520.674 Information Theoretic Methods in Statistics

Information theoretic ideas (entropy, mutual information and K-L divergence) for solving problems of statistical inference will be discussed: information geometry, maximum entropy methods, the EM-algorithm & alternating minimization, Sanov's theorem & large deviations, redundancy, MDL & other universal data compression schemes. Khudanpur. 3 hours.