ECE 520.447 Introduction to Information Theory and Coding

Spring 2014

Mon Wed Fri 1:30-2:20 PM in Hackerman 320 (Occasionally 1:30-2:45)

Information Theory addresses some fundamental questions about systems that store or communicate data.

The answer to the first set of questions leads to the concept of Channel Capacity, and the second set to Entropy. Information Theory is therefore an essential weapon in a communication engineer's arsenal.

Information Theory, due to the nature of its subject matter, also makes fundamental contributions to statistical physics (thermodynamics), computer science (string complexity), economics (optimal portfolios), probability theory (large deviations) and statistics (Fisher information, hypothesis testing). This makes Information Theory a useful tool for students of other disciplines.


E mail:
Office Location:
Office Hours:

Course Assistants:

E mail:
Office Location:
Office Hours:
Sanjeev Khudanpur

MyLastName at jhu dot edu
Hackerman Hall Room 325
By appointment: W 3:00-4:00. Please e-mail Ms. Ruth Scally [rscally1] in advance.

Congyuan Yang and Dung Tran

[yangcy dot ee at gmail dot com] and [FirstName.n.Lastname at jhu dot edu] respectively
Hackerman 322 and Barton 320 respectively
Monday 3:00-4:00 PM and Wednesday 10:00-11:00 AM respectively

Text and Reference Books

The Fall 2002 offering of the course will utilize the book by Cover and Thomas as the primary textbook, with some additional reading material from Csiszár and Körner.

Homeworks, Exams and Class Notes

Evaluation will be based on a combination of homework assignments (20%), two midterm examinations (20% each, on March 14 and April 30), and a comprehensive final examination (40%) on May 14, 9AM -12 Noon.

Homework assignments and notes will be posted here throughout the semester. Check here frequently!

An Important Note on Academic Ethics:

The strength of the university depends on academic and personal integrity. In this course, you must be honest and truthful. Ethical violations include cheating on exams, plagiarism, reuse of assignments, improper use of the Internet and electronic devices, unauthorized collaboration, alteration of graded assignments, forgery and falsification, lying, facilitating academic dishonesty, and unfair competition. Report any violations you witness to the instructor. You may consult the associate dean of students and/or the chairman of the Ethics Board beforehand. See the guide on Academic Ethics for Undergraduates and the Ethics Board web site for more information.

Old Homeworks

Homework assignments from the Fall 2002 offering of the course are provided below as examples. These or similar homeworks will be assigned in Spring 2014. See above for the actual homeworks for this semester.

Old Exams

Old Notes

Some notes on Joint Typicality ( pdf, ps)

Some notes on Rate Distortion ( pdf, ps)

Some notes on Relative Entropy ( pdf, ps)

Some notes on the Method of Types (large pdf file, or even larger ps file)