Information Theory and Elements of Coding

General overview. Basic concepts. Definition and quantitative measure of Information. Useful measures and operations. Basic concepts of probability. Discrete sources, entropy-information and their relationship, redundancy, efficiency, information rate, source codes. Memory-less sources, Markov sources. channel capacity and noise. Ideal channel, ideal system, channel with noise, continuous channel. Continuous signals, transmission of band-limited signals, sampling theorems and reconstruction. Comparison of analog signal communication systems. Elements of error control coding.