"Babes-Bolyai" University of Cluj-Napoca
Faculty of Mathematics and Computer Science

Information theory
Code
Semes-
ter
Hours: C+S+L
Credits
Type
Section
MC013
8
2+2+0
7
optional
Matematica Economica
MC013
8
2+2+0
7
optional
Matematici Aplicate
Teaching Staff in Charge
Prof. MIHOC Ion, Ph.D., imihoc@math.ubbcluj.ro
Aims
To pressent to the students the principal notions of the information theory and some important applications of their.
Content
1. Information- the secondary concept and the probability - the primary concept. Hartley's formula. Shannon's entropy. The Shannon-Hincin and the Fadeev characterizations of the Shannon entropy. The information characterizations for some probability distributions.
2. Information - the primary concept and the probability - the secondary concept. Axiomatic of J.Kampe de Feriet-Bruno Forte. Information of the type M.
3. Renyi entropies. The information of order $\alpha$ for a ordinary distributions and for the incomplete distributions.
4. Information theory and statistics. Kullback-Leibler information. Properties of information: additivity, convexity. Fisher's information. Information and sufficiency.
References
1. Ash, R., Information Theory, John Wiley and Sons, New York. London . Sydney, 1965
2. Guiasu, S., Theodorescu, R., Teoria matematica a informatiei, Ed. Academiei RSR, Bucuresti, 1966.
3. Kullback, S., Information Theory and Statistics, John Wiley and Sons, London, 1959
4. Renyi, A., Probability Theory, Akademiai Kiado, Budapest, 1970.
Assessment
Exam.