“When in the late 1940s C.E.Shannon created Information Theory there was a general belief he should call it Communication Theory, but he insisted on the word “information”, and it is exactly that word which has been the constant source of both interest and of disappointment in the theory. One wants to have a theory of “information” but it is simply a theory of strings of symbols. Again, all we suppose is there is such a source, and we are going to encode it for transmission.”
Excerpt From: Richard R. Hamming. “Art of Doing Science and Engineering: Learning to Learn.”