6 years, 5 months ago

Laying Foundation for Digital Age: Claude Shannon, 1948

Link: http://feedproxy.google.com/~r/ElementalLinks/~3/zQVTyMm5umY/

“The informational value of a message, whether it’s spoken, written, or encoded in silicon, is a function of how much the recipient already knows. If an outcome is highly unpredictable, a message communicating the outcome carries more information (and requires more bits) than if the outcome is more predictable (e.g., learning that a two-headed coin is going to come up heads). Shannon’s entropy allows programmers to create software that packs the most important information into the smallest file size.”