Information theory

History of information theory

The decisive event which established the discipline of information theory, and brought it to immediate worldwide attention, was the publication of Claude E. Shannon's classic paper "A Mathematical Theory of Communication" in the Bell System Technical Journal in July and October 1948. In this revolutionary and groundbreaking paper, the work for which Shannon had substantially completed at Bell Labs by the end of 1944, Shannon for the first time introduced the qualitative and quantitative model of communication as a statistical process underlying information theory, opening with the assertion that "The fundamental problem of communication is that of reproducing at one point, either exactly or approximately, a message selected at another point." With it came the ideas of * the information entropy and redundancy of a source, and its relevance through the source coding theorem; * the mutual information, and the channel capacity of a noisy channel, including the promise of perfect loss-free communication given by the noisy-channel coding theorem; * the practical result of the Shannon–Hartley law for the channel capacity of a Gaussian channel; and of course * the bit - a new way of seeing the most fundamental unit of information. (Wikipedia).

Video thumbnail

(IC 1.6) A different notion of "information"

An informal discussion of the distinctions between our everyday usage of the word "information" and the information-theoretic notion of "information". A playlist of these videos is available at: http://www.youtube.com/playlist?list=PLE125425EC837021F Attribution for image of TV static:

From playlist Information theory and Coding

Video thumbnail

From information theory to learning via Statistical Physics by Florent Krzakala

26 December 2016 to 07 January 2017 VENUE: Madhava Lecture Hall, ICTS Bangalore Information theory and computational complexity have emerged as central concepts in the study of biological and physical systems, in both the classical and quantum realm. The low-energy landscape of classical

From playlist US-India Advanced Studies Institute: Classical and Quantum Information

Video thumbnail

From information theory to learning via Statistical Physics: Introduction: by Florent Krzakala

26 December 2016 to 07 January 2017 VENUE: Madhava Lecture Hall, ICTS Bangalore Information theory and computational complexity have emerged as central concepts in the study of biological and physical systems, in both the classical and quantum realm. The low-energy landscape of classical

From playlist US-India Advanced Studies Institute: Classical and Quantum Information

Video thumbnail

Information Theory Meets Quantum Physics: The magic of wave dynamics by Apoorva Patel

26 December 2016 to 07 January 2017 VENUE: Madhava Lecture Hall, ICTS Bangalore Information theory and computational complexity have emerged as central concepts in the study of biological and physical systems, in both the classical and quantum realm. The low-energy landscape of classical

From playlist US-India Advanced Studies Institute: Classical and Quantum Information

Video thumbnail

From information theory to learning via Statistical Physics: From statistical by Florent Krzakala

26 December 2016 to 07 January 2017 VENUE: Madhava Lecture Hall, ICTS Bangalore Information theory and computational complexity have emerged as central concepts in the study of biological and physical systems, in both the classical and quantum realm. The low-energy landscape of classical

From playlist US-India Advanced Studies Institute: Classical and Quantum Information

Video thumbnail

Sergio Verdu - Information Theory Today

Founded by Claude Shannon in 1948, information theory has taken on renewed vibrancy with technological advances that pave the way for attaining the fundamental limits of communication channels and information sources. Increasingly playing a role as a design driver, information theory is b

From playlist NOKIA-IHES Workshop

Video thumbnail

CERIAS Security: WHAT IS INFORMATION? 6/6

Clip 6/6 Speaker: Wojciech Szpankowski · Professor of Computer Science · Purdue University Information permeates every corner of our lives and shapes our universe. Understanding and harnessing information holds the potential for significant advances. The breadth and depth of underlyin

From playlist The CERIAS Security Seminars 2007

Video thumbnail

The State of the Universe - J. Hartle - 12/9/2013

A conference celebrating the 50th anniversary of quarks honoring Murray Gell-Mann was held at Caltech on December 9-10, 2013. For more information, visit: http://hep.caltech.edu/gm/

From playlist The 50th Anniversary of Quarks Honoring Murray Gell-Mann

Video thumbnail

CERIAS Security: WHAT IS INFORMATION? 1/6

Clip 1/6 Speaker: Wojciech Szpankowski · Professor of Computer Science · Purdue University Information permeates every corner of our lives and shapes our universe. Understanding and harnessing information holds the potential for significant advances. The breadth and depth of underlyin

From playlist The CERIAS Security Seminars 2007

Video thumbnail

The Creation of Historical Effects - L. Taub - 4/26/19

On April 26-27 2019, the Division of Humanities & Social Sciences at Caltech hosted a conference in honor of Jed Z. Buchwald, “Looking Back as We Move Forward: The Past, Present, and Future of the History of Science.” This event was sponsored by the Division of the Humanities & Social Sci

From playlist Looking Back as We Move Forward - A Conference in Honor of Jed Z. Buchwald - 4/26-27/2019

Video thumbnail

Black Holes | History and Philosophy of Astronomy 8.05

Learn about the history and philosophy of astronomy from Professor Impey, a University Distinguished Professor of Astronomy at the University of Arizona, with our Knowing the Universe: History and Philosophy of Astronomy course here on YouTube. This video is part of module 8, Relativity.

From playlist History and Philosophy Course Module 8: Relativity

Video thumbnail

Ancient Aliens: Einstein’s Theories Prove Time Travel to an Alternate Universe (Season 5) | History

Ancient Aliens theorists believe that Albert Einstein’s Theory of Relativity creates the possibility for time travel through wormholes - allowing for contact with extraterrestrials, in this clip from Season 5, "The Einstein Factor". Watch all new episodes of Ancient Aliens, returning soon

From playlist Ancient Aliens: Official Series Playlist | New Episodes Fridays at 9/8c | History

Video thumbnail

The quantum measure (and how to measure it) by Rafael Sorkin

21 November 2016 to 10 December 2016 VENUE Ramanujan Lecture Hall, ICTS Bangalore Quantum Theory has passed all experimental tests, with impressive accuracy. It applies to light and matter from the smallest scales so far explored, up to the mesoscopic scale. It is also a necessary ingredie

From playlist Fundamental Problems of Quantum Physics

Video thumbnail

Leonard Mlodinow on his friendship with Stephen Hawking | CTT Live

Theoretical physicist Len Mlodinow, who spent nearly two decades collaborating with Stephen Hawking, joined Robert to discuss his new book, Stephen Hawking: A Memoir of Friendship and Physics. Buy the book here: https://amzn.to/2PqdyJ1

From playlist Closer To Truth Chats

Video thumbnail

Thermodynamics of Information by Juan MR Parrondo (Lecture 1)

26 December 2016 to 07 January 2017 VENUE: Madhava Lecture Hall, ICTS Bangalore Information theory and computational complexity have emerged as central concepts in the study of biological and physical systems, in both the classical and quantum realm. The low-energy landscape of classical

From playlist US-India Advanced Studies Institute: Classical and Quantum Information

Video thumbnail

HSC Science Extension Module 1 Epistemology and Navigation

HSC Science Extension Module 1 Foundations of Scientific Thinking Epistemology and Scientific Knowledge Navigation and the Longitude problem

From playlist Y12 Sci Ex Mod 1 Foundations of Scientific Thinking

Video thumbnail

Bernard Geoghegan, “The Difficulty of Gift-Giving: Cybernetics and Postwar French Thought”

A historian and theorist of digital media, Geoghegan is a senior lecturer in Media and Communications at Coventry University and a visiting associate professor in Film and Media Studies at Yale University. He also works as a curator and educational programmer for the Anthropocene Project a

From playlist Whitney Humanities Center

Video thumbnail

mod-05 Lec-33 Extensive Games: Introduction

Game Theory and Economics by Dr. Debarshi Das, Department of Humanities and Social Sciences, IIT Guwahati. For more details on NPTEL visit http://nptel.iitm.ac.in

From playlist IIT Guwahati: Game Theory and Economics | CosmoLearning.org Economics

Video thumbnail

Thermodynamics of Information by Juan MR Parrondo (Lecture 4)

26 December 2016 to 07 January 2017 VENUE: Madhava Lecture Hall, ICTS Bangalore Information theory and computational complexity have emerged as central concepts in the study of biological and physical systems, in both the classical and quantum realm. The low-energy landscape of classical

From playlist US-India Advanced Studies Institute: Classical and Quantum Information

Related pages

Qubit | Harry Nyquist | Channel capacity | Entropy (classical thermodynamics) | Noisy-channel coding theorem | Mutual information | H-theorem | Claude Shannon | Information theory | John von Neumann | Kullback–Leibler divergence | A Mathematical Theory of Communication | Alan Turing | Redundancy (information theory) | Reed–Muller code | Shannon's source coding theorem | Shannon–Hartley theorem | Entropy in thermodynamics and information theory | Discrete cosine transform | Bit | David E. Muller | Bandwidth (signal processing) | Elwyn Berlekamp | Berlekamp–Massey algorithm | Timeline of information theory | Irving S. Reed | Hartley (unit)