Introduction to Data Compression∗ Guy E. Blelloch Computer Science Department Carnegie Mellon University blellochcs.cmu.edu January 31, 2013 Contents 1 Introduction 3 2 Information Theory 5 2.1 Entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.2 The Entropy of the English Language . . . . . . . . . . . . . . . . . . . . . . . . 6 2.3 Conditional Entropy an