Principles of compression

Image, video, and audio
As mentioned earlier, formats like GIF, JPG, TIFF and many others, not to mention MP3, Ogg, WMV and the rest are all borne from compressors optimised for these target sources. These are also, of themselves, a science. For example unlike text and binary, which have their own patterns, the data in images can often consist of lots of data that, while not always changing from one sequence to the next, can consist of variance of data - such as a color gradient. Rather than repeatedly store the actual data for every pixel, for example, a compressed image format might store simply the variance between one pixel and the next. Then, if you lax the threshold of what constitutes a variance, you can suddenly save a lot of data, though of course with a trade-off, which we'll get onto in a moment.

Video, too, has its own specific traits. Some video compressors treat each frame as an image, and compress the images individually, but just as the data within an image can be repetitive (or closely related), the same is true of video as well. When an actor is talking on screen, their mouth might move but the background is static, and the data between all those repeated frames can be substituted, and thus compressed, to save space. These two methods are known as intraframe and interframe compression respectively.

And audio, of course, is another beast. Using music as an example, rather than store the data of the actual waveforms, the differences between the change in a waveform itself can be stored. Some techniques work by modelling what is expected next in a waveform, and storing the variance of the actual waveform to the predicted model.

Of course all of these methods also share something in common (with the exception of some formats) - they are a lossy approach to compression. While we generally notice if words go missing in a text document, and binaries don't take kindly to bits disappearing, lossless compression isn't necessarily required for images, video, and audio. As humans we have limits to our perception, and tend not to mind or notice degradation of quality up to a certain point. Said another way, there's sometimes a lot of data a compressor can simply throw away entirely with these formats because its loss is imperceptible, or acceptable, to us. And this, of course, can save a lot more space. The art of good lossy techniques lies in determining what can be dropped and what can be compressed to approximate equivalents, and still provide an almost indistinguishable result from the original. Or, of course, electing to save even more space and be more generous with approximations until we do notice the difference, but it's a choice we make for the quality/space tradeoff - such as low or high bit-rate audio and video formats.