On audio compression

In the last few days, I have been reading and thinking a lot about audio compression.

Lossy v. lossless compression

As most of you will know, there are two major types of compression: lossless and lossy. In the first case, we take a string of digital information and reduce the amount of space it takes to store without actually destroying any information at all. For example, we could take a string like:

1-2-1-7-3-5-5-5-5-5-5-5-5-5-5-5-5-5-2-2-2-3-4

And convert it into:

1-2-1-7-3-5(13)-2(3)-3-4

Depending on the character of the data and the kinds of rules we use to compress it, this will result in a greater or lesser amount of compression. The upshot is that we can always return the data to its original state. If the file in question is an executable (a computer program), this is obviously required. A file that closely resembles Doom, as a string of bits, will nonetheless probably not run like Doom (or at all).

Lossless compression is great. It allows us, for instance, to go back to the original data and then manipulate it with as much freedom as we had to begin with. The cost associated with that flexibility is that files compressed in lossless compression are larger than those treated with lossy compression. For data that is exposed to human senses (especially photos, music, and video), it is generally worthwhile to employ ‘lossy’ compression. A compact disc stores somewhere in the realm of 700MB of data. Uncompressed, that would take up an equivalent amount of space on an iPod or computer hard drive. There is almost certainly some level of lossy compression at which it would be impossible for a human being with good ears and the best audio equipment to tell if they were hearing the compressed or uncompressed version. This is especially true when the data source is CDs, which have considerable limitations of their own when it comes to storing audio information.

Lossy compression, therefore, discards bits of the information that are less noticeable in order to save space. Two bits of sky that are almost-but-not-quite the same colour of blue in an uncompressed image file might become actually the same colour of blue in a compressed image file. This happens to a greater and greater degree as the level of compression increases. As with music, there is some point where it is basically impossible to distinguish the original uncompressed data from a compressed file of high quality. With music, it might be that a tenth of a second of near silence followed by a tenth of a second of the slightest noise becomes a twentieth of a second of near silence.

MP3 and AAC are both very common kinds of music compression. Each can be done at different bit-rates, which determines how much data is used to represent a certain length of time. Higher bit rates contain more data (which one may or may not be able to hear), while lower bit rates contain less. The iTunes standard is to use 128-bit AAC. I have seen experts do everything from utterly condemn this as far too low to claim that at this level the sound is ‘transparent:’ meaning that it is impossible to tell that it was compressed.

But what sort to use, exactly?

Websites on which form of compression to use generally take the form of: “I have made twenty five different versions of the same three songs. I then listened to each using my superior audio equipment and finely tuned ear and have decided that X is the best sort of compression. Anyone who thinks you should use something more compressed than X obviously doesn’t have my fine ability to discern detail. Anyone who wants you to use more than X is an audiophile snob who is more concerned about equipment than music.”

This is not a very useful kind of judgment. Most problematically, the subject/experimenter knows which track is which, when listening to them. It has been well established that taking an audio expert and telling them that they are listening to a $50,000 audiophile quality stereo will lead to a good review of the sound, even if they are really listening to a $2,000 system. (There are famous pranks where people have put a $100 portable CD player inside the case for absurdly expensive audio gear and passed the former off as the latter to experts.) The trouble is both that those being asked to make the judgement feel pressured to demonstrate their expertise and that people actually do perceive things which they expect to be superior as actually being so.

Notoriously, people who are given Coke and Pepsi to taste are more likely to express a preference for the latter if they do not know which is which, but for the former when they do. Their pre-existing expectations affect the way they taste the drinks.

What is really necessary is a double-blinded study. We would make a large number of versions of a collection of tracks with different musical qualities. The files would then be assigned randomized names by a group that will not communicate with either the experimenters or the subjects. The subjects will then listen to two different versions of the same track and choose which they prefer. Each of these trials would produce what statisticians call a dyad. Once we have hundreds of dyads through which to compare versions, we can start to generate statistically valid conclusions about whether the two tracks can be distinguished, and which one is perceived as better. On the basis of hundreds of such tests, in differing orders, we would gain knowledge about whether a certain track is preferred on average to another.

We would then analyze those frequencies to determine whether the difference between one track (say, 128-bit AAC) and another (say, 192-bit AAC) is statistically significant. I would posit that we will eventually find a point where people are likely to pick one or the other at random, because they are essentially the same (640-bit AAC v. 1024-bit AAC, for instance). We therefore take the quality setting that is lowest, but still distinguishable from the one below based on, say, a 95% confidence level and use that to encode our music.

This methodology isn’t perfect, but it would be dramatically more rigorous than the expertly-driven approach described above.

Let the market provide?

The way the Oxford County Library deals with audio-visual materials strikes me as rather illegitimate. They charge for renting CDs and DVDs at rates comparable to commercial venues. As such, they are using a tax subsidized situation (free rent, plus funding from local taxes) to complete directly with private enterprise in an area where there is no market failure. I don’t need a governmental service to charge me three quid to rent The Life Aquatic, and the existence of one that does quite likely crowds out commercial venues that would do a better job (have more than one copy, offer deals for frequent renters, etc).

I am all for libraries having a collection of educational CDs and DVDs, but they really should be lent out in a way that is in keeping with the idea of libraries as publicly funded providers of public goods.

That said, I have been enjoying the use of my new headphones with the opera CDs that I paid £6 to borrow. Reproduction of classical instruments and male vocalists seem to be the two areas where there is the most difference between these and the default Apple earbuds. It can also be amusing to not how often you faintly hear people shuffling around the studio, coughing in muffled fashion, or turning pages in CDs that you have heard a hundred times. You can even hear someone’s watch ticking faintly in a Nine Inch Nails B-side I have.

iTunes data integrity question

I find that a small number of the songs that I have in iTunes have become mysteriously truncated: suddenly coming to an abrupt end somewhere between twenty seconds and two minutes before the song is over. These are songs that worked properly before. In total, less than one in thirty songs are affected, in my estimation. Even so, in a library of 3786 songs, that’s a non-trivial proportion. In particular, Bob Marley songs seem to be vulnerable for some reason. Of the small collection I have, more than half have been thus clipped over the course of the last few years. I don’t know if this is simply corruption due to entropic increase in my hard drive, or whether something more complex is at work.

Is this an issue anybody knows anything about? Google has not been forthcoming with information from other people similarly affected. The problem is especially annoying because it is difficult to remember exactly which songs have been affected, and thus to replace them. Also, most of the CDs from which my iTunes music was generated are back in boxes of mine in Vancouver.

PS. This is definitely not related to the option in iTunes to have songs end before the track normally does (useful for cutting applause off in live tracks, or removing annoying band chatter before and after songs).

First eBay sale

I’ve joined the ranks of those who have at least listed an item on eBay. In this case, it’s the Sony headphones that I want to sell in order to get money for a snazzier pair. These are brand new and in the original packaging.

I may have set the minimum bid a bit high, but you can’t set a reserve price under £50 and I’m really not willing to sell these for less than £15 after spending almost £25 on them. In any case, we will see how this experiment in commerce goes.

[Update: 21 June 2006] With exactly 12 seconds left in the auction, someone placed a bid. Looks like I am offloading these headphones for £15 plus the cost of shipping.

The economics of it all:

Price initially paid on Amazon: £25.66 C$53.01

Payment received from eBay: £15.00 C$30.99
Shipping fee from eBay: £2.00 C$4.13

eBay listing fee: £1.29 C$2.68
PayPal currency fee: £0.86 C$1.77
Cost of packaging: £0.49 C$1.01
Cost of shipping: £0.68 C$1.40
Net eBay income: £13.68 C$28.26

Amazon cost – eBay income: -£11.98 -C$24.75

In the end, choosing to buy these headphones cost me about twenty-five bucks for three months’ usage. Let’s hope the ones I choose to replace them with last much longer.

Sony headphones, cont.

In response to my complaints by letter and phone, as well as sending back the broken old pair, Sony sent me a new set of MDR-EX71SL Fontopia earbuds (£25.67 on Amazon). While this is welcome, I would have preferred to have them send me a refund. On past form, the new pair will only last about three months before completely falling apart due to cheap materials and shoddy construction. My earlier complaint about them is here. To anyone considering buying Sony Fontopia earbuds: don’t. They are no longer the nice-sounding, solid things they were back in 2000 or so. Now, they are made of plastic so soft, you can literally peel it off the wires gently with your fingers. It is less tough than the brand-new shoestring licorices that you are surprised to bite into and not find toughened by months of exposure to 7-11 air and fluorescent lighting.

Rather than put the refund towards new Sony headphones, I would probably have gone with either the Shure E2Cs (£51.49) or Etymotic ER6Is (£68.51), which many people have told me are more durable and have better sound. They need to be earbuds, because I want to be able to wear them under a bike helmet and carry them virtually everywhere. While big ear-covering headphones would be great for my room, they hardly work with bikes and iPod Shuffles.

Since the replacement EX71s are new and in the original packaging, I could try selling them online somewhere, thereby generating some funds to put towards a better set. Of course, that would mean at least another couple of weeks with ear canals aching from button-shaped and unyielding hard white generic Apple headphones. What do cost-conscious audiophiles suggest?

Ten songs you should hear

I am extremely grateful to the many friends of mine who have introduced me to new music over the course of the last few years. Below are a list of ten songs that most people probably will not have heard, but which I heartily endorse. Unfortunately, I cannot actually give you the songs, because goons from the RIAA would break down my door in the night. The list is therefore provided for the benefit of those in search of new music, and capable of acquiring it for themselves by means that satisfy the legal and ethical codes that apply to them.

Since most of these artists are relatively unknown, I would definitely think better of you if you actually went out and bought their albums.

1) Nina Simone, “Feeling Good” from the album Nina Simone in Concert

This energetic song was sent to me by Lauren Priest. I have always been an appreciator of strong female vocalists and, along with her jazz accompaniment, Nina Simone definitely rises to that level. This is the kind of song you’ll find yourself singing automatically while walking briskly home from something really enjoyable.

2) Idan Raichel
, “Mima’amakim (From the deep)” from the album Mimamakayim

While I can’t speak or understand a word of Hebrew, I love the complex melodies in this song, as well as the interesting texture of the language itself. The interplay between vocals and instrumentation in this track is really fascinating, in a way that reminds me of Robert Miles transported into an entirely different genre and mood. I got it from Lindi.

3) Antony & the Johnsons, “Beautiful Boyz” from the album Noah’s Ark

Alison introduced me to this sombre and mournful, yet beautiful, track. This is the kind of music that wanders into my mind during really long solitary walks of the sort that it’s sometimes advisable to take when sad.

4) KT Tunstall, “Black Horse And The Cherry Tree” from the album Eye to the Telescope

Almost the exact opposite tone from the previous song, this song is energetic to the point of being positively jaunty. I could easily imagine Astrid dancing to this track, after she lent it to me.

5) Neko Case, “Hold On, Hold On” from the album Fox Confessor Brings the Flood

This is the only really superb song that I ever got as a free song of the week from the iTunes music store. From Neko Case’s new album, this autobiographical piece has the same combination of powerful female vocals and enigmatic lyrics that I so appreciate from Tori Amos. A video of this song being performed can be viewed here.

6) Lorraine a’ Malena, “Just Me and Eve” from the album Mirror Mirror

A somewhat comic retelling of Genesis, this song was apparently written by Neil Gaiman and is performed by his assistant. It may be my appreciation for Paradise Lost manifesting itself again, but I quite enjoy the tune. I am grateful to Jessica for introducing me to it. This one is actually available free online. How cool is that?

7) Feist, “When I was a Young Girl” from the album Let It Die

Many thanks to Jonathan for sending me this song. I love the rhythmic interplay between the drums and vocals. It reminds me, to a certain extent, of a few songs from the Oh Brother, Where Art Thou soundtrack, insofar as it accompanies a kind of moralizing message (probably ironic) with really enjoyable sounds.

8) Mad Pudding, “First We Take Manhattan” from the album Grand Hotel

This cover of Leonard Cohen’s song is done by a Vancouver folk/celtic band that is somewhat similar to Spirit of the West. The contrast between their energetic interpretation and Cohen’s determinedly poetic style of delivering is striking. I first heard this song from a CD owned by my father.

9) The Vincent Black Shadow, “This Road is Going Nowhere” from the album The Vincent Black Shadow

Since Neal gave me the superb debut album of this Vancouver band, I have listened to it more than sixty times. This is the kind of song that alters the way you experience everything for a good fifteen minutes, at least, after you hear it. I especially like the musical interludes with strings and saxophone.

10) Rae Spoon, “To Find You” from an unknown album

On the first CD from which I ever heard Tori Amos and Tegan and Sara, my friend Jenny also included this wonderfully downbeat vocal and guitar track by a Vancouver artist. A complex and evocative piece of music.

This is by no means an exhaustive list, so please don’t feel insulted if a song you sent me isn’t listed here. These are just songs that happen to have found particular resonance with me, during the time when I’ve been in Oxford.

[Update: 25 February 2007] Since so many people were looking for them, some Idal Raichel lyrics translated into English have been added.

Sony Fontopia headphones are very poorly made

The Sony Fontopia MDR-EX71SL headphones that I got on the 3rd of March are already broken. This is hardly what you expect from a pair that cost more than $50. The cladding around the wires is made of really cheap plastic that bunches up and breaks down: even under the kind of delicate use to which I have been subjecting them. I was warned too late about how poor their durability is. It makes quite the contrast with the pair of Fontopias I bought in 2000, and which only failed immediately before I bought these ones.

I will try to have them replaced under the warranty. Otherwise, I am soliciting opinions about earbuds that have comparably good sound and dramatically better construction.

Second summer day

Contrail and tree branches

With Kai at a party in London and Alex in Vienna for the marathon, this is my first night alone in the Church Walk flat. It follows a day that was excellent in many ways. The weather had the same brightness and warmth of yesterday, and it was accompanied nicely by the Feist CD that Jonathan recommended to me, called “Let it Die.” A few weeks will be necessary to really comprehend the style, but I could tell immediately that I like it. Already, the CD strikes me as unusually versatile – with a style that’s hard to pin down. The tone is similarly liable to shift dramatically between songs: from playful to forlorn. I rather like the song in French.

After meeting with Dr. Hurrell and reading in the Wadham MCR for a few hours, I spent the evening walking and conversing with Roz. At one point, we got excellent veggie burgers from a place on Walton Street called Peppers. The smell and clientele reminded me of the Jamaican place across the street from the hostel in Manhattan where I stayed in the days after the blackout in 2003. Their burgers are both very filling and surprisingly tasty, for a vegetarian product in the UK. Given that it’s just a few blocks away and is open late, it risks becoming the Pita Pit of this place of residence (a reference that anyone from UBC should understand).

Earlier, Rosalind and I wandered through Trinity College, which I had previously seen only from the outside, and the Wadham gardens. They have begun to change dramatically, with the coming of sun and longer days. For someone who arrived in Oxford in late September, it’s still something of a surprise to see groups of trees with leaves on them. I am looking forward to a summer of working, cycling, and researching here.

Migration news: geeky stuff

The migration from Blogger to WordPress is going well. I have the colours and formatting on the new blog more or less where I want them. I already much prefer the commenting and management system of WordPress. I just need to come up with a sharp new banner and tweak a few small things. Then, I will shift the WordPress version to the front page. I think I can do so without breaking the links to the old Blogger posts: at least until Google indexes them on the basis of their new permalinks.

Now that I am getting used to it a bit, I prefer the cleaner lines and overall layout of the new blog, as well as the greater versatility of the content management system.


  • Between the 20th and 24th of June, the Oxford Playhouse is staging Paradise Lost. I shall make a point of going. Roz says that she is also keen to come, if she hasn’t headed off to Rome for the summer by that point.
  • Tomorrow afternoon is the first OUSSG executive meeting, meant to sort out what to do about the dinners this term. It will be good to finally meet the rest of both the new and old executive in a context meant for planning.
  • I got a quartet of very diverse fictional books from a free box in Nuffield that Margaret directed me towards, when I was waiting for my supervision. I’ll have a look through them in the summer.

End of term festivities I

The most interesting photo available

Happy Birthday Alison Atkinson

I spent a while at the Wadham MCR party tonight, but it was so absurdly loud that I could only occupy the purlieus. Within ten metres of the speakers, the sound was so distorted as to make even familiar songs seem bizarrely warped. As a consequence of tonight being a guest dinner night, I only recognized about one in ten people there. I fairly quickly adopted the rational strategy of going back to my room to listen to Tracy Chapman, eat tofu sandwiches, and talk with Bryony over MSN.

Today did not involve a great deal of progress on the take-home exam. I’ve decided to whom I will write the hypothetical letter requesting an interview. I can therefore also begin formulating appropriate interview questions. Since I’ve never conducted a formal interview and our course didn’t actually involve any training on what kind of questions to ask, I am essentially on my own in terms of coming up with them. I suppose that if I make them pretty heavily technical, it will seem reasonably impressive to whoever marks it, though it may or may not represent an effective way of getting useful information.

I went to Beeline Cycles today and learned that their two cheapest bikes – both new – are a generic steel framed mountain bike for £80 or a hybrid for £130. They were really pushing the hybrid, saying it’s less likely to get stolen and better suited to Oxford commuting, but I’m not sure it’s worth almost twice the price. I won’t buy anything until my mother brings my D-lock and helmet from Vancouver. To have a bike and neither of those would be to court disaster.

Music and frustration: copy protection schemes

Chained pig, BathHaving spent the last few minutes explaining to a friend why a brand-new, legitimately purchased CD will not play in her computer due to the copy protection EMI has included, I am reminded of my considerable indignation about how the music industry is treating their customers. Yes, in this case, it was possible to disable the copy protection program just by holding shift as the CD was inserted into a Windows computer, but there is no guarantee at all that music you buy today is either usable or safe.

In the worst case, such as the notorious Sony BMG rootkit, inserting a legitimate music CD into your computer intentionally breaks it. It also causes it to report what you listen to to Sony, even if you choose ‘no’ when a screen comes up asking for permission to install software. It also creates really sneaky back doors into your system that can be exploited for any number of purposes, by Sony or random others. While Sony is currently facing lawsuits for this particular, infamous piece of malware, it isn’t nearly enough to put my mind at ease. If some 16 year old had written something comparably dangerous, they would probably be in jail.

Legitimately downloaded music is little better. Songs you buy from the iTunes music store may work with your iPod today, but they won’t work with another portable player. They won’t even play in software other than iTunes, and there is no guarantee that they will still work at some point in the future. Spending a great deal of money on songs from there (and they’ve just had their billionth download), is therefore probably not very wise. You don’t actually own the music you are buying – you’re just buying the right to use it on someone else’s terms: terms that they have considerable freedom to change.

Personally, I will not buy any CD that contains copy protection software. I will not buy a Sony BMG CD, regardless of whether it does or not, nor will I be buying any of Sony’s electronics in the near future. This is a business model that needs to change.