A Ripping Yarn : Converting CDs to WAV files
The Audio CD Format
The 16-bit/44.1 kHz specifications for Compact Disc were laid down in the early 1980s. They reflect the capability of mass-produced technology of the day, and 30 years later have been superseded by higher resolution formats. The main limiting parameters of the CD standard are bandwidth, dynamic range and low level distortion. A sampling rate of 44.1 kHz acommodates a bandwidth of just over 20 kHz, and 16 bits provide a digital dynamic range of 96 dB, which, although it sounds a lot, cannot be compared in audible effect with an analog dynamic range of 96 dB. There are those who consider that the high frequencies should not be limited to 20 kHz, although the frequency range of professional analog tapes at 38 cm/s was similarly curtailed by the practical limitation of the length of the gap in the playback head, and analog master tapes often sound better than CDs. But, while one might perhaps not worry about the bandwidth restriction, the 16 bit boundary was without question an unfortunate compromise, reflecting state of the art digital technology of 1980. Indeed, studies by the BBC had by then already shown that at least 20 bits were needed for what was identified as the worst case — big band jazz.
One can readily calculate how many bits are needed per hour in the 16/44.1 system:
44,100 samples per channel per second, each sample represented by 16 bits:
= 44100 x 16 = 705,600 bits per channel per second
But the CD is in stereo:
= 1,411,200 bits per two channels per second
= 84,672,000 bits per minute
= 5,080,320,000 bits per hour
We usually count in bytes, each of which is a grouping of 8 bits, making an hour of stereo 635,040,000 bytes, or about 606 Megabytes. Then there are some added extras, the table of contents (TOC), which is effectively an extra track at the beginning of the CD, and the bits used for inline control data and error correction. All this is effectively in a single file and represents a lot of data in 1982 when the CD was introduced. Since in those days a 10 MB hard drive was an expensive, large and heavy monster in a special case, the mass production of a CD, which was much smaller and lighter and acommodated more than 60 times the amount of data held by such a hard drive, was quite amazing.
The Standard CD Player
Because the data is read off the CD a bit at a time, a lot of processing has to happen in the player if the audio is to be as perfect as was initially promised by the record companies. This involves separating out the control data, applying error correction where needed (and possible), rearranging the samples into left and right channels and then converting them back to analog audio. At the same time, the datastream is being used to adjust the speed of rotation of the disc, which varies as the spiral is tracked from inside to outside radius. With all this work in real time, A standard CD player is too busy to go back and have a second look at the data: it has to do the best it can with what comes off the CD and then move on to the next chunk.
Storing Audio on a Computer
Modern personal computers, with fast processors and cheap, fast, high-capacity hard drives are much more adept at playing back audio in real time. The data are recorded in files with a single header at the beginning that identifies the sample and bit rates, and the number of channels, followed by the actual data. The channels are still interleaved, but unlike the CD the number of these is not limited to two, and the sample and data rates can be higher. Taking advantage of this, current professional recordings are made at sample rates of up to 192 kHz, which certainly answers concerns about the frequency range of a CD, and 24 bit representation of the samples gives a further 48 dB of dynamic range. We are no longer limited by the medium, which can be a DVD-A, a Bluray disc or a USB data stick.
Although sales have fallen dramatically over the past few years, CDs are still with us, and most of them sound better than mp3 files. But it has been my experience that playing high quality wavefiles of the same 16/44.1 resolution as CDs from a humble iPod produces higher quality sound than playing the same music from a CD player of the quality that I can afford. Of course, excellent sounding CD players can be bought for as little (or as much, depending on your point of view) as $5,000 and more, but the same quality can be achieved more cheaply by transferring the data from CD to hard drive in a process called “ripping”) and then playing the result directly from the hard drive through a decent convertor.
There are several programs that can be used to rip CDs and convert the audio to a wavefile that can be played directly. I personally only have experience of three that work under Windows: iTunes, EAC and Foobar2000; but there are others. Any of these free programs will rip most CDs in as little as a 20th of the time it takes to play it, and EAC is optimised to rip damaged CDs, though the penalty for this is the long time required.
Some of my old CDs, although still in new condition, present an additional challenge — particularly those on the CBS Masterworks and Denon labels from the middle 1980s. The quality of the CBS transfers from analog masters holds up extremely well today, even against later Super Bit Mapping transfers; but they and the Denon CDs have "pre-emphasis".
What is Pre-emphasis?
Generally, pre-emphasis refers to raising (emphasising) the higher frequencies before transfer to the storage or transmission medium, in order to keep them above the noise level and optimise the signal to noise ratio. It is done with simple component networks and the inverse function is applied on replay, so as to restore the music to its original levels at all frequencies, while reducing any unwanted noise added in the recording and replay process. Pre-emphasis was first used with 78 rpm records, which clearly benefited from reduction in noise, LPs and analog tapes used it, it is used in FM radio, and Dolby-B noise reduction, applied to cassettes, is a more sophisticated implementation. The engineers who designed CDs made provision for pre-emphasis, but it was optional and was generally dispensed with from about 1990, though it is still easily available if the engineers select it.
Pre-emphasis and the CD
The pre-emphasis of the high frequencies in the music on CDs is similar to the pre-emphasis that is used in the old analog media. It consists of an increase in the treble frequencies, starting around 500 Hz (+0.1dB) and rising gradually to a level of 9.5 dB at 20 kHz (see graph and table below). The difference between music that has been pre-emphasised in this way, without the corresponding de-emphasis, and music that has not been pre-emphasised is quite audible, though in varying degrees depending upon the style of the music and the instruments playing, and sounds similar to playing a Dolby-B cassette with the Dolby playback turned off.
When pre-emphasis is employed, the official standard demands a signal on the CD indicating this. But as generating this signal and feeding it into the datastream is a separate operation, it is by no means certain that all CDs that are pre-emphasised have one, and there may be CDs that have the signal without pre-emphasis. There is no automatic check for such potential discrepancies. See here.
In addition to the music samples, the datastream of an audio CD has what are called subcodes. There are eight of these, but only two are actually used for audio CDs. These codes are chopped up, interleaved with the music data and repeated 75 times a second so that they can be recovered by the player without errors. The codes contain the time information and the track numbers that are used by the CD player to find the music you want to play, and to show this information on its display. The Q code also contains a "flag" that is either "up" or "down" (in digital terms "1" or "0") to indicate that pre-emphasis has been used, or not. In addition to their presence in the datastream during the tracks, the subcodes are also a feature of the "lead-in", which is what the player looks at when you insert the CD. The first thing that the player does is to read this lead-in or Table of Contents (TOC) for information about the number and position of tracks on the CD. This is what is happening while you are anxiously waiting to start listening to the music, and the more tracks there are on the CD the longer it takes to read the TOC. Pre-emphasis can only be turned on or off between tracks: when the player detects that the flag is "up", it engages a part of the circuit that does the de-emphasis.
CDs with Incomplete Subcode Data
As required by the Red Book standard, most CDs that use pre-emphasis advertise the fact to the player by the raised flag, both during the track and in the TOC. My CBS CDs all do this, but I have come across CDs that raise the flag only during the tracks and not in the TOC. Playing these CDs in a normal player is not a problem, as players do not rely on the TOC flag and actually use the flag embedded in the music to switch the de-emphasis network. (I am assuming that all CD players are able to detect and correct pre-emphasis, though it has been suggested by others that this is not so.) But when these CDs are loaded into the current version of the three ripping programs I have investigated, they are dealt with differently by each program.
The six CD recordings that I have with this problem were all mastered by Denon in Japan around 1985.
Three of them are music CDs:
33C37–7546: Schubert String Quartets
33C37–7601: Schubert String Quintet
60CO–1566/7: Mahler Symphonies 9 & 10 (Adagio)
Three of them are Test CDs:
HFN003: HiFi News and Record Review Test Disc
In the accompanying booklet, only Track 54 is listed as having pre-emphasis, but actually, Tracks 1–14 (all the music tracks) have pre-emphasis also.
PG–6006: Anechoic Orchestral Music Recording
Tracks 1–44 (the music tracks) of this CD have pre-emphasis.
In addition the NAB Broadcast and Audio System Test CD uses the same announcements to identify left and right channels as those used on HFN003, with the same pre-emphasis. The remainder of the tracks — the important ones — have no pre-emphasis.
Apart from HFN003, the test CDs are not widely used by music lovers, but there are undoubtedly other Denon CDs, and perhaps some made by other manufacturers, that present this problem of identifying the presence of the pre-emphasis flag when ripped.
How to Rip CDs that have Pre-emphasis.
At the time of writing, the current version of EAC (v.1.0 beta 3) will indicate in the track listing upon loading the CD whether the pre-emphasis flags are set in the TOC, but to discover whether they are set in the actual tracks (as with the Denon CDs discussed above) you need an earlier version (e.g. v.0.95 prebeta 3). These flags in the Q subcode of the music tracks can be read by using the menu item “Action/TOC Alterations/Detect TOC Manually” — you can hear EAC doing its job as it goes through the tracks. This option was removed from later versions of EAC to address possible legal problems in Germany. Neither of the other two programs gives information about the flags.
I have used three different ripping methods:
1. If the flags are set in the TOC, iTunes will recognise them (though not report this) and tacitly apply de-emphasis automatically. How well the de-emphasis is done is shown below. iTunes will not process correctly a CD with a flag only in the music track and not in the TOC.
2. Foobar2000, also, does not alert you to the fact that you are dealing with a CD that has some or all of its tracks pre-emphasised: for that you need EAC. Foobar2000 with the IIR filter plug-in set to “CD de-emphasis” will de-emphasise properly. This involves downloading the plug-in and setting it up within Foobar2000. Since it is necessary to select the de-emphasis manually, care needs to be taken when dealing with CDs that have a mixture of tracks with and without pre-emphasis that those tracks that are not pre-emphasised do not pass through the filter. (Obviously, this demands two runs of the software.) A pre-emphasised CD in good condition can be ripped accurately with Foobar2000 but, given that the CDs that need de-emphasis are up to 30 years old, it is possible that EAC is the only program that can read data from some of them accurately if they have been abused over the years.
3. Use EAC to rip and then deal with the de-emphasis separately. The question is then, how to de-emphasise the music after it has been accurately ripped by EAC.
A. The obvious solution is to make up an analog circuit with the required frequency response. This makes perfect sense, as the original pre-emphasis was most likely done in the analog domain. A suitable circuit is similar to one used for RIAA correction and such a circuit can be modified quite easily. The RIAA circuit has three turnover points at 3180 µs, 318 µs and 75µs. CDs require only two turn-over points, at 50 µs and 15 µs. The shape, plotted in dB versus log(frequency) is shown below (see the table at the end of the page for identification of the frequencies):
B. Synthesize the response in the digital domain. If you know how to write a program to do this, you are ahead of me. I use my DAW, Samplitude Pro-X, which has two options:
i) the FFT plug-in has a de-emphasis preset.
ii) I have made a de-emphasis preset for the EQ116 plug-in, the possible advantage of which is that it can be made to have the same phase response as an analog circuit, though I am not sure whether this makes any difference to the sound or not.
Using the tone generation facility of Cool Edit Pro, I made a mono wavefile of 37 different frequencies, each 10 seconds long and spaced at 1/3 octave intervals. I read this into Samplitude and burned a CD-R. I set the pre-emphasis flag (Samplitude puts this in the tracks and in the TOC!) but did not actually pre-emphasise the signal, as all I wanted to do was to test the accuracy of the de-emphasis applied after the ripping process. I ripped this CD with iTunes (with automatic de-emphasis), EAC (flat) and Foobar (with its IIR de-emphasis filter). Then I played the ripped files in Samplitude and measured the responses using RME's Digicheck.
The table below shows the results. The first two columns list the third octave frequencies used, identified in the graph above as f1-f37, and the third column gives the calculated response in decibels of a perfect de-emphasis circuit (all the decibel quantities should have a negative sign in front of them). In column four the response of the iTunes (v.126.96.36.199) de-emphasis is seen to be rather inaccurate. Column five shows Foobar2000 with its filter: better than iTunes, but not perfect. The last two columns show the result of correcting the flat EAC rip with the two Samplitude plug-ins. I tuned the EQ116 plug-in for the best result that I could achieve, and the FFT plug-in used the CD de-emphasis preset provided. These two are close to each other and to the ideal, though (at least theoretically) the EQ116 method may be preferable as it has the same phase response as an analog filter and will thus correct the phase of an analog pre-emphasis filter.
Ripping CDs with pre-emphasis is clearly fraught with difficulties, and there may actually be more problem cases than I have discussed here. Since music is meant to be enjoyed, perhaps the best advice is "When in doubt, trust your ears!"