Penguin
Note: You are viewing an old revision of this page. View the current version.

InNeedOfRefactor

Work in progress.

http://freshmeat.net/projects/phonoripper/ might be interesting --AristotlePagaltzis

As a Christmas present, I have decided to convert some my of parents vinyls to CD. In doing so, I hope to remove some of the hiss and clicks you get on old vinyls, and also make sure the contents are preserved - they were either vinyl-only recordings (Tararua College choir, 1961) or unavailable in any other medium these days.

I'll add to this wiki as I go, so excuse the journal-like nature. When I've finished I'll tidy it up, or let a WikiGnome do it for me :)

Sorry, this is too longwinded and unwieldly a mess for this gnome to clean up, despite following along with great interest. :) --AristotlePagaltzis

It'll be finished soon, and most of it can be condensed down into 'Use decent recording hardware, preamp if possible, gwc will clean your audio nicely, but if you are using 2.6 and an ATAPI CDWriter, use Nero under Windows to burn to CD. --DanielLawson

Hardware used:

I have a Celeron-1300, running Windows 98 (wait a bit...), which has an SB Audidy 2 Platinum eX. As far as I know, there aren't decent drivers for the Audigy under linux, hence staying under Windows 98. That said, I'm not using it very much - it has RCA inputs on its external unit, which means I can plug my turntable directly in.

This machine is probably fast enough for most processing, and has 256 MB of ram.

I also have an Athlon 1800+ XP, with 512 MB of ram, running Gentoo linux. This is what I'm doing most of my processing under, if I can at all help it.

First Attempt

I used DartPro under Win98 to record the vinyl. I made two wav files, one for each side of the vinyl. I used !DartPro? because trogs suggested it as a useful tool, and I thought it might do all of my processing for me very easily. If it will, I couldn't work out how in the 10 minutes I spent playing with it (mainly, I couldn't get it to play any wave files, so it was very hard to tell how well its effects were working).

I had seen an article on gramofile on LinuxJournal or SlashDot or somewhere, and thought I'd give it a try. It's located at http://panic.et.tudelft.nl/costar/gramofile/. One thing I noticed - it's had no work done in over two years!. There is actually a fork of the project at http://www.schou.dk/linux/gramofile/ which has advanced it somewhat, although the author of the fork calls it a hostile takeover, apparently in complete ignorance of how OSS projects work.

Using gramofile 1.6 (the 'old' version) was pretty easy. It locates track boundaries, separates the input .wav into tracks, and then performs some click reduction on it. The track finding algorithm was a bit enthusiastic, and the click reduction was ok but there was still a large amount of noise.

So, I tried using audacity, which also featured in an article I read recently. Despite some UI problems (you cant skip to an arbitrary point in a .wav while its playing, you have to stop, wait a bit, click where you want the cursor to go to, then press play again) it seemed ok. It provides noise reduction and so on, but most of its effects were more focussed on 'effects' (wahwah, etc) rather than cleaning up old vinyl recordings.

I also had a play with !DartPros? cleanup functions. It has a wide range of predefined tools - !DeHiss?, !DeClick?, !DeNoise?, !DeHum? and so on, and allows you to build up a custom filter using mixtures of these filters.

Both Audacity and !DartPro? allowed me to amplify the tracks, which was needed as the output from my turntable was not amplified, and I dont have access to an external amplifier/eq unit.

I wrote a CD with my freshly cleaned and split tracks. Well, I wrote 2 coasters under linux (never authored an audio CD under linux before, I must be missing something like padding or reversing audio bits or something). I copied the tracks to the windows box, and wrote them to CD using Nero, at 52x. The resulting CD sounded ok, but the noise got progressively worse as the CD played (ie, later tracks sounded awful). Does anyone know if writing audio cds at high speed is a BadThing? ?

Second attempt:

Found some more information at http://www.xena.uklinux.net/Linux/audio.html. This guy started using gramofile, and then switched to using gwc, the Gnome Wave Cleaner. I'm installing this right now, as well as snd and sox, which both claim to be good tools

GWC works really well. I followed the suggested notes at the above URL, which I'll summarize here:

First of all, I amplified the track. GWC seems a bit buggy with this - it won't always amplify correctly. If the waveform doesn't increase in amplitude, it hasn't been amplified. I found if you wait till after processing to amplify, some noise still gets through.

(Jeff Welty here - see my comments below about GWC amplify)

  • Run the strong declick tool. The first time I ran this I got about 7000 clicks removed. The second time, about 400, and the third time, about 20. This was 'good' enough, so I moved onto the next step:
  • Run the decrackle tool. This removed most of the remaining crackle on the recording.

(Jeff Welty here -- it's worth a try at skipping this step to see if the denoise tool will clean out the remaining crackle. Avoid decrackling if possible because it attenuates the high frequencies)

  • Run the denoise tool. Its suggested to use any lead-in you can, but I only lead-out noise available. It seemed to do just as well. I ran it first with the defaults, then used the suggested settings on the url above. It seemed about the same really, although with the speakers I have on this computer it is hard to tell.

At this point, the tracks were a LOT cleaner and sounded great.

Unfortunately, Gramofile doesn't split the tracks out correctly any more. I figured I could run Gramofile over the unprocessed track and use its timing, or I could split the waves up manually. Gramofile did a fairly good job of picking the tracks out of the original wave file, although it needed some tweaking, and sometimes removed too much between songs. It writes out a filename.tracks file which you can edit, and in this case copy to a different name so that Gramofile will use on the processed wave file.

By this time I have a directory with 12 .wavs, named side1-1.wav to side1-6.wav and side2-1.wav to side2-6.wav. Nearly there!

A bit more reading and I found that cdrdao should let me write Audio cds where cdrecord failed earlier. It was worth a shot, so I decided to try it. I previously found another page describing the use of gramofile and audacity to clean up vinyl recordings, which suggested using cdrdao and a .toc file to write the audio tracks. Based on the notes at this page: http://www.togaware.com/linux/survivor/Recommended_Recording.html , the following shell script built a nice .toc file for me
(echo "CD_DA"; for i in -; do echo "TRACK AUDIO"; echo FILE \"$i\" 0; done )> cd.toc
The resulting file looks like
CD_DA TRACK AUDIO FILE "side1-01.wav" 0 ,...
And I should now be able to write an audio cd using the following command
cdrdao write cd.toc

cdrdao expects a symlink from /dec/cdrecorder to the cdrecorder device. Recent cdrdao support the ATAPI interface, and its actually sufficient to symlink /dev/cdrecorder to your IDE CDWriter and it'll work fine. I also found i had to specify the driver option, and I set a speed for safety (my first write failed with a buffer underrun)

cdrdao write --speed 16 --device generic-mmc cd.toc

This time the write succeeded, although upon a visual inspection of the CD afterwards it seemed as though the disk was not written to, and it wouldn't play in either the computer or the stereo. Back to the drawing board.

I burnt a CD in Nero, but had the same problem as with the very first copy I made - or a similar problem anyway. The music started off alright, but very quickly got very noisy, despite sounding OK on the computer.

Third Attempt

After a bit of thought, I realised one thing I might have been doing wrong. The GWC amplify tool didn't seem to work particularly well, and when it did work I think I was using it wrong. It puts up a prompt with 5 entry boxes - Left Channel Stard, End, Right Channel Start, End, and Feather Width (set to 2000, I left it). It stated the the maximum gain without clipping for this track was 25.something, which was a figure I'd seen elsewhere. So I set the 'End' values to 25. Which, in hindsight, probably meant an increasing amplification as the wave progressed, and which would have accounted for the increased noise further on in the CD and the poor noise correction performance - I doubt it can handle fixing an increasing noise source.

(Jeff Welty here -- Yes, the Left Channel Start value is the amplification to be applied to the start of the selected region in the left channel, and the Left Channel End value is the amplification to be applied to the end of the selected region in the left channel. This allows you to apply fade ins, or fade outs if you desire. The feather width is a little window that "feathers in" the amplification. If there was no feathering, then the amplification could introduce a click into the audio. End Jeff Welty's comment)

So this time, I used DartPro? (couldn't be assed finding another tool to do this, in case it wasn't the problem after all) to increase the gain, and then ran my declick/decrackle/denoise routine over this file. The waveform looked a lot better, and it sounded fine.

I also discovered that GWC will mark track boundaries just as well as gramofile will, and will export a .toc file for me too. Better and better. I processed both sides, merged the two .toc files (the .toc format allows you to specify a filename and a start/end offset into the file, so you dont actually need to split the original file up :)

Console output from GWC giving some timing information
6638 clicks repaired, 0 clicks marked, but remain unrepaired DECLICK in 65.811 real seconds 224 clicks repaired, 0 clicks marked, but remain unrepaired DECLICK in 18.114 real seconds 24 clicks repaired, 0 clicks marked, but remain unrepaired DECLICK in 17.278 real seconds DENOISE in 141.310 real seconds

This is on an Athlon 1800 XP+ with 512 MB of ram, running 2.6.0-test11 / gentoo, and with an 80 GB 7200 RPM 8 MB Cache Maxtor HDD. DECRACKLE information wasn't given, but it didn't take overly long - less than DENOISE. Each step requires some undo information (if you let it), and in the case of DECLICK and DENOISE requires a full undo file - as long as your original .wav

At this point, I still can't get cdrdao to actually write anything to CD. It says its writing, it seems to work ok, but it just doesn't do it. If I let it write at full speed, however, it writes, but then dies at some point and requires a reboot. Grrr.

Emerge yourself K3B - a very Nero-like interface on top of cdrdao. Has burnt me music CDs fine (using IDE-SCSI however) - give it a try and see what it does? -- CraigBox

I have k3b already, it looks good. However, it disclaims any support at all for ATAPI writing, and while my version of cdrdao should be fine, if I can't get it to write using cdrdao I don't see that a gui frontend will help. Also, at the point I was trying this, I had two large .wav files with a .toc splitting them up on offsets (generated by gwc) - k3b doesn't grok this (cdrdao does). It turns out neither does nero, so I split the tracks up anyway using gramofile and then burnt to cd using nero. I'll try k3b next time, for writing separated tracks to cd.

It seems a lot of the tools are still playing catchup with the cdwriting interface in 2.6. IDE-SCSI is either deprecated or just plain doesn't work, I've not yet established which. cdrecord has support for ATAPI, and it seems to work now from the command line, but its not so good from xcdroast. k3b claims to only understand the IDE-SCSI interface. Not tried eroaster or gcombust yet.

As horible as it is, XCDroast at least claims that it can burn using 2.6 + cdrecord, have you tried that? -- AlastairPorter

It does indeed work, however I've never had much luck with getting it to write audio CDS correctly. Having to wait 30 seconds for it to detect my cdwriter (known bug with xcdroast / atapi ) is also a problem. And, once more, its just a front end. I don't really see how using a GUI frontend over cdrdao or cdrecord will magically solve my problems. The exception is of course, if they are using command line options to those programs that I don't know about / aren't using, however I've tried a lot of things to no avail. I'll give them all another shot next CD I write, and when I have more blanks (15 or so wasted so far - when I write something with cdrdao, it seems to write the leadin and nothing else, yet claim to write everything correctly )

(later)

Having written this attempt to CD, it seems pretty good. I don't have the turntable hooked up to the stereo to compare the quality, but there are no pops, very little crackle. There is some residual noise still, and the gain is still too low, which makes the noise somewhat worse. When I amplified it before processing, it was amplified to the highest gain possible without clipping - however that was based (I think) on the highest amplitude seen, which was any number of the pops that occured there were about 10 - 15 times louder than the rest of the track. I'm not sure if this will improve the effect of the noise reduction algorithms, or if it will screw things up totally, but its probably worth a shot.

Selecting a better noise sample may also help, although I suspect its about as good as I can get. I'll move onto the next vinyl for now, which I know is in worse condition in some ways (at least one scratch bad enough to cause the needle to get stuck), but may have better properties overall. And the other thing to do is to try an external amplifer/equaliser unit, although I only have the stereo the turntable is connected to, which isn't overly great. I'm open to offers :)

Fourth Attempt. Or something.

OK. If you're going to bother doing this, you'll probably want at least the following equipment:

  • A good turntable. Turns out mine wasn't so hot. The stylus is probably the most important bit of this, and I suspect mine has been mistreated.
  • A decent set of headphones (for monitor purposes). And probably a decent set of speakers to check the full range
  • A preamp. The output gain from most turntables is low. This is normal. In one of the vinyls I was trying to restore, it was so low the computer wouldn't record anything. A preamp with monitor output, and simple input/output gain controls is probably fine.

Turntable sound output is down around 5mVolts compared to CD output at around 150mVolts. Also the signal needs something called frequency equalisation to restore the output to proper proportions. Disk Smith sells a turntable with inbuilt preamp for $100 that will do the job. --Paul Cleary

You should also make sure your soundcard has a good SNR. The Creative Audigy 2 Platinum Ex has a good one, although it's taken ages for Creative Labs to turn this product into anything more than gamers toy. There's plenty of high-quality professional level cards that will do this very well, and cost less than the Audigy / Extigy (I checked earlier, I cant remember the manufacturer of the cards I was looking at tho, will append later).

I use a cheap $60 Creative soundblaster card for this work and I get very good SNR. The thing to realise is that in all low to med cost soundcards, the audio input part that converts the analog signal to digital is virtually the same. The difference in the cost of the cards is paying for is all the fancy signal manipulation and sounds effects. A person recording from LP to a WAV file just needs the conversion part. I recommend you read Clive Backham's webpages on converting vinyl LPs to CDs (Google:Clive Backham convert lp cd). He discusses the whole process from start to finish, including equipment, soundcards and software. I use his Waverepair software for LP to CD conversions but it only works on Windows. --Paul Cleary

It's now Christmas day. I made a few CD copies of what I acheived last night, which sounded at the time like a great improvement. I'm sure it was a great improvement, however its wise to use the correct speed setting when playing back records. I'd been testing the input with a Faith No More LP (45), and forgot to flick the speed setting back down when I started with Music for the Feast of Christmas.

I'm thinking of stamping a label on the CDs I wrote calling them "Music for the Feast of Christmas, by the Chipmunks of Ely Cathedral"

Fifth Attempt.

It turns out my last two attempts were both almost there. Had I initially recorded at 33rpm, not 45, I would have had a perfect encoding I think. When I reprocessed the track after recording at 33, I had set the noise reduction parameter up to high, and I think this was causing a lot of clipping in any high 's' sounds (quite common in choral pieces, especially in latin, as a lot of this album is in).

So, back to drawing board.

Some notes about CD Burning, while I'm here. I burned a copy of my album with k3b, and it seemed fine. It played in my stereo, etc. I checked a few tracks. I then burned another copy, which I took round to my fiancees families place. It wouldn't play in their stereo. Upon closer inspection, it hadn't burnt. Same problem I was having with using cdrdao from the command line. I phoned home at some point to get them to play the one I had burned and tested, and it didn't work either!

So, once more, it seems as though I'm forced to say that writing audio cds under 2.6 with an ATAPI cdwriter (not using ide-scsi) is broken at this stage. If someone can do this reliably (that is, write audio cds under a 2.6.0-test11 or more recent, using cdrdao 1.1.7 or cdrtools 2.01-alpha20, with an ATAPI cdwriter using ide-cd, NOT ide-scsi - I dont care if you can do it under 2.4, or using ide-scsi, or with a scsi cdwriter), let me know how you manage it? Especially if you are authoring audio cds and not just copying them.


I successfully wrote an audio cd using ATAPI under kernel 2.4.24 and cdrecord version 2.0.something under Debian Testing (although my graphical front-end didn't know about passing the dev=ATAPI option to cdrecord, so I had to use the command line. --JohnMcPherson

$ cdrecord -v -dao dev=ATAPI:0,1,0 driveropts=burnfree -audio -pad *.wav

If you read my SoundProcessingNotes page, you'll note I continued to have problems, regardless of whether I used a command line approach or an ATAPI-aware GUI. However, my command differed from yours above, which could well have been the problem. I will investigate :) -- DanielLawson