Archiving C64 Tapes Correctly(pagetable.com)
pagetable.com
Archiving C64 Tapes Correctly
https://www.pagetable.com/?p=1002
36 comments
We know that he transmitted on illegal frequencies. (We sometimes heard his voice on our TV, I'm totally not kidding.)
That's most likely about power levels, and not illegal frequencies.
I worked at an AM radio station that was built in the middle of nowhere. Eventually the suburbs surrounded it, and the neighbors would complain about the sound of our station coming out of all kinds of electrical things.
That's what you get when you move into a house next to a radio station.
That's most likely about power levels, and not illegal frequencies.
I worked at an AM radio station that was built in the middle of nowhere. Eventually the suburbs surrounded it, and the neighbors would complain about the sound of our station coming out of all kinds of electrical things.
That's what you get when you move into a house next to a radio station.
I had an obscure machine called an Interact[1]. It had a built-in tape drive, and my issue with that was the read head kept getting out of alignment. I had to regularly adjust it. (Probably because I transported it to friends' houses in a wagon.)
My first hardware hack was cutting the trace to the HLT line on the 8080A CPU and wiring up to a toggle switch to act as a pause button.
Also good times. :)
[1]: http://oldcomputers.net/interact.html
My first hardware hack was cutting the trace to the HLT line on the 8080A CPU and wiring up to a toggle switch to act as a pause button.
Also good times. :)
[1]: http://oldcomputers.net/interact.html
unlike the C64, the CoCo used normal, consumer level cassette machines
The problem I had was tape reliability
And now you know why the C=64 didn't use consumer tape recorders.
Ordinary tape recorder speed and volume of recording can vary from machine to machine. The 64's datasette was more strictly engineered, for compatibility between systems.
It's why on CP/M machines, you could hook up a generic tape recorder to record your programs, or you could spend extra money for a "data" cassette recorder, which helped improve reliability.
(The C=64 part is according to what I've read. I never had a datasette for it. I was one of the fortunate who started out right with a 1541.)
The problem I had was tape reliability
And now you know why the C=64 didn't use consumer tape recorders.
Ordinary tape recorder speed and volume of recording can vary from machine to machine. The 64's datasette was more strictly engineered, for compatibility between systems.
It's why on CP/M machines, you could hook up a generic tape recorder to record your programs, or you could spend extra money for a "data" cassette recorder, which helped improve reliability.
(The C=64 part is according to what I've read. I never had a datasette for it. I was one of the fortunate who started out right with a 1541.)
Also longer tapes could be affected more by stretching.
For computers they sold cassettes in short lengths such as 15-minutes.
The Datasette had a digital interface to the computer. This and the known calibration went a long way to help it.
For computers they sold cassettes in short lengths such as 15-minutes.
The Datasette had a digital interface to the computer. This and the known calibration went a long way to help it.
Can anyone recommend decent method of decoding such signal? Here is description[1], it uses variant of frequency-shift keying (continuous phase FSK?).
According to Wikipedia[2], which cites service manual[3], actual hardware just clips this signal, and (probably) software on C64 just detects zero crossing, using CPU clock for sampling. Are there better approaches, more tolerant to noise and inconsistent tape speed? Maybe using matched filters? Almost everything that I found was about radio which does not have problem of "changing speed of time": you have to correct drifting phase, but not frequency. I know only basics of DSP.
I'm interested primarily in ZX Spectrum, not Commodore 64, but it has almost the same modulation/encoding. It uses regular tape recorder, input port with two levels (above zero and below zero), no dedicated clock for sampling, and software decoding. Routine in ROM finds zero crossings and measures time of pulses. I think it's possible to do better with modern computers, and achieve better noise and tape speed tolerance.
[1] http://wav-prg.sourceforge.net/tape.html
[2] https://en.wikipedia.org/wiki/Commodore_Datasette#Physical_c...
[3] ftp://www.zimmers.net/pub/cbm/schematics/datassette/C2N-1530-1531_Service_Manual_Preliminary_314002-002_(1984_Oct).pdf
According to Wikipedia[2], which cites service manual[3], actual hardware just clips this signal, and (probably) software on C64 just detects zero crossing, using CPU clock for sampling. Are there better approaches, more tolerant to noise and inconsistent tape speed? Maybe using matched filters? Almost everything that I found was about radio which does not have problem of "changing speed of time": you have to correct drifting phase, but not frequency. I know only basics of DSP.
I'm interested primarily in ZX Spectrum, not Commodore 64, but it has almost the same modulation/encoding. It uses regular tape recorder, input port with two levels (above zero and below zero), no dedicated clock for sampling, and software decoding. Routine in ROM finds zero crossings and measures time of pulses. I think it's possible to do better with modern computers, and achieve better noise and tape speed tolerance.
[1] http://wav-prg.sourceforge.net/tape.html
[2] https://en.wikipedia.org/wiki/Commodore_Datasette#Physical_c...
[3] ftp://www.zimmers.net/pub/cbm/schematics/datassette/C2N-1530-1531_Service_Manual_Preliminary_314002-002_(1984_Oct).pdf
In FSK (radioteletype or 'RTTY' is a well-documented case used commercially for decades) the most important factor in accurate recovery (especially for deteriorated signals) is the quality of the filters and clocking. Cheap circuits avoid using them, and so the methods they fall back on give poorer results.
High-quality circuits will have excellent results even when the signal level is below the noise level. Since your software has to work with the output of an A-D converter, that has to be as good as you need it to be.
A couple of widely-used, vintage standards:
Manchester encoding: https://www.allaboutcircuits.com/technical-articles/manchest...
Kansas City Standard: http://www.dabeaz.com/py-kcs/index.html
On the other side of archiving, get the best tapes and recorders you can manage, and get a good, strong signal to the tape. (Time will weaken that signal.) Then keep them shielded from magnetic fields.
High-quality circuits will have excellent results even when the signal level is below the noise level. Since your software has to work with the output of an A-D converter, that has to be as good as you need it to be.
A couple of widely-used, vintage standards:
Manchester encoding: https://www.allaboutcircuits.com/technical-articles/manchest...
Kansas City Standard: http://www.dabeaz.com/py-kcs/index.html
On the other side of archiving, get the best tapes and recorders you can manage, and get a good, strong signal to the tape. (Time will weaken that signal.) Then keep them shielded from magnetic fields.
For Spectrum, at least as far as ROM loader is concerned, decoding the signal is basically the matter of deciding whether a pulse is length t0 or t1. Technically it's FSK, but since each symbol is only one period long, it's more convenient to talk about pulse lengths. I don't think traditional matched filters are any help here.
With a modern computer you could record lengths for all impulses present in the signal and then find a threshold that best divides the two groups. You could then do probabilistic decoding. E.g. if checksum doesn't match, try flipping the bits that are closest to the threshold and are most likely to have been decoded incorrectly.
With a modern computer you could record lengths for all impulses present in the signal and then find a threshold that best divides the two groups. You could then do probabilistic decoding. E.g. if checksum doesn't match, try flipping the bits that are closest to the threshold and are most likely to have been decoded incorrectly.
But maybe such "binarization" throws out information which might be useful? ZX Spectrum generates rectangular signal, after writing to tape and reading back, it becomes heavily low-pass filtered. So, I thought, if we take typical tape frequency response, pass ~1KHz and ~2KHz rectangle pulses through it, we get typical "0" and "1" signals as passed through tape, and if we cross-correlate them with tape signal, maybe we can achieve better results than by zero crossing detection?
Also, what I felt unattractive in binarization/zero-crossing is that you should do band pass filtering anyways to fix DC offset and ripples that can affect zero crossings: https://imgur.com/ys78f1M. So, maybe if single filter is used for both frequencies of "marks" and "spaces", then we are doing it wrong and filters matched to marks and spaces could do it better? As alternative to filtering we can use adaptive binarization threshold and then morphology on binarized signal (AFAIK, this approach is popular in detecting various barcodes on photos).
Also, what I felt unattractive in binarization/zero-crossing is that you should do band pass filtering anyways to fix DC offset and ripples that can affect zero crossings: https://imgur.com/ys78f1M. So, maybe if single filter is used for both frequencies of "marks" and "spaces", then we are doing it wrong and filters matched to marks and spaces could do it better? As alternative to filtering we can use adaptive binarization threshold and then morphology on binarized signal (AFAIK, this approach is popular in detecting various barcodes on photos).
As a kid I sometimes played our PET tapes through a tape recorder to listen to the data.
At the time I wondered if data could be written to the tape to play a particular song. Anyone else wonder this and actually try it?
At the time I wondered if data could be written to the tape to play a particular song. Anyone else wonder this and actually try it?
There was a “Waz” hack back in the day that did the opposite - it read the audio off the cassette and somehow replayed it from the SID ...
Otherwise the closest thing I can think of is Aphex Twin’s windowlicker where a particular sound in the song displays Richard D James face in a spectrum analyser ...
Otherwise the closest thing I can think of is Aphex Twin’s windowlicker where a particular sound in the song displays Richard D James face in a spectrum analyser ...
The super-simple approach to this is to set up the sid and then flick the volume from min to max and back for every 1 read from the tape input in a tight loop. You can probably do many much better variations of this, but that's enough to get recognisable music out.
(http://www.eeggs.com/items/34824.html if you haven’t seen it before)
Don't throw away the old unreadable tapes, I'm sure they can be read other ways.
Long ago I built an interface to the 1531 to allow tape duplication via my external Aiwa tape deck. I tapped the signal somewhere near the output of the device, sent it through some TTL gates to make its shape more squarish, then it went through a voltage divider into the deck input. Results were astonishing: I didn't have any tool at hand to condition the signal, but merely making it more loud and clean did wonders; no more hours spent adjusting the head azimuth. To be clear, a cassette which required adjustment could turn into a perfectly readable copy if recorded without adjusting the azimuth. In other words the problem wasn't the azimuth but low signal level (which is also a product of bad head alignment).
Now back to those old cassettes. If saving them from one device to another or sampling and filtering them doesn't work, it may just be the poor 1531 that cannot cope with a signal too much dirty because it lacks the brains to recognize it, but our eyes could indeed spot the problem. No kidding, In the old days I spent hours for every single music track cleaning them from vinyl scratches by zooming the waveform in Cool Edit Pro until I could see every single sample. Scratches became clearly visible for having a very different shape, often a few samples at a level not compatible with normal music. With time I learnt to spot them even more easily, and after finding them and verifying that they were actual scratches, I spent even hours for a single track cleaning them by reshaping the waveform, by hand, one sample at a time. The reason behind this was Cool Edit's vinyl cleaning algorithm which at the time sounded like garbage.
Cleaning digital data should be much easier: one can load the waveform into an editor then zoom it until the signal becomes visible. It doesn't have to be readable by a 35 years old 1531, but merely recognizable by a human. Then either correct it manually (slow!), or write some macros to put x 1 or 0 samples where the mouse is according to a keypress. Then one can scroll the wave to a point where it starts an unknown bit; we know both data speed and carriers speed, therefore we know how many samples a bit is made of, this way when an unknown bit is met we know what to search for and based on this interpretation press a key which turns that series of unreadable samples into a nice 0 or 1 of the right duration and frequency. Still slow, but doable.
Analyzing the waveform at single sample level would also make possible to correct wow and flutter: again, we know the carriers frequencies and tape speed, so we know for sure how much long a bit must be and apply corrections where necessary.
Also, it might be interesting to sample all tracks at the same time into a multitrack file just like the cassette was a single side media. The reason being if the tape got damaged at a point, we can analyze the other track as well (yes, it's reversed but we're aware) to see if recovering one of them can help to save the other one too just by comparing a probably very similar damage done to two signals, one of which hopefully is easier to recover.
Now back to those old cassettes. If saving them from one device to another or sampling and filtering them doesn't work, it may just be the poor 1531 that cannot cope with a signal too much dirty because it lacks the brains to recognize it, but our eyes could indeed spot the problem. No kidding, In the old days I spent hours for every single music track cleaning them from vinyl scratches by zooming the waveform in Cool Edit Pro until I could see every single sample. Scratches became clearly visible for having a very different shape, often a few samples at a level not compatible with normal music. With time I learnt to spot them even more easily, and after finding them and verifying that they were actual scratches, I spent even hours for a single track cleaning them by reshaping the waveform, by hand, one sample at a time. The reason behind this was Cool Edit's vinyl cleaning algorithm which at the time sounded like garbage.
Cleaning digital data should be much easier: one can load the waveform into an editor then zoom it until the signal becomes visible. It doesn't have to be readable by a 35 years old 1531, but merely recognizable by a human. Then either correct it manually (slow!), or write some macros to put x 1 or 0 samples where the mouse is according to a keypress. Then one can scroll the wave to a point where it starts an unknown bit; we know both data speed and carriers speed, therefore we know how many samples a bit is made of, this way when an unknown bit is met we know what to search for and based on this interpretation press a key which turns that series of unreadable samples into a nice 0 or 1 of the right duration and frequency. Still slow, but doable.
Analyzing the waveform at single sample level would also make possible to correct wow and flutter: again, we know the carriers frequencies and tape speed, so we know for sure how much long a bit must be and apply corrections where necessary.
Also, it might be interesting to sample all tracks at the same time into a multitrack file just like the cassette was a single side media. The reason being if the tape got damaged at a point, we can analyze the other track as well (yes, it's reversed but we're aware) to see if recovering one of them can help to save the other one too just by comparing a probably very similar damage done to two signals, one of which hopefully is easier to recover.
I wonder if they ever tried duplicating a bad tape on a higher end audio double cassete deck, to a brand new tape.
It sounds like most of the devices used in the article would have low end quality motors, heads, etc. Such that a (still glitchy) copy on a new tape might read better.
I suppose it's possible the important bits of a high end cassete player aren't any better. Audiophile equipment is sometimes funny that way.
Edit: Found the full service manual for the Commodore 1530/1531 Datasette: ftp://www.zimmers.net/pub/cbm/schematics/datassette/C2N-1530-1531_Service_Manual_Preliminary_314002-002_(1984_Oct).pdf (hrm, guess HN doesn't linkify ftp:// urls?)
It sounds like most of the devices used in the article would have low end quality motors, heads, etc. Such that a (still glitchy) copy on a new tape might read better.
I suppose it's possible the important bits of a high end cassete player aren't any better. Audiophile equipment is sometimes funny that way.
Edit: Found the full service manual for the Commodore 1530/1531 Datasette: ftp://www.zimmers.net/pub/cbm/schematics/datassette/C2N-1530-1531_Service_Manual_Preliminary_314002-002_(1984_Oct).pdf (hrm, guess HN doesn't linkify ftp:// urls?)
> I suppose it's possible the important bits of a high end cassete player aren't any better. Audiophile equipment is sometimes funny that way.
It's a different problem domain too. A non-linear tape head/amplifier may help with recovering pulses but have unpleasant distortion for audio.
It's a different problem domain too. A non-linear tape head/amplifier may help with recovering pulses but have unpleasant distortion for audio.
Instead of duplicating the tape, you could avoid the generation loss by connecting the high quality tape player directly to the computer. You might need to insert an equalizer between them to make the frequency response match the cheap tape players these computers were designed for.
A key method I've learned through various sources for ensuring good conversion from WAV to TAP is to invert the whole waveform in Audacity through Effects -> Invert before the actual conversion process.
I can also recommend the command line tool UberCassette for easy conversion: http://www.retroreview.com/iang/UberCassette/
I can also recommend the command line tool UberCassette for easy conversion: http://www.retroreview.com/iang/UberCassette/
Huh, that's weird! How does inverting a waveform affect the process?
According to post #5 in this Lemon 64 thread[1], sound cards or TV amplifiers may invert signals from tape. I have actually just tried it when running recorded tapes in emulators but it has proved useful in cases when non-inverted tapes have failed to load.
[1] https://www.lemon64.com/forum/viewtopic.php?t=45206
[1] https://www.lemon64.com/forum/viewtopic.php?t=45206
I don‘t have tapes but more than 100 old floppies.
I still have the original 1541 II the disks were written with. I‘m willing to throw some money at the task but can‘t spend more than a day or two for this. Any tips for a realistic plan to save my floppies?
Get in touch with 4am (@a2_4am) and DiskBlitz (@diskblitz) on Twitter — although they're doing it on Apple II disks, there's some pretty sophisticated tooling, and it should theoretically be able to read C64 disks too. 4am also has a lot of experience reading old disks using soap and water :-)
c64.com and other archive sites often take donations of floppies and will try to make images of them and return copies of the data in return for being able to keep any non-personal/private data.
But if you have the original drive you might want to try to copy it off first rather than risk mailing it off anywhere. There are multiple options for that including the 1541 Ultimate II (1) if you have a C64 to do the actual copying (as far as I understand anyway), and USB adapters for the 1541 called XU1541(2) and XUM1541/ZoomFloppy (3). The USB adapters might be your easiest option; the ZoomFloppy is listed at $35 (note: I have not dealt with this company, so can not guarantee they're reliable).
If you do image it, I'd strongly recommend you submit any non-private content to one or more of the archives - there's still plenty of stuff floating around that's not archived anywhere.
(1) https://en.wikipedia.org/wiki/1541_Ultimate
(2) https://spiro.trikaliotis.net/xu1541
(3) http://www.go4retro.com/products/zoomfloppy/ and http://www.root.org/~nate/c64/xum1541/
But if you have the original drive you might want to try to copy it off first rather than risk mailing it off anywhere. There are multiple options for that including the 1541 Ultimate II (1) if you have a C64 to do the actual copying (as far as I understand anyway), and USB adapters for the 1541 called XU1541(2) and XUM1541/ZoomFloppy (3). The USB adapters might be your easiest option; the ZoomFloppy is listed at $35 (note: I have not dealt with this company, so can not guarantee they're reliable).
If you do image it, I'd strongly recommend you submit any non-private content to one or more of the archives - there's still plenty of stuff floating around that's not archived anywhere.
(1) https://en.wikipedia.org/wiki/1541_Ultimate
(2) https://spiro.trikaliotis.net/xu1541
(3) http://www.go4retro.com/products/zoomfloppy/ and http://www.root.org/~nate/c64/xum1541/
I can confirm that the 1541 Ultimate II works very well.
I got my floppies copied directly to a USB stick, with no problems.
http://1541ultimate.net https://ultimate64.com
http://1541ultimate.net https://ultimate64.com
I went through this exercise about 15 years ago. At the time, the solution was to build a cable that attaches to the computer's parallel printer port at one end, and the 1541 at the other, and run special software under DOS. There may have been software for newer Microsoft OS's and/or even Linux later. Start here: http://sta.c64.org/xcables.html but beware of bit/link rot.
[deleted]
Send them to the Internet Archive.
If one was able to collect, say, a couple dozen tapes of the same game or magazine supplement, and the tapes all had different bits lost, would it be possible to recreate the tape with the WAV representations?
May be a way to find a bunch of different tapes on ebay, and recreate old classics.
EDIT: this is actually performed later in the article, whoops, I should have read the whole thing before commenting.
May be a way to find a bunch of different tapes on ebay, and recreate old classics.
EDIT: this is actually performed later in the article, whoops, I should have read the whole thing before commenting.
In the article, they did this at the file level.
I admit I did not read that far. I'm remedying that right now.
That's what I was thinking as well; I wonder if it would be as simple as recording all of them, aligning them in Audacity and playing back the resultant mix? Any dropouts on one tape would be largely overruled by the (strong) signal from all the good tapes?
Just have each tape owner read it to the best ability of their drive, and then compare the results centrally: for each byte, take that which occurs in the most results. I would assume that where tapes fail, they all fail differently.
And that should also work for floppy disks, right?
And that should also work for floppy disks, right?
Aligning them is non-trivial because of "wow and flutter" caused by variation of the tape speed. Even if your tape player has perfectly stable tape speed the original recorder wouldn't have had it. Maybe you could calculate some inverse-wow-and-flutter distortion that minimizes the difference between the tracks.
[deleted]
I was quite busy with that system, and ended up building a library of several dozen tapes, containing off the shelf software, but mostly software and data that I'd created and saved.
The problem I had was tape reliability. At the time, I estimated that each tape had about a 2-3% chance of becoming corrupt per day. Having more time than sense, I ended up going forward with an intensive, structured tape backup/duplication scheme, not unlike tape backup rotations I'd end up administering professionally some years later.
With the gear that I had, a tape duplication took 30-40 minutes, so this was a time consuming process, but it worked.
Why did my tapes go bad so frequently? In retrospect, there were two factors:
1. There was a high voltage step down transformer on a power pole about 20 feet from my room. I remember as a child that this transformer had problems at least a couple of times. In one case, it spectacularly sprayed sparks down on our back yard, and the electric company came out and messed with it multiple times besides.
2. In the other direction, our neighbor had a very tall CB antenna. Probably 30 feet or more, and the base of this thing was only 60 or so feet from my room. We know that he transmitted on illegal frequencies. (We sometimes heard his voice on our TV, I'm totally not kidding.) It would not surprise me if he was using illegal power levels too.
Good times.