The printer that wouldn't print: Fixing an IBM 1401 mainframe from the 1960s(righto.com)
righto.com
The printer that wouldn't print: Fixing an IBM 1401 mainframe from the 1960s
http://www.righto.com/2018/09/the-printer-that-wouldnt-print-fixing.html
48 comments
The very first programming job I had out of college was working on a DEC PDP-8 (not an 8-I or and 8-E, a plain 8), and I remember one time when it failed. I ran a diagnostic program (from paper tape) and the result indicated a hardware failure. We had a service contract, so we called DEC and they sent out a service person. He re-ran the diagnostic and confirmed the fault. I expected him to swap out the failed board, but no. He pulled the board out of the chassis, un-soldered a transister, and soldered in a new one. I can remember to this day, my amazement that he could repair a computer like that.
I vividly remember walking into my high school computer lab to find our PDP-11/34A partially disemboweled and someone hunched over the guts of the machine. An update tape from Digital required wire wrap changes which were included in the release notes.
I'm lucky if I follow 20% of these posts, but they are consistently my favorite items and I look forward to each and every one. Similar to the Dolphin Emulator blog, it's just a joy reading deep technical dives from people who are passionate about their work.
Even not understanding most of it, I always manage to learn several interesting tidbits.
Even not understanding most of it, I always manage to learn several interesting tidbits.
I find this stuff so incredibly fascinating that my hands are shaking...
love that this is done using a garbage scope (rigol). even the garbage of today is quite capable enough to debug the start of the art of yesteryear.
Oh come on, the Rigol is not a "garbage" scope! It is excellent at what it was designed to do and competitive in every way with similarly spec'd scopes from any other vendor.
I'm quite happy with the Rigol scope I used (DS1054Z), and would recommend it to anyone who wants an inexpensive scope.
I am a die hard LeCroy user and fan, but recommend the rigol when asked what to buy. For most people, my criticisms of it are of no concern (I am frequently chaining instruments with GPIB and operating them remotely, and the usb stuff on the rigol doesn't fit into that and just too slow.)
I remember reading an old urban legend about someone that got the exact sequence of characters on the print chain and wrote some code to print exactly that line on the 1403.
The legend tells of the 1403 dutifully printing that line by firing all 132 hammers at once, causing the chain to violently break and whipsaw itself out of the side of the printer cabinet.
The legend tells of the 1403 dutifully printing that line by firing all 132 hammers at once, causing the chain to violently break and whipsaw itself out of the side of the printer cabinet.
> You might expect that the 132 hammers align with 132 type slugs, so the matching hammers all fire at once, but that's not what happens. Instead, the hammers and type slugs are spaced slightly differently, so only one hammer is aligned at a time, and a tiny movement of the chain lines up a different hammer and type slug. (Essentially they form a vernier.) Specifically, every 11.1 microseconds, the chain moves 0.001 inches. This causes a new hammer / type slug alignment. For mechanical reasons, every third hammer lines up in sequence (1, 4, 7, ...) until the end of the line is reached; this is called a "subscan" and takes 555 microseconds. Two more subscans give each hammer in the line an option to fire, forming a print scan of 1.665 milliseconds. 48 print scans give each hammer a chance to print each character, and then the 49th print scan is used for error checking. (For more details of this timing, see Manual of Instruction, page 37.)
Based on this note at the bottom of the article that seems impossible, could be I'm just misunderstanding what he's saying though.
Based on this note at the bottom of the article that seems impossible, could be I'm just misunderstanding what he's saying though.
You are correct; it's impossible to fire all the hammers at once on the 1403 since they fire one at a time. Maybe the self-destructing printer is a different line printer, or maybe firing the hammers in the right sequence over-stresses the chain, but I suspect an urban legend.
I could imagine that setting up some kind of sequence that could make it either vibrate or resonate should still be possible. Basically set it up so that you get the maximum speed of hitting any characters. But like the sibling said, it could easily be a different printer too.
I quoted a number of those stories in my comment here: https://news.ycombinator.com/item?id=15876947#15893800
Approximate translation of the "print storage" paragraph: "Fetching characters to print via DMA consumed the entire memory bandwidth of the mainframe, so printing caused all computation to stop; unless you leased the 132-word 'print storage' hardware option for an extra $3000 per month (in 2018 dollars)." This add-on looks to be about the size of a largish suitcase. One word in this context was 12 bits, so a grand total of 1584 bits, plus lots of control circuitry.
Since I started college and really knew what breadboards were I always believed they were only every used in schools or else for prototyping, anything in production/commercial used etched silicon, that assumption was killed today.
It now makes sense why it cost $20,000/month (today's $) to use this computer, if you had to manually build each board, the cost for that alone is astronomical...
I think I owe my ECE 101 professor an apology.
It now makes sense why it cost $20,000/month (today's $) to use this computer, if you had to manually build each board, the cost for that alone is astronomical...
I think I owe my ECE 101 professor an apology.
There are no "breadboards" in an IBM 1401. There's a wire-wrap backplane, but that's a production technology. Even manual wire wrapping is very organized. You have a list of X-Y coordinates to attach, and a supply of pre-cut and pre-stripped wires of the right length. You put in all the shortest wires first, then the next length, and so on. A wire-wrap gun-like tool does the twisting and stops when the wire is tight. It's easy, and the result is reliable. Much better than those solderless breadboards the Arduino crowd uses.
By the IBM 360 era, IBM had much of the production process automated. Wire-wrap backplanes were wired by a Gardner-Denver CNC wire-wrapping machine. IBM's "solid logic technology",a pre-IC technology, placed components in little ceramic bases which were then canned. IBM pioneered automated weaving of core memory. For 15 years, IBM was way ahead on automated electronics manufacturing, and extremely profitable.
All this was a dead end. Making ICs by photolithography became cheaper in the 1970s.
By the IBM 360 era, IBM had much of the production process automated. Wire-wrap backplanes were wired by a Gardner-Denver CNC wire-wrapping machine. IBM's "solid logic technology",a pre-IC technology, placed components in little ceramic bases which were then canned. IBM pioneered automated weaving of core memory. For 15 years, IBM was way ahead on automated electronics manufacturing, and extremely profitable.
All this was a dead end. Making ICs by photolithography became cheaper in the 1970s.
Good answer. A couple additional notes: the 1401's backplane was also wire-wrapped by a Gardner-Denver machine controlled by punch cards. Field engineers could modify the wiring with hand wire-wrap tools (to fix problems or add new features), but most wiring was automated.
IBM was using CAD for their systems way back; the 1401's wiring diagrams were called Automated Logic Diagrams because they were computer-generated, originally using a vacuum tube IBM 704 or 705 computer.
I'm still not sure if IBM's solid logic technology (SLT) was a good idea, or if they should have moved to integrated circuits much sooner.
IBM was using CAD for their systems way back; the 1401's wiring diagrams were called Automated Logic Diagrams because they were computer-generated, originally using a vacuum tube IBM 704 or 705 computer.
I'm still not sure if IBM's solid logic technology (SLT) was a good idea, or if they should have moved to integrated circuits much sooner.
I worked on a project in the mid 90's where we were prototyping a single board replacement for a PC and several specialty boards and we used a breadboard that was about 12 inches by 8 inches and we kept adding components until we had problems with the digital lines being too noisy - things stopped working consistently so we looked and the 1/0 levels were obviously pretty dirty on the oscilliscope so we had to switch to using wirewrap for the prototypes. This happened again with wirewrap so then we switched to PCB prototypes. Obviously this depends on clock speed for the data lines and application (we had a couple of CPU's, memory, and a/d, d/a, NTSC, forget how far we got with just a breadboard.)
Better than our office couldn't print story. The admin was in a car accident in the late 1990s.
She was gone 3 days before we could no longer print. I remember talking to her and her complaining that we were using too much space for emails, and whoever set up the Network put the email and the print spooler on the same drive. I told people if they cleaned out their inboxes printing should work again, and it did. She returned a couple weeks later and fixed the problem in a more permanent way.
She was gone 3 days before we could no longer print. I remember talking to her and her complaining that we were using too much space for emails, and whoever set up the Network put the email and the print spooler on the same drive. I told people if they cleaned out their inboxes printing should work again, and it did. She returned a couple weeks later and fixed the problem in a more permanent way.
Fixing 1960s and 1970s electronics is hard. But they're fixing IBM's good stuff, not junk.
It's rare to see a transistor opened up today to see what went wrong. From the 1950s to the 1980s, the USAF did that on occasion. The USAF had a project, the Air Force Reliability Program, to make electronics more reliable. About 1% of the electronics boxes in the USAF were stickered with a note like "USAF Reliability Program - do not repair in the field, replace and ship back to ..." Back at Reliability Program HQ, technicians would figure out exactly what had broken and document it. All the way down to the individual component level and below. Transistors would be opened up and photographed under a microscope to see exactly what went wrong. They found problems like this bad transistor in the 1401, where a lead wasn't attached properly.
The USAF followed up by naming and shaming manufacturers. Microphotographs of bad parts were published in Aviation Week. Parts got better.
It's rare to see a transistor opened up today to see what went wrong. From the 1950s to the 1980s, the USAF did that on occasion. The USAF had a project, the Air Force Reliability Program, to make electronics more reliable. About 1% of the electronics boxes in the USAF were stickered with a note like "USAF Reliability Program - do not repair in the field, replace and ship back to ..." Back at Reliability Program HQ, technicians would figure out exactly what had broken and document it. All the way down to the individual component level and below. Transistors would be opened up and photographed under a microscope to see exactly what went wrong. They found problems like this bad transistor in the 1401, where a lead wasn't attached properly.
The USAF followed up by naming and shaming manufacturers. Microphotographs of bad parts were published in Aviation Week. Parts got better.
I expected the debugging session to end when they identified the transistor that was bad, but these guys were dedicated!
I've never seen a transistor can opened like that. I love how it's essentially just a shrunken down version of the very first transistor ever made[1]
Great story about the USAF. It's a shame that circuitry is essentially a black box nowadays.
1: https://physics.aps.org/assets/c5065b1e-4bf2-4501-a05b-6a841...
I've never seen a transistor can opened like that. I love how it's essentially just a shrunken down version of the very first transistor ever made[1]
Great story about the USAF. It's a shame that circuitry is essentially a black box nowadays.
1: https://physics.aps.org/assets/c5065b1e-4bf2-4501-a05b-6a841...
A transistor is just as much of a black box, only incredibly simpler.
You can decap hardware today in roughly the same way, and given of course enough time, understand it, and eventually control it.
You can decap hardware today in roughly the same way, and given of course enough time, understand it, and eventually control it.
These days decapping involves sandpaper and a magnifying glass.
> It's a shame that circuitry is essentially a black box nowadays.
Yeah, quite a shame that we have high speed low power electronics today. I pine for vacuum tube computers.
You get my point. Most advances come at a cost. The cost in this case is worth it.
Yeah, quite a shame that we have high speed low power electronics today. I pine for vacuum tube computers.
You get my point. Most advances come at a cost. The cost in this case is worth it.
Vacuum tube machines computed with more pleasing harmonics, resulting in the output having a richer, warmer sound.
And we all know that germanium is waaaaay better than silicon in your diodes and transistors.
I just installed a germanium Dallas-Arbiter Fuzzy Logic unit, and boy did I ever see some benchmark improvements
How much HP exactly? If I put one in my Civic does it mount on the trunk-wing bleacher seat spoiler or to the muffler resonator tip?
Oh, God, "germanium Dallas-Arbiter Fuzzy" isn't just word salad ...
I'm dying over here ...
I'm dying over here ...
And have you ever tried to compute the "Pride and Joy" algorithm using a silicon-based computer? Exercise in futility, to say the least.
> Fixing 1960s and 1970s electronics is hard. But they're fixing IBM's good stuff, not junk.
Depends. I found my various Tektronix pieces quite easily repaired, with insights gained. But then again their documentation is quite good.
Depends. I found my various Tektronix pieces quite easily repaired, with insights gained. But then again their documentation is quite good.
Tektronix was over the top on quality. Ceramic terminal strips, silver solder, and all the component color codes facing in the same direction. They even put a little spool of silver solder inside the scope in case you had to make repairs. [1]
[1] https://www.youtube.com/watch?v=BvbNBZX6kNE
[1] https://www.youtube.com/watch?v=BvbNBZX6kNE
On top of that, if a board has utterly gave up the ghost and let the magic smoke out there's still a large market for donor boards, probably owing to the fact that they were such well built, documented, and repairable pieces of equipment.
Tektronix is a scientific instrument company that happened to get into computer peripherals. You could bet their gear was rock solid, and designed for engineers to poke at.
Like the mid 1970s Tektronix 4014 storage tube graphics display terminals. Saw one that was retired but still operational in early '00s used originally for CAD. Interesting how back then you could draw a circle in a box then morph the box (using scroll wheels) into a rectangle to form an ellipse.
I wonder how long it would have taken to debug this back in 1960
Probably half the time or less since the people working on it would be more familiar with debugging these sorts of systems.
Yes, I expect field engineers back in the 1960s would have found the problem much faster than we did. They had oscilloscopes and test equipment, so we don't have much advantage there. They would be much more familiar with the system and wouldn't need to keep looking stuff up like we did. Plus, they would be motivated by anxious customers breathing down their necks demanding a quick fix.
I think the digitizing oscilloscope is quite an advantage compared to the old stuff ;)
Wonderful article, as always. It's impressive how detailed IBM's documentation is.
What I thought was cool is that it was machine generated.
If you like stuff like this, follow curiousmarc on youtube. He volunteers there and helps with restorations, and does all kinds of cool restorations at home, too.
Get to work on the Holmes here with pliers and a blowtorch.
Chain printers like this were amazing to watch, the chain had to move amazingly fast and could break ... I worked with a Burroughs mainframe, we were always cautioned never to stand either side of the printer, because reputedly if the chain broke it could fly out the side and embed itself in the wall (never actually happened to us)
On some printers you could change chains for particular print jobs - when we printed some jobs we'd load up a chain with just digits on it (so the printer could print an entire line while the chain moved 10 locations rather than 48 locations - theoretically 5 times faster.
Page layout was done with a loop of paper tape N positions long (that matched the number of lines on a page) and 10 widem each one of 10 horizontal holes corresponded to one of the fortran carriage control commands "1" meant "skip until there's a hole in position 1 on the tape" - you'd change to a special loop when printing labels for example, or for different sized paper - if you made a loop and were stupid enough not to have a hole in each horizontal position at least once around a loop skipping to that position would result in a "page throw" fanfold paper would exit the printer horizontally until it hit the wall and would continue to do so until you ran out or someone noticed
On some printers you could change chains for particular print jobs - when we printed some jobs we'd load up a chain with just digits on it (so the printer could print an entire line while the chain moved 10 locations rather than 48 locations - theoretically 5 times faster.
Page layout was done with a loop of paper tape N positions long (that matched the number of lines on a page) and 10 widem each one of 10 horizontal holes corresponded to one of the fortran carriage control commands "1" meant "skip until there's a hole in position 1 on the tape" - you'd change to a special loop when printing labels for example, or for different sized paper - if you made a loop and were stupid enough not to have a hole in each horizontal position at least once around a loop skipping to that position would result in a "page throw" fanfold paper would exit the printer horizontally until it hit the wall and would continue to do so until you ran out or someone noticed