Ask HN: Why is there so little info on the web about IBM mainframe programming?
137 comments
At companies I've worked at with mainframes the teams running them were typically very well tenured on both the systems and software running on them.
They weren't the types to go looking on StackOverflow or Youtube for answers.
They would also help train new joiners, and IBM offers good training too.
The kinds of systems running on mainframes don't lend themselves to copy/paste from StackOverflow type programming. It's a lot of credit-card and banking transactions where you want experienced people writing and helping on it.
Secondly, even if IBM is selling more than before, that's still an incredibly tiny fraction of the computing scene. Companies don't tend to have more than one (or possibly 2, with one for DR mainframes). So, that 80-90% of the Fortune 500 is really just 1000 mainframes total. Generously, if every Fortune 500 had a pair in each geography (roughly EU, Americas, Asia for big corps), that ends up being 6, so 3000 globally, in production. There are also some spread around academia, government labs, etc. But the numbers are dwarfed by standard x86 machines. eg. at one bank we had "the mainframe" which was actually one hot and one DR, and then 40k windows servers and 40k linux servers. Similarly, there were thousands of engineers in technology, but the mainframe team with single digits.
Given the wealth of information out there, even if there was "a lot" of information for some value of lots, it's dwarfed by the incredible amount of standard tech info.
They weren't the types to go looking on StackOverflow or Youtube for answers.
They would also help train new joiners, and IBM offers good training too.
The kinds of systems running on mainframes don't lend themselves to copy/paste from StackOverflow type programming. It's a lot of credit-card and banking transactions where you want experienced people writing and helping on it.
Secondly, even if IBM is selling more than before, that's still an incredibly tiny fraction of the computing scene. Companies don't tend to have more than one (or possibly 2, with one for DR mainframes). So, that 80-90% of the Fortune 500 is really just 1000 mainframes total. Generously, if every Fortune 500 had a pair in each geography (roughly EU, Americas, Asia for big corps), that ends up being 6, so 3000 globally, in production. There are also some spread around academia, government labs, etc. But the numbers are dwarfed by standard x86 machines. eg. at one bank we had "the mainframe" which was actually one hot and one DR, and then 40k windows servers and 40k linux servers. Similarly, there were thousands of engineers in technology, but the mainframe team with single digits.
Given the wealth of information out there, even if there was "a lot" of information for some value of lots, it's dwarfed by the incredible amount of standard tech info.
> So, that 80-90% of the Fortune 500 is really just 1000 mainframes total. Generously, if every Fortune 500 had a pair in each geography (roughly EU, Americas, Asia for big corps), that ends up being 6, so 3000 globally, in production.
As others have posted, your estimates are WILDLY off.
First, fortune 500 companies dont own "two" mainframes, but a LOT more.
Next, many non-F500 companies own a lot of them as well.
Years ago I worked for a local cable company in Canada.
Not a "F500" company, owned at least two mainframes for the "head office" and each different 'subdivision' owned their own mainframe.
> Really just 1000 mainframes total.
I'd be willing to bet that any major city has more within its borders.
As others have posted, your estimates are WILDLY off.
First, fortune 500 companies dont own "two" mainframes, but a LOT more.
Next, many non-F500 companies own a lot of them as well.
Years ago I worked for a local cable company in Canada.
Not a "F500" company, owned at least two mainframes for the "head office" and each different 'subdivision' owned their own mainframe.
> Really just 1000 mainframes total.
I'd be willing to bet that any major city has more within its borders.
Even if OP's estimates are "WILDLY off", there still aren't many out there
Multiply his estimate to 1000 per F500 company ... that's only half a million
Put a pair in every major geography ... that's only 5,000,000
And, for kicks and giggles, multiply by another 10 for funzies
That 50,000,000
Yes, it's "a lot" ... but it's nothing compared to commodity hardware
According to a quick web search, Google alone had in excess of 1,000,000 x86 servers about a decade ago
That's one (large) company
I've worked with myriad companies who're running 10s or 100s of 1000s of x86 servers (let alone x86 desktops/workstations/laptops) - with nary a mainframe in sight (or maybe a couple ... which is on the order of "none" next to 100s of 1000s or millions of x86 boxes)
Per Microsoft [0], there are 1.4 billion Windows 10 and 11 devices (yes, I know that's not a server count, but it speaks to the orders of magnitude difference between x86 devices and mainframes ...even with my generous estimates posted above)
-----------
[0] https://news.microsoft.com/bythenumbers/en/windowsdevices
Multiply his estimate to 1000 per F500 company ... that's only half a million
Put a pair in every major geography ... that's only 5,000,000
And, for kicks and giggles, multiply by another 10 for funzies
That 50,000,000
Yes, it's "a lot" ... but it's nothing compared to commodity hardware
According to a quick web search, Google alone had in excess of 1,000,000 x86 servers about a decade ago
That's one (large) company
I've worked with myriad companies who're running 10s or 100s of 1000s of x86 servers (let alone x86 desktops/workstations/laptops) - with nary a mainframe in sight (or maybe a couple ... which is on the order of "none" next to 100s of 1000s or millions of x86 boxes)
Per Microsoft [0], there are 1.4 billion Windows 10 and 11 devices (yes, I know that's not a server count, but it speaks to the orders of magnitude difference between x86 devices and mainframes ...even with my generous estimates posted above)
-----------
[0] https://news.microsoft.com/bythenumbers/en/windowsdevices
The flip side of that is that commodity hardware runs commodity software. It’s entirely possible to run a very large company without having any bespoke software.
Mainframes on the other hand or more likely than not to have software that only runs on a handful of machines at most.
So number of machines is not a good proxy for amount of software written.
On the other hand I’ve never touched a mainframe so it’s all speculation…
Mainframes on the other hand or more likely than not to have software that only runs on a handful of machines at most.
So number of machines is not a good proxy for amount of software written.
On the other hand I’ve never touched a mainframe so it’s all speculation…
>It’s entirely possible to run a very large company without having any bespoke software.
I find that claim remarkably improbable - the overwhelming majority of all code written is for in-house business tasks
I find that claim remarkably improbable - the overwhelming majority of all code written is for in-house business tasks
>So number of machines is not a good proxy for amount of software written.
It is a good proxy for the relative need of developers, however - with billions of x86 devices out there vs max a couple-few million mainframes...the scale of need is just wildly different
It is a good proxy for the relative need of developers, however - with billions of x86 devices out there vs max a couple-few million mainframes...the scale of need is just wildly different
I don't know if iSeries (AS/400) counts as Mainframe... But the software stack is similar and I know that hundreds (if not thousands) of casinos run old Main Frame era software.
[deleted]
Having worked in BigCo land for 20 years, your estimates are way off.
There are F500s that have 1k mainframes each, today, in 2022.
There are F500s that have 1k mainframes each, today, in 2022.
Thank you for setting that straight. The more I read through the answer, the more ludicrous it became.
There should be a way to flag and remove comments that don't violate any guidelines, but are just objectively wrong, in a way that doesn't penalize the author. Like "you tried, but nothing about this is correct, so it goes to the bottom of the comment section above all the dead/shadowbanned comments."
I prefer the back-and-forth the way it is now. I perversely enjoy reading a comment and "buying into it" only to realize from a reply that I was naive or too trusting. Or, at least, that there are more angles to the story.
This helps you build up an immune system that is skeptical. If most wrong answers were removed/flagged/hidden then the remaining wrong but unflagged would be even more a problem.
This helps you build up an immune system that is skeptical. If most wrong answers were removed/flagged/hidden then the remaining wrong but unflagged would be even more a problem.
Exactly. This helps train the "don't believe everything you read" muscle.
Sure for the percentage of people who do read all the way through and then rethink things. This is <1% in most communities
Assuming everyone is an idiot, and designing around that basis, seems like a great way to foster a community of idiots.
And unlike stackoverflow, there’s not really any good reason for anyone to think a top-level comment is somehow authoritative, especially because the best comments on HN tend to be replies
And unlike stackoverflow, there’s not really any good reason for anyone to think a top-level comment is somehow authoritative, especially because the best comments on HN tend to be replies
> I perversely enjoy reading a comment and "buying into it" only to realize from a reply that I was naive or too trusting.
Good for debate and free speech in general, but bad for your grasp of the facts in the case every time there don't happen to be any corrective replies.
Hard to weighs pros vs cons.
(ObSillyPun: Naah, just put them on the scales! But remember that a lighter pro will often beat a heavier con anyway!)
Good for debate and free speech in general, but bad for your grasp of the facts in the case every time there don't happen to be any corrective replies.
Hard to weighs pros vs cons.
(ObSillyPun: Naah, just put them on the scales! But remember that a lighter pro will often beat a heavier con anyway!)
Your bullshit detector should be trained continuously, not put to sleep by others labeling a post true or not. A dangerous path.
So you DO search for answers to mainframe questions on s**overflow?
It seems like their point still stands. If there were 100 million mainframes, x86 would still have more, and the standards for skills and experience would still be different due to the nature of deployment.
there are many medium size banks that have a mainframe or two at the heart of their system.
Even small banks. Basically every bank that was around 20 or 30 years ago started with mainframes (or very large minis) and still uses them. I'm not in that business, but I've seen the server rooms, and I haven't heard of a bank switching over to a rack of PCs, yet. There might be, but that would still hundreds, perhaps a thousand banks in Europe alone running on large-ish iron.
There might be less mainframes in total, though. The bank where my parents worked had 4 at different sites, but now runs on a single one, AFAIK.
There might be less mainframes in total, though. The bank where my parents worked had 4 at different sites, but now runs on a single one, AFAIK.
> I haven't heard of a bank switching over to a rack of PCs, yet.
Right. The main limitation is the core software platform. This is an extraordinarily complex system that requires expensive regulatory sign-off/audit/etc.
There are some vendors in the space who are trying to do a new bank + new core. Even if someone had an awesome x86 clustered core system with 100% regulatory approval, any existing mid-tier bank converting to it would be an ~8 figure ordeal.
Right. The main limitation is the core software platform. This is an extraordinarily complex system that requires expensive regulatory sign-off/audit/etc.
There are some vendors in the space who are trying to do a new bank + new core. Even if someone had an awesome x86 clustered core system with 100% regulatory approval, any existing mid-tier bank converting to it would be an ~8 figure ordeal.
Unisys ClearPath Dorado is just a fancy Xeon machine running an emulator.
But the OS, database, network definition and workflow languages are still proprietary. Which goes back to the original question: where is that knowledge store/discussed?
Inside the company. Not gonna be a youtube tutorial lmao
"Sup nerds, today we're going to be writing some COBOL, but first don't forget to hit the bell and smash that subscribe button..."
I can confirm after working with the IBM mainframe people a bit. The internal training of these systems make SO/ random other information sources on the subject pointless.
StackOverflow, I feel, inherently ties into the "Release early. Release often." philosophy: if you ship broken code , oh well , there'll be a release soon enough to fix it. This can not be borne in a mainframe environment. This is not to say there aren't bugs because of course there are but the expectation is completely different. This kind of "by your bootstraps" education is no bueno. You will get lots of training and excellent documentation. We used to measure the amount of documentation IBM ships with the mainframe not in pages but in yards it takes on the shelves.
StackOverflow in itself is the answer to the question "who you gonna call" -- because you might be alone, or you might be working on a small team where no one has better expertise, and usually you are working with open source and so no one is obligated to pick up the phone at the software "manufacturer". This scenario does not play in a mainframe environment: you are going to be part of a larger team, full of literal graybeards and when push comes to shove then you do have a contact number: your employer is paying absolutely ridiculous amounts of money to the software company and they will have a contact person who will pick up the phone.
StackOverflow in itself is the answer to the question "who you gonna call" -- because you might be alone, or you might be working on a small team where no one has better expertise, and usually you are working with open source and so no one is obligated to pick up the phone at the software "manufacturer". This scenario does not play in a mainframe environment: you are going to be part of a larger team, full of literal graybeards and when push comes to shove then you do have a contact number: your employer is paying absolutely ridiculous amounts of money to the software company and they will have a contact person who will pick up the phone.
Yeah, mainframe coding is much closer to "code NASA runs on spacecraft" than what most people consider coding to be today. And the level of hardware support (mainframes are basically redundant on everything) and software support is unheard of. Consider that these things are so integrated into major banks, etc, that even minutes of downtime would cost millions upon millions, and think of all the things you can spend millions on to reduce the chance of it happening.
> mainframes are basically redundant on everything
Not only that but that redundancy is hidden from everything. Already decades ago the Hungarian electricity distributor had a "RAID" storage unit where one bunch of disks were at the capitol, the other bunch connected with fiber at another city -- and you talked to this thing as if it were an ordinary disk. Short of an invading army nothing could cause data loss on that thing -- simple aerial bombing wouldn't because the unit in the capitol was tens of meters underground... that's not concerns you have on your average web app.
Not only that but that redundancy is hidden from everything. Already decades ago the Hungarian electricity distributor had a "RAID" storage unit where one bunch of disks were at the capitol, the other bunch connected with fiber at another city -- and you talked to this thing as if it were an ordinary disk. Short of an invading army nothing could cause data loss on that thing -- simple aerial bombing wouldn't because the unit in the capitol was tens of meters underground... that's not concerns you have on your average web app.
Yeah, one thing that I've been really disappointed with is how UNRELIABLE the "cloud" is - unless you engineer all the reliability in the software itself.
We have the technology and the capability to do things like run a VM and have it move between hardware when that hardware fails or needs replacement.
but that would cost money.
We have the technology and the capability to do things like run a VM and have it move between hardware when that hardware fails or needs replacement.
but that would cost money.
> the capability to do things like run a VM and have it move between hardware when that hardware fails
This is one of my favorite features of Google Cloud:
https://cloud.google.com/compute/docs/instances/live-migrati...
Unfortunately, Google continues to be the third-string provider in the US, so I don't often get the opportunity to work with Google Cloud.
This is one of my favorite features of Google Cloud:
https://cloud.google.com/compute/docs/instances/live-migrati...
Unfortunately, Google continues to be the third-string provider in the US, so I don't often get the opportunity to work with Google Cloud.
I want to be able to justify using Google Cloud for projects, but the fact that it’s a Google product is the main obstacle to that.
If history is anything to go by, relying on Google products and services seems like a great way to set yourself up for major problems down the line. Either the product you’re using is shut down on short notice or some automation flags a false-positive and you’re left screaming into the void trying to get yourself unbanned.
It’s not a live migration. It’s running an a VM in a sysplex which is in lockstep with its peer. There’s not a failover in the tradition sense..the I/O just goes from one side to the other.
VMs can often be configured to do "live migration" but that doesn't protect a running instance if the hardware it is running on fails completely. There is such a thing as fault tolerant hardware/software stacks but, yes, most modern architectures shift application resiliency higher up the stack.
> This can not be borne in a mainframe environment.
I mean, maybe not in an old-school 1970s batch-SIMD mainframe environment with limited storage memory.
But in a modern mainframe environment, where
1. every tenant being processed is hitting their own virtualized environment locked into their own legacy versions of everything (think: separately-SCMed "Enterprise" deployments of SaaS software — just all living together on one machine cluster rather than on-prem per customer); and
2. you've got the spare storage capacity to do everything via async-queued CQRS/ES + writing secondary representations and cubes to a local data lake, rather than ever updating anything in place...
...then why not do "agile" blue-green deploys of your ACH reconciliation logic, starting with the smallest tenants first?
If you ever produce a bug in such an environment, you can always just: stop the job; emit tombstone records for all the CQRS Commands it produced so they'll be ignored on "outbox processing" rather than pushed to consumers; fix the bug; up the version number for OLAP report generation, invalidating + requiring recomputation for any reports produced from the data in the reporting period; and then start the job up again. No external system will ever have to know, because your own system should be set up such that no other external system is synchronously dependent on its outputs. (This is half the reason things like ACH are designed with the async "flex" they have.)
And, in fact, assuming you've got the spare CPU power (which you do), you can even run entire reporting / integration cycles twice — once under the old logic, once under the new logic — and examine the resulting queued commands + reports to see that they all come out the same, before allowing anything to go through. You can do this not just in production, but experimentally, against a corpus of all old input data for each tenant (which you've almost certainly kept around.) And, as such, you don't even have to stick with those virtualized legacy environments that mainframe architectures like z/OS were designed to enable — you can actively migrate clients away from them (even automatically, with CI/CD-triggered graduated rollouts et al) as soon as you can prove that their workloads (both present and historical!) won't be affected by the change. Each tenant can be re-pinned over time to the newest release that can be proven to be equivalent to the original release provided to them on all known data.
One of the key insights about mainframe programming, is that the software architectures of the kinds of systems deployed on mainframes, are designed to be resilient against individual bugs; or, one might say, against individual programmers who don't know what they're doing. Systems engineering in e.g. finance, is to bad individual-component / individual-release logic, as NASA's Mars-rover systems engineering is to bad hardware: designed under the assumption of the potential for failure, and designed with built-in remediation strategies for that failure.
I mean, maybe not in an old-school 1970s batch-SIMD mainframe environment with limited storage memory.
But in a modern mainframe environment, where
1. every tenant being processed is hitting their own virtualized environment locked into their own legacy versions of everything (think: separately-SCMed "Enterprise" deployments of SaaS software — just all living together on one machine cluster rather than on-prem per customer); and
2. you've got the spare storage capacity to do everything via async-queued CQRS/ES + writing secondary representations and cubes to a local data lake, rather than ever updating anything in place...
...then why not do "agile" blue-green deploys of your ACH reconciliation logic, starting with the smallest tenants first?
If you ever produce a bug in such an environment, you can always just: stop the job; emit tombstone records for all the CQRS Commands it produced so they'll be ignored on "outbox processing" rather than pushed to consumers; fix the bug; up the version number for OLAP report generation, invalidating + requiring recomputation for any reports produced from the data in the reporting period; and then start the job up again. No external system will ever have to know, because your own system should be set up such that no other external system is synchronously dependent on its outputs. (This is half the reason things like ACH are designed with the async "flex" they have.)
And, in fact, assuming you've got the spare CPU power (which you do), you can even run entire reporting / integration cycles twice — once under the old logic, once under the new logic — and examine the resulting queued commands + reports to see that they all come out the same, before allowing anything to go through. You can do this not just in production, but experimentally, against a corpus of all old input data for each tenant (which you've almost certainly kept around.) And, as such, you don't even have to stick with those virtualized legacy environments that mainframe architectures like z/OS were designed to enable — you can actively migrate clients away from them (even automatically, with CI/CD-triggered graduated rollouts et al) as soon as you can prove that their workloads (both present and historical!) won't be affected by the change. Each tenant can be re-pinned over time to the newest release that can be proven to be equivalent to the original release provided to them on all known data.
One of the key insights about mainframe programming, is that the software architectures of the kinds of systems deployed on mainframes, are designed to be resilient against individual bugs; or, one might say, against individual programmers who don't know what they're doing. Systems engineering in e.g. finance, is to bad individual-component / individual-release logic, as NASA's Mars-rover systems engineering is to bad hardware: designed under the assumption of the potential for failure, and designed with built-in remediation strategies for that failure.
Thanks for sharing. TBH I envy those good trainings if they are that good. The trainings we get (e.g. GCP) are OK but definitely not exceptionally good. I actually wonder what training IBM gives.
There is an emulator for running mainframe code called Hercules. It comes with no software, so you'll need to acquire compilers & database apps separately.
The equivalent from IBM is called "IBM Z Development and Test Environment" and the license for that costs $5,540 per user per year. There was (maybe still is?) a "learners edition" that you have to contact them to "see if you qualify" and it is a far more reasonable $120/year (there are no links on IBM Marketplace to even try to order it). This includes several compilers and 3 databases: CICS, DB2 & IMS.
IBM has a very different model for supporting developers than Microsoft does with MSDN.
Links:
https://en.wikipedia.org/wiki/Hercules_(emulator)
http://www.hercules-390.org/
https://www.ibm.com/products/z-development-test-environment/...
https://ibm.github.io/zdt-learners-edition-about/
https://old.reddit.com/r/mainframe/
Ballmer being excited: https://www.youtube.com/watch?v=wNy2zBG79zU
The equivalent from IBM is called "IBM Z Development and Test Environment" and the license for that costs $5,540 per user per year. There was (maybe still is?) a "learners edition" that you have to contact them to "see if you qualify" and it is a far more reasonable $120/year (there are no links on IBM Marketplace to even try to order it). This includes several compilers and 3 databases: CICS, DB2 & IMS.
IBM has a very different model for supporting developers than Microsoft does with MSDN.
Links:
https://en.wikipedia.org/wiki/Hercules_(emulator)
http://www.hercules-390.org/
https://www.ibm.com/products/z-development-test-environment/...
https://ibm.github.io/zdt-learners-edition-about/
https://old.reddit.com/r/mainframe/
Ballmer being excited: https://www.youtube.com/watch?v=wNy2zBG79zU
It always struck me as odd that one the one hand, finding people with experience in this field seem to be very hard to find, but on the other hand, IBM makes it so hard for people to learn this stuff. I wonder if it didn't pay off for them in the long run to just put up one beefy machine on the Internet, offer free accounts to people or just a free hobbyist license like you can get for VMS. Are they paranoid about customers cheating to hitch a free ride?
Because it's relatively easy to hire someone with relevant experience in other fields and give them the internal tooling and training from IBM.
After all, IBM wants you to pay IBM to do this stuff for you.
It's not like Bob from IT is going to play around with mainframe coding for a year or so and convince the boss to buy one.
After all, IBM wants you to pay IBM to do this stuff for you.
It's not like Bob from IT is going to play around with mainframe coding for a year or so and convince the boss to buy one.
This. The mainframe model seems bizarre to people experienced in modern programming, because they can't imagine all the things that were different.
So imagine you're a company selling low volumes (by modern standards) of exquisite computing hardware to highly demanding customers, in a world where... (1) there is no Internet, (2) there are no personal computers, (3) most people don't know how to use a computer, much less program one.
You'd probably come up with a model that looked a lot like "We'll sell you the machine + train you how to program and use it!" as a package.
And then after the world changed, but your offering was very polished, you probably just wouldn't overhaul it. Because it's been working fine for 50 years.
So imagine you're a company selling low volumes (by modern standards) of exquisite computing hardware to highly demanding customers, in a world where... (1) there is no Internet, (2) there are no personal computers, (3) most people don't know how to use a computer, much less program one.
You'd probably come up with a model that looked a lot like "We'll sell you the machine + train you how to program and use it!" as a package.
And then after the world changed, but your offering was very polished, you probably just wouldn't overhaul it. Because it's been working fine for 50 years.
> It's not like Bob from IT is going to play around with mainframe coding for a year or so and convince the boss to buy one.
not currently, IBM pretty much actively prevents this with their current pricing.
it could happen, and would, if people were allowed to learn this on their own.
Mainframes are very good at some things. Far better than commodity hardware is today.
not currently, IBM pretty much actively prevents this with their current pricing.
it could happen, and would, if people were allowed to learn this on their own.
Mainframes are very good at some things. Far better than commodity hardware is today.
> offer free accounts to people
AFAIR, they offer the possibility for a one year free access to a node (Linux/IFL?). I do not remember if it is part of IBM Z Xplore and I didn't find it again with a quick search, but if you contact them they for sure will tell you more.
Edit: I just found https://www.openmainframeproject.org/
AFAIR, they offer the possibility for a one year free access to a node (Linux/IFL?). I do not remember if it is part of IBM Z Xplore and I didn't find it again with a quick search, but if you contact them they for sure will tell you more.
Edit: I just found https://www.openmainframeproject.org/
It wouldn’t directly generate any sales commissions.
See also:
"Hello World on z/OS" (2018) https://medium.com/the-technical-archaeologist/hello-world-o...
Hn discussion:
https://news.ycombinator.com/item?id=17642846
I seem to recall IBM launched z/os in the cloud as partly a production service, and partly an education/hobby resource a while back - but I'm not sure if that's what's now:
https://www.ibm.com/cloud/pricing
I think it came with access to db2 as well?
Ed: I was probably thinking of IBM LinuxOne:
https://developer.ibm.com/articles/get-started-with-ibm-linu...
"Hello World on z/OS" (2018) https://medium.com/the-technical-archaeologist/hello-world-o...
Hn discussion:
https://news.ycombinator.com/item?id=17642846
I seem to recall IBM launched z/os in the cloud as partly a production service, and partly an education/hobby resource a while back - but I'm not sure if that's what's now:
https://www.ibm.com/cloud/pricing
I think it came with access to db2 as well?
Ed: I was probably thinking of IBM LinuxOne:
https://developer.ibm.com/articles/get-started-with-ibm-linu...
From the pricing FAQ [1]
"Learner's Edition is currently being updated and will return soon"
Let's hope "soon" is actually soon :)
[1] https://www.ibm.com/products/z-development-test-environment/...
"Learner's Edition is currently being updated and will return soon"
Let's hope "soon" is actually soon :)
[1] https://www.ibm.com/products/z-development-test-environment/...
> IBM has a very different model for supporting developers than Microsoft does with MSDN.
Interesting comparison, because when I was teaching myself to code in the early 00s, that's how I perceived Microsoft.
MSDN was a paid subscription, and the tools you needed to write MS software were also paid.
Hence me ending up forming a career on open source software, and a generation of programmers who still turn their nose up at Microsoft languages.
Interesting comparison, because when I was teaching myself to code in the early 00s, that's how I perceived Microsoft.
MSDN was a paid subscription, and the tools you needed to write MS software were also paid.
Hence me ending up forming a career on open source software, and a generation of programmers who still turn their nose up at Microsoft languages.
Except .. they're using VSCode, GitHub and Typescript, probably still on a Windows laptop.
None of those things were around in the early 00s.
Well, Windows was, but that's not the point.
The parent comment noted that there was a generation who shunned Microsoft because of their corporate misbehavior. More recent generations of programmers have forgotten why that was, and Microsoft has, once again, successfully established a dominant position in the development ecosystem.
I fully expect that position to be leveraged for profit, and that would see a new generation, 25 years later, who will never trust Microsoft again.
Most things are cyclical.
The parent comment noted that there was a generation who shunned Microsoft because of their corporate misbehavior. More recent generations of programmers have forgotten why that was, and Microsoft has, once again, successfully established a dominant position in the development ecosystem.
I fully expect that position to be leveraged for profit, and that would see a new generation, 25 years later, who will never trust Microsoft again.
Most things are cyclical.
A shop I worked in ages ago had a mainframe on site supporting lots of batched transactions over the course of a 24 hour period, along with the daily operations for a 35 facility logistics company, spread throughout the southeastern United States. They had a staff of 8 mainframers that ran the entire operation. All of the PCs onsite (my area of responsibility) were basically 3270 terminal clients for z/OS.
Sometime later, a connection on LinkedIn pointed out the following resources:
https://www.openmainframeproject.org/
https://github.com/openmainframeproject
https://github.com/openmainframeproject/cobol-programming-co...
Also, mainframes are really good at high throughput transactional jobs. That's why you see them in banks, transportation, insurance, etc. Big Tech™ doesn't see it as "cool" and are too focused on the Next Big Thing™, so there's not a lot of attention there. Sometimes, boring just gets the job done.
Sometime later, a connection on LinkedIn pointed out the following resources:
https://www.openmainframeproject.org/
https://github.com/openmainframeproject
https://github.com/openmainframeproject/cobol-programming-co...
Also, mainframes are really good at high throughput transactional jobs. That's why you see them in banks, transportation, insurance, etc. Big Tech™ doesn't see it as "cool" and are too focused on the Next Big Thing™, so there's not a lot of attention there. Sometimes, boring just gets the job done.
I know a few mainframe folks. I don't think there's a single answer, but there are definitely a few inhibiting factors.
* The documentation for mainframe stuff is incredibly comprehensive compared to what most people are used to, so there's less need for something like Sewer Overflow.
* The code people are working on is likely to be proprietary, so no GitHub.
* Mainframes are often used in security-conscious environments where "openness" is not a core value and might even be considered negative.
* People working in that area might like being part of a relatively closed community (lower supply = better pay) so they don't advertise.
And then there's just critical mass. Since there aren't already a lot of mainframe programmers Out There, and mainframe programmers' work has few technical "touch points" with the rest of the world (e.g. different languages, libraries), it almost makes more sense to flip the question around.
Why would there be more info on the web about IBM mainframe programming?
* The documentation for mainframe stuff is incredibly comprehensive compared to what most people are used to, so there's less need for something like Sewer Overflow.
* The code people are working on is likely to be proprietary, so no GitHub.
* Mainframes are often used in security-conscious environments where "openness" is not a core value and might even be considered negative.
* People working in that area might like being part of a relatively closed community (lower supply = better pay) so they don't advertise.
And then there's just critical mass. Since there aren't already a lot of mainframe programmers Out There, and mainframe programmers' work has few technical "touch points" with the rest of the world (e.g. different languages, libraries), it almost makes more sense to flip the question around.
Why would there be more info on the web about IBM mainframe programming?
I think that hits a lot of it. And while "mainframe" (really IBM z/OS, z/VM, CICS, etc. in this context) is probably the most prominent example, the same applies to a lot of other "legacy" computing like MUMPS, ADA, IBM i, other mainframes and minicomputers still in use, etc. Collectively, there's still a lot of that sort of thing but it's essentially all proprietary and there's not much in the way of community except informally though, in many cases, there are still user groups, etc.
I know a bunch of MUMPS folks too and yes, it's a very similar vibe. Ditto with other "niche" tech communities. Some people just really like those little pockets outside of the mainstream. One of my mainframe friends frequently posts pictures of RL gatherings of that community where he is, and it definitely seems like a good scene to be in. If I ever had to un-retire, it might well be into something like that.
It's not the sort of thing I'd really recommend a young person to get into. I've seen people who were basically world experts on $LEGACY_THING as interest in said thing declined to zero and it wasn't pretty. But there are also some fairly large niches and they're probably a comfortable enough place for a veteran to coast late career.
The mainframe programmers are skilled in the programming, so the questions they'll be asking will often be business logic questions, not "how do I assign a variable".
So even if you do see a mainframe programmer in the wild asking a question, you probably won't recognize it.
So even if you do see a mainframe programmer in the wild asking a question, you probably won't recognize it.
There isn't a lot on Stack Overflow and the rest of the web because it isn't needed.
Mainframe programming is not like web programming. There are not multiple frameworks and languages competing for developer attention and rising and falling in popularity every few years. It is much more "there is one way to do it, it's the same way we've done it for 15 years" and that is what you do. There are very few new questions. All the common questions are covered in training and the more uncommon ones are covered in the documentation. If there isn't already a standard way to do what you want to do, you are probably doing it wrong.
Mainframe programming is not like web programming. There are not multiple frameworks and languages competing for developer attention and rising and falling in popularity every few years. It is much more "there is one way to do it, it's the same way we've done it for 15 years" and that is what you do. There are very few new questions. All the common questions are covered in training and the more uncommon ones are covered in the documentation. If there isn't already a standard way to do what you want to do, you are probably doing it wrong.
There's also the existing code at the site for reference. That's how you should be doing things. You need a really good reason to not follow the local conventions. So, most answers you seek will be in the existing code base.
Also, I would add that most mainframe programming is flat procedural (no objects, no complex data structures - heck, COBOL doesn't have real strings). So the vendor manuals cover most of what you need to know, as things don't get too complicated and obscure. And there are always colleagues to ask too.
Also, I would add that most mainframe programming is flat procedural (no objects, no complex data structures - heck, COBOL doesn't have real strings). So the vendor manuals cover most of what you need to know, as things don't get too complicated and obscure. And there are always colleagues to ask too.
> Most of the results are on IBM's own website, and even that contains only high-level details.
As other people have stated in this thread, IBM Redbooks are what you need and they span the whole range from high level overflow to low level.
The ABC's of System Programming is probably where you want to start, there are 13 volumes I think.
https://colinpaice.blog/2020/06/22/whats-in-abcs-of-z-os-sys...
You just have to know what to look for to find low level stuff - for an example, here is the reference for IBM's High Level Assembler Language (HLASM).
https://www-40.ibm.com/servers/resourcelink/svc00100.nsf/pag...
> And if there are, then why is there so little info for them on the web? Is the knowledge proprietary and secret?
IMO it is just a cultural thing. A lot of mainframe developers and especially the experts who have deep knowledge have been programming for a long time. Like, started on punch cards while they were getting a math degree.
The IBM culture has always been comprehensive documentation available from the source and predates the internet by a few decades, so the kind of public stack overflow culture we are used to never developed over there.
The book Hackers: Heroes of the Computer Revolution by Steven Levy kind of talks about this, the difference between the mainframe guys and the mini computer guys. Its a good read.
Mainframe people are descendants of the "high priests" that worked on computers before the invention of the mini computer and the culture reflects that to this day.
As other people have stated in this thread, IBM Redbooks are what you need and they span the whole range from high level overflow to low level.
The ABC's of System Programming is probably where you want to start, there are 13 volumes I think.
https://colinpaice.blog/2020/06/22/whats-in-abcs-of-z-os-sys...
You just have to know what to look for to find low level stuff - for an example, here is the reference for IBM's High Level Assembler Language (HLASM).
https://www-40.ibm.com/servers/resourcelink/svc00100.nsf/pag...
> And if there are, then why is there so little info for them on the web? Is the knowledge proprietary and secret?
IMO it is just a cultural thing. A lot of mainframe developers and especially the experts who have deep knowledge have been programming for a long time. Like, started on punch cards while they were getting a math degree.
The IBM culture has always been comprehensive documentation available from the source and predates the internet by a few decades, so the kind of public stack overflow culture we are used to never developed over there.
The book Hackers: Heroes of the Computer Revolution by Steven Levy kind of talks about this, the difference between the mainframe guys and the mini computer guys. Its a good read.
Mainframe people are descendants of the "high priests" that worked on computers before the invention of the mini computer and the culture reflects that to this day.
I suspect a lot of mainframe development today is Java EE centered, for which there is an abundance of stuff on the web.
During my vocational training, I did a bit of work with mainframes and found a forum back then I liked, but my work (and the forum) was mostly about sysadmin-stuff, and the little coding I did was in Perl.
There is a sizable library of RedBooks on IBM's web site somewhere, but again, the stuff I was interested in was about system administration.
A lot of the culture in the mainframe world developed a long time ago, and in relative isolation from the rest of the IT world, so I suspect by the time Stack Overflow became popular, their communities had already found other ways and places to mingle and pass knowledge on and around.
With hardware, software, and support contract being so expensive, and corporate culture being what it is, I also suspect the threshold to call up the the vendor for support instead of Stack Overflow is much lower than in most other areas.
(Most of this is speculation on my part, I only spend a few months in this fascinating corner of IT. If I'm wrong, I'd love to hear to from someone with actual experience.)
During my vocational training, I did a bit of work with mainframes and found a forum back then I liked, but my work (and the forum) was mostly about sysadmin-stuff, and the little coding I did was in Perl.
There is a sizable library of RedBooks on IBM's web site somewhere, but again, the stuff I was interested in was about system administration.
A lot of the culture in the mainframe world developed a long time ago, and in relative isolation from the rest of the IT world, so I suspect by the time Stack Overflow became popular, their communities had already found other ways and places to mingle and pass knowledge on and around.
With hardware, software, and support contract being so expensive, and corporate culture being what it is, I also suspect the threshold to call up the the vendor for support instead of Stack Overflow is much lower than in most other areas.
(Most of this is speculation on my part, I only spend a few months in this fascinating corner of IT. If I'm wrong, I'd love to hear to from someone with actual experience.)
I suspect a lot of mainframe development today is Java EE centered, for which there is an abundance of stuff on the web.
Indeed, and not just Java. zLinux runs a great many workloads on M/Fs these days, which looks a lot like a normal Linux env.
Then ofc K8S/docker/etc can run on mainframes, presenting another layer of abstraction.
Modern mainfame programming is just a question of getting to know a slightly different set of tools.
Indeed, and not just Java. zLinux runs a great many workloads on M/Fs these days, which looks a lot like a normal Linux env.
Then ofc K8S/docker/etc can run on mainframes, presenting another layer of abstraction.
Modern mainfame programming is just a question of getting to know a slightly different set of tools.
> Then ofc K8S/docker/etc can run on mainframes, presenting another layer of abstraction.
wtf why even plow $$$ into hardware at that point?
wtf why even plow $$$ into hardware at that point?
Mainframes are more energy efficient for a given level of performance than an equivalent farm of servers. Additionally, the redundancy, hot-swapping, and resource utilisation is generally better.
A corporation might well calculate that, for its typical workloads, mainframes make a better K8s platform in the long run than server farms.
A corporation might well calculate that, for its typical workloads, mainframes make a better K8s platform in the long run than server farms.
I find that claim pretty hard to believe tbh. Any actual benches published? It’s like Oracle claiming they are the shit but then actually not allowing any performance benching whatsoever. The 14nm process that they use to build those kinda leads me to believe it’s all marketing
Why should it be so hard to believe? A mainframe box is about the size of a rack of servers, and all the components are much more tightly integrated. Plus the utilisation on m/fs is much higher than on normal servers.
I don't have benchmarks, but googling "mainframe energy efficiency" brings up a load of results, eg:
https://www.bmc.com/blogs/mainframe-sustainability-green-it/
https://www.serverwatch.com/guides/for-energy-savings-consid...
https://www.suse.com/c/mainframe-versus-server-farm-comparis...
I don't have benchmarks, but googling "mainframe energy efficiency" brings up a load of results, eg:
https://www.bmc.com/blogs/mainframe-sustainability-green-it/
https://www.serverwatch.com/guides/for-energy-savings-consid...
https://www.suse.com/c/mainframe-versus-server-farm-comparis...
These all seems like pure marketing. I would love to see actual technical benchmark (this would also make prime HN material even today). Usually companies that actually think they have a performance edge publish those ;)
There was a funny ad years back where an exec walks into the data center and finds it nearly empty after the young newbie has consolidated all of their servers into VMs running on a single z990. (In retrospect I notice there are no disk drives or tape drives to be seen, though.)
In the late 1990s, when I joined a bank that used IBM mainframes, there was a group of 30ish new CS graduates who would spend many months being trained internally and by IBM to learn to program and operate those. As I understand it, that's still the case today.
The people who operate these mainframes have tremendous access to internal IBM resources. I remember having to stifle my laughter as, during a meeting about a major outage, a seasoned programmer explained that he "took a dump and sent it to IBM" and was expecting to hear back any time.
The channels that exist for this knowledge are expansive and just aren't on the internet, though they do occasionally spill over there.
The people who operate these mainframes have tremendous access to internal IBM resources. I remember having to stifle my laughter as, during a meeting about a major outage, a seasoned programmer explained that he "took a dump and sent it to IBM" and was expecting to hear back any time.
The channels that exist for this knowledge are expansive and just aren't on the internet, though they do occasionally spill over there.
This was the model when I joined Eastern Airlines' reservation system (aka, System One) out of college, although the training was entirely in-house; IBM was not directly involved, that I could tell. I enjoyed spending the first few months of my career getting paid to sit in a classroom and learn this stuff.
I would love to hear about that experience, if you’ve written about it
I haven't yet, beyond some quick comments I've posted on other HN threads. I may capture it in long form after I've retired. My career has taken a meandering path, and there have been aspects of it that I think might be entertaining to the right audience. If there's something specific you're curious about, I'd be happy to try to address that.
Much of the expertise on the platform has been slowly internalized by IBM over the years. The experts on the platform work for them.
Example. I used to work at a massive organization that did have some core applications on mainframes. The guys who maintained it retired and they couldn't find anyone to fill the position. So they contracted with IBM to maintain the system. Guess who showed up on site wearing IBM polos? The guys who just retired. Now doing the same job but making even more while doing it.
Example. I used to work at a massive organization that did have some core applications on mainframes. The guys who maintained it retired and they couldn't find anyone to fill the position. So they contracted with IBM to maintain the system. Guess who showed up on site wearing IBM polos? The guys who just retired. Now doing the same job but making even more while doing it.
I went through a mainframe cobol bootcamp at a company in the 90's and worked on their code for a few years. The thing to know today is most all the jobs (in the US) were offshored, very few of those jobs are left in the US today.
The other thing to know is; it's really hard to learn if you don't have access to one (obviously). They are from an era when computing was not ubiquitous, there is a whole parallel universe of computing in mainframes where things have different names and conventions, while the basic concepts are the same. It can be very hard to translate.
For me, my time on mainframes was enlightening and interesting, but a major career dead end. I had to go back to entry level in a different stack to get out of it. Glad I did, those jobs are all dead end, miserable, cost centers. Every company who has mainframe applications would love to get off them, but they can't because the cost is too high.
The other thing to know is; it's really hard to learn if you don't have access to one (obviously). They are from an era when computing was not ubiquitous, there is a whole parallel universe of computing in mainframes where things have different names and conventions, while the basic concepts are the same. It can be very hard to translate.
For me, my time on mainframes was enlightening and interesting, but a major career dead end. I had to go back to entry level in a different stack to get out of it. Glad I did, those jobs are all dead end, miserable, cost centers. Every company who has mainframe applications would love to get off them, but they can't because the cost is too high.
A lot of the documentation is printed in manuals/books written before the 90s. I have inherited a whole closet worth of COBOL programs (on both magnetic tape and print out on continuous form paper via dot matrix) and mainframe documentation from my parents who made a career in COBOL programming. Many of the people who worked on those systems predated the internet and the concept of sharing information via the internet was not part of the culture. It's one thing I admire about today's devs- the amount of free and high value information on YouTube, Twitch, GitHub and Stack Overflow.
I encourage young software engineers to seriously look at Mainframe development as a possible career path. Currently there is an aging knowledge base that needs to be replaced. Also, that world lends itself more to continually leveling up (i.e. experience matters more) than the typical development world where technologies change so fast.
On the down side it's much harder to bootstrap an environment for doing side projects on the mainframe. The hercules emulator makes it possible locally but it's a pain [0]. However, I think IBM provides sandox's for people to mess around with and all that requires is some minimal sign up and a TN3270 terminal emulator.
[0] http://www.hercules-390.org/
On the down side it's much harder to bootstrap an environment for doing side projects on the mainframe. The hercules emulator makes it possible locally but it's a pain [0]. However, I think IBM provides sandox's for people to mess around with and all that requires is some minimal sign up and a TN3270 terminal emulator.
[0] http://www.hercules-390.org/
The world runs on them but so far isn’t willing to pay for the skills. It also resigns you to the world of legacy code and tech debt. The person who has to ask for budget to fix problems from 30 years ago is very under appreciated. (“What new functionality am I getting for all that money?”)
IBM is also a very painful vendor. They’re in the audit and share buyback business. They struggle to innovate. (One can argue with their track record on innovation they really should just milk the existing contracts and buy back shares)
IBM is also a very painful vendor. They’re in the audit and share buyback business. They struggle to innovate. (One can argue with their track record on innovation they really should just milk the existing contracts and buy back shares)
> Currently there is an ageing knowledge base that needs to be replaced.
No, there are lots of companies that need to modernise their tech stack :)
I've been using a personal and business bank account with no mainframes involved for about 6 years now. Golang, microservices, Cassandra/Postgres. Progress is great to see.
No, there are lots of companies that need to modernise their tech stack :)
I've been using a personal and business bank account with no mainframes involved for about 6 years now. Golang, microservices, Cassandra/Postgres. Progress is great to see.
Ditto. COBOL and RPG aren’t sexy, but they pretty much run the world from healthcare to airlines.
You'll find mainframes in many/most large older (founded before 1990) organizations but almost nowhere else. There doesn't tend to be a lot of detailed information publicly available for proprietary enterprise products generally. This is partly because that's the way many companies prefer it (i.e. trade secrets and NDA's) and partly because they just aren't of general interest to hobbyists/hackers/smaller customers. In addition to the lack of information, mainframes also have significant physical/electrical/cost constraints which further limits interest in terms of actually owning one... which would be a driver of interest.
Compare this to your typical PC/mobile device which fits on/in a desk/lap/pocket, is practically free in comparison, has a deluge of information available everywhere and is actually useful to most people in everyday life. About the only thing a mainframe really gets you is reliability in a monolithic device which from a cost/benefit standpoint isn't worthwhile for most customers and can be approximated via other means.
I've been inside many organizations that run mainframes and if you're not on one of the mainframe teams, which tend to be tiny relative to other IT groups, it's pretty much plumbing as far as the rest of the organization is concerned... which is the point.
Compare this to your typical PC/mobile device which fits on/in a desk/lap/pocket, is practically free in comparison, has a deluge of information available everywhere and is actually useful to most people in everyday life. About the only thing a mainframe really gets you is reliability in a monolithic device which from a cost/benefit standpoint isn't worthwhile for most customers and can be approximated via other means.
I've been inside many organizations that run mainframes and if you're not on one of the mainframe teams, which tend to be tiny relative to other IT groups, it's pretty much plumbing as far as the rest of the organization is concerned... which is the point.
A few years back I did something called 'master the mainframe' where they provided a TSO login for you on an instance running in one of their labs. They also provided you with a bunch of fun exercises to learn more about the system.
I think this is the equivalent - https://ibmzxplore.influitive.com/users/sign_in
I think this is the equivalent - https://ibmzxplore.influitive.com/users/sign_in
"The IBM Z Xplore Learning Platform (formerly called Master the Mainframe) is a fun way to get hands-on experience across a variety of technologies, to develop valuable skills, and to earn digital badges – with no prior knowledge required!"
I've had this bookmarked for years, never gotten around to starting it, though. Web development continues to entertain, challenge, and sustain me.
I've had this bookmarked for years, never gotten around to starting it, though. Web development continues to entertain, challenge, and sustain me.
There's stuff out there, it's just not always easy to find. A lot of the IBM mainframe stuff, for example, is buried in their catalog of "Red Books" found at www.redbooks.ibm.com.
Check out:
https://www.ibm.com/z/trials
https://www.redbooks.ibm.com/search?query=%22system%20progra...
https://developer.ibm.com/?q=z%2Fos
Also search ibm.com for terms like "PL/I", "COBOL", "CICS", "RPG" and so on and you'll likely turn up more of the mainframe stuff. You can, of course, program mainframes in other languages, especially Java. But from an IBM perspective, references to the former batch of terms will generally turn up stuff about their mainframes, or their minicomputer series (mostly what's called iSeries these days, as far as I know. Formerly the AS/400, S/38, etc).
There are also various print books from 3rd party sources that can be found on Amazon. Ex:
https://www.amazon.com/IBM-COBOL-Complete-Guide-2020/dp/0655...
https://www.amazon.com/Quick-Start-Training-Application-Deve...
etc.
The reasons for this are, so far as I can tell, largely what other commenters have already posted. It's just a different culture and way of doing things, and the mainframe world just works a bit differently. And I think a lot of that is because that world is so insular now with IBM being more or less the only mainframe vendor left. I suppose Unisys or Hitachi or Fujitsu or somebody may still make a mainframe here and there, but the market is almost completely dominated by IBM. So the "mainframe way of doing things" is more or less "the IBM way of doing things" and in particular it's a very old skool version of "The IBM way" at that.
Check out:
https://www.ibm.com/z/trials
https://www.redbooks.ibm.com/search?query=%22system%20progra...
https://developer.ibm.com/?q=z%2Fos
Also search ibm.com for terms like "PL/I", "COBOL", "CICS", "RPG" and so on and you'll likely turn up more of the mainframe stuff. You can, of course, program mainframes in other languages, especially Java. But from an IBM perspective, references to the former batch of terms will generally turn up stuff about their mainframes, or their minicomputer series (mostly what's called iSeries these days, as far as I know. Formerly the AS/400, S/38, etc).
There are also various print books from 3rd party sources that can be found on Amazon. Ex:
https://www.amazon.com/IBM-COBOL-Complete-Guide-2020/dp/0655...
https://www.amazon.com/Quick-Start-Training-Application-Deve...
etc.
The reasons for this are, so far as I can tell, largely what other commenters have already posted. It's just a different culture and way of doing things, and the mainframe world just works a bit differently. And I think a lot of that is because that world is so insular now with IBM being more or less the only mainframe vendor left. I suppose Unisys or Hitachi or Fujitsu or somebody may still make a mainframe here and there, but the market is almost completely dominated by IBM. So the "mainframe way of doing things" is more or less "the IBM way of doing things" and in particular it's a very old skool version of "The IBM way" at that.
We actually published a research paper recently on COBOL communities, but I still don't have a great answer to your question.
Much of the information is locked away in developers' heads and it was difficult for us to find many resources on getting started with COBOL. In fact, most of the COBOL code on GitHub is homework assignments but almost no real-world programs that we could analyze.
Our paper which contributes a defects benchmark: https://austinhenley.com/pubs/Lee2022ICSME_COBOL.pdf
Much of the information is locked away in developers' heads and it was difficult for us to find many resources on getting started with COBOL. In fact, most of the COBOL code on GitHub is homework assignments but almost no real-world programs that we could analyze.
Our paper which contributes a defects benchmark: https://austinhenley.com/pubs/Lee2022ICSME_COBOL.pdf
Have you seen https://www.ibmmainframer.com/?
I would also dig through Project Guttenburg https://www.gutenberg.org/ for IBM Mainframe Programming books if you're interested in the history.
I would also dig through Project Guttenburg https://www.gutenberg.org/ for IBM Mainframe Programming books if you're interested in the history.
I worked somewhere where almost everything was run on some kind of mainframe, but it was abstracted from 90% or more of the developers, who wrote more or less normal code in normal languages. The “mainframe programming” was basically a systems level or infra type role providing this platform to everyone else. I imagine this is equivalent to a k8s team in other companies. However I’m sure there are plenty of banks and other places that are actually writing business logic with the mainframes’ toolset.
I did hear IBM has free courses for people wanting to learn mainframe. For example:
https://www.ibm.com/blogs/ibm-training/free-course-announcin...
I am sure there are others out there too.
https://www.ibm.com/blogs/ibm-training/free-course-announcin...
I am sure there are others out there too.
Maybe because someone needing a fact on the mainframe realizes that almost no one on StackOverflow will help them, plus many will make fun of them for being a dinosaur?
Edit: I should add that I'd bet they DO have their own web forums, and a lot of them are probably run by IBM itself. (I'm not a mainframer, in case you're wondering)
Edit: I should add that I'd bet they DO have their own web forums, and a lot of them are probably run by IBM itself. (I'm not a mainframer, in case you're wondering)
These forums are even worse than Stack Overflow in terms of newbie friendliness. 'Ask your systems programmer' seems to be the reply du jour just as SO's default is 'closed as a dupe'.
Reading Red Books has better ROI than trawling for help on the 'Net.
Everybody is welcome to watch Moshix channel on YouTube, though. It's refreshing.
Reading Red Books has better ROI than trawling for help on the 'Net.
Everybody is welcome to watch Moshix channel on YouTube, though. It's refreshing.
Have you seen: https://www.ibm.com/docs/en/zos-basic-skills
I've no knowledge of IBM mainframes - but I assume they're sold as part of a large consultancy/training package.
I've no knowledge of IBM mainframes - but I assume they're sold as part of a large consultancy/training package.
This is an interesting question actually and it leads me to draw parallels with some IRC channels, like for example #vmware and #rhel at libera (formerly freenode).
These are what I would call Enterprise environments, IBM, VMware, RHEL. And they all have one thing in common, lack of community documentation. Because in Enterprise environments you're expected to be trained.
And in the IRC channels people mostly talk about politics, or just everyday stuff. Very rarely do they nerd out about Linux or hypervisors. Because most people in there are trained engineers and neither need help, nor are willing to give help to someone.
If you go up to a crowd of enterprise people to ask a question, you better ask a good question. The old adage "there are no stupid questions" does not apply at all. Because these are trained people who have very specific knowledge of their Enterprise systems. They don't mess around with community projects or open source for fun. They do it to achieve a goal.
I know someone who started at IBM about a year ago and he's learning quickly how to program on IBM mainframes. By other senior engineers, and trainers.
These are what I would call Enterprise environments, IBM, VMware, RHEL. And they all have one thing in common, lack of community documentation. Because in Enterprise environments you're expected to be trained.
And in the IRC channels people mostly talk about politics, or just everyday stuff. Very rarely do they nerd out about Linux or hypervisors. Because most people in there are trained engineers and neither need help, nor are willing to give help to someone.
If you go up to a crowd of enterprise people to ask a question, you better ask a good question. The old adage "there are no stupid questions" does not apply at all. Because these are trained people who have very specific knowledge of their Enterprise systems. They don't mess around with community projects or open source for fun. They do it to achieve a goal.
I know someone who started at IBM about a year ago and he's learning quickly how to program on IBM mainframes. By other senior engineers, and trainers.
While others addressed different parts already, I just wanted to mention that the IBM (well, any Corp) marketing data should be treated like it's written to be technically/legally correct but stretched as positive as possible.
> "ninety percent of Fortune 500 companies rely on the IBM Mainframe"
This doesn't say they own them. It's also very vague given they know the exact number.
> "eighty percent of the worlds corporate data resides or originates on the mainframe."
This doesn't say it's the primary store, or is that by volume or dataset count, or what constitutes "data". It doesn't say where the chain ends either. If the user record is created on a mainframe, did everything related to the user originate on the mainframe?
Take it with appropriate amount of salt.
> "ninety percent of Fortune 500 companies rely on the IBM Mainframe"
This doesn't say they own them. It's also very vague given they know the exact number.
> "eighty percent of the worlds corporate data resides or originates on the mainframe."
This doesn't say it's the primary store, or is that by volume or dataset count, or what constitutes "data". It doesn't say where the chain ends either. If the user record is created on a mainframe, did everything related to the user originate on the mainframe?
Take it with appropriate amount of salt.
Look for IBM Redbooks, also the "Mainframe" has 5 different operating systems, zVM, OS2, zOS, Linux (s390x architecture), and somehow I forgot the other.
There is Code in a multitude of languages even when it is not Linux, from C++ to Java, but nowadays also nodejs and whatever you want to run in that architcture.
Edit:
Basically, you should look for "System Z" which is the actual name of the machine, and "redbooks" the name of the documentation manuals.
Example: https://www.redbooks.ibm.com/domains/zsystems
There is Code in a multitude of languages even when it is not Linux, from C++ to Java, but nowadays also nodejs and whatever you want to run in that architcture.
Edit:
Basically, you should look for "System Z" which is the actual name of the machine, and "redbooks" the name of the documentation manuals.
Example: https://www.redbooks.ibm.com/domains/zsystems
https://www.redbooks.ibm.com
has a lot off z howto books
has a lot off z howto books
Ah yes the yearly Mainframe thread on HN. Many years ago (10+) one of these threads convinced me that I should get into mainframe programming. But I couldn't figure out how to get in. The actual job postings all require 5+ years of experience with Mainframes and I'm starting from zero. And there aren't that many job postings, I'd say one mainframe job posting for every 1,000 or 10,000 web developer roles. So why bother?
What are some mainframe jobs (developer, sysadmin, others) and what do they pay? Are any of these jobs in SV or remote?
I cut my teeth on A/UX and SunOS so I'm of a certain age, but besides having school accounts for intro/101 courses using 3270 emulation I never really got hands on. For the same reasons that today I still wish Solaris had won, I think I would enjoy certain aspects of the mainframe environment.
I wonder if any of my modern stack skillset would be useful or not.
I cut my teeth on A/UX and SunOS so I'm of a certain age, but besides having school accounts for intro/101 courses using 3270 emulation I never really got hands on. For the same reasons that today I still wish Solaris had won, I think I would enjoy certain aspects of the mainframe environment.
I wonder if any of my modern stack skillset would be useful or not.
Google now has a jobs answer box. First hit is mainframe app dev, apparently contract for $520/week in San Jose. They can actually hire people at that rate?
There aren't resources to practice it but IBM documentation is actually really really good it reminds me of MSDN in the 2000s. So if you can get access to an environment to practice the documentation is really solid. It surprises me that you complain about IBM website because you can get everything there, even detailed machine instruction reference for z/Architecture if you want.
Companies have been trying to bury the mainframe for 40+ years in search of "cheaper" alternatives. Software companies, with few exceptions, have followed suit. The early implementations of "Open Systems" software on the mainframe were generally unmitigated performance disasters. It's a bit better now, but not something you'd want to bet the bank on.
Not that it proves anything, but here's a S.O. post from yesterday...
https://stackoverflow.com/questions/74593657/dsnacics-stored...
Compared to other stacks/platforms, it may seem a dearth, but given the number of programmers working in that area (and as others here have pointed out, given the very different nature of mainframe programming in general), it's no so surprising that there's relatively little info that's searchable on the public internet. TBH I'm still frequently surprised by not being able to find information about a particular error I'm getting with a particular API that I would have thought was in relatively common use even in the C#/.NET world!
https://stackoverflow.com/questions/74593657/dsnacics-stored...
Compared to other stacks/platforms, it may seem a dearth, but given the number of programmers working in that area (and as others here have pointed out, given the very different nature of mainframe programming in general), it's no so surprising that there's relatively little info that's searchable on the public internet. TBH I'm still frequently surprised by not being able to find information about a particular error I'm getting with a particular API that I would have thought was in relatively common use even in the C#/.NET world!
You get physical manuals as part of your support agreement, like a bread truck full. In my experience it was something you learned by sitting next to someone who had been doing it for decades and they just call each other if they forgot something. The EBCDC chart was the most used doc.
You can finds lots of books and online resources on COBOL programming or JCL. There are also encyclopedia sized publications called IBM Redbooks (available in pdf if you Google for them) which provide tons of detail on hardware and OS level concepts you need background understanding of.
As for how you'd actually play around with writing usable software.. you'd need access to a mainframe. Basically, the insane complexity of mainframes and the walled garden nature of the ecosystem makes it extremely difficult to teach or to learn.
I worked on the z/OS network stack for a few years (tech support and later some actual development) and it was the most complex thing I've ever tried to learn in my career.
As for how you'd actually play around with writing usable software.. you'd need access to a mainframe. Basically, the insane complexity of mainframes and the walled garden nature of the ecosystem makes it extremely difficult to teach or to learn.
I worked on the z/OS network stack for a few years (tech support and later some actual development) and it was the most complex thing I've ever tried to learn in my career.
[deleted]
IBM historically was the most proprietary of the most proprietary. Like, if there were ANY information sharing out there in public about mainframe programming the way there is about Java/JavaScript/Node/Go/Rust or even Windows programming today, that would get you visited by an army of men in suits. Not government agents, but just about as scary: IBM lawyers.
So that cultural value of keep everything close to the vest and only learn through official channels, still persists in IBM mainframe culture today. Especially since there is little to no "hacker culture" surrounding mainframes since it's not like you can download z/OS and run it on your PC (things like Hercules notwithstanding).
So that cultural value of keep everything close to the vest and only learn through official channels, still persists in IBM mainframe culture today. Especially since there is little to no "hacker culture" surrounding mainframes since it's not like you can download z/OS and run it on your PC (things like Hercules notwithstanding).
This might have become true mostly after 1987 to 1990, when the IBM managers began to regret the former openness of IBM, (wrongly) believing that it might have contributed to their diminishing share in the computer market, and they have made various, mostly failed, attempts to replace more open products with proprietary products, like the IBM PS/2.
At least before 1980, during the IBM System/360 and IBM System/370 generations, abundant documentation about any aspect of any IBM product was available, much more detailed than any modern documentation.
Moreover, for many of the programs provided by IBM with the hardware, the source code was available, even if they were not open-source in the modern sense, because the users were not encouraged to modify and redistribute them, even if such activities happened sometimes, e.g. within the IBM SHARE user group.
When the IBM PC was introduced in 1981, this policy of providing exhaustive documentation was still active, which is why the IBM PCs were delivered with a ton of manuals, including descriptions of the hardware interfaces and the complete listing of the source code for the BIOS firmware.
At least before 1980, during the IBM System/360 and IBM System/370 generations, abundant documentation about any aspect of any IBM product was available, much more detailed than any modern documentation.
Moreover, for many of the programs provided by IBM with the hardware, the source code was available, even if they were not open-source in the modern sense, because the users were not encouraged to modify and redistribute them, even if such activities happened sometimes, e.g. within the IBM SHARE user group.
When the IBM PC was introduced in 1981, this policy of providing exhaustive documentation was still active, which is why the IBM PCs were delivered with a ton of manuals, including descriptions of the hardware interfaces and the complete listing of the source code for the BIOS firmware.
There's a ton of information on IBM mainframe programming, they're the IBM redbooks. They're all here:
https://www.redbooks.ibm.com/domains/zsystems
IBM's mainframes have some of the best technical documentation for any product line, anywhere, and if you can't find what you're looking for in the redbooks, the expensive support contract that you likely got with your z/Series comes with support that's sometimes described as... aggressive at solving high priority incidents with their internal support databases.
There's also the online manuals that come with z/OS, and you can get a free account via z Xplore: https://www.ibm.com/it-infrastructure/z/education/zxplore
You can also get a Linux LPAR on a s390x machine (LinuxONE, basically a regular mainframe with the processors fused to only run Linux) as part of the free tier on IBM Cloud.
That being said, most real-world deployments have a ton of other stuff running on top of the OS, usually from Broadcom and other close software partners, and dealing with those packages usually requires a support account with those companies as well. The applications themselves are very weird if you're not used to living in a mainframe environment. For example, you can get PKZIP for z/OS, and there is a 437 page PDF describing just the error messages alone.
Generally, learning how to get around in a mainframe environment requires working in a mainframe shop, nearly all of which are very mature and have converged on certain ways of dealing with testing, deployment, upgrades, migrations, etc. These procedures are usually not documented because they're simply well-understood by all the staff, or it would be dangerous for them to be working in the mainframe environment. Learning to use just ISPF, the closest thing z/OS has to a management shell, requires not only learning ISPF, but also all the installation-specific details for a given environment.
https://www.redbooks.ibm.com/domains/zsystems
IBM's mainframes have some of the best technical documentation for any product line, anywhere, and if you can't find what you're looking for in the redbooks, the expensive support contract that you likely got with your z/Series comes with support that's sometimes described as... aggressive at solving high priority incidents with their internal support databases.
There's also the online manuals that come with z/OS, and you can get a free account via z Xplore: https://www.ibm.com/it-infrastructure/z/education/zxplore
You can also get a Linux LPAR on a s390x machine (LinuxONE, basically a regular mainframe with the processors fused to only run Linux) as part of the free tier on IBM Cloud.
That being said, most real-world deployments have a ton of other stuff running on top of the OS, usually from Broadcom and other close software partners, and dealing with those packages usually requires a support account with those companies as well. The applications themselves are very weird if you're not used to living in a mainframe environment. For example, you can get PKZIP for z/OS, and there is a 437 page PDF describing just the error messages alone.
Generally, learning how to get around in a mainframe environment requires working in a mainframe shop, nearly all of which are very mature and have converged on certain ways of dealing with testing, deployment, upgrades, migrations, etc. These procedures are usually not documented because they're simply well-understood by all the staff, or it would be dangerous for them to be working in the mainframe environment. Learning to use just ISPF, the closest thing z/OS has to a management shell, requires not only learning ISPF, but also all the installation-specific details for a given environment.
z/OS Library
https://www-40.ibm.com/servers/resourcelink/svc00100.nsf/pag...
There's also a z/VM library.
You can fill a large room with the printed manuals.
When I was working on the mainframe, I usually had about 20' of shelves for the manuals I used.
The trick is knowing where to look in which manual.
I characterised my job as being a professional [IBM] manual reader.
There was a gazillion times a programmer came to me with a question and I pulled out a manual.
Source code was also available for many pieces. I made several modifications and patches.
https://www-40.ibm.com/servers/resourcelink/svc00100.nsf/pag...
There's also a z/VM library.
You can fill a large room with the printed manuals.
When I was working on the mainframe, I usually had about 20' of shelves for the manuals I used.
The trick is knowing where to look in which manual.
I characterised my job as being a professional [IBM] manual reader.
There was a gazillion times a programmer came to me with a question and I pulled out a manual.
Source code was also available for many pieces. I made several modifications and patches.
Everything was books. There weren't very many of the computers anyway.
Also maybe you should know about Seattle's 'Living Computer History Museum'. It's been closed for a couple years but I'm ready to visit once they reopen.
Also maybe you should know about Seattle's 'Living Computer History Museum'. It's been closed for a couple years but I'm ready to visit once they reopen.
There is plenty of info I thought? The platform is z/os and JCL is basically what you need to learn aside from assembly, cobol,etc...
https://www.ibm.com/docs/en/zos-basic-skills?topic=sdsf-what...
https://www.ibm.com/docs/en/zos-basic-skills?topic=zos-assem...
https://www.ibm.com/docs/en/zos-basic-skills?topic=sdsf-what...
https://www.ibm.com/docs/en/zos-basic-skills?topic=zos-assem...
Pub400.com – Your public IBM I 7.5 server
https://news.ycombinator.com/item?id=33032095 2022-09-30
https://news.ycombinator.com/item?id=33032095 2022-09-30
I haven't touched a mainframe in over a decade(and i was only ever doing user administration on one) but from my experience with hpux and other legacy platforms a big part of the reason is that the actual information about the core platform lives in old hardcopy textbooks and paywalled vendor documentation, and because a lot of those platforms are "feature frozen" the old textbooks are still valid enough.
And on the application side you are typically dealing with databases and Runtimes(Java mostly) that also exist on Linux and have an large Linux centric community where any legacy programmers and admins kind of drown in the masses.
There is a whole bunch of companies out there with large COBOL and ALGOL code-bases but but as the problem here is less about generic mainframe skills and more about knowing how to work large COBOL or ALGOL projects.
And on the application side you are typically dealing with databases and Runtimes(Java mostly) that also exist on Linux and have an large Linux centric community where any legacy programmers and admins kind of drown in the masses.
There is a whole bunch of companies out there with large COBOL and ALGOL code-bases but but as the problem here is less about generic mainframe skills and more about knowing how to work large COBOL or ALGOL projects.
I worked on these systems at IBM. The reason they are not documented on the net is that IBM has more thorough and useful documentation on its own internal network and in it's "pubs" (copyrighted and available only to customers).
IBM also has very advanced diagnostics tools and error tracking and those were in production use before the internet came along. It makes sense to use those resources rather than the web.
IBM also has very advanced diagnostics tools and error tracking and those were in production use before the internet came along. It makes sense to use those resources rather than the web.
This doesn't answer your question completely, but the user group communities are very strong for both IBM Mainframe and IBM i platforms. https://www.share.org/ and https://www.gse.org for example.
This is typical in other fields of engineering. Details and best methods for running advanced design tools, some of the most advanced engineering knowledge, etc, are not found online in a easily-digestible Stack Overflow format. This is the norm, software engineering/Stack Overflow are the exception.
See if your local library card gives you access to /learning-oreilly-com
There is lots of content there.
There is lots of content there.
I was an IBM mainframe systems programmer (VM/370) for years.
All of the information you need is in the manuals. IBM is exceptionally good at documentation.
Also, the hardware instruction set is set in stone.
All of the information you need is in the manuals. IBM is exceptionally good at documentation.
Also, the hardware instruction set is set in stone.
https://www.redbooks.ibm.com
a lot of howto books
a lot of howto books
I used to be the community manager for a StackOverflow-style (pre-dated SO by a number of years) site that catered to enterprise technology and we had ~200k-400k monthly visitors just on AS/400 topics — it was a huge and active community, but there was very little advertiser interest in it. A few years after I left, they completely shuttered the site and nuked the archives, sadly, so that's probably part of it — my site and others like it were bought up, monetized, then nuked when the money moved elsewhere.
The other partial reason is that a ton of those conversations happen in old email groups that aren't indexed or index a lot worse than spam sites do. So if you're interested, searching around for mailing lists, users groups, and old forums and there's still good material out there, just buried and mostly catering to folks who know the basics, so a bit tough to parse if you're new to it.
The other partial reason is that a ton of those conversations happen in old email groups that aren't indexed or index a lot worse than spam sites do. So if you're interested, searching around for mailing lists, users groups, and old forums and there's still good material out there, just buried and mostly catering to folks who know the basics, so a bit tough to parse if you're new to it.
my theory:
- incredibly proprietary
- heavily customized for the environments they are running in (and boy howdy are there many)
- COBOL/Fortran/4GL not popular or desirable (for younger developers)
- incredibly proprietary
- heavily customized for the environments they are running in (and boy howdy are there many)
- COBOL/Fortran/4GL not popular or desirable (for younger developers)
1. Google doesn't want you to program for mainframes. Google wants you to program for the cloud.
2. developerworks site on IBM had some info on mainframes and mainframe programming.
3. It is not "cool".
2. developerworks site on IBM had some info on mainframes and mainframe programming.
3. It is not "cool".
Obligatory mention of ‘Master the Mainframe’
Just do a search at Amazon Books for:
IBM Mainframe or z/OS or etc.
Designed their model to resemble retail distributorships and then shocked pickachu when they realize software doesn't need the same logistics.
The most likely explanation would be that there are hardly any mainframe developers. But IBM's website proclaims that "ninety percent of Fortune 500 companies rely on the IBM Mainframe" and "eighty percent of the worlds corporate data resides or originates on the mainframe." If that's true then surely there must be a reasonable number of mainframe developers out there? And if there are, then why is there so little info for them on the web? Is the knowledge proprietary and secret? Or is it just somewhere other than the usual places?
I would be really interested to hear from anyone with knowledge about this.