I think some kind of discipline around releasing products in a timely way by cutting features would have done wonders. However, the kinds of computers SGI built were on the way out, so they couldn't have survived without moving in the direction that people wanted. Maybe it was a company whose time had come. SGI wasn't set up to compete with the likes of NVIDIA and Intel.
the framebuffer had a recursive rasterizer which followed a hilbert curve through memory, the thinking being that you bottom out the recursion instead performing triangle clipping, which was really expensive for the hardware at the time.
The problem was that when you take some polygons which come close to W=0 after perspective correction, their unclipped coordinates get humongous and you run out of interpolator precision. So, imagine you draw one polygon for the sky, another for the ground, and the damn things Z-fight each other!
SGI even came out with an extension to "hint" to the driver whether you want fast or accurate clipping on Octane. When set to fast, it was fast and wrong. When set to accurate, we did it on the CPU [1]
Oh, it's a lot longer story than that. I worked as SGI from just around its peak, to its downfall, seeing the company shrink to a tenth of its size while cutting products.
At the time, I was a fairly junior employee doing research in AGD, the advanced graphics division. I saw funny things, which should have led me to resign, but I didn't know better at the time. Starting in the late 90's, SGI was feeling competitive pressure from 3DFx, NVIDIA, 3DLabs, Evans and Sutherland (dying, but big), and they hadn't released a new graphics architecture in years. They were selling Infinite Reality 2's (which were just a clock increase over IR1), and some tired Impact graphics on Octanes. The O2 was long in the tooth. Internally, engineering was working on next gen graphics for both, and they were both dying of creeping featureitis. Nothing ever made a deadline, they kept slipping by months. The high end graphics pipes to replace infinite reality never shipped due to this, and the "VPro" graphics for Octane were fatally broken on a fundamental level, where fixing it would mean going back to the algorithmic drawing board, not just some Verilog tweak, basically, taping out a new chip. Why was it so broken? Because some engineers decided to implement a cool theory and were allowed to do it (no clipping, recursive rasterization, hilbert space memory organization).
At the same time, NVIDIA was shipping the GeForce, 3DFx was dying, and these consumer cards processed many times more triangles than SGI's flagship Infinite Reality 2, which was the size of a refrigerator and pulled kilowatts. SGI kept saying that anti-aliasing is the killer feature of SGI and that this is why we continue to sell into visual simulation and oil and gas sector. The line rendering quality on SGI hardware was far better as well. However, given SGI wasn't able to ship a new graphics system in perhaps 6 years at that point, and NVIDIA was launching a new architecture every two years, the reason to use SGI at big money customers quickly disappeared.
As for Rick Beluzzo, man, the was a buffoon. My first week at SGI was the week he became CEO, and in my very first allhands ever, someone asked something along the lines of, "We are hemmoraging a lot of money, what are you going to do about it"? He replied with, "Yeah, we are, but HP, E&S, etc, are hemmoraging a lot more and they have less in the bank, so we'll pick up their business". I should have quit my first week.
I worked at Google for eight years, and fell into a funk, because I picked up new challenges and moved teams, and learned a whole lot, but I also worked on backend infra projects, not shippable features, and you know how well that goes over with the perf review and promo committee.
So, I left for a startup. It was trial by fire, because Google does thing the Google way, and everyone else uses other technologies. Gone were borg, stubby, tap, and in came Kubernetes, REST, Jenkins. It took a long time to learn how the rest of the world works, and Google wasn't my first job, I started there after already working for fifteen years, but in eight years, the world changes a lot.
Now, I'm the main tech lead for a large startup on the verge of success. It's been a crap ton of work, grueling, I've probably made 30% of the income I would have if I stayed at Google in the years that I've been gone, but I've also worked with the best people I've ever encountered - better than at Google, and I feel professionally successful, albeit not financially.