Can GPUs run reliably in an orbital radiation environment at all? The combination of extreme transistor density and zero error checking seems like the opposite of what you'd want in a space processor.
It's far worse than that: to attribute a particular output to an input you don't just need the input data, you need the gradient updates that data caused in the training run. A couple hundred gigabytes input tokens times 175 billion parameters equals... impossibility.
This quote seems to fundamentally misunderstand what transformers are doing at all. Technically I suppose you could save all gradient updates from every input token, and do some weighted averaging to show what inputs affected the particular output the most, but saving all those gradient updates would be unimaginably space consuming. "Feasible" is doing a lot of work there.
It's very hard for people to get away from the idea that GPT is "copying" something, but that's not what it's doing. The reality is, to get the exact artifact which produced the code in question, you need "Call me Ishmael" from Moby Dick just as much as the Linux kernel source.