Ask HN: Why don't frontier AI model providers continuously improve their models?
Just what the title says: I'm wondering why we're still, years after ChatGPT, having to wait weeks or months for "the next version" of a model when so much else in the software world has moved toward continuous improvement?
1 comments
It just doesn't work that way.
To some extent you can append some knowledge to a model with low-rank adaptation and other techniques but if you want to train a model which is substantially better than your old model you need to train a new model which is much bigger and/or more efficient than your old model and it learns a whole new representation.
See https://en.wikipedia.org/wiki/Catastrophic_interference for one problem.
To some extent you can append some knowledge to a model with low-rank adaptation and other techniques but if you want to train a model which is substantially better than your old model you need to train a new model which is much bigger and/or more efficient than your old model and it learns a whole new representation.
See https://en.wikipedia.org/wiki/Catastrophic_interference for one problem.