poormathskills

vor 7 Monatendiscuss

Is it still “scraping” when the purpose of these transparency logs is to be used for this purpose?

vor 7 Monatendiscuss

OpenAI has never compared their models to models from other labs in their blog post. Open literally any past model launch post to see that.

poormathskills

vor 7 Monatendiscuss

For a minor version update (5.1 -> 5.2) that's a way bigger improvement than I would have guessed.

poormathskills

letztes Jahrdiscuss

Who has a (publicly released) model that is SOTA is constantly changing. It’s more interesting to see who is driving the innovation in the field, and right now that is pretty clearly OpenAI (GPT-3, first multi-modal model, first reasoning model, ect).

poormathskills

letztes Jahrdiscuss

Right. Those labs aren’t leading the industry.

poormathskills

letztes Jahrdiscuss

Go look at their past blog posts. OpenAI only ever benchmarks against their own models.

poormathskills

letztes Jahrdiscuss

Go look at their past blog posts. OpenAI only ever benchmarks against their own models.

This is pretty common across industries. The leader doesn’t compare themselves to the competition.

comments