poormathskills

Is it still “scraping” when the purpose of these transparency logs is to be used for this purpose?

OpenAI has never compared their models to models from other labs in their blog post. Open literally any past model launch post to see that.

poormathskills

7 mesi fadiscuss

For a minor version update (5.1 -> 5.2) that's a way bigger improvement than I would have guessed.

poormathskills

anno scorsodiscuss

Who has a (publicly released) model that is SOTA is constantly changing. It’s more interesting to see who is driving the innovation in the field, and right now that is pretty clearly OpenAI (GPT-3, first multi-modal model, first reasoning model, ect).

poormathskills

anno scorsodiscuss

Right. Those labs aren’t leading the industry.

poormathskills

anno scorsodiscuss

Go look at their past blog posts. OpenAI only ever benchmarks against their own models.

poormathskills

anno scorsodiscuss

Go look at their past blog posts. OpenAI only ever benchmarks against their own models.

This is pretty common across industries. The leader doesn’t compare themselves to the competition.

comments