A lot of this post relies on the recent open ai result they call GDPval (link below). They note some limitations (lack of iteration in the tasks and others) which are key complaints and possibly fundamental limitations of current models.
But more interesting is the 50% win rate stat that represents expert human performance in the paper.
That seems absurdly low, most employees don’t have a 50% success rate on self contained tasks that take ~1 day of work. That means at least one of a few things could be true:
1. The tasks aren’t defined in a way that makes real world sense
2. The tasks require iteration, which wasn’t tested, for real world success (as many tasks do)
I think while interesting and a very worthy research avenue, this paper is only the first in a still early area of understanding how AI will affect with the real world, and it’s hard to project well from this one paper.
This seems to be a theme, but I wonder how it passes muster with any semi-smart VCs. Don't they ask about this? If the unit economics are so obviously bad, how did they get through in the first place?
Isn't that frequently the case though? The number of times I've walked into a building and been captured by some security camera must be enormous. Maybe the resolution is lower, but 1.) that may not matter that much, they can still recognize/have a record of you and 2.) I'm sure security cameras get better over time and eventually that's going to be high res too.
As for the name<->picture mapping, many many buildings have a log book for visitors, doesn't take too much work to associate a name with video of who wrote in the book at that time.
My point is that it's quite likely you've already given out tons pictures of you to the locations you visit. We-work might be more explicit, but they're not unique.
I am a fairly experienced road cyclist and even though I prefer to ride outside AND have a computer-controlled bike trainer, I would consider one if I had the space. I also have several friends who prefer this to all their other options.
1.) It's way easier to get started up without the faff of setting up your trainer. If you have a dedicated bike+trainer this is less of an issue, but that is going to cost a fair amount (something like $1-1.5K) and take up about as much space, and not be anywhere near as seamless.
2.) The coaching you get is extremely rich compared to TrainerRoad and Zwift, both costing $15/person/month. So if you have 2 people on the same account you pay a bit more per person and get better coaching. They are also branching out into strength and other types of video coaching as part of the package.
I initially thought it was useless if you had a trainer for the bike, but I see the appeal now. If the app worked with my wahoo trainer, then I'd very likely buy the service and have it work with my existing equipment.
But more interesting is the 50% win rate stat that represents expert human performance in the paper.
That seems absurdly low, most employees don’t have a 50% success rate on self contained tasks that take ~1 day of work. That means at least one of a few things could be true:
1. The tasks aren’t defined in a way that makes real world sense
2. The tasks require iteration, which wasn’t tested, for real world success (as many tasks do)
I think while interesting and a very worthy research avenue, this paper is only the first in a still early area of understanding how AI will affect with the real world, and it’s hard to project well from this one paper.
https://cdn.openai.com/pdf/d5eb7428-c4e9-4a33-bd86-86dd4bcf1...