Sibyl – Prediction Service at DoorDash (2020)(doordash.engineering)
doordash.engineering
Sibyl – Prediction Service at DoorDash (2020)
https://doordash.engineering/2020/06/29/doordashs-new-prediction-service/
36 comments
The "Shadow predictions" bit is interesting. They could set that up in a way to score it versus actuals...where the shadow implementation automatically becomes the live one once it's better than the incumbent.
Sibyl, was also the name of the Delphi clone for OS/2 :)
I'm yet to have the 'wow' moment from any recommendation service.... Netflix, Amazon, Deliveroo... At best I get the 'meh why not'. It's obviously a hard problem to solve.
What about Youtube? Im surprised to not see them mentioned in any of the comments under yours. For all their faults, I regularly am recommended amazing(ly interesting to me) videos.
I read that something like 60 hours of video are uploaded to YouTube every second, so I find it hard to believe that the best stuff they can recommend is the same handful of 9 year old videos I've already watched sprinkled with dogshit right wing American political content.
My view on this is that recommendation services are made to try to funnel people to the same thing. While they sell some kind of "hyper personalization", they're just another kind of classic marketing. Especially for Netflix. Talking about series you watched seems to be a big part of watching series, so pushing people to watch the same stuff works better.
> It's obviously a hard problem to solve.
Or maybe that is not the problem they are trying to solve. They optimize to sell as much as possible, not to get you the best experience.
It's like asking for the most shopped articles that are the ones farther away. Is it difficult to design better supermarkets? Or are they just optimizing for you to spend more time and money inside?
Or maybe that is not the problem they are trying to solve. They optimize to sell as much as possible, not to get you the best experience.
It's like asking for the most shopped articles that are the ones farther away. Is it difficult to design better supermarkets? Or are they just optimizing for you to spend more time and money inside?
If they had the full catalogue of all movies and series ever made then it'd easily be much easier to actually make more relevant recommendations, but their catalogue is actually shrinking as they have less old content they've been able to renew since those content owners have been starting their own streaming services.
Amazon, the book store has everything published under the sun and still cannot recommend something better than "oh, here you go, this is sci-fi as well!".
They arguably don't have your view history though, though their relatively new streaming service will be likely changing that slowly.
Try TikTok for a few weeks, genuinely
TikTok is optimized to get as much signal as possible from users, as well as they were smart about getting some signal from the videos themselves: https://www.eugenewei.com/blog/2020/9/18/seeing-like-an-algo...
Something like Amazon gets very little signal after you bought something whether you really liked it or not.
Something like Amazon gets very little signal after you bought something whether you really liked it or not.
Soundcloud has amazing recommendations, which is the best feature of the platform in my view. It's so easy to find great music that's related to tracks you already like. I have found literally thousands of new tracks from its 'Station' feature.
I feel like Netflix gave up. All I get is shit I already saw.
tiktok is the only one i think
I see posts like this and just think that as far as I can tell DD's recommendations are:
Ads
Ads
Ads
Shitty place nobody would want to eat
Shitty place nobody would want to eat
Shitty place nobody would want to eat
Shitty place nobody would want to eat
Oh look something interestingI have the same experience as you, and it's one of the main reasons why I try to call places for takeout instead (or use a local app).
I (idly) wonder whether they're optimizing for the wrong(?) things: throwing a bunch of garbage up front maximizes the amount of time that I spend in-app, which is probably one of their key metrics. What they don't(?) realize is that I'm spending all that time filtering through the dreck, and so the metric is really measuring my frustration and not my engagement.
I (idly) wonder whether they're optimizing for the wrong(?) things: throwing a bunch of garbage up front maximizes the amount of time that I spend in-app, which is probably one of their key metrics. What they don't(?) realize is that I'm spending all that time filtering through the dreck, and so the metric is really measuring my frustration and not my engagement.
Doordash: Hey try this new place!
lists taco bell
Relevancy is hard!
I just cancelled my membership (was free with a credit card i already had).
They offered a "free membership" for 10 months "at a $120 value" (which used to be $60 a year), however I couldn't figure out how to activate it.
I can't figure out what the value is over other services/direct from restaurant. Like another poster, most of the suggestions were bad. "Do you want dairy queen?" Or "quick you can order free from Kwik Trip in the next 10 minutes".
The post makes the suggestions look good. My top recommendation is the cheesecake factory. Followed by McDonalds. Who are they targeting? Who is getting McDonalds delivered?
They offered a "free membership" for 10 months "at a $120 value" (which used to be $60 a year), however I couldn't figure out how to activate it.
I can't figure out what the value is over other services/direct from restaurant. Like another poster, most of the suggestions were bad. "Do you want dairy queen?" Or "quick you can order free from Kwik Trip in the next 10 minutes".
The post makes the suggestions look good. My top recommendation is the cheesecake factory. Followed by McDonalds. Who are they targeting? Who is getting McDonalds delivered?
McDonalds is one of the most popular restaurants in all of America. Forget DD, probably 10% of America orders McDonald's every day. It's not that surprising. It probably keeps pretty well during delivery because it's designed to be under a warmer for long periods of time.
We get McDonalds delivered all the time. We don’t have a car.
One of my wife's assistants likes McDonald's. A weekly reward for no products breaking in shopping when she does packaging is lunch. The cost of having McDonald's delivered is far less than the opportunity cost for her to pick it up (and obviously for us too).
I get McDonald's delivered all the time. Can't beat 13 minute delivery for coffee in the morning.
My coffee routine only takes 8 mins from whole beans to v60 pour over, most of that the kettle warming.
How many minutes of active work, though? Presumably OPs 13 minute wait time involves about 30 seconds of work and 12 1/2 minutes of doing whatever they feel like.
(in any case let’s be honest, you’re both optimising for very different parameters. Which is fine!)
(in any case let’s be honest, you’re both optimising for very different parameters. Which is fine!)
... with MUCH better-tasting coffee than e.g. McDonald's as a bonus!
Do you order anything besides coffee? Maybe I'm misunderstanding, but why would you wait 13 minutes for someone to drop off $1 of distinctly average coffee?
(I say this as someone who has ordered McDonald's through DoorDash before.)
(I say this as someone who has ordered McDonald's through DoorDash before.)
It's a $12 minimum order where I am anyway. So you pay $12 whether you have $12 of items or not.
So, I am guessing GP has a couple egg-mcmuffins in there somewhere.
So, I am guessing GP has a couple egg-mcmuffins in there somewhere.
Yep. I get egg mcmuffins with my distinctly average coffee.
I get McDonalds delivered. Why not?
I predict they'll get my order wrong or leave something off despite charging me and deliver my food cold.
Wait, so they are actively encouraging Sybil attacks?
See https://en.wikipedia.org/wiki/Sybil_attack
See https://en.wikipedia.org/wiki/Sybil_attack
I assume it's a reference to the ancient Greek prophetesses[1].
[1]: https://en.wikipedia.org/wiki/Sibyl
[1]: https://en.wikipedia.org/wiki/Sibyl
I think it's a coincidence. Sybil is reference to the book / movie about multiple personality disorder. Both deal with multiple personality systems.
I would be interested in knowing how much improvement they saw by using C++ or Kotlin. Also, I don’t really understand what compute service is actually used to run the model predictions in this framework.