Launch HN: Mirror (YC W17) – turn a selfie into emoji that look like you
80 comments
I installed this and pointed it at my six year old son. It came back with a big image of an angry girl shouting "SHUT THE FUCK UP!"
Not really what I was expecting. Glad I wasn't showing him the screen at the time, at least. But you might want to fix that.
Not really what I was expecting. Glad I wasn't showing him the screen at the time, at least. But you might want to fix that.
ugh. thanks for telling about this. we don't detect the gender of kids too well, but we do detect age and should just turn off the more explicit stickers.
thank you, very useful. and sorry you had that experience.
thank you, very useful. and sorry you had that experience.
Well that was a bit of a waste of time - the stickers (they’re not emoji, they’re stickers) look nothing like me - got the color of my hair totally wrong, the shape of the face is nothing like mine, and the facial hair is completely wrong.
To top all of that off, I was immediately presented with profanity thats completely out of line with the store rating.
Nice try, but bitmoji was more fun and way more accurate.
To top all of that off, I was immediately presented with profanity thats completely out of line with the store rating.
Nice try, but bitmoji was more fun and way more accurate.
thanks for your feedback – we know it doesn't work for some people and are working to make it better. we'll get there :)
regarding profanity – thinking how to both let people who want to use it use it (as many do) while not annoying anyone with it.
thanks for trying the product, sorry it didn't work for you
regarding profanity – thinking how to both let people who want to use it use it (as many do) while not annoying anyone with it.
thanks for trying the product, sorry it didn't work for you
Um, yeah the profanity was completely unexpected. I happen to be one of those weird people that basically doesn't swear and presenting "me" with a big ole (non abbreviated) STFU word bubble feels bad. Uninstalled.
If you're going to have them, put them in a "you gotta look for them corner" of the app, not front and center.
If you're going to have them, put them in a "you gotta look for them corner" of the app, not front and center.
Suggestion: default it to off, make turning it on accessible, pop up the terms of service that it’s a TOS violation for minors to use it when it is turned on (or if they told you they’re under 18, don’t make it accessible) and tell anyone who complains about it, “tough noogies” (or their kid is a TOS violator) and don’t engage any further. Keep the ban hammer at the ready.
And thanks for making an R-rated option. I will use it.
And thanks for making an R-rated option. I will use it.
I tried it, and I know there is a lot of work behind this, but you asked for an opinion and here it is: I look nothing like the emojis, I could not identify myself with them.
Seconded. It got my hair color wrong (I have light mousey brown hair, with a sprinkle of gray, the app creates an avatar with dark brown hair), and it got my eye color wrong (it chose brown over my blue)
While the idea is admirable, there is very little scope for error in a space as personal as thjs
While the idea is admirable, there is very little scope for error in a space as personal as thjs
agreed re scope of error – this is the key challenge given the complexity of face perception.
this is a beta, we will improve quickly.
this is a beta, we will improve quickly.
yeah we get it – it gets good results for some people, but by no means all. thanks for the feedback, we will make it better!
I don't get it. The technology looks super interesting but the end result maybe is more targeted at teenagers?
> AI needs data that informs it whether its decisions are right
Well, aren't the emojis all out there for you to analyze? Isn't there a lot of videos with real faces for you to analyze?
> VR characters that look like you in the gameworld
XBox does this today, doesn't it?
> personalized advertising that has your face in it the way you like
I couldn't care less about advertising. Maybe you're pitching to investors?
> Everyone asks us why after starting a big “serious" company, we decided to work with emoji, and this is the reason.
What's the reason? It's not clear.
As an user of this, it looks like a gimmick (like Tom The Talking Cat).
> AI needs data that informs it whether its decisions are right
Well, aren't the emojis all out there for you to analyze? Isn't there a lot of videos with real faces for you to analyze?
> VR characters that look like you in the gameworld
XBox does this today, doesn't it?
> personalized advertising that has your face in it the way you like
I couldn't care less about advertising. Maybe you're pitching to investors?
> Everyone asks us why after starting a big “serious" company, we decided to work with emoji, and this is the reason.
What's the reason? It's not clear.
As an user of this, it looks like a gimmick (like Tom The Talking Cat).
hey! let me illustrate with an (oversimplified) example:
– we tweak eyes to be larger-than-real and observe the reaction by letting users pick from several options or just observing usage frequency. that way we find the optimal point and then slice it down by age, sex, ethnic etc. groups; as well as to personal preference. - do that with a large amount of data + genetic algos that do much less-obvious things than eye size. - when a user wants a World of Warcraft orc avatar, we can use the data to make the orc look both like an orc and like he wants to see himself.
you can't do this with generic emoji – you actually have to keep tweaking and seeing user reactions to understand how people truly want to see themselves and others. and then you can use this to other contexts.
thanks for the feedback, i agree this is not explained clearly enough in the post.
– we tweak eyes to be larger-than-real and observe the reaction by letting users pick from several options or just observing usage frequency. that way we find the optimal point and then slice it down by age, sex, ethnic etc. groups; as well as to personal preference. - do that with a large amount of data + genetic algos that do much less-obvious things than eye size. - when a user wants a World of Warcraft orc avatar, we can use the data to make the orc look both like an orc and like he wants to see himself.
you can't do this with generic emoji – you actually have to keep tweaking and seeing user reactions to understand how people truly want to see themselves and others. and then you can use this to other contexts.
thanks for the feedback, i agree this is not explained clearly enough in the post.
Thanks for explaining it in more detail. This is very interesting (and if it doesn't pan out, you got the hottest skills in the market right now). All the best.
thank you! really appreciated :)
Meh.
Your payoff says I could turn a selfie into an emoji that looks like me, but I got stickers instead of emoji, and they looked nothing like me.
Your payoff says I could turn a selfie into an emoji that looks like me, but I got stickers instead of emoji, and they looked nothing like me.
we'll make it better. matter of time + data. right now it works for some people, but not for others – just tough to get really good results because of how complex human facial perception is.
thanks for your feedback!
thanks for your feedback!
Just tried it out, only one attempt and got good results!
Couple of ideas, you may already have considered:
1) Use a color-correction algo to help with tinted hair. Then use something like MASK-RCNN to get pixels of hair and get dominate cluster of color. 2) If lighting/shadows/environment causes so much trouble maybe have user rotate/tilt their head in circle-motion (like Apple has users do to train FaceID) then these run ~30 images through your algos and average the results? Maybe have torch on during capture to make the data more homogenous. (Or do once with torch, once without and train two different networks for both types of captures.) 3) If user manually overrides hair color show them your algos next 5 best predictions as other color choices so you can more easily retrain if user selects one of these other predictions. Maybe have them manually adjust bounding polygon of hair detection and even adjust image color correction sliders to show "what my hair actually looks like" (so you can train color correction).
Just some ideas. Cool product! Good luck!
Couple of ideas, you may already have considered:
1) Use a color-correction algo to help with tinted hair. Then use something like MASK-RCNN to get pixels of hair and get dominate cluster of color. 2) If lighting/shadows/environment causes so much trouble maybe have user rotate/tilt their head in circle-motion (like Apple has users do to train FaceID) then these run ~30 images through your algos and average the results? Maybe have torch on during capture to make the data more homogenous. (Or do once with torch, once without and train two different networks for both types of captures.) 3) If user manually overrides hair color show them your algos next 5 best predictions as other color choices so you can more easily retrain if user selects one of these other predictions. Maybe have them manually adjust bounding polygon of hair detection and even adjust image color correction sliders to show "what my hair actually looks like" (so you can train color correction).
Just some ideas. Cool product! Good luck!
thank you! passing to the ML team!
This app sounds really cool!
I installed it but wasn't too impressed with the quality of the emoji. (This could be because I look kind of weird; eg I have pink hair.) It's really easy to take photos and see how the emoji look; I encourage other people to try that out.
I'm excited to see whether this technology ends up working well in future.
I installed it but wasn't too impressed with the quality of the emoji. (This could be because I look kind of weird; eg I have pink hair.) It's really easy to take photos and see how the emoji look; I encourage other people to try that out.
I'm excited to see whether this technology ends up working well in future.
hey!
yea we don't have pink hair – basically it was really hard to get the color palettes working right in all kinds of different lighting conditions, so we didn't include rare/custom hair.
we'll get better at it :) thanks for using the product!!
yea we don't have pink hair – basically it was really hard to get the color palettes working right in all kinds of different lighting conditions, so we didn't include rare/custom hair.
we'll get better at it :) thanks for using the product!!
I have a nice lumberjack-kind-of-beard and my avatar showed up with a designer chinstrap and glossy lips...
yeah – it's definitely not perfect. we are quickly making it better. in particular beards are very limited because there are no good existing datasets/trained networks for them, so we have to do everything ourselves.
thanks for mentioning, we'll improve!
thanks for mentioning, we'll improve!
This appears to be a keyboard? What other data does your app have access to in that regard? Do you get to read the contents of SMS or email or anything else with that? Also, what data or images of mine are stored on your servers?
we don't log your messages – in theory keyboard developers could do this, but Apple would wipe the floor with us if we did. we do plan to have a bag-of-words RNN that runs locally on the device to suggest emoji to you depending on context of your SMS etc without sending anything to a server.
we do send your images to our servers at this point, and we do store them + standard usage data about how you interact with app flows (conversions, retention etc).
in the future we'll try to have ML on the device and not even send your images to the cloud. both better for privacy and saves us money.
thanks for bringing this up. i have zero sense of privacy for myself so it helps when others point out how they feel about this.
we do send your images to our servers at this point, and we do store them + standard usage data about how you interact with app flows (conversions, retention etc).
in the future we'll try to have ML on the device and not even send your images to the cloud. both better for privacy and saves us money.
thanks for bringing this up. i have zero sense of privacy for myself so it helps when others point out how they feel about this.
Thanks for the detailed response. In the interest of total transparency, I have to say I'd be a little concerned with installing a keyboard app that sends photos of my face to servers in Russia, particularly with everything going on right now.
I'm not sure what could be done to alleviate that concern, and perhaps I'm being paranoid, but for now, that has stopped me from using the app. Obviously a 10 year old wouldn't have that concern, and perhaps that is a different problem.
I hope you understand--nothing personal.
I'm not sure what could be done to alleviate that concern, and perhaps I'm being paranoid, but for now, that has stopped me from using the app. Obviously a 10 year old wouldn't have that concern, and perhaps that is a different problem.
I hope you understand--nothing personal.
they aren't in Russia, they are in AWS :)
if we kept it in Russia latency would suck + we'd have to comply with local regulations we don't want to deal with.
if we kept it in Russia latency would suck + we'd have to comply with local regulations we don't want to deal with.
oh and just to be clear – the reason it is a keyboard is that people are really used to having emoji in their keyboard. this is how bitmoji and other manual-custom-emoji products work too.
Any possibility of integrating as iOS iMessage Stickers to avoid using a custom keyboard?
yes – there is an iMessage app included! can you try setting it up and let us know about the experience? i found iMessage to be confusing from time to time so want to make sure nothing wrong there...
Cool app! read my facial shape, eyes, hair color, and style from an indoor selfie. Wish I could have just uploaded a photo to start. People might have a better photo of themselves in their photo stream.
we initially let people do that, but they started uploading much worse photos, group photos etc. – the selfies help consistency. glad you liked it :)
Just wanted to say, this is a fantastic app in my opinion. I took one photo and it was already more accurate & personally identifiable to some friends as a 'better' Bitmoji.
I'm impressed
I'm impressed
really glad to hear!!! hoping we can get everyone to have this experience :)
thanks for giving us a try! if at some point you have other feedback pls let us know.
thanks for giving us a try! if at some point you have other feedback pls let us know.
Is the app only available in the States? I'm in the UK and when I click the download link on my iPhone I get this: https://imgur.com/zHytq4u (url: https://itunes.apple.com/us/app/mirror-emoji-keyboard/id1199...)
Maybe clarify that, or open it up to the rest of us :)
Maybe clarify that, or open it up to the rest of us :)
Could you please try to open this link in Safari?
I guess Chrome failed to redirect to App Store.
Safari on iOS works correctly and opens the App Store.
thanks for finding this, we'll fix it!
whoa thank you for catching this. will look into right away.
Excuse my (potential) ignornace, but isn’t this just Bitmoji (which has huge market share), just with slightly reduced setup from face scanning?
not quite. let me point several things out:
- bitmoji takes a long time and a lot of mental effort to set up. i have no idea what my face shape is and had to ask my ex-gf whether my eyes are green or brown.
- with bitmoji you cannot add other people to your emoji because even if you DO go through mental effort and time to create yourself, you won't do a bunch of other people. and often you really want this – e.g. i want to send an image of YOU as Captain Obvious if the situation calls for it, not of ME.
- over time this will get better and better in terms of quality, while Bitmoji will not.
- bitmoji has no market share in any other language than English, and little market share in non-snapchat messengers.
- bitmoji takes a long time and a lot of mental effort to set up. i have no idea what my face shape is and had to ask my ex-gf whether my eyes are green or brown.
- with bitmoji you cannot add other people to your emoji because even if you DO go through mental effort and time to create yourself, you won't do a bunch of other people. and often you really want this – e.g. i want to send an image of YOU as Captain Obvious if the situation calls for it, not of ME.
- over time this will get better and better in terms of quality, while Bitmoji will not.
- bitmoji has no market share in any other language than English, and little market share in non-snapchat messengers.
My experience trying it just now with my girlfriend: I think we both like bitmoji better. With Mirror my girlfriend took her selfie probably 5 or 6 times to get one she was happy with, and had to try her hair in different styles to see what she liked. She also asked if there was more customization options like bitmoji. I don't think people really mind the setup time of bitmoji and possibly actually like playing with it.
Anyway, it's a cool app. I'm not trying to be a downer, just wanted to give you honest feedback.
Anyway, it's a cool app. I'm not trying to be a downer, just wanted to give you honest feedback.
thanks for sharing! i wonder if there are just different demographics here where some people want to customize while others don't.
i remember that at our hotel booking company, only 10-15% of people used any filters/sorts for hotels, and >80% just picked from the top list of what we gave them. to me this is shocking given a hotel is a big and rare purchase. but it makes me feel the majority of the world want the fewest taps and the fewest mental barriers.
plus we kind of feel like we want to force ourselves to do a good job out of automatic creation, and a lot of manual customization is defocusing.
i remember that at our hotel booking company, only 10-15% of people used any filters/sorts for hotels, and >80% just picked from the top list of what we gave them. to me this is shocking given a hotel is a big and rare purchase. but it makes me feel the majority of the world want the fewest taps and the fewest mental barriers.
plus we kind of feel like we want to force ourselves to do a good job out of automatic creation, and a lot of manual customization is defocusing.
If character creation screens in games has taught me anything, it's that people love taking time to customise things like this. I don't think you should try to make your competitive advantage speed.
i don't think it is merely speed though, it is simplicity too. i hear you about character creation screens, but there are hardcore gamers and there are casual gamers. the latter are far more prevalent. and i say this as someone who loves my Divinity 2 and Baldur's Gate :)
(also see comment below about hotels, same discussion)
what do you think?
(also see comment below about hotels, same discussion)
what do you think?
Cool idea! I'd allow more options to customize - I didn't feel that the emoji accurately looked like me and wouldn't feel comfortable sending this to people. However some slight tweaks (manually changing the hair color, beard style, etc) would go a long way to giving me a sense of ownership over the emoji.
thanks for the feedback! yep, we are planning to add exactly the tweaks you mentioned. plus eye color.
will do this soon :)
will do this soon :)
As a male with long hair past my shoulders, there seem to be no such hair option when creating the profile
hmm, obvious mistake. will fix.
Tried the app on my Nexus. It didn't work, at all. When I open the app and try to take a selfie, the image is completely blurred out and it can't detect my face. (The built-in Camera app works just fine, though.)
huh. this definitely shouldn't be happening. does it replicate every time? any chance you can try a couple times and post the exact time you are doing this? will try to hunt down where it went wrong.
thanks for testing
thanks for testing
This is great. I had an idea similar to this, with the added concept of being able to generate animated emojis using video and either keeping the audio of the user or generating synthetic audio from text.
yeah we decided against that because it would interfere with sharing frequency – people are much more willing to share images than voice/video. plus animation makes content volumes much harder to scale.
I wonder if there's any potential for this to be used for 3D avatars in VR. I imagine it would be fairly easy to do. Not much of a market yet, but perhaps in the future.
definitely. we think doing this with emoji first is much better than going after VR first. this is where the market (and the data) is today. and that will help build the applications of tomorrow like VR.
My vanity made me click on this. My curiosity about using AI with human interaction made me install the app.
Very interesting problem set. I look forward to seeing what you do with it.
Very interesting problem set. I look forward to seeing what you do with it.
vanity... definitely my favorite sin :)
it does tap into a lot of human emotions. one of the reasons we decided to pursue this area.
it does tap into a lot of human emotions. one of the reasons we decided to pursue this area.
I just installed and tried it:
- very easy to install and use - took a pic (I am not that good looking) and my emoji looks great! - I sent it to a couple of friends who liked it
- very easy to install and use - took a pic (I am not that good looking) and my emoji looks great! - I sent it to a couple of friends who liked it
awesome :) really happy you liked it!! thanks a lot for the feedback!
I went through the trouble of actually logging into my Hacker News account just to say that the generated version of me looks exactly like me. O_o
awesome :) really happy to hear this!!
huh what does 'emoji' mean in this context -- are you using some combination of Unicode characters?
no. a more accurate description is definitely "stickers." but everyone knows and searches "emoji" much more. this is more of a marketing naming decision, i agree that it isn't accurate.
Ah gotcha. So is the idea to signal to future partner companies that the product is like a 3rd party version of "Animoji", using cool pictures to collect a big corpus of face feature data for future products?
if you mean about the word "emoji" it is more to pick up traffic from people searching for emoji keyboards and the like.
if you mean in general, right now we are focused on building a great, highly engaging product that people enjoy. and once that is there and at large scale, we'll start thinking about other applications.
impossible to do anything in life well without obsessive focus...
if you mean in general, right now we are focused on building a great, highly engaging product that people enjoy. and once that is there and at large scale, we'll start thinking about other applications.
impossible to do anything in life well without obsessive focus...
My emoji turned out fine, but full access to keyboard data is a non-starter for me. Sorry.
well you can use it without the keyboard, or with an iMessage app if you are on iOS.
unfortunately full access is the only way to make image pasting work. Bitmoji and all other apps that utilize this mechanic do the same thing.
For what it's worth, Apple would brutalize us if we were offloading customer keystrokes to our servers or something like that.
unfortunately full access is the only way to make image pasting work. Bitmoji and all other apps that utilize this mechanic do the same thing.
For what it's worth, Apple would brutalize us if we were offloading customer keystrokes to our servers or something like that.
What's the business model?
in the short-term, people are very willing to pay for gift/status-boosting content related to faces and identity. plus stickers. LINE is a $10bn public company basically off of selling stickers. this has very high margins combined with viral nature of product that has low distribution cost. so good economics.
longer-term it is about the data and using it for other technologies.
longer-term it is about the data and using it for other technologies.
Thanks for the insights. I've never bought a sticker, which is why I guess why I wasn't sure what the play is.
> longer-term it is about the data and using it for other technologies.
I'm not sure what this means. Better adtech? Or is there something else unique about this data that I'm not seeing?
> longer-term it is about the data and using it for other technologies.
I'm not sure what this means. Better adtech? Or is there something else unique about this data that I'm not seeing?
AI that knows how you want to be seen => VR avatar and personalized advertising applications. lots of other ideas, but no sense in thinking too much about them so early.
Thanks for your reply. Curious to see how the business develops. Good luck!
Be bought by another company ;)
we don't want that. we think there's a big business model/opportunity related to being the best in the world at understanding how humans perceive their own faces and the faces of others.
also i'm very fortunate in that i already have one company that was bought, plus another that will hopefully go public. so want to build something bigger. plus in Silicon Valley nobody truly respects you if your only goal is to be bought by another company.
also i'm very fortunate in that i already have one company that was bought, plus another that will hopefully go public. so want to build something bigger. plus in Silicon Valley nobody truly respects you if your only goal is to be bought by another company.
Looks like something facebook would buy. Do you consider them as competition ?
Facebook are clearly a very smart and capable team that is very interested in this exact area. So yes, they are competition.
But it is a classic case of that for a large company it is hard to execute on many priorities at once and dedicate a lot of organizational resources and attention. Plus they seem to have bet on going directly to VR.
We are laser-focused on doing this particular usecase extremely well, have a large/growing/strong engineering team, and have business leaders that don't have any other priorities beyond making this product excellent.
But it is a classic case of that for a large company it is hard to execute on many priorities at once and dedicate a lot of organizational resources and attention. Plus they seem to have bet on going directly to VR.
We are laser-focused on doing this particular usecase extremely well, have a large/growing/strong engineering team, and have business leaders that don't have any other priorities beyond making this product excellent.
We are launching an app (Mirror) that uses a single selfie to create ~250 emoji/stickers that look like you and your friends.
We previously started a large online travel company in Russia (Ostrovok, >$500m in sales this year) and while there got very interested in AI as a tool to solve business objectives like predicting cancelations or conversion rates to inform marketing spend. And when we started something new, we wanted a product that had extremely high-frequency usage to build feedback loops into + something that a lot of humans really want to do, but that AI can do better.
Emoji keyboards sound like trivial toys, but this is actually a path to teaching AI one of the most complex human social behaviors. Offline, our faces communicate the majority of information that passes between us; far more than words do. In fact there is a whole piece of our brain (fusiform gyrus) dedicated to facial analysis and if lesioned humans lose ability to read faces while losing nothing else. Basically humans pay closer attention to faces than to any other object in the universe – from the day we are born.
AI needs data that informs it whether its decisions are right. And emoji are by far the largest structured source of facial data + usage in the world – >15 billion sent a day. Getting that ocean of data will help train AI that can solve all kinds of future problems – from VR characters that look like you in the gameworld, to personalized advertising that has your face in it the way you like, and a lot of other things. Everyone asks us why after starting a big “serious" company, we decided to work with emoji, and this is the reason.
On to the actual product. We think custom emoji apps like Bitmoji are a good step towards bringing our real, unique faces into messaging. We take this further and let you:
- Get your emoji in 3 seconds with one tap.
- Not spend mental energy wondering what your head shape/nose shape etc. are
- Add your friends/family (soon your pets and more) to your own emoji in seconds. Manually constructing 10+ friends is just too hard.
- Use all text-containing emoji in 8 languages (will expand to >25 shortly). We really have no idea why Bitmoji and other such apps tend to be English-only.
- Get emoji that become more and more recognizable/liked with time by learning from data (explicit choices + implicit metrics). For example we see that people want larger-than proportional eyes and darker skin. Except in Japan where it seems users want lighter skin. Really excited to find more like this with usage data.
Google Allo (and some others) tried doing this but got very little done. The reason is that while the idea is extremely obvious, the tech is quite hard. Here's why:
- You have to get a lot of facial features right. Some of these (e.g. hair, facial hair) are very diverse across age/sex/ethnicity. Some (e.g. head shape) are 3d but need to be inferred from shadows in a 2d image. Some (e.g. lower eyelids or wrinkles) can be detected but have a tradeoff of authenticity vs beauty. Some (colors of skin, hair, lips, eyes, eyebrows) are highly dependent on lighting conditions. All are must-haves. And this is still not enough – you have to detect and place wrinkles, birthmarks, sunspots etc. too
- You have to compensate for a lot of variability in selfie-taking – lighting/shadows, turn/pitch/tilt, camera distance, device camera type, background, whether the user is smiling or not etc.
- You have to use one photo to infer and reconstruct how that person looks with different emotions.
- There’s a whole dedicated part of the brain for this task, with a lot of neural machinery (fusiform gyrus). Humans can’t formally explain how they recognize other humans, but are very demanding to result quality. Humans take our exceptional skill at face recognition for granted, although for us a face is THE most well-perceived object in the universe. Think about it: can you instantly recognize a piece of a random wall that looks similar to every other wall, and that you stared at for an hour 10 years ago? You can do this with a face.
All the above makes training neural nets that solve recreating (rather than simply recognizing) faces much harder.
It took us close to a year and a large engineering team + multiple people with many years of experience with faces to get this to beta.
We know there is still a long way to go to make the product great. Right now we are working on several things to make it better:
- improving the tech from just “personalized” to “recognizable by friends” – both by utilizing feedback loops and by looking at new developments in AI. Particularly excited about GANs although we don't see a pure-GAN approach as viable (longer discussion).
- adding new features – e.g. auto-detection of groups of people, pets, clothes, backgrounds. doing a lot of this automatically based on your Photo Stream content and on-device ML technologies.
- expanding our content library to ~10,000 emoji in 25 languages.
You can get the app here: https://www.mirror-ai.com/
Let us know what you think – we really appreciate all your advice!