So I reverse engineered two dating apps(push32.com)
push32.com
So I reverse engineered two dating apps
https://push32.com/post/dating-app-fail/
149 comments
Probably this? https://themoth.org/stories/data-mining-for-dates
And here's an article https://www.wired.com/2014/01/how-to-hack-okcupid/
And here's an article https://www.wired.com/2014/01/how-to-hack-okcupid/
Not to throw this into standard male internet complaining about dating, but it's funny that the Wired article waits until almost the end to mention that the dude is over six feet tall with blue eyes (and per the photo at the beginning, not bad-looking!). I wonder if this kind of gaming would have worked if he wasn't already in a highly desired male demographic. It's possible that just being noticed is a big marginal benefit when you're already in that demographic.
[deleted]
It starts with his photo...
Of course it wouldn't. Why would women suddenly find a man attractive just because some app brings up a match? The matching algorithms are not there to show you who you find attractive, they are there to filter out the 95% (for women), or 50% (for men) of people you definitely don't find attractive.
If 95% of women reject your profile, would you get more matches if your profile was in front of 10 women or 10,000?
Generally it’s not 95% of women rejecting a given profile, but all women (or near enough to make no difference) rejecting mostly the same 95% of profiles and not rejecting the rest.
I.e. boosting visibility probably brings proportionate benefits to the most desirable men, but little additional benefit to the ugly/short/older guy.
I.e. boosting visibility probably brings proportionate benefits to the most desirable men, but little additional benefit to the ugly/short/older guy.
Source? With 51.5% of American men presently married I have a hard time believing 95% of men are consistently rejected by every woman!
It's probably a misremembering of actual findings:
At least on OKCupid, women rate 80% of men as below-average attractiveness, while men rate women at right about 50% as below-average and 50% as above-average [0].
OKCupid has since deleted the post with their findings [1], but references to it still exist all over, as does an archived copy [2].
[0] https://techcrunch.com/2009/11/18/okcupid-inbox-attractive/
[1] https://theblog.okcupid.com/your-looks-and-your-inbox-8715c0...
[2] http://web.archive.org/web/20100324074028/http://blog.okcupi...
At least on OKCupid, women rate 80% of men as below-average attractiveness, while men rate women at right about 50% as below-average and 50% as above-average [0].
OKCupid has since deleted the post with their findings [1], but references to it still exist all over, as does an archived copy [2].
[0] https://techcrunch.com/2009/11/18/okcupid-inbox-attractive/
[1] https://theblog.okcupid.com/your-looks-and-your-inbox-8715c0...
[2] http://web.archive.org/web/20100324074028/http://blog.okcupi...
Their blog was a goldmine but decommissioned when they were bought by match.com
It sucks that they deleted it. It's so hard to know the truth in this area because the truth offends people.
Maybe men just answer the question more literal and find the real average (50%). Women understand the question more figuratively. Below average is what they don't like and they don't like 80% of men.
Why even ask for above/below average ratings? What people like/don'tlike is more interesting.
Why even ask for above/below average ratings? What people like/don'tlike is more interesting.
Maybe he most attractive men in the population aren't on dating apps.
It’s 95% of the men on dating apps, not 95% of all men. The men who get married presumably delete their profile. It’s not hard to see that all of the gold had already been panned out and there’s nothing left but mud (in the eyes of the women on those dating apps).
That sounds like how hiring managers describe the tech job market.
Sellers' markets all have this dynamic.
Edit: wtf @ stripping emoji HN
Edit: wtf @ stripping emoji HN
[deleted]
The trouble is it's not "95% of women reject your profile" but rather "women reject 95% of profiles" -- which is something else entirely.
If someone had good experiences with people the app recommended previously, then the app recommending someone can be enough to make them willing to date them.
It's basically the same mechanism that works for physical attraction, i.e. if you had good interactions with people of a certain "type" then you'll be attracted to other people of that "type".
It's basically the same mechanism that works for physical attraction, i.e. if you had good interactions with people of a certain "type" then you'll be attracted to other people of that "type".
Are those percentages based on data you got somewhere, or are they a part of the 84% of statistics that are made up on the spot?
It is probably based off the OkCupid data that found that women rated 80% of men as below average attractiveness or the Tinder data which has the top 50% of women chasing 5-10% of the men.
https://medium.com/@worstonlinedater/tinder-experiments-ii-g...
https://qz.com/1051462/these-statistics-show-why-its-so-hard...
http://archive.is/BtTP2
I can't find the actual statistic the above people are repeating, but these are similar.
https://medium.com/@worstonlinedater/tinder-experiments-ii-g...
https://qz.com/1051462/these-statistics-show-why-its-so-hard...
http://archive.is/BtTP2
I can't find the actual statistic the above people are repeating, but these are similar.
Not sure why folks are downvoting me... perhaps giving the humorous out is rubbing folks the wrong way.
I looked at the links and didn’t see mention of those values, and I don’t know if they are commonly known, so I’m going to ask again.
Where are these numbers (95% of women and 50% of men) coming from?
I looked at the links and didn’t see mention of those values, and I don’t know if they are commonly known, so I’m going to ask again.
Where are these numbers (95% of women and 50% of men) coming from?
The OKCupid and Tinder studies, as linked elsewhere. The OKCupid study showed women rate 80% of men as "below average". But if you assume that most women don't want an average or even just slightly above average man, it easily becomes more than 90%.
This was fascinating all around - both for the means by which he optimized OKCupid for himself, and for the the insights about women that it yielded. He was able to type women in to 7 clusters, but that 7th cluster ('diverse') could have been optimized into further groups, it seems.
[deleted]
> Probably this? https://themoth.org/stories/data-mining-for-dates
That's it, I was really hoping to hear this in its entirety but could never find it: thanks!
That's it, I was really hoping to hear this in its entirety but could never find it: thanks!
I did this on OkCupid just by answering common questions in the most common way. But more dates isn't really the point, at least not for me. So I weighted the questions in the way they mattered to me and I met my current gf. Cheaters only cheat themselves.
Yup, met my wife this way too (OkCupid as well). I in general was very upfront. I didn't try to sell myself much, which meant my flaws in interaction were present too. I didn't like the idea of wasting time with people I'd slowly push away as I let my crafted guard down.
[deleted]
Yes! The thing was that he only got 3 second dates and 1 third date. In the end, he gave up and found love the way the algos intended; she messaged him. The conclusion I took was that the math worked. His gaming the systems was pointless, he needed patience more than anything.
Leaking data is one of the issues with all those CRUD framework tutorials.
It is nice marketing: look how easy it is to output entities with our framework!
The problem is: you almost never want to output complete entities. You want to output data based on the user's role/rights and on the context. I wish frameworks would put more focus on this.
It is nice marketing: look how easy it is to output entities with our framework!
The problem is: you almost never want to output complete entities. You want to output data based on the user's role/rights and on the context. I wish frameworks would put more focus on this.
This is a major problem with REST / API / SPA-based applications. When you build around the concept of resources, you either get lazy or forget to build individual endpoints for each object.
That is, it's not just "/api/people/" that shows everything. You'd need: "/api/people/matched/" with a few fields, and /api/people/before-match/" etc. One end point per use case. Almost like one endpoint per... screen.
Suddenly, the old school MVC starts to make more sense. Your view dictates what data you need only for that view. Across resources. And it factors in entitlements. Often, people think this means no AJAX or SPA. That's not quite right, they are orthogonal.
What you really want is a data-on-the-wire protocol but with RPC calls that are still domain based rather than resource based. Everything should be its own endpoint. Django Forms and Formsets, and the Rails and Laravel equivalents are much closer to this than REST (even if they can be RESTful themselves). Think "UpdateProfile" instead of "UserResource". The UpdateProfile may have a lot more fields, some which don't even belong on the User database table.
Lot of rambling here, but I've been meaning to consolidate this in a blog post.
Some related thoughts here: https://news.ycombinator.com/item?id=21875331
That is, it's not just "/api/people/" that shows everything. You'd need: "/api/people/matched/" with a few fields, and /api/people/before-match/" etc. One end point per use case. Almost like one endpoint per... screen.
Suddenly, the old school MVC starts to make more sense. Your view dictates what data you need only for that view. Across resources. And it factors in entitlements. Often, people think this means no AJAX or SPA. That's not quite right, they are orthogonal.
What you really want is a data-on-the-wire protocol but with RPC calls that are still domain based rather than resource based. Everything should be its own endpoint. Django Forms and Formsets, and the Rails and Laravel equivalents are much closer to this than REST (even if they can be RESTful themselves). Think "UpdateProfile" instead of "UserResource". The UpdateProfile may have a lot more fields, some which don't even belong on the User database table.
Lot of rambling here, but I've been meaning to consolidate this in a blog post.
Some related thoughts here: https://news.ycombinator.com/item?id=21875331
Actually, there is a simple and good way to think about this. We often espouse the "onion model of applications" describing the responsibilities of different layers of applications.
Just apply that here - some API endpoints should be backend "system APIs" offering as much access to entities as possible - think "data layer APIs".
When you want to expose data to consumers for use in a UI, these are "experience APIs". These are APIs that might be built as part of the backend that directly interacts with the frontend.
Mulesoft talks about this a lot [1]. I am a user of it at work, not necessarily a big fan, but I think their architecture idea is applicable anywhere and helpful for thinking about layers of APIs in a large system. It's similar to the onion model, just applied to APIs.
Now, if you are building a smaller system this might all seem overkill. For a small system it might be true, unless you believe it will scale and need to be architected like this from the beginning. I think scale of the end result should always be factored into architecture decisions, because not one size fits all.
I think personally there is a model out there that hasn't been expressed fully/fully realized, that integrates ideas of APIs, GraphQL, CQRS, microservices and the onion model.
[1] https://blogs.mulesoft.com/dev/api-dev/what-is-api-led-conne...
Just apply that here - some API endpoints should be backend "system APIs" offering as much access to entities as possible - think "data layer APIs".
When you want to expose data to consumers for use in a UI, these are "experience APIs". These are APIs that might be built as part of the backend that directly interacts with the frontend.
Mulesoft talks about this a lot [1]. I am a user of it at work, not necessarily a big fan, but I think their architecture idea is applicable anywhere and helpful for thinking about layers of APIs in a large system. It's similar to the onion model, just applied to APIs.
Now, if you are building a smaller system this might all seem overkill. For a small system it might be true, unless you believe it will scale and need to be architected like this from the beginning. I think scale of the end result should always be factored into architecture decisions, because not one size fits all.
I think personally there is a model out there that hasn't been expressed fully/fully realized, that integrates ideas of APIs, GraphQL, CQRS, microservices and the onion model.
[1] https://blogs.mulesoft.com/dev/api-dev/what-is-api-led-conne...
I'll have to give this a read.
Many of these technologies (GraphQL) seem like layers on top of layers. Really, I just want a page with 3 fields specific to that page. They may come from 3 different tables.
GraphQL's original solution to this was to batch the request to 3 different resources, but the underlying problem was still there. Of course, you can make ad-hoc GraphQL equivalents to a random REST ad-hoc /api/get-info-for-my-profile-page/.
The problem becomes especially noticeable when you want to update nested resources. Say, a profile's subscription status. Instead, in "MVC" world, I would just bump up the subscription status to be a first class field on that view. So no profile->subscription->status. Just "subscription_status". Hope that makes sense.
So far, I've reverted to using Django Forms (across tables) and breaking them down to JSON to use them in a SPA. My endpoints are still very much domain specific, but I can use XHR/fetch to make the page interactive.
Many of these technologies (GraphQL) seem like layers on top of layers. Really, I just want a page with 3 fields specific to that page. They may come from 3 different tables.
GraphQL's original solution to this was to batch the request to 3 different resources, but the underlying problem was still there. Of course, you can make ad-hoc GraphQL equivalents to a random REST ad-hoc /api/get-info-for-my-profile-page/.
The problem becomes especially noticeable when you want to update nested resources. Say, a profile's subscription status. Instead, in "MVC" world, I would just bump up the subscription status to be a first class field on that view. So no profile->subscription->status. Just "subscription_status". Hope that makes sense.
So far, I've reverted to using Django Forms (across tables) and breaking them down to JSON to use them in a SPA. My endpoints are still very much domain specific, but I can use XHR/fetch to make the page interactive.
In REST you return all the data because it's better to have some geek reading about which girls rejected him than to have to create a ticket with the backend team to get them to add it later #tea
I agree with you up to the point of RPC calls. In general it doesn't matter what specific technology is used, as long as each data source is structured around the use-case and not around backend details like how entity models are actually stored. A lot of these ideas are captured well by Robert Martin ("Uncle Bob") in his book Clean Architecture
I guess I just use RPC as a catch all for "server call that could involve resources, actions, effects, etc." So even a REST PUT is an RPC in this world of mine.
I mentioned the Onion Model above, I saw your comment about Clean Architecture, which is a refinement of the Onion Model.
I think we might already have all the architectural solutions we need, we just haven't put together how to express the architectures we want in the new technologies we have.
Some combination of the onion model/clean architecture, CQRS, APIs, data federation like GraphQL and microservices is probably how it can all go together. I know I am mixing implementations and patterns together in this sentence, but am just trying to express patterns and examples of things that have pieces of those patterns implemented.
I think we might already have all the architectural solutions we need, we just haven't put together how to express the architectures we want in the new technologies we have.
Some combination of the onion model/clean architecture, CQRS, APIs, data federation like GraphQL and microservices is probably how it can all go together. I know I am mixing implementations and patterns together in this sentence, but am just trying to express patterns and examples of things that have pieces of those patterns implemented.
> The majority of the testing is done inside a rooted Android emulator running Android 8 Oreo. Tests that require more capabilities are done on a real Android device running Lineage OS 16 (based on Android Pie), rooted with Magisk.
I don't have much experience with such things. Can such an analysis be done with a non-rooted device?
I don't have much experience with such things. Can such an analysis be done with a non-rooted device?
It sounds like they're doing at least some reverse engineering via the Xposed framework, which requires root. However you could probably get similar findings by just sniffing your own network traffic.
Unfortunately with modern iOS/Android you need a rooted device.
Modern iOS/Android have something called SSL Pinning. Which means if you want to use a self-signed cert to Man in the Middle the HTTPS traffic it won't work. So you need to patch the network stack to allow this. I have a spare iPhone I keep rooted for this reason.
Modern iOS/Android have something called SSL Pinning. Which means if you want to use a self-signed cert to Man in the Middle the HTTPS traffic it won't work. So you need to patch the network stack to allow this. I have a spare iPhone I keep rooted for this reason.
You can actually get by without root by using Frida (https://frida.re/docs/android/), at least on Android. By repackaging the APK to include frida-gadget (such as in this tutorial: https://koz.io/using-frida-on-android-without-root/), you can get all of Frida's capabilities on that app without root.
That said, it is more convenient to have a spare rooted phone.
That said, it is more convenient to have a spare rooted phone.
Do all apps use cert pinning? I think it got popular some years ago but I remember some downsides existing
(I know things like Google Apps and such use it, but I'm not sure about less popular ones)
(I know things like Google Apps and such use it, but I'm not sure about less popular ones)
Android apps don't trust user CA certificates by default (they only work with web browsers.) You have to explicitly enable this option in app manifest.
Rooting the device enables you to put custom certificates in system store, and bypass this check.
Rooting the device enables you to put custom certificates in system store, and bypass this check.
I've done a few and always ran into cert pinning issues.
IMO it's fortunately tho!
> Modern iOS/Android have something called SSL Pinning.
Yep, the mobile OSes have that option, but it’s not very commonly used. I fairly regularly snoop traffic on apps and it’s not super often that I’m blocked temporarily by cert pinning.
Yep, the mobile OSes have that option, but it’s not very commonly used. I fairly regularly snoop traffic on apps and it’s not super often that I’m blocked temporarily by cert pinning.
Interesting. The last 6 apps I've reverse engineered they all had pinning on.
Even I wouldn't recommend generating your bearer tokens client side if they are actually doing it, but collision thing is a bit far fetched. Any randomly generated UUID like UUID4 has a randomness in space of 2^122(ignoring pre determined bits). For practical purposes this is almost unique, you will need to generate ~2^61 ids for 0.5 probability of a collision happening.
I think the issue is that a hacked client could send a bad "UUID", not that properly randomly generated tokens would collide.
What will that accomplish though? The way he described their auth APIs, you will be able to set an auth token generated by you for an account that you anyway have the user credentials for.
What is a bad “UUID”? A wouldn’t the hacked client do things anyway like uploading UID or local credentials to its servers?
It's pretty much industry standard for a distributed system to use uuid to generate statistically-guaranteed unique ids.
How feasible is it to go from doing software development to stuff like this for a living without a decrease in earning potential? I find reverse engineering interesting, but never really know who exactly is paying for stuff like this. Maybe this is more of a side gig?
I've had a couple people trying to convince me to join Noblis for unrelated work. Saw these today as I browsed the listings:
https://jobs-noblis.icims.com/jobs/8246/junior-reverse-engin...
https://jobs-noblis.icims.com/jobs/6640/software-reverse-eng...
https://jobs-noblis.icims.com/jobs/6415/software-reverse-eng...
But in the past I've worked with malware analysts using IDA Pro to reverse engineer stuff. They worked for SAIC. That's another place to try.
https://jobs-noblis.icims.com/jobs/8246/junior-reverse-engin...
https://jobs-noblis.icims.com/jobs/6640/software-reverse-eng...
https://jobs-noblis.icims.com/jobs/6415/software-reverse-eng...
But in the past I've worked with malware analysts using IDA Pro to reverse engineer stuff. They worked for SAIC. That's another place to try.
> Must be able and willing to obtain a Top Secret security clearance.
I've had a confidential security clearance for years because of my job working on government military applications. I remember the process being insanely rigorous. I wonder how much longer it would take to get a Top Secret clearance?
I've had a confidential security clearance for years because of my job working on government military applications. I remember the process being insanely rigorous. I wonder how much longer it would take to get a Top Secret clearance?
In 2017 it took >18 months (I don't know how much greater, because I couldn't handle being paid to do busywork for so long). I would assume it takes longer for people farther along in their career (I was fresh), but would probably be faster because you've held a confidential.
Other anecdata: 60% of my cohort got a clearance in less than 8 months (I believe their sponsoring project paid for expediting?), at least one person was >3 years in queue last I checked.
Other anecdata: 60% of my cohort got a clearance in less than 8 months (I believe their sponsoring project paid for expediting?), at least one person was >3 years in queue last I checked.
Livelihood aside, how the heck do you go from being a dev to becoming proficient in reverse engineering? Are these stuff all self-taught through years of tinkering/interest before it becomes a profession?
I can already imagine the amount of technical barrier and knowledge gap one needs to fill even before getting started..
Holy shiet. It's impressive.
I can already imagine the amount of technical barrier and knowledge gap one needs to fill even before getting started..
Holy shiet. It's impressive.
I do a bit of reverse engineering both professionally and for fun, and the two bits of “proper” education that have helped the most were my Hardware/Software Interface and Intro to Operating Systems classes in undergrad.
Learning how this stuff works in the forward direction makes spotting patterns a whole lot easier. It’s a lot easier to start RE when you’re already familiar with stuff like calling conventions or memory layout (for example).
From there, there isn’t a ton of formal education as far as I’ve seen. I am really fond of Smash the Stack’s IO wargame if you’re interested in CTF-style challenges. I also spent a good bit of time compiling my own small programs and then using them to learn the tools. When you’re starting off, RE is a lot easier when you know what you’re looking for.
Learning how this stuff works in the forward direction makes spotting patterns a whole lot easier. It’s a lot easier to start RE when you’re already familiar with stuff like calling conventions or memory layout (for example).
From there, there isn’t a ton of formal education as far as I’ve seen. I am really fond of Smash the Stack’s IO wargame if you’re interested in CTF-style challenges. I also spent a good bit of time compiling my own small programs and then using them to learn the tools. When you’re starting off, RE is a lot easier when you know what you’re looking for.
I worked with some insanely smart individuals good at RE, and they describe it as a slog where you don't know how much, if any, progress one is making on REing something until you have a breakthrough. And then you don't know how much longer until the nest breakthrough. Ad infinitum. OTOH I know web app vulnerability researchers and they describe their job as going through the OWASP Top 10 and trying to find vulnerabilities. Sounds like the latter requires significantly fewer hard skills and you get more frequent intellectual reward.
If you want a better look at the basics I've found LiveOverflow on YouTube to be entertaining and fairly accessible for myself.
Fundamentally though, you will probably need to build up some low level knowledge of whatever you are targeting (whether it's an app platform, TCP packets, C/ASM code, etc).
If you have a web background there are lots of sites that have fun challenges you can start with. You could try Google's XSS game: https://xss-game.appspot.com/
It's a field that really interests me since I always liked puzzles and games and am happiest at work debugging tricky issues - but I'm not sure how easy it would be to break in as a 'junior' at age 35+.
Fundamentally though, you will probably need to build up some low level knowledge of whatever you are targeting (whether it's an app platform, TCP packets, C/ASM code, etc).
If you have a web background there are lots of sites that have fun challenges you can start with. You could try Google's XSS game: https://xss-game.appspot.com/
It's a field that really interests me since I always liked puzzles and games and am happiest at work debugging tricky issues - but I'm not sure how easy it would be to break in as a 'junior' at age 35+.
You just start. Pick a layer and analyze. Run it through a debugger, analyze the network, etc.
Time in front of the screen is all you need. All code translates to an execution layer.
It helps if you have a few years of dev under your belt. Bonus for low level languages like C or assembly.
Time in front of the screen is all you need. All code translates to an execution layer.
It helps if you have a few years of dev under your belt. Bonus for low level languages like C or assembly.
> how the heck do you go from being a dev to becoming proficient in reverse engineering?
have to do a lot of multi-system level debugging, and/or low-level debugging and optimization
have to do a lot of multi-system level debugging, and/or low-level debugging and optimization
[deleted]
Look for companies that make software that by has to integrate with existing/competing/legacy software systems.
I know that's vague but there are companies that make new products that for one reason or another need to talk to old products and the source, APIs, even binary interfaces are opaque, undocumented, obfuscated or otherwise unaccessible.
Developer tools, CAD/EDA, embedded systems... lots of people are building new stacks that have to talk to older ones at some point.
I know that's vague but there are companies that make new products that for one reason or another need to talk to old products and the source, APIs, even binary interfaces are opaque, undocumented, obfuscated or otherwise unaccessible.
Developer tools, CAD/EDA, embedded systems... lots of people are building new stacks that have to talk to older ones at some point.
Getting into security might be a good starting point. I have recently started with Web-App security and Malware analysis. There is significant reverse engineering involved in these. You may apply to jobs once you're proficient.
Is there really any way to securely host static assets in a cloud bucket? If you host profile images, etc... locally you can easily give or deny access to static assets without having to obfuscate the file path. From what I read here, the dating app was using UUIDs in the URL paths of the images to obfuscate their identity (I’ve done this sort of thing before but afaik it’s not considered secure?). Is it possible to set up ACLs for bucket objects though? From my limited experience with Google Cloud Buckets are object can be simply public or private.
It depends on the data you are storing, using unique UUIDs for something like profile image(avatar?) is good enough, as I commented here the probability of a collision is so low that you can assume them to be not iterable for practical purposes. Just be sure to not enable list APIs for public use in your bucket.
The other solutions you have where you authenticate the path as such, which should ideally be used for more sensitive assets:
- Proxy asset requests through application server, let it authenticate the user access. You can even generate short lived S3 urls this way, and your objects don't have to be public.
- Proxy asset requests through a reverse proxy like Nginx, authenticate there, you have a lot of options here like using something like JWT to authenticate the resource access.
- Proxy asset requests through your reverse proxy CDN like Cloudflare, authenticate there, like in case of Cloudflare JWT is supported.
The other solutions you have where you authenticate the path as such, which should ideally be used for more sensitive assets:
- Proxy asset requests through application server, let it authenticate the user access. You can even generate short lived S3 urls this way, and your objects don't have to be public.
- Proxy asset requests through a reverse proxy like Nginx, authenticate there, you have a lot of options here like using something like JWT to authenticate the resource access.
- Proxy asset requests through your reverse proxy CDN like Cloudflare, authenticate there, like in case of Cloudflare JWT is supported.
Not really in the way you're describing. But S3 supports (and most S3-compatible alternatives as well) signing URLs with the secret key, and with a user-set expiry.
The problem is that it means the requests can't really be cached, so for static assets it's not ideal. S3 supports etags though but I've never touched that so I don't know how well it all works; I personally don't use signed URLs for stuff I want cacheable.
The problem is that it means the requests can't really be cached, so for static assets it's not ideal. S3 supports etags though but I've never touched that so I don't know how well it all works; I personally don't use signed URLs for stuff I want cacheable.
1) Long UUID are generally considered safe if you don’t allow parties to list the contents of the bucket
2) You can encrypt the asset and include the key with the asset URL (mentioned in the article)
3) Use pre-signed URL (infra load, but smaller than you serving it)
4) Implement your logic in Lambda@Edge (AWS) and link it with CloudFront distribution. You can deny or allow access based on that (Auth, Tokens, etc.)
4) I would say is the best and the safest solution at really low cost. Followed by 1) at price/value ratio.
4) I would say is the best and the safest solution at really low cost. Followed by 1) at price/value ratio.
What is a “short” UUID?
I don’t think GP intended to signify there’s long and short UUIDs, just used “long” to additionally describe UUIDs. I could be wrong, but site guidelines say to assume best intent in your replies.
Pretty much :)
I don't know how would I classify a short UUID, but they should follow the same precautions as if it was a password composed only of lowercase alphanumeric chars. The longer you can afford, the better.
There are variants of uuids that encode a MAC address and timestamp, reducing the randomness quite a bit. If you knew that info, generating a collision would be a lot easier (still impractical). Maybe that's what GP meant?
Short term signatures are the generally recommended approach, especially if you control the client and producer. You give the client a 1/5/60 minute signature, and consumers you authorized can get it.
Protecting from MITM is a separate (important) concern for that approach to be valid though, and it's not 100%.
If you use an integrated cloud+cdn solution, the problem is largely solved for you, assuming you pick up the best practices and run with them
Protecting from MITM is a separate (important) concern for that approach to be valid though, and it's not 100%.
If you use an integrated cloud+cdn solution, the problem is largely solved for you, assuming you pick up the best practices and run with them
The problem was most likely that the image server had directory listing turned on!?
Another problem is if the random image id is leaked.
Another problem is if the random image id is leaked.
You can serve them from the application layer and use authentication there. It's just slower.
“When the phone number is registered, it returns 200 OK, but when the number is not registered, it returns 418 I'm a teapot.“
Good reminder that fun little Easter eggs like using the bizarre 418 return code can bite you. I’d hate to be the engineer defending that decision after this vulnerability was discovered.
Good reminder that fun little Easter eggs like using the bizarre 418 return code can bite you. I’d hate to be the engineer defending that decision after this vulnerability was discovered.
I have a question because recently I had to implement a way to expire static assets and I would like to hear creative/new ideas.
For example, I have all my static assets on S3 and I want to generate a link that will make data available for a long time (let's say 1 year) but with S3 signing you can only generate a link available for a week max.
How would you go about doing this without relying on another server?
I think this is what happened with the public bucket. They thought about how to deliver static assets without relying on a server and the only way they found is to make the bucket public.
For example, I have all my static assets on S3 and I want to generate a link that will make data available for a long time (let's say 1 year) but with S3 signing you can only generate a link available for a week max.
How would you go about doing this without relying on another server?
I think this is what happened with the public bucket. They thought about how to deliver static assets without relying on a server and the only way they found is to make the bucket public.
I actually had to solve this exact problem recently. I set up a lifecycle policy for a known prefix in the bucket, so any item with that prefix is deleted after N amount of time. Then, when a link is requested to a static asset (which is stored at rest with a private ACL), it gets copied into the prefix, with a random name and a public ACL, and the new link gets served to the client.
So far I haven't seen any big drawbacks. It does mean storing the same objects multiple times in S3. But S3 storage is relatively cheap unless you have a huge amount of data. If bandwidth was ever a problem, it would be simple enough to wrap the transient prefix in a CDN.
So far I haven't seen any big drawbacks. It does mean storing the same objects multiple times in S3. But S3 storage is relatively cheap unless you have a huge amount of data. If bandwidth was ever a problem, it would be simple enough to wrap the transient prefix in a CDN.
If you actually want to expire the assets, you could use S3 lifecycle configuration - https://docs.aws.amazon.com/AmazonS3/latest/dev/intro-lifecy....
If you want "clever" ideas, maybe a Lambda that moves off objects of a certain age to a private bucket?
I disagree on what the issue was with their S3 bucket. As these were all public static assets, the real problem was just the ability to bulk enumerate them. As mentioned in the post, the two issues were: 1. ListObjects was enabled 2. The filenames lacked sufficient entropy (debatable in my opinion)
If you want "clever" ideas, maybe a Lambda that moves off objects of a certain age to a private bucket?
I disagree on what the issue was with their S3 bucket. As these were all public static assets, the real problem was just the ability to bulk enumerate them. As mentioned in the post, the two issues were: 1. ListObjects was enabled 2. The filenames lacked sufficient entropy (debatable in my opinion)
You can auto expire the contents of an s3 bucket.
[deleted]
[deleted]
[deleted]
Sounds like a quick run through burp suite
> I think startups could certainly offer bug bounties. It is a nice gesture, and more importantly, platforms like HackerOne provide researchers a legal path to the disclosure of vulnerabilities.
Unfortunately most startups are obsessed with growth and don't really care about security, privacy, etc.
Unfortunately most startups are obsessed with growth and don't really care about security, privacy, etc.
I disagree for another reason. I don't think bug bounty is the best fit for a few reasons:
* HackerOne is very expensive for most startups, IIRC it's around 30k USD/year only to be hosted on their platform (the price is probably different for different companies though.)
* If one doesn't want to use a platform like HackerOne you're in a tax hell doing payments to people all over the world
* The cost of paying for the bugs reported which can be massively expensive if you want to be competitive.
* The time it takes to handle all reports is massive. You might imagine you get cool and interesting research but 99.9% are people copy-pasting output from automated scanners, people reporting on out of scope things, usually low severity with things like "missing x-frame-options" or just straight up false positives. It's both frustrating, demotivating and time consuming.
I think a self hosted responsible disclosure program is better and more sustainable for most startups. Add a security.txt [0] to the website. You might still get a bunch of low quality reports but at least you give people a structured way of disclosing findings.
0. https://securitytxt.org/
* HackerOne is very expensive for most startups, IIRC it's around 30k USD/year only to be hosted on their platform (the price is probably different for different companies though.)
* If one doesn't want to use a platform like HackerOne you're in a tax hell doing payments to people all over the world
* The cost of paying for the bugs reported which can be massively expensive if you want to be competitive.
* The time it takes to handle all reports is massive. You might imagine you get cool and interesting research but 99.9% are people copy-pasting output from automated scanners, people reporting on out of scope things, usually low severity with things like "missing x-frame-options" or just straight up false positives. It's both frustrating, demotivating and time consuming.
I think a self hosted responsible disclosure program is better and more sustainable for most startups. Add a security.txt [0] to the website. You might still get a bunch of low quality reports but at least you give people a structured way of disclosing findings.
0. https://securitytxt.org/
> If one doesn't want to use a platform like HackerOne you're in a tax hell doing payments to people all over the world
Why would the company paying be in tax hell? The recipient is the one who needs to pay income tax
Why would the company paying be in tax hell? The recipient is the one who needs to pay income tax
I actually don't know the exact details, I don't work with finance but that's one of the problems we identified when looking into it.
I think it has do do with how bug bounties generally work. Since bug hunters aren't sending an invoice from a registered company you have to pay employer tax or similar. But as I mentioned, I'm not entirely sure since I don't work with that part. It might also differ depending on where the company is registered.
I think it has do do with how bug bounties generally work. Since bug hunters aren't sending an invoice from a registered company you have to pay employer tax or similar. But as I mentioned, I'm not entirely sure since I don't work with that part. It might also differ depending on where the company is registered.
Unfortunately I think that is because customers and investors broadly don't prioritize security + privacy.
And that's because users broadly don't prioritize security + privacy.
> users broadly don't prioritize security + privacy
Users broadly have no way of analyzing security + privacy. Even most software developers don't have the time or expertise to reverse engineer and analyze even one app, let alone every release of every app they use. They just have to take it on trust. For things like cars we have mandatory standards for vehicle design, crash testing requirements, fuel efficiency and emissions standards etc. to try and make sure that people can expect a certain level of performance and safety. For software there's nothing like that.
Users broadly have no way of analyzing security + privacy. Even most software developers don't have the time or expertise to reverse engineer and analyze even one app, let alone every release of every app they use. They just have to take it on trust. For things like cars we have mandatory standards for vehicle design, crash testing requirements, fuel efficiency and emissions standards etc. to try and make sure that people can expect a certain level of performance and safety. For software there's nothing like that.
I think users have no idea what’s involved in software development, but that they expect any company takes care of the important things in order to bring a product to market.. that includes security and privacy. the “users don’t care” argument, i believe, is a cop out to make ourselves feel good.
I would agree with you, except the number of people still using facebook after cambridge analytica kinda proves that people don't actually care, even the ones that say they do.
I think, "Don't actually care" is pretty different than, "aren't convinced enough by an apparent one-time issue to cut off a lot of relationships".
If a massive privacy breach that potentially shaped history is not sufficient and all the follow up stories on it were not sufficient, customers have basically decided that they practically don't care for the purpose of conducting business operations.
What percentage of the general public do you imagine could give a coherent summary of Cambridge Analytica scandal? I'd be amazed if it were as high as 5%. I get that a lot of people here know about it. But it had no direct impact on most users. It's not surprising that they don't understand this any better than a host of other subtle but important things.
People can't all devote time and energy to everything that's important. It's too big a world. We shouldn't conclude that those things don't matter to them. Our current emergency is a fine example: until a pandemic happened, few knew enough to worry about it. But that doesn't mean we were indifferent to the outcome.
People can't all devote time and energy to everything that's important. It's too big a world. We shouldn't conclude that those things don't matter to them. Our current emergency is a fine example: until a pandemic happened, few knew enough to worry about it. But that doesn't mean we were indifferent to the outcome.
> People can't all devote time and energy to everything that's important.
Well if the users can't spend any time on it, it isn't sufficiently important for the business to care about it as the users will be briefly miffed and then go on to what they consider important.
User unhappiness matters only so far as it changes behavior.
Well if the users can't spend any time on it, it isn't sufficiently important for the business to care about it as the users will be briefly miffed and then go on to what they consider important.
User unhappiness matters only so far as it changes behavior.
That's just not the only way things change. Look, for example, at the establishment of the FDA: https://en.wikipedia.org/wiki/Pure_Food_and_Drug_Act
Industrial food companies were taking significant liberties with food quality and safety. And it was apparently fine! People appeared not to care. Then Upton Sinclair's book The Jungle happened to catch the popular imagination, leading to a wave of outrage and investigations. People still "didn't care" in the sense of, say, not buying industrially produced food. But they did very much care, leading to a regulatory regime that has lasted more than a century.
Industrial food companies were taking significant liberties with food quality and safety. And it was apparently fine! People appeared not to care. Then Upton Sinclair's book The Jungle happened to catch the popular imagination, leading to a wave of outrage and investigations. People still "didn't care" in the sense of, say, not buying industrially produced food. But they did very much care, leading to a regulatory regime that has lasted more than a century.
> but that they expect any company takes care of the important things in order to bring a product to market.
They show that expectation by withholding their money or not using the product when something bad happens. I don't see them doing much of that when companies have data breaches.
They show that expectation by withholding their money or not using the product when something bad happens. I don't see them doing much of that when companies have data breaches.
There already are companies that advertise privacy as an edge, but either they are sexy enough for consumers or they aren't. DuckDuckGo isn't sexy. Firefox isn't sexy. Linux-based desktops aren't sexy.
If these things caught the sparkle in customer's eyes, we'd all know it by now.
If these things caught the sparkle in customer's eyes, we'd all know it by now.
I think this is an important point. Startups trying to make privacy ‘cool’ need to pay more attention to branding. While I love DDG, the name and the whole flavor of their site will necessarily limit their reach to technical people.
In terms of popular media, I thought Mr. Robot did a pretty good job of making privacy/security tech seem sexy, even if the main character is usually bypassing it.
Here’s a good example: https://youtu.be/i9CBKGLVCME
In terms of popular media, I thought Mr. Robot did a pretty good job of making privacy/security tech seem sexy, even if the main character is usually bypassing it.
Here’s a good example: https://youtu.be/i9CBKGLVCME
Agreed. There's just no way consumers can be expected to usefully verify things like privacy. In the real world, "it should just work" expectations get baked into things like building codes, the Uniform Commercial Code, commoditization rules, and food safety standards.
If we don't come up with something like that as an industry, eventually somebody else is going to do it for us. And we won't like that one bit.
If we don't come up with something like that as an industry, eventually somebody else is going to do it for us. And we won't like that one bit.
I agree, at some point we're going to get regulated into a painful place
> the “users don’t care” argument, i believe, is a cop out to make ourselves feel good.
Strongly disagree. It's just the simple reality that most users don't care about security. The vast majority of potential consumers in the world don't choose digital products based on security. I always see this security angle touted on Hacker News, but I'm quite frankly shocked that people here don't have the self-awareness to realize that we live in an uber-tech geek's echo chamber.
Have you ever met an "average" Facebook user? They really, truly, do not understand or care about security. I'm very confident that even if you sat one down and walked them through all of the implications of what poor security even means, they would walk away and not change their behavior whatsoever.
Strongly disagree. It's just the simple reality that most users don't care about security. The vast majority of potential consumers in the world don't choose digital products based on security. I always see this security angle touted on Hacker News, but I'm quite frankly shocked that people here don't have the self-awareness to realize that we live in an uber-tech geek's echo chamber.
Have you ever met an "average" Facebook user? They really, truly, do not understand or care about security. I'm very confident that even if you sat one down and walked them through all of the implications of what poor security even means, they would walk away and not change their behavior whatsoever.
The whole "users don't care" is really ignoring consumer's cognitive dissonance on security.
Adopting the stance that "vast majority of potential consumers in the world don't choose digital products based on security" time-and-time again bites organizations in the ass when there's a breach.
Adopting the stance that "vast majority of potential consumers in the world don't choose digital products based on security" time-and-time again bites organizations in the ass when there's a breach.
> time-and-time again bites organizations in the ass when there's a breach.
The bite isn't very hard though. The largest data breach of the 21st century in terms of users was Adobe and it cost them just 2 million in legal.
The only painful data breach I can think of financially has been Equifax. Everyone else just sent out a "reset your password" email, paid for a couple lawyers and PR people, and went on with their companies.
Can you name a company killed by a data breach? I can't think of one.
The bite isn't very hard though. The largest data breach of the 21st century in terms of users was Adobe and it cost them just 2 million in legal.
The only painful data breach I can think of financially has been Equifax. Everyone else just sent out a "reset your password" email, paid for a couple lawyers and PR people, and went on with their companies.
Can you name a company killed by a data breach? I can't think of one.
I believe users won't care until it's common for there to be repercussions from not caring. If not caring means my odds of having my savings account stolen are %20 a year then people would start caring. As it is there are few directly attributable repercussions. Just vague worries about future problems.
As it is what are the odds of any one user running into issues because of lack of security and privacy? It seems fairly low.
It seems like an opportunity for Apple (assuming they're actually better) to run some scare ads (99% of people scammed via there computer were running Windows/Android) if that's true. If it was true a good ad campaign could get people to care?
For internet stuff how about 98% of the people who got their bank accounts hacked were hacked by leaks of data on Facebook. (probably not true and not provable)
Maybe we need some security insurance who will then audit software and only insure customers that use certified software? They'd have an incentive for their audits to be good because they pay out if it turns out the software is not secure And if their market was big enough then software creators would want to be certified.
It's even possible some standard sandboxes could help make it easy to certify. Add this sandbox to your app and you're certified? Maybe some OSes that already have sandboxes would automatically get certified but server side you'd need audits?
Just throwing out ideas.
As it is what are the odds of any one user running into issues because of lack of security and privacy? It seems fairly low.
It seems like an opportunity for Apple (assuming they're actually better) to run some scare ads (99% of people scammed via there computer were running Windows/Android) if that's true. If it was true a good ad campaign could get people to care?
For internet stuff how about 98% of the people who got their bank accounts hacked were hacked by leaks of data on Facebook. (probably not true and not provable)
Maybe we need some security insurance who will then audit software and only insure customers that use certified software? They'd have an incentive for their audits to be good because they pay out if it turns out the software is not secure And if their market was big enough then software creators would want to be certified.
It's even possible some standard sandboxes could help make it easy to certify. Add this sandbox to your app and you're certified? Maybe some OSes that already have sandboxes would automatically get certified but server side you'd need audits?
Just throwing out ideas.
Users do care, but they tend to guess the quality of the entire product, including security, from the slick look of the product or marketing pages with fancy words. In reality, all of these are hard to verify even for tech savvy users.
Amazon/MS/Google should do a better job of making it hard to leave things unprotected. They are no longer a new service and no longer have the excuse of having to avoid friction.
Amazon/MS/Google should do a better job of making it hard to leave things unprotected. They are no longer a new service and no longer have the excuse of having to avoid friction.
I think users care, given the outrage when leaks or “creepy” ads occur. But the problem is that users have an impossible time reasonably evaluating apps for security and privacy, so there’s very little market incentive for app makers to make secure and private apps.
Consumers also spend billions on homeopathic medicine. That doesn't mean fake medicine or privacy violations should be legal.
I work in a startup and unfourtunately we don’t have cash lying around to offer any bug bounty that would be reasonably priced. And, I think, that applies propably to most of the startups.
> ...identified several misconfigured S3 buckets...
For the love of all that is holy... have your AWS, Azure, etc. cloud service setup/config wire-brushed by a 3rd party expert before deploying!
That, or perhaps these cloud services need a clippy like wizard: "Are you sure these files should be 777? Perhaps you want 644?".
For the love of all that is holy... have your AWS, Azure, etc. cloud service setup/config wire-brushed by a 3rd party expert before deploying!
That, or perhaps these cloud services need a clippy like wizard: "Are you sure these files should be 777? Perhaps you want 644?".
If you are using the AWS console, S3 actually asks you if you want the bucket to be public and gives you an option to create non public buckets.
Thing is, S3 ACL is pretty ugly. UX is unpleasant and testing is terrible. It could be some accidental catch-all rule letting everyone in. I guess that the only lesson here is to set up an automated test that verifies whether the resource you wanted to block is actually blocked.
Presumably there are some tools that will do that ?
Such as DivvyCloud: https://docs.divvycloud.com/docs/amazon-web-services
paulie_a(2)
Sounds like you can greatly decrease your chances of being hacked by simply not offering an android app.
since this person was doing network level I/O work, they could do similar things with an iOS device leaking network packages. there are even work arounds for "pinned" SSL certifications so that your package sniffer can read the contents of network requests and responses.
In the process revealing how erroneously so many put stock in matching via a system with a very clearly vulnerable system (compatibility rating algorithm) and went out of his way to optimize the system further with less than desirable results: he was the most desired Male profile and was going out on several dates a day and rejecting women he'd have coffee dates with in more and more absurd ways as he got more and more requests.
It was really funny, albeit cautionary, and I only caught like 65% of it, so if anyone finds it can you post it here?
Jack Dorsey's recent interview on the AI podcast hinted at how perilous this could be, too. The 'why' part of Machine learning/data Science/AI has to be just as tightly examined as the optimization development when deployed into Society.