Make anonymous HTTP GET requests with proxies via Python(github.com)
github.com
Make anonymous HTTP GET requests with proxies via Python
https://github.com/rootVIII/proxy_requests
23 comments
What's the best way to browse the web anonymously?
Full anon is very hard. If you have to ask you're not going to be able to pull it off. I did a contract once where my client had to deal with some really scary people.
The steps I took were so extreme. Cash phone, burner computer, custom proxies, relays, delays, fake data, fake traffic.
And you still never know.
Your adversary could have something you didn't even think of and you get owned anyway.
If all you want to do is avoid tracking scripts from companies, however, use a VM on another user account on your computer and write up a script to light up a new DigitalOcean droplet automatically when you browser. Cycle the IP frequently and use a bunch of different VMs with different browsers and OSes. It's enough for 99.99%.
The steps I took were so extreme. Cash phone, burner computer, custom proxies, relays, delays, fake data, fake traffic.
And you still never know.
Your adversary could have something you didn't even think of and you get owned anyway.
If all you want to do is avoid tracking scripts from companies, however, use a VM on another user account on your computer and write up a script to light up a new DigitalOcean droplet automatically when you browser. Cycle the IP frequently and use a bunch of different VMs with different browsers and OSes. It's enough for 99.99%.
Depends on your definition of anonymous. Who are you trying to hide from? You ISP, government, the site you visit? The answer might lie somewhere in between "just sign up with a VPN provider" and "reinstall your OS, drive to a Starbucks at least 50 miles away and use their WiFi, never logging in to any site you visit."
Fixed a slight bug... it should not ever leak your IP... And you do not lose HTTPS with this module whatsoever
Uses the list of free proxies from here: https://www.us-proxy.org/
Anonymous to anyone but the operator of the proxy and you also lose HTTPS so....
P.S. proxies do not completely mask DNS requests so timing analysis is still possible for uncached requests.
And worse the payload returned can be used to unmask the user behind a proxy e.g. redirect to uniqueid.myhost.com my honey pot does that to unmask requests originating from TOR or from proxies and it’s quite successful.
P.S. proxies do not completely mask DNS requests so timing analysis is still possible for uncached requests.
And worse the payload returned can be used to unmask the user behind a proxy e.g. redirect to uniqueid.myhost.com my honey pot does that to unmask requests originating from TOR or from proxies and it’s quite successful.
This uses HTTP CONNECT, it does HTTPS just fine.
The only DNS request this script will make locally is for www.us-proxy.org.
The only DNS request this script will make locally is for www.us-proxy.org.
You're saying people using Tor make unmasked DNS requests? Are these users using some non-standard configuration? Because I'm pretty sure the Tor browser protects against that.
It does but anyone hitting a honey pot server most likely won't be doing it within Tor browser. It's easy to send traffic via Tor's socks port but forget about DNS lookups.
I could be wrong but I'm not sure this is completely accurate. There are HTTPS proxies listed on the page it's fetching proxies from.
And it's been a while but I could have sworn you can do DNS requests through a proxy.
Feel free to correct me if I'm wrong though.
And it's been a while but I could have sworn you can do DNS requests through a proxy.
Feel free to correct me if I'm wrong though.
You can proxy DNS requests through SOCKS proxies and trough normal ones but it’s not common or at least airtight (e.g. a none http/https resource request can bypass proxy even in Chrome, addons, flash and other system calls also may be used to reveal the original IP), and proxies do not preserve HTTPS without being able to MITM it, if you want to anonymize something use TOR at least non-state actors likely can’t trace the source of the request since no one is operating enough nodes and has the capability to run timing analysis on the entire network.
If you have something that is sensitive enough to require anonymity you do not want to disclose it to another party.
Proxies are also finicky my honeypot tries DNS resolving via multiple vectors including applets, flash and more recently the dns.resolve API Firefox implemented with 60 onwards. I also return an SSL cert and use OCSP and CRL resolutions to try and get the actual IP address.
Overall proxies and OpenVPN provides are unasked in about 50% of the cases TOR in the high single digits and im not doing anything super sophisticated.
You can ofc go beyond that and fingerprint the browser, use zero days or abuse headless browsers or other frameworks but I’m not that bored yet.
If you have something that is sensitive enough to require anonymity you do not want to disclose it to another party.
Proxies are also finicky my honeypot tries DNS resolving via multiple vectors including applets, flash and more recently the dns.resolve API Firefox implemented with 60 onwards. I also return an SSL cert and use OCSP and CRL resolutions to try and get the actual IP address.
Overall proxies and OpenVPN provides are unasked in about 50% of the cases TOR in the high single digits and im not doing anything super sophisticated.
You can ofc go beyond that and fingerprint the browser, use zero days or abuse headless browsers or other frameworks but I’m not that bored yet.
>and proxies do not preserve HTTPS without being able to MITM
This is simply not true in the slightest.
This is simply not true in the slightest.
Web newb here. Why are these things so hard to mask? Can't redirect be denied/caught by the Python requests libraries?
It can ofc, but the question is always what do you do with the content :)
But in any case anything that is too sensitive to send from your own IP should not be sent over a medium that you have no ability to verify if it's being logged and by whom.
Open proxy operators aren't charities most of them are dubious at best.
If you want anonymity cryptocurrency VPS in data heaven jurisdiction or TOR is the best way to go, if you don't need that much anonymity than "proven" no-logging VPN providers are also better since they guarantee higher privacy than unauthenticated open proxies.
But in any case anything that is too sensitive to send from your own IP should not be sent over a medium that you have no ability to verify if it's being logged and by whom.
Open proxy operators aren't charities most of them are dubious at best.
If you want anonymity cryptocurrency VPS in data heaven jurisdiction or TOR is the best way to go, if you don't need that much anonymity than "proven" no-logging VPN providers are also better since they guarantee higher privacy than unauthenticated open proxies.
There was a good thread here about scraping and the use of paid proxy services. Seems like free isn’t a solution for anything at scale.
Cool!
The proxies from this list don't usually last that long, so you might have connection problems.
The proxies from this list don't usually last that long, so you might have connection problems.
What is the average shelf life of these free proxies in your estimation?
The page seems to be updated every 10 mins or so. Any time a request is made, the newest procies are scraped:)
I was literally about to write this same program in nodejs for a client's scraping project. Serendipity at its best.
Mine's a bit different though because I need to discard proxy server IPs that are flagged by cloudflare on target sites and only be left with those that work flawlessly. This should be done every couple of hours to ensure the 'freshness' of the proxy server IPs that I have.
Mine's a bit different though because I need to discard proxy server IPs that are flagged by cloudflare on target sites and only be left with those that work flawlessly. This should be done every couple of hours to ensure the 'freshness' of the proxy server IPs that I have.
No one should be expecting privacy if proxies are the only solution you rely on.
https://www.whonix.org/Comparison_Of_Tor_with_CGI_Proxies,_P...
"Conclusion
Proxies have a high susceptibility of misusing and stealing user data: Many proxies (HTTP/HTTPS/SOCKS) are PCs hijacked by hackers or criminals, or honeypots exclusively offered for the purpose of user observation. Even if they were legitimate, a single operator can decide to enable logging. Additionally, some proxies automatically give your IP address away to the destination server.
Proxies offer, at best, only weak protection against destination website logging, and they offer no protection from third party eavesdropping. Their use is discouraged"
A good starting point if you wish to appreciate privacy more is: https://www.whonix.org/wiki/Security_Guide
Also look into Qubes OS: https://www.whonix.org/wiki/Qubes