GoodbyeDPI: Deep Packet Inspection circumvention utility(github.com)
github.com
GoodbyeDPI: Deep Packet Inspection circumvention utility
https://github.com/ValdikSS/GoodbyeDPI
123 comments
Well, the good point of GoodbyeDPI is exactly so that it preserves your IP address. Normally, when trying to circumvent censorship, you would need a VPN server in a different country. But the downsides are that the bank will deny all transactions and call you (OK, answered, they added the VPN IP to the whitelist), that you will miss local-only content, you won't be able to register for a doctor appointment online (the city uses a geo-restricting filter out of security concerns), and you will see foreign prices (which might be 5x higher in some cases!) on sites that differentiate based on the country. Also, extra latency in games. None of that applies with GoodbyeDPI.
Edit: the above is written from the viewpoint of the past myself, before emigration to the Philippines.
Edit: the above is written from the viewpoint of the past myself, before emigration to the Philippines.
I wonder why a VPN is the default solution (with all complications it ensues, some of which you've listed), when a simple SSH tunnel to any server in a sane location does just fine. `ssh server -D12345`, point your applications to socks5 at localhost:12345, and it's done. It's dead simple to only allow/deny those sites that you (don't) need to go through another server, and the traffic is encrypted (and optionally compressed), and looks just like another SSH connection.
I've used many other solutions (including WireGuard, etc.) on and off, but always come back to SSH.
I've used many other solutions (including WireGuard, etc.) on and off, but always come back to SSH.
It's easier to detect that someone is using a SOCKS tunnel – from memory, one way it might be exposed is if the packet TTL is incongruous [1] as I don't think SOCKS rewrites those.
At the height of the pandemic I travelled to Denmark for work (on a clinical trial) and had a UK negative covid test to report to the UK government that I hadn't got around to – whose website geo-blocks people reporting covid tests from outside a UK IP address (even if, e.g. you'd just left it and wanted to report a negative test taken the day before). A SOCKS proxy was detected and I got a "we cannot verify you are in the UK" message. A wireguard VPN worked fine.
[1] https://incolumitas.com/2021/03/13/tcp-ip-fingerprinting-for...
At the height of the pandemic I travelled to Denmark for work (on a clinical trial) and had a UK negative covid test to report to the UK government that I hadn't got around to – whose website geo-blocks people reporting covid tests from outside a UK IP address (even if, e.g. you'd just left it and wanted to report a negative test taken the day before). A SOCKS proxy was detected and I got a "we cannot verify you are in the UK" message. A wireguard VPN worked fine.
[1] https://incolumitas.com/2021/03/13/tcp-ip-fingerprinting-for...
point your applications to
That's the problem. Not all of them will implement tunneling their own traffic through SOCKS, and there's still other things like DNS that you might also want to go through the tunnel, but can't easily do so. A VPN sits at a lower layer, just looking like a regular network connection, so applications don't need to be aware.
That's the problem. Not all of them will implement tunneling their own traffic through SOCKS, and there's still other things like DNS that you might also want to go through the tunnel, but can't easily do so. A VPN sits at a lower layer, just looking like a regular network connection, so applications don't need to be aware.
Well, yeah, that's the problem (or the main advantage depending on your viewpoint). The post I was replying to mentioned how painful it is to avoid routing through VPN where it's not needed (although it's pretty easy to do on Linux with network namespaces, and IIRC policy routing, which I've never tried).
I just want to point out the simplest solution which for some reason doesn't seem to be very popular, although it covers most users' use-cases better than a VPN connection does (IMHO).
Don't know about other browsers, but Firefox is able to send DNS requests through socks, whether you're using DNS-over-HTTPS or not.
I just want to point out the simplest solution which for some reason doesn't seem to be very popular, although it covers most users' use-cases better than a VPN connection does (IMHO).
Don't know about other browsers, but Firefox is able to send DNS requests through socks, whether you're using DNS-over-HTTPS or not.
In China at least, the GFW can detect tunneled SSH traffic and cut it off.
Thank you, I do too. I though it was only me.
SSH is very simple and there’s almost nothing a SSH tunnel can’t do.
SSH is very simple and there’s almost nothing a SSH tunnel can’t do.
> SSH is very simple and there’s almost nothing a SSH tunnel can’t do.
You cannot disguise your SSH traffic mimicking HTTPS traffic which help you to bypass DPI solutions.. so its easy to block/filter/log your traffic or even pinpoint you in an adverse environment.
You cannot disguise your SSH traffic mimicking HTTPS traffic which help you to bypass DPI solutions.. so its easy to block/filter/log your traffic or even pinpoint you in an adverse environment.
Please expand. How can an Apache server, for instance, know if I’m accessing through and SSH tunnel. And how would that be different on a Wireguard VPN?
Why apache would care? We are talking about DPI solutions, aka deep packet inspection. They are normally deployed inline, and SSH tunnels are so often blocked, that in some solutions you have it one click away from you https://www.sonicwall.com/support/knowledge-base/how-to-bloc.... Other solutions try to make the traffic similar with Apache + Firefox to make it harder to be detectable and blocked by DPI solutions..
Interesting, thanks. And would Wireguard be transparent to DPI?
there are many implementations of DPI out there, each one with your own set of rules and heuristics... this discussion[1] talks about it, but the short answer is: it depends
References:
[1] https://www.reddit.com/r/WireGuard/comments/ajv0eq/wireguard...
References:
[1] https://www.reddit.com/r/WireGuard/comments/ajv0eq/wireguard...
@patrakov I can't reply to you directly.
If the only thing a web server could do is differentiate tunnel from direct IP connection with or without firewall/NAT, which are ubiquitous, it's an interesting effort, but a tour de force with little gain IMO.
If the only thing a web server could do is differentiate tunnel from direct IP connection with or without firewall/NAT, which are ubiquitous, it's an interesting effort, but a tour de force with little gain IMO.
Encapsulating TCP in TCP results in exponential backoff and retransmissions in the event of loss.
SSH tunneling is encapsulating byte streams in TCP, not TCP (which means "packets with sequence numbers, acknowledgements, and retransmissions") in TCP, and therefore doesn't suffer.
Well the byte stream includes everything you throw in (TCP/UDP/L2), so while the tunnel will not suffer from any signaling issues in the payload flow, the opposite is not true - the internal traffic is affected by any TCP problems on the SSH tunnel so a short blip on the tunnel can cause a cascade of blips depending on whats in there.
[deleted]
For what it’s worth as well, there are other solutions than whole-network VPNs and such.
Personally, I chose to generate a domain list for V2Ray from the Russian government’s blocklist when I lived there [1].
I prefer to do that typically because it avoids the pain of the ever-growing whitelists and it allows me to keep the traffic encrypted in case someone does actually figure out that you’ve bypassed DPI. And if you use something like V2Ray or ShadowSocks, they’ll disguise the traffic much better than something like OpenVPN typically would, making it less obvious to anyone monitoring that you’re using a proxy in the first place.
There’s a load of references and pre-generated lists for different needs if anyone else is interested in doing something similar [2].
(Also, I hope this doesn’t come across as missing the point of the tool — I think it’s really useful and a good solution. I just figured I’d note some others too)
[1]: https://github.com/OmarAssadi/AntiZapret-V2Ray
[2]: https://github.com/v2ray/domain-list-community
Personally, I chose to generate a domain list for V2Ray from the Russian government’s blocklist when I lived there [1].
I prefer to do that typically because it avoids the pain of the ever-growing whitelists and it allows me to keep the traffic encrypted in case someone does actually figure out that you’ve bypassed DPI. And if you use something like V2Ray or ShadowSocks, they’ll disguise the traffic much better than something like OpenVPN typically would, making it less obvious to anyone monitoring that you’re using a proxy in the first place.
There’s a load of references and pre-generated lists for different needs if anyone else is interested in doing something similar [2].
(Also, I hope this doesn’t come across as missing the point of the tool — I think it’s really useful and a good solution. I just figured I’d note some others too)
[1]: https://github.com/OmarAssadi/AntiZapret-V2Ray
[2]: https://github.com/v2ray/domain-list-community
GoodbyeDPI also includes Russian blacklist built from zapret-info, to apply censorship circumvention only for the websites from the list, to reduce the risk of breaking the website due to mangled traffic.
The newest issue are unlisted filtering performed on so-called TSPU DPI boxes. Two years ago we had only ISP DPI boxes, but now there's a government TSPU black box which they control themselves and block the websites/VPNs/SSH/IP ranges out-of-the-registry.
The newest issue are unlisted filtering performed on so-called TSPU DPI boxes. Two years ago we had only ISP DPI boxes, but now there's a government TSPU black box which they control themselves and block the websites/VPNs/SSH/IP ranges out-of-the-registry.
Yeah, I've come across your tool a few times before and seen the default lists. Really useful stuff, by the way!
Interesting, though. I had heard talks about introducing proper government-level filtering -- I think after the Telegram/AWS/etc blocks in, like, 2018 (?), but I wasn't aware of anything actually going into effect.
If you've got time to answer or link me anything, I am a little curious. How are the TSPU boxes setup? Are these provided by the government to different datacenters/IXs or at some sort of higher level than that? And are they currently just used to filter additional out-of-registry domains/IP addresses or do they also filter the semi-public, known blacklist? Is there anything like the unofficial Chinese gfwlist that tries to maintain a list of the out-of-registry stuff?
I haven't lived in Russia in a little while now, but when I was last there, although virtually every residential ISP enforced the government list, a number of the domestic server providers weren't, so a good option for low-latency and keeping a Russian IP address was just renting a gigabit VPS from the city next to me and using it as a proxy server.
Interesting, though. I had heard talks about introducing proper government-level filtering -- I think after the Telegram/AWS/etc blocks in, like, 2018 (?), but I wasn't aware of anything actually going into effect.
If you've got time to answer or link me anything, I am a little curious. How are the TSPU boxes setup? Are these provided by the government to different datacenters/IXs or at some sort of higher level than that? And are they currently just used to filter additional out-of-registry domains/IP addresses or do they also filter the semi-public, known blacklist? Is there anything like the unofficial Chinese gfwlist that tries to maintain a list of the out-of-registry stuff?
I haven't lived in Russia in a little while now, but when I was last there, although virtually every residential ISP enforced the government list, a number of the domestic server providers weren't, so a good option for low-latency and keeping a Russian IP address was just renting a gigabit VPS from the city next to me and using it as a proxy server.
https://bitbucket.org/anticensority/russian-unlisted-blocks/... — my list of unlisted blocks
>How are the TSPU boxes setup? Are these provided by the government to different datacenters/IXs or at some sort of higher level than that?
They are provided by the government and should be installed topologically close to the client, before CGNAT. This is a modified RDP.RU EcoFilter, and currently are required to be installed only on residential connections, not in DCs/IXes. ISPs do not have any configuration access, and it's prohibited to route traffic not via the boxes. The abbreviation TSPU means Technical Measures to Combat Threats, and these boxes are capable of collecting, saving and centralized sharing of NetFlow data, but currently are almost always used only for blocking, however the general idea is to centrally control BGP flows and collect SNMP data from other ISP routers.
The company which controls the boxes is called Center of Public Network Monitoring and Control (ЦМУ ССОП, Центр мониторинга и управления сетью связи общего пользования).
>do they also filter the semi-public, known blacklist?
Yes, they do. I suppose the idea is to replace filtering DPI boxes which were installed on the ISP network all these years with this one. Right now most ISPs have both TSPU and one of commercial DPI systems.
More information in English from Alexander Isavnin on RIPE: https://ripe83.ripe.net/archives/video/630/
>How are the TSPU boxes setup? Are these provided by the government to different datacenters/IXs or at some sort of higher level than that?
They are provided by the government and should be installed topologically close to the client, before CGNAT. This is a modified RDP.RU EcoFilter, and currently are required to be installed only on residential connections, not in DCs/IXes. ISPs do not have any configuration access, and it's prohibited to route traffic not via the boxes. The abbreviation TSPU means Technical Measures to Combat Threats, and these boxes are capable of collecting, saving and centralized sharing of NetFlow data, but currently are almost always used only for blocking, however the general idea is to centrally control BGP flows and collect SNMP data from other ISP routers.
The company which controls the boxes is called Center of Public Network Monitoring and Control (ЦМУ ССОП, Центр мониторинга и управления сетью связи общего пользования).
>do they also filter the semi-public, known blacklist?
Yes, they do. I suppose the idea is to replace filtering DPI boxes which were installed on the ISP network all these years with this one. Right now most ISPs have both TSPU and one of commercial DPI systems.
More information in English from Alexander Isavnin on RIPE: https://ripe83.ripe.net/archives/video/630/
The problem with such lists is the inherent assumption that all ISPs block the same "bad stuff". However, in reality, this is not the case, because each ISP has to implement the blocking on their own, and there are multiple DPI solutions with different sets of false positives. This "provider-specific overblocking" is especially common with IPv6.
So in addition to using such lists with one of the ISPs, I tried to detect signs of non-prevented blockage using iptables (matching on stuff like unusually-high TTL of an RST packet, or a string that occurs in the SSL certificate that they try to use for MITM - yes, they were not even consistent, or maybe there were two layers of DPI), and add the addresses learned this way to an ipset, so that next time they are routed through a VPN.
On the other ISP at a different location, just dropping all packets with ID=0 was for some time enough to avoid the censorship.
So in addition to using such lists with one of the ISPs, I tried to detect signs of non-prevented blockage using iptables (matching on stuff like unusually-high TTL of an RST packet, or a string that occurs in the SSL certificate that they try to use for MITM - yes, they were not even consistent, or maybe there were two layers of DPI), and add the addresses learned this way to an ipset, so that next time they are routed through a VPN.
On the other ISP at a different location, just dropping all packets with ID=0 was for some time enough to avoid the censorship.
Why isn’t SSL working for you? You should be end-to-end encrypted and no censor or government should be able to see your https requests.
The only time this doesn’t apply is if someone controls your computer or the destination website and is able to MITM your TLS traffic. Is that what has happened?
Your HTTPS headers are not visible to anyone. So, for example, why is GoodbyDPI modifying the Host header? This is inside the end-to-end TLS encrypted connection that your ISP can’t see, and that the destination web host can’t see.
The only time this doesn’t apply is if someone controls your computer or the destination website and is able to MITM your TLS traffic. Is that what has happened?
Your HTTPS headers are not visible to anyone. So, for example, why is GoodbyDPI modifying the Host header? This is inside the end-to-end TLS encrypted connection that your ISP can’t see, and that the destination web host can’t see.
SNI (the thing in the ClientHello packet which indicates the domain name for which to get a certificate, just in case if there is more than one SSL website on one IP) is not encrypted. DPI solutions (and even plain old Squid) can look into this without the need to break any encryption.
You don't control the myriad of routers between you and your server, and between the server and its destination. One reason to use commercial VPNs is to at least attempt to launder your packets.
Haven’t you just replaced trust in your ISP with trust in the hosting provider of your VPN?
(Depending on yourISP and provider this can be a good trade-off)
(Depending on yourISP and provider this can be a good trade-off)
It's widely employed by Russian and Indian ISPs to censor access to websites. I've had a largely successful run with GoodbyeDPI 's circumvention.
More importantly, it's widely deployed by HNers in US to snoop though everything you do on your phone, tablet and computer if you dare to use their wifi.
The ops topics are full of people claiming how critically important it is for them to sniff through everything you do on "their" network for security.
You don't need to go to Russia, China or India to have your privacy violated. Just go to work.
The ops topics are full of people claiming how critically important it is for them to sniff through everything you do on "their" network for security.
You don't need to go to Russia, China or India to have your privacy violated. Just go to work.
Your privacy isn't violated if you avail yourself of someone else's property.
Violation and legality are two entirely different things.
Depends whether you were aware of the consequences of accessing that property. If you walk through my front gate and I pants you, take photos of your junk and upload them, you'll probably be upset unless I had a sign on the gate saying (in clear language, not 14 pages of irrelevant 6pt legalese hiding a link to the actual 'privacy policy' document which admits in obfuscated language that pantsing is an option) that this would happen.
No. When all "gates" in a domain have this sign, it isn't less upsetting.
The networks of ISPs you're "availing yourself to" are their property.
South Korea also does this, and this tool is well known as one of the best circumvention methods (alongside with VPNs, but they cost money)
The problem / disconnect with that DPI is dead article is that corporations still want to use use, as they control their employees computers.
Hence I've recently been asked to implement a URL inspecting firewall, which implies a HTTPS intercepting MiTM proxy. They will probably require some exception list where things get passed through 'unmolested', but for a white listed sub-set of sites (say well known health, banking, etc).
The one thing which can break that is cert pinning, but then the corporations can simply mandate that apps requiring/using cert pinning not be used from the corporate machines.
There are some bumps in the techniques caused by HTTPS RRs, AltSvc and HTTP/3, but they will be worked around, at worst by forced downgrades.
The "they'll never no argument" carries no weight, as employers up front tend to inform their employees that the communications are monitored, hence giving the corporation legal cover.
That said, I do generally agree with the aim of that article, but changing the corporate mindset is a different (non technical) problem; it is all about providing the corporation with a colorable argument towards reducing their liability in certain scenarios.
Hence I've recently been asked to implement a URL inspecting firewall, which implies a HTTPS intercepting MiTM proxy. They will probably require some exception list where things get passed through 'unmolested', but for a white listed sub-set of sites (say well known health, banking, etc).
The one thing which can break that is cert pinning, but then the corporations can simply mandate that apps requiring/using cert pinning not be used from the corporate machines.
There are some bumps in the techniques caused by HTTPS RRs, AltSvc and HTTP/3, but they will be worked around, at worst by forced downgrades.
The "they'll never no argument" carries no weight, as employers up front tend to inform their employees that the communications are monitored, hence giving the corporation legal cover.
That said, I do generally agree with the aim of that article, but changing the corporate mindset is a different (non technical) problem; it is all about providing the corporation with a colorable argument towards reducing their liability in certain scenarios.
FYI, your setup is great if you don’t trust your ISP but it doesn’t provide any privacy.
Public VPNs act as a mixer and hide your tracks. IP correlation is very easy nowadays, so persistent single connection to your own VPN won’t protect you from certain entities that correlate your traffic.
DoH is better than DNS but it doesn’t provide privacy. You should switch to DNSCrypt.
Public VPNs act as a mixer and hide your tracks. IP correlation is very easy nowadays, so persistent single connection to your own VPN won’t protect you from certain entities that correlate your traffic.
DoH is better than DNS but it doesn’t provide privacy. You should switch to DNSCrypt.
> I religiously route all my traffic through a VPN that I own and control.
But then you are losing the anonymizing effect of the VPN. If you are doing something illegal this is obviously bad but I guess it otherwise doesn't really matter. Companies like Mullvad have tens of thousands of users which means that the actions of one VPN IP address can not be attributed to just one person. You're just transferring the ability to surveil you from your ISP to whoever hosts your VPN server.
But then you are losing the anonymizing effect of the VPN. If you are doing something illegal this is obviously bad but I guess it otherwise doesn't really matter. Companies like Mullvad have tens of thousands of users which means that the actions of one VPN IP address can not be attributed to just one person. You're just transferring the ability to surveil you from your ISP to whoever hosts your VPN server.
religiously route all my traffic through a VPN that I own and control
Can you please share an easy way to do that?
I’m usually not afraid of wading through configuration swamps, but when it comes to openvpn, I curl up in a corner crying.
Can you please share an easy way to do that?
I’m usually not afraid of wading through configuration swamps, but when it comes to openvpn, I curl up in a corner crying.
Personally I use dsvpn (https://github.com/jedisct1/dsvpn) which is super simple to setup and use, and it's made by the guy who created libsodium.
Thanks! But from what I can see in sources (please correct me if that’s wrong), it is not a vpn, and a regular vpn client cannot connect to it. Nice idea along the lines of SOCKS proxy.
You can just use Tailscale if all of this seems like a pushup. It's wireguard under the hood and everything JustWorks™
This is probably the easiest way. It uses wireguard instead of openvpn but I'm pretty sure it's better
https://github.com/angristan/wireguard-install
I have had good luck with Algo: https://github.com/trailofbits/algo
But why exactly do you do this? Own a server somewhere, admin it, replace or repair it in case of trouble - what for?
What VPN service did you choose, and what were the contenders if any?
Why not run everything through TOR then? Or is that too limiting?
Looking at the circumvention techniques GoodbyeDPI uses makes me want to cry. Is this really the state of DPI in 2022: changing Host to hoSt, or adding white spaces between method and URI actually works?
I suspect it is also due to the scale at which DPI is used; every additional bit of complexity quickly adds up to increase the amount of processing power required.
Censorship companies probably can't hire good talent. I'm actually glad to see this, however it might get patched if this tool becomes too widespread.
It's not as much about talent but about tradeoffs. I work in traffic monitoring tools (not censorship, just observability tools for infrastructure) there's always the decision of how many edge cases you want to cover vs how fast you want your tool to go. At millions of packets per second, an extra "if" might make a big difference in the throughput you're able to monitor. So maybe it's actually reasonable to ignore the .1% that use "hoSt" instead of "Host" to avoid losing .5% of the packets.
More importantly, using “hoSt” is a self-declaration of being an enemy of the censorship regime which is probably dangerous in Russia.
Or perhaps some good talent somehow ending up working there made sure the system is full of plausibly deniable holes.
I would like to believe that; it makes me feel warm and comfortable
Interesting to see the two meanings of good appear in both your and your parent's comment.
I suspect it's not even about not being able to, there's very little motivation. I had a brief contact with ZScaler who operates approximately in this area of traffic inspection, they literally have no clue and they don't care. Their service can be hot flaming trash but people will still pay them money because they check some boxes. I'm sure it applies to other companies in the same area as well.
That was true 3-4 years ago but nowadays they're getting surprisingly good, and that's alarming.
It is harder problem that it sounds. Deep packet inspection needs to happen at some linespeed. More work you do the harder it is to process it all fast enough. You can write stuff for single packets, but when you have lot of connections happening it becomes much harder problem.
Isn’t this stuff typically specially built HW? I feel like an ASIC can accelerate this stuff fairly quickly although the volumes/pricing may not warrant building that. Also if you’re matching on host name there’s no reason you even need to keep up with line rate. All you need is to do is keep up with the connection establishment rate and you can always do the processing in the background and just issue a TCP reset after the fact.
Can you update ASICs with new firmware?
No but I suspect many hardware products use FPGAs, which can be updated as you would firmware, and indeed are often updated during firmware updates.
Your suspicion is wrong. Sorry.
I actually work in the field (networking) and FPGAs are very common in professional telecommunication equipments, hence my suspicion/guess that DPI are the same, especially since I'm also guessing that this is the sort of thing that may be updated often. So I think my 'suspicion' is at least as good as yours.
As long as we are pulling ranks,
I worked in a DPI/firewall company and my work ran on the ASIC accelerator, so nah, my 'guess' is probably better.
FPGA is not worth the trouble. You get neither the (line) speed of ASIC, nor the flexibility of running everything in the CPU. Most serious DPI hardware vendors have stopped using it.
But you are right that it's no fun trying to workaround ASIC bugs.
I worked in a DPI/firewall company and my work ran on the ASIC accelerator, so nah, my 'guess' is probably better.
FPGA is not worth the trouble. You get neither the (line) speed of ASIC, nor the flexibility of running everything in the CPU. Most serious DPI hardware vendors have stopped using it.
But you are right that it's no fun trying to workaround ASIC bugs.
Well you made a laconic, non-substantive reply so you ought to expect pushback.
FGPAs allow near-ASIC speeds with effectively the flexibility of software in that they can be updated via firmware upgrades, with much cheaper dev. costs than ASICs. They do have a higher unit cost than ASICs but only at high volume. For anything that is 'low' volume an ASIC may not make financial sense at all in any case.
I am no expert in DPI specifically but Google suggests that using FPGAs for DPI is an active commercial topic.
FGPAs allow near-ASIC speeds with effectively the flexibility of software in that they can be updated via firmware upgrades, with much cheaper dev. costs than ASICs. They do have a higher unit cost than ASICs but only at high volume. For anything that is 'low' volume an ASIC may not make financial sense at all in any case.
I am no expert in DPI specifically but Google suggests that using FPGAs for DPI is an active commercial topic.
Don’t google “request smuggling vuln”; this is the state of all proxies and load balancers. Everyone tests the happy path and calls it a day.
You can get really far with cheap techniques when your goal is to dissuade. The bigger concern I’d have is statistical analysis of top offender.
Every OSI layer offers more bypass techniques and is the halting problem where your goal is to get value without making everything break when a new browser comes out. You can’t cover all options as a 3rd party and get it perfect.
The higher up application layer, the easier it is to bypass. The more you try to classify without impact (dpi,ids,waf,spam,av), the easier bypasses are.
The domains that get effective like spam have quicker feedback loops. Network middle boxes have the slowest response cycle where they are explicitly called out in RFCs
<script> In a url might get blocked but <script >… bc it’s string matching and not layer aware.
Every OSI layer offers more bypass techniques and is the halting problem where your goal is to get value without making everything break when a new browser comes out. You can’t cover all options as a 3rd party and get it perfect.
The higher up application layer, the easier it is to bypass. The more you try to classify without impact (dpi,ids,waf,spam,av), the easier bypasses are.
The domains that get effective like spam have quicker feedback loops. Network middle boxes have the slowest response cycle where they are explicitly called out in RFCs
<script> In a url might get blocked but <script >… bc it’s string matching and not layer aware.
If you're spying on people who use plain HTTP, why would you expect them to be clever.
most the engines out there weren't made for security but performance. It's disturbing and relaxing at the same time to see how easy it is to bypass them. Something that works 100% is to multiplex a channel, changing it protocols after some packages. You do the SSL handshake, than after some amount of time, you switch it to SSH, I think something like that https://github.com/yrutschle/sslh (couldn't find the real repository that I used, but that one looks similar) could be used after the detection to bypass filters
No, unfortunately almost no of these naive methods longer works. However the protocol spoofing ("fake packet" in GoodbyeDPI) with Auto-TTL is pretty effective on most ISPs of Russia, Korea, Indonesia, Turkey.
Haha that’s cute. Here in china the GFW now simply routes the IPs to blackhole. False positive by virtual host or CDN you ask? They don’t care :)
That's what domain fronting is for, and even though the GFW attempts to filter by SNI, genetic algorithms like Geneva are able to find workarounds: https://geneva.cs.umd.edu/papers/foci21.pdf
ESNI/ESH is coming soon, I guess we'll see soon enough how that plays out.
Since the project at the OP link is mostly/initially aimed at Russia, in my own experience, it's not always just DPI — it's often DPI combined with a firewall. Roskomnadzor can order to block access to a domain, but then they can also specify an IP or a subnet instead. For example, that's what happened when they tried to block Telegram, which does not use DNS at all.
IP Fronting too: https://research.cloudflare.com/publications/Fayed2021/
Looks like it uses WinDivert as the filtering driver, which says on its site that "Windows Server 2016 systems must have secure boot disabled"; not entirely surprising and a reminder of what a lot of the "security" stuff is really securing.
"secure boot must be disabled" is the equivalent of "in case of permissions error, just chmod 777". secure boot literally protects the user first, and is configurable with user provided keys on the vast vast majority of open platform computers (i.e. amd64)
"prevent injection of a driver that can divert all my shit at the kernel level" is exactly what you want secure boot protecting you from. there is no limitation of user rights because the user can turn it off and/or load their own keys at will.
secure boot fear mongering is bullshit nonsense
"prevent injection of a driver that can divert all my shit at the kernel level" is exactly what you want secure boot protecting you from. there is no limitation of user rights because the user can turn it off and/or load their own keys at will.
secure boot fear mongering is bullshit nonsense
secure boot literally protects the user first
No it doesn't. It's merely a convenient excuse to divert attention away from the truth, which is that it prevents users from doing things like defeating DRM and modifying the system to not be so hostile to themselves in other ways.
secure boot fear mongering is bullshit nonsense
Your position is the corporate propaganda.
No it doesn't. It's merely a convenient excuse to divert attention away from the truth, which is that it prevents users from doing things like defeating DRM and modifying the system to not be so hostile to themselves in other ways.
secure boot fear mongering is bullshit nonsense
Your position is the corporate propaganda.
Both are right -- is it so difficult to see that?
"prevent injection of a driver that can divert all my shit at the kernel level" is exactly what you want secure boot protecting you from.
The only thing Secure Boot is doing here is preventing you from loading a driver not blessed by Microsoft. They would happily bless "a driver that can divert all my shit at the kernel level", but it costs too much for the maintainer of WinDivert.
The only thing Secure Boot is doing here is preventing you from loading a driver not blessed by Microsoft. They would happily bless "a driver that can divert all my shit at the kernel level", but it costs too much for the maintainer of WinDivert.
It is kind of sad that no one seems to bother enough to actually learn how to use Secure Boot to their advantage. Everyone is just disabling it the first time it gets in their way. Reminds me of how Firewalls used to be treated like 20 years ago. Yes, by default most implementations will only accept signatures from Microsoft. But the thing is: You can always enroll your own keys. My Laptop is currently booting a non-mainline Linux kernel with secure boot enabled. Just enroll your own Certificate and (automate the process to) sign binaries yourself. I really wonder why no one yet started some kind of project to provide a "community trust root" of some sorts which you could enroll on your machine.
Would it really help in this case? I believe the HLK-signing is enforced by Windows, not UEFI. Uploading your own cert to UEFI won't change that.
I think that for the vast majority of threat models, the insecurity of having your own signing certificate (it needs to be hot if it's going to be automated) is not much different from the risk of not having secure boot at all. Also, the effort v. security balance.
Windows Server 2016+ with Secure Boot enabled won't allow loading of any third-party non-HLK-certified drivers, whether they are signed or unsigned. That's why.
And HLK is only for device drivers, not for any regular drivers, as far as I know.
And HLK is only for device drivers, not for any regular drivers, as far as I know.
Sad to see this downvoted without any response, because as far as I see he's exactly right.
No. That's just propaganda. The purpose is to further vendor lock you. MS has a minute share in the world of OS, completely outclassed by Linux. They're trying to force their way in by fooling people like you that something like Pluton is required.
The only reason they dominate the PC market share is because their spyware OS is installed by default and people don't go around switching OSes.
An anecdote about security; at my workplace, one of the top 5 security firms in the world, secure boot isn't required nor is MS. Makes you wonder.
The only reason they dominate the PC market share is because their spyware OS is installed by default and people don't go around switching OSes.
An anecdote about security; at my workplace, one of the top 5 security firms in the world, secure boot isn't required nor is MS. Makes you wonder.
What? Secure Boot doing anything to force you into Microsoft solutions is a hardware vendor/supplier issue. They're the one that configure the secure boot parameters, and with minor exception (don't buy bad products...) they do not lock you out of booting non-Microsoft solutions.
I fail to see in any way how preventing the loading of unsigned drivers in the secure boot chain is "vendor lock-in".
Furthermore, that signature does _not_ have to be Microsoft's. You can sign a driver with a private CA and provided that signer is in the trust store, it will be loaded.
>at my workplace, one of the top 5 security firms in the world
Cool, my dad works for Nintendo tho. Agree with other poster, not sure if you even know what Secure Boot is. Seems like you read an article on Slashdot about it 10 years ago.
I fail to see in any way how preventing the loading of unsigned drivers in the secure boot chain is "vendor lock-in".
Furthermore, that signature does _not_ have to be Microsoft's. You can sign a driver with a private CA and provided that signer is in the trust store, it will be loaded.
>at my workplace, one of the top 5 security firms in the world
Cool, my dad works for Nintendo tho. Agree with other poster, not sure if you even know what Secure Boot is. Seems like you read an article on Slashdot about it 10 years ago.
I'm not sure if you don't know or don't want to know what SB does, but it's not propaganda, neither is it vendor-locking. Most widespread vendor locks don't use SB, neither is the functionality only usable for that (well duh, it could be, because you can choose your root of trust, but you can choose the root).
You might as well call other tech Linux doesn't play well with "vendor-locking" as this point with no concern, even if there are no real "locks" like that, just lack of support. "TME doesn't work? Literally vendor-locking and Microsoft propaganda!1!"
Security corporations, known for their cargo culting in addition to the usual corporate bullshit, aren't a great example. But bringing then up like they somehow were... makes you wonder indeed.
You might as well call other tech Linux doesn't play well with "vendor-locking" as this point with no concern, even if there are no real "locks" like that, just lack of support. "TME doesn't work? Literally vendor-locking and Microsoft propaganda!1!"
Security corporations, known for their cargo culting in addition to the usual corporate bullshit, aren't a great example. But bringing then up like they somehow were... makes you wonder indeed.
Censorship bypass needn't necessarily be a security solution.
Windows Server 2016+ with Secure Boot enabled won't allow loading of any third-party non-HLK-certified drivers, whether they are signed or unsigned. That's why.
Wow, it's like the average vendor's support of SELinux or AppArmor. "Just turn it off" facepalm
Doesn't work in China, though:
https://github.com/ValdikSS/GoodbyeDPI/issues/123
https://github.com/ValdikSS/GoodbyeDPI/issues/123
If anyone gets me a network link to China, I'm ready to test and implement new techniques.
Last time I asked why such tools are not built in China, the developer of very popular anti-censorship tool told me that there are punishment for censorship circumvention in China, and such tools which openly punch DPI could be more dangerous for the end user in the legal sense.
Last time I asked why such tools are not built in China, the developer of very popular anti-censorship tool told me that there are punishment for censorship circumvention in China, and such tools which openly punch DPI could be more dangerous for the end user in the legal sense.
Websocket tunneling is known to work against the GFW.
Many people reported me that they managed to use wstunnel to bypass
https://github.com/erebe/wstunnel (linux + mac + windows)
https://github.com/erebe/wstunnel (linux + mac + windows)
For that, you need something more like obfsproxy or some other steganographic tunnel, which of course also requires an endpoint on the other side much like a VPN.
> Active DPI is more tricky to fool. Currently the software uses 7 methods to circumvent Active DPI:
> TCP-level fragmentation for first data packet
> TCP-level fragmentation for persistent (keep-alive) HTTP sessions
> Replacing Host header with hoSt
> Removing space between header name and value in Host header
> Adding additional space between HTTP Method (GET, POST etc) and URI
> Mixing case of Host header value
> Sending fake HTTP/HTTPS packets with low Time-To-Live value, incorrect checksum or incorrect TCP Sequence/Acknowledgement numbers to fool DPI and prevent delivering them to the destination
DPI middleboxes are truly terrible. They're incompatible with even basic TCP without any good reason. I wonder if these ISPs use the same vendors as your average "enterprise" network.
> TCP-level fragmentation for first data packet
> TCP-level fragmentation for persistent (keep-alive) HTTP sessions
> Replacing Host header with hoSt
> Removing space between header name and value in Host header
> Adding additional space between HTTP Method (GET, POST etc) and URI
> Mixing case of Host header value
> Sending fake HTTP/HTTPS packets with low Time-To-Live value, incorrect checksum or incorrect TCP Sequence/Acknowledgement numbers to fool DPI and prevent delivering them to the destination
DPI middleboxes are truly terrible. They're incompatible with even basic TCP without any good reason. I wonder if these ISPs use the same vendors as your average "enterprise" network.
> DPI middleboxes are truly terrible. They're incompatible with even basic TCP without any good reason.
Thanks to them things like TCP Fast Open are unfortunately still a rarity.
> I wonder if these ISPs use the same vendors as your average "enterprise" network.
Most likely, yes. Takes a bit too much effort to handle the bandwith necessary, that everyone would be able to do and sell it as a service.
Thanks to them things like TCP Fast Open are unfortunately still a rarity.
> I wonder if these ISPs use the same vendors as your average "enterprise" network.
Most likely, yes. Takes a bit too much effort to handle the bandwith necessary, that everyone would be able to do and sell it as a service.
Is there a network wide DPI circumvention tool that can be easily deployed using Docker?
If you have a server/cheap vpc outside the blocking zone, you can use websocket tunneling.
https://github.com/erebe/wstunnel (linux + mac + windows)
https://github.com/erebe/wstunnel (linux + mac + windows)
It's only for Windows. Linux didn't supported. :(
There is a similar project that supports Linux, but its much more focused on Russian censorship.
https://github.com/bol-van/zapret
https://github.com/bol-van/zapret
It IS supported on Linux, they have a tar file on the downloads page.
https://github.com/ValdikSS/GoodbyeDPI/releases
https://github.com/ValdikSS/GoodbyeDPI/releases
You mean the source tarball? It doesn't work on Linux, it doesn't even compile — the Makefile hardcodes CC to the MinGW compiler. Almost every .c file includes windows.h.
Looking at the project's issue list, they don't support other operating systems.
Looking at the project's issue list, they don't support other operating systems.
That source tarball is autogenerated by github when you make a release and it in no way implies linux compatibility, it is simply a way to download the source code without git.
Is this some kind of bad joke?
The nice thing about this is that passive mode should be undetectable.
Active mode on the other hand will probably not be useful for long because it is basically based in negligent implementation on the the DPI side and trivial to fix.
Active mode on the other hand will probably not be useful for long because it is basically based in negligent implementation on the the DPI side and trivial to fix.
[deleted]
Sorry, might be a dumb question, but isn't this simply a matter of encrypting your packets or using a VPN?
Several issues:
* VPNs are mostly not free, and with current situation in Russia, you can't easily pay for the Europe/US service due to absent Visa/MC service
* Popular VPN providers are getting blocked in Russia recently
* For the major websites, such as Instagram and Twitter, VPN access almost instantly triggers additional checks, cellphone number validation, account block, etc.
* VPN connection increases latency and reduces speed
While GoodbyeDPI is free and autonomous.
AND it can unblock VPNs, too! For example, the latest build can unblock blocked ProtonVPN by inserting fake TLS packet during OpenVPN TCP handshake.
* VPNs are mostly not free, and with current situation in Russia, you can't easily pay for the Europe/US service due to absent Visa/MC service
* Popular VPN providers are getting blocked in Russia recently
* For the major websites, such as Instagram and Twitter, VPN access almost instantly triggers additional checks, cellphone number validation, account block, etc.
* VPN connection increases latency and reduces speed
While GoodbyeDPI is free and autonomous.
AND it can unblock VPNs, too! For example, the latest build can unblock blocked ProtonVPN by inserting fake TLS packet during OpenVPN TCP handshake.
Default user just want to access blocked resources. Free VPN... Can we trust them? Paid VPN are paid.
>Free VPN... Can we trust them? Paid VPN are paid.
Paid VPNs are paid. Can we trust them more?
Paid VPNs are paid. Can we trust them more?
People who live in a country with strict censorship often can't pay foreign vpn company due to the lack of payment processors support. Can't pay with visa/mastercard if you live in Russia or Iran.
[deleted]
Wireguard solves this, no need this
That’s assuming that the wireguard protocol is not blocked by that same DPI that you’re trying to circumvent.
The more tools - the better
Not at all. The protocol has a very well defined structure, and one only has to foul the handshake packets to block connections.
No, it used to as a side-effect, but DPI circumvention is an unpopularity contest... wiregaurd is no longer unpopular enough, DPI targets it, and wiregaurd's purpose is not to play the circumvent DPI game, so it's not going to continually make arbitrary changes to do so.
Witeguard isn't made to do this . Even the author said so . It's blocked in many countries
This tool is great, but I religiously route all my traffic through a VPN that I own and control. I’ve hardened the box I use to have zero logs and I don’t need to blindly trust a commercial provider whether they’ve been audited or not. There’s no way of really knowing they’re not logging in some capacity bar from being physically in their server room and inspecting their setup.
Add to the VPN a DoH resolver that I own and control too, and it makes things even better. I also block port 80 on my machine as an extra measure. No need to be using port 80 in this day and age except for captive portals which I rarely ever have to use.