OpenSSH 9.0(openssh.com)
openssh.com
OpenSSH 9.0
https://www.openssh.com/releasenotes.html
112 comments
We are making [NTRU Prime + x25519 key exchange the default] now (i.e. ahead of cryptographically-relevant quantum computers) to prevent "capture now, decrypt later" attacks where an adversary who can record and store SSH session ciphertext would be able to decrypt it once a sufficiently advanced quantum computer is available.
This situation is actually more complex than the summary implies.
NIST is conducting a competition for post-quantum key exchange and signature algorithms. NTRU Prime did not make the cut as a key exchange finalist.
It appears that NTRU Prime is going ahead in OpenSSH, without any formal endorsement from NIST.
In the notes listing NTRU Prime as an alternate (and rejection as a finalist), Daniel J. Bernstein filed a complaint with his experience at NIST:
https://csrc.nist.gov/Projects/post-quantum-cryptography/rou...
"Formal complaint regarding 8 June 2021 incident - 2021.06.15, Daniel J. Bernstein..."
"Executive summary. A week ago Dr. Daniel Apon from NIST publicly accused me of professional misconduct. Specifically, he accused me of initiating private contact with NIST so as to provide false information to NIST regarding the timing of an upcoming announcement relevant to NIST’s ongoing decisions..."
It is unfortunate that this disfunction has a practical impact upon OpenSSH.
NIST is conducting a competition for post-quantum key exchange and signature algorithms. NTRU Prime did not make the cut as a key exchange finalist.
It appears that NTRU Prime is going ahead in OpenSSH, without any formal endorsement from NIST.
In the notes listing NTRU Prime as an alternate (and rejection as a finalist), Daniel J. Bernstein filed a complaint with his experience at NIST:
https://csrc.nist.gov/Projects/post-quantum-cryptography/rou...
"Formal complaint regarding 8 June 2021 incident - 2021.06.15, Daniel J. Bernstein..."
"Executive summary. A week ago Dr. Daniel Apon from NIST publicly accused me of professional misconduct. Specifically, he accused me of initiating private contact with NIST so as to provide false information to NIST regarding the timing of an upcoming announcement relevant to NIST’s ongoing decisions..."
It is unfortunate that this disfunction has a practical impact upon OpenSSH.
The text of the complaint is available at [0].
[0] https://03283664099418252878.googlegroups.com/attach/6f5422d...
[0] https://03283664099418252878.googlegroups.com/attach/6f5422d...
The feds don't like DJB, he sued them and won to allow us all to publish fun stuff like this.
Looks like Daniel Apon no longer works for NIST, FWIW.
It is well accepted that djb is more trustworthy than NIST, with the latter assisting the NSA to insert kleptographic backdoors into standards.
Is there anywhere I can learn (or learn to learn) about NTRU Prime? I have a fairly decent knowledge of x25519 but in know nothing about lattice-based cryptography. I tried reading the NTRUP paper but most of it goes over my head.
Very future proof! So it is quantum brute force crack proof?
It isn't, you can always brute force the symmetric key used for traffic if you wanted. The handshake itself can also in theory be brute forced if you know all input parameters except the input key.
>sftp(1): add a "cp" command to allow the sftp client to perform server-side file copies.
Well, goes to show how little I know. Been using this for years but apparently not!
Well, goes to show how little I know. Been using this for years but apparently not!
Maybe you have been using scp instead of sftp? Maybe you have been using a different sftp client than the OpenSSH one?
https://superuser.com/questions/686394/scp-between-two-remot...
https://unix.stackexchange.com/questions/8707/whats-the-diff...
https://superuser.com/questions/686394/scp-between-two-remot...
https://unix.stackexchange.com/questions/8707/whats-the-diff...
> Been using this for years but apparently not!
I don't know what you're trying to say here. Perhaps you misunderstood what's been added? Example:
I don't know what you're trying to say here. Perhaps you misunderstood what's been added? Example:
$ sftp HOST
Connected to HOST.
sftp> cp filea fileb
Invalid command.I like how they do new side effect of improvement with downsides. We can get used to this, and then they can deprecate scp/rcp in a future release.
What's actually a convenient way of transferring files across a network if scp is being removed?
Update: Thanks for the replies. I should've done a bit of extra research about scp first. Guess this is an example of Cunningham's Law.
Update: Thanks for the replies. I should've done a bit of extra research about scp first. Guess this is an example of Cunningham's Law.
The scp protocol is being removed, but the scp command you find convenient isn't going anywhere. So that convenience ought not to change, except that now the quoting does what you probably expect, because it's just a local command and so nothing weird is happening.
I'm somewhat of an old fart, but rsync(1) has worked well for me. It integrates well with ssh on both sides, using ssh channels to execute the binary on the server side and transporting data nicely back and forth.
It isn't suited for millions of files, but neither is scp.
It isn't suited for millions of files, but neither is scp.
It handles as many files as I've ever thrown at it - often in the millions. The great thing about rsync is it is restart-able and the source and destination don't even have to be local.
the biggest weakness of rsync is that it's single-threaded. Single-thread-copy with millions of small files is painfully slow, Rsync is as good as it can be but you just need more threads.
I think that's what a sibling is getting into with the "better to tar files and send them over ssh in some cases" thing. And yes, you can hack it in after-the-fact with xargs/etc but it's clunky compared to just having native multithreading like rclone/etc.
I think that's what a sibling is getting into with the "better to tar files and send them over ssh in some cases" thing. And yes, you can hack it in after-the-fact with xargs/etc but it's clunky compared to just having native multithreading like rclone/etc.
If you use GNU Parallel, there's a fairly easy way to parallelize [1].
That being said, I hadn't heard of rclone - thanks for mentioning it, it looks amazing. I'll definitely be trying this out for my use cases...
[1] http://www.gnu.org/software/parallel/man.html#example-parall...
That being said, I hadn't heard of rclone - thanks for mentioning it, it looks amazing. I'll definitely be trying this out for my use cases...
[1] http://www.gnu.org/software/parallel/man.html#example-parall...
Wow, Parallel also looks awesome. Never seen it before.
It handles millions, but it can be a lot faster to just pipe output from tar through the ssh connection.
Got any benchmarks/write ups on the subject? I did a bit of testing myself a long time ago and basically the answer just ended up being to use rsync because any differences were marginal. That said, I didn't test with millions of files.
I think perhaps this was a bigger issue back in the day when we were using rotating harddisks. In those day doing a seek would be a lot slower than doing a write.
Today seeks are mostly instants, so maybe my experience isn't valid anymore.
Today seeks are mostly instants, so maybe my experience isn't valid anymore.
Your experience is valid today but for a different reason: if you're comparing millions of tiny files there's a lot of back and forth. If you're streaming a single archive, it only checks if that single file has been modified.
Like everyone here I've no benchmarks but have got burned trying to rsync around too many small files.
Like everyone here I've no benchmarks but have got burned trying to rsync around too many small files.
I got into the « lot of small files » case, and I ended up generating a file list, split it and feed multiple rsync instances with xargs.
We've put rsync in a HPC scheduler and used it, or some tooling ontop of it really, to copy billions of files for a large-ish HPC compute cluster with many P of data.
That's my experience - millions or billions of files - who cares?
rsync is time served. It just works. I'm sure there are other funky solutions but they are not proven over decades.
rsync is time served. It just works. I'm sure there are other funky solutions but they are not proven over decades.
we have had problems with millions of rsunk files.
Also, rsync can create more faithful copies than scp. E.g. scp can't copy a symlink as a symlink, instead it will follow the symlink and copy the file or directory.
This is one of the main reasons I never use scp and always use rsync -a. IMO, seems a poor design of scp.
A simple unix-like way would be to use tar with plain ssh.
To transfer a set of files from local device to a remote device is as easy as:
To transfer a set of files from local device to a remote device is as easy as:
tar -c path/to/file1 path/to/file2 ... | ssh user@hostname tar -x
-c = create (archive file)
-x = extract (archive file)This could bite you on platforms where tar observes absolute links (commercial UNIX comes to mind).
There is wild variation in tar implementations beyond what is required by POSIX.
There is wild variation in tar implementations beyond what is required by POSIX.
I use "gtar" on the command-line in that case; it either fails because gnu-tar is missing, or it works as expected.
"-H pax" is also an important item to specify on the side creating the tar file, as pax support is much older than pax as a default, and the pre-pax default (gnu or USTAR?) has a short enough maximum path length as to have caused me problems in the past.
"-H pax" is also an important item to specify on the side creating the tar file, as pax support is much older than pax as a default, and the pre-pax default (gnu or USTAR?) has a short enough maximum path length as to have caused me problems in the past.
Or dd | tar | ssh so you can control the size of compressed data (as in very large files)
https://www.linode.com/docs/guides/copying-a-disk-image-over...
https://www.linode.com/docs/guides/copying-a-disk-image-over...
The scp tool continues to exist, but implementation switches to SFTP as protocol from SCP protocol.
They plan to deprecate scp the protocol, not scp the tool, which will use SFTP under the hood.
the only snag about this plan is that the old `scp` tool works with any server, whereas the new tool will require additional configuration on the server to enable SFTP.
It's not enabled by default in some distributions and many "how to secure your server" guides recommend disabling it/keeping it disabled due to the best practice of only enabling necessary services.
This will break `scp` for many corporate cases (where arguing that sftp is necessary is an impossible fight)
It's not enabled by default in some distributions and many "how to secure your server" guides recommend disabling it/keeping it disabled due to the best practice of only enabling necessary services.
This will break `scp` for many corporate cases (where arguing that sftp is necessary is an impossible fight)
You're actually making the opposite argument that you think you're making: if the scp command stops working, from a "corporate" perspective, from all intents and purposes, it means that "our server no longer works with scp due to an update". Regardless of implementation details, the opposite will then happen: whoever is in charge will request for "SCP to be fixed", which will actually mean turning on SFTP support.
Removing options from "corporate" users usually results in saner choices being made, because you stop being able to bikeshed the implementation details. People have a server and they need a way to transfer files. If you have two protocols that can be used, arbitrary decisions will be made. If the readily available tools all break, there is no more decision to be made – whatever needs to be enabled will be enabled.
Removing options from "corporate" users usually results in saner choices being made, because you stop being able to bikeshed the implementation details. People have a server and they need a way to transfer files. If you have two protocols that can be used, arbitrary decisions will be made. If the readily available tools all break, there is no more decision to be made – whatever needs to be enabled will be enabled.
if regular ssh is available, nothing stops people from building pipes with tar on both ends. This will be integrated into additional tools (future "show HN - scp replacement written in $FANCY_LANGUAGE that works even when sftp is disabled") and will be the "new way" of doing things because it won't require any configuration change on the server.
Never underestimate the inertia of server configurations.
Never underestimate the inertia of server configurations.
> (future "show HN - scp replacement written in $FANCY_LANGUAGE that works even when sftp is disabled")
Too late:
https://en.wikipedia.org/wiki/Files_transferred_over_shell_p...
In KDE this is implemented as a standard protocol handler ("ioslave") and you can access it from most KDE apps using URLs like "fish://user@server/":
https://docs.kde.org/trunk5/en/kio-extras/kioslave5/fish/ind...
Too late:
https://en.wikipedia.org/wiki/Files_transferred_over_shell_p...
In KDE this is implemented as a standard protocol handler ("ioslave") and you can access it from most KDE apps using URLs like "fish://user@server/":
https://docs.kde.org/trunk5/en/kio-extras/kioslave5/fish/ind...
The snag in tbis plan is that wildcard copies dont work in the new scp tool. Serious usability downgrade to the new system. :(
The fact scp "worked" illustrates that such "How to secure your server" guides are... at best... poorly thought out. If "remote file access" is not in fact a "necessary service" then you can't very well allow scp to work, and thus you can't allow basic ssh service.
Actually a more sane setup, which I've used, is only SFTP is enabled, no shells. You can fetch files, you can send files, but you can't... for example... run a race exploitation shell script to seize root permissions.
This also has the advantage that if we can only send and receive files, the "server" we're connecting to needn't really exist, like an HTTP virtual host it's just a bunch of "files" and could actually be backed by a database with some blob tables.
There's a lot of scp usage in a typical corporate environment that should actually be files living in a shared filestore anyway, I've seen way too many files which ought to actually be:
* A wiki entry
* Checked into a git repository
* Shared in some cloud service e.g. OneCloud
... but instead they live on Steve's F:\Documents and if Steve is off sick well, here's the copy I made last week, I hope it isn't missing any important updates and somebody make sure Steve gets this when we change it...
The file living in /usr/project/ on testserver03 instead of F:\Documents on Steve's laptop is not a real improvement.
Actually a more sane setup, which I've used, is only SFTP is enabled, no shells. You can fetch files, you can send files, but you can't... for example... run a race exploitation shell script to seize root permissions.
This also has the advantage that if we can only send and receive files, the "server" we're connecting to needn't really exist, like an HTTP virtual host it's just a bunch of "files" and could actually be backed by a database with some blob tables.
There's a lot of scp usage in a typical corporate environment that should actually be files living in a shared filestore anyway, I've seen way too many files which ought to actually be:
* A wiki entry
* Checked into a git repository
* Shared in some cloud service e.g. OneCloud
... but instead they live on Steve's F:\Documents and if Steve is off sick well, here's the copy I made last week, I hope it isn't missing any important updates and somebody make sure Steve gets this when we change it...
The file living in /usr/project/ on testserver03 instead of F:\Documents on Steve's laptop is not a real improvement.
I'm pretty sure scp isn't a protocol. It is just a tool that relies on doing a request for execution over the regular ssh2 protocol.
I've written a few custom ssh servers and you don't really need to do anything to them to support scp.
sftp uses the sftp subsystem. scp just uses regular ssh requests and transports data over the ssh channels.
I've written a few custom ssh servers and you don't really need to do anything to them to support scp.
sftp uses the sftp subsystem. scp just uses regular ssh requests and transports data over the ssh channels.
See <https://en.wikipedia.org/wiki/Secure_copy_protocol>
SFTP also transports data over SSH channels.
SFTP also transports data over SSH channels.
I guess you can call it a (application) protocol. However, when implementing this I've not needed to anything to make scp works as opposed to anything else. Just supporting ssh executions requests and connecting the channels as you do with any remote invocation.
[deleted]
The protocol is being deprecated not the scp command. It's been reimplemented over sftp. They have had to handle wildcard/glob expansion, and how scp tried to deal with it.
TL;DR it's just the same command, mostly, for now.
TL;DR it's just the same command, mostly, for now.
[deleted]
scp the protocol is being replace by the sftp protocol, scp the tool will use that new protocol
sftp, rsync, scp using the sftp protocol in the background (which is now the default).
scp and sftp make sense, but is that really the case for rsync?
As far as I remember, rsync does a few neat tricks like hashing files on both sides of a connection to determine whether it's worth transferring them over a potentially slow connection, which would not work with pure sftp.
As far as I remember, rsync does a few neat tricks like hashing files on both sides of a connection to determine whether it's worth transferring them over a potentially slow connection, which would not work with pure sftp.
You're correct, rsync operates over SSH (or its own native protocol) but runs a copy of the rsync binary on the remote side rather than using the SFTP subsystem.
I would love an stb single header style version of this. I had to build this recently and it was a fucking nightmare.
[deleted]
Before this thread gets too long, there is already a lengthly discussion in play.[1]
Hint for the future : There's a search box on the bottom of HN. ;-)
[1]https://news.ycombinator.com/item?id=30954800
Hint for the future : There's a search box on the bottom of HN. ;-)
[1]https://news.ycombinator.com/item?id=30954800
Merged hither. Thanks!
(The current thread was posted earlier - you can tell from the item IDs.)
(The current thread was posted earlier - you can tell from the item IDs.)
[deleted]
Ironically, that post is marked as dupe atm.
Well, the dupe marked post is hours younger than this post.
Because it was posted after this post.
Ironically what we needed was a hint for the past, not for the future.
[deleted]
'Release X' threads are mostly repetitive anyway, like an infinite dupe. A subscription to dupery.
https://news.ycombinator.com/item?id=30937469
https://news.ycombinator.com/item?id=30937469
That would make us confederacy of dunces hawking free lifetime subscriptions to dupery, no less. I think you’re making the right call here, on the whole.
You should post your hint to the other thread as it's the dupe.
That was posted after this post...
indeed. he should have used that. I was first, because when I updated my openssh mirror this morning, I was pleasently surprised by this.
One nice thing about recent versions of SSH is that they include support for signing arbitrary messages with your ssh key. I know you're not supposed to re-use keys but there have been situations where I needed to prove my identity to people who only had my ssh public key. This would have been super useful in the past.
The OpenSSH arbitrary signing comes with a rationale for why this is actually safe. Basically they know these keys are intended for SSH, for which they will sign messages with a particular structure, and so they can choose the structure signed for the "arbitrary data signing" feature so that the two are necessarily disjoint - so no possible "arbitrary data" signature can be confused for a "I'm signing in to this SSH server" signature, and you're at no risk.
Thus, even if you allow some adversary to literally pick the arbitrary data, when you'll sign that data and so on, your signing it cannot possibly allow them to impersonate you to a SSH server.
This is easier to pull off than the rationale for key re-use in encryption because signing opaque blobs is a neutral action your SSH client already does, if adversaries could learn your keys or whatever from seeing you sign such a blob, then SSH authentication itself would be unsafe already. In contrast decrypting data an adversary sent to you might reveal something, especially if you can be persuaded (as happened for HTTPS with older TLS versions and most popular implementations) to tell the adversary what happened when you tried so this will usually be dangerous and a rationale for why it's safe must be thorough if we want non-experts to do it.
Thus, even if you allow some adversary to literally pick the arbitrary data, when you'll sign that data and so on, your signing it cannot possibly allow them to impersonate you to a SSH server.
This is easier to pull off than the rationale for key re-use in encryption because signing opaque blobs is a neutral action your SSH client already does, if adversaries could learn your keys or whatever from seeing you sign such a blob, then SSH authentication itself would be unsafe already. In contrast decrypting data an adversary sent to you might reveal something, especially if you can be persuaded (as happened for HTTPS with older TLS versions and most popular implementations) to tell the adversary what happened when you tried so this will usually be dangerous and a rationale for why it's safe must be thorough if we want non-experts to do it.
> In contrast decrypting data an adversary sent to you might reveal something, especially if you can be persuaded (as happened for HTTPS with older TLS versions and most popular implementations) to tell the adversary what happened when you tried so this will usually be dangerous and a rationale for why it's safe must be thorough if we want non-experts to do it.
What exactly happened for HTTPS with older TLS versions? Sounds like you’re alluding to some sort of oracle attack.
What exactly happened for HTTPS with older TLS versions? Sounds like you’re alluding to some sort of oracle attack.
A sort of oracle, yes. I believe there's a vulnerability with servers that are willing to attempt RSA kex (which no longer exists in TLS 1.3 and has been a bad idea for many years), where you can (at least in principle) arrange for them to do an operation which either reveals some bits of their private RSA key (extremely bad) or is equivalent to RSA signing a message of your choosing. RSA kex involves the server attempting to RSA decrypt messages chosen by an adversary.
It's fuzzy, perhaps somebody will remind me of the specifics.
It's fuzzy, perhaps somebody will remind me of the specifics.
> there have been situations where I needed to prove my identity to people who only had my ssh public key
Just wondering what kind of situation can this be where someone has only your ssh public key? Usually people know each other via say, github user names or irc handles but ssh public keys? (genuinely curious)
Just wondering what kind of situation can this be where someone has only your ssh public key? Usually people know each other via say, github user names or irc handles but ssh public keys? (genuinely curious)
For github: https://github.com/$user.keys
Every time I pay enough attention to an OpenSSH release that I trick myself into reading some of the code, I'm left smh in disbelief that I actually rely on this software still.
Can you tell why that is? Did you find the OpenSSH source code especially bad? I am under the belief that OpenSSH is a really high quality, security focused software project.
> Can you tell why that is?
Yeah, I too find it a really bizarre comment to make, especially as it is not substantiated with any detail whatsoever.
OpenSSH comes from the house of OpenBSD and LibreSSL.
I am no security guru, but as I understand it, all three have a well deserved security reputation, and the core maintainers run a tight ship.
We've already covered in recent weeks on HN how OpenSSH has maintained its robustness in the face of an increasingly hostile cyber world.[1]
So, if you are going to imply that OpenSSH code is shit, back it up with hard facts.
[1] https://news.ycombinator.com/item?id=30830749
Yeah, I too find it a really bizarre comment to make, especially as it is not substantiated with any detail whatsoever.
OpenSSH comes from the house of OpenBSD and LibreSSL.
I am no security guru, but as I understand it, all three have a well deserved security reputation, and the core maintainers run a tight ship.
We've already covered in recent weeks on HN how OpenSSH has maintained its robustness in the face of an increasingly hostile cyber world.[1]
So, if you are going to imply that OpenSSH code is shit, back it up with hard facts.
[1] https://news.ycombinator.com/item?id=30830749
> I am no security guru, but as I understand it, all three have a well deserved security reputation, and the core maintainers run a tight ship.
Among us security gurus, OpenSSL has a really bad reputation. It's a giant pile of spaghetti code with a huge history of critical system vulnerabilities stemming from trying to be everything to everybody. Bitcoin is in the process of writing replacement functionality in order to remove OpenSSL entirely from its dependency tree because they got burned by major bugs multiple times in the past. A lot of modern alternatives like libsodium list "not OpenSSL" as a selling point. The situation is so bad that multiple organizations have forked OpenSSL to create stripped down alternatives (BoringSSL, LibreSSL) that are more likely to be secure.
I know less about the OpenSSH code base specifically, except that it came out of the same culture and the same developers. I wouldn't use it in new projects unless I had no other choice.
Among us security gurus, OpenSSL has a really bad reputation. It's a giant pile of spaghetti code with a huge history of critical system vulnerabilities stemming from trying to be everything to everybody. Bitcoin is in the process of writing replacement functionality in order to remove OpenSSL entirely from its dependency tree because they got burned by major bugs multiple times in the past. A lot of modern alternatives like libsodium list "not OpenSSL" as a selling point. The situation is so bad that multiple organizations have forked OpenSSL to create stripped down alternatives (BoringSSL, LibreSSL) that are more likely to be secure.
I know less about the OpenSSH code base specifically, except that it came out of the same culture and the same developers. I wouldn't use it in new projects unless I had no other choice.
> I know less about the OpenSSH code base specifically, except that it came out of the same culture and the same developers
ummm what? Afaik openssh and openssl have no major common history? openssh comes from openbsd, openssl has always been independent project.
ummm what? Afaik openssh and openssl have no major common history? openssh comes from openbsd, openssl has always been independent project.
If you think this [0] is quality workmanship, I never want to work on your team.
I haven't even exhaustively looked through this code, only took a quick glance at clientloop.c [1] and was curious about how the channels were integrated into the poll loop.
This is a mess, and I'm pretty sure I already see a bug.
[0] https://github.com/openssh/openssh-portable/blob/master/chan...
[1] https://github.com/openssh/openssh-portable/blob/master/clie...
Edit: I'm not looking at a future CVE, but I've definitely found a bug. For those of you tossing up low-effort straw men, actually go grok the code. The way they've integrated the poll loop into all the actual handling of the returned events is a sprawling mess, and the bug I've found is a direct result of that.
I haven't even exhaustively looked through this code, only took a quick glance at clientloop.c [1] and was curious about how the channels were integrated into the poll loop.
This is a mess, and I'm pretty sure I already see a bug.
[0] https://github.com/openssh/openssh-portable/blob/master/chan...
[1] https://github.com/openssh/openssh-portable/blob/master/clie...
Edit: I'm not looking at a future CVE, but I've definitely found a bug. For those of you tossing up low-effort straw men, actually go grok the code. The way they've integrated the poll loop into all the actual handling of the returned events is a sprawling mess, and the bug I've found is a direct result of that.
Have you created a new issue ticket for it on GitHub? If I'd spotted a bug, and felt strongly enough to comment on HN about it, opening a new ticket would be my next move.
Hi Greg, that's the first thing I tried to do. Alas they have issues disabled on the github repo; https://github.com/openssh/openssh-portable/issues redirects to /pulls.
So I cloned the repo to start looking more closely at the code in a real editor with an eye towards making a PR.
Fortunately the nature of the bug I've found seems unlikely to be a security issue, it's a busy-loop CPU-burning bug in clientloop.c's poll() integration.
So I cloned the repo to start looking more closely at the code in a real editor with an eye towards making a PR.
Fortunately the nature of the bug I've found seems unlikely to be a security issue, it's a busy-loop CPU-burning bug in clientloop.c's poll() integration.
Good point on the /issues redirect. They have a separate bug tracker it seems [0]. Thanks for taking the time to look into it. This sort of thing is what makes open source software great.
P.S. I'm not Greg (added it to my profile->about section to clarify).
[0] https://www.openssh.com/report.html
P.S. I'm not Greg (added it to my profile->about section to clarify).
[0] https://www.openssh.com/report.html
Out of curiosity, can you explain what's wrong with that code? I dabbled in C a few years back but I'm nowhere near competent enough to find anything wrong with it.
I'm also no C guru, but this seems to dereference a pointer before checking if it's in-bounds:
This is similar to a famous Linux kernel bug from ages ago, where a !NULL check was optimized away in such a situation, leading to an exploit: https://lwn.net/Articles/342330/
In this case I'm not aware if something like that happens, or if it happens to be harmless.
struct pollfd *pfd = &pfds[p];
if (fd == -1)
return;
if (p == -1 || (u_int)p >= npfd)
fatal_f("channel %d: bad pfd %d (max %u)", c->self, p, npfd);
dump_channel_poll(__func__, what, c, p, pfd);
This is similar to a famous Linux kernel bug from ages ago, where a !NULL check was optimized away in such a situation, leading to an exploit: https://lwn.net/Articles/342330/
In this case I'm not aware if something like that happens, or if it happens to be harmless.
The first line is not actually dereferencing anything - it's taking the address of the pointer, and returning another pointer with offset p from the original one.
As far as I know, C allows arbitrary pointer arithmetic (including pointing to an out-of-bounds address) as long as you don't read/write into the invalid memory. If the p check fails, nothing will be done with pfd.
If I'm wrong, please correct me.
As far as I know, C allows arbitrary pointer arithmetic (including pointing to an out-of-bounds address) as long as you don't read/write into the invalid memory. If the p check fails, nothing will be done with pfd.
If I'm wrong, please correct me.
Unfortunately you are not correct. Pointer arithmetic is only valid if the resulting calculated address is within the bounds of the same array as the original pointer, or calculates an address that is just past the end of the array. Otherwise it is our old friend undefined behavior. See section 6.5.6 of the C11 spec.
Given that the function fd_ready is static, it is limited to calls within this translation unit. It looks like the function which calls fd_ready has already confirmed that p is a valid index.
This:
struct pollfd *pfd = &pfds[p];
struct pollfd *pfd = pfds + p;
`pfds + p` is only defined if both the original pointer and the result pointer are pointing at elements of the same array or one past the end of that array. Note that executing `pfds - 1` when `pfds` points at the first element of an array is undefined behaviour and may fail on some platforms.
https://en.cppreference.com/w/c/language/operator_arithmetic...
https://en.cppreference.com/w/c/language/operator_arithmetic...
It's not undefined behavior, but it is implementation-defined behavior [0]:
Also note that you've quoted C++ reference. Just another example of where C++ stops being "compatible" with C.
Indirection through an invalid pointer value and passing an invalid pointer value to a deallocation function have undefined behavior. Any other use of an invalid pointer value has implementation-defined behavior.
Also note that you've quoted C++ reference. Just another example of where C++ stops being "compatible" with C.
cppreference.com hosts references for both the C and C++ languages. My quote is from the C language part of the site.
Furthermore, we are not discussing operations on an invalid pointer value. We are talking about the addition operator operating with a pointer value and when its behaviour is defined.
Lastly, it is your reference here that is to a C++ language draft, not the C language.
Furthermore, we are not discussing operations on an invalid pointer value. We are talking about the addition operator operating with a pointer value and when its behaviour is defined.
Lastly, it is your reference here that is to a C++ language draft, not the C language.
What is the difference between "operations with an invalid pointer value" and "addition operator operating with a pointer value"?
EDIT (since I can't reply yet):
Official C standard documents are surprisingly hard to come by. The only place I've found one was genesis. Here's a direct quote about pointer arithmetic from ISO/IEC 9899: 1990, page 47:
---
Maybe I just suck at reading technical documents, but I can't figure out which part says that storing an invalid pointer is undefined behavior. Would you care to point it out for me?
EDIT (since I can't reply yet):
Official C standard documents are surprisingly hard to come by. The only place I've found one was genesis. Here's a direct quote about pointer arithmetic from ISO/IEC 9899: 1990, page 47:
---
When an expression that has integral type is added to or subtracted from a pointer, the result has the type of the pointer operand.
If the pointer operand points to an element of an array object, and the array is large enough, the result points to an element offset from the original element such that the difference of the subscripts of the resulting and original array elements equals the integral expression. In other words, if the expression P points to the i-th element of an array object, the expressions (P)+N (equivalently. N+(P) ) and (P)-N (where N has the value n) point to, respectively, the i+n-th and i-n-th elements of the array object, provided they exist.
Moreover, if the expression P points to the last element of an array object, the expression (P)+1 points one past the last element of the array object, and if the expression Q points one past the last element of an array object, the expression (Q)-1 points to the last element of the array object.
If both the pointer operand and the result point to elements of the same array object, or one past the last element of the array object, the evaluation shall not produce an overflow; otherwise, the behavior is undefined.
Unless both the pointer operand and the result point to elements of the same array object, or the pointer operand points one past the last element of an array object and the result points to an element of the same array object, the behavior is undefined if the result is used as an operand of the unary * operator.
Maybe I just suck at reading technical documents, but I can't figure out which part says that storing an invalid pointer is undefined behavior. Would you care to point it out for me?
If both the pointer operand and the result point to elements of the same array object, or one past the last element of the array object, the evaluation shall not produce an overflow; otherwise, the behavior is undefined.
In our example here the argument `pfds` to the addition operator in the expression `pfds + p` is (presumably) a perfectly valid pointer. And `p` is a perfectly valid `int`. So there are no invalid pointers passed as arguments to the addition operation. You might think the result of the addition expression is an invalid pointer, but the result is simply undefined behaviour.
Typically an invalid pointer is created by deallocating some memory, which transforms previously valid pointers to the memory into invalid pointers without the pointer values themselves changing their value.
Typically an invalid pointer is created by deallocating some memory, which transforms previously valid pointers to the memory into invalid pointers without the pointer values themselves changing their value.
Ah, okay.
Personally I'd still want to do the check before, because someone could still be adding something in-between the assignment and the check, but if that's the order it's resolved in it seems to be working now.
(I'm guessing they declare and assign the variable first because of C version constraints - wasn't that something that changed in C99?)
Personally I'd still want to do the check before, because someone could still be adding something in-between the assignment and the check, but if that's the order it's resolved in it seems to be working now.
(I'm guessing they declare and assign the variable first because of C version constraints - wasn't that something that changed in C99?)
Yes; it's C99 that let you declare variables anywhere in a function, rather than only before any statements. So it could be rewritten as:
if (fd == -1)
return;
if (p == -1 || (u_int)p >= npfd)
fatal_f("channel %d: bad pfd %d (max %u)", c->self, p, npfd);
struct pollfd *pfd = &pfds[p];
dump_channel_poll(__func__, what, c, p, pfd);
You don't need C99 for this, just some extra braces:
if (fd == -1)
return;
if (p == -1 || (u_int)p >= npfd)
fatal_f("channel %d: bad pfd %d (max %u)", c->self, p, npfd);
{
struct pollfd *pfd = &pfds[p];
dump_channel_poll(__func__, what, c, p, pfd);
}
For consideration, "declare variable" is just that; one could still choose to only assign to it after having done the validation (assuming fatal_f causes control flow termination)
struct pollfd *pfd;
if (p == -1 || (u_int)p >= npfd)
fatal_f("channel %d: bad pfd %d (max %u)", c->self, p, npfd);
pfd = &pfds[p];
A good compiler would produce the same code in both cases.
I don't see anything so obviously wrong with [0].
I guess I'll never make it into your team ¯\_(ツ)_/¯
I guess I'll never make it into your team ¯\_(ツ)_/¯
It's one of the most deployed pieces of software on the planet, and for that reason has a million eyes on it, yet there doesn't seem to be any alarming concern about the source quality. Maybe it's all right after all...
Maybe. You've expressed the same sentiment as you'd have seen for OpenSSL just a few months before everybody decided OpenSSL was unacceptable and they'd fork it (producing BoringSSL, LibreSSL and so on).
OpenBSD has some very peculiar preferences, some of which are in step with modern thinking (e.g. they like sandboxes, privilege dropping and defence in depth very much) but some very much are not (they like C despite its lack of Type Safety, Memory Safety, and generally just Safety)
SecSH (the SSH v2 standard) is a fairly clean modern encrypted protocol design. You do a key exchange, you bring up an encrypted link and then everything else happens inside this, alleviating many of your security concerns, the same way TLS 1.3 works (but TLS 1.3 is much more recent).
But the OpenSSH codebase dates back to even before SSHv2, it's basically the original SSH codebase, from back when encrypted remote shells was an entirely new idea, the mid-1990s, the SSH codebase went proprietary and OpenSSH is the fork. That's some crufty old code, and in C that's not an insignificant problem because problems in code you never read has potentially drastic consequences for unrelated code in the same binary.
OpenBSD has some very peculiar preferences, some of which are in step with modern thinking (e.g. they like sandboxes, privilege dropping and defence in depth very much) but some very much are not (they like C despite its lack of Type Safety, Memory Safety, and generally just Safety)
SecSH (the SSH v2 standard) is a fairly clean modern encrypted protocol design. You do a key exchange, you bring up an encrypted link and then everything else happens inside this, alleviating many of your security concerns, the same way TLS 1.3 works (but TLS 1.3 is much more recent).
But the OpenSSH codebase dates back to even before SSHv2, it's basically the original SSH codebase, from back when encrypted remote shells was an entirely new idea, the mid-1990s, the SSH codebase went proprietary and OpenSSH is the fork. That's some crufty old code, and in C that's not an insignificant problem because problems in code you never read has potentially drastic consequences for unrelated code in the same binary.
The reason they use C is as you said it's a fork. Being able to compile to almost anything is another factor I'd expect.
That would be an obvious reason why their SSH implementation is in C. But for example their ACME implementation (to get certificates from e.g. Let's Encrypt) is also in C, even though various more famous ACME clients that were started earlier and have better features are in more modern languages like Python. That C ACME client complies with OpenBSD's other preferences, it is privilege separated resulting in a bunch of separate programs running to handle different privileges for example, but it is written in C and lacks some obvious security features that aren't platform specific.
Thus on OpenBSD a separate program handles your private key data from the one talking to the network. But, if you used a more capable ACME implementation for other platforms written in say, bash, a completely separate piece of software even on another machine can be talking to the network since that's just how the certificate process works. OpenBSD does not support this (last time I looked).
Thus on OpenBSD a separate program handles your private key data from the one talking to the network. But, if you used a more capable ACME implementation for other platforms written in say, bash, a completely separate piece of software even on another machine can be talking to the network since that's just how the certificate process works. OpenBSD does not support this (last time I looked).
> You've expressed the same sentiment as you'd have seen for OpenSSL just a few months before everybody decided OpenSSL was unacceptable and they'd fork it
I remember recurring complaints about the state of OpenSSL for years before e.g. mBed TLS and LibreSSL happened, but granted it's a decent example.
> it's basically the original SSH codebase
Would you be able to prove that by showing a bisection of the current sources and one of the releases from 22 years ago? I'd be surprised if not almost every single function in the code base hadn't seen significant changes.
I remember recurring complaints about the state of OpenSSL for years before e.g. mBed TLS and LibreSSL happened, but granted it's a decent example.
> it's basically the original SSH codebase
Would you be able to prove that by showing a bisection of the current sources and one of the releases from 22 years ago? I'd be surprised if not almost every single function in the code base hadn't seen significant changes.
OpenSSH is very well regarded in security community, has seen no major vulnerability for a long time.
[deleted]