Curl 8.0.0(daniel.haxx.se)
daniel.haxx.se
Curl 8.0.0
https://daniel.haxx.se/blog/2023/03/20/curl-8-0-0-is-here/
72 comments
Service comment: Daniel was live at https://www.twitch.tv/curlhacker.
How many applications will delay getting those security patches because they assume that 8.x is a breaking change?
There's probably a fair number of setups that more-or-less automatically upgrade to the next minor version, but not the next major.
There's probably a fair number of setups that more-or-less automatically upgrade to the next minor version, but not the next major.
> There is no API nor ABI break in this version.
I don’t know why, but this really, really bothers me. Why even use semver then?
I don’t know why, but this really, really bothers me. Why even use semver then?
Semver 2.0 on major version:
“Major version X (X.y.z | X > 0) MUST be incremented if any backwards incompatible changes are introduced to the public API. It MAY also include minor and patch level changes. Patch and minor versions MUST be reset to 0 when major version is incremented.”
Curl on changes in 8.0.0:
“There is only one actual “change” in this release. This is the first curl release to drop support for building on a systems that lack a working 64 bit data type. curl now requires that ‘long long‘ or an equivalent exists.”
This is a backwards-incompatible breaking change, though not to the public API (but I believe semver as used is understood to include ABI, supported environments, etc), and it’s likely only “theoretically” breaking (are there any existing platforms that don’t have 64-bit support that did build curl, and now can’t? Looks like Daniel posed this question back in Sep 22 and found there were not https://curl.se/mail/lib-2022-09/0099.html).
So semver is not respected to the letter (semver specifies that only public API matters), but it is respected in spirit (the breaking change is in supported build environments, which people do care about when picking versions), but it is also the smallest possible unit of change that still counts as a breaking change, so it’s a bit of a technicality as well.
Personally I’m quite satisfied with it, it takes a decent understanding of semver to produce a case this finely balanced on the edge of two technicalities, which gives me the impression of being playful with semver rather than disrespecting, or being unaware of, semver. YMMV
“Major version X (X.y.z | X > 0) MUST be incremented if any backwards incompatible changes are introduced to the public API. It MAY also include minor and patch level changes. Patch and minor versions MUST be reset to 0 when major version is incremented.”
Curl on changes in 8.0.0:
“There is only one actual “change” in this release. This is the first curl release to drop support for building on a systems that lack a working 64 bit data type. curl now requires that ‘long long‘ or an equivalent exists.”
This is a backwards-incompatible breaking change, though not to the public API (but I believe semver as used is understood to include ABI, supported environments, etc), and it’s likely only “theoretically” breaking (are there any existing platforms that don’t have 64-bit support that did build curl, and now can’t? Looks like Daniel posed this question back in Sep 22 and found there were not https://curl.se/mail/lib-2022-09/0099.html).
So semver is not respected to the letter (semver specifies that only public API matters), but it is respected in spirit (the breaking change is in supported build environments, which people do care about when picking versions), but it is also the smallest possible unit of change that still counts as a breaking change, so it’s a bit of a technicality as well.
Personally I’m quite satisfied with it, it takes a decent understanding of semver to produce a case this finely balanced on the edge of two technicalities, which gives me the impression of being playful with semver rather than disrespecting, or being unaware of, semver. YMMV
I don't think it violates SEMVER.
> It MAY also include minor and patch level changes.
I ultimately understand this as "I can release a patch as a major if I feel like it"
> It MAY also include minor and patch level changes.
I ultimately understand this as "I can release a patch as a major if I feel like it"
Seems like an ambiguity in semver, actually.
I understood that to mean you don’t have to break updates into their major, minor, and patch changes. Like if you’re at 7.88.2, you don’t have to put all your patches into update 7.88.3, then ten minutes later release 7.89.0 with your minor changes, and then ten minutes after that release 8.0.0 with just the major breaking changes. You can instead bundle them all as 8.0.0 because a major change can contain minor changes and patches, and besides it’s far more convenient to not have to break apart the update.
I do appreciate the logic of your understanding, that “breaking change therefore major version bump” does not imply “major version bump therefore breaking change”.
I understood that to mean you don’t have to break updates into their major, minor, and patch changes. Like if you’re at 7.88.2, you don’t have to put all your patches into update 7.88.3, then ten minutes later release 7.89.0 with your minor changes, and then ten minutes after that release 8.0.0 with just the major breaking changes. You can instead bundle them all as 8.0.0 because a major change can contain minor changes and patches, and besides it’s far more convenient to not have to break apart the update.
I do appreciate the logic of your understanding, that “breaking change therefore major version bump” does not imply “major version bump therefore breaking change”.
It clearly says that breaking changes MUST be released in a major version. And it also says nothing about how “minor” and “patch” changes alone MUST NOT be released in a major version. So releasing only “minor” and “patch” changes is fine according to my interpretation of it.
I would expect to see guidance exactly for that claim. I don't see it as ambiguous, I see it as a misunderstanding. The rules are not "must not contain no breaking changes".
> semver specifies that only public API matters
People get really hung up on what is, or isn't "technically" a change to the API. "This now breaks the build" is a change to the public API in the only sense that matters.
People get really hung up on what is, or isn't "technically" a change to the API. "This now breaks the build" is a change to the public API in the only sense that matters.
> Personally I’m quite satisfied with it, it takes a decent understanding of semver to produce a case this finely balanced on the edge of two technicalities, which gives me the impression of being playful with semver rather than disrespecting, or being unaware of, semver. YMMV
What?
> > We decided it was about time to reset the minor number down to more a manageable level and doing it exactly on curl’s 25th birthday made it extra fun. There is no API nor ABI break in this version.
Intentions-wise this has nothing to do with the Venerable SemVer.
What?
> > We decided it was about time to reset the minor number down to more a manageable level and doing it exactly on curl’s 25th birthday made it extra fun. There is no API nor ABI break in this version.
Intentions-wise this has nothing to do with the Venerable SemVer.
They wanted to move to 8.0.0 on curl’s 25th birthday, so they made the smallest possible breaking change they could (changing which environments curl can build in), thus justifying a major version bump.
“There is no API nor ABI break in this version.”, he wrote.
First: You should never trust semver anyways, not where it matters. Second: even in semver, you can bump all the way you want, especially the major version. Just don't skip bumping when you break stuff.
Being able to trust it is the entire raison d'être of semver. Everyone keeps complaining about it, but it's working splendid for hundreds of thousands of code bases.
It's best effort, and can make your life easier, but please treat it that way (best effort). For everything that matters, review and test even revision bumps.
With third-party code, you're always relying on the best effort of others. Of course a version number doesn't relieve you of the need to run tests on your code base, but the probability for semver issues and code bugs should be about equally high - so there's no reason to trust one but not the other.
I think it can be okay in ecosystems where developers understand and respect semver
AFAIK, semantic versioning does not prohibit you from voluntarily increasing a version number at a more "important" place than the one you need to increase considering the changes you did. It just requires that you increase it if you do "hard" API/ABI changes that mandate an increase.
The line before the one you quoted gives their explanation: "We decided it was about time to reset the minor number down to more a manageable level and doing it exactly on curl’s 25th birthday made it extra fun."
I read that too, but I feel like a project as widely used as curl should be more “conservative” in their version increments.
Well linux incremented from 2 to 3 mostly because they felt like it, so they are in good company.
They really are: https://daniel.haxx.se/blog/2013/03/23/why-no-curl-8/
Well, Windows jumped from 4.0 to 2000
Internally Windows 2000 was WinNT 5.0. The marketing name doesn’t have to reflect software versions.
Why? It's just a number.
Major instead of patch is far better than patch instead of major. Nothing breaks due to this new version.
> Why even use semver then?
They don’t?
They don’t?
Well yeah, but it looks like semver. Again, I don’t know why it bothers me, but it just... does.
That’s a version number. Semver is a way of versioning. Now, saying "it bothers me that they don’t use semver" (which has them in the company of other small projects like typescript or the linux kernel) is a statement you can make, but "why use semver" when they don’t makes little sense.
True. I could have phrased that better. I’ll take this as a sign to go to bed. Good night!
Semver is a set of conventions on an older versioning scheme. The semver 1.0.0 spec (I think from 2010?) even points out:
> This is not a new or revolutionary idea. In fact, you probably do something close to this already. The problem is that “close” isn’t good enough. Without compliance to some sort of formal specification, version numbers are essentially useless for dependency management.
Thus, X.Y.Z does not automatically mean "semver". Nor does even X.0.0 mean semver.
To give some 20th century counter-examples found through Google Scholar:
"System Description: Spass Version 1.0.0" (1999) https://link.springer.com/chapter/10.1007/3-540-48660-7_34
"This is version 2.0.0 of the software." (1999) https://econpapers.repec.org/software/bocbocode/s365703.htm
"This report provides the NIKE3D user's manual update summary for changes made from version 3.0.0 April 24, 1995 to version 3.3.6 March 24, 2000" - https://www.osti.gov/servlets/purl/15004757
> This is not a new or revolutionary idea. In fact, you probably do something close to this already. The problem is that “close” isn’t good enough. Without compliance to some sort of formal specification, version numbers are essentially useless for dependency management.
Thus, X.Y.Z does not automatically mean "semver". Nor does even X.0.0 mean semver.
To give some 20th century counter-examples found through Google Scholar:
"System Description: Spass Version 1.0.0" (1999) https://link.springer.com/chapter/10.1007/3-540-48660-7_34
"This is version 2.0.0 of the software." (1999) https://econpapers.repec.org/software/bocbocode/s365703.htm
"This report provides the NIKE3D user's manual update summary for changes made from version 3.0.0 April 24, 1995 to version 3.3.6 March 24, 2000" - https://www.osti.gov/servlets/purl/15004757
It's rather than semver looks like it. Curl predates semver by 15 years. People have been numbering their software release as $major.$minor.$patch forever.
Oh yeah. The `x.y.z` notation that already existed and that SemVer just adopted is totally a surefire signal that “SemVer Here”.
You’ve got the order of events all mixed up.
You’ve got the order of events all mixed up.
[deleted]
Still no IRC support. Which - considering the project that became curl started out as an IRC bot - is slightly ironic.
How would that work ? cURL is built for request-reply usecases, with potentially long-lived connections, but once connected the handling happens outside of curl. irc is an interactive protocol, meaning that requests and replies would be handled inside cURL.
An example: A git hook that joins a channel every time a commit is pushed to its master branch, to inform an existing IRC bot, that it is time to pull the newest changes from the repository.
Use netcat instead?
printf "NICK gitbot42\nUSER foo bar baz\nPRIVMSG #gitevents :pull now\nQUIT :foobar\n" | nc irc.server.foo 6667Oh of course, I am not waiting on cURL to support IRC. Rather I was suggesting a use case, where essentially the client logs in, performs a series of commands and then disconnects. And I feel that is fitting with how cURL operates.
This _really_ sounds out of scope for what curl is
Yet, Curl supports TELNET.
Another potential use case for IRC is scripting a data collector to connect, run /list on a number of IRC networks, and disconnect. Eg, something similar to whatever netsplit.de does to collect its statistics.
But then, given the ability to connect to IRC via TELNET, perhaps Curl can already claim support for IRC.
Another potential use case for IRC is scripting a data collector to connect, run /list on a number of IRC networks, and disconnect. Eg, something similar to whatever netsplit.de does to collect its statistics.
But then, given the ability to connect to IRC via TELNET, perhaps Curl can already claim support for IRC.
Yeah that's my point: that would be like saying cURL supports XMPP because there are HTTP/web socket bridges, but that's not the point.
I guess cURL could connect, with, send a message and leave, or list rooms, but that's already a hack on the original use case of IRC. And it's only a small fraction of what IRC is used for, namely people interacting with each other. "Supporting IRC" would mean at least that.
I guess cURL could connect, with, send a message and leave, or list rooms, but that's already a hack on the original use case of IRC. And it's only a small fraction of what IRC is used for, namely people interacting with each other. "Supporting IRC" would mean at least that.
You're welcome to make a PR :-)
No big deal. I could write it in a 3 day weekend.
What is this a reference to?
Found it;
“I Could Rewrite Curl” - https://news.ycombinator.com/item?id=27220136
20 May 2021
In before somebody is going to say they can rewrite curl in a weekend hackaton.
There is a blogpost about this exactly where Daniel collects similar responses:
https://daniel.haxx.se/blog/2021/05/20/i-could-rewrite-curl/
https://daniel.haxx.se/blog/2021/05/20/i-could-rewrite-curl/
"We sold a curl exploit" is a nice short story of everything wrong with our rotten industry.
Lobsters and HN have really become a hotspot for arrogant contrarians. You can observe this best in the comment section of any thread that dares to mention modern web development.
The most charitable view I can take of such comments is to mentally paraphrase them to: "I can rewrite the small subset of the curl CLI that I personally use, in a modern language, on top of an existing HTTP library, in a weekend hackathon". That doesn't seem so crazy.
Yeah, especially using libcurl.
writing stuff I use is not that much of an effort. However providing various configuration options, handling edge cases and scenarios millions of programmers would like to use... different league.
writing stuff I use is not that much of an effort. However providing various configuration options, handling edge cases and scenarios millions of programmers would like to use... different league.
of course I can ! just pull in libcurl and then it is just parsing cli arguments
A whole weekend?! With GPT4 and C++23 on the horizon that's more of an afternoon delight.
Hah. Goddamn. This comment was depressing enough to shock me into leaving HN for the night. We have so little time left. I should spend it reading good books.
Half of which is just waiting for the code to compile.
You could ask for gpt to pretend to be a Linux machine and predict how the c++ code would run without compiling with 99% accuracy and just a little chance of hallucinating the target endpoint responding with 418
25 years of CVE:s, why not.
to be fair, if you have ever used the curl C api, you can see why there are 25 years of CVEs, and not 25 years of reliable secure bug free networking (not that its possible).
I love C as much as the next guy, hell, I even write it for fun when I'm feeling down -- but the one thing I won't do, beyond a little "i wrote an http 1.0 server in C" joke project, is networking.
Keep your C offline. If I do networking, it has to be modern C++ with static analyzers, good practices, boost asio, unit testing, and sanitizers, or just Rust, Erlang/Elixir, or whatever other non-C language.
Ive never seen a library get as abused and misused as Curl in source code - well maybe zlib. I think it's great that curl exists, and Im glad its so old that the bugs are mostly worked out, but writing a subset of curl that doesnt have a million issues in a weekend is not so unrealistic.
I love C as much as the next guy, hell, I even write it for fun when I'm feeling down -- but the one thing I won't do, beyond a little "i wrote an http 1.0 server in C" joke project, is networking.
Keep your C offline. If I do networking, it has to be modern C++ with static analyzers, good practices, boost asio, unit testing, and sanitizers, or just Rust, Erlang/Elixir, or whatever other non-C language.
Ive never seen a library get as abused and misused as Curl in source code - well maybe zlib. I think it's great that curl exists, and Im glad its so old that the bugs are mostly worked out, but writing a subset of curl that doesnt have a million issues in a weekend is not so unrealistic.
Why try not, then?
Edit: Just found this[0], hehe. :)
[0]: https://daniel.haxx.se/blog/2021/05/20/i-could-rewrite-curl/
Edit: Just found this[0], hehe. :)
[0]: https://daniel.haxx.se/blog/2021/05/20/i-could-rewrite-curl/
what have you created in the last 25 years?
What's stopping you from making a basic version? Just about everything supports HTTP 1.0 still so you can make a quick program that sends:
GET / HTTP/1.0
Host: example.com
"Figure out TLS."
Is this sarcasm? A joke?
Is this sarcasm? A joke?
Try eTLS, the e stands for security /s
OpenSSL or your TLS library of choice will likely have documentation and examples showing you how to use it. I'm not sure I understand your point.
[deleted]
Have you looked at the documentation or are you just guessing? When you realize how impenetrable OpenSSL or any TLS lib documentation looks to people untrained in cryptography, you would gain a new appreciation for Git man pages!
I used mbedTLS a while ago when implementing my own HTTPS server from scratch. I had no particular crypto or TLS knowledge, though I did know about ciphers, hashes and certificates from a high level.
Between the docs[1][2] and the examples[3] I was up and running using my own IO stack in a couple of evenings. If I had used the IO stack from the library it would have been very easy.
Another couple of evenings and I had it integrated with Windows' certificate store. In the end it wasn't very difficult to implement, but it was a bit more of a challenge figuring out the documentation and how it related to mbedTLS.
Definitely a rewarding experience, gave me some nice insights into the plumbing.
edit: Note, not trying to argue replacing libcurl is a weekend project!
[1]: https://mbed-tls.readthedocs.io/projects/api/en/latest/
[2]: https://mbed-tls.readthedocs.io/projects/api/en/latest/api/g...
[3]: https://github.com/Mbed-TLS/mbedtls/blob/development/program...
Between the docs[1][2] and the examples[3] I was up and running using my own IO stack in a couple of evenings. If I had used the IO stack from the library it would have been very easy.
Another couple of evenings and I had it integrated with Windows' certificate store. In the end it wasn't very difficult to implement, but it was a bit more of a challenge figuring out the documentation and how it related to mbedTLS.
Definitely a rewarding experience, gave me some nice insights into the plumbing.
edit: Note, not trying to argue replacing libcurl is a weekend project!
[1]: https://mbed-tls.readthedocs.io/projects/api/en/latest/
[2]: https://mbed-tls.readthedocs.io/projects/api/en/latest/api/g...
[3]: https://github.com/Mbed-TLS/mbedtls/blob/development/program...
Kind of makes you think that the subject at hand is quite complex in its nature, thus untrained people might do best to steer away until properly trained.
Not much, but the word "basic" does some heavy lifting there.
Even with just http(s), what you describe is just a tiny fraction of typical use cases, even if you ignore all the rarely used bits.
Even with just http(s), what you describe is just a tiny fraction of typical use cases, even if you ignore all the rarely used bits.
Just don't forget to implement CLI params so we can have drop-in replacement: https://curl.se/docs/manpage.html
Othwerise for basic HTTP version, I'll use Fetch API that has TLS already figured out. https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API
Othwerise for basic HTTP version, I'll use Fetch API that has TLS already figured out. https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API