I agree that the maximum SSD performance can be achieved with carefully planned asynchronous reads, using either the modern liburing (io_uring) or the older libaio (Linux asynchronous I/O).
Nevertheless, I do not think that "this is not really possible with mmap, even with madvise".
If the kernel is not eager enough to prefetch pages from the SSD when you use madvise with MADV_WILLNEED, then you can use madvise with MADV_POPULATE_READ, which should force the reading of the pages that you request.
Using madvise with MADV_POPULATE_READ for appropriate page ranges at the right moments should be able to provide a performance not much lower than when using explicit asynchronous reads.
I do not think that I have ever seen any benchmark for a PCIe 5.0 SSD that did not have sequential read speeds well over 10 GB/s.
If there were such a slow SSD, it would not make sense to buy it instead of a cheaper PCIe 4.0 SSD.
For PCIe 4.0 SSDs, I have seen a very large number of benchmarks where the SSDs achieved read speeds close to the theoretical limit, i.e. around 7 GB/s.
Searching now randomly for recent SSD reviews, I find many reviews for "SanDisk WD_BLACK SN8100", which achieve between 13 GB/s and 15 GB/s reading speed, which is better than most other consumer PCIe 5.0 SSDs.
Of course, if you write a very simple program that invokes something like "fread" or "read" in a loop, you will not reach such speeds. Achieving a SSD throughput close to the limit requires a more complex program that can ensure that the SSD controller is permanently busy with pipelined read commands.
Fortunately all the "weird key bindings" can be easily changed.
The only difficulty is caused by the fact that already each Emacs mode may change some of the key bindings. Because of that, before deciding on some key bindings you prefer, it is wise to first check the bindings used in all the Emacs modes that you are likely to use frequently, in order to choose bindings that would not conflict with any of those modes.
Certain kinds of GOTOs are indeed proven mathematically to be necessary to write certain programs, in the sense that eliminating the GOTOs can be done only by making the program more inefficient, i.e. by increasing the number of executed instructions and the amount of memory that is used.
Because of this, any decent programming language must include GOTOs, but in many modern languages they are not named GOTOs, because of the bad reputation of the word.
For instance the language Scheme does not have GOTOs, but it has mandatory tail call optimization, which means that a "tail call" is just an alternative name for "GOTO". For example, this allows the writing of a state machine in Scheme, exactly like it would be written in a language with GOTO, but using tail calls instead of GOTOs.
Other languages have labelled loops and they allow exit a.k.a. break with a label and next a.k.a. cycle a.k.a. continue with a label. Such instructions with labelled targets are just GOTOs with a bad placement of the label, which makes reading the source more tedious than with a classic GOTO.
For providing the benefits of GOTO, a restricted variant is sufficient, i.e. a GOTO that may jump only forwards and which cannot jump inside nested blocks.
You say that because you lack experience in writing programs for scientific-technical computing, where an abundance of complicated mathematical formulae are necessary.
Most programmers have seen only programs that do more data movement and comparison than actual computation, but nonetheless there are programs with abundant computations, the kind of programs that were originally written in Fortran, but nowadays they may be ported to other languages, or alternatives for them may be now written in more modern languages.
In such programs, overloaded operators do not make programs "slightly shorter to type", but they reduce the amount of text at least an order of magnitude or even much more.
The main advantage is not that you type less, but that you can read the source of some function in one page, so you can see it in its entirety, instead of having the source spread on many pages, forcing you to wander through all those pages continuously, when you try to understand what it does and whether it does it correctly.
I have written such programs, for instance which were full of long formulae where most variables were complex vectors or complex matrices (for computing some radiated electromagnetic fields with the method of boundary elements). Without the operator overloading of C++, reading them would have been extremely tedious and maintaining them would have been very error prone.
For myself, any programming language that lacks operator overloading, e.g. Java, is disqualified, because it definitely is not "powerful" enough.
Operator overloading also increases the safety of a programming language, because you can define distinct data types for each kind of physical quantity, allowing the compiler to detect and reject the invalid operations. Without operator overloading that would inflate too much the size of the source text.
The rules can be enforced by a static code checker.
That is really not very different of rules enforced by the Rust compiler.
For someone who does a fresh start, using a Rust compiler may ensure safer programs out of the box, but that does not mean that the same results cannot be achieved by alternative means when using other languages, when the use of those languages makes sense for other reasons, and it is worthwhile to invest resources in making appropriate libraries and tooling.
In general, I recommend against the use of C++ in new projects, but I see much too often claims about things that are supposedly difficult or impossible to do in C++, which are just false.
No, I mean by using only custom types for things like arrays, strings and pointers, which do access checks and automatic memory release, and not using unsafe features like the built-in arrays, strings and pointers, or the incorrect integer type conversions inherited from C.
For maximum safety, beyond what Rust offers by default, in C++ it is easy to replace the built-in integer types with custom integer types, which check for overflows and allow only the correct type conversions. It is also easy to define distinct types for various kinds of physical quantities, for increased safety.
You do not need to run anything in your head. With appropriate type definitions, a C++ compiler will do anything that is required.
The problem is that because of the requirement for backwards compatibility, C++ is a huge junk collection. I think that more than half of C++ consists of obsolete features, which should never be used in new programs, and this is a serious difficulty for newbies. There are various C++ style guides, but in my opinion even most of those are not very inspired.
Despite of its defects, C++ still has the advantage of extreme customizability. It is easy to write programs that appear to be written in a language that has no resemblance with C++ (inclusively by having different keywords and what appears to be a different syntax), but nonetheless they are valid C++ programs.
Such a customized C++ variant can mimic any safer language.
The old Cascade Lake based server found by the previous poster is still new enough to have instructions for relatively fast AI inference with the INT8 format.
So for optimal speed the models must be quantized in this format.
It is very likely that with INT8 models those CPUs are fast enough so that the inference throughput is limited by the memory bandwidth (384-bit interface to DDR4-2933 per socket, i.e. 282 GB/s for both sockets).
The memory throughput for such an old server is very similar to an AMD Ryzen Strix Halo, NVIDIA DGX Spark or Apple M5 Pro, but it has much more memory.
The inference speed should be very similar to those, but with bigger LLMs.
Now many mini-PCs and desktops are able to read simultaneously from 1 PCIe 5.0 SSD and 1 PCIe 4.0 SSD. This can ensure a reading throughput around 20 GB/s, i.e. 20 times faster than on author's system.
With only 1 PCIe 5.0 SSD, the reading throughput is still significantly more than 10 times faster than on author's system.
So it is likely that inference speeds around 1 token/s are achievable on something like a NUC mini-PC.
I have discovered Ghostty only this year, and I have switched to it a few months ago.
For decades, I have used a great number of video terminal emulators. I have used for long time intervals at least 8 or 9 video terminal emulators, from the original xterm until the one used immediately before ghostty, which was kitty (and including Konsole, the Gnome terminal, the XFCE terminal, WezTerm and others).
I consider Ghostty the best video terminal emulator that I have used. For now, I have not encountered any noticeable bug yet, even if I spend a lot of time using it.
Nor have I encountered any feature that I really miss (though a few things that worked in other emulators, do not work, at least not with the default configuration, e.g. setting the title of a tab window with the standard escape sequence of characters; but the ghostty titles are fine, i.e. the pwd value while in shell and the invoked command otherwise).
I do not doubt that there may exist some bugs or missing features, but it seems unlikely that they can be seen during typical workflows. I have used it only in Linux, so I do not know anything about bugs under macOS.
Rust is one of the safer languages, but saying that it is "the safest language" is just a baseless exaggeration.
Decades before Rust and long before the simplified language that was C, there were safe programming languages, where all invalid operations, numeric overflows or out-of-bounds accesses generated exceptions and where use-after-free was impossible, because either garbage collectors or reference counts were used.
Rust is much safer than C compiled with its bad default compilation options, but it did not bring much in comparison with other languages.
Even in C++, with appropriate rules, restrictions and discipline you can write programs that are guaranteed to be at least as safe as any Rust program, but unfortunately very few use C++ in this way, i.e. by strictly avoiding the features that are obsolete or unsafe.
There are several substances that are made in the human body, but apparently in lower quantities than necessary.
These include at least choline, creatine and taurine, and also vitamin K2 (which is made in humans by conversion from vitamin K1, which is abundant in vegetables, but with a low efficiency). Vitamin A is also made with a low efficiency from carotene, so a vegan who does not supplement it needs to eat something like a big carrot every day.
It is not known with certainty which are the optimum daily intakes for such substances. All that is known for sure is that their concentration in the blood of vegans (who do not take supplements) is lower than in the blood of non-vegans and that there are studies like this, which suggest that the higher amounts are beneficial.
Because I eat mostly vegan food, I supplement it with these substances. As long as the amount of supplements is small (especially for choline, where both too low and too high daily intakes are known to be harmful), in the worst case they are worthless, but in the best case they improve your health.
> All in all, using zero-knowledge proofs for research may not have much benefit. “There is almost no situation in research where one research group is so far ahead of all the other research groups that they can keep novel results secret for long,”
> Given this experience, Gidney says in a blog post, “I don’t think it’s the right strategy moving forward” to publish such results with zero-knowledge proofs. “The benefits are negligible, and the costs are many. We should just publish openly.”
Bernstein as a "software publisher" cannot really be compared with anyone else, because what can be considered as great flaws in comparison with others are compensated by equally great advantages.
He has published a very large quantity of open-source software, but after publication he never bothered with any kind of maintenance, which is understandable, because he went on doing other work.
On the other hand, unlike almost any other software packages, those published by him almost did not need any maintenance. The only required changes, after decades since they were written, have been caused by external changes, e.g. the continuous evolution and instability of the Linux APIs and the replacement of various IETF RFCs.
I am still using several software packages written by DJB, which have been run continuously 24/7 for more than a quarter of century, on many servers, without ever causing any kind of problems or incidents, unlike a lot of much more notorious software packages, which had various bugs despite permanent maintenance.
It does not matter much how long it goes back, because before its standardization very few people have bothered to study it.
Like for any other cryptographic algorithms, where one or more decades were necessary for a good understanding of their properties, we can expect much more relevant publications about lattice key establishment in the next years, than until now.
The CPU of Ryzen is better than that of DGX Spark, especially for modern programs that have been updated to use AVX-512 (i.e. it has a significantly higher multithreaded performance).
Only for GPU applications the NVIDIA system is likely to be better.
Only the top variants of M5 Max have more than twice the memory bandwidth of AMD, but those are much more expensive, i.e. over $10,000.
M5 Pro has as slightly higher memory bandwidth, but currently it is available only with up to 64 GB of DRAM. Even with this small memory, it is more expensive than either AMD or NVIDIA, especially when you do not want a puny SSD, but a normal-sized SSD, i.e. big enough to put there an LLM if you want to compute a quantization yourself (i.e. more than $5600 with a 4 TB SSD & 64 GB DRAM).
If you want to do LLM inference, I do not see Apple as a competitive solution, as their price is much, much higher, while also having limited expansion for SSDs, where you need a lot of space if you want to store a few LLMs.
The only thing that is correct is that the AMD Strix Halo system used to be much cheaper than NVIDIA, but now it has the same price. The CPU of Strix Halo is better than that of NVIDIA, but the NVIDIA GPU is likely to be better than the AMD GPU and CUDA is guaranteed to work fine.
I agree with the explanation you give at that link.
For linear algebra, the Fortran convention, "column-major" is indeed superior.
This is most obvious when computing a matrix-vector product, i.e. a linear transformation of a vector, which is a very common operation.
In schools this operation is typically taught in the wrong way, i.e. by computing scalar products of row vectors from the matrix with the column vector operand.
This naive method is inefficient. The correct method that must be used in computers is to avoid scalar products, but use the so-called AXPY operation (from its BLAS name. i.e. scalar A times vector X Plus vector Y), where the operands are column vectors from the matrix and the column vector that is the second operand of the matrix-vector product.
Therefore, to compute the product one needs to read columns from the matrix, so for maximum throughput the elements of a column must be stored sequentially.
In schools, also the matrix-matrix product is taught in the wrong way, with scalar products between row vectors and column vectors.
The correct way also avoids the inefficient scalar products and replaces them with tensor products of column vectors.
No, Simula (1962) was not object-oriented. Only its successor "Simula 67", which was described publicly in some papers in the first half of 1968, was the first "object-oriented" language, by inventing the "virtual" procedures and functions, and also class inheritance. Simula 67 inherited most of its concepts and terminology, including keywords like "class" and "new", from a crucial paper published by C. A. R. Hoare in November 1965-11, "Record Handling". Except for its new object-oriented features and for the features taken from Hoare, Simula 67 was a language very similar to ALGOL 60.
> ALGOL (1958)
"ALGOL" did not exist in 1958. In December 1958, a description of IAL (International Algebraic Language) was published. IAL is the predecessor of ALGOL 60, which was published in May 1960.
A significant part of ALGOL 60 was already present in IAL (1958), but some important parts of IAL were deleted and some new very important features were added in ALGOL 60. The differences between IAL and ALGOL 60 are comparable with the differences between C and C++, i.e. they are related languages, but they are also very different.
ALGOL 60 introduced the concept of "block" with local variables that are allocated upon block entry and released upon block exit. This required the use of stacks and stack pointers and it allowed the implementation of recursive procedures. ALGOL 60 also introduced several other less important, but still significant features, like the "while" loops.
But ALGOL 60 lost a very important feature of IAL (1958) and this loss has crippled many modern programming languages, which have followed too subserviently ALGOL 60. In IAL, unlike in the followers of ALGOL 60, the result of a function could consist of multiple values, i.e. a function could have both multiple input parameters and multiple output parameters and the syntax of assignment allowed the assignment of the output parameters to multiple variables.
Besides ALGOL 60, IAL (1958) had another descendant, the language JOVIAL (Jules' Own Version of IAL), which was used extensively by the US Military and later it had an important influence on the Department of Defense requirements for the programming language that eventually became Ada (1979).
> Note that the original stylised name “FORTRAN” changed to the modern form “Fortran” with the publication of the Fortran 1990 standard.
In fact, the early names of many programming languages, like FORTRAN, ALGOL and COBOL were not printed in capitals, except where good typography was not available, e.g. when using a typewriting machine.
Where normal typographic features where available, like in a book or in a journal, the names of the programming languages used a capital only for the initial letter, while the following letters were written with small capitals.
Nowadays the number of people familiar with the use of small capitals has decreased a lot and many low quality fonts do not include small capitals (in which case the word processor simulates the small capitals with smaller big capitals, which do not look as good as true small capitals).
Because of this, like with Fortran, many words that in the past used small capitals are written now with lower-case letters.
Nevertheless, I do not think that "this is not really possible with mmap, even with madvise".
If the kernel is not eager enough to prefetch pages from the SSD when you use madvise with MADV_WILLNEED, then you can use madvise with MADV_POPULATE_READ, which should force the reading of the pages that you request.
Using madvise with MADV_POPULATE_READ for appropriate page ranges at the right moments should be able to provide a performance not much lower than when using explicit asynchronous reads.