> I get the impression nvidia puts out a lot of their hardware supporting software themselves because they are hostile to open source community collaboration in general. This could be because nvidia is a big fan of vendor lock in.
It's not hostility, it's about agility. More so than other hardware vendors, we rely on really tight integration between hardware and software.
In some situations, we find a hardware bug that would require another manufacturer to do a "respin" (e.g. restart the manufacturing process with a new, fixed design). Because we have tight control over the software stack, we can workaround that bug. It's faster for us to do this when we have full control. Also, sometimes these bugs have security implications, etc.
That said, we've been moving in the direction of open source for a long time.
The other fellow was suggesting we should write no software at all. Nouveau's struggles is an excellent example of how difficult it is to write software for hardware without the engagement and interaction of the manufacturer of that hardware.
TL;DR you're talking about whether our software should open source versus closed source; the other fellow was suggesting we shouldn't have software at all.
That involves a few diagrams, but essentially, we have two layers:
- the libcu++ layer, which has some of our extensions and implementations specific to our platform.
- the libc++ layer, which is a modified upstream libc++.
A header in the libcu++ layer defines the libc++ internal macros in a certain way, and then includes the applicable libc++ header.
This is the current architecture, but we're moving away towards a more integrated approach where almost everything is in the libc++ layer.
We change the mangling of all the symbols by changing the inline namespace that they are in, regardless of whether or not functional ABI breaks occurred. That's why it says the ABI is broken on major releases. We do this to try and loudly break people who are trying to depend on ABI stability, instead of silently failing them.
Since Unified Memory. UVA, or Unified Virtual Addressing, just ensured that a GPU-private object wouldn't have the same address as a CPU-private object.
The majority of software engineers at NVIDIA write software that runs on the host CPU.
The majority of software written at NVIDIA (by any metric, lines of code, number of projects, etc) runs either solely on the CPU, or on both the CPU and the GPU.
> I did not say the library controls anything, Nvidia controls the library : its features, its roadmap, its bugs corrections, development efforts (people) etc. All these choices are made by Nvidia.
libcu++ is a fork of LLVM's libc++, which we do not control. We contribute upstream and engage with that community.
libcu++ is an implementation of the C++ Standard Library, which is controlled by an ISO standardization committee, which has about ~300 members, 10 of whom work at NVIDIA.
> The same way I don't like them making software, I also don't like them selling CPUs or seafood. They are good at GPUs and that's OK.
We employ more software engineers than hardware engineers. We don't sell hardware, we sell software + hardware.
We manufacture and sell CPUs today. I like to think we're quite good at it.
> The drivers are usually running in the kernel space and do not involve much of interaction with users.
Incorrect. The core part of the driver, called the Resource Manager (RM), runs in the kernel space. Each different SDK (CUDA, OpenGL, Vulkan, etc) has its own "user mode driver", which is a shared library that interacts with RM. It's hard to say what the split is, but I'd say roughly half of what you think of as "the driver" is in user mode.
> Firmware, on the other side, is hardware-close software and can be gradually replaced by specific hardware continuous improvements without the user/software or the OS noticing.
Firmware runs on the GPU. You can't do everything from the GPU.
But neither firmware nor drivers have anything to do with the toolchain that you use to write heterogeneous programs. That's what this is a part of.
> A lot of hardware has builtin software, either inside a firmware or as a driver.
Correct.
> Keeping the software part in firmware lets customer free to use any kind of OS.
Do you mean firmware, or firmware and driver?
You can't do everything in firmware.
> Using host cpu and memory is bad design IMHO.
How do you propose that you program the GPU then?
The CPU has to interact with the GPU. Some software has to manage that interaction.
That said, we are not talking about either a driver or firmware. This is a part of our toolchain. It is a library that you use when writing a heterogeneous program.
> IMHO, the question is not that we need code to run on CPUs and GPUs , we do need that, The question is whether the GPU seller has to control both sides.
The question is not about running code on CPUs, or running code on GPUs. It's about running code on both CPUs and GPUs at the same time. It's about enabling the code on the CPU and the code on the GPU to seamlessly interoperate with each other, communicate with each other, move objects and data to and from each other.
Who do you expect to make that happen?
> Until I buy a CPU from nvidia I want to keep some kind of independence
You can buy a CPU from NVIDIA, check out our Tegra systems. We also sell full systems, like DGX platforms, which use a 3rd party CPU.
> When will we be able to use a future riscv-64 CPU with an nvidia GPU ? we will let the answer to nvidia ?
Who else would answer this question?
Okay, you want to use <insert some future CPU> with our GPU.
Who is going to design and build the interconnect between the CPU and the GPU?
Who is going to provide the GPU driver?
The CPU manufacturer? Why would they do that? They don't make any money from selling NVIDIA products. Why should they invest effort in enabling that?
NVIDIA employs more software engineers than hardware engineers.
> We need smarter and more shining GPUs from nvidia, not software.
Software is a part of the GPU. You get better GPUs by having hardware and software engineers collaborate together.
It is extremely expensive to put features into hardware. It costs a lot of money and takes a very long time. It takes 2-4 years at a minimum to put features into hardware. And there are physical constraints; we only have so many transistors.
If we make a mistake in hardware, how are we supposed to fix it? At NVIDIA we have a status for hardware bugs called "Fix in Next Chip". The "Next Chip" is 2-4 years away.
So what do we do? We solve problems in software whenever possible. It's cheaper to do so, it has a quicker turnaround time, and most importantly, we can make changes after the product has shipped.
> I would say .... The hardware must be sold independently of the software ... but it is a bit too complex, I know.
We don't sell hardware and you don't want to buy hardware. Trust me, you wouldn't know what to do with it. It's full of bugs and complexity.
We sell a platform that consists of hardware and software. The product doesn't work without software.
If we tried to make the same product purely in hardware, the die would be the size of your laptop and would cost a million dollars.
> apparently only a small bit of the library is actually implemented.
Yep. It's an incremental project. But stay tuned.
> I'm somewhat suspicious of the presumption of us using NVIDIA's version of the standard library for our host-side work.
Today, when using libcu++ with NVCC, it's opt-in and doesn't interfere with your host standard library.
I get your concern, but a lot of the restrictions of today's GPU toolchains comes from the desire to continue using your host toolchain of choice.
Our other compiler, NVC++, is a unified stack; there is no host compiler. Yes, that takes away some user control, but it lets us build things we couldn't build otherwise. The same logic applies for the standard library.
We wanted an implementation that intended to conform to the standard and had deployment experience with a major C++ implementation. EASTL doesn't have that, so it never entered our consideration; perhaps we should have looked at it, though.
At the time we started this project, Microsoft's Standard Library wasn't open source. Our choices were libstdc++ or libc++. We immediately ruled libstdc++ out; GPL licensing wouldn't work for us, especially as we knew this project had to exchange code with some of our other existing libraries that are under Apache- or MIT-style licenses (Thrust, CUB, RAPIDS).
So, our options were pretty clear; build it from scratch, or use libc++. I have a strict policy of strategic laziness, so we went with libc++.
> What do you mean about running in the same memory space? Your operating system doesn’t allow that. Is your concern about using host memory?
Actually, the basis of our modern GPU compute platform is a technology called Unified Memory, which allows the host and device processor to share access to memory spaces. We think this is the way going forward.
Of course, there's still the process isolation provided by your operating system.
NVIDIA employs more software engineers than hardware engineers.
> Why do they need to run software in the same memory space as my mail reader ?
It is a lot more expensive to build functionality and fix bugs in silicon than it is to do those same things in software.
At NVIDIA, we do as much as we possible can in software. If a problem or bug can be solved in software instead of hardware, we prefer the software solution, because it has much lower cost and shorter lead times.
Solving a problem in hardware takes 2-4 years minimum, massive validation efforts, and has huge physical material costs and limitations. After it's shipped, we can't "patch" the hardware. Solving a problem in software can sometimes be done by one engineer in a single day. If we make a mistake in software, we can easy deploy a fix.
At NVIDIA we have a status for hardware bugs called "Won't Fix, Fix in Next Chip". This means "yes, there's a problem, but the earliest we can fix it is 2-4 years from now, regardless of how serious it is".
Can you imagine if we had to solve all problems that way? Wait 2-4 years?
On its own, our hardware is not a complete product. You would be unable to use it. It has too many bugs, it doesn't have all of the features, etc. The hardware is nothing without the software, and vice versa.
We do not make hardware. We make platforms, which are a combination of hardware and software. We have a tighter coupling between hardware and software than many other processor manufacturers, which is beneficial for us, because it means we can solve problems in software that other vendors would have to solve in hardware.
> I really do not understand why a (very good) hardware provider is willing to create/direct/hint custom software for the users.
Because we sell software. Our hardware wouldn't do anything for you without the software. If we tried to put everything we do in software into hardware, the die would be the size of your laptop and cost a million dollars each.
You wouldn't buy our hardware if we didn't give you the software that was necessary to use it.
> Isn't this exactly what a GPU firmware is expected to do ?
Firmware is a component of software, but usually has constraints that are much more similar to hardware, e.g. long lead times. In some cases the firmware is "burned in" and can't be changed after release, and then it's very much like hardware.
It's not hostility, it's about agility. More so than other hardware vendors, we rely on really tight integration between hardware and software.
In some situations, we find a hardware bug that would require another manufacturer to do a "respin" (e.g. restart the manufacturing process with a new, fixed design). Because we have tight control over the software stack, we can workaround that bug. It's faster for us to do this when we have full control. Also, sometimes these bugs have security implications, etc.
That said, we've been moving in the direction of open source for a long time.
> It's not(https://nouveau.freedesktop.org/wiki/)
The other fellow was suggesting we should write no software at all. Nouveau's struggles is an excellent example of how difficult it is to write software for hardware without the engagement and interaction of the manufacturer of that hardware.
TL;DR you're talking about whether our software should open source versus closed source; the other fellow was suggesting we shouldn't have software at all.