Linux Inside – How the Linux Kernel Works(0xax.gitbooks.io)
0xax.gitbooks.io
Linux Inside – How the Linux Kernel Works
https://0xax.gitbooks.io/linux-insides/content/
32 comments
Every time I start to fall into the conceit that I'm an exceptionally good developer, I look at things like kernels or low level hardware programming and eat a big ol' slice of humble pie.
Why? It's just a different level of detail than you're used to. It's not magic.
As someone that used to do kernel and RTOS development for many years and then migrated into web and app dev for a living because it allowed a more flexible lifestyle for paying the bills - this equivocation is disingenuous.
There is some room for disagreement, but I agree with most of the following. https://developer.apple.com/library/content/documentation/Da...
Kernel development, similar to algorithmic trading (incl HFT), some types of games, various embedded, actual production compilers etc requires a higher level of rigorous understanding from a variety of areas compared to the vast majority of webapps (often CRUD) and most software that IT professional work on. Kernel dev similar to other performance critical or safety critical systems has a much reduced set of tradeoffs and more numerous constraints, for one.
Of course, most applications require knowledge in disciplines such as design and domain specific knowledge that kernel development does not require, but that is besides the point. It requires a different type of thinking overall (one that is more unforgiving to lack of rigor and not as open to subjectivity cf. design) that low-level systems development/engineering and most app development are virtually unrelated disciplines outside of trivial similarities relating to them both targeting computers.
I don't say this to discourage anyone. But these reductions are unhelpful and outright condescending to those that might struggle to learn (oh it's just a slightly different domain than your JS TODO app....um no it's not.. it's harder for you to learn because it is actually harder and more complex).
There is some room for disagreement, but I agree with most of the following. https://developer.apple.com/library/content/documentation/Da...
Kernel development, similar to algorithmic trading (incl HFT), some types of games, various embedded, actual production compilers etc requires a higher level of rigorous understanding from a variety of areas compared to the vast majority of webapps (often CRUD) and most software that IT professional work on. Kernel dev similar to other performance critical or safety critical systems has a much reduced set of tradeoffs and more numerous constraints, for one.
Of course, most applications require knowledge in disciplines such as design and domain specific knowledge that kernel development does not require, but that is besides the point. It requires a different type of thinking overall (one that is more unforgiving to lack of rigor and not as open to subjectivity cf. design) that low-level systems development/engineering and most app development are virtually unrelated disciplines outside of trivial similarities relating to them both targeting computers.
I don't say this to discourage anyone. But these reductions are unhelpful and outright condescending to those that might struggle to learn (oh it's just a slightly different domain than your JS TODO app....um no it's not.. it's harder for you to learn because it is actually harder and more complex).
This is true insofar that bugs resulting from an incomplete understanding of the system have far more significant implications in kernel development (kernel panics, seg faults, OOM killer) than in web dev, where you might get an Exception that's handled by your framework.
> this equivocation is disingenuous
i'm super confused. what equivocation are you refering to?
i'm super confused. what equivocation are you refering to?
I think he may have meant equiparation.
This is probably the most important lesson I learned when I started doing kernel and firmware work - the Linux kernel is, fundamentally, just a C program - albeit quite a complex one where you don't have the standard library and you see "volatile __asm__" more than you'd like to, but it's still just a big C program.
I'd absolutely encourage anyone with some C background to go and write themselves a "hello world" Linux kernel module to go convince themselves of this.
I'd absolutely encourage anyone with some C background to go and write themselves a "hello world" Linux kernel module to go convince themselves of this.
[deleted]
I work in JIT compilers and sometimes people say this about my field. But it's really not true for one major reason - not all the knowledge you need in this field is written down. So if you don't have someone you can ask questions nearly full time who knows it all, you can't just get started like you can in web programming, where there are endless books and tutorials.
It's not magic indeed:
——
Looking up and down the aisles at Lowes and not finding a raised garden bed that met my needs, I inquired with a staff member:
“Sorry, we don’t carry anything like that.”
“Darn. Ok, I’ll try another store then,” I sighed with resignation.
“You know,” he said to me, “it’s just wood.”
It took me a moment to realize how liberating this statement was. To him, woodworking held no mystery at all - it was just a thing you did. If he wanted a garden bed, there’d be no question in his mind that he’d make it himself. Why spend more on something suboptimal when you can build exactly what you want?
— Bryan Kennedy (http://plusbryan.com/its-just-wood)
——
Looking up and down the aisles at Lowes and not finding a raised garden bed that met my needs, I inquired with a staff member:
“Sorry, we don’t carry anything like that.”
“Darn. Ok, I’ll try another store then,” I sighed with resignation.
“You know,” he said to me, “it’s just wood.”
It took me a moment to realize how liberating this statement was. To him, woodworking held no mystery at all - it was just a thing you did. If he wanted a garden bed, there’d be no question in his mind that he’d make it himself. Why spend more on something suboptimal when you can build exactly what you want?
— Bryan Kennedy (http://plusbryan.com/its-just-wood)
Nice quote.
At least half of what I have learned has been because I was dissatisfied with what was available and thought I could do better myself.
Sometimes I could make a better thing, and sometimes I couldn't, but I always learned something. Even sometimes just the knowledge that it's harder than it looks.
At least half of what I have learned has been because I was dissatisfied with what was available and thought I could do better myself.
Sometimes I could make a better thing, and sometimes I couldn't, but I always learned something. Even sometimes just the knowledge that it's harder than it looks.
Eh... It's far more complex than what most devs are accustomed to, and often too complex for most devs to be good at. It's not just "different", it's harder. You have to solve hard problems if you're going to build and maintain an OS. You're not solving hard problems when you write your next CRUD app.
Thats right. We're each of us experts in our own systems.
+1 and then I feel the same and sometimes even worse when it comes to programming language design.
Nice. Way back in the day, maybe 20-25 years ago, I had a first look at the boot process and wish I'd had something similar to the first chapter then. Man, it was hard to find information and ones you did you realized understanding it meant learning an entirely new subject.
Those were the days really. Barely having started my English studies I went looking for a single piece of information and found so much else on the way.. The tinkering and nights of frustration gave insights and a feeling of accomplishment that set me on the path to the profession I have today.
Then I started high school and someone showed me a shaded and textured cube they had rotating on their screen and I found out what math is good for :)
Those were the days really. Barely having started my English studies I went looking for a single piece of information and found so much else on the way.. The tinkering and nights of frustration gave insights and a feeling of accomplishment that set me on the path to the profession I have today.
Then I started high school and someone showed me a shaded and textured cube they had rotating on their screen and I found out what math is good for :)
Besides some scattered text files and the Linux man pages, there is this: https://www.kernel.org/doc/html/latest/
It's fairly new - started about a year ago, but there's quite a lot of stuff in there already (bootstrapped from the old XML stuff, I believe).
It's fairly new - started about a year ago, but there's quite a lot of stuff in there already (bootstrapped from the old XML stuff, I believe).
... That guide is actually fucking hilarious.
'However, that’s not too bad, because even the makers of GNU indent recognize the authority of K&R (the GNU people aren’t evil, they are just severely misguided in this matter)'
'However, that’s not too bad, because even the makers of GNU indent recognize the authority of K&R (the GNU people aren’t evil, they are just severely misguided in this matter)'
but the preferred way, as shown to us by the prophets Kernighan and Ritchie, is to put the opening brace last on the line, and put the closing brace first, thusly:
So much for the idea that kernel developers are super-smart programming "gods". Formatting code that way is brain-dead and Kernighan and Ritchie are false prophets. The real prophet, Allman, tells us the Truth - curly braces go on a line by themselves, aligned vertically and indented to the same level as their corresponding control structure.
So much for the idea that kernel developers are super-smart programming "gods". Formatting code that way is brain-dead and Kernighan and Ritchie are false prophets. The real prophet, Allman, tells us the Truth - curly braces go on a line by themselves, aligned vertically and indented to the same level as their corresponding control structure.
These theories were ultimately settled by trial by combat between the champions representing each: Linux and BSD. Linux has clearly won this battle, and thus, the gods have spoken. ~
For a quick overview, I like http://www.makelinux.net/kernel_map/ - even though it's a bit dated (the map mentions kernel 2.6.36), I find it a good starting point for digging deeper.
I've taken a class in college about the Linux kernel, and it's still amazing to see how complicated it actually is.
I've got a fairly good idea of how many pieces work, but the whole thing together still puzzles me at times.
I think that it's one of the biggest achievements of the internet as a tool for progress, and it shows that good intentions and pragmatism (together with some good Linus insults as well) can go a long way
I've got a fairly good idea of how many pieces work, but the whole thing together still puzzles me at times.
I think that it's one of the biggest achievements of the internet as a tool for progress, and it shows that good intentions and pragmatism (together with some good Linus insults as well) can go a long way
> I've taken a class in college about the Linux kernel, and it's still amazing to see how complicated it actually is.
Complexity doesn't necessarily mean it's a masterpiece. A lot of it is just old and has been incrementally changed without being simplified in the mean time.
As a concrete example, the guide goes through the rather complicated mess of how syscalls worked. That complexity was the result of a series of optimizations and features added over time, but it wasn't a good design. These days Linux syscalls (on x86_64 anyway) are a good deal simpler than they are in the guide.
(Hey 0xAX, I don't suppose you'd like to update the guide?)
Complexity doesn't necessarily mean it's a masterpiece. A lot of it is just old and has been incrementally changed without being simplified in the mean time.
As a concrete example, the guide goes through the rather complicated mess of how syscalls worked. That complexity was the result of a series of optimizations and features added over time, but it wasn't a good design. These days Linux syscalls (on x86_64 anyway) are a good deal simpler than they are in the guide.
(Hey 0xAX, I don't suppose you'd like to update the guide?)
I'm not 0xAX, but care to make a pull request to get the ball rolling?
I love this work, it inspired me to work on Inside LibreOffice.
https://www.gitbook.com/book/chrissherlock1/inside-libreoffi...
https://www.gitbook.com/book/chrissherlock1/inside-libreoffi...
In fact, StarView was in essence what we now call the Visual Components Library (VCL)
Any relation to the Borland/Delphi VCL?
Any relation to the Borland/Delphi VCL?
Does not look like it.
https://github.com/LibreOffice/core/tree/master/vcl
But I can see clear lines for confusion.
https://github.com/LibreOffice/core/tree/master/vcl
But I can see clear lines for confusion.
clear lines. confusion.
No, afraid not.
Oh my God, the first article has a link to Ralph Brown's Interrupt List. Good ol' days!
[1] http://www.ctyme.com/intr/int.htm
[1] http://www.ctyme.com/intr/int.htm
Holy shit, yeah. Last time I looked at those pages was like 15 years ago.
Torvald's thesis on portability of Linux is also a quite interesting article
https://www.cs.helsinki.fi/u/kutvonen/index_files/linus.pdf
https://www.cs.helsinki.fi/u/kutvonen/index_files/linus.pdf
Is there any documentation that talks about x86_64 and the modern UEFI boot process? This guide still talks about BIOSs, MBRs, real mode, 16-bit registers, etc.