Red flag questions to ask in a Linux interview(blog.abctaylor.com)
blog.abctaylor.com
Red flag questions to ask in a Linux interview
https://blog.abctaylor.com/five-red-flag-questions-to-ask-in-a-linux-job-interview/
56 comments
Containers are also an abstraction, adding complexity, where bugs lurk. Maybe the 'Perl script someone wrote in the early 00s' has been revised multiple times to address security concerns, and if it's not broken, don't try and fix it by architecting large abstractions.
Until you got to deploy it on a new, supported linux OS, and realize that CPAN modules that load are different that the old server, and the developers own workstation, and now things go to hell, but they worked great on the developers machine.
(not just perl, python and others can be fun too).
As an admin, not a dev, I appreciate the additional complexity and security concerns, but I much more appreciate the portability and reproducability.
(not just perl, python and others can be fun too).
As an admin, not a dev, I appreciate the additional complexity and security concerns, but I much more appreciate the portability and reproducability.
Portability and reproducibility means that the same code runs nicely in different envrionments.
A container or VM with a specific version of an OS and packages is the exact opposite of portability.
A container or VM with a specific version of an OS and packages is the exact opposite of portability.
Saying "apprehension towards containers is a red flag" is as subjective as the apprehension itself. I like to work with professionals that have a justified opinion, rather than knee jerk reactions.
I am old, I've seen it all, sometimes I use containers. Are they useful? Sure! Are they "the final form"? No!
Tools are just tools, don't get obsessed about them... They only matter when used properly to solve the right problem. Obsession about tooling generates so much waste... and burnout!
I am old, I've seen it all, sometimes I use containers. Are they useful? Sure! Are they "the final form"? No!
Tools are just tools, don't get obsessed about them... They only matter when used properly to solve the right problem. Obsession about tooling generates so much waste... and burnout!
Containers have been around for 15 years. They are not "newfangled" anymore.
Containers in UNIX and mainframes predate Linux containers, thus a bit longer than that.
The big difference is that we did not use them to ship the developer's computer into production.
The big difference is that we did not use them to ship the developer's computer into production.
At the rate bank IT changes, maybe they kind of still are.
[deleted]
You're specifically talking about Docker/OCI containers; but "containers" have actually been around a lot longer than that — specifically on the sorts of mainframe systems that banks use.
A workload on e.g. IBM z/VM is/was basically "a container" — in the sense that it was an application filesystem or binary, packaged up with a manifest that specified what environment the application expects to be run under; where it's the workload manager's responsibility to arrange a sandbox to run the application within that'll meet the application's needs.
Of course, this was less about supplying an operating system to wrap around the container-image (ala Elastic Beanstalk), and more about supplying a particular flavor of emulated hardware for the workload to run "on bare metal" against. But applications of the day were bare metal — unikernels, if you want to call them that — and mainframe ISAs/memory-maps of the day were much like older-generation game console ISAs/memory-maps, in that the "bare metal" directly exposed advanced host services through CPU instructions and IO ports specific to the microarchitecture of the machine. The hardware "was" the OS, per se; and so emulating different hardware was in effect a virtualization of different operating systems.
A good way to understand these mainframe ISAs, might be to picture a high-level and complex runtime-based compilation target like Macromedia Flash bytecode: sure, it's "an ISA", but each op can cause extremely complex things to happen at the runtime level. The "runtime" of a physical mainframe would originally be "in hardware" — consisting of "accelerator" boards triggered by the CPU. But in z/VM, the "runtimes" of each "hardware profile" a workload might ask to be run under, are just regular old runtimes. Like Flash. Or a JVM. Or a software x86 emulator, with hardware paravirtualization care of x86 "application processors" living on accelerator cards.
To me, this abstraction — of a hypervisor that can support running workloads built for many different ISAs and microarchitectures, with these representing anything from low-level-emulated real machines (like x86) to high-level-virtualized real machines (like some specific mainframe) to high-level-virtualized abstract machines (like a JVM) — is obviously something that could have been used as "containerization" by those interested in doing so, even as far back as 1972's VM/370. And even those who weren't thinking in terms of "building a container", were still doing something very similar, in creating workloads as disk-image-based unikernels that assume full memory access, and which rely upon a single-tasking single-user exokernel library OS like DOS or CPM (but the z/VM equivalent, because they were writing programs that manipulated batch database records, not files, so regular DOS routines would be pretty useless.) Taking your "application disk" that you had developed to boot on your workstation, and uploading it byte-for-byte to your mainframe to be executed with 1000x concurrency over a batch workload, was exactly the sort of "shipping development to production" that we expect from containerization today.
A workload on e.g. IBM z/VM is/was basically "a container" — in the sense that it was an application filesystem or binary, packaged up with a manifest that specified what environment the application expects to be run under; where it's the workload manager's responsibility to arrange a sandbox to run the application within that'll meet the application's needs.
Of course, this was less about supplying an operating system to wrap around the container-image (ala Elastic Beanstalk), and more about supplying a particular flavor of emulated hardware for the workload to run "on bare metal" against. But applications of the day were bare metal — unikernels, if you want to call them that — and mainframe ISAs/memory-maps of the day were much like older-generation game console ISAs/memory-maps, in that the "bare metal" directly exposed advanced host services through CPU instructions and IO ports specific to the microarchitecture of the machine. The hardware "was" the OS, per se; and so emulating different hardware was in effect a virtualization of different operating systems.
A good way to understand these mainframe ISAs, might be to picture a high-level and complex runtime-based compilation target like Macromedia Flash bytecode: sure, it's "an ISA", but each op can cause extremely complex things to happen at the runtime level. The "runtime" of a physical mainframe would originally be "in hardware" — consisting of "accelerator" boards triggered by the CPU. But in z/VM, the "runtimes" of each "hardware profile" a workload might ask to be run under, are just regular old runtimes. Like Flash. Or a JVM. Or a software x86 emulator, with hardware paravirtualization care of x86 "application processors" living on accelerator cards.
To me, this abstraction — of a hypervisor that can support running workloads built for many different ISAs and microarchitectures, with these representing anything from low-level-emulated real machines (like x86) to high-level-virtualized real machines (like some specific mainframe) to high-level-virtualized abstract machines (like a JVM) — is obviously something that could have been used as "containerization" by those interested in doing so, even as far back as 1972's VM/370. And even those who weren't thinking in terms of "building a container", were still doing something very similar, in creating workloads as disk-image-based unikernels that assume full memory access, and which rely upon a single-tasking single-user exokernel library OS like DOS or CPM (but the z/VM equivalent, because they were writing programs that manipulated batch database records, not files, so regular DOS routines would be pretty useless.) Taking your "application disk" that you had developed to boot on your workstation, and uploading it byte-for-byte to your mainframe to be executed with 1000x concurrency over a batch workload, was exactly the sort of "shipping development to production" that we expect from containerization today.
I wasn't aware of that. Even more reasons to not call it a new thing then.
It reads like the advice of a mid-career engineer who is not aware that their time has been spent in a very small corner of the overall software engineering world.
I thought the same about every item in the list. A more honest title would be "questions to ask to determine if you think like me"
I don't know if it's a red flag but it's something that should be followed up on to determine if it's a sign of one. Just like there are situations where containers aren't appropriate there are situations where they most definitely are and a lack of nuance with reservations towards containers from a candidate would be a red flag for me.
Containers don't inherently reduce stability (quite the opposite in most cases). If I'm being generous to the author, they probably meant "unfounded apprehension", such as assuming out of ignorance that containers are somehow less stable.
Containers merely namespace/segment various OS resources.
Containers merely namespace/segment various OS resources.
An affection for containers are a green flag in some sense. Management probably has really low expectations if they accept “works on my system, let’s just ship my system” at scale as a solution.
Shipping containerized beta as production as a service, SCBaPaaS? You can promise fantastic turn-around, cut staff, and "crowdsource" testing! You offer a pretty dashboard and you are cooking with fire.
Shippable Containerized Actionalized Betas for Productive Ideation and Customer Creation.
Our goal is to minimize the difference between your environment and that of your customers. When you think “converged external and internal environments” think SCABPICC.
Our goal is to minimize the difference between your environment and that of your customers. When you think “converged external and internal environments” think SCABPICC.
Move to add 'and Retention' to the name so as to make the acronym SCABPICCR.
I mean, if you read the entire statement, it's clear that it's not exactly an unqualified statement. If your view on containers is that you've evaluated them and found that they aren't useful for your organizational needs, then it's not a red flag... but it's also not "apprehension toward containers."
Getting a response like this to the question "What are the firm’s views on containers?" is the actual red flag.
Use things you need and don't use things you don't. If someone makes a general statement that you need something for everything - that's a flag more red than Soviet Union's.
> Is everything glued together with a few bash scripts
The whole GNU/Linux world is glued together with a ton of {ba,}sh scripts (typically - there are exceptions, of course). Sometimes, all you need is another small one. And, of course, sometimes (and quite frequently) it's not a great idea. Depends on what you need to do and what it could evolve into.
> Apprehension toward containers is a red flag. With tools like KubeVirt, even VMs can now be run on platforms like k8s.
Same principle applies. Extra technology to set up and maintain is a liability. But if adding something helps you to remove a need for something more complex, then it's logically best to make smaller sacrifices than bigger ones. Unless, of course, you shift this liability to a contracted party and you're happy about it - that also works.
> Is everything glued together with a few bash scripts
The whole GNU/Linux world is glued together with a ton of {ba,}sh scripts (typically - there are exceptions, of course). Sometimes, all you need is another small one. And, of course, sometimes (and quite frequently) it's not a great idea. Depends on what you need to do and what it could evolve into.
> Apprehension toward containers is a red flag. With tools like KubeVirt, even VMs can now be run on platforms like k8s.
Same principle applies. Extra technology to set up and maintain is a liability. But if adding something helps you to remove a need for something more complex, then it's logically best to make smaller sacrifices than bigger ones. Unless, of course, you shift this liability to a contracted party and you're happy about it - that also works.
> Does prod hang on by a Perl script someone wrote in the early 00s, or is there an active drive to address technical debt? Consider a push toward web apps a green flag.
A script being in Perl is is not "technical debt". There is nothing wrong with Perl. On a personal level I don't especially like Perl, but it's a perfectly functional language that's fit for purpose and I would never claim there is something with with a script just because it's written in Perl.
Of course there can be plenty of good reasons to replace such a script: if it doesn't work well, if its too hard to adapt to new requirements, or something else. But merely "it's Perl" is a complete non-reason.
And such scripts don't need to be "old" in the first place; maybe people actually like Perl and is something that works well for them? So the assumption is some old crufty script is wrong in the first place.
This entire list is an enormous red flag of "I have one preferred way of doing things and anything else is wrong". These are the sort of people that will join a team, spend 2 years rewriting perfectly working systems to ... other perfectly working systems, and then leave after two years with a far greater mess than when they arrived for everyone else to clean up while they can spunk off keywords on their CV.
A script being in Perl is is not "technical debt". There is nothing wrong with Perl. On a personal level I don't especially like Perl, but it's a perfectly functional language that's fit for purpose and I would never claim there is something with with a script just because it's written in Perl.
Of course there can be plenty of good reasons to replace such a script: if it doesn't work well, if its too hard to adapt to new requirements, or something else. But merely "it's Perl" is a complete non-reason.
And such scripts don't need to be "old" in the first place; maybe people actually like Perl and is something that works well for them? So the assumption is some old crufty script is wrong in the first place.
This entire list is an enormous red flag of "I have one preferred way of doing things and anything else is wrong". These are the sort of people that will join a team, spend 2 years rewriting perfectly working systems to ... other perfectly working systems, and then leave after two years with a far greater mess than when they arrived for everyone else to clean up while they can spunk off keywords on their CV.
It definitely can be a reason. You need someone to maintain it, which means you need a Perl expert. Perl is relatively hard to hire for, and you really do need an expert in most cases, the Perl language being what it is. Unlike, say, Go or Python, you can't just hire someone that "knows" Perl, you probably do need an expert.
That's a completely different thing than merely "it's written in Perl".
What can you do in Perl that can't be done in Python? Everybody has bought into Python nowadays. Even Program Managers can bash out Python scripts.
Nothing. But that's the answer to "what can do in $X that can't be done in $Y" for most values of $X and $Y.
The point isn't about Perl or Python, it's about that if there's a working solution that works well then embarking on a project to "modernize" it for no reason other than the choice of language perceived to be "old-fashioned" (a completely subjective assessment) is almost always bad thing.
The point isn't about Perl or Python, it's about that if there's a working solution that works well then embarking on a project to "modernize" it for no reason other than the choice of language perceived to be "old-fashioned" (a completely subjective assessment) is almost always bad thing.
Would the following be a red flag?
"We promote modern languages like Python! We've had our own Python 1.5 based fork for 20 years now; there's some documentation, mostly from 10yr ago when the guy who started the project died. But we've got a good maintenance team and they understand almost all the magic he wrote in."
> Apprehension toward containers is a red flag.
"(almost) all our servers are in containers: we started buying cases after Ptyhon dude died, too. Previous server generations that live as stacks of unsecured hardware on shelves are being upgraded as can by the maintenance team."
"We promote modern languages like Python! We've had our own Python 1.5 based fork for 20 years now; there's some documentation, mostly from 10yr ago when the guy who started the project died. But we've got a good maintenance team and they understand almost all the magic he wrote in."
> Apprehension toward containers is a red flag.
"(almost) all our servers are in containers: we started buying cases after Ptyhon dude died, too. Previous server generations that live as stacks of unsecured hardware on shelves are being upgraded as can by the maintenance team."
I love python.
But not entirely sure it’s as new as go or rust. Old isn’t bad.
But not entirely sure it’s as new as go or rust. Old isn’t bad.
Very often old==tested==proven==goog
And sometimes old==python2==painful to work on.
Most things have pain points in some way at some point.
The pursuit of a relatively subjective panacea of programming languages is what keeps software developers ceaselessly borne into the past.
.like code aspiring Gatsbys reinventing the same libraries and frameworks in different programming syntaxes over and over and over.
The pursuit of a relatively subjective panacea of programming languages is what keeps software developers ceaselessly borne into the past.
.like code aspiring Gatsbys reinventing the same libraries and frameworks in different programming syntaxes over and over and over.
[deleted]
I assume you work in edge case testing :)
BOFH. Hiring for the maintenance team.
(Note: Not "a position on", you will be the team)
Also: "RHEL 6?" Bah. Modern heresy. RedHat 4.2 FTW.
(Note: Not "a position on", you will be the team)
Also: "RHEL 6?" Bah. Modern heresy. RedHat 4.2 FTW.
Uhh... Win 2008 DCs as well I assume I should be grateful for, right? Right?
Win98 on a personal airgapped system should you need it for testing.
your workstation will be aix, xenix, or solaris 1; dependng on which desk is free any given day.
OS diversity is a good thing. get here early, people fight for the Sun box, it has `xeyes`
your workstation will be aix, xenix, or solaris 1; dependng on which desk is free any given day.
OS diversity is a good thing. get here early, people fight for the Sun box, it has `xeyes`
This list reflects the patterns of thought that get continual transition to the current “best practice” technologies and ballooning cost with no real business value. I would rather know that the firm runs on a twenty year old Perl script orchestrating Common Lisp and COBOL services than hear that it’s “pushing modern languages”
for me as am experienced senior-engineer these are not "red flags" but challenges:
lets introduce modern ways of solving problems & automating processes, if they are feasible and meaningful ... (eg. sadly even container are not the proper tool for every use-case ... ;))
but a red flag would be for example, if those things are "carved in stone", eg.:
1. "if you work for us, you are not allowed to introduce modern methods", because $EXCUSE
2. we have to cling on to our inefficient/manual way of doing things, because $COLLEAGUE doesn't want to change his_hers habits originating from the mid to late 1990ties
3. etc...
been there, tried to change things despite such signs of resistance against change - didn't work out ... in every single case ;))
just my 0.02€
lets introduce modern ways of solving problems & automating processes, if they are feasible and meaningful ... (eg. sadly even container are not the proper tool for every use-case ... ;))
but a red flag would be for example, if those things are "carved in stone", eg.:
1. "if you work for us, you are not allowed to introduce modern methods", because $EXCUSE
2. we have to cling on to our inefficient/manual way of doing things, because $COLLEAGUE doesn't want to change his_hers habits originating from the mid to late 1990ties
3. etc...
been there, tried to change things despite such signs of resistance against change - didn't work out ... in every single case ;))
just my 0.02€
Same experience here, to a T. Equivocation, excuses, fear for their ancient jobs, I've seen it all, got the t-shirt and am recovering from the PTSD.
I've been fired with pride from places like this for butting heads against similar thinking (and of course, failing) and moved on to much greener pastures.
I've been fired with pride from places like this for butting heads against similar thinking (and of course, failing) and moved on to much greener pastures.
Someone asking some of these questions in an interview would be a red flag for me.
And here I thought I was the crazy one for having been burned by the negative answers to these questions in the past and thought I was the only one asking. It's been my experience that the actual content of the answer doesn't really matter, the technology or process could lean one way or another for all I care, it's the candor by which the questions are answered:
"oh god no we don't use container shit" (actual answer I've had) versus "it's something we're looking into and evaluating".
"why do we need to use config management shit" (actual answer, same company as above) versus "we've had some success with it, struggles otherwise, do you have experience that you could lend us with that?"
"change control? you mean doing everything as root and cowboying it doesn't count? what kind of idiot are you?" (less verbatim answer, same company, but hints at what I'm getting at).
If the interviewer answers like a child (and I have been the receiving end of that), I don't really care about the answers at all. If they are thoughtful, I'll considering going along with the rest of the process.
"oh god no we don't use container shit" (actual answer I've had) versus "it's something we're looking into and evaluating".
"why do we need to use config management shit" (actual answer, same company as above) versus "we've had some success with it, struggles otherwise, do you have experience that you could lend us with that?"
"change control? you mean doing everything as root and cowboying it doesn't count? what kind of idiot are you?" (less verbatim answer, same company, but hints at what I'm getting at).
If the interviewer answers like a child (and I have been the receiving end of that), I don't really care about the answers at all. If they are thoughtful, I'll considering going along with the rest of the process.
I will "sticky" this comment at the end of the blog. This is pretty much the entire point of these questions, to assess culture.
Gentlepeople! Remeber people have opinions, like they have fingers to type them...
These could be red flags for the author, I respect that. These are not universal, yet they could be good conversation starters.
Consider that if you don't hear the answer that you expect, maybe you can contribute something of great value to the company... Or receive a lesson on the complexities of a particular business.
Also, language-itis itself can be a problem. We love the shiny, and forget the old... I am not saying that punch cards and FORTRAN are the tool of the future, but maybe FORTRAN is the right tool for a specific problem...
I don't know if you have noticed... But programming languages could potentially mean very little, soon.
These could be red flags for the author, I respect that. These are not universal, yet they could be good conversation starters.
Consider that if you don't hear the answer that you expect, maybe you can contribute something of great value to the company... Or receive a lesson on the complexities of a particular business.
Also, language-itis itself can be a problem. We love the shiny, and forget the old... I am not saying that punch cards and FORTRAN are the tool of the future, but maybe FORTRAN is the right tool for a specific problem...
I don't know if you have noticed... But programming languages could potentially mean very little, soon.
"Apprehension toward containers is a red flag."
This is a garbage take. There are obvious security considerations that a firm may or may not be ready to take one that come with container use, much less organizational knowledge.
This is a garbage take. There are obvious security considerations that a firm may or may not be ready to take one that come with container use, much less organizational knowledge.
Thoughts on my list? What "green flag" questions do you like asking?
> What "green flag" questions do you like asking?
Though it wasn't a direct question i asked, i once turned down what would have otherwise been a dream job because the interviewer claimed that they worked at "an athletic pace." i.e. they ran like hell for too many hours per day. In my 20s that would have not bothered me, but i was close to 40 at the time and reaching the point where constant overtime was in no way appealing.
Though it wasn't a direct question i asked, i once turned down what would have otherwise been a dream job because the interviewer claimed that they worked at "an athletic pace." i.e. they ran like hell for too many hours per day. In my 20s that would have not bothered me, but i was close to 40 at the time and reaching the point where constant overtime was in no way appealing.
The blog is currently down for me, so here's an archived version in case it helps anyone else:
https://web.archive.org/web/20231117165517/https://blog.abct...
https://web.archive.org/web/20231117165517/https://blog.abct...
People use containers because their work (code, scripting around it, ...) is so fragile, it needs a very specific version of a specific Linux distro, and probably has a huge dependency footprint.
Being gung ho for containers is the red flag: it communicates that their shit is not clean, portable and with low dependencies.
Being gung ho for containers is the red flag: it communicates that their shit is not clean, portable and with low dependencies.
> Apprehension toward containers is a red flag.
Maybe they have been burned before? If your workload does well without containers, porting it to a container-based infrastructure and operating that is a huge sink of person hours with no business gain.
Maybe they have been burned before? If your workload does well without containers, porting it to a container-based infrastructure and operating that is a huge sink of person hours with no business gain.
Modern languages like Python was kind of amusing. Don't get me wrong Python serves it's purpose but it first appeared in 1991...
> Most large places have legacy tech you’ll work on, but an “ain’t broke don’t fix” approach to technical debt is a red flag. A push to web apps is a green flag.
Very happy to see this up near the top so I know I can disregard the rest of the article. That's the attitude that helps drive enshittification of everything.
Very happy to see this up near the top so I know I can disregard the rest of the article. That's the attitude that helps drive enshittification of everything.
> A push to web apps is a green flag.
This is the part that had me going 'WTF?!'.
I get that there is some signal there, but calling out 'web apps' as green flag pointing toward a better culture against technical debt had me confused.
This is the part that had me going 'WTF?!'.
I get that there is some signal there, but calling out 'web apps' as green flag pointing toward a better culture against technical debt had me confused.
Yeah, I almost feel like a better way to find red flags is to ask if I have to use web apps to manage systems or if there’s a nice suite of command line tools to use for day to day tasks.
A "Linux interview" is in of itself a huge red flag
Only at a hyperscaler should any job description be "Linux" - otherwise you should be interviewing for solving business problems - the solution to which could include Linux (but probably should be at a much higher level of abstraction)
Only at a hyperscaler should any job description be "Linux" - otherwise you should be interviewing for solving business problems - the solution to which could include Linux (but probably should be at a much higher level of abstraction)
The world does not divide between hyperscaler and not hyperscalers. Plenty of companies need linux specialised positions. Redhat, IBM and a small ISP are not hyperscalers.
Unqualified statements like that are unqualified B.S.
Not all environments gain anything from containers, and if you have "ultra-stability" requirements like, e.g., those found in banks, avoiding newfangled containers for oldfangled methods which are known to work well and for which the organization has long-proven fallbacks is not only legitimate, but arguably preferred.