One main problem that we observed was that big parts of their IAM / auth setup was overloaded / down which led to all kinds of cascading problems. It sounds as if Dynamo was reported to be a root cause, so is IAM dependent on dynamo internally?
Of course, such a large control plane system has all kinds of complex dependency chains. Auth/IAM seems like such a potentially (global) SPOF that you'd like to reduce dependencies to an absolute minimum. On the other hand, it's also the place that needs really good scalability, consistency, etc. so you probably like to use the battle proof DB infrastructure you already have in place. Does that mean you will end up with a complex cyclic dependency that needs complex bootstrapping when it goes down? Or how is that handled?
It's a bit funny that they say "most service operations are succeeding normally now" when, in fact, you cannot yet launch or terminate new EC2 instance, which is basically the defining feature of the cloud...
The max-unused percentage feature is well worth it to 80/20 the prune process and only prune the data which is easiest to prune away (i.e. not try to remove small files big packs but focus on packs which have lots of garbage).
In general, there's an unavoidable trade-off between creating many small packs (harder on metadata throughout the system, inside restic and on the backing store but more efficient to prune) versus creating big packs which are more easy on the metadata but might create big repack cost.
I guess a bit more intelligent repacking could avoid some of that cost by packing stuff together that might be more likely to get pruned together.
Isn't he dismissing culture with his rant? Culture as the currently common way to do and describe things?
He seems to imply that there could be an easier, more straight-forward way to describe things in some more common language. And that while he doesn't give any evidence how the current ways are overly complex.
Of course, there is broken or outdated software, and some things were crap from the start. Of course, there are always concrete things to improve but you won't get anywhere by dismissing all of it and starting anew.
For me, understanding the current state as part of our culture and our humanity and improving gradually on it, has guided me well in the past.
Actually, that the malware contained a root exploit is fixable sooner or later.
The next insight will be, that even if the sandbox had worked, this type of attack still is possible by using the user's trust in the brand of a well-known app to use the permissions granted to it for malicious intents. There's no easy way to avoid that up front automatically.
The downside is that a bunch of processes started from one TTY doesn't get as much CPU as before.
It basically shifts the scheduling granularity a level higher from processes to (interactive) sessions. Because that's what the question is: on which level do we want to have a fair scheduling? For a desktop user, processes have little meaning. Sessions, instead, are much more useful because they correspond better to his different tasks, for which he expects that the CPU power is distributed in a fair way.
What's the reasoning for your reproduction of the complete text here?
With all possibly valid reasoning in this particular case, there should be high barriers for just copying someone else's text, esp. here given its personal nature.
An interesting side note here is that the coming Unity will be based upon compiz, instead of relying on mutter. Canonical apparently hired the lead developer of compiz:
I think that's a valid point. For long years mostly the live performance of media was paid for. Then with the manifestation of media as data, the guys who had the machines to record and create copies claimed the stakes and the exclusive rights as a form to create economic value. Time has passed and now we have found out that every data are equal and data can hardly be bounded and the exclusivity of copying has gone away. Perhaps it's time to rethink the claims.
I think so, too. If you think about what is making Android devices run so slow, it's almost always some misbehaving background services. Unless there isn't some smarter way controlling the scheduling of foreground applications, there won't be multitasking and fast speed at the same time with a comparable CPU.
About Javascript: "Javascript is a fairly poorly thought-out language. Considering its influences were Scheme and Smalltalk it's rather sad what came out - but it got lucky, it's in a crucial place, it simply fell into the right place in the universe."
I had great expectations when I heard from Wolfram Alpha. The things Mathematica is able to do backed by a giant structured data store seemed mind blowing.
As the author said the visualization tools and the (still small) dataset _is_ impressive. I thought: We have all this data, all this visualization options and now we can create aggregate and munch all the facts of the world with an ad-hoc query language. But this simply does not work. There is hardly any functionality to really process data (aside from the predefined ways). And if you come to think about it: One cannot think of a way more complex transformations could be expressed with WA's natural language interface.
For me a better interface would look like this: The basic interface is some kind of full-featured expression language to navigate the data hierarchy (perhaps similar to SQL). You could then build some GUI to interactively navigate/parametrize data/transformations to build more complex expressions. To make the system more easily accessible you could then - and only then - add the natural language recognition system on top of this, which tries to guess some expressions from your input string and gives you some suggestions to start from.
The authors speculate that this is because some people just can't handle the meaninglessness of programming
I think he cited the paper wrong here: It's not the meaninglessness of programming but the meaninglessness of programs. And it is obviously not meant, that a program makes no sense, but that a formalism can stand on its own, without any purpose. Programming, however, is creating programs for a purpose, thus, abstracting facts into a formalism and after running the program interpreting its results is not meaningless at all.
I think I can back the observations of the original paper. I occured people which seem to plainly don't get what a program does. They are not talking about what they themselves told the computer to do but they are speculating why the program/computer isn't doing what it ought to do (what its purpose is).
I disagree with Steve's opinion that the level of abstraction is a general problem. I started programming early, and I hit walls all the time. Back from the early times to now I always found things I just couldn't grok. For long years I couldn't understand programming graphical things at all. I watched with awe how my three years older neighbor could program animations on his C64, all I managed to do were Fahrenheit to Celsius converters. Later I was crazy enough to start learning C++ with only knowing BASIC before. It took days to get 'Hello World' running. I kept trying and eventually I found the right book which made me understand OOP. Programming some graphics came very late. And the list still goes on...
I can read a book without understand a thing. I come back to the book (or suddenly remember details) some weeks, months, years later and manage to understand what is going on.
The same applies to other areas: Playing piano is something where you hit your personal wall all the time. Sure, there is a natural anatomical caused level of virtuosity which simply can't be overcome. (See Robert Schumann, who tried to hack his weakest finger which ultimately failed) But there are "mental barriers" which can only be overcome (aside from exercising) by keeping cool and sometimes stop trying to come back to it later.
Another example: As a child I always divided the physical world from the living world (the christian heritage?). As such it was for a long time unthinkable that creatures are made from the same stuff as everything else. I recall that accepting this knowledge was really hard...
Regarding the original paper (they've done much research, I can only guess and tell from observation): The humps they observe, couldn't they just be the indication of a clear mental barrier which has to be overcome? Perhaps being taught and some pressure is just not enough to grok these abstract facts about programming? Perhaps, what is needed is just time to (un)consciously chew on the ideas? So, shouldn't be children exposed to 'some programming stuff' as early as possible?
In Germany at least it is left to chance when children become exposed to programming. In most regions there is no teaching at all before a pupil is about 16 years old (11-th grade).
Of course, such a large control plane system has all kinds of complex dependency chains. Auth/IAM seems like such a potentially (global) SPOF that you'd like to reduce dependencies to an absolute minimum. On the other hand, it's also the place that needs really good scalability, consistency, etc. so you probably like to use the battle proof DB infrastructure you already have in place. Does that mean you will end up with a complex cyclic dependency that needs complex bootstrapping when it goes down? Or how is that handled?