Why does sorting in computer science mean ordering rather than categorizing?(twitter.com)
twitter.com
Why does sorting in computer science mean ordering rather than categorizing?
https://twitter.com/random_walker/status/1036704758696960001
152 comments
No, that's a misreading of computing history. The big problem, all the way into the early 1970s, was storage, not compute power. Leibniz built the first mechanical multiplier in 1694. But significant amounts of storage (kilobytes) didn't become available until the 1950s, and memory was still a million dollars a megabyte in the early 1970s.
Read the history of the IBM 400 and 600 series, the business line from IBM. First machines that could add and count, then print, then multiply, then divide, and then a long struggle to get some memory. Machines had a few memory locations, then tens of memory locations, then a hundred or so. The people involved knew they could do more if they could store more info. It wasn't a conceptual problem. Finally they got to the IBM 650, with drum memory, the first business machine we'd recognize today as a computer. Knuth learned on one of those.
Read "IBM's Early Computers", von Neumann's report on the EDVAC, learn what Zuse and Atanasoff did, find an Analytical Engine simulator and learn what the machine could do, and read up on the history of punched card and tote machines.
Read the history of the IBM 400 and 600 series, the business line from IBM. First machines that could add and count, then print, then multiply, then divide, and then a long struggle to get some memory. Machines had a few memory locations, then tens of memory locations, then a hundred or so. The people involved knew they could do more if they could store more info. It wasn't a conceptual problem. Finally they got to the IBM 650, with drum memory, the first business machine we'd recognize today as a computer. Knuth learned on one of those.
Read "IBM's Early Computers", von Neumann's report on the EDVAC, learn what Zuse and Atanasoff did, find an Analytical Engine simulator and learn what the machine could do, and read up on the history of punched card and tote machines.
Regardless of whether your correct or not, this is still a disrespectful comment.
Nevertheless, he's referring to Ada Lovelace's comments from 1843 and saying they weren't understood for ~100 years i.e. 1940s. That pre-dates most of what you're talking about.
In the twitter thread the OP clearly has read up on the history of punched cards and the history of IBM machines.
Nevertheless, he's referring to Ada Lovelace's comments from 1843 and saying they weren't understood for ~100 years i.e. 1940s. That pre-dates most of what you're talking about.
In the twitter thread the OP clearly has read up on the history of punched cards and the history of IBM machines.
Didn't read as disrespectful at all to me, just a firm rebuttal.
OK, that was a bit harsh.
My point is that machine arithmetic preceded machine data storage. Early computing was a struggle to get something done with very, very little memory. Like 5 to 20 numbers. Babbage wanted to build something the size of a locomotive for storage - a huge drum of counter wheels, able to store 1000 numbers. Atanasoff had a rotating wheel of capacitors, refreshed on each turn, like DRAM, which was good thinking for 1939. The code-breaking machines of WWII were all very limited memory, more like a Bitcoin ASIC than a general purpose computer. IBM was plugging away with punched cards, which were their form of permanent data storage. Shannon wrote a paper on the minimum memory requirements for a telephone switch.
In the IBM punched card world, "sorting" meant ordering, and distributing into groups was "selecting". Sorting was usually required before selecting, but if you only needed two or three output groups, you could select cards using a collator, which was capable of moderately complex logic operations. You might do that for "customers who are past due", for example.
Building a stored program computer wasn't a conceptual problem. It was that, despite massive efforts, there was nothing to store the program in. The concept of algorithms dates back to at least the 9th century, and probably back to Euclid.
(Although there's a long history of programmable cam-driven machinery. I've seen the Jaquet-Droz automata. built in the 1770s, demonstrated at the museum in Neuchâtel, where they are operated once a month. That was the high point of cam driven machinery for a long time.)
My point is that machine arithmetic preceded machine data storage. Early computing was a struggle to get something done with very, very little memory. Like 5 to 20 numbers. Babbage wanted to build something the size of a locomotive for storage - a huge drum of counter wheels, able to store 1000 numbers. Atanasoff had a rotating wheel of capacitors, refreshed on each turn, like DRAM, which was good thinking for 1939. The code-breaking machines of WWII were all very limited memory, more like a Bitcoin ASIC than a general purpose computer. IBM was plugging away with punched cards, which were their form of permanent data storage. Shannon wrote a paper on the minimum memory requirements for a telephone switch.
In the IBM punched card world, "sorting" meant ordering, and distributing into groups was "selecting". Sorting was usually required before selecting, but if you only needed two or three output groups, you could select cards using a collator, which was capable of moderately complex logic operations. You might do that for "customers who are past due", for example.
Building a stored program computer wasn't a conceptual problem. It was that, despite massive efforts, there was nothing to store the program in. The concept of algorithms dates back to at least the 9th century, and probably back to Euclid.
(Although there's a long history of programmable cam-driven machinery. I've seen the Jaquet-Droz automata. built in the 1770s, demonstrated at the museum in Neuchâtel, where they are operated once a month. That was the high point of cam driven machinery for a long time.)
First off, thanks for the comment—pretty interesting stuff. One thing continues to bother me though.
> My point is that machine arithmetic preceded machine data storage.
> The concept of algorithms dates back to at least the 9th century, and probably back to Euclid.
I don't think that's the issue at hand. Special-purpose computers implement algorithms and may use arithmetic—those are not the key differentiators from general purpose computation. Perhaps they were important first steps, but clearly what e.g. Turing/Church did was something else entirely. And I'm aware Leibniz was very much concerned with the problem earlier on, but where I get a bit fuzzy is whether general purpose computers really needed the more fleshed out results of e.g. Turing (or perhaps a predecessor) in order to really build a general purpose computer—and whether that was a major limiting factor even after we had access to the necessary memory, or if we knew conceptually that what we were shooting for all along was fully general purpose computation and we were really just waiting for the hardware to be possible.
> My point is that machine arithmetic preceded machine data storage.
> The concept of algorithms dates back to at least the 9th century, and probably back to Euclid.
I don't think that's the issue at hand. Special-purpose computers implement algorithms and may use arithmetic—those are not the key differentiators from general purpose computation. Perhaps they were important first steps, but clearly what e.g. Turing/Church did was something else entirely. And I'm aware Leibniz was very much concerned with the problem earlier on, but where I get a bit fuzzy is whether general purpose computers really needed the more fleshed out results of e.g. Turing (or perhaps a predecessor) in order to really build a general purpose computer—and whether that was a major limiting factor even after we had access to the necessary memory, or if we knew conceptually that what we were shooting for all along was fully general purpose computation and we were really just waiting for the hardware to be possible.
See Eckert's work at Columbia University.[1] In the early 1930s, IBM came out with punched card equipment that could multiply. Eckert managed to kludge several IBM tabulating machines into a sort of programmable computer. "The control box had settings that told the different machines what they should be doing at a given time and these gave the broad mechanical programming of the problem. The punches on the cards were used to program the specific details." This was all electromechanical, with relays, gears, and clutches, not vacuum tubes. IBM eventually turned that mess into the IBM Card Programmed Calculator. Still not enough memory for programs. Eckert went on to design many more computing machines, including the ENIAC. Finally, no moving parts, but still mostly plugboard-wired programs.
Eckert is credited with inventing delay line memory.[2] That was the first useful computer memory technology. Suddenly, everybody in number-crunching was building delay line computers - EDVAC, EDSAC, LEO, IAS, UNIVAC I... All were stored program machines. As soon as there was memory, people started putting programs in it. That's when computing took off.
[1] http://www.columbia.edu/cu/computinghistory/brennan/index.ht... [2] https://en.wikipedia.org/wiki/Delay_line_memory
Eckert is credited with inventing delay line memory.[2] That was the first useful computer memory technology. Suddenly, everybody in number-crunching was building delay line computers - EDVAC, EDSAC, LEO, IAS, UNIVAC I... All were stored program machines. As soon as there was memory, people started putting programs in it. That's when computing took off.
[1] http://www.columbia.edu/cu/computinghistory/brennan/index.ht... [2] https://en.wikipedia.org/wiki/Delay_line_memory
Interesting—though I'm not convinced that the IBM approach to generalizing computation would have led us to general purpose computers as we know them now. Sure, it was more general than what had come before, allowing some parameterization of behavior—but 'more general' and 'universal' may be worlds apart (I don't know enough of the details of IBMs machines to say how much though...). Or even an architecture technically capable of complete generality, but not doing it so well, may be worlds apart from an architecture based on a simple theoretical grounding of that complete generality.
Eckert it seems was inspired by Von Neumann whom he worked with on the ENIAC, and Neumann was an early appreciator of Turing's work on universal computation(from 1936)—no surprise that his eponymous architecture mirrors a universal turing machine not only in power but in form (i.e. "the concept of a computer able to store in its memory its program of activities and of modifying that program in the course of these activities"[1]).
In view of that, I'm inclined to give similar importance to Turing's theoretical work still. Maybe I'm overestimating the importance of the architectural insights it led to—but I wonder if, say, we even had the Lambda Calculus, but no Turing—would we have fully generally programmable computers as we do now (which are as general in practice as well as in theory, if that makes sense)? I'm not sure...
[1] http://www.alanturing.net/turing_archive/pages/Reference%20A...
Eckert it seems was inspired by Von Neumann whom he worked with on the ENIAC, and Neumann was an early appreciator of Turing's work on universal computation(from 1936)—no surprise that his eponymous architecture mirrors a universal turing machine not only in power but in form (i.e. "the concept of a computer able to store in its memory its program of activities and of modifying that program in the course of these activities"[1]).
In view of that, I'm inclined to give similar importance to Turing's theoretical work still. Maybe I'm overestimating the importance of the architectural insights it led to—but I wonder if, say, we even had the Lambda Calculus, but no Turing—would we have fully generally programmable computers as we do now (which are as general in practice as well as in theory, if that makes sense)? I'm not sure...
[1] http://www.alanturing.net/turing_archive/pages/Reference%20A...
Eckert was building working hardware in the early 1930s, long before von Neumann was involved.
There was a performance penalty to doing only one thing at a time, the stored program way. Stored programs have a lot of overhead - fetching instructions, decoding instructions, doing instructions that didn't directly do arithmetic - that were not present in the plugboard-wired machines. Those usually did multiple operations on each cycle. When clock rates were a few kilohertz, this mattered, a lot.
One result was a split between "scientific" and "business" computers. Scientific computers were usually binary, funded by the military, and were mostly one-off machines in the early days. Business machines were decimal, had to be cost-effective, and were mass produced. The two sides finally came together with the IBM System/360. By that point, both were stored-program.
As for an "an architecture technically capable of complete generality, but not doing it so well", that was the IBM 1401, a very successful machine with a very strange architecture. The Computer Museum in Mountain View has two of them working. It was a true stored program computer, quite different from any of the scientific computers. It had a much lower cost and parts count, and ran most of America's middle sized businesses for years.
There was a performance penalty to doing only one thing at a time, the stored program way. Stored programs have a lot of overhead - fetching instructions, decoding instructions, doing instructions that didn't directly do arithmetic - that were not present in the plugboard-wired machines. Those usually did multiple operations on each cycle. When clock rates were a few kilohertz, this mattered, a lot.
One result was a split between "scientific" and "business" computers. Scientific computers were usually binary, funded by the military, and were mostly one-off machines in the early days. Business machines were decimal, had to be cost-effective, and were mass produced. The two sides finally came together with the IBM System/360. By that point, both were stored-program.
As for an "an architecture technically capable of complete generality, but not doing it so well", that was the IBM 1401, a very successful machine with a very strange architecture. The Computer Museum in Mountain View has two of them working. It was a true stored program computer, quite different from any of the scientific computers. It had a much lower cost and parts count, and ran most of America's middle sized businesses for years.
It’s a tad condescending.
It asserts authoritatively that is a misreading of computer science history - in history, there is rarely a single “correct reading”, but rather many interpretations that at time reinforce and at times weaken each other.
Using the imperative “read”, “learn”, etc is also heavy handed and unnecessary. It’s the kind of language bad teachers - the teachers who have to use their authority rather than mentorship skills - tend to use (“Check out those books for more” flows much better, for instance).
As a result, the comment comes a little too much across as “no, you don’t have the knowledge, I have the knowledge!” which is sadly can be a little too common in hacker circles and doesn’t really encourage open, fruitful conversations the way “You know stuff, I know other stuff, let’s put it together and see what we come up with” does.
It asserts authoritatively that is a misreading of computer science history - in history, there is rarely a single “correct reading”, but rather many interpretations that at time reinforce and at times weaken each other.
Using the imperative “read”, “learn”, etc is also heavy handed and unnecessary. It’s the kind of language bad teachers - the teachers who have to use their authority rather than mentorship skills - tend to use (“Check out those books for more” flows much better, for instance).
As a result, the comment comes a little too much across as “no, you don’t have the knowledge, I have the knowledge!” which is sadly can be a little too common in hacker circles and doesn’t really encourage open, fruitful conversations the way “You know stuff, I know other stuff, let’s put it together and see what we come up with” does.
> ... in history, there is rarely a single “correct reading”, but rather many interpretations that at time reinforce and at times weaken each other.
As I was reading the GP's interpretation of these historical events, it occurred to me how difficult it really is to get at a reliable interpretation of something like this: I've already read a couple books on the subject—but I can see room the cracks so to speak, where the GP's view may be more accurate.
That said, difficulty isn't the same as impossibility—and yes, it is technically impossible (or at least meaningless) for certain definitions of 'correct' interpretation, but if you're okay with correct up to "for all intents and purposes"—then it's a non-issue. For instance, in this case there is a pretty definite question we'd like answered: which innovation was the limiting factor on our original development of general purpose computation: increased memory capacity, or theoretical knowledge of general purpose computation—or was their role more alike than different?
Depending on how things actually played out, there are definite things you can say about that. For instance, perhaps we need at least 500 bits of memory for general purpose computation—did we have access to that while no one had yet thought to build the general purpose computer because we were still waiting for Zuse, Turing/Church? If so, we answer one way; if not, we answer the other way. The only case where it gets endlessly complicated, perhaps giving apparent grounds for claiming impossibility of correct interpretation, actually fits neatly into the last option I mentioned in the original question statement: that there wasn't a significant difference in the two innovations' roles as limiting factors.
It seems valuable to understand the difficult and limits of interpretation, but I think more harm is being done than good speaking so generally of "correct readings" being impossible.
> Using the imperative “read”, “learn”, etc is also heavy handed and unnecessary. It’s the kind of language bad teachers - the teachers who have to use their authority rather than mentorship skills - tend to use (“Check out those books for more” flows much better, for instance).
I really wish this was better known. It's a pretty consistent pattern and useful to be able to recognize. My read on it is that it's typically used as a defensive strategy—a way to prevent an interlocutor from questioning the speaker any further on the subject.
As I was reading the GP's interpretation of these historical events, it occurred to me how difficult it really is to get at a reliable interpretation of something like this: I've already read a couple books on the subject—but I can see room the cracks so to speak, where the GP's view may be more accurate.
That said, difficulty isn't the same as impossibility—and yes, it is technically impossible (or at least meaningless) for certain definitions of 'correct' interpretation, but if you're okay with correct up to "for all intents and purposes"—then it's a non-issue. For instance, in this case there is a pretty definite question we'd like answered: which innovation was the limiting factor on our original development of general purpose computation: increased memory capacity, or theoretical knowledge of general purpose computation—or was their role more alike than different?
Depending on how things actually played out, there are definite things you can say about that. For instance, perhaps we need at least 500 bits of memory for general purpose computation—did we have access to that while no one had yet thought to build the general purpose computer because we were still waiting for Zuse, Turing/Church? If so, we answer one way; if not, we answer the other way. The only case where it gets endlessly complicated, perhaps giving apparent grounds for claiming impossibility of correct interpretation, actually fits neatly into the last option I mentioned in the original question statement: that there wasn't a significant difference in the two innovations' roles as limiting factors.
It seems valuable to understand the difficult and limits of interpretation, but I think more harm is being done than good speaking so generally of "correct readings" being impossible.
> Using the imperative “read”, “learn”, etc is also heavy handed and unnecessary. It’s the kind of language bad teachers - the teachers who have to use their authority rather than mentorship skills - tend to use (“Check out those books for more” flows much better, for instance).
I really wish this was better known. It's a pretty consistent pattern and useful to be able to recognize. My read on it is that it's typically used as a defensive strategy—a way to prevent an interlocutor from questioning the speaker any further on the subject.
In case downvotes on the above confuse others, the comment Animats made is disrespectful but they are not some random Internet commentor, and in Jobsian fashion, disrespect is cool so long as you can back it up (and John Nagle can back it up).
Well I didn't read it as disrespectful, and that's my explanation of the downvotes (which doesn't have any value and that I only offer as a counterpoint here). I have no idea who that poster is, I don't care, and that shouldn't be a factor when considering rudeness tbh. As long as you can back up your reasonable claims and don't attack the person behind the keyboard, what's disrespectful ?
Or is arguing considered rude now ?
Or is arguing considered rude now ?
He's telling the OP to go and read up on various books with the implication that they haven't read anything on the topic. Where as from the twitter thread they clearly have read books on the topics mentioned (IBM and punch cards).
From the guidelines:
> Be civil. Don't say things you wouldn't say face-to-face.
I don't think the comment made by Animats meets this guideline.
From the guidelines:
> Be civil. Don't say things you wouldn't say face-to-face.
I don't think the comment made by Animats meets this guideline.
> He's telling the OP to go and read up on various books with the implication that they haven't read anything on the topic.
It is possible that OP hasn't read a sufficient amount of books, or read the wrong books, or misunderstood parts of the books he read.
The implication you are taking offense to (OP hasn't read books) does not follow from that.
It is possible that OP hasn't read a sufficient amount of books, or read the wrong books, or misunderstood parts of the books he read.
The implication you are taking offense to (OP hasn't read books) does not follow from that.
Out of curiosity, if the comment started with "In my opinion that's a misreading of computer history...", with everything else left unchanged, would it still be uncivil in your opinion?
Yes, in my opinion it still would be uncivil as per my above comment:
> He's telling the OP to go and read up on various books with the implication that they haven't read anything on the topic.
that to me is disrespectful. But the whole tone of the comment was dismissive, so changing the first part makes it less so, but not entirely.
> He's telling the OP to go and read up on various books with the implication that they haven't read anything on the topic.
that to me is disrespectful. But the whole tone of the comment was dismissive, so changing the first part makes it less so, but not entirely.
I see. For me, dismissive is one thing (and I could agree with that), uncivil is something much worse.
I typically say "I recommend" rather than "read".
Arguing isn't rude, but phrasing comments as reading assignments somewhat implies ignorance.
I'm aware of who Animats is but the OP is also a Princeton Professor of CS. Given that Animats has so much reputation on here, I'd hope he would be more likely to be civil and uphold the guidelines.
To be very blunt, this appears so nuanced as to not merit discussion.
I wouldn't call it a politely worded comment. That's all.
I wouldn't call it a politely worded comment. That's all.
The real problem is that Animats' comment is very misguided, so doesn't justify its own tone.
This is a case of someone jumping in with both feet and not realizing they're at the wrong pool.
This is a case of someone jumping in with both feet and not realizing they're at the wrong pool.
“We are all smart here. Distinguish yourself by being kind”.
> what abstractions are we missing today that will be obvious to future generations?
I think we're missing a science of meaning, and (closely related) trust.
We have a science of raw data and bits (basic thermodynamics into information theory), a science of sending streams of bits (information theory), and provide rigorous numerical bounds for e.g. if you know the medium, how quickly one can signal.
We don't have a predictive, numerical theory of meaning, which is how interpretations of data or information cohere to the real world and how those meanings can be trusted and transmitted from one another in a robust way.
We're seeing the failure of meaning all around us today as we're drowned by real data and fake data and noise, as competing voices and groups spread selfish memes and narratives that ultimately obscure truth.
We're seeing the failure of meaning as our overcompetition in academia spawns a raft of fake or non-replicable studies and p-hacking, threatening to drown the signal with noise.
Anyway, there are hints of a 'science of meaning' in different fields: Graphical statistical models hint at how causality can be inferred from downstream facts, game theory tells us some (very limited) conditions under which selfish players can learn to trust one another, evolutionary theory tells us how groups can learn to cooperate and share models of the world. But none of these provide predictive and quantitative bounds, so as of yet, meaning is an art left for cultural leaders and manipulated by advertisers and politicians, and not yet a science.
I think we're missing a science of meaning, and (closely related) trust.
We have a science of raw data and bits (basic thermodynamics into information theory), a science of sending streams of bits (information theory), and provide rigorous numerical bounds for e.g. if you know the medium, how quickly one can signal.
We don't have a predictive, numerical theory of meaning, which is how interpretations of data or information cohere to the real world and how those meanings can be trusted and transmitted from one another in a robust way.
We're seeing the failure of meaning all around us today as we're drowned by real data and fake data and noise, as competing voices and groups spread selfish memes and narratives that ultimately obscure truth.
We're seeing the failure of meaning as our overcompetition in academia spawns a raft of fake or non-replicable studies and p-hacking, threatening to drown the signal with noise.
Anyway, there are hints of a 'science of meaning' in different fields: Graphical statistical models hint at how causality can be inferred from downstream facts, game theory tells us some (very limited) conditions under which selfish players can learn to trust one another, evolutionary theory tells us how groups can learn to cooperate and share models of the world. But none of these provide predictive and quantitative bounds, so as of yet, meaning is an art left for cultural leaders and manipulated by advertisers and politicians, and not yet a science.
A lot of the cultural and political issues that surround digital technology appear because there's a predisposition to consider all bits equally valuable and useful.
Another word for meaning could be quality. Socially and culturally, not all information is of equivalent quality and value.
In fact we use the profit motive and political power as metrics of quality, and in practice they're turning out to be a bad way to maximise long-term social value.
Another word for meaning could be quality. Socially and culturally, not all information is of equivalent quality and value.
In fact we use the profit motive and political power as metrics of quality, and in practice they're turning out to be a bad way to maximise long-term social value.
Rather, the limitation was that people didn't have the abstractions, vocabulary, and mental tools to properly conceive of general purpose computers as a concept and to understand their usefulness.
Maybe that was it or maybe society had not achieved a scale and a regularity where a more primitive and expension version of a programmable computer would have been useful (ie, where the cost-benefit exchange was obvious). I'd note that computers effectively came into use during WWII, where there was immense pressure to optimize the large-scale organizational activity as well as large scale industrial processes (the Manhattan Project esp).
Maybe that was it or maybe society had not achieved a scale and a regularity where a more primitive and expension version of a programmable computer would have been useful (ie, where the cost-benefit exchange was obvious). I'd note that computers effectively came into use during WWII, where there was immense pressure to optimize the large-scale organizational activity as well as large scale industrial processes (the Manhattan Project esp).
There was also an era of “human computers”, and when someone ultimately decided to automate their jobs away, people gradually realized that a computer could do things we never thought of as computation before, and that information could be encoded as numbers, and a lot of wild things like that.
One thing I learned in college was a means of encoding a secret: n people all learn a separate point on the curve and the degree of the polynomial for the curve, and you encode the secret as the Y-intercept. That way any k of n of those people (depending on the degree of the polynomial) can recover the secret. I thought, that’s not so hard, and I asked why this wasn’t discovered centuries ago. It’s because encoding arbitrary information as a number is an idea that never occurred to anyone before computers, aside from things like bible codes and numerology that are usually lossy anyway. But you can do this with a pad and paper if you wanted.
One thing I learned in college was a means of encoding a secret: n people all learn a separate point on the curve and the degree of the polynomial for the curve, and you encode the secret as the Y-intercept. That way any k of n of those people (depending on the degree of the polynomial) can recover the secret. I thought, that’s not so hard, and I asked why this wasn’t discovered centuries ago. It’s because encoding arbitrary information as a number is an idea that never occurred to anyone before computers, aside from things like bible codes and numerology that are usually lossy anyway. But you can do this with a pad and paper if you wanted.
That’s Shamir’s secret sharing algorithm. Very clever, yet easy to understand.
The one caveat is that it should be done over a finite field.
The one caveat is that it should be done over a finite field.
Who was the guy who patiently explained IN THE 70S to an interviewer why general purpose computers would be needed? I remember hearing the interview and the guy genuinely couldn’t fathom why they’d be needed to replace regular libraries and so on.
This is really great stuff you are collecting. Would be awesome to read it in a more organized manner than a twitter posts: twitter really isn't so great for organizing knowledge. Don't you keep a blog or something? Maybe considering?
By the way, I remember you by your course on coursera, it was great. Thank you a lot for that.
By the way, I remember you by your course on coursera, it was great. Thank you a lot for that.
Thanks!
The main reason I'm using Twitter for this is that it's a bit too preliminary for a blog post. I don't yet have as good an understanding of the history as I would like. This way, when I discover new stuff, I can simply add at a tweet to the thread.
But TBH I think Twitter is underrated as a publishing medium. For example, I've had probably 10x the number of responses from other people as I would have gotten in the form of blog comments.
In any case, I'm definitely planning to make this more organized once I'm happy with my level of understanding. At least a series of blog posts; probably a paper and/or online lecture.
The main reason I'm using Twitter for this is that it's a bit too preliminary for a blog post. I don't yet have as good an understanding of the history as I would like. This way, when I discover new stuff, I can simply add at a tweet to the thread.
But TBH I think Twitter is underrated as a publishing medium. For example, I've had probably 10x the number of responses from other people as I would have gotten in the form of blog comments.
In any case, I'm definitely planning to make this more organized once I'm happy with my level of understanding. At least a series of blog posts; probably a paper and/or online lecture.
> But TBH I think Twitter is underrated as a publishing medium. For example, I've had probably 10x the number of responses from other people as I would have gotten in the form of blog comments.
What was your impression of the quality of those comments, compared to the ones you get on blog comments?
What was your impression of the quality of those comments, compared to the ones you get on blog comments?
> What abstractions are we missing today that will be obvious to future generations?
The first that comes to mind is rooted in the problem that our concept of "tech" is still incredibly mechanistic. Look at Foucault's "technologies of the self" and you realize that technology is not just something we create in teams at work using this or that set of tools. Rather than a "thing" outside of us, tech is very much already a part of us. We are compelled to develop it (externalize it; give it objective manifestation) simply by living.
A computer is us, but even the word computer is no longer sufficient; people needlessly mechanize humans when they think in those terms and this creates fear, fear of AI and fear of becoming mere batteries; meanwhile our fault-protection circuits have already accounted for this and we can trust them (while leaving them engaged).
We must zoom even farther out from the typical computer model in order to compute at the next level. "Tech" as an industry (in hindsight to be found a ridiculous and temporary construction) has turned too far inward. In fact there is nothing to fear outside, even in the foreign-to-tech irrational world. We can accept that world, live in it, give it the respect it seems now to unfairly demand, and learn it as a technological mode. We can bridge with it, build more technologies to abstract our relationship with it. At that point we become truly powerful, able to spin technology out of limitless resources and into limitless solutions.
The first that comes to mind is rooted in the problem that our concept of "tech" is still incredibly mechanistic. Look at Foucault's "technologies of the self" and you realize that technology is not just something we create in teams at work using this or that set of tools. Rather than a "thing" outside of us, tech is very much already a part of us. We are compelled to develop it (externalize it; give it objective manifestation) simply by living.
A computer is us, but even the word computer is no longer sufficient; people needlessly mechanize humans when they think in those terms and this creates fear, fear of AI and fear of becoming mere batteries; meanwhile our fault-protection circuits have already accounted for this and we can trust them (while leaving them engaged).
We must zoom even farther out from the typical computer model in order to compute at the next level. "Tech" as an industry (in hindsight to be found a ridiculous and temporary construction) has turned too far inward. In fact there is nothing to fear outside, even in the foreign-to-tech irrational world. We can accept that world, live in it, give it the respect it seems now to unfairly demand, and learn it as a technological mode. We can bridge with it, build more technologies to abstract our relationship with it. At that point we become truly powerful, able to spin technology out of limitless resources and into limitless solutions.
Regarding externalized systems, a relevant quote from the classic computer game Deus Ex (2000):
> Helios: The checks and balances of democratic governments were invented because human beings themselves realized how unfit they were to govern themselves. They needed a system, yes, an industrial-age machine.
> JC Denton: Human beings may not be perfect, but a computer program with language synthesis is hardly the answer to the world's problems.
> Helios: Without computing machines, they had to arrange themselves in crude structures that formalized decision-making - a highly imperfect, unstable solution.
> Helios: The checks and balances of democratic governments were invented because human beings themselves realized how unfit they were to govern themselves. They needed a system, yes, an industrial-age machine.
> JC Denton: Human beings may not be perfect, but a computer program with language synthesis is hardly the answer to the world's problems.
> Helios: Without computing machines, they had to arrange themselves in crude structures that formalized decision-making - a highly imperfect, unstable solution.
O'Neill likewise skewers the human-condition-social-upgrade-solution (referring to various -isms like Socialism) in his futurism work, _2081_. IMO it's the bipolar casting of the thought process there that is "wrong" if anything is. We are gradually becoming capable of merging and finessing subject and object using various technologies (I count therapeutic-environment-building and institutional life-examination-encouragement as examples of those technologies, but wouldn't exclude something more mechanistic than those) to where the question of "did a computing machine help or was it old fashioned social structuring" will seem pretty uneducated if it doesn't already.
> fear of becoming mere batteries
I really wish that the Matrix had stuck with its original concept [0] that the people were being used for their processing power, not as batteries.
[0] Or so the rumour goes.
I really wish that the Matrix had stuck with its original concept [0] that the people were being used for their processing power, not as batteries.
[0] Or so the rumour goes.
If that was the original concept, I wish they did.
I mean, it's such an obvious one. I must have independently thought of it a dozen times as a teenager. Hell, the movie itself sort of makes use of it - after all, AFAIR, the Matrix is simulated in peoples' heads. Why would they go with this human battery nonsense is beyond me. I guess someone with influence over the movie script must have though people are too dumb for the plot to make sense.
I mean, it's such an obvious one. I must have independently thought of it a dozen times as a teenager. Hell, the movie itself sort of makes use of it - after all, AFAIR, the Matrix is simulated in peoples' heads. Why would they go with this human battery nonsense is beyond me. I guess someone with influence over the movie script must have though people are too dumb for the plot to make sense.
It’s not important to the plot at all and they probably didn’t want to get into the task of explaining the processing stuff to the dumber half of the audience. Everyone in the 90’s was familiar with disposable AA batteries, but not everyone had used a computer much.
I asked the same thing several years ago and got an answer [0]: You're correct, they didn't care so much about that specific detail and let it be changed because it wasn't actually too important to the main plot.
[0] https://scifi.stackexchange.com/q/19817/2242
[0] https://scifi.stackexchange.com/q/19817/2242
Feyerabend explores the difficult process of scientific progress in the classic Against Method. He describes how society finds it difficult to develop new scientific theories to better explain new evidence because of our confirmation bias to conventional theory.
For example the earth was once understood to be at the center of the solar system, when telescopes could better observe planetary behavior it first lead simply to more precise epicycles.
In the book he points out that one way we tend to break through this hold on how we frame evidence, is by considering metaphors from the Arts. For example, religious writings helped inform the works of Copernicus and Newton.
So to generally answer your question, I think some insights will come to us from the Arts, like science fiction books/movies/games. For example, the show Black Mirror provides some ideas that we may at some point take for granted.
For example the earth was once understood to be at the center of the solar system, when telescopes could better observe planetary behavior it first lead simply to more precise epicycles.
In the book he points out that one way we tend to break through this hold on how we frame evidence, is by considering metaphors from the Arts. For example, religious writings helped inform the works of Copernicus and Newton.
So to generally answer your question, I think some insights will come to us from the Arts, like science fiction books/movies/games. For example, the show Black Mirror provides some ideas that we may at some point take for granted.
Near the end of your proposed course, I would heartily recommend including https://www.theatlantic.com/magazine/archive/1945/07/as-we-m.... It was the 1945 paper that introduced the ideas behind both hypertext and the science citation index. The two ideas famously got recombined some decades later by Google's PageRank algorithm.
Now reading that article, one might wonder how someone could have such a solid grasp of how computing and people could work together in practice when the transistor had not yet been invented, and stored program computers were first described only a month earlier.
The reason is that Vanevar Bush had been building and working with computers for close to 20 years!
Now reading that article, one might wonder how someone could have such a solid grasp of how computing and people could work together in practice when the transistor had not yet been invented, and stored program computers were first described only a month earlier.
The reason is that Vanevar Bush had been building and working with computers for close to 20 years!
> After all, Babbage's machine could have been built in his time if funding hadn't run out.
Funding ran out in large part because of cost overruns due to the fact that the technology of the time wasn’t capable of building the analytic engine design.
Funding ran out in large part because of cost overruns due to the fact that the technology of the time wasn’t capable of building the analytic engine design.
Had anyone in the 19th century realized the generality of computing in binary there would have been successful mechanical computers of the age. Doing everything in decimal was part of the problem.
Charles S. Peirce in the 1880s understood that switching circuits embody Boolean logic, though he never published except in a letter to a student/colleague who was going to work on this but abandoned it. That Babbage and Peirce both had trouble dealing with people may have had a pretty big effect on history.
That's the trick of "autistic" personalities -- better with computers but worse with people. A person better with people but worse with computers wouldn't have succeeded either. Being better at both is rare.
According to the London museum of science, Babbage looked at binary, as well as many other bases and made an informed decision against it. The supremacy of binary is not as obvious with mechanical systems as it is with electrical semiconductors. These machines are built with gears, not switches.
This was my understanding as well. The tight machining tolerances required simply didn't exist in his time.
Given resources, they could potentially have made a machine to machine.
The ability was potentially there, but the need perhaps was not.
It would have required a generation dedicated to that one task.
The ability was potentially there, but the need perhaps was not.
It would have required a generation dedicated to that one task.
It was likely possible with a lot of effort and extensive hand-fitting, just not economically at scale.
That's possible, but an alternative explanation for the cost overruns that I've read is that Babbage had terrible project management skills.
Wikipedia has this to say:
In 1991, the London Science Museum built a complete and working specimen of Babbage's Difference Engine No. 2, a design that incorporated refinements Babbage discovered during the development of the Analytical Engine. This machine was built using materials and engineering tolerances that would have been available to Babbage, quelling the suggestion that Babbage's designs could not have been produced using the manufacturing technology of his time.
https://en.wikipedia.org/wiki/Analytical_Engine
Wikipedia has this to say:
In 1991, the London Science Museum built a complete and working specimen of Babbage's Difference Engine No. 2, a design that incorporated refinements Babbage discovered during the development of the Analytical Engine. This machine was built using materials and engineering tolerances that would have been available to Babbage, quelling the suggestion that Babbage's designs could not have been produced using the manufacturing technology of his time.
https://en.wikipedia.org/wiki/Analytical_Engine
I just finished reading The Difference Engine yesterday, an entire book about this project, by the project lead!
https://www.amazon.com/Difference-Engine-Charles-Babbage-Com...
(unfortunately overshadowed in Google Search results by a William Gibson book of the same name)
It gives a lot of color on Babbage, but yes the conclusion was that Babbage design basically worked, and could have been built. There were errors in his drawings that they had to correct, but nothing fundamental.
The group at the Science Museum spent over 6 years doing this! This is the group that holds most of his papers, drafts, and unfinished machines.
Although there are a couple things I want to follow up on. They weren't that specific about what computation they did. And does it still work today? It was extraordinarily finicky. It produced a lot of bit errors, as did mechanical computing devices that came later, which sort of defeated the purpose (it was supposed to calculate tables of logarithms and such with higher accuracy than humans.)
https://www.amazon.com/Difference-Engine-Charles-Babbage-Com...
(unfortunately overshadowed in Google Search results by a William Gibson book of the same name)
It gives a lot of color on Babbage, but yes the conclusion was that Babbage design basically worked, and could have been built. There were errors in his drawings that they had to correct, but nothing fundamental.
The group at the Science Museum spent over 6 years doing this! This is the group that holds most of his papers, drafts, and unfinished machines.
Although there are a couple things I want to follow up on. They weren't that specific about what computation they did. And does it still work today? It was extraordinarily finicky. It produced a lot of bit errors, as did mechanical computing devices that came later, which sort of defeated the purpose (it was supposed to calculate tables of logarithms and such with higher accuracy than humans.)
The examples built do indeed keep on working with non-prohibitive maintenance - the 2nd #2-design engine (built in the 2000s for Nathan Myhrvold), was on display at the CHM in Mountain View for 8 years with daily or twice-daily demonstration runs. It sadly went off display in 2016 (probably to go to Myhrvold's private collection) but I saw the demonstration a couple of times and can answer some of your questions:
1. The concrete computation performed was to use the Finite Difference Method (https://en.wikipedia.org/wiki/Finite_difference_method - hence the "Difference Engine" name) to calculate arbitrary polynomials of degree up to IIRC 10. By using Taylor Series, this method could be used to calculate arbitrary functions, like log and sine. This was in fact the same method used to construct logarithmic tables by hand at the time, and had similar nominal precision; the singular goal was to eliminate the bit errors rampant in the old, manual process.
2. The machine removed not just errors in calculation, but also in typesetting; about half the part-count of the original design was in its printer, which could be configured with all kinds of options for typesetting the results. It would output a "print preview" onto paper locally (this was not publicly demonstrated at the CHM because of the enormous mess of ink spills, but the machinery was run dry), and an identical wax mold ready for use in mass printing. This was because many of the bit errors in the existing log/sine/etc. tables were introduced not by the (human) computers, but by the multiple copying steps involved in transforming calculated values into printed pages.
3. Computation was quite reliable - the machine worked in base 10, and mechanisms were carefully designed to freeze up (and be easily resettable to a known-good state, as demonstrations showed) before introducing errors. As far as I know bit errors were unheard of in the demonstration runs. This reliability, like in later electronic computation, was the motivation for using digital rather than analog logic. (Finickiness was mostly limited to those halting conditions - it proved quite sensitive to clock speed (rate of crank turn), but only by the standards of the hand cranking used in demonstrations; connected up to a steam engine with 19th-century rate governors, input power could have been kept clean enough to run with long MTTF.)
1. The concrete computation performed was to use the Finite Difference Method (https://en.wikipedia.org/wiki/Finite_difference_method - hence the "Difference Engine" name) to calculate arbitrary polynomials of degree up to IIRC 10. By using Taylor Series, this method could be used to calculate arbitrary functions, like log and sine. This was in fact the same method used to construct logarithmic tables by hand at the time, and had similar nominal precision; the singular goal was to eliminate the bit errors rampant in the old, manual process.
2. The machine removed not just errors in calculation, but also in typesetting; about half the part-count of the original design was in its printer, which could be configured with all kinds of options for typesetting the results. It would output a "print preview" onto paper locally (this was not publicly demonstrated at the CHM because of the enormous mess of ink spills, but the machinery was run dry), and an identical wax mold ready for use in mass printing. This was because many of the bit errors in the existing log/sine/etc. tables were introduced not by the (human) computers, but by the multiple copying steps involved in transforming calculated values into printed pages.
3. Computation was quite reliable - the machine worked in base 10, and mechanisms were carefully designed to freeze up (and be easily resettable to a known-good state, as demonstrations showed) before introducing errors. As far as I know bit errors were unheard of in the demonstration runs. This reliability, like in later electronic computation, was the motivation for using digital rather than analog logic. (Finickiness was mostly limited to those halting conditions - it proved quite sensitive to clock speed (rate of crank turn), but only by the standards of the hand cranking used in demonstrations; connected up to a steam engine with 19th-century rate governors, input power could have been kept clean enough to run with long MTTF.)
Thanks for those details. I am glad to have seen the demonstration. I didn't know they removed it in 2016.
I believe the better link is https://en.wikipedia.org/wiki/Divided_differences
I believe the better link is https://en.wikipedia.org/wiki/Divided_differences
They require tolerances that could be achieved in the day, but not reliably was certainly not standard practice. When the funding was cut they were basically running a research program to develop better machining. With the research program failed to bad project management is kind of irrelevant. My point was it was still in the area of being a research program.
I think an important difference between Babagge’s attempt and that of the London Science Museum is that, for the latter, it was much more economical to produce the parts to the required tolerances.
The difference between “takes an expert engineer a month to create” and “can be CNC’d in a day” is enormous, even if the end results are the same.
The difference between “takes an expert engineer a month to create” and “can be CNC’d in a day” is enormous, even if the end results are the same.
They still haven’t built the Analytical Engine, which was the general-purpose computer and significantly more complex.
Also, building something with the benefit of 150 years of technological progress gives you quite a leg up on the past even if you use techniques that would have technically been available back then.
Also, building something with the benefit of 150 years of technological progress gives you quite a leg up on the past even if you use techniques that would have technically been available back then.
Probably if Babbage had employed watchmakers, he could have got it done, but clockwork run on springs isn't as impressive as massive, clanking machinery driven by steam engines.
He would still have had the problem of driving typesetting equipment, but that seems like a smaller problem. Output on punched paper tape was the solution the later generation chose, and would have worked.
Amusingly, the original telegraph inventions (plural) output dots on paper, but accidents of finance left us with the inferior audio clickery. Sort of like how we are still using x86.
He would still have had the problem of driving typesetting equipment, but that seems like a smaller problem. Output on punched paper tape was the solution the later generation chose, and would have worked.
Amusingly, the original telegraph inventions (plural) output dots on paper, but accidents of finance left us with the inferior audio clickery. Sort of like how we are still using x86.
[deleted]
If you haven't seen it, physicists used physical lenses to compute fourier transforms before computers.
Also, although I didn't read closely enough, I think "collation" or "partitioning" is the right term for sorting things and then placing them into buckets. Sorting is just ordering, not partitioning.
Also, although I didn't read closely enough, I think "collation" or "partitioning" is the right term for sorting things and then placing them into buckets. Sorting is just ordering, not partitioning.
The question is why you believe that "sorting" is "ordering"?
Dictionaries show that "sorting" meand "ordering" in computing, but means "classifying" or "organizing" in all other contexts.
Putting documents in order is "filing", from the French word meaning "string", as in attaching things in a string to put them in order in a line.
As an example, pre-computer card catalogs are "filed", not "sorted": https://libguides.ala.org/filing-rules
Dictionaries show that "sorting" meand "ordering" in computing, but means "classifying" or "organizing" in all other contexts.
Putting documents in order is "filing", from the French word meaning "string", as in attaching things in a string to put them in order in a line.
As an example, pre-computer card catalogs are "filed", not "sorted": https://libguides.ala.org/filing-rules
Because the dictionary is wrong.
Dictionaries are not authoritative sources for word definitions. Rather they are, and can only be, historical references.
Words mean what people use them to mean.
“Sort these pages” with no other instructions will almost always mean, to the average person in a general context, put the pages in ascending order using the most obvious sequence present.
“File these documents” will generally mean put these pages in their relevant folders, either in a filing cabinet or computer storage.
Well, that’s their common usage outside of computer specific applicant ions here in Australia.
YMMV
Dictionaries are not authoritative sources for word definitions. Rather they are, and can only be, historical references.
Words mean what people use them to mean.
“Sort these pages” with no other instructions will almost always mean, to the average person in a general context, put the pages in ascending order using the most obvious sequence present.
“File these documents” will generally mean put these pages in their relevant folders, either in a filing cabinet or computer storage.
Well, that’s their common usage outside of computer specific applicant ions here in Australia.
YMMV
At this point my mind is blown.
English is my second language, and to this moment, I've always believed "filing" is just "putting somewhere in the right cabinet".
English is my second language, and to this moment, I've always believed "filing" is just "putting somewhere in the right cabinet".
Both are incorrect. "Filing" is bucketing according to an external value.
For example, when a doctor files some test results, the test results aren't being put in order with other test results. Those papers are first bucketed by patient, and the buckets (files) are what's kept sorted.
For example, when a doctor files some test results, the test results aren't being put in order with other test results. Those papers are first bucketed by patient, and the buckets (files) are what's kept sorted.
There were also things like Michelson's harmonic analyzer:
https://www.asme.org/engineering-topics/articles/technology-...
https://www.asme.org/engineering-topics/articles/technology-...
This leads me to wonder: what abstractions are we missing today that will be obvious to future generations?
Assuming that at some point we will stop writing code and machines will generate programs for us instead, then we need abstractions that make this possible: a way to describe the purpose of a program that is precise enough to generate code satisfying the purpose and that can be derived from normal human communication instead of having to be explicitly coded. In other words: a way of describing why the program exists, instead of describing what it should do.
Additionally, to avoid paperclip maximizers we would need a system of ethics and values robust enough that it becomes computable. Then it can be integrated into the code-writing code in such a way as to constrain the code it generates to only choose approaches that are in line with the value system, and to refuse to implement purposes at odds with the value system. As long as our best stab at ethics largely boils down to long lists of behavioral rules written in holy books with few underlying principles we aren't ready to build truly advanced AI.
Probably some philosopher has already figured out the right model of abstract ethics, but has gone mostly undiscovered so far.
Assuming that at some point we will stop writing code and machines will generate programs for us instead, then we need abstractions that make this possible: a way to describe the purpose of a program that is precise enough to generate code satisfying the purpose and that can be derived from normal human communication instead of having to be explicitly coded. In other words: a way of describing why the program exists, instead of describing what it should do.
Additionally, to avoid paperclip maximizers we would need a system of ethics and values robust enough that it becomes computable. Then it can be integrated into the code-writing code in such a way as to constrain the code it generates to only choose approaches that are in line with the value system, and to refuse to implement purposes at odds with the value system. As long as our best stab at ethics largely boils down to long lists of behavioral rules written in holy books with few underlying principles we aren't ready to build truly advanced AI.
Probably some philosopher has already figured out the right model of abstract ethics, but has gone mostly undiscovered so far.
> Assuming that at some point we will stop writing code and machines will generate programs for us instead, then we need abstractions that make this possible: a way to describe the purpose of a program that is precise enough to generate code satisfying the purpose
The industry-standard term for a specification precise enough to enable building a program from it is "code"
The industry-standard term for a specification precise enough to enable building a program from it is "code"
>> This leads me to wonder: what abstractions are we missing today that will be obvious to future generations?
That's a very nice question. But I tend to see it like this : if you'd know, it'd mean that somehow you predicted the future.
I'd say that abstractions are built everytime but only the good ones survive. So the only way to answer your question is to wait 50 years. Then we'll know what are those missing abstractions.
But the simple act of asking the question already opens the way to a critic of current abstractions, which will lead to new ones and maybe, one of the future :-)
Are there public places where such things are discussed ? I mean, places where you don't need to be inside a university to actually participate.
That's a very nice question. But I tend to see it like this : if you'd know, it'd mean that somehow you predicted the future.
I'd say that abstractions are built everytime but only the good ones survive. So the only way to answer your question is to wait 50 years. Then we'll know what are those missing abstractions.
But the simple act of asking the question already opens the way to a critic of current abstractions, which will lead to new ones and maybe, one of the future :-)
Are there public places where such things are discussed ? I mean, places where you don't need to be inside a university to actually participate.
> what abstractions are we missing today that will be obvious to future generations?
And how many people have already recognized them, only to have been ignored because they're working at too high a level for their listeners to appreciate?
And how many people have already recognized them, only to have been ignored because they're working at too high a level for their listeners to appreciate?
[deleted]
Given the great perspective you put here, Ada was truly amazing.
I have a bit more appreciation for her insight at such an early time.
How frustrating! She could see it, but too early.
A bit like maybe Tesla, others in times just before things really opened up.
I have a bit more appreciation for her insight at such an early time.
How frustrating! She could see it, but too early.
A bit like maybe Tesla, others in times just before things really opened up.
> Babbage's machine could have been built in his time if funding hadn't run out.
And if it hadn't, so what? The world would not have changed, we'd just have an absurdly impactical computer which no one had a use for. Had it been useful enough to keep making them and improving them, we'd have developed the abstractions.
And if it hadn't, so what? The world would not have changed, we'd just have an absurdly impactical computer which no one had a use for. Had it been useful enough to keep making them and improving them, we'd have developed the abstractions.
I don’t follow.
When you have to build a device pretty much by hand, why bother building a general purpose machine?
And here we are, 181 years after Babbage first described his Analytic Engine and Aplication Specific Integrated Circuits and Graphics Processing Units are all the rage.
When you have to build a device pretty much by hand, why bother building a general purpose machine?
And here we are, 181 years after Babbage first described his Analytic Engine and Aplication Specific Integrated Circuits and Graphics Processing Units are all the rage.
> When you have to build a device pretty much by hand, why bother building a general purpose machine?
Because you'd conquer many industries with it simultaneously. Instant path to richness.
> And here we are, 181 years after Babbage first described his Analytic Engine and Aplication Specific Integrated Circuits and Graphics Processing Units are all the rage.
It's a different thing. We usually first figure a description of a computation - one that could be run on a general-purpose machine - and then turn it into ASIC to make it more efficient, by performing only the work necessary for that one particular calculation. Back in Babbage's times, they didn't have a coherent notion of computation yet.
Because you'd conquer many industries with it simultaneously. Instant path to richness.
> And here we are, 181 years after Babbage first described his Analytic Engine and Aplication Specific Integrated Circuits and Graphics Processing Units are all the rage.
It's a different thing. We usually first figure a description of a computation - one that could be run on a general-purpose machine - and then turn it into ASIC to make it more efficient, by performing only the work necessary for that one particular calculation. Back in Babbage's times, they didn't have a coherent notion of computation yet.
> Because you'd conquer many industries with it simultaneously. Instant path to richness.
I just mean to say, from my limited knowledge, the available applications were all (mostly?) bespoke anyways, so the inputs and outputs probably needed to be also. So perhaps a modular mechanical general purpose computer may have helped.
> It's a different thing.
Yes, you've made a good point, I agree. Perhaps, then, if you were to build a general purpose mechanical computer ~180 years ago you might use it for whatever application is required, observe it's most commonly used computational paths, then swap it out for an application specific computer. of (possibly?) reduced complexity. You can then take your whizz-bang-high-tech-expensive computer to the next client, rinse and repeat. Maybe.
I just mean to say, from my limited knowledge, the available applications were all (mostly?) bespoke anyways, so the inputs and outputs probably needed to be also. So perhaps a modular mechanical general purpose computer may have helped.
> It's a different thing.
Yes, you've made a good point, I agree. Perhaps, then, if you were to build a general purpose mechanical computer ~180 years ago you might use it for whatever application is required, observe it's most commonly used computational paths, then swap it out for an application specific computer. of (possibly?) reduced complexity. You can then take your whizz-bang-high-tech-expensive computer to the next client, rinse and repeat. Maybe.
> So perhaps a modular mechanical general purpose computer may have helped.
That's what I was imagining when writing that comment. And I suppose experience turning computation into work in one industry would let you design bespoke endpoints for another faster.
That's what I was imagining when writing that comment. And I suppose experience turning computation into work in one industry would let you design bespoke endpoints for another faster.
We are missing thought to action devices. We have voice action devices.
[deleted]
Maybe it's just because I've spent too long with computers, but I thought the everyday meaning of "sorting" was ordering, not categorizing.
(Though now that I think about it, it's both: if I handed a random non-coder a deck of cards and asked them to sort them, I'd expect them to group by suit and then order by value. I wouldn't expect four stacks by suit in random order, or a deck of all the cards in numerical order regardless of suit...)
That nitpick aside, it was a good excuse to get into the interesting pre-digital computing history stuff.
(Though now that I think about it, it's both: if I handed a random non-coder a deck of cards and asked them to sort them, I'd expect them to group by suit and then order by value. I wouldn't expect four stacks by suit in random order, or a deck of all the cards in numerical order regardless of suit...)
That nitpick aside, it was a good excuse to get into the interesting pre-digital computing history stuff.
I had the same initial reaction, but the dictionary supports the idea that the everyday usage of sort means to group by category, not to order. I've probably just been coding too long as well.
You’re right, it does! I wonder if the definition is going to drift towards “ordering” as computing becomes ever more ubiquitous, or if we’re just in a linguistic cul de sac by virtue of our profession...
Are you referring to a dictionary of word definitions? The kind that is structured as a list of words, ordered lexicographically? Where is the categorization in that?
He's referring to the dictionary definition of the word "sort".
Dictionary.com has it as "to arrange according to sort, kind, or class; separate into sorts; classify". I looked it up in the OED too -- mostly because we have one and I don't get enough excuses to use its nifty little magnifying glass -- and it also supports the notion that it means by category, not by order. The general verb definition is "to arrange things according to kind or quality", and there's a separate definition for software: "sort: Computers: the action of arranging items of data in a prescribed sequence".
Dictionary.com has it as "to arrange according to sort, kind, or class; separate into sorts; classify". I looked it up in the OED too -- mostly because we have one and I don't get enough excuses to use its nifty little magnifying glass -- and it also supports the notion that it means by category, not by order. The general verb definition is "to arrange things according to kind or quality", and there's a separate definition for software: "sort: Computers: the action of arranging items of data in a prescribed sequence".
What do you mean? I agree that dictionaries are ordered and not categorized, but do dictionaries claim to be sorted?
I thought parent was using "the sortedness of dictionaries" as his argument, I now realize he meant "the dictionary definition of 'sort'".
I definitely think it's reasonable that someone might say e.g. "sort these coats by color" and expect them to be bucketed like-with-like, but not imply any kind of ordering (either within or across buckets).
What if the coats all had different but similar colors? The process would be the same, just that you have to split them into arbitrary "buckets" at the end.
There are phrases using word sort as a category: "this sort of people" , "sort of liked".
> but I thought the everyday meaning of "sorting" was ordering, not categorizing
I think I've always thought of ordering as a subset of sorting, not something different, both in computing and in "real life" language use.
When sorting washed laundry I categorise by type and order by a fairly arbitrary mix of target position (undies in the top left draw, socks top right, daily t-shirts the draw below, "better" t-shirts the draw below that, then trousers, ...), hue/pattern/subject, and a few other properties. As with your cards example there are both grouping and ordering operations present in the sorting operation (in the cards example there might be two orderings: one for the suits if you want them to be in a specific order and one by the value of the cards within the suit).
Sorting numbers in a computer seem quite different: each category has at most one item and the are ordered by the same property they are categorised by, but the process is similar enough.
And sorting in computing is not always simply ordering. There are many examples where it is categorising (just by some variety of ordinal value). Some sorting algorithms explicitly categorise the input data as a divide-and-conquer process. Sometimes you don't care about the precise ordering (but it is done precisely anyway because a fuzzy sort would be no more performant).
The only examples that immediately jumps to mind where there is a definite distinction made are data management languages where there is a distinction between ordering and grouping (quite explicitly seen in SQL's "order by" and "group by" clauses), and even there I know that internally ordering/grouping/both are performed by what I would identify as a sort operation.
Sorting doesn't mean something different in computing really, people just tend to pick out a more specific definition in that context.
I think I've always thought of ordering as a subset of sorting, not something different, both in computing and in "real life" language use.
When sorting washed laundry I categorise by type and order by a fairly arbitrary mix of target position (undies in the top left draw, socks top right, daily t-shirts the draw below, "better" t-shirts the draw below that, then trousers, ...), hue/pattern/subject, and a few other properties. As with your cards example there are both grouping and ordering operations present in the sorting operation (in the cards example there might be two orderings: one for the suits if you want them to be in a specific order and one by the value of the cards within the suit).
Sorting numbers in a computer seem quite different: each category has at most one item and the are ordered by the same property they are categorised by, but the process is similar enough.
And sorting in computing is not always simply ordering. There are many examples where it is categorising (just by some variety of ordinal value). Some sorting algorithms explicitly categorise the input data as a divide-and-conquer process. Sometimes you don't care about the precise ordering (but it is done precisely anyway because a fuzzy sort would be no more performant).
The only examples that immediately jumps to mind where there is a definite distinction made are data management languages where there is a distinction between ordering and grouping (quite explicitly seen in SQL's "order by" and "group by" clauses), and even there I know that internally ordering/grouping/both are performed by what I would identify as a sort operation.
Sorting doesn't mean something different in computing really, people just tend to pick out a more specific definition in that context.
The example that comes to my mind is a pile of wooden blocks. If I was asked to sort them I sort them by shape (i.e., by category). Here is a pile of the square ones, here are the circle ones, etc.
If I was sorting lego blocks I don't know if I would do it first by color and then by size/shape (here are all the 2x8s, here are all the 1x4s, etc.), but it would definitely be by category.
If I was sorting lego blocks I don't know if I would do it first by color and then by size/shape (here are all the 2x8s, here are all the 1x4s, etc.), but it would definitely be by category.
lego is funny, if you have a pile of red 'stuff' it's hard to pick out the 2x8. however, if you have a pile of 2x8s, it's really easy to pick out the red one.
size/shape is far more useful.
size/shape is far more useful.
This is a well known idea in data visualisation -- first articulated by Jaques Bertin back in the 60s (I think). Colour is known as a selective visual variable; selective variables are better used to represent categorical data as they "enable us to immediately isolate all correspondences belonging to the same category". i.e. sort your lego by bricks type rather than colour, though sorting by colour will lead to more instagram likes :)
Yes, color looks nicer, but shape is more useful.
> but I thought the everyday meaning of "sorting" was ordering, not categorizing
Go ask your wife how she sorts the laundry.
Go ask your wife how she sorts the laundry.
The meanings are the same. In one case you're sorting a set of items which are all unique, whereas in the other there are some duplicates under the sorting key. If you sort by just the suit of the cards, you have clusters that are of the same suit. It's only in the very-precise world of programming that a set versus a set of sets makes much of a difference.
no, its more like a set of sets vs an ordered set of sets
[deleted]
I think this is a valid critique of the title. Sorting means whatever it means in a specific context (dewey decimal system, post addresses, etc...).
Most languages are context sensitives, but generalized computing is fundamentally defined by a context free language in which a processing transition can execute and not care was previous states were. This gets blurry with recursion and generics and other syntactic sugar, but fundamentally state changes are still defined by the operation and instantaneous state and not the language itself.
So in our context-free language of generalized computers, sorting and ordering are the same.
Most languages are context sensitives, but generalized computing is fundamentally defined by a context free language in which a processing transition can execute and not care was previous states were. This gets blurry with recursion and generics and other syntactic sugar, but fundamentally state changes are still defined by the operation and instantaneous state and not the language itself.
So in our context-free language of generalized computers, sorting and ordering are the same.
I can’t really tell what you mean but I suspect that it doesn’t make any sense.
What on Earth do you mean when you say “generalized computing is fundamentally defined by a context free language in which a processing transition can execute and not care was previous states were”?
Perhaps you are just writing context free/sensitive when you mean something different from “the set of words that may be formed by starting with some nonterminal symbol S, following rules of the form A -> xy where A is any nonterminal and x and y are either terminal or nonterminal symbols, repeating until there are no nonterminals left.”
If you really mean something about all computations being expressible in a context free language then I suppose I would have to agree but I put it to you that a regular language would be sufficient.
The other thing I can’t understand is what this has to do with either the parent or the article?
What on Earth do you mean when you say “generalized computing is fundamentally defined by a context free language in which a processing transition can execute and not care was previous states were”?
Perhaps you are just writing context free/sensitive when you mean something different from “the set of words that may be formed by starting with some nonterminal symbol S, following rules of the form A -> xy where A is any nonterminal and x and y are either terminal or nonterminal symbols, repeating until there are no nonterminals left.”
If you really mean something about all computations being expressible in a context free language then I suppose I would have to agree but I put it to you that a regular language would be sufficient.
The other thing I can’t understand is what this has to do with either the parent or the article?
These "twitter stories" are horrible to read.
Someone else posted a much more readable version, but it is buried under a downvoted post:
https://threadreaderapp.com/thread/1036704758696960001.html
Someone else posted a much more readable version, but it is buried under a downvoted post:
https://threadreaderapp.com/thread/1036704758696960001.html
This is quite interesting. From my German background I associate "sortieren" with alphabetic/numerical /calendaring order. Whereas "ordnen" ist more like "putting in order"/"categorize" or "sorting". So maybe it's as easy as blaming the Germans ;-)
Except that -ieren ending on sortieren is indicative of an import from Romantic languages, typically French or Italian. [0] So it's likely that the English and German word both share French sortir as an imported root. Etymonline lists it as 14th century English, so too late to have inherited it through German. [1]
[0] https://en.wiktionary.org/wiki/-ieren#German
[1] https://www.etymonline.com/word/sort#etymonline_v_24299
[0] https://en.wiktionary.org/wiki/-ieren#German
[1] https://www.etymonline.com/word/sort#etymonline_v_24299
> Except that -ieren ending on sortieren is indicative of an import from Romantic languages, typically French or Italian.
Except that it is Romance or Romanic languages, not Romantic languages.
Except that it is Romance or Romanic languages, not Romantic languages.
although french and italian are seen as romantic by some :-)
I think that’s a false cognate: in French, sortir means to leave or go/take out. Trier is means to sort or classify.
The meaning you listed seems to correlate across many Romance languages and caused me to start looking deeper. I found an Anglo-Norman dictionary [0], which lists meanings very much in line with those listed at Etymonline [1]. I can't explain the difference... The listed Latin root seems to allow the noted Anglo-Norman and later English meaning [2]. But modern Romance languages seem to have standardized a meaning similar to what you mention.
There does seem to be widely-attested noun forms like French sorte, Italian sorta with a meaning along the lines of "type" or "kind".
[0] http://www.anglo-norman.net/gate/
[1] http://www.anglo-norman.net/D/sortir
[2] https://en.wiktionary.org/wiki/sortior#Latin
There does seem to be widely-attested noun forms like French sorte, Italian sorta with a meaning along the lines of "type" or "kind".
[0] http://www.anglo-norman.net/gate/
[1] http://www.anglo-norman.net/D/sortir
[2] https://en.wiktionary.org/wiki/sortior#Latin
yes, but german also has the term "Sorte" which is used to categorize the variations of fruits and other stuff. the macintosh apple is one sort and the boskop apple is another. you can have different sorts of flour, etc. so the meaning of "sortieren" has changed, but it's definetly related to categorizing.
greetings, eMBee
greetings, eMBee
Sorting is categorizing, just with an order imposed on the categories.
If the categories have a natural order, then people will exploit that when they sort.
Suppose you had to put some cards with names on them into bins labeled by first letter. You might go through the cards one by one and stick each in the appropriate bin. It would help you if the bins are in alphabetic order, rather than scrambled.
Sorting into categories involves a search to find the category for an item; the search benefits from order.
If the categories have a natural order, then people will exploit that when they sort.
Suppose you had to put some cards with names on them into bins labeled by first letter. You might go through the cards one by one and stick each in the appropriate bin. It would help you if the bins are in alphabetic order, rather than scrambled.
Sorting into categories involves a search to find the category for an item; the search benefits from order.
> Sorting is categorizing, just with an order imposed on the categories.
…and infinite categories.
…and infinite categories.
Infinite, or recursive? Alphabetic, lexographic, and numeric sorts are recursive over "significant digits", which is why we can write radix sorts. (Scare quotes because that term doesn't necessarily directly translate to alphabetic and lexographic sorting, but the basic prinicple is the same.)
"recursively" is but one way to build a potentially infinite set. They're not mutually exclusive.
I actually asked a non-math/cs person the question, "if I gave you a shuffled deck of cards and asked you to sort them, what would you give me?" Her response was, I'd give you the deck order them by suit (spade, club, heart, diamond), then by number (with j,q,k after 9).
Which was what I expected. The twitter post seems to suggest that his expectation for the given question would be four decks, one for each suit (i.e., categorized).
Which was what I expected. The twitter post seems to suggest that his expectation for the given question would be four decks, one for each suit (i.e., categorized).
Soundex, the word similarity hashing algorithm, also predates computers. It was developed in early 1900, to help perform census in USA. One of its goals was to sort similar last names together.
https://en.wikipedia.org/wiki/Soundex
https://en.wikipedia.org/wiki/Soundex
Knuth discusses this in Vol 3 Sorting and Searching ... we mean "collate" by sorting. Unfortunately I don't have that volume at hand to quote. Anybody?
> 5.2.4 Sorting by Merging
> Merging (or collating) means the combination of two or more ordered files into a single ordered file.
So it's not quite the same idea.
> Merging (or collating) means the combination of two or more ordered files into a single ordered file.
So it's not quite the same idea.
I dunno... phonebooks we're sorted alphabetically before they were created with computers. That's a single sorted list and not a set of categories. The card sorting is interesting but it seems like the word is simply used in multiple ways.
If you decorate each item with a category tag and then sort using those tags, you are categorizing. The power of decorate-sort-undecorate.
This is great stuff, too bad it's on Twitter instead of a real website or platform.
That threadreadapp pollutes the thread with pointless replies
Regardless of if you like twitter or their practices they are a real website and platform.
It's pretty clear from context that by "a real platform" OP means "a platform suited to this sort of work" (long-form writing).
I agree.
I hate to post anything longer than random thoughts on Twitter, but this is an unfortunate consequence of the culture shift in past 10 years driven by the social media giants. Like it or not, Twitter is now the center of gravity of the entire Internet, and currently the most effective online platform to make posts with high visibility is Twitter...
Other platforms like forums declined, even forums did nothing wrong by themselves, and they are in fact much more suitable for real discussion. But the culture is already changed, people got used to make random 20 words posts on their mobile phones instead of reading a long threads of discussion, microblog-like, social-media-like platforms would prevail in the shortterm future. The only thing we can do is to promote a OStatus-based, open, alternative social media implemented by GNU/Social, Mastodon, etc, to make the situation less harmful.
I'm happy that we can still have real discussion on Hacker News.
I hate to post anything longer than random thoughts on Twitter, but this is an unfortunate consequence of the culture shift in past 10 years driven by the social media giants. Like it or not, Twitter is now the center of gravity of the entire Internet, and currently the most effective online platform to make posts with high visibility is Twitter...
Other platforms like forums declined, even forums did nothing wrong by themselves, and they are in fact much more suitable for real discussion. But the culture is already changed, people got used to make random 20 words posts on their mobile phones instead of reading a long threads of discussion, microblog-like, social-media-like platforms would prevail in the shortterm future. The only thing we can do is to promote a OStatus-based, open, alternative social media implemented by GNU/Social, Mastodon, etc, to make the situation less harmful.
I'm happy that we can still have real discussion on Hacker News.
As of the end of 2016, 24% of active internet users in America use Twitter. According to Twitter, they have 335 million active users, which is declining - thanks largely to declines in America [2]. One can only speculate on the percent of those that are bots. By contrast Facebook still has north of 2billion monthly active users. Both sites, on top of bots, of course also have an incentive to count as loosely as possible within the realm of plausible deniability.
Ultimately, if Twitter is the center of gravity of the internet, then Pluto must be the center of gravity of our solar system. The only reason I mention this is because I find people make the sort of arguments like you're making to convince themselves that they're not just wasting time on these platforms. I don't think there's been any shift in culture. There probably has been a shift in demographics as the internet has become much more common among less technically oriented individuals, but I think the real player is simply that these platforms are designed to be addictive, and they are. People keep using them even when they want to quit or do not find the time spent on them beneficial. Dopamine's a hell of a drug.
[1] - http://www.pewinternet.org/2016/11/11/social-media-update-20...
[2] - https://venturebeat.com/2018/07/27/twitter-loses-1-million-m...
Ultimately, if Twitter is the center of gravity of the internet, then Pluto must be the center of gravity of our solar system. The only reason I mention this is because I find people make the sort of arguments like you're making to convince themselves that they're not just wasting time on these platforms. I don't think there's been any shift in culture. There probably has been a shift in demographics as the internet has become much more common among less technically oriented individuals, but I think the real player is simply that these platforms are designed to be addictive, and they are. People keep using them even when they want to quit or do not find the time spent on them beneficial. Dopamine's a hell of a drug.
[1] - http://www.pewinternet.org/2016/11/11/social-media-update-20...
[2] - https://venturebeat.com/2018/07/27/twitter-loses-1-million-m...
Twitter is now the center of gravity of the entire Internet
Journalists and celebrities certainly seem to think so, and I'm glad they do, because I can easily avoid them by simply ignoring Twitter.
Journalists and celebrities certainly seem to think so, and I'm glad they do, because I can easily avoid them by simply ignoring Twitter.
Same here. Maybe if there were decent tools for aggregating and filtering. But there aren't.
I think there are sub-populations who think Twitter encompasses all, particularly in the pop culture and journalism crossroads.
I had a strange professional experience encountering media school academics about 5-7 years ago who were all so gung-ho about it. This was when the "firehose" could be acquired with a budget plausible within a grant-funded research group, and they seemed to imagine themselves having total information awareness like they had joined the five eyes. They spoke as if they were on the cusp of a future where this tool replaced all other forms of polling and data-gathering for social science, with no apparent concern for the incompleteness of this perspective nor of the ethical quandaries if it really were that powerful.
I had a strange professional experience encountering media school academics about 5-7 years ago who were all so gung-ho about it. This was when the "firehose" could be acquired with a budget plausible within a grant-funded research group, and they seemed to imagine themselves having total information awareness like they had joined the five eyes. They spoke as if they were on the cusp of a future where this tool replaced all other forms of polling and data-gathering for social science, with no apparent concern for the incompleteness of this perspective nor of the ethical quandaries if it really were that powerful.
[deleted]
It's a "tweetstorm".
https://www.merriam-webster.com/words-at-play/tweeting-up-a-...
https://www.merriam-webster.com/words-at-play/tweeting-up-a-...
“tweetstorm” is somewhat pejorative; here it’s just a “thread”.
Some words do have different meaning in computer science, as is the case with other fields as well.
Perhaps a more illustrative example is the word "or", in plain english it normally means an "exclusive or", but in CS it means an "inclusive or".
Perhaps a more illustrative example is the word "or", in plain english it normally means an "exclusive or", but in CS it means an "inclusive or".
Consider the following four combinations of {stable, unique}:
- stable sort of unique values is ordering
- stable sort of non-unique values is categorization + ordering; this is a most general algorithm which could reasonably be used for either operation
- non-stable sort of non-unique values is categorization without ordering
- non-stable sort of unique values technically could be implemented as a NOP.
Given that various strategies of ordering and categorizing have different trade-offs between {memory,computational} complexity, and linear vs random access patterns, it makes sense to consider them pluggable algorithms that can be swapped depending on requirements.
- stable sort of unique values is ordering
- stable sort of non-unique values is categorization + ordering; this is a most general algorithm which could reasonably be used for either operation
- non-stable sort of non-unique values is categorization without ordering
- non-stable sort of unique values technically could be implemented as a NOP.
Given that various strategies of ordering and categorizing have different trade-offs between {memory,computational} complexity, and linear vs random access patterns, it makes sense to consider them pluggable algorithms that can be swapped depending on requirements.
"That sort of person would sort the blocks into buckets" uses "sort" twice (differently) without implying some underlying metric for "people" or "blocks" and it's pretty clear in context. I saw a few people in the comments say they are surprised that "sort" would be used that way, but I'd bet money that they understood my example correctly.
"The blocks were sorted by size" implies that there is some metric, and again, it's pretty clear in context.
"The blocks were sorted by size" implies that there is some metric, and again, it's pretty clear in context.
> If the electric telegraph hadn’t been invented, would engineers have continued to optimize the optical telegraph?
Isn't that today's Free Space Optical Communications? Bandwidth is measured in Gigabits per second.
https://en.wikipedia.org/wiki/Free-space_optical_communicati...
Isn't that today's Free Space Optical Communications? Bandwidth is measured in Gigabits per second.
https://en.wikipedia.org/wiki/Free-space_optical_communicati...
I'm not sure what the difference is in sorting by category vs. order.
For example, consider an array of one million arbitrary alpha-numeric strings. If you were to sort them by category, would that mean simply making contiguous regions of the array where each string begins with the same character?
For example, consider an array of one million arbitrary alpha-numeric strings. If you were to sort them by category, would that mean simply making contiguous regions of the array where each string begins with the same character?
Assoc: Edge-notched cards - https://kk.org/thetechnium/one-dead-media/
Ordering is just a special case of categorizing.
I'd say it's arguably a degenerate case, but it's very arguable.
Few people would claim that each position in an order is a different category. And if I asked someone to categorise a pile of papers and they handed me back a single stack, I'd be unimpressed.
Few people would claim that each position in an order is a different category. And if I asked someone to categorise a pile of papers and they handed me back a single stack, I'd be unimpressed.
Ordering is categorizing organized into a vertical grid.
In MapReduce, they're the same thing.
Yawn, words mean things in context... yada yada.
That said, the details about old school punch card sorting is pretty neat.
That said, the details about old school punch card sorting is pretty neat.
That's exactly what bucket sort is. Don't they teach that in Comp Sci anymore?
Rather, the limitation was that people didn't have the abstractions, vocabulary, and mental tools to properly conceive of general purpose computers as a concept and to understand their usefulness. They couldn't see that devices as seemingly disparate as tide prediction machines[1], census tabulation machines, and loom controllers were all instances of a single, terrifyingly general idea.
From what I can tell, Babbage mostly understood this, but it was Ada Lovelace who grasped it fully. But her writings weren't understood in her time and had to be "rediscovered" a century later. For example, she wrote [2]:
Supposing, for instance, that the fundamental relations of pitched sounds in the science of harmony and of musical composition were susceptible of such expression and adaptations, the engine might compose elaborate and scientific pieces of music of any degree of complexity or extent.
This leads me to wonder: what abstractions are we missing today that will be obvious to future generations?
BTW I have a follow-up thread on the optical telegraph, a form of networking that long predates the Internet. [3] My long-term goal is to teach a course on computing/networking/information processing before computers, with a view to extracting lessons that are still applicable today.
[1] https://en.wikipedia.org/wiki/Tide-predicting_machine
[2] https://googleblog.blogspot.com/2012/12/honouring-computings...
[3] https://twitter.com/random_walker/status/1037031465735860224