Computer Power and Human Reason (1976) [pdf](web.archive.org)
web.archive.org
Computer Power and Human Reason (1976) [pdf]
https://web.archive.org/web/20230215010739if_/http://blogs.evergreen.edu/cpat/files/2013/05/Computer-Power-and-Human-Reason.pdf
16 comments
Phenomenal book with a number of pages about the hacker mindset. I’m still not done.
Excerpts: - A person who endeavors to solve problems with a computer should be distinguished from someone who does so in order to spend more time at the computer. A real scientist can go home at the end of the day and think about something important, like their family, and not be kept up by thoughts of the computer. A scientist who is ever-tweaking their code for more dopamine is not a scientist but a computer addict with a side job as a scientist.
- A computer’s memory is a perfect medium wherein any memory configuration is possible. As the programmer’s power over this domain is effectively absolute, one must acknowledge that their corruption and will to exploit this domain is also absolute.
- Technology amplifies power. It is a lever. Just because your choice was impactful doesn’t make it right.
Excerpts: - A person who endeavors to solve problems with a computer should be distinguished from someone who does so in order to spend more time at the computer. A real scientist can go home at the end of the day and think about something important, like their family, and not be kept up by thoughts of the computer. A scientist who is ever-tweaking their code for more dopamine is not a scientist but a computer addict with a side job as a scientist.
- A computer’s memory is a perfect medium wherein any memory configuration is possible. As the programmer’s power over this domain is effectively absolute, one must acknowledge that their corruption and will to exploit this domain is also absolute.
- Technology amplifies power. It is a lever. Just because your choice was impactful doesn’t make it right.
From the book, he mentions how quickly people began to anthropomorphize his program ELIZA. His secretary even asked him to leave the room to have a private conversation with it.
He then had an idea to rig the program so that he could be the one responding, but quickly dropped this idea as privacy concerns were raised by his colleagues.
People should be very cautious of giving their power over to machines that are, in the end, programmed by humans that seek power.
He then had an idea to rig the program so that he could be the one responding, but quickly dropped this idea as privacy concerns were raised by his colleagues.
People should be very cautious of giving their power over to machines that are, in the end, programmed by humans that seek power.
This is literally the point of the fictional Butlerian Jihad. Computers may appear to do one thing, but ultimately they are propelled into the future by their service to their masters.
Don’t use centralized and closed-source AI.
Don’t use centralized and closed-source AI.
That, and the part about the gamblers constructing irrational belief systems stuck with me, decades later.
This should be a required reading for everyone at OpenAI. If such simple programs can fool even intelligent people, I can't imagine what an infinitely more capable gpt4 will be able to achieve socially.
As Yuval Harari correctly points out, with the current mastery of language, AIs are starting to hack the OS on which humans operate: words and ideas.
As Yuval Harari correctly points out, with the current mastery of language, AIs are starting to hack the OS on which humans operate: words and ideas.
I'm not so sure he'd want to be called an “AI” pioneer, seeing how the mislabeling as “AI” is pure marketing hype (which is a polite way of calling it a lie) and has nothing to do with reality. The ELIZA he's so famous for is pure satire on the lie that is “AI”.
Well, the ELIZA we all know and love, was just one possible script for ELIZA. With another script, ELIZA could walk you through diagnosing an engine repair. (Sort of, with silly open-ended questions latching on to keywords you used.) It was, and still is, a substantial advance in natural language processing. Semi-flexible handling of the incredible complexity of natural English. Seems quaint today, but impressive at the time.
Giving computers the ability to handle natural language with some kind of flexibility, context-sensitivity, and appearance of understanding (even if they lack it) is one of the core goals of AI. AI was, and still is, a field with decades of serious history and work before anyone heard of Stable Diffusion. And machine learning is just one tiny subset of AI.
AI went through a similar hype bubble at the time of ELIZA, funnily enough, with natural language processing! ELIZA and SHRDLU, along with early machine translation systems, gave the impression of far more ability, especially in careful demonstrations, than they actually had. Much hype over machines that could seemingly handle natural language because they can produce coherent sentences sometimes... then disillusionment as the limits of the techniques used were quickly encountered. We may be repeating that today.
Giving computers the ability to handle natural language with some kind of flexibility, context-sensitivity, and appearance of understanding (even if they lack it) is one of the core goals of AI. AI was, and still is, a field with decades of serious history and work before anyone heard of Stable Diffusion. And machine learning is just one tiny subset of AI.
AI went through a similar hype bubble at the time of ELIZA, funnily enough, with natural language processing! ELIZA and SHRDLU, along with early machine translation systems, gave the impression of far more ability, especially in careful demonstrations, than they actually had. Much hype over machines that could seemingly handle natural language because they can produce coherent sentences sometimes... then disillusionment as the limits of the techniques used were quickly encountered. We may be repeating that today.
[deleted]
AI is real though. Or did I miss an innuendo?
Related:
Three Reviews of J. Weizenbaum’s Computer Power and Human Reason (1976) [pdf] - https://news.ycombinator.com/item?id=36752501 - July 2023 (11 comments)
Three Reviews of J. Weizenbaum’s Computer Power and Human Reason (1976) [pdf] - https://news.ycombinator.com/item?id=36370926 - June 2023 (1 comment)
The continued relevance of Weizenbaum’s warnings - https://news.ycombinator.com/item?id=34543518 - Jan 2023 (90 comments)
Weizenbaum examines computers and society (1985) - https://news.ycombinator.com/item?id=28175870 - Aug 2021 (2 comments)
Weizenbaum examines computers and society (1985) - https://news.ycombinator.com/item?id=21245979 - Oct 2019 (4 comments)
Weizenbaum examines computers and society (1985) - https://news.ycombinator.com/item?id=15859854 - Dec 2017 (1 comment)
Weizenbaum examines computers and society (1985) - https://news.ycombinator.com/item?id=8341074 - Sept 2014 (2 comments)
Three Reviews of J. Weizenbaum’s Computer Power and Human Reason (1976) [pdf] - https://news.ycombinator.com/item?id=36752501 - July 2023 (11 comments)
Three Reviews of J. Weizenbaum’s Computer Power and Human Reason (1976) [pdf] - https://news.ycombinator.com/item?id=36370926 - June 2023 (1 comment)
The continued relevance of Weizenbaum’s warnings - https://news.ycombinator.com/item?id=34543518 - Jan 2023 (90 comments)
Weizenbaum examines computers and society (1985) - https://news.ycombinator.com/item?id=28175870 - Aug 2021 (2 comments)
Weizenbaum examines computers and society (1985) - https://news.ycombinator.com/item?id=21245979 - Oct 2019 (4 comments)
Weizenbaum examines computers and society (1985) - https://news.ycombinator.com/item?id=15859854 - Dec 2017 (1 comment)
Weizenbaum examines computers and society (1985) - https://news.ycombinator.com/item?id=8341074 - Sept 2014 (2 comments)
This (archived) recent article from the Guardian has an account of Weizenbaum's early life that I hadn't seen before, about his childhood in Nazi Germany, emigration to the US and service in the Army during WWII, and the start of his career in computing:
https://archive.is/oxWZ5
https://archive.is/oxWZ5
Interesting insights! The article's perspective on technology's role in society sparks meaningful discussions. Embracing ethical tech practices is vital for a balanced future.
[deleted]
New title: Weizenbaum, "AI Pioneer" - Computer Power and Human Reason (1976)
The topic was "the special responsibility of a computer scientist" or something. He was talking about students at MIT, and how much of their research is funded by the military. How seemingly harmless programs like a talking Teddy bear would in the end up becoming part of a military drones etc. He gave a couple more examples and was suddenly talking about the sundown. How one can wonder when it is really setting. Is it when the lower side touches the horizon, or the middle. Or should one use the actual horizon or the apparent. Then all of a sudden he slammed his hands on the table, stood up and said in a said very loudly: But there is a difference between night and day!
That one hit hard. I think it changed me as a person.
He died a few weeks later. I never checked where he was buried. Maybe i should.