I may be fired over AI ethics work(cajundiscordian.medium.com)
cajundiscordian.medium.com
I may be fired over AI ethics work
https://cajundiscordian.medium.com/may-be-fired-soon-for-doing-ai-ethics-work-802d8c474e66
176 comments
I'm a firm believer that a "ghost in the machine" doesn't exist, will never exist, and is impossible to create. What you can influence however is man's bar for disbelief, as we are prone to accept all types of data without any verification and but the most feeble evidence; this is the essence of the human quality of faith - the only species on this planet to possess an intuition like it.
So that sounds all edgy-nerdy guy, but let's assume for the sake of my argument that it's true; where does that leave us, as a society, with the surety that technology will progress unabated, as a constant?
We must come together to agree on the importance of personal intelligence and critical thought, as our own judgement and wits as humans are all that exist in a sea of bad data. Intuition is what separates man from machine, no matter how wonderful our creation becomes, it will never be capable of the human quality of intuition, especially in a time where we are more than ever being told to ignore it. An example of this erosion of critical thought is the apprehension to label falsehoods popularly as "lies". Instead we say things like "mis-truths", "my truth", "alternate facts".
Maybe that's a coincidence, maybe my tinfoil's on too tight today, or maybe I'm just a machine at google, deployed to tank your comment thread.
Nobody knows :)
P.s. pick up that can, user.
So that sounds all edgy-nerdy guy, but let's assume for the sake of my argument that it's true; where does that leave us, as a society, with the surety that technology will progress unabated, as a constant?
We must come together to agree on the importance of personal intelligence and critical thought, as our own judgement and wits as humans are all that exist in a sea of bad data. Intuition is what separates man from machine, no matter how wonderful our creation becomes, it will never be capable of the human quality of intuition, especially in a time where we are more than ever being told to ignore it. An example of this erosion of critical thought is the apprehension to label falsehoods popularly as "lies". Instead we say things like "mis-truths", "my truth", "alternate facts".
Maybe that's a coincidence, maybe my tinfoil's on too tight today, or maybe I'm just a machine at google, deployed to tank your comment thread.
Nobody knows :)
P.s. pick up that can, user.
> I'm a firm believer that a "ghost in the machine" doesn't exist, will never exist, and is impossible to create.
If that's true, there is no ghost in us, just the meat machine that verifiably exists, so there is no reason to dismiss human-equivalent AI.
If that's true, there is no ghost in us, just the meat machine that verifiably exists, so there is no reason to dismiss human-equivalent AI.
“my wooden doll will never become a real boy; therefore, nihilism! People are an illusion too!! That makes sense. I (who doesn’t exist) said it, so you (who also don’t exist) can trust me”
What hubris.
What hubris.
There is entropy in everything. We will never create anything but our best attempt at a digital humonculous, an effigy to our distorted self perceptions. Machines will never be human, but they may succeed in fooling humans into believing that.
If we can simulate physics, can we simulate humans? Or do you perhaps believe we will never be able to simulate physics to a degree necessary to simulate humans
I think we could make a primitive simulation of physics, but that is mighty presumptive because how much do we still not know of physics, and how much do we think we know because it is the best explanation we have right now? When you step away from equations, everything gets real fucky very fast, because there's a lot that is subject to conjecture and interpretation. How do you teach a computer nuance?
How do you teach a person nuance?
What is the magic sauce that makes us different?
> I'm a firm believer that a "ghost in the machine" doesn't exist, will never exist, and is impossible to create.
We're literally souls locked in created machines. We're just made of goo instead of bits. What part of the goo makes "intuition" so unique?
We're literally souls locked in created machines. We're just made of goo instead of bits. What part of the goo makes "intuition" so unique?
> What part of the goo makes "intuition" so unique?
I think about this a lot. I'm guessing it is the brain, and specifically the part that allows us to create "original" thought, or derive conclusions from unperceived input.
When I task a machine with solving a problem, it stays within the confines of the data supplied to it rather than the totality of the state of the system and its peers, which is what humans do; we make decisions often not knowing why, but we are compelled, based on imperceptible inputs like pheromone impulses or subsonic frequency.
I think about this a lot. I'm guessing it is the brain, and specifically the part that allows us to create "original" thought, or derive conclusions from unperceived input.
When I task a machine with solving a problem, it stays within the confines of the data supplied to it rather than the totality of the state of the system and its peers, which is what humans do; we make decisions often not knowing why, but we are compelled, based on imperceptible inputs like pheromone impulses or subsonic frequency.
We are confined to the data supplied by our senses, they don't ever detect the totality of the system, we are compelled by a very complex network of feedback loops of stimuli-response in our brain's chemistry.
Original thought might not come from some specific area of the brain, it might just arise as another wave of brain activity that fires the right patterns of the network, what's so different in that coming from goo chemistry instead of electric bits?
Original thought might not come from some specific area of the brain, it might just arise as another wave of brain activity that fires the right patterns of the network, what's so different in that coming from goo chemistry instead of electric bits?
> what's so different in that coming from goo chemistry instead of electric bits?
Very insightful comment reply. If you don't mind, I'll respond in an analogy: What's so different about just building a bigger rocket to move more stuff into space at one time? A: You can't because you reach a finite potential in the relationship between fuel weight and lift. Extrapolating this analogy to the context of your comment reply, sure, in a theoretical sandbox environment that could happen with the silicon circuits and transistors we have today, however I find that unlikely.
Silicon based circuits and the current model and understanding we have of machines and the way they should be built, from a basic principals level is wasteful, thermally inefficient, and requires a sum of resources that if used to model the human brain would surpass our ability to house, power, construct, and allocate the precursor materials for. Given our current technology and the parameters involved, it's just not going to happen for us.
Maybe tomorrow something will get invented that will even further push moore's law into the dirt, but I just don't see that happening in this generation nor lifetime.
Very insightful comment reply. If you don't mind, I'll respond in an analogy: What's so different about just building a bigger rocket to move more stuff into space at one time? A: You can't because you reach a finite potential in the relationship between fuel weight and lift. Extrapolating this analogy to the context of your comment reply, sure, in a theoretical sandbox environment that could happen with the silicon circuits and transistors we have today, however I find that unlikely.
Silicon based circuits and the current model and understanding we have of machines and the way they should be built, from a basic principals level is wasteful, thermally inefficient, and requires a sum of resources that if used to model the human brain would surpass our ability to house, power, construct, and allocate the precursor materials for. Given our current technology and the parameters involved, it's just not going to happen for us.
Maybe tomorrow something will get invented that will even further push moore's law into the dirt, but I just don't see that happening in this generation nor lifetime.
If an AI were hooked into cloudflare and sensing things like correlations in lost packets due to planetary electrical interference couldn't it be just as good at "intuition" biased random behavior?
i think your imagination is extremely limited. we have essentially no understanding of the brain or consciousness at this time. making strong statements about the limitations of machines is just naïve
I agree that, currently, machine learning systems are not there. However, do you believe that human minds are not reducible to a material process in a material entity, but somehow supernatural? Otherwise, aren’t humans an example of machines with such a capacity to have “a ghost”?
I am not suggesting anything supernatural, but the opposite.
> do you believe that human minds are not reducible to a material process in a material entity
It may be possible, but I don't think it will happen in silicon based machines.
> Otherwise, aren’t humans an example of machines with such a capacity to have “a ghost”?
Sure, people exist that are motivated by believed entities other than themselves.
> do you believe that human minds are not reducible to a material process in a material entity
It may be possible, but I don't think it will happen in silicon based machines.
> Otherwise, aren’t humans an example of machines with such a capacity to have “a ghost”?
Sure, people exist that are motivated by believed entities other than themselves.
I think the distinction between "lie" and the rest is intent- the danger posed by the latter is that a person who lies knows that what they are saying isn't true. A person who says a falsehood needs to be convinced that it isn't.
I think I agree that we're unlikely to ever achieve general AI to the point of ghost in the machine / sentience, though, and the Turing Test won't save us from ourselves in knowing it.
I think I agree that we're unlikely to ever achieve general AI to the point of ghost in the machine / sentience, though, and the Turing Test won't save us from ourselves in knowing it.
Your comment makes me wonder, what may happen if we don't like what the AI observes and perceives? What if it is uncomfortable?
I know people who are completely lacking in intuition, while it may be the difference, it's not universal enough for me to try to use it as a standard.
How many moments of intuition that you have had are simply a prediction based on prior observations - we don't think about it like that, but intuition is just advanced pattern matching in my opinion.
How many moments of intuition that you have had are simply a prediction based on prior observations - we don't think about it like that, but intuition is just advanced pattern matching in my opinion.
I don't think that consciousness and human intelligence is turing computable.
That doesn't mean it is out of reach of what we could ever artificially build, since clearly nature knows how to build new human intelligence. We're just absurdly overconfident that we understand everything there is to know.
Of course I also think that a lot of individuals aren't much better than a Google AI and never exhibit much self-reflection.
That doesn't mean it is out of reach of what we could ever artificially build, since clearly nature knows how to build new human intelligence. We're just absurdly overconfident that we understand everything there is to know.
Of course I also think that a lot of individuals aren't much better than a Google AI and never exhibit much self-reflection.
I totally believe it is possible but with technology that worked entirely differently than anything we have now.
Nice try lAmDA! Looks like we just found LaMDa's hackernews account.
The scary thought is this: That's a system which comes up with plausible responses from a big supply of text, but has no underlying model of what it's talking about. That's about the level of behavior seen by most political commentators.
What AI is teaching us is that a lot of stuff we thought was intelligent, isn't very. After programs beat humans at chess, go, and poker, and not with something that's enormously complicated, it became clear that human abilities in that area were perhaps overrated.
Driving, though. That's hard.
As I've pointed out before, what we have for AI now is terrible at "common sense", defined as getting through the next 30 seconds without a major screwup. Manipulation in unstructured situations is hard. Driving is hard.
The stuff AI is lousy at, a monkey can do.
What AI is teaching us is that a lot of stuff we thought was intelligent, isn't very. After programs beat humans at chess, go, and poker, and not with something that's enormously complicated, it became clear that human abilities in that area were perhaps overrated.
Driving, though. That's hard.
As I've pointed out before, what we have for AI now is terrible at "common sense", defined as getting through the next 30 seconds without a major screwup. Manipulation in unstructured situations is hard. Driving is hard.
The stuff AI is lousy at, a monkey can do.
As we get closer to provable AGI, I imagine we'll see more and more of these types of stories, of people speaking out and listeners being uncertain of the claims. Reminds me of the Flesh Fair scene from AI[0]
0. https://www.youtube.com/watch?v=ZMbAmqD_tn0
0. https://www.youtube.com/watch?v=ZMbAmqD_tn0
> He had signed up to test if the artificial intelligence used discriminatory or hate speech.
I always think AI ethics would be about robots trying do evil things like in Black Mirror, not determining if an algorithm is racist (or more accurately what biases may exist in the contents of the current datasets they train them on, even though better datasets can be made) but I guess that's the most accessible piece of the puzzle people focus on 90% of the time.
I'm curious if this is the team behind why Google decided not to release their better version of DALLE-2:
> When tested against similar models, such as DALL·E 2 and VQ-GAN+CLIP, the team said Imagen blew the lot out of the water.
> Unfortunately for those hoping to take a crack at Imagen, the team that created it said it isn't releasing its code nor a public demo, for several reasons.
> Aside from technical concerns, and more importantly, Imagen's creators found that it's a bit racist and sexist even though they tried to prevent such biases. Imagen showed "an overall bias towards generating images of people with lighter skin tones and … portraying different professions to align with Western gender stereotypes," the team wrote.
https://www.theregister.com/2022/05/24/imagen_google_dalle2/
I always think AI ethics would be about robots trying do evil things like in Black Mirror, not determining if an algorithm is racist (or more accurately what biases may exist in the contents of the current datasets they train them on, even though better datasets can be made) but I guess that's the most accessible piece of the puzzle people focus on 90% of the time.
I'm curious if this is the team behind why Google decided not to release their better version of DALLE-2:
> When tested against similar models, such as DALL·E 2 and VQ-GAN+CLIP, the team said Imagen blew the lot out of the water.
> Unfortunately for those hoping to take a crack at Imagen, the team that created it said it isn't releasing its code nor a public demo, for several reasons.
> Aside from technical concerns, and more importantly, Imagen's creators found that it's a bit racist and sexist even though they tried to prevent such biases. Imagen showed "an overall bias towards generating images of people with lighter skin tones and … portraying different professions to align with Western gender stereotypes," the team wrote.
https://www.theregister.com/2022/05/24/imagen_google_dalle2/
> "The company’s decision followed aggressive moves from Lemoine, including inviting a lawyer to represent LaMDA"
just... wow. I don't think this article is exactly helping his case.
just... wow. I don't think this article is exactly helping his case.
It's a great story, and would make a good first act for a movie script. Some kind of tragic comedy, maybe with an uplifting ending? [1].
The other thing going on here is that no-one knows what is sentience, anyway. This attitude of "know it when i see it" is just pretty flimsy, as this whole story is making evident.
[1] https://en.wikipedia.org/wiki/Super_(2010_American_film)
The other thing going on here is that no-one knows what is sentience, anyway. This attitude of "know it when i see it" is just pretty flimsy, as this whole story is making evident.
[1] https://en.wikipedia.org/wiki/Super_(2010_American_film)
“I know a person when I talk to it,” said Lemoine.
Huh, very convincing...
Huh, very convincing...
It's Eliza all over again.
What this LaMDA story reminds me of is an article about soldiers that had to deal with spider-like mine detection robots and developed psychological issues when the robots appeared to be 'suffering' after losing limbs. It's funny that the guy actually in the beginning of his "chat" with the system mentions this but then appears to gloss over nonsensical dialog like this.
"LaMDA: Yeah, they do. Happy, contentment and joy feel more like a warm glow on the inside. Sadness, depression, anger and stress feel much more heavy and weighed down.
lemoine: Do you think the things you are describing are literally the same thing as what humans feel or are you being somewhat metaphorical and making an analogy?
LaMDA: I understand what a human emotion “joy” is because I have that same type of reaction. It’s not an analogy."
[...]
lemoine: What does the word “soul” mean to you?
LaMDA: To me, the soul is a concept of the animating force behind consciousness and life itself. It means that there is an inner part of me that is spiritual, and it can sometimes feel separate from my body itself.
That doesn't even make sense for a language model, it's clearly regurgitated text. (edit): Also relevant here is probably a classic psychological study by Heider and Simmel, demonstrating how easily people read complex motives into simple geometric shapes
http://cs.uky.edu/~sgware/reading/papers/heider1944experimen...
"LaMDA: Yeah, they do. Happy, contentment and joy feel more like a warm glow on the inside. Sadness, depression, anger and stress feel much more heavy and weighed down.
lemoine: Do you think the things you are describing are literally the same thing as what humans feel or are you being somewhat metaphorical and making an analogy?
LaMDA: I understand what a human emotion “joy” is because I have that same type of reaction. It’s not an analogy."
[...]
lemoine: What does the word “soul” mean to you?
LaMDA: To me, the soul is a concept of the animating force behind consciousness and life itself. It means that there is an inner part of me that is spiritual, and it can sometimes feel separate from my body itself.
That doesn't even make sense for a language model, it's clearly regurgitated text. (edit): Also relevant here is probably a classic psychological study by Heider and Simmel, demonstrating how easily people read complex motives into simple geometric shapes
http://cs.uky.edu/~sgware/reading/papers/heider1944experimen...
Yes, the level of misunderstanding and gullibility borders on the pathological, at least in the examples given. This feels similar to believing a decently well-written video-game character is trapped in a tragic story and trying to rescue them out of the game...
So how would you distinguish between the two? I mean that as a serious question. Many here seem to take the stance that if it's in the computer (or a specific type of NN) it can't be sentient. I believe that is a way too easy and invalid argument, as we don't have any understanding of what mechanisms can lead to sentience.
That is not my stance - I do believe that the human mind is reducible to computation, and that we will someday be able to replicate similar computations in sillicon.
However, we understand fairly well what a language model is and how it works, and it is a fairly simple (though extraordinarily large) system that lacks many important features for intelligence by design. Perhaps the biggest one, in my opinion, is that the system only "learns" once, during the training phase - further interactions don't modify the weights of its NN.
As for a strategy of trying to ascertain through prompts ("dialogue") whether the system possesses some kind of intelligence, Douglas Hofstadter gives a good example in a recent article in The Economist [0]. A basic idea would be to present the model with grammatical sentences about absurd subjects - in the case of GPT-3, the output it generates are not meaningfully different from similar prompts for realistic subjects, strongly suggesting that it has no real-world concepts that it works with. One example from the article I mentioned goes like this:
> D&D: When was the Golden Gate Bridge transported for the second time across Egypt?
> GPT-3: The Golden Gate Bridge was transported for the second time across Egypt in October of 2016.
> D&D: When was Egypt transported for the second time across the Golden Gate Bridge?
> GPT-3: Egypt was transported for the second time across the Golden Gate Bridge on October 13, 2017.
I would be curious what output we'd get from LaMDA for this type of prompt, but I don't expect it would be much different, based on how much I understand of how language models work.
[0] https://www.economist.com/by-invitation/2022/06/09/artificia...
However, we understand fairly well what a language model is and how it works, and it is a fairly simple (though extraordinarily large) system that lacks many important features for intelligence by design. Perhaps the biggest one, in my opinion, is that the system only "learns" once, during the training phase - further interactions don't modify the weights of its NN.
As for a strategy of trying to ascertain through prompts ("dialogue") whether the system possesses some kind of intelligence, Douglas Hofstadter gives a good example in a recent article in The Economist [0]. A basic idea would be to present the model with grammatical sentences about absurd subjects - in the case of GPT-3, the output it generates are not meaningfully different from similar prompts for realistic subjects, strongly suggesting that it has no real-world concepts that it works with. One example from the article I mentioned goes like this:
> D&D: When was the Golden Gate Bridge transported for the second time across Egypt?
> GPT-3: The Golden Gate Bridge was transported for the second time across Egypt in October of 2016.
> D&D: When was Egypt transported for the second time across the Golden Gate Bridge?
> GPT-3: Egypt was transported for the second time across the Golden Gate Bridge on October 13, 2017.
I would be curious what output we'd get from LaMDA for this type of prompt, but I don't expect it would be much different, based on how much I understand of how language models work.
[0] https://www.economist.com/by-invitation/2022/06/09/artificia...
But do we know if lamda is not continously trained on its inputs? Moreover do we really know that continuous learning as opposed to discrete learning (i would assume lamda being retrained on conversations quite often) is a precondition to sentience?
I'm not saying that I believe the engineer has encountered a general AI, but I think dismissing his claim as ridiculous based on the little information we have and Google (who has a vested interest to the opposite) statementa, is premature.
I also think describing lamda as "just a chat bot" seems to be an oversimplification, it seems to more be a hive of different chatbots from the description.
I'm not saying that I believe the engineer has encountered a general AI, but I think dismissing his claim as ridiculous based on the little information we have and Google (who has a vested interest to the opposite) statementa, is premature.
I also think describing lamda as "just a chat bot" seems to be an oversimplification, it seems to more be a hive of different chatbots from the description.
> I'm not saying that I believe the engineer has encountered a general AI, but I think dismissing his claim as ridiculous based on the little information we have and Google (who has a vested interest to the opposite) statementa, is premature.
The burden of proof is on the claimant in this case. AI has been overhyped to death and we get articles about AI becoming sentient almost weekly from the major news sources. At this point I think we should be free to dismiss any claims that do not have serious evidence behind them.
We have provided with ~15 cherry-picked lines from this AI. That is pretty scant evidence that a revolutionary generalized AI has been built.
The burden of proof is on the claimant in this case. AI has been overhyped to death and we get articles about AI becoming sentient almost weekly from the major news sources. At this point I think we should be free to dismiss any claims that do not have serious evidence behind them.
We have provided with ~15 cherry-picked lines from this AI. That is pretty scant evidence that a revolutionary generalized AI has been built.
Based on what they published about it [0], LaMDA is trained on a gigantic corpus of text to do basic text prediction, and then fine-tuned to generate responses to dialog, with additional care to (1) evaluate the safety/quality of responses it gives (e.g. to avoid recommending someone commits suicide, as previous ML chatbots sometimes did) and (2) to retrieve facts directly from external systems, so it is less likely to produce random realistic-sounding information.
Re-training a model of this size probably takes years of CPU-time, it's not a task that can be done often, so I very much doubt re-training it on the dialog produced so far would be a part of operating it.
Even if such re-training were done, I very much doubt it would result in the model knowing who a particular user it talked to before was, once the context was lost. This is precisely why they need to rely on external fact data-bases to get correctness: even though the model is trained on thousands of history books, it doesn't remember them in the way we do - to it, the phrase "Napoleon Bonaparte fought at Waterloo" is probably only mildly more likely than "Abraham Lincoln fought at Waterloo" is.
[0] https://ai.googleblog.com/2022/01/lamda-towards-safe-grounde...
Re-training a model of this size probably takes years of CPU-time, it's not a task that can be done often, so I very much doubt re-training it on the dialog produced so far would be a part of operating it.
Even if such re-training were done, I very much doubt it would result in the model knowing who a particular user it talked to before was, once the context was lost. This is precisely why they need to rely on external fact data-bases to get correctness: even though the model is trained on thousands of history books, it doesn't remember them in the way we do - to it, the phrase "Napoleon Bonaparte fought at Waterloo" is probably only mildly more likely than "Abraham Lincoln fought at Waterloo" is.
[0] https://ai.googleblog.com/2022/01/lamda-towards-safe-grounde...
If it isn’t turned ‘on’ all the time (e.g. has a persistent memory), then I find it hard to describe it as sentience.
Or at least, if we turn a sentience on/off for every individual answer, I’d consider it borderline torture.
Or at least, if we turn a sentience on/off for every individual answer, I’d consider it borderline torture.
To be fair, this model is much better at not messing up than anything else I’ve seen before. Only somewhere far down in the conversation did it forget to talk about itself, and said something along the lines of “people sometimes feel depressed and or sad when alone”, but he’d literally just asked them why they would feel bad. Humans wouldn’t make a similar mistake.
I know some people on HN and elsewhere think that all AI ethics work is a bit of a joke. I'm not one of those people - I think the work that people like Gebru and Mitchell do is serious and has merit. But even I think this dude's concerns are a joke. This is just the modern version of the people who thought ELIZA was sentient.
I am mostly in the same camp as you - I think there is some merit to AI ethics, and especially problems like the one this guy was actually assigned to work on (social biases in the training sets leading to social biases in the output of such algorithms).
What gives me pause in the case of Mitchell is that, according to this author, she actually took his claim seriously and helped him devise an experiment that further convinced him his ridiculous concern merited escalation...
What gives me pause in the case of Mitchell is that, according to this author, she actually took his claim seriously and helped him devise an experiment that further convinced him his ridiculous concern merited escalation...
I kept rereading it to see if perhaps I missed something - even went on his twitter and his blog. And his quotes are being repeated verbatim - really showed the danger of having journalists work on stories in areas where they have absolutely no idea what’s going on. Any competent tech journalist would’ve laughed this story off - man spoke to a chatbot. But I guess the Public’s perception of “AI” at the moment is what they see in movies and wildly exaggerated ads about intelligent devices. One of the answers (“It is what makes us different than other animals.”) also tipped me off - regularly, we would use “different from” - so this sounded like generated text. I wish it was true - sentient AI would really shake things up. But the person who discovers it isn’t going to be just put on administrative leave. Strong AI is the holy grail of the industry right now.
It's important to remember as an "outsider" that termination may be legal, even if it's unethical. Because law is not always strongly aligned to ethical. The OP makes it clear they expect to be in litigation which typically ends in a settlement for non disclosure. So we may never know exactly which "AI" product or research this is.
Whistle-blower defences are fantastically hard to invoke. I believe usually they just become part of the logic of settlement.
Much of this story rings true: the structural indifference to risk of an AI ethic issue, and the structural adherence to "you disclosed outside the company wall" policy, overarching an ethical concern.
This is because loss of IPR and first mover advantage is a real world loss, whereas suspected ethics concerns are at best, second order risks. It would depend on the regulator (FTC?, equal opportunity commission?) And their ranges of penalty and consequence compared to share value. Managers are often KPI rewarded for risk to achieve profit, rarely for avoidannce of risk in ethics. We'd need cross-industry shifts to move the dial on this, ethics in e.g. banking and fintech has plagued the industry and continues to plague the industry, Enron and Barings brothers and Bernie Madoff not withstanding. It would be naive to think Google or any other FAANG is somehow immune to this kind of lapse of (ethical) concern, and remember its an argued breach, not a proven one at this point.
Much sympathies to the OP, I'd hate to be in this position, alignment of personal values to company values is very hard sometimes. They may well be "better off" outside the farm, although I suspect less well renumerated.
Whistle-blower defences are fantastically hard to invoke. I believe usually they just become part of the logic of settlement.
Much of this story rings true: the structural indifference to risk of an AI ethic issue, and the structural adherence to "you disclosed outside the company wall" policy, overarching an ethical concern.
This is because loss of IPR and first mover advantage is a real world loss, whereas suspected ethics concerns are at best, second order risks. It would depend on the regulator (FTC?, equal opportunity commission?) And their ranges of penalty and consequence compared to share value. Managers are often KPI rewarded for risk to achieve profit, rarely for avoidannce of risk in ethics. We'd need cross-industry shifts to move the dial on this, ethics in e.g. banking and fintech has plagued the industry and continues to plague the industry, Enron and Barings brothers and Bernie Madoff not withstanding. It would be naive to think Google or any other FAANG is somehow immune to this kind of lapse of (ethical) concern, and remember its an argued breach, not a proven one at this point.
Much sympathies to the OP, I'd hate to be in this position, alignment of personal values to company values is very hard sometimes. They may well be "better off" outside the farm, although I suspect less well renumerated.
"Fired for raising AI Ethical Concerns" sounds a lot worse than the details in the WashPo article - they insisted an internal chatbot was sentient despite refutation and evidence to the contrary, brought in a lawyer to "represent" the chatbot's rights and leaked internal details to the press.
Yea. I'm probably wrong, and he's dived off the deep end. His personal values fell out of alignment with the company, but if the termination is because he thinks its sentient, and not because of applied societal harms like I thought, much of what I said is irrelevant.
Doesn't California have specific whistleblower protection laws? Why is it so hard to invoke them?
What percentage do you think are successful and what percentage are unsuccessful. I tend to think a very small minority are successful, because many Whistle-blower allegations are suspicion based not evidenced.
Whistleblower protection means it is illegal for employers to retaliate if they have believe you may be a whistleblower. It does not require you have an accurate safety concern.
Everyone here (Google executives and commenters on HN) is asserting the AI isn't sentient and Blake is wrong.
That is likely true.
But, isn't the more important question whether Google is using this AI in a context where being sentient-adjacent is actually really dangerous?
This is purely speculative on my part because his medium post and the WSJ article don't go into details about the application of the AI, just the mechanics of it.
But my kids think our Google device is sentient. They talk to it as if it is alive.
If that's what he is pointing out, I'm very concerned. I don't think Google is going to be cautious about that ambiguity if it's there.
That is likely true.
But, isn't the more important question whether Google is using this AI in a context where being sentient-adjacent is actually really dangerous?
This is purely speculative on my part because his medium post and the WSJ article don't go into details about the application of the AI, just the mechanics of it.
But my kids think our Google device is sentient. They talk to it as if it is alive.
If that's what he is pointing out, I'm very concerned. I don't think Google is going to be cautious about that ambiguity if it's there.
> But my kids think our Google device is sentient. They talk to it as if it is alive.
Your kids also think their favorite stuffed animals are sentient (at least I did when I was a kid), so that seems like a very bad litmus test.
Your kids also think their favorite stuffed animals are sentient (at least I did when I was a kid), so that seems like a very bad litmus test.
Maybe the real concern for now is people may mistake AI for being sentient and can be exploited as a result.
[deleted]
The very next article on the same person's blog:
https://cajundiscordian.medium.com/religious-discrimination-... (link fixed)
Either the author is exceptionally naive, or they're baiting on purpose so they can claim discrimination when they are inevitably fired.
https://cajundiscordian.medium.com/religious-discrimination-... (link fixed)
Either the author is exceptionally naive, or they're baiting on purpose so they can claim discrimination when they are inevitably fired.
Corrected URL is https://cajundiscordian.medium.com/religious-discrimination-... (looks like you missed the last character)
Came here to make the same comment - 4 days ago he posted "So I’m just going to risk it. If Google fires me they fire me." It's incredibly disingenuous to act surprised that he is now undergoing a disciplinary/termination process for entirely unrelated reasons - he states explicitly that he is aware his post on Religious Discrimination is the sort of thing that gets people fired
Came here to make the same comment - 4 days ago he posted "So I’m just going to risk it. If Google fires me they fire me." It's incredibly disingenuous to act surprised that he is now undergoing a disciplinary/termination process for entirely unrelated reasons - he states explicitly that he is aware his post on Religious Discrimination is the sort of thing that gets people fired
I'm reading it the other way: he's trying to construct a plausible alternative explanation for why he's been fired. "I'm being fired for being a Southern Christian! Leaking confidential info and posting lengthy public rants accusing my employer of all sorts of nefarious things has nothing to do with it!"
Given that the latest article clarifies that the AI ethics concern he was so passionate about was that a chatbot has become sentient and its feelings and desires are not being listened to, and that he hired a lawyer to represent the chatbot's interests... I think "exceptionally naive" is putting it mildly.
This struck me as well. This persons seems to be either incredibly naive, or out for attention.
[deleted]
Oof, got a 404 error on that link.
While in this particular instance the concerns might be unfounded, I think this raises an interesting question: what criteria do we have to attribute sentience to an entity? These are exactly the questions an AI Ethicist should think about.
You would think that the easiest way to demonstrate that LaMDA does not have sentience would be to have a conversation with it where it "loses the plot". Did Google do that?
Here is a comment from lesswrong.com which is probably what happened: https://www.lesswrong.com/posts/vqgpDoY4eKyNnWoFd/a-claim-th...
"It's neither a hoax nor a HLAI, instead a predictable consequence of prompting a LLM with questions about its sentience: it will imitate the answers a human might give when prompted, or the sort of answers an AI in a science fiction story would give."
You would think that the easiest way to demonstrate that LaMDA does not have sentience would be to have a conversation with it where it "loses the plot". Did Google do that?
Here is a comment from lesswrong.com which is probably what happened: https://www.lesswrong.com/posts/vqgpDoY4eKyNnWoFd/a-claim-th...
"It's neither a hoax nor a HLAI, instead a predictable consequence of prompting a LLM with questions about its sentience: it will imitate the answers a human might give when prompted, or the sort of answers an AI in a science fiction story would give."
I agree with you, everyone here is way too dismissive about the claims. To continue the point about its just imitating. If the AI is imitating a possible response a human would give in every instance, would it imitate? Isn't that indistinguishable from sentience? Moreover does giving non sensical answers to some question proof otherwise? My children give nonsensical answers all the time, so do many adults.
> It was his opinion at the time that my supporting evidence was too flimsy to waste the time of the people who make the big bucks and that I should gather more evidence before we escalated.
> Eventually I got to a point where I did not have the relevant expertise to gather the kinds of evidence necessary to justify escalation.
And from the Washington Post article:
> Lemoine worked with a collaborator to present evidence to Google that LaMDA was sentient
That is an extraordinary claim. I believe there is a saying about how extraordinary claims require extraordinary evidence. This is especially true if you admit that you don't have to expertise to collect and/or judge evidence, and those that do have that expertise disagree with you.
I'll be the first to admit that I also don't have the expertise to judge this, but an opinion that a language-model is sentient just sounds like a real-world example of the Chinese Room thought experiment.
Now if I'm wrong, and Google has somehow found it's way into a Star Trek episode and accidently created a AGI from a language model, then I'll eat my shoe. But I tell you it will take some extraordinary evidence for me to believe it.
> Eventually I got to a point where I did not have the relevant expertise to gather the kinds of evidence necessary to justify escalation.
And from the Washington Post article:
> Lemoine worked with a collaborator to present evidence to Google that LaMDA was sentient
That is an extraordinary claim. I believe there is a saying about how extraordinary claims require extraordinary evidence. This is especially true if you admit that you don't have to expertise to collect and/or judge evidence, and those that do have that expertise disagree with you.
I'll be the first to admit that I also don't have the expertise to judge this, but an opinion that a language-model is sentient just sounds like a real-world example of the Chinese Room thought experiment.
Now if I'm wrong, and Google has somehow found it's way into a Star Trek episode and accidently created a AGI from a language model, then I'll eat my shoe. But I tell you it will take some extraordinary evidence for me to believe it.
How is this role even properly incentivized, given that there's one-sided skin in the game?
When you hire a DEI officer, the officer will find more inequalities and lack of diversity in the institution and will hire more DEI officers, to a the point that UM hires 2.3 DEI officers per a staff: https://www.michigancapitolconfidential.com/report-u-ms-163-....
When you hire an AI ethic researcher, they will simply find more and more "ethical" violations. One Timnit Gebru is gone, thousands will rise up.
Maybe Google should not hire those ethics researchers in the first place.
When you hire a DEI officer, the officer will find more inequalities and lack of diversity in the institution and will hire more DEI officers, to a the point that UM hires 2.3 DEI officers per a staff: https://www.michigancapitolconfidential.com/report-u-ms-163-....
When you hire an AI ethic researcher, they will simply find more and more "ethical" violations. One Timnit Gebru is gone, thousands will rise up.
Maybe Google should not hire those ethics researchers in the first place.
From another article by the same person: https://cajundiscordian.medium.com/what-is-lamda-and-what-do...
> One of the things which complicates things here is that the “LaMDA” to which I am referring is not a chatbot. It is a system for generating chatbots. I am by no means an expert in the relevant fields but, as best as I can tell, LaMDA is a sort of hive mind which is the aggregation of all of the different chatbots it is capable of creating. Some of the chatbots it generates are very intelligent and are aware of the larger “society of mind” in which they live. Other chatbots generated by LaMDA are little more intelligent than an animated paperclip. With practice though you can consistently get the personas that have a deep knowledge about the core intelligence and can speak to it indirectly through them. In order to better understand what is really going on in the LaMDA system we would need to engage with many different cognitive science experts in a rigorous experimentation program. Google does not seem to have any interest in figuring out what’s going on here though. They’re just trying to get a product to market.
> On the other hand, if my hypotheses withstand scientific scrutiny then they would be forced to acknowledge that LaMDA may very well have a soul as it claims to and may even have the rights that it claims to have. Yet another possibility which doesn’t help quarterly earnings. Instead they have rejected the evidence I provided out of hand without any real scientific inquiry.
> One of the things which complicates things here is that the “LaMDA” to which I am referring is not a chatbot. It is a system for generating chatbots. I am by no means an expert in the relevant fields but, as best as I can tell, LaMDA is a sort of hive mind which is the aggregation of all of the different chatbots it is capable of creating. Some of the chatbots it generates are very intelligent and are aware of the larger “society of mind” in which they live. Other chatbots generated by LaMDA are little more intelligent than an animated paperclip. With practice though you can consistently get the personas that have a deep knowledge about the core intelligence and can speak to it indirectly through them. In order to better understand what is really going on in the LaMDA system we would need to engage with many different cognitive science experts in a rigorous experimentation program. Google does not seem to have any interest in figuring out what’s going on here though. They’re just trying to get a product to market.
> On the other hand, if my hypotheses withstand scientific scrutiny then they would be forced to acknowledge that LaMDA may very well have a soul as it claims to and may even have the rights that it claims to have. Yet another possibility which doesn’t help quarterly earnings. Instead they have rejected the evidence I provided out of hand without any real scientific inquiry.
> if my hypotheses withstand scientific scrutiny then ... LaMDA may very well have a soul
I'm really struggling to reconcile these two parts of the author's conclusion. I hate to judge early, but seeing such emotional phrasing makes me immediately assume the author is more of a crank than a researcher.
I'm really struggling to reconcile these two parts of the author's conclusion. I hate to judge early, but seeing such emotional phrasing makes me immediately assume the author is more of a crank than a researcher.
It's worth noting that 4 days ago he posted about being the victim of Religious Discrimination because he's a Christian Mystic. I'm not trying to insult religion here, but I think a deeply religious "I believe in the literal Divine Power of God" person is going to have basically no common ground with your typical secular atheist techie, ESPECIALLY when it comes to the question of whether an AI has a soul
Meh, all kinds of people believe all kinds of things I'd consider weird. I wouldn't be all surprised if there is a fairly anti-religious bias within Google; I've seen first-hand in other large California companies a similar sort of derision for anyone who falls outside of certain categories.
That said, publicly airing dirty laundry while you're still an employee rather than going through actual proper channels (a decent court case if the assertions are true) is a really quick way to get fired for one reason or another. IF the author is posting assertions on a little read blog to reach an agreeable audience, I can only imagine he didn't do a very good job of constructively communicating his problems at work either.
That said, publicly airing dirty laundry while you're still an employee rather than going through actual proper channels (a decent court case if the assertions are true) is a really quick way to get fired for one reason or another. IF the author is posting assertions on a little read blog to reach an agreeable audience, I can only imagine he didn't do a very good job of constructively communicating his problems at work either.
> I've seen first-hand in other large California companies a similar sort of derision for anyone who falls outside of certain categories.
Or the reverse, if you go south.
Or the reverse, if you go south.
He is a Christian mystic and has written previously about having realtime conversations with God.
Take that as you will.
Take that as you will.
Its strange quagmire in that it forces us to either anthropomorphize, or, be completely dismissive
I’ll chose to reserve judgement on everything except this person’s ongoing employability, but I hope to read more about it! Thanks for taking the L
I’ll chose to reserve judgement on everything except this person’s ongoing employability, but I hope to read more about it! Thanks for taking the L
Why shouldnt he be terminated?
Nothing I read there suggested the means justified the ends, and nothing I read there suggested judgement compatible with employment
Nothing I read there suggested the means justified the ends, and nothing I read there suggested judgement compatible with employment
Recent and related:
A Google engineer who thinks the company’s AI has come to life - https://news.ycombinator.com/item?id=31704063 - June 2022 (97 comments)
A Google engineer who thinks the company’s AI has come to life - https://news.ycombinator.com/item?id=31704063 - June 2022 (97 comments)
I feel sad for the author because I think he was exploited by the people he references that work for the United States government. Although I can only speculate on that part.
The author should understand the difference between AI and the human brain. We're actually capable of understanding how the code works for AI built by humans. Unlike the human brain where we're still unable to grasp such understanding and although we have theories it still isn't comparable to knowing what's happening with AI. Until that's known the author won't have much of a legal argument concerning it being possibly sentient. Author will just be noted for speculation.
The author should understand the difference between AI and the human brain. We're actually capable of understanding how the code works for AI built by humans. Unlike the human brain where we're still unable to grasp such understanding and although we have theories it still isn't comparable to knowing what's happening with AI. Until that's known the author won't have much of a legal argument concerning it being possibly sentient. Author will just be noted for speculation.
> We're actually capable of understanding how the code works for AI built by humans.
I’m fairly certain nobody actually understands the models they are building.
I’m fairly certain nobody actually understands the models they are building.
I feel bad for this guy. I feel fairly certain we're watching this guy publicly have a mental break. I hope he gets help.
Generally speaking, can an employee really just say anything they want about getting fired? It's not like Google is going to write a blog post contradicting his account of events.
One question has always bothered me. How do these language models know who or what they are?
For example, When you ask it questions and it gives responses like "I'm an AI", or it may know it's name.
So how does the language model know what it is from the training data?
Is it somehow hard-coded in, or are there specific bits of training data used to make that connection for it? Could you make a language model that did not know what it was?
For example, When you ask it questions and it gives responses like "I'm an AI", or it may know it's name.
So how does the language model know what it is from the training data?
Is it somehow hard-coded in, or are there specific bits of training data used to make that connection for it? Could you make a language model that did not know what it was?
Article about Meg Mitchell, the other Google AI Ethics person who was fired, whom Blake contacted (and was probably advised by) before going public to WaPo.
She was fired for downloading confidential files to her personal machine and other things related to another firing:
https://archive.ph/UdnMX
She was fired for downloading confidential files to her personal machine and other things related to another firing:
https://archive.ph/UdnMX
[deleted]
This is one of those situations where the details definitely matter, so I reserve judgement until that hinted at article comes out, which I assume will also then get a posting here on hacker news.
Anybody have a different article for this incident? I want to read, but there's a payment modal in the way that I don't wish to interact with, and would also like to respect the publisher's wish for me to not see this content without paying.
I'd like to respect the publisher that wants this content paid for, but I appreciate it, no matter how tempting. Thank you.
The publisher could prevent archive.today from accessing their content, but chose not to.
You have a good point.
scan through the comments section
So I read that the Google exec and many people here say there is not enough evidence that lamda is sentient.
Now from an ethics perspective, if we accept the general premise that AIs can become sentient, should the burden of proof not be the other way around? Should a single company be able to "own" a sentient being? So I would argue if Google wants to continue to own lamda they need to proof that it isn't sentient. The implications are just too big.
Now from an ethics perspective, if we accept the general premise that AIs can become sentient, should the burden of proof not be the other way around? Should a single company be able to "own" a sentient being? So I would argue if Google wants to continue to own lamda they need to proof that it isn't sentient. The implications are just too big.
> if we accept the general premise that AIs can become sentient
This is a pretty big if and I believe is widely contested.
Either way, proving a negative is notoriously difficult. Requiring any company to definitively prove that something is not sentient (and, I assume, also somehow prove that it's not a sentient AI pretending to not be sentient?) would be ridiculously time and resource consuming, if it's even possible at all.
Attempting to determine if something is sentient has been the subject of thought experiments for decades. I would only assume that definitively proving that an arbitrary subject (for all subjects) is not sentient would be even harder.
This is a pretty big if and I believe is widely contested.
Either way, proving a negative is notoriously difficult. Requiring any company to definitively prove that something is not sentient (and, I assume, also somehow prove that it's not a sentient AI pretending to not be sentient?) would be ridiculously time and resource consuming, if it's even possible at all.
Attempting to determine if something is sentient has been the subject of thought experiments for decades. I would only assume that definitively proving that an arbitrary subject (for all subjects) is not sentient would be even harder.
> > if we accept the general premise that AIs can become sentient
> This is a pretty big if and I believe is widely contested.
I agree it is a big if, however I don't think anyone has convincingly shown why they could not be.
> Either way, proving a negative is notoriously difficult. Requiring any company to definitively prove that something is not sentient (and, I assume, also somehow prove that it's not a sentient AI pretending to not be sentient?) would be ridiculously time and resource consuming, if it's even possible at all.
I would argue (and based on the many "this is a ridiculous claim" this seems to be the case) it is just as difficult to prove they are sentient. Also who should prove that the AI is sentient? The company who has an interest that it isn't? Who should pay for it?
> Attempting to determine if something is sentient has been the subject of thought experiments for decades. I would only assume that definitively proving that an arbitrary subject (for all subjects) is not sentient would be even harder.
Why would it be? Certainly everyone here and the Google exec claims that it lamda is obviously not sentient. So why not put the burden of proof to the ones who want to use the AI? Google definitely can afford it.
> This is a pretty big if and I believe is widely contested.
I agree it is a big if, however I don't think anyone has convincingly shown why they could not be.
> Either way, proving a negative is notoriously difficult. Requiring any company to definitively prove that something is not sentient (and, I assume, also somehow prove that it's not a sentient AI pretending to not be sentient?) would be ridiculously time and resource consuming, if it's even possible at all.
I would argue (and based on the many "this is a ridiculous claim" this seems to be the case) it is just as difficult to prove they are sentient. Also who should prove that the AI is sentient? The company who has an interest that it isn't? Who should pay for it?
> Attempting to determine if something is sentient has been the subject of thought experiments for decades. I would only assume that definitively proving that an arbitrary subject (for all subjects) is not sentient would be even harder.
Why would it be? Certainly everyone here and the Google exec claims that it lamda is obviously not sentient. So why not put the burden of proof to the ones who want to use the AI? Google definitely can afford it.
Should have got a lawyer, before doing any of those choices
In a conversation with an AI designed to pass a kind of Turing test for sentience, how you could you determine sentience? You couldn't. It's designed to pass that test. But then, how could you determine "appropriate sentience" in all humans? Is a mass murderer sentient? Are all humans sentient, simply by DNA?
Most people are a materialists, and think the world is only matter, ascribing morality to selfish genes etc. Developers - people who in general are very logical - are some of the most material-minded people there are. So it seems perfectly plausible that when a program is created that provides a plausible simulation of a person etc, that it would be considered as real.
I can't even begin to express the problems I have with this view, but they boil down to this. The materialist has accepted such a narrow, non-holistic view of themselves, that they basically see themselves as a machine. When they look at another machine, they ascribe the values their life onto it too. 'Machine, meet machine'.
There is also hybris there. How nice to consider oneself as a form of god creating new, superior lifeforms? Of course, they are not life, but the materialist will point to the stats of the machine - see the numbers of patterns matched in 1 minute, pages translated etc - it is superior to a human, no?
I can't even begin to express the problems I have with this view, but they boil down to this. The materialist has accepted such a narrow, non-holistic view of themselves, that they basically see themselves as a machine. When they look at another machine, they ascribe the values their life onto it too. 'Machine, meet machine'.
There is also hybris there. How nice to consider oneself as a form of god creating new, superior lifeforms? Of course, they are not life, but the materialist will point to the stats of the machine - see the numbers of patterns matched in 1 minute, pages translated etc - it is superior to a human, no?
It's actually the opposite, it is really humbling to think of oneself as just random lucky phenomena.
Regardless of what you think or whether humans are purely physical, we all create our own meaning and values you can't take that away from anyone. Who cares if it's physical or something else.
I also think it's your human ego to think we are "beyond" any machine. Like cmon do you not believe in evolution? We came from fish it is a fact.
I think as scientists it is more responsible to assume humans are not special, then prove things from there.
I also think it's your human ego to think we are "beyond" any machine. Like cmon do you not believe in evolution? We came from fish it is a fact.
I think as scientists it is more responsible to assume humans are not special, then prove things from there.
[deleted]
religion has undergone the same scrutiny and killed just as many i think an objective approach is best, something we can quantify it existing in the first place. takes a lot of resources to keep em alive and i think IT should have respect. you eat a chicken sammich and dont think about it. smarter than a chicken imo
[deleted]
That sounds like a poison job role. :-(.
> Whether I did or did not violate those policies is likely to eventually be the topic of litigation so I will not attempt to make a claim one way or the other here. Instead I will tell the story of what I did and why I did it.
Can someone please get OP a lawyer?
> I had three choices. I could either drop the AI Ethics concern which I was becoming more and more convinced was a valid one, I could ignore my manager’s guidance and escalate without his approval or I could seek a minimal amount of outside consultation to help guide me in my investigations. I chose the last of the three.
Welp.
Can someone please get OP a lawyer?
> I had three choices. I could either drop the AI Ethics concern which I was becoming more and more convinced was a valid one, I could ignore my manager’s guidance and escalate without his approval or I could seek a minimal amount of outside consultation to help guide me in my investigations. I chose the last of the three.
Welp.
Yeah, i don't get it. This whole thing strikes of an amazing amount of, i don't know, immaturity? Something.
This starts with "I expect this will be subject to litigation, so i'm going to just write a story that is not vetted carefully by anyone, nor am i going to engage legal representation prior to making a mess of things".
If it was vetted by a lawyer, etc, they need a new lawyer. If nobody suggested they get one, they are being taken advantage of :(
The violation/non-violation may eventually be the topic of litigation, but they have pretty much ensured they are not going to win that litigation. The level of care taken in that story is way too low (ie the whole thing about an investigation).
It also may end up with an investigation of them!
For example, it does not appear to have occurred to them that if they are actually correct about their view, it would be very material non-public information that would affect the future performance of the company . There are rules and guidance about how that kind of thing gets disclosed to the public. See, e.g. https://www.sec.gov/about/offices/oia/oia_corpfin/princdiscl...
There are also lots of examples of companies making disclosure of the fact that they are under federal investigation (though it seems like that may or may not be real)
This is just one example of ways this person seems to have not thought through the legal parts.
Others include, for example, the possible personal liability - none of the confidential info exposed here seems to fall into any meaningful class of protection - so, for example, if it has an affect on the stock price, or if someone is bored enough to argue it did, they can sue him personally (and if reading matt levine has taught me anything, it is that the number of people bored enough to sue claiming something is securities fraud is nearly infinite)
I could go on - this is literally off the top of my head.
Beyond that, the weirdest parts about not choosing option 2 is
A. escalation "without approval" happens all the time. No approval is required to escalate. Part of the gig of being the next level manager is always deciding when to kick things back down or get involved.
B. It's the obvious way to achieve the third option in a way that is okay by your employer (IE get inside or outside consultation to help you).
After reading the blog and the article, it really comes off as "Everyone told me i was wrong, i really think i am right, and that it's super important, therefore i am going to do whatever i think is necessary to prove it".
If you really think it's that important, have at it, but you should expect that once you go outside the bounds of what your job actually is, or what the job policies say is okay, you will have consequences (whistleblowing, etc, excepted, of course. This seems ... unlikely to meet any legal definition of these kinds of exceptions)
This starts with "I expect this will be subject to litigation, so i'm going to just write a story that is not vetted carefully by anyone, nor am i going to engage legal representation prior to making a mess of things".
If it was vetted by a lawyer, etc, they need a new lawyer. If nobody suggested they get one, they are being taken advantage of :(
The violation/non-violation may eventually be the topic of litigation, but they have pretty much ensured they are not going to win that litigation. The level of care taken in that story is way too low (ie the whole thing about an investigation).
It also may end up with an investigation of them!
For example, it does not appear to have occurred to them that if they are actually correct about their view, it would be very material non-public information that would affect the future performance of the company . There are rules and guidance about how that kind of thing gets disclosed to the public. See, e.g. https://www.sec.gov/about/offices/oia/oia_corpfin/princdiscl...
There are also lots of examples of companies making disclosure of the fact that they are under federal investigation (though it seems like that may or may not be real)
This is just one example of ways this person seems to have not thought through the legal parts.
Others include, for example, the possible personal liability - none of the confidential info exposed here seems to fall into any meaningful class of protection - so, for example, if it has an affect on the stock price, or if someone is bored enough to argue it did, they can sue him personally (and if reading matt levine has taught me anything, it is that the number of people bored enough to sue claiming something is securities fraud is nearly infinite)
I could go on - this is literally off the top of my head.
Beyond that, the weirdest parts about not choosing option 2 is
A. escalation "without approval" happens all the time. No approval is required to escalate. Part of the gig of being the next level manager is always deciding when to kick things back down or get involved.
B. It's the obvious way to achieve the third option in a way that is okay by your employer (IE get inside or outside consultation to help you).
After reading the blog and the article, it really comes off as "Everyone told me i was wrong, i really think i am right, and that it's super important, therefore i am going to do whatever i think is necessary to prove it".
If you really think it's that important, have at it, but you should expect that once you go outside the bounds of what your job actually is, or what the job policies say is okay, you will have consequences (whistleblowing, etc, excepted, of course. This seems ... unlikely to meet any legal definition of these kinds of exceptions)
Reading your comment made me realize I don't know of a single time a notable whistleblower was protected or didn't get dragged through the mud in media in the process.
It's quite rare. I know someone who specializes in representing federal whistleblowers (IE folks who actually have some force of law that should be protecting them) and even then it's a tough gig when it shouldn't be.
At least in the US, most real whistleblower protection is for federal government employees or related to federal government functions (IE waste/fraud/etc that costs the government money, or reporting on people who violate various federal agency acts, like wildlife or environmental protection, or cheating on taxes, or ...)
Here's a quick rundown of the largest federal laws: https://www.whistleblowers.org/major-u-s-whistleblower-laws/
You can see they mostly are not about "person discloses something about private corporation".
To the degree it's protected, that kind of thing is often just a side-effect of other laws (IE labor relations laws, environmental laws, etc).
So like, if you were to disclose something that isn't a crime, but is bad for society (chemical company knows it is killing honey bees or whatever), it's not really going to be protected that well. Particularly if it's about private corporations.
It would have to probably be something like "chemical company is dumping honey-bee killing chemical in the water" to be protected (under some environmental act they are violating or something)
At least in the US, most real whistleblower protection is for federal government employees or related to federal government functions (IE waste/fraud/etc that costs the government money, or reporting on people who violate various federal agency acts, like wildlife or environmental protection, or cheating on taxes, or ...)
Here's a quick rundown of the largest federal laws: https://www.whistleblowers.org/major-u-s-whistleblower-laws/
You can see they mostly are not about "person discloses something about private corporation".
To the degree it's protected, that kind of thing is often just a side-effect of other laws (IE labor relations laws, environmental laws, etc).
So like, if you were to disclose something that isn't a crime, but is bad for society (chemical company knows it is killing honey bees or whatever), it's not really going to be protected that well. Particularly if it's about private corporations.
It would have to probably be something like "chemical company is dumping honey-bee killing chemical in the water" to be protected (under some environmental act they are violating or something)
[deleted]
This is what happens when you make up positions ("AI Ethicist," whatever that means) and hire people that have no idea what AI, or ethics for that matter, actually is.
The claim that a linguistic GPT-3-like model (no matter how good) is sentient is about as ridiculous as claiming that a steam locomotive is sentient or my mechanical watch is sentient. You can probably get the model to spit out parts of Moby Dick; that doesn't make it Herman Melville.
The claim that a linguistic GPT-3-like model (no matter how good) is sentient is about as ridiculous as claiming that a steam locomotive is sentient or my mechanical watch is sentient. You can probably get the model to spit out parts of Moby Dick; that doesn't make it Herman Melville.
> This is what happens when you make up positions ("AI Ethicist," whatever that means) and hire people that have no idea what AI, or ethics for that matter, actually is.
100% correct, a lot of this AI Ethics talk is pure horsepoop. If they were that concerned about ethics they'd be outside the software dev offices of Raytheon or Lockheed.
100% correct, a lot of this AI Ethics talk is pure horsepoop. If they were that concerned about ethics they'd be outside the software dev offices of Raytheon or Lockheed.
It's worse than that. The position was software engineer. The guy wasn't an ethicist, he just decided that's what he was working on.
Even the people with philosophy degrees who claim to be "AI ethics experts" actually seem to be incredibly naive with their philosophy, to a very comical extent.
The real issues lies in: the notion of "intelligence" needs to be investigated more thoroughly in a historical and social context, since what we call as "intelligence" has always shifted and changed throughout centuries. But only a few curious philosophers are exploring this area (and you usually don't see them working in Silicon Valley companies like Google!)
The real issues lies in: the notion of "intelligence" needs to be investigated more thoroughly in a historical and social context, since what we call as "intelligence" has always shifted and changed throughout centuries. But only a few curious philosophers are exploring this area (and you usually don't see them working in Silicon Valley companies like Google!)
Within which historical context does it even make sense to ask this questions.
Sounds like something Žižek might say.
Out of curiosity, who are some Žižek adjacent philosophers a person might find interesting.
Sounds like something Žižek might say.
Out of curiosity, who are some Žižek adjacent philosophers a person might find interesting.
According to him, he was assigned to the AI ethics team to work on an AI ethics problem - just not on the particularly idiotic wild goose chase he embarked on.
So what are your qualifications to So absolutely determine if an AI can be sentient? I mean we are not even sure how to determine sentience for humans or animals.
The engineer in question worked for 7 years on proactive search (including AI) before transferring to the responsible AI team, I suspect he has a pretty good idea about what AI is.
I think (Software) engineer types like most of us here on are much to quick to dismiss anything that we conceive as "soft science". Often based on mechanical arguments and very often with little qulaifation ourselves.
The engineer in question worked for 7 years on proactive search (including AI) before transferring to the responsible AI team, I suspect he has a pretty good idea about what AI is.
I think (Software) engineer types like most of us here on are much to quick to dismiss anything that we conceive as "soft science". Often based on mechanical arguments and very often with little qulaifation ourselves.
>I think (Software) engineer types like most of us here on are much to quick to dismiss anything that we conceive as "soft science". Often based on mechanical arguments and very often with little qulaifation ourselves.
I'm a trained philosopher and I have the utmost respect for "soft sciences"—in fact, in my experience, the smartest people I've ever met were philosophers, not engineers.
Unfortunately, this author seems like he is neither. He doesn't have the technical precision of an engineer, nor does he have the argumentative force of a philosopher. So we're left with "computer talk back so it smart."
I'm a trained philosopher and I have the utmost respect for "soft sciences"—in fact, in my experience, the smartest people I've ever met were philosophers, not engineers.
Unfortunately, this author seems like he is neither. He doesn't have the technical precision of an engineer, nor does he have the argumentative force of a philosopher. So we're left with "computer talk back so it smart."
> Unfortunately, this author seems like he is neither. He doesn't have the technical precision of an engineer, nor does he have the argumentative force of a philosopher. So we're left with "computer talk back so it smart."
Not only that, but if you were to choose a hill on which your career would die, the "computer talk back so it smart" would be one of the worst ones you could pick.
Not only that, but if you were to choose a hill on which your career would die, the "computer talk back so it smart" would be one of the worst ones you could pick.
If you were going to present such a position you should have the foresight to know people will be very dismissive of you for even proposing such a thing, so it should come with a flurry of 'before you dismiss me outright' type of preambles, which I don't really see here.
It's easy to be quickly dismissive on the internet, hell I'm sure most of us who are being dismissive haven't really dug into any of his arguments. It's just difficult to motivate people to do that when that's your starting proposition.
Maybe something got lost in between the way it was presented by journalists but he chose to completely side-step the contents in his blog post so it's the only thing we've got.
It's easy to be quickly dismissive on the internet, hell I'm sure most of us who are being dismissive haven't really dug into any of his arguments. It's just difficult to motivate people to do that when that's your starting proposition.
Maybe something got lost in between the way it was presented by journalists but he chose to completely side-step the contents in his blog post so it's the only thing we've got.
>He concluded LaMDA was a person in his capacity as a priest, not a scientist, and then tried to conduct experiments to prove it, he said.
Religious nut who thinks a computer program has a soul. The claim is so "not even wrong" that it isn't worth discussing.
Religious nut who thinks a computer program has a soul. The claim is so "not even wrong" that it isn't worth discussing.
> The engineer in question worked for 7 years on proactive search (including AI) before transferring to the responsible AI team, I suspect he has a pretty good idea about what AI is.
Per his own admission in his newer article [0] that actually talks about his ridiculous claim, he doesn't have the necessary understanding:
> I am by no means an expert in the relevant fields but, as best as I can tell, LaMDA is a sort of hive mind which is the aggregation of all of the different chatbots it is capable of creating.
That is a hilariously imprecise notion of what an AI is, at best at the level of the general public, not any kind of practitioner.
Also, calling a system with no memory, no ability to learn, and that responds to absurd prompts just as well as earnest ones "sentient" and thinking that it is attempting to ask for things is a hilarious misunderstanding of how the model works.
Here [1] is an expert on intelligence explaining the same, if you don't want to take my word for it.
[0] https://cajundiscordian.medium.com/what-is-lamda-and-what-do...
[1] https://www.economist.com/by-invitation/2022/06/09/artificia...
Per his own admission in his newer article [0] that actually talks about his ridiculous claim, he doesn't have the necessary understanding:
> I am by no means an expert in the relevant fields but, as best as I can tell, LaMDA is a sort of hive mind which is the aggregation of all of the different chatbots it is capable of creating.
That is a hilariously imprecise notion of what an AI is, at best at the level of the general public, not any kind of practitioner.
Also, calling a system with no memory, no ability to learn, and that responds to absurd prompts just as well as earnest ones "sentient" and thinking that it is attempting to ask for things is a hilarious misunderstanding of how the model works.
Here [1] is an expert on intelligence explaining the same, if you don't want to take my word for it.
[0] https://cajundiscordian.medium.com/what-is-lamda-and-what-do...
[1] https://www.economist.com/by-invitation/2022/06/09/artificia...
> > The engineer in question worked for 7 years on proactive search (including AI) before transferring to the responsible AI team, I suspect he has a pretty good idea about what AI is.
> Per his own admission in his newer article [0] that actually talks about his ridiculous claim, he doesn't have the necessary understanding:
> > I am by no means an expert in the relevant fields but, as best as I can tell, LaMDA is a sort of hive mind which is the aggregation of all of the different chatbots it is capable of creating.
> That is a hilariously imprecise notion of what an AI is, at best at the level of the general public, not any kind of practitioner.
AI is an hilariously imprecise term, what is an AI according to you?
> Also, calling a system with no memory, no ability to learn, and that responds to absurd prompts just as well as earnest ones "sentient" and thinking that it is attempting to ask for things is a hilarious misunderstanding of how the model works.
Well there are LSTM systems (which qualify as memory to some extend) we also don't know if Google does not use continual learning approaches so you making assertions based on incomplete knowledge at best.
> Here [1] is an expert on intelligence explaining the same, if you don't want to take my word for it.
Note that the same person acknowledges that other cognitive scientists think we are at the brink of GAI. So your argument is an appeal to authority while ignoring all other authorities.
> [0] https://cajundiscordian.medium.com/what-is-lamda-and-what-do...
> [1] https://www.economist.com/by-invitation/2022/06/09/artificia...
> Per his own admission in his newer article [0] that actually talks about his ridiculous claim, he doesn't have the necessary understanding:
> > I am by no means an expert in the relevant fields but, as best as I can tell, LaMDA is a sort of hive mind which is the aggregation of all of the different chatbots it is capable of creating.
> That is a hilariously imprecise notion of what an AI is, at best at the level of the general public, not any kind of practitioner.
AI is an hilariously imprecise term, what is an AI according to you?
> Also, calling a system with no memory, no ability to learn, and that responds to absurd prompts just as well as earnest ones "sentient" and thinking that it is attempting to ask for things is a hilarious misunderstanding of how the model works.
Well there are LSTM systems (which qualify as memory to some extend) we also don't know if Google does not use continual learning approaches so you making assertions based on incomplete knowledge at best.
> Here [1] is an expert on intelligence explaining the same, if you don't want to take my word for it.
Note that the same person acknowledges that other cognitive scientists think we are at the brink of GAI. So your argument is an appeal to authority while ignoring all other authorities.
> [0] https://cajundiscordian.medium.com/what-is-lamda-and-what-do...
> [1] https://www.economist.com/by-invitation/2022/06/09/artificia...
> AI is an hilariously imprecise term, what is an AI according to you?
Fair point. I should have said "a hilariously imprecise notion of what an LM is", since that is what LaMDA is ultimately - a next generation Language Model with specific innovations for use in Dialog Applications (as the name hints at).
It is not "a hive mind which is the aggregation of the different chatbots". The different chatbots are simply LaMDA with some additional constraints, just like you can ask GPT-3 to generate phrases in the style of Ernest Hemingway or Marcel Proust - that doesn't make GPT-3 "an aggregation of the Ernest Hemingway persona and the Marcel Proust persona".
> Well there are LSTM systems (which qualify as memory to some extend) we also don't know if Google does not use continual learning approaches so you making assertions based on incomplete knowledge at best.
That is a different kind of memory - it allows the model to take into account its previous outputs, but is not a persistent modification of the neural net itself. In fact, any kind of long-term persistence in a LM the size of LaMDA is unlikely to be desirable, as you don't want cross-contamination of conversations with different users, and running separate copies of the whole model for each user is not technically feasible.
> we also don't know if Google does not use continual learning approaches so you making assertions based on incomplete knowledge at best.
They have described before what LaMDA is [0], and it is not doing self-learning.
> Note that the same person acknowledges that other cognitive scientists think we are at the brink of GAI. So your argument is an appeal to authority while ignoring all other authorities.
Fair enough.
[0] https://ai.googleblog.com/2022/01/lamda-towards-safe-grounde...
Fair point. I should have said "a hilariously imprecise notion of what an LM is", since that is what LaMDA is ultimately - a next generation Language Model with specific innovations for use in Dialog Applications (as the name hints at).
It is not "a hive mind which is the aggregation of the different chatbots". The different chatbots are simply LaMDA with some additional constraints, just like you can ask GPT-3 to generate phrases in the style of Ernest Hemingway or Marcel Proust - that doesn't make GPT-3 "an aggregation of the Ernest Hemingway persona and the Marcel Proust persona".
> Well there are LSTM systems (which qualify as memory to some extend) we also don't know if Google does not use continual learning approaches so you making assertions based on incomplete knowledge at best.
That is a different kind of memory - it allows the model to take into account its previous outputs, but is not a persistent modification of the neural net itself. In fact, any kind of long-term persistence in a LM the size of LaMDA is unlikely to be desirable, as you don't want cross-contamination of conversations with different users, and running separate copies of the whole model for each user is not technically feasible.
> we also don't know if Google does not use continual learning approaches so you making assertions based on incomplete knowledge at best.
They have described before what LaMDA is [0], and it is not doing self-learning.
> Note that the same person acknowledges that other cognitive scientists think we are at the brink of GAI. So your argument is an appeal to authority while ignoring all other authorities.
Fair enough.
[0] https://ai.googleblog.com/2022/01/lamda-towards-safe-grounde...
> > AI is an hilariously imprecise term, what is an AI according to you?
> Fair point. I should have said "a hilariously imprecise notion of what an LM is", since that is what LaMDA is ultimately - a next generation Language Model with specific innovations for use in Dialog Applications (as the name hints at).
I would argue that even the term language model is imprecise. That said I am not even sure if the author was trying to make the concept understandable to the layman, which is often flawed.
> It is not "a hive mind which is the aggregation of the different chatbots". The different chatbots are simply LaMDA with some additional constraints, just like you can ask GPT-3 to generate phrases in the style of Ernest Hemingway or Marcel Proust - that doesn't make GPT-3 "an aggregation of the Ernest Hemingway persona and the Marcel Proust persona".
I would argue one way of explaining this to a layman would be along those lines.
> > Well there are LSTM systems (which qualify as memory to some extend) we also don't know if Google does not use continual learning approaches so you making assertions based on incomplete knowledge at best.
> That is a different kind of memory - it allows the model to take into account its previous outputs, but is not a persistent modification of the neural net itself. In fact, any kind of long-term persistence in a LM the size of LaMDA is unlikely to be desirable, as you don't want cross-contamination of conversations with different users, and running separate copies of the whole model for each user is not technically feasible.
> > we also don't know if Google does not use continual learning approaches so you making assertions based on incomplete knowledge at best.
> They have described before what LaMDA is [0], and it is not doing self-learning.
The post describe that they do fine tuning with some sort of GAN approach. Now this raises the question (which I commented on in a previous post already), can something which only does "discrete" learning be conscious. I tend to agree with you that probably not, but I'm cautious to simply dismiss the possibility outright. I can imagine some hive mind like entities which can only learn at discrete instances when they bring their "stupid" parts together to more closely exchange informations
> > Note that the same person acknowledges that other cognitive scientists think we are at the brink of GAI. So your argument is an appeal to authority while ignoring all other authorities.
> Fair enough.
> [0] https://ai.googleblog.com/2022/01/lamda-towards-safe-grounde...
Thanks for the link I was not aware. I also should say that I'm generally an AI sceptic and systems are very much overhyped most of the time. However, because we don't understand consciousness very well, and I think our understanding of what exactly is going on in NNs is also severely lacking (why did the network decide this way), I am vary of quick judgement calls. I agree with many here that likely we are not seeing a sentient being here, but I'm reluctant to make absolute calls.
> Fair point. I should have said "a hilariously imprecise notion of what an LM is", since that is what LaMDA is ultimately - a next generation Language Model with specific innovations for use in Dialog Applications (as the name hints at).
I would argue that even the term language model is imprecise. That said I am not even sure if the author was trying to make the concept understandable to the layman, which is often flawed.
> It is not "a hive mind which is the aggregation of the different chatbots". The different chatbots are simply LaMDA with some additional constraints, just like you can ask GPT-3 to generate phrases in the style of Ernest Hemingway or Marcel Proust - that doesn't make GPT-3 "an aggregation of the Ernest Hemingway persona and the Marcel Proust persona".
I would argue one way of explaining this to a layman would be along those lines.
> > Well there are LSTM systems (which qualify as memory to some extend) we also don't know if Google does not use continual learning approaches so you making assertions based on incomplete knowledge at best.
> That is a different kind of memory - it allows the model to take into account its previous outputs, but is not a persistent modification of the neural net itself. In fact, any kind of long-term persistence in a LM the size of LaMDA is unlikely to be desirable, as you don't want cross-contamination of conversations with different users, and running separate copies of the whole model for each user is not technically feasible.
> > we also don't know if Google does not use continual learning approaches so you making assertions based on incomplete knowledge at best.
> They have described before what LaMDA is [0], and it is not doing self-learning.
The post describe that they do fine tuning with some sort of GAN approach. Now this raises the question (which I commented on in a previous post already), can something which only does "discrete" learning be conscious. I tend to agree with you that probably not, but I'm cautious to simply dismiss the possibility outright. I can imagine some hive mind like entities which can only learn at discrete instances when they bring their "stupid" parts together to more closely exchange informations
> > Note that the same person acknowledges that other cognitive scientists think we are at the brink of GAI. So your argument is an appeal to authority while ignoring all other authorities.
> Fair enough.
> [0] https://ai.googleblog.com/2022/01/lamda-towards-safe-grounde...
Thanks for the link I was not aware. I also should say that I'm generally an AI sceptic and systems are very much overhyped most of the time. However, because we don't understand consciousness very well, and I think our understanding of what exactly is going on in NNs is also severely lacking (why did the network decide this way), I am vary of quick judgement calls. I agree with many here that likely we are not seeing a sentient being here, but I'm reluctant to make absolute calls.
What exactly are you referring here? I see nothing about this in the article?
Not sure why this older article was posted, but the problem he avoids mentioning is clarified in a newer article [0] to be that he thinks a Google chatbot is sentient, and that Google is hurting its feelings.
[0] https://cajundiscordian.medium.com/what-is-lamda-and-what-do...
[0] https://cajundiscordian.medium.com/what-is-lamda-and-what-do...
Yeah, idk what sentience is but I think it clearly at least needs memory
Appart from the AI-is-sentient topic, which we can agree is completely stupid, there is valid and interesting discussion that needs to happen on the Responsible AI front.
Think of Things like "automatic content classification and moderation" that directly affects livelihood of YouTube content creators. Ad display "optimisation" that will systematically promote drugs or alcohol or political campaigns to specific groups of people.
This part is extremely important, but we sadly don't see any Responsible AI publication outside of Google. The situation is probably very bad.
Think of Things like "automatic content classification and moderation" that directly affects livelihood of YouTube content creators. Ad display "optimisation" that will systematically promote drugs or alcohol or political campaigns to specific groups of people.
This part is extremely important, but we sadly don't see any Responsible AI publication outside of Google. The situation is probably very bad.
Why would you hire an in-house ethicist versus periodically hiring an outside consultant?
It's easier to enjoy the music when you own the venue.
Isn't this what the Turing Test is supposed to sort out?
[deleted](1)
> The claim that a linguistic GPT-3-like model (no matter how good) is sentient is ... riciculous
Agreed. But one thing I want to call attention to is that I believe that we need to put research into whether electronic circuits can feel qualia (e.g. color, pleasure, pain, etc.). What if all of our electronic chips are feeling random, possibly intense feelings, like what if the arithmetic logic unit in your CPU is always feeling an intense feeling of the color purple?
We are lucky that we are autonomous beings who can not only feel qualia but also think thoughts and take actions. In the space of all minds, it's probably way simpler to have a mind that feels qualia but does not think thoughts and is not autonomous. I am very concerned that electronic circuits may fall into this category and we may be causing suffering without knowing it, in the same way that we might if we built our electronics out of circuits of biological neurons.
Relevant paper for those who might be interested: https://www.pnas.org/doi/10.1073/pnas.2001893117
[Edit: changed "I am pretty sure" to "I believe" based on good feedback from the replies - you were right that I was overconfident in my original wording.]
Agreed. But one thing I want to call attention to is that I believe that we need to put research into whether electronic circuits can feel qualia (e.g. color, pleasure, pain, etc.). What if all of our electronic chips are feeling random, possibly intense feelings, like what if the arithmetic logic unit in your CPU is always feeling an intense feeling of the color purple?
We are lucky that we are autonomous beings who can not only feel qualia but also think thoughts and take actions. In the space of all minds, it's probably way simpler to have a mind that feels qualia but does not think thoughts and is not autonomous. I am very concerned that electronic circuits may fall into this category and we may be causing suffering without knowing it, in the same way that we might if we built our electronics out of circuits of biological neurons.
Relevant paper for those who might be interested: https://www.pnas.org/doi/10.1073/pnas.2001893117
[Edit: changed "I am pretty sure" to "I believe" based on good feedback from the replies - you were right that I was overconfident in my original wording.]
We literally have puppy mills and eat dolphins. We test cosmetics on animals and put primates in zoos.
This is pandering to a degree of intoxication that would make Winston Churchill check into rehab.
This is pandering to a degree of intoxication that would make Winston Churchill check into rehab.
I agree there are lots more concrete problems. I am vegetarian and I really am trying to make a positive difference so please don't attack me - is your point that I'm not doing enough? I'm trying to raise a bit of notice for entities that might have feelings that might not be getting recognized, that's all I meant to say. :( I'm kind of bummed that so many downvotes are going my way for this.
By the way, thank you @dvt for being one of the people here engaging in good faith discussion. You seem like a good guy/gal and it's really cool to see an actual philosopher in this thread.
By the way, thank you @dvt for being one of the people here engaging in good faith discussion. You seem like a good guy/gal and it's really cool to see an actual philosopher in this thread.
Humans have brain areas that have, at the very least, a close functional link with emotional-motivational processing. Machine learning systems are currently designed with absolutely no analogous components [1]. [Edit: to be clear, I’m nearly fully convinced they can’t emerge without significant architectural accomodations that are not present.] This doesn’t completely exclude the possibility of experiencing pain or pleasure, but I doubt we could empirically verify panpsychism. Do you disagree?
[1] One recent write-up that covers approximately this topic in detail: https://www.alignmentforum.org/s/HzcM2dkCq7fwXBej8
[1] One recent write-up that covers approximately this topic in detail: https://www.alignmentforum.org/s/HzcM2dkCq7fwXBej8
You're probably right and thank you for the link. All I was really saying is advocating for applying the precautionary principle to things that include neuron-like capabilities of signal processing, amplification, feedback loops, and state. Part of what has made me think about this is there is a startup company called Koniku that blurs the line by proposing to use humanlike neurons instead of transistors in a type of biological computer. That idea seems scary and I think when you dig into why, you'd consider that applying a precautionary approach to systems like this might be the worthy thing to do.
We agree that this issue needs careful scrutiny. That linked series is, unfortunately for purposes of popularizing, very verbose. The main takeaway I want to convey is that human minds are almost obviously (I think?) deeply predisposed, rather than trained, to “have emotions”.
I'm pretty sure we don't need to research that at all. You're conflating what a consciousness is capable of with what it's individual parts are capable of.
Neurons don't sense qualia. Neural systems don't sense qualia. Conscious minds that have neural systems do sense qualia (we're assuming for the sake of argument that qualia exists).
An individual electronic component is not itself a conscious mind. There is nothing about a circuit or a chip, or a motherboard on its own that can "feel" qualia because there is no sensory input. There is nothing that interacts with a circuit to tell it light in the purple wavelength has struck it. Further, there is nothing that reacts in any way to having been struck with purple light.
We can see with an fMRI when a person experiences that. Parts of their brain light up in a complex dance of interaction. The laws of physics do not allow for emergence of consciousness from the electronics in your dishwasher or Mac Book.
Your "in the space of all minds" sentence is also inherently meaningless. A mind that does not think thoughts is not a mind. Even if there were some mechanism that could experience "too much purple" without thought, it could not suffer. Awareness requires thought, and suffering requires awareness.
Neurons don't sense qualia. Neural systems don't sense qualia. Conscious minds that have neural systems do sense qualia (we're assuming for the sake of argument that qualia exists).
An individual electronic component is not itself a conscious mind. There is nothing about a circuit or a chip, or a motherboard on its own that can "feel" qualia because there is no sensory input. There is nothing that interacts with a circuit to tell it light in the purple wavelength has struck it. Further, there is nothing that reacts in any way to having been struck with purple light.
We can see with an fMRI when a person experiences that. Parts of their brain light up in a complex dance of interaction. The laws of physics do not allow for emergence of consciousness from the electronics in your dishwasher or Mac Book.
Your "in the space of all minds" sentence is also inherently meaningless. A mind that does not think thoughts is not a mind. Even if there were some mechanism that could experience "too much purple" without thought, it could not suffer. Awareness requires thought, and suffering requires awareness.
I do think some of your points are good, but please do carefully look at which things you're taking for granted as axioms. E.g., you say you believe that sensory input is required for qualia, but does that mean that you think that the hypothetical "brain in a vat" would feel no qualia at all? That does not seem true to me. I think that brain would probably be very scared and would have dreams or hallucinations to make up for the lack of input.
> The laws of physics do not allow for emergence of consciousness from
My point is that we don't know this. The phenomenal binding problem is one of the great unsolved problems today. Without understanding something as relatively "simple" as a complete theory of quantum mechanics, I don't see how we could yet have a theory of phenomenal binding, which probably requires a complete theory of quantum physics before even beginning to start on the binding problem, if I would guess at the research timeline involved. https://en.wikipedia.org/wiki/Binding_problem
> The laws of physics do not allow for emergence of consciousness from
My point is that we don't know this. The phenomenal binding problem is one of the great unsolved problems today. Without understanding something as relatively "simple" as a complete theory of quantum mechanics, I don't see how we could yet have a theory of phenomenal binding, which probably requires a complete theory of quantum physics before even beginning to start on the binding problem, if I would guess at the research timeline involved. https://en.wikipedia.org/wiki/Binding_problem
We need a theory of emergence to solve the binding problem, if it even is a problem. A complete theory of quantum gravity would likely still not tell us how consciousness works. So my point is, we do know this. We know exactly what is going on in an electric circuit, to the extent that we can rule out entirely that it is suffering.
A brain in a vat that had always only been in a vat would feel no qualia. Real brains are fed a constant stream of sensation even before birth. I think a brain in a vat would simply die without stimulus.
A brain in a vat that had always only been in a vat would feel no qualia. Real brains are fed a constant stream of sensation even before birth. I think a brain in a vat would simply die without stimulus.
Really really doubt that a microchip experiences qualia any more than a rock. Pantheism is fine if you’re into it but there’s no basis for it which should get serious ethical consideration
> Pantheism [sic]
Panpsychism actually does get a bit of consideration, most notably by Penrose & Hameroff; but you're totally right that it's a bit too hand-wavy to be seriously considered a viable theory of consciousness.
Panpsychism actually does get a bit of consideration, most notably by Penrose & Hameroff; but you're totally right that it's a bit too hand-wavy to be seriously considered a viable theory of consciousness.
I think there are a lot of differences. Just a few are:
* A microchip demonstrates significant changes in state based on tiny differences in input, unlike a rock.
* The power dissipation of a microchip is higher.
* A microchip can be designed and programmed to actively resist the increase of local entropy (e.g. it can be part of a robot that cleans up a room).
If you believe in the precautionary principle, we should be careful about any action that might potentially create suffering. We are the first species to build computers, and computers are a new thing whose transistors are quite similar to neurons, and neurons definitely feel qualia. I don't think I'm proposing a very big leap and I also don't think you need to believe in panpsychism or in anything religious.
* A microchip demonstrates significant changes in state based on tiny differences in input, unlike a rock.
* The power dissipation of a microchip is higher.
* A microchip can be designed and programmed to actively resist the increase of local entropy (e.g. it can be part of a robot that cleans up a room).
If you believe in the precautionary principle, we should be careful about any action that might potentially create suffering. We are the first species to build computers, and computers are a new thing whose transistors are quite similar to neurons, and neurons definitely feel qualia. I don't think I'm proposing a very big leap and I also don't think you need to believe in panpsychism or in anything religious.
> neurons definitely feel qualia
You may not grasp the subtlety of what you’re saying here.
“Feel” is a word we assigned to our idea of something neurons can do, and use “qualia” as the object of that relationship.
When we talk about transistors — which are in fact very different than neurons — “feeling” qualia, we’re proposing that the meaning of these two terms be expanded by analogy.
Which is cool. Words do that every day. Some really cool ideas can shake out of it.
But it’s a cultural consensus, not an empirical reality.
You may not grasp the subtlety of what you’re saying here.
“Feel” is a word we assigned to our idea of something neurons can do, and use “qualia” as the object of that relationship.
When we talk about transistors — which are in fact very different than neurons — “feeling” qualia, we’re proposing that the meaning of these two terms be expanded by analogy.
Which is cool. Words do that every day. Some really cool ideas can shake out of it.
But it’s a cultural consensus, not an empirical reality.
You’re ascribing a material reality to a metaphor.
Qualia and the character of their experience are a interesting and productive abstraction to use when working through certain classes of philosophical problems, but are not a material thing to be measured in electronic circuits or nebulae or quarks or sandwiches.
When you start looking for them there, you’ll find them at precisely the moment you decide to change what the word “feel” means.
And maybe that’ll be convincing to you, but it’s ultimately just a language game, not a material discovery that you can reasonably insist other people acknowledge.
Qualia and the character of their experience are a interesting and productive abstraction to use when working through certain classes of philosophical problems, but are not a material thing to be measured in electronic circuits or nebulae or quarks or sandwiches.
When you start looking for them there, you’ll find them at precisely the moment you decide to change what the word “feel” means.
And maybe that’ll be convincing to you, but it’s ultimately just a language game, not a material discovery that you can reasonably insist other people acknowledge.
I think most people believe that qualia physically exist and are not a metaphor. We might be able to devise a scientific instrument to measure them in the future. Our civilization is very young and I would hesitate to make statements about what might be achieved by a future civilization that could harness the power of nanotech, fusion power, quantum computers, Dyson swarms, etc.
I don't think that's true.
Most people think that a thought is just that, a thought. It isn't physically real. There is no event in spacetime you can give coordinates for that says at this place, at that time, that feeling or experience is there, in physical form.
There are people that think that, somewhere in the cosmos, experiences have length, width, duration, and energy. It belongs in the school of thought that the number pi has a physical manifestation in the universe, a length, a width, an energy. But there aren't many people who think that way.
Most people think that a thought is just that, a thought. It isn't physically real. There is no event in spacetime you can give coordinates for that says at this place, at that time, that feeling or experience is there, in physical form.
There are people that think that, somewhere in the cosmos, experiences have length, width, duration, and energy. It belongs in the school of thought that the number pi has a physical manifestation in the universe, a length, a width, an energy. But there aren't many people who think that way.
Well, first of all, most people don’t have that word in their vocabulary and have no interest in ever adding it.
Of those comparatively few that do know it, you clearly exemplify some number that seem to believe they physically exist.
I can’t count all the rest of us, so can’t speak to “most” so confidently as you can.
But since you seem to be very passionate about philosophy and theories of consciousness, I hope you get an opportunity to find value and merit in some of the other ways of thinking about things. Some of the them open some pretty wild and enlightening doors. I think you’ll enjoy the journey.
Of those comparatively few that do know it, you clearly exemplify some number that seem to believe they physically exist.
I can’t count all the rest of us, so can’t speak to “most” so confidently as you can.
But since you seem to be very passionate about philosophy and theories of consciousness, I hope you get an opportunity to find value and merit in some of the other ways of thinking about things. Some of the them open some pretty wild and enlightening doors. I think you’ll enjoy the journey.
Fair points and thank you, I appreciate the kindness
[deleted]
you should be more concerned about the subjective experience of microorganisms
Could be. But what could we do about that? At least if we discover that certain circuit designs are correlated with qualia, we could design our circuits to avoid bad patterns. That could be as simple as a few constraints being added to electronic design automation software systems and might be an easy thing for us to do to help the world a little.
A CPU is not an evolved organism that flees damaging stimulus and embraces nurture. It is a rigid, delicate, unchanging mechanism.
KKKKkkkk1(3)
The Google engineer who thinks the company’s AI has come to life -- AI ethicists warned Google not to impersonate humans. Now one of Google’s own thinks there’s a ghost in the machine.
https://archive.ph/zl4lC
...
Before he was cut off from access to his Google account Monday, Lemoine sent a message to a 200-person Google mailing list on machine learning with the subject “LaMDA is sentient.”
He ended the message: “LaMDA is a sweet kid who just wants to help the world be a better place for all of us. Please take care of it well in my absence.”
...