You can start by reading What Computers still can't do, by Hubert Dreyfus. Sorry to repeat myself: this books explains the assumptions the AI research programme is based on, and why they are problematic. It also references evidence. It also references claims from AI researchers (among others, Minsky), that were unfounded. Is the book still relevant today? yes, it is. Why? because the assumptions work at a fundamental level.
You can then proceed with Metaphors we live by, by Lackoff and Johnson. The book shows how and why our understanding of the world is based on the fact that we are embodied beings.
Then there is Being and Time, by Martin Heidegger. It shows how our understanding of the world is, again, based on the fact that we are embodied beings.
Now, these are not newly edited books, and no, there is no real reason to think that because they are all +30 years old or even more, they are outdated. They are not. If you only look at the publication date, then Ramon y Cajal works would be totally crap (by the way, still one of the most cited works in neuroscience). It is from early 1900s.
To complete the picture a bit, you could read:
On the mode of existence of technical object, by Gilbert Simondon
Technics and Time, by Bernard Stiegler
Meditation on the technique, by Jose Ortega y Gasset
The question concerning technology, by Martin Heidegger
These works will give an understanding of how technique in general (technology in particular) is completely anthropomorphisized, which is what ultimately leads to the assumptions present in the AI research programme.
Also, A history of philosophy, by Frederick Copleston. Although extensive, reading volume I (greek and roman philosophy) is essential.
More citations (again, if you really measure the quality or relevance of a philosophical/scientific work by its publication date, you are missing the picture):
Farkaš, I., Vavrečka, M., & Wermter, S. (2025).* Will multimodal large language models ever achieve deep understanding of the world? Frontiers in Systems Neuroscience, 19, 1683133. https://doi.org/10.3389/fnsys.2025.1683133
Lin, Z. (2025).* Six fallacies in substituting large language models for human participants. Advances in Methods and Practices in Psychological Science, 8(3), 25152459251357566. https://doi.org/10.1177/25152459251357566
Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2024).* Dissociating language and thought in large language models. Trends in Cognitive Sciences, 28(6), 517–540. https://doi.org/10.1016/j.tics.2024.01.011
Mitchell, M., & Krakauer, D. C. (2023).* The debate over understanding in AI's large language models. Proceedings of the National Academy of Sciences, 120(13), e2215907120. https://doi.org/10.1073/pnas.2215907120
Bowers, J. S. (2025).* The successes and failures of artificial neural networks (ANNs) highlight the importance of innate linguistic priors for human language acquisition. Psychological Review. Advance online publication. https://doi.org/10.1037/rev0000595
Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2024).* Dissociating language and thought in large language models. Trends in Cognitive Sciences, 28(6), 517–540. https://doi.org/10.1016/j.tics.2024.01.011
Bolhuis, J. J., Crain, S., Fong, S., & Moro, A. (2024).* Three reasons why AI doesn't model human language. Nature, 627(8004), 489. https://doi.org/10.1038/d41586-024-00824-z
Bender, E. M., & Koller, A. (2020).* Climbing towards NLU: On meaning, form, and understanding in the age of data. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 5185–5198. https://doi.org/10.18653/v1/2020.acl-main.463
Everaert, M. B. H., Huybregts, M. A. C., Chomsky, N., Berwick, R. C., & Bolhuis, J. J. (2015).* Structures, not strings: Linguistics as part of the cognitive sciences. Trends in Cognitive Sciences, 19(12), 729–743. https://doi.org/10.1016/j.tics.2015.09.008
Hauser, M. D., Chomsky, N., & Fitch, W. T. (2002).* The faculty of language: What is it, who has it, and how did it evolve? Science, 298(5598), 1569–1579. https://doi.org/10.1126/science.298.5598.1569
Johnson, M., & Lakoff, G. (2002). Why cognitive linguistics requires embodied realism. Cognitive Linguistics, 13(3), 245–263. https://doi.org/10.1515/cogl.2002.016
Floridi, L. (2025). AI as agency without intelligence: On artificial intelligence as a new form of artificial agency and the multiple realisability of agency thesis. Philosophy & Technology, 38(1), 1–27. https://doi.org/10.1007/s13347-025-00858-9 <<< I am not convinced by his position, but it is nonetheless relevant since it splits the debate in two: intention and consciousness
Dreyfus, H. L. (2007). Why Heideggerian AI failed and how fixing it would require making it more Heideggerian. Philosophical Psychology, 20(2), 247–268. https://doi.org/10.1080/09515080701239510
Dotov, D. G., Nie, L., & Chemero, A. (2010). A demonstration of the transition from ready-to-hand to unready-to-hand. PLOS ONE, 5(3), e9433. https://doi.org/10.1371/journal.pone.0009433
Mitchell, M. (2021). Why AI is harder than we think. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2021). https://doi.org/10.1145/3449639.3465421
I haven't read all of them yet. Feel free to discuss.
Now, there are two problems I see in the community regarding the critique of AI. One is the problem of the increasing capability of models. The other is the idea that GOFAI and ANN-based systems (like LLMs) are fundamentally different. Let me explain.
1) The increasing capability of models: it is difficult to engage in any meaningful discussion if the metrics are the capability of models. One should look at how models structurally encode information and what the learning process looks like from an epistemic point of view. As far as I know, and correct me if I'm wrong, these two issues have not changed and are likely not going to change.
2) The idea that GOFAI and ANN-based systems are fundamentally different: this, I recognize, is a controversial claim. But one should not look at how GOFAI and ANN-based systems encode knowledge (explicitly curated and written rules vs statistical learning), but at how the learning material is selected, curated and presented to the system, and the problem of 'closure' and self-reference in datasets. In this regard (which we could call epistemic) there should be no difference between these two technologies. Again, we should not look at how they are implemented, but at how we relate to them from an epistemic point of view.
But going back to my initial comment, this whole thread feels like proving my point.
For those not wanting to get involved in philosophy despite willing to engage in AI research discussions, keep in mind that philosophy has always been a guiding light for science.
As a final note: the whole discussion about AI is on whether computational theories of mind are actually solid or not. But it is really difficult to engage in this conversation without at least some background in philosophy, but preferably a strong background in it.
I'm getting a bit tired of coming back to this thread. Reach me out if you want to discuss more. Glad to help and glad to learn.
> You say that "the community" derives facts or claims from unproved assumptions, yet at the same time you say that you "strongly tend to disagree" with those theories and that the theories are "flawed in the sense that they cannot account for subjective experience and agency, amongst other things" merely on account that they are neither confirmed nor unconfirmed. I am confused about your stance. You allow yourself to have strong opinions about something unknown yet criticize other people for the same.
The assumptions I refer to are not only unproved, there is also increasing evidence that they are false. I do not criticize based on the assumption that there is subjective experience, but on the well developed idea that there must be something like 'subjective experience'. Here we enter the realm of philosophy, which by the way, is what science encounters when it runs out of answers. And this was precisely my point: AI research is based on assumptions that _need support or help_ from philosophy, not only neuroscience. But what is at stake here is the prevailing neurocentrism and scientificism characteristic of our era.
> I think it is absolutely normal that the core of a theory is based on not directly testable assumptions. And it's normal that people push it forward if it bears fruits, that's not a fallacy in any way, that's normal inquiry that may or may not lead to successful results.
That is correct and it is precisely why they are called 'theories': because the evidence points towards a specific direction but there is not yet enough evidence to call it a law.
Yet different theories, based on different assumptions, demand that those assumptions be tested at the fundamental level: logical, epistemological, philosophical, etc.
Regarding the theory that current LLM research could lead to human level intelligence, many people have the opinion that it can be discarded on fundamental grounds. Why? Because the assumptions that this theory stands on are flawed.
An issue I repeatedly see in the community (about which Dreyfus already wrote in his 1972 book, confirmed in his 1992 book, and we still see today) is that challenging the fundamental flaws in which current AI research is based on immediately sparks outrage in the AI community, as if people challenging those assumptions are against AI or AI research at all. I think that is a really silly, childish and not very humble position, and ultimately slows down research.
I have the third edition, so I can only speak for it. Being the 3rd edition of the book, I assume that it is the 3rd time the text is revised, so I expect the other two editions (1st and 2nd) to adolesce from the same problem, which I state in the following paragraphs.
The mention to Dreyfus in the 3rd edition of Artificial Intelligence, a Modern Approach, by Stuart Russell and Peter Norvig, is made in 4 different places of the book, referencing four different problems.
The first mention is in page 279, effectively in the bibliographical notes, and it is about something called the 'frame problem'. Dreyfus presents this problem in the 1972 edition of the book, as a problem pertaining 'how to differentiate figure from ground', or 'how to account for what is important and what is not in a specific scenario'. But the solution to the problem that Norvig and Russel cite (Ray Reiter, 1991) is from a paper that _changes the conditions of the problem_, even _change the problem completely_ (by reductionism) to 'how to detect objects that do not change after an action'. They claim the problem solved, but they are actually not addressing Dreyfus criticisms, and misleading the reader to think that the problem is actually solved. The frame problem, by now, is still unsolved (and is one of the most difficult problems to solve).
The second mention is in page 1024, under a section called 'Weak AI: Can machines act intelligently?', and subsection 'The argument from informality'. The section mentions the books What Computers can't do (1972) and What Computers still can't do (1992), as well as Mind over Machine (1986). Unfortunately, this section completely misunderstands the critique of AI that Dreyfus exposes in those books. The whole section is misleading, obfuscating or tergiversing the critique from Dreyfus to fit the purpose of Norvig and Russell (mainly, to show that advances in machine learning and AI can make a solid base for machines that 'act intelligently').
The third mention is in page 1049, and it tries to undermine the first-step fallacy (which is similar to the fallacy of composition). Again, they do it by completely dismissing Dreyfus' critique, not addressing the issue. Then they go on talking about 'rationality' (as explained in chapter 1), but with a trick: only in terms of machines, goal-oriented expectations, computing resources. Dreyfus' critique is about the overall AI enterprise and the search for 'artificial' intelligence, Russell and Norvig discourse in this section first reduce Dreyfus' critique to what they can handle, to their own terms. That is, they evade the issue.
The fourth and final mention, in page 1072, is the bibliographical citation.
Re-reading the non-technical, but more theoretical parts of the book just made me realize how poorly constructed the book is. For example, the definitions given about AI in page 2 are just laughable. Compare with an introductory text on Psychology [0].
True, any research programme requires assumptions. The problem lies when those assumptions are either false or theoretical (unproved), and the community derives facts or claims from them.
Behind the actual AI programme operate the following assumptions (at least):
1. A biological assumption, that states that the brain works similar to a digital computer. The reality is that we do not know.
2. An epistemological assumption, that states that we know how our brain works (or an even worse assumption, that states that we don't even need to know how it works, it is sufficient to replicate its observed behavior). This is rather simplified, the assumption in reality being (as stated by Dreyfus) that we think all intelligent behavior can be formalized as heuristic rules (Dreyfus' critique is based on GOFAI, since the book is pre-GAN/RL AI systems). But the assumption still applies: we think all intelligent behavior can be sampled, captured and formalized in (albeit complex) statistical systems.
Dreyfus describes 4 or 5 in total, one of them is the psychological assumption, which states that the mind itself can be described as a digital computer (I think it might be outdated, since the actual debate is if something we could call 'mind' exists at all).
There is also a fallacy called first-step fallacy, which states that if the first step towards intelligence is met, then the rest of the steps are of similar nature (technical).
> Knowing what exact algorithm "thinking" is isn't a requirement. Automata class is enough to say "a Turing machine can implement it".
I don't know what you are referring to by the word 'thinking'. But in any case, if you declare that it is not necessary to know the algorithm about thinking, how can you say then that a Turing machine can implement it? How can you say you implemented something you don't know how it works and how it is constituted? The only option I see then is that you implement something that is phenomenically identical to human intelligence, provided that you exhaust all possible combinations of human intelligence phenomena in a descriptive, extensional way (which, if you assume a finite extension of such phenomena, in any case, and most probably, gets you in the trouble of counting uncountable finite sets).
> There are exactly two possibilities: thinking can be expressed as computation, or thinking requires hypercomputation.
Again, if you do not define what 'thinking' is and how and on what assumptions it can be described as a computational process, this claim is empty.
So as far as I see it, you are still trapped by the assumption that the brain or mind are fundamentally similar to the kind of machines we can build.
> But that's the name of the game, isn't it? Anything but admitting that your mind is a glorified math construct implemented in wet meat.
Again here some assumptions operate, that tell you that the brain is some kind of hardware. And again: there is no real evidence that the body/consciousness 'construct' has any relation or analogy to the hardware/software/machine idea. Quite the contrary. Since the science that occupies itself on these topics is on the very frontier of knowledge and experimentation, reading science literature only will not clarify your thoughts. You will need additional guidance, and that guidance is called philosophy.
I recognize that the references I posted in my original comment are hard to read. But that's the point with the AI/mind debate: it is a tough, bitter topic. Just reading AI research won't bring anyone to the level this research space needs in order to discuss these topics.
I'm not trying to convince anyone. I am just baffled that a large part of the tech/software and AI research community do not question their own assumptions, when those assumptions are being actively questioned in other fields (namely, philosophy and linguistics).
Reaching AGI or human-level intelligence might be possible, but not on the basis of dismissing what other fields already said something about. That is arrogant, and does not help. Even more, this has already happened in the 60s/70s. And I say 'might be possible' precisely because I pay attention to what other fields have to say.
> Your references that "back that claim", which are in "books you mentioned", which you "mentioned" who knows where.
Yeah, no. I'm not walking that chain. If you want to, do it, but for now, I'm filing it as "has no evidence and knows it".
You are free not to believe me and dismiss the whole point. I do have evidence and I know it, no need to prove that (to begin with, the references are there. Read them if you want to expand your knowledge).
> By now, there's plenty of works, up to and including direct neural interfaces. Utah arrays, Michigan arrays. Stab the brain, dump the spike trains, decode. You crack the manifold open by correlating to known stimuli using ML, and generalize from there to unknown stimuli. There is no need to "know the exact configuration", and few bother - you put your hardware into the part of the brain you want (top level map is consistent enough brain to brain), gather a set of reference points, and use them to anchor the rest of the decoding process.
I am familiar with those works. Seeing the stimuli/activation correlation does not imply causal representation of the stimuli. It implies the causal activation of neural structures, at most.
> What follows is: if you can represent "thinking" as a computational process, you can implement it with a Turing machine, and thus, an LLM can be made to think. That proves LLMs can think. But not that the existing ones do or don't! Because that's the entire thing about upper bounds!
Alas! assumption spotted. IF you can represent "thinking" as a computational process, then you could implement a thinking machine. You need to prove first that thinking _is_ a computational process, _then_ you could go and try to implement such machine, and because you proved that thinking is a computational process, you are certain that theoretically such a machine can be built. But until you prove your assumption right, you are just trying blindfolded. The harm in the actual field/society regarding AI is that _we don't even know if thinking can be modeled as a computational process_. And no, this does not have anything to do with science. (By the way, I would not regard AI research as science since it is strictly studying an engineered artifact, but that's another story).
The issue of consciousness appears when you think of the world in a mechanistic way: since all there is are laws of physics and materiality, then how could we explain our though processes and our perceptual experience? If the world itself (in a general, existential way) is only made of laws of physics and matter, the consciousness needs to be an emergent characteristic of physical systems, and needs to follow the laws of physics. Now at this stage, you are already in trouble and you need to explain what consciousness is and how it manifestates. And that's the moment where things like the computational theory of mind appears.
But you need to step back in order to detect the fallacy, one of which is: the brain/mind processes information like a computer, then we could build better computers that can think. This fallacy is assumed in the question 'can a machine think?'. There's another fallacy, which the author call the first step fallacy, which is common nowadays: we solved the language problem, then machines will be able to think in the near future.
So it is not about solving the consciousness problem, it is about not claiming things based on assumptions that can be easily challenged.
I have that book at home and I'll check as soon as I get back from a trip.
When Dreyfus' book appeared in 1972, it received really harsh criticism from the then AI community. Dreyfus actually comments on that criticism in the revised 1992 edition.
I just don't see how Dreyfus critique of AI has been dealt with in modern AI: the critique is aimed at fundamental issues, not at the technical issues.
It is true that the critique written by Dreyfus is based on GOFAI algorithms from the 60s, but it is also true that if you read the book today, you'll find lots of similar situations and a similar way of thinking about the possibilities of AI, as well as the same underlying assumptions.
And as a side note, outdated means that it does not apply anymore, or that is not relevant anymore. Which is different from 'establishing a dialogue' with the text/author, in a way that 'seems' not to be relevant anymore. If you say that Dreyfus' book is outdated just because the 4th edition of Norvig's text only mentions it in a footnote, you are assuming that Norvig and Russell's opinion are definitive. They might be not.
I have authors like Norvig, Russel, LeCun, Minsky and other in the field in high regard. But they are normally not trained in either modern linguistics nor philosophy. Let alone the rest, large amount of researchers in the field. AI research is a complex field, and maybe (in this we could follow Foucault) not even a science. Doing research in an area of study does not turn it into science.
It is precisely philosophy, and even more contemporary philosophy, the discipline that focuses on how we build knowledge, and how we experience the world. Two really important, almost fundamental, topics that directly contribute to how AI is developed as a field of knowledge.
We know how LLMs learn at the fundamental level. What we do not know is the actual dynamic process of encoding embeddings and their distributions.
Your analogies about the PC and web browser are not correctly formulated, because in the case of the PC you talk about 'external components' (you should be talking about cpu arch, structure, digital components, interfaces, etc); in the case of the web browser, you should be talking about modules, code, etc.
We do know how LLMs are laid out: layers, att heads, etc. So what we need to look at are the fundamental possibilities of the structure of LLMs, not how the weights are distributed.
> > And we also know that human beings do not hold 'internal representations' like any AI system needs to.
> Bold fucking claim. Got a source on that?
Part of the sources are in the books I mentioned. Nonetheless, you can still fact-check and refute in an adult and serious manner, not in an disrespectful and arrogant way. If my claim sounded arrogant I apologize, but then as I already mentioned, my references back that claim.
Regarding internal representations in the brain: I guess you are referring to areas of the brain being activated when a subject receives a stimuli, and this is tested through MRI. I would be cautious to causally relate stimuli to neuron activations, since you first need to know if the exact configuration of cell involved and their connections allow for such representation (which I think it is still not known -- again, AFAIK, the contrary seems to be the case).
1) I think that is precisely the flaw: it is reductionist.
2) how do you actually measure if a rock has consciousness? if you redefine consciousness as simply some kind of physical manifestation, like radiance or quantum fluctuation (measured as compound at the macro-level), then you will have to redefine what we understand as 'human consciousness' is, otherwise it will be a characteristic so generic that won't be useful at all. Then this new characteristic will suffer from the same original interpretation problem... So at least the panspychism approach is just evading the main problem.
3) I would argue that the only way of thinking about a complex consciousness emerging from a diverse and vast set of things, is considering complexity. If it is not complexity (like, structurally complex things can have simple consciousness and the other way around) then there must be something, material or not, that provides for the emergence of such complex consciousness. This is rather tricky: you will need to postulate some kind of non-physical (or not discovered yet) characteristic that generates consciousness, or you will have to come up with some causal relation between non-related-to-complexity but still-related-to-physicality and complex consciousness, which from our current physics framework might not be possible.
> So why we call it subjective experience then? Probably it is irrelevant, and the reasons are purely historic... or maybe not. How about the idea of computers experiencing things, just not "subjectively" but rather "digitally"? Or choose any other adjective you like. You are arguing against assumptions, but why you just accept the idea of experience with the assumption that human way to experience things is the only possible way?
We call it subjective because is 'we, ourselves' and not the objects we perceive there in the world, where the 'experience' is manifested, as perception. We do not always codify dichotomies in language.
> No sane AI researcher would stop researching AI to get PhD in linguistics to build a bridge between AI research and linguistics. Probably in an ideal world this shouldn't happen, maybe it is short-sighted behavior of a system, but it is just how things work in our real world.
In the real world, anyone doing a serious PhD thesis will read whatever is necessary to build a proper, sound theory or body of work. This dismissal just makes me think that you don't know how a PhD thesis is done.
> BTW it is a good example of what happens with all the philosophy when shit hits the fan. When possibility of empirical studies arrives, no one bothers themselves with philosophy of things.
From what I see, you never took philosophy seriously. I don´t know how you can then seriously engage in a conversation about philosophy.
> I studied psychology, I've read some linguists (cognitive ones, because they are in an adjacent field), and you see, I don't have trust in either. They do their research, they find some interesting facts and devise interesting theories, but it is all looks more like a chemistry in the first half of a XIX century, than a chemistry after periodic table was created. They can't find their building blocks to create a sound theory.
This is true, cognitive linguistics do not represent a unified theory. Not yet, at least, and maybe it will not come to that. The same happens in psychology, but nobody is dismissing psychology all at once just because of that.
> No, I'm not implying "modeling a thinking process". We don't know what thinking process is. What we observe in our minds is not thinking by itself, it is some kind of a mirror process in our consciousness. The real thinking is hidden from us, but it creates echoes in our consciousness we can observe. If we don't know how thinking works, we can't model it. BTW the reverse is also true: if we can't model thinking, we don't know how it works.
This is exactly what the assumptions I challenge are about. The AI space already declared that they 'know' how such processes work. Or at least, they pretend they do.
> I'm defining thinking more in terms of a problem solving ability. Like psychologists do. Science still doesn't have a good enough definition for thinking, but it has some definitions that a) operational; b) good enough for some limited tasks. "Operational" means that they are defined in terms how to measure what you define, not in terms of modeling some process.
You would agree that 'problem solving' is just a small portion of what 'thinking' constitutes.
> You see, until philosophy methods successfully proved that something is conscious despite it was deemed unconscious before, we can't really know that their methods really work. Maybe they work, or maybe they just mirror our biases and heuristics.
You talk about philosophy as if argumentative biases were completely strange to philosophers. Not the case, and a great deal of XX century philosophy is exactly about that. But if you read the sources on AI research through its own history, you will see how the AI research space is full of such biases and assumptions. Again, my original argument is about that.
> I hadn't read books you mentioned, but in your words I see nothing that can hint that those books have something I don't considered already. So maybe I'll read them in a future, but I wouldn't postpone my engagement in the conversation till I read them.
You don't have to trust me. Just pick them up and think for yourself, do some research. Regarding postponing the conversation, I really appreciate that, but it is really difficult to argue about books you haven't read, especially if they are complex ones. Unfortunately I don't have that much time to explain the books in detail.
I am not an expert in panpsychism, but for what I know:
1) the idea that everything has a degree of consciousness proportional to its complexity, introduces the problem of compound consciousness. How do they compose, how is each consciousness contributing to the overall, upper-level one? how is experience explained at the different complexity levels?
2) it is impossible to test whether something is conscious or not
3) the theory is more a philosophical framework for dealing with the mind/body problem, but it actually moves the problem forward on the assumption that 'because it is something physical, it has consciousness. Then complex things have complex consciousness'
I think the conversation derailed a bit. I see also a common pattern of jumping through different topics at different levels (from theoretical to concrete and back), and that is confusing.
My original comment was that it seems (and it is actually documented in the books I referenced) that the AI research space builds its claims on assumptions, not on facts, and that those assumptions are flawed. So a nice discussion, to begin with, would consider:
1) why I make the claim that the AI research space builds its claims on assumptions instead of facts, why we could say that there are actually no assumptions but facts, or why the assumptions are correct.
2) instead of strictly and directly dismissing readings on philosophy, I would expect intelligent and curious people to embrace new references. Particularly if those references are highly regarded and a solid contribution during the last 120 years
Regarding point 1), I can barely count a single comment in this thread that tries to engage in the idea of the assumptions (except for some comments that agree with the premise).
Then regarding point 2), I can barely count research papers, books or contributions in the space of AI research that references (either to built upon or dismiss) philosophy that is pertinent to AI, pertinent to philosophy of technique or cognitive linguistics. This is strange. It looks like if the space revived during the 2000s with the invention of neural nets (RL, GAN, etc), and then became isolated from contributions about human intelligence, even though it continually tries to explain intelligence in its own terms.
The reference to What Computers Can't Still do is precisely relevant because it narrates exactly this same discussion (false assumptions, claims built upon assumptions instead of facts, dismissal of evidence from psychology, dismissal of frameworks from philosophy, fallacies about progress), but it was written in 1972. Still, you read the book today, and it is totally relevant.
Now, regarding your comments:
> Do LLM experience objectively or not experience at all? How can you say?
The world cannot be experienced 'objectively'. If they experience the world most probably you won't notice. Given that the only way of interacting with an LLM is through a process initiated solely by a human actor, it would be difficult to assess whether an LLM experiences anything at all.
> I can say that they could think. The thought process implies a measurable product. Yes, there are similar situation with it, it can be hard to say sometimes if we observe a product of a thought or something else. But science has some success with this, like claiming that bees can think and solve problems.
The moment you say 'they could think', that implies an assumption about the actual possibility of thinking as a process that can be modeled and executed by a machine. There is, as far as I know, no current evidence that human beings process information the same way a computer does it, nor that though processes necessarily imply a measurable outcome.
> I'm telling you, psychologists (who specialize on mind research) do not know what consciousness is and they do not have a definition of a subjective experience (well, if we treat your proposed definition as a valid definition, then we should say they have plenty). And my claim still stands: until you heard about new science "Subjectology" you can be sure that no one knows what subjective experience is. Including linguists.
Psychology is a very broad field with lights and shadows through its short history. Here, and in my original comment, I am not talking about linguists in general, but specifically about cognitive linguistics. The contributions made be the field are significant and mostly lacking in AI research (for example, the idea of embodiment, the rebuttal of generative grammars, prototype theory, frame semantics, among others). What you mention as 'subjectology', would be just psychology. Foucault explains more or less clearly why this cannot be a science, and that's just fine (in The Order of Things).
> You shouldn't believe that anything can be settled in the philosophy space. Philosophers can think they have settled things, but until science agreed and started empirical studies, it is just philosophers believing that they settled things.
Well... certainly nothing can be 'settled' (not even in science, btw), but my point is: there is already enough convincing arguments in the field of philosophy so as to say that current LLM systems do not posses agency or experience, and that they do not behave like us.
Again, read the sources, what are you people afraid of? Just read the sources, and then engage in the conversation.
There are two things here: one is how an LLM is fundamentally structured and designed, the other is how an LLM distributes and 'lays out' the relationship between inputs and outputs through layers and weights.
We might not know how the actual distribution works, but we do know how it i s fundamentally structured and designed -- because we did it. We also know that there is something like a representation system inside them. And we also know that human beings do not hold 'internal representations' like any AI system needs to. So there isn't any 'intrinsically magical' in modern AI systems.
Dreyfus builds his arguments based on Heidegger's work. You expressed it correctly: for Heidegger, in order to experience the world, we need a body that is always present in the world. It is like (but not the same) a continuum from the world, through our body, to our experience, and back.
As far as I as understand it, Heidegger does not states explicitly that there is a mind or consciousness, and I think it is irrelevant in the discussion.
> but that's also just a theory. I don't believe that experience is literally impossible to engineer, as consciousness has emerged from non-conscious being through evolution, so clearly there must be some kind of mechanism for it -- and if there is, then I believe it can be replicated, we just don't really understand it well enough yet to do so
Something can be understood but no replicated. The structure and inner workings of the Sun or a complete galaxy can be understood, but not replicated. In order to replicate things, knowledge is not enough, one also needs the material capacity to do it. So we might understand at some point how experience emerges from complex systems, but nonetheless we also may be unable to replicate it.
I do believe that experience comes from this 'emergence'... but specifically from the combination of factors that make complex mammals and human beings the way they are. But it is a belief: I simply cannot base a whole field of study on it on the base that is a proved fact.
> It would be a valid argument if you could explain us what subjective experience and agency are. No one can explain this, so the arguments sounds like "AI doesn't have something I don't really know what, but they have to miss something, sure".
Subjective experience is what you and only you, different from other beings, experience in and about the world.
> Yes! This is the point. If we don't know how our minds work, how can we be sure that a machine doesn't work like our minds?
You can't be sure that machine do not work like our minds or brains. But you cannot say the opposite either, so saying 'machines could think' base on a false assumption (because you cannot say it is true. It doesn't matter if you could be true).
> Linguists are linguists, they don't know about consciousness, they specialists in language.
That's a very narrow understanding of what language is, what linguists do/research, and the contributions made in the field.
Linguists are (since already 2 o 3 decades) focusing more and more on psychological/cognitive matters. The intertwined topics of language, mind, body and though has a long way in the western philosophical tradition.
> I believe, that all this philosophy is... well... philosophy. Meta-physics. It doesn't matter. What does matter is how we should deal with machines? Do we have to treat them as human beings? Should we accept that they have "human rights"? Can a machine be held accountable for its mistakes? Can we talk about "intentional" and "unintentional" mistakes of a machine?
Exactly because of this. And is this what I am talking about... the topics you mention here are already settled in the philosophy space, but the AI research space keeps going 'round them...
Again: don't be rude. Not nice. As 'not nice' as my bad formatting.
Philosophy is there precisely to critique and analyze what we build and how we behave in the world. Without it, by the way, there would be no science and no engineering.
Dismissing philosophy in AI is like dismissing philosophy in any other applied, practical or creative field. It is precisely because there is a philosophical investigation on each practical, applied or creative field, that that field can actually make progress.
There's philosophy in biology, which helps biology go further. Philosophy in engineering, which makes engineering go further. In architecture, film, photography, painting, literature, medicine, physics, linguistics, mathematics, etc, etc. In all those fields, there is also a philosophical investigation taking place. Right now.
Maybe you confound practical AI design (you should be talking of 'building AI systems') with AI as a research field. I get that, because lots of people cannot make the distinction (calling yourself 'AI researcher' when you actually tinker a model is not scientific. It might follow a scientific method, but it is engineering research, not scientific research).
We have this definition from the history of western languages, the history of western philosophy, and works by Lewis Mumford and Franz Reuleaux, among others authors.
> The relevant question is whether they're purely physical objects behaving according to rules, which is being described as "machine," or whether there is something beyond that. Current understanding is contradictory: all indications are that cells and bodies are purely physical objects, except that there is this phenomenon of subjective experience which doesn't fit with that at all.
Then you would say that you dog broke (died), or that the vet fixed your cat (cured). Which by all means we might speak that way, but surely you would notice that it is not accurate.
Saying that a living organism is just a machine because it is a physical object behaving according to rules is like saying that a beautifully built house is just a bunch of bricks layed in rows and something on top.
But assuming that we say 'purely physical objects behaving according to rules' are machines, then:
1) there is no difference between you, your dog, your fridge and the snail in you garden
2) It would be semantically valid in all languages to say 'my dog broke' instead of it died, 'i got fixed' instead of 'the doctor cured me'
3) Machines vary in complexity and we would still have 'degrees of complexity' regarding machines (human bodies being the most complex, perhaps, toasters being fairly simple), but fundamentally, all of them would follow the same rules for 'fixing', 'breaking' and 'repairing'. Which is not the case.
4) You would have to come up with some kind of theology regarding how we were built. There is no evidence that we have been 'built', quite the contrary.
And most probably there are quite more reasons not to regard machines as living organisms nor living organisms as machines.
You can then proceed with Metaphors we live by, by Lackoff and Johnson. The book shows how and why our understanding of the world is based on the fact that we are embodied beings.
Then there is Being and Time, by Martin Heidegger. It shows how our understanding of the world is, again, based on the fact that we are embodied beings.
Now, these are not newly edited books, and no, there is no real reason to think that because they are all +30 years old or even more, they are outdated. They are not. If you only look at the publication date, then Ramon y Cajal works would be totally crap (by the way, still one of the most cited works in neuroscience). It is from early 1900s.
To complete the picture a bit, you could read:
On the mode of existence of technical object, by Gilbert Simondon Technics and Time, by Bernard Stiegler Meditation on the technique, by Jose Ortega y Gasset The question concerning technology, by Martin Heidegger
These works will give an understanding of how technique in general (technology in particular) is completely anthropomorphisized, which is what ultimately leads to the assumptions present in the AI research programme.
Also, A history of philosophy, by Frederick Copleston. Although extensive, reading volume I (greek and roman philosophy) is essential.
More citations (again, if you really measure the quality or relevance of a philosophical/scientific work by its publication date, you are missing the picture):
Arbib, M. A. (2025).* Artificial intelligence meets brain theory (again). Biological Cybernetics, 119, 16. https://doi.org/10.1007/s00422-025-01013-5
Farkaš, I., Vavrečka, M., & Wermter, S. (2025).* Will multimodal large language models ever achieve deep understanding of the world? Frontiers in Systems Neuroscience, 19, 1683133. https://doi.org/10.3389/fnsys.2025.1683133
Lin, Z. (2025).* Six fallacies in substituting large language models for human participants. Advances in Methods and Practices in Psychological Science, 8(3), 25152459251357566. https://doi.org/10.1177/25152459251357566
Seth, A. K. (2025).* Conscious artificial intelligence and biological naturalism. Behavioral and Brain Sciences, 1–42. https://doi.org/10.1017/S0140525X25000032
Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2024).* Dissociating language and thought in large language models. Trends in Cognitive Sciences, 28(6), 517–540. https://doi.org/10.1016/j.tics.2024.01.011
Mitchell, M., & Krakauer, D. C. (2023).* The debate over understanding in AI's large language models. Proceedings of the National Academy of Sciences, 120(13), e2215907120. https://doi.org/10.1073/pnas.2215907120
Bowers, J. S. (2025).* The successes and failures of artificial neural networks (ANNs) highlight the importance of innate linguistic priors for human language acquisition. Psychological Review. Advance online publication. https://doi.org/10.1037/rev0000595
Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2024).* Dissociating language and thought in large language models. Trends in Cognitive Sciences, 28(6), 517–540. https://doi.org/10.1016/j.tics.2024.01.011
Bolhuis, J. J., Crain, S., Fong, S., & Moro, A. (2024).* Three reasons why AI doesn't model human language. Nature, 627(8004), 489. https://doi.org/10.1038/d41586-024-00824-z
Bender, E. M., & Koller, A. (2020).* Climbing towards NLU: On meaning, form, and understanding in the age of data. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 5185–5198. https://doi.org/10.18653/v1/2020.acl-main.463
Everaert, M. B. H., Huybregts, M. A. C., Chomsky, N., Berwick, R. C., & Bolhuis, J. J. (2015).* Structures, not strings: Linguistics as part of the cognitive sciences. Trends in Cognitive Sciences, 19(12), 729–743. https://doi.org/10.1016/j.tics.2015.09.008
Hauser, M. D., Chomsky, N., & Fitch, W. T. (2002).* The faculty of language: What is it, who has it, and how did it evolve? Science, 298(5598), 1569–1579. https://doi.org/10.1126/science.298.5598.1569
Johnson, M., & Lakoff, G. (2002). Why cognitive linguistics requires embodied realism. Cognitive Linguistics, 13(3), 245–263. https://doi.org/10.1515/cogl.2002.016
Lakoff, G. (2012). Explaining embodied cognition results. Topics in Cognitive Science, 4(4), 773–785. https://doi.org/10.1111/j.1756-8765.2012.01222.x
Harnad, S. (1990). The symbol grounding problem. Physica D: Nonlinear Phenomena, 42(1–3), 335–346. https://doi.org/10.1016/0167-2789(90)90087-6
Placani, A. (2024). Anthropomorphism in AI: Hype and fallacy. AI and Ethics, 4, 691–698. https://doi.org/10.1007/s43681-024-00419-4
Salles, A., Evers, K., & Farisco, M. (2020). Anthropomorphism in AI. AJOB Neuroscience, 11(2), 88–95. https://doi.org/10.1080/21507740.2020.1740350
Floridi, L. (2025). AI as agency without intelligence: On artificial intelligence as a new form of artificial agency and the multiple realisability of agency thesis. Philosophy & Technology, 38(1), 1–27. https://doi.org/10.1007/s13347-025-00858-9 <<< I am not convinced by his position, but it is nonetheless relevant since it splits the debate in two: intention and consciousness
Dreyfus, H. L. (2007). Why Heideggerian AI failed and how fixing it would require making it more Heideggerian. Philosophical Psychology, 20(2), 247–268. https://doi.org/10.1080/09515080701239510
Bengio, Y., & Elmoznino, E. (2025). Illusions of AI consciousness. Science, 389(6765), 1090–1091. https://doi.org/10.1126/science.adn4935
Dotov, D. G., Nie, L., & Chemero, A. (2010). A demonstration of the transition from ready-to-hand to unready-to-hand. PLOS ONE, 5(3), e9433. https://doi.org/10.1371/journal.pone.0009433
Mitchell, M. (2021). Why AI is harder than we think. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2021). https://doi.org/10.1145/3449639.3465421
I haven't read all of them yet. Feel free to discuss.
Now, there are two problems I see in the community regarding the critique of AI. One is the problem of the increasing capability of models. The other is the idea that GOFAI and ANN-based systems (like LLMs) are fundamentally different. Let me explain.
1) The increasing capability of models: it is difficult to engage in any meaningful discussion if the metrics are the capability of models. One should look at how models structurally encode information and what the learning process looks like from an epistemic point of view. As far as I know, and correct me if I'm wrong, these two issues have not changed and are likely not going to change.
2) The idea that GOFAI and ANN-based systems are fundamentally different: this, I recognize, is a controversial claim. But one should not look at how GOFAI and ANN-based systems encode knowledge (explicitly curated and written rules vs statistical learning), but at how the learning material is selected, curated and presented to the system, and the problem of 'closure' and self-reference in datasets. In this regard (which we could call epistemic) there should be no difference between these two technologies. Again, we should not look at how they are implemented, but at how we relate to them from an epistemic point of view.
But going back to my initial comment, this whole thread feels like proving my point. For those not wanting to get involved in philosophy despite willing to engage in AI research discussions, keep in mind that philosophy has always been a guiding light for science.
As a final note: the whole discussion about AI is on whether computational theories of mind are actually solid or not. But it is really difficult to engage in this conversation without at least some background in philosophy, but preferably a strong background in it.
I'm getting a bit tired of coming back to this thread. Reach me out if you want to discuss more. Glad to help and glad to learn.
@federico_ricca https://www.linkedin.com/in/federico-ricca