I think it's not as bad as a snake oil salesman that knew their product was not good.
In reality, most software design and dev advice is well intentioned but is just based on personal opinion, they don't really have much science behind them.
This is not to say they should be completely ignored, it means there should be a healthy does of using your own brain and experience on any of this stuff.
> I still think it's far more likely that people aren't describing the same thing. People are very, very bad at interpreting and experiencing their mind's inner workings.
Are you thinking that the term "internal monologue" is being interpreted too literally and that any system of organized thought, even one without the apparent usage of words, could still be considered an internal monologue?
> Have you ever reasoned through a complex coding or maths problem? Tell me, how did you do that without words?
When I'm working through complex software design issues it's almost all abstract images/conceptual.
Testing different design approaches and options are all non-verbal, I imagine the system using abstract imagery that represents the different concepts.
> I recently encountered a query that deadlocked on itself because it used a parallel execution plan and updated multiple indexes in a manner that the different threads could conflict with each other.
MS SQL?
We've had to set MAXDOP 1 on some specific queries for a long time.
> A biological synapse's weight takes effect whenever its input changes.
I don't think it makes sense to try to compare our brains to ANN's, they are apples and oranges.
A synapse's weight is dynamically modulated by the astrocyte on multiple time scales (millisecond, sub-second, minutes), and the astrocyte itself is receiving inputs and performing computation (in addition to impacting the neural network).
Ya, I just started using them in File Explorer recently and I really like them because I frequently had multiple windows open within the same tree, this is much cleaner. I can't believe it took me so long to actually click the "+" and try it.
Those example are exactly the problem that has not been solved yet. They are the best we can do so far but are pretty inadequate to communicate complex designs in a way that a person can easily absorb all of that info and then reason about whether the resulting system is adequate.
As complexity grows, the value of those artifacts reaches significant limits that are typically dealt with through brute-force mental effort of the people involved.
We were working through many complex business flows and running into exactly what you describe in many areas. Some of the project people with less experience complained that we were spending too much time on exceptions and slowing down the project. We had to explain that when each exception happens 100 times a day with significant impact on business productivity, then it doesn't matter whether you call it an exception or not, it's important to solve for.
> But I don’t think we should be calling these people “domain experts”. I think we should reserve that name for the other group, for the people who truly and deeply understand the domain, the whys and whats and why nots.
I've spent a lot of time in my career extracting info from the business and I think most do understand the whys/why nots but aren't practiced at organizing all of those decades of experience into a higher level abstract model that can easily be communicated.
It's typically layers and layers of information with dependencies in many directions littered with exceptions. Just like our software design+dev experience, it takes a lot of practice to try to organize all of that info into a coherent presentable model.
> It's very difficult to define a specification that works as intended, even with tools.
Agree, we are in the stone age in software design and dev. We have not figured out a good way to communicate the design of complex systems in a way the business can understand.
> Seems like tacit acknowledgment that IBM mothership is not the right place for a speculative growth play from both a management and capital perspective.
I'm not understanding your logic, can you explain?
What I see with the program and amounts companies were awarded is some level of acknowledgment of the current state of quantum research (i.e. IBM is generally considered the leader) and their pragmatic approach that piggy-backs on current technologies (for obvious speed+cost benefits).
> What I'm saying is that this is incorrect. An "idea" exists within a model before it generates tokens.
If that were the case, then the systems would generate words based on the fully resolved idea, but that is not how the LLM systems currently work (per vendors descriptions).
They choose words sequentially and both the specifics of the input as well as the chosen output words significantly impacts not just the rest of the output but the very correctness of the output.
> but not so differently that 'learning stats' would be an inaccurate description.
Agreed, humans are generalizing using some mechanism that can be modeled with math.
But the execution of our reasoning and thought processes is not obviously similar to LLM's next word generation based on probabilities.
> I've seen proposals for Product Managers to define those conditions themselves by speaking with the LLM.
But the LLM is not aware of how the business works and why, so someone needs to work with the business to extract the information. Typically it's not well documented.
> And then, a follow-up: what is actually the bottleneck at most companies? What causes "requirements gathering" to take long?
Complexity.
In my experience (medium size businesses, i.e. 200 million to 2 billion annual revenue) we're trying to understand how a complex set of systems and business processes and different businesses (external partners) interact and then trying to morph all of that into a shape that now has capability X layered on top or in the middle.
Here's a concrete example, business X that makes their own products and has retail stores as well as an ecom site wanted to add the ability to put complementary items built by other companies on the website and have them drop shipped from the vendors to the consumers. The final solution involved 21 different interfaces between 4 different systems (ecom system, store system, omni channel system, external drop ship mgmt system) as well as a new internal system to manage this activity. It's takes a significant amount of time to understand and solve for all of the low level details.
A typical example of trying to add a new significant capability involves many meetings (days, weeks, months, etc. )with the business to understand how their work flows between systems X, Y and Z as well as all of the significant exceptions (e.g. we handle subset A this way and subset B that way, but for the final step we blend those groups together, except for subset C which requires special process 97).
Then with that understanding comes the system solutioning across multiple systems that can be a blend of internal system or vendor's system, each with different levels of ability to customize, which pushes the shape of the final solution in different directions.
There is certainly value in speeding up coding, but it's just one piece of the puzzle and today LLM's can't help with gathering the domain information and defining a solution.
> Thoughts don't happen in a vacuum, they are triggered by external or internal stimuli, and these stimuli/thought precursors could very easily be tokens (dense info packets), which then map to latent space vectors, which very well could be thoughts.
Yes, possible, that's why I asked you above if that's what you meant by "token". Someone else responded and I didn't notice it wasn't you.
> Claims like "humans don't operate the same way" has no basis. Not only do we literally not know how humans operate mechanistically, and so we literally don't know the logical structure of human thought, but any system that is Turing complete is so easy to create that many wildly different mechanistic systems are fundamentally equivalent/interconvertible.
I think this position is too extreme, we do have some information.
We know how LLM's work when generating a sequence of words and I know that my brain does not work the same way for word generation because I am fully aware of the complete thought in advance of any words getting generated by me externally or internally.
I know prior to generating words that my thought is X and the words I'm about to produce need to express that thought.
But with LLM's we know that the essence of what they produce is not known in advance, that it must complete the word generation process to fully realize the end result and that multiple different end results are possible.
> "We don't understand intelligence, so you literally have no idea whether what we recognize as intelligence is some suitable arrangement of "statistical token generation""
Do you mean "token" as in the LLM sense?
Or are you thinking that thoughts in the human brain are also constructed out of some sort of underlying "token" even though the abstract thought happens and is held before any words are used to try to communicate that thought to an external party?
I think it's not as bad as a snake oil salesman that knew their product was not good.
In reality, most software design and dev advice is well intentioned but is just based on personal opinion, they don't really have much science behind them.
This is not to say they should be completely ignored, it means there should be a healthy does of using your own brain and experience on any of this stuff.