Don’t get me wrong, I very much agree with the danger. As I highlighted - I saw it this morning when someone used Claude to draw the wrong conclusions.
I’m saying there is a way in which they can be used where there isn’t scope for numerical hallucinations at all. They can write sql queries, for example, without ever being allowed to even see numbers.
What invariably does and will happen though is they’ll inner join instead of left join and some data will get missed. Or there will be some missing context (users in this set already have a certain class of property by virtue of some selection bias and that will be mistreated as some signal etc).
I use it a lot now for knocking up grafana charts etc. It’s not so much that the LLM is feeding the numbers through. You can still use real tools to analyse and summarise the numbers, it’s just much quicker at driving them.
As ever with data analysis, two things will continue to be true. Real insights come from spotting something that looks off and digging into it deeper. Secondly, it’s really easy to connect data in a misleading way.
I’ve had a Claude analysis handed to me this morning including a summary list of actions we’re going to take next which falls into this very trap.
The insights you’ll get from your data will only be as deep as the curiosity of the person at the helm.
For the people that didn’t live through this time, lining these images up makes it obvious why those that did speak of how visually impressive the Amiga was.
I loved all the early stuff. Freaky Styley, Mothers Milk, The Uplift Mofo party plan. With Rick Rubin at the controls I just think Blood Sugar Sex Magik took their sound to another level.
It always makes me a bit sad that everyone knows RHCP but less so their early stuff. Blood sugar sex magik is a funk masterpiece. Didn’t help that for years Spotify used the singles versions of the tracks so the levels were all over the place and it was basically unstreamable.
> To what degree did I expand scope because I knew I could do more using the AI?
Someone at work recently termed this “Claude Creep”. It’s so easy to generate things push you towards going further but the reality is that’s you’re setting yourself up for more and more work to get them over the line.
Yup same. Going a little insane trying to set up a new device that's having network issues. Claude was helping me diagnose. Seems like I've infected the whole internet.
One annoying thing about that flow is that when you change the world outside the model it breaks its assumptions and it loses its way faster (in my experience).
Ha! I had someone doing a task the other day and the llm they used wrote a regex in Perl. I joked that in 25 years all the seems to have changed is the number of layers between me and Perl.
For me, all the code in the PR at that point is in question. The code was changed as part of a bigger part of work and reverting it without understanding why might cause issues back the other way.
The other thing to say - that is missing from this conversation - is that if you’re using github, the original commits are still against the PR.
Maybe a slight misinterpretation of what I meant. The commit that goes with a PR is definitely useful context, but I’ve found more granular than that is seldom useful. Even big ones like “move from angular to react” - the details of someone getting something wrong in there don’t matter, it’s the scale of it that just makes me go “oh yeah, this is bound to be a mistake”.
Maybe different in other places, but after 15 years in my codebase, I’m still happy with a simple linear history.
It’s pretty hard to keep the commits in a working branch in a good legible state - certainly it takes work to do it.
In 25 years of professional development I’ve never really had a situation where the commits on a branch would have helped me understand what was going on a year ago when the work was done. That includes pretty big bits of project work.
I’d much rather have a trunk with commits at the granularity of features.
> allows functions to read the context they’re called in
Can you show an example? Seems interesting considering that code knowing about external context is not generally a good pattern when it comes to maintainability (security, readability).
I’ve lived through some horrific 10M line coldfusion codebases that embraced this paradigm to death - they were a whole other extreme where you could _write_ variables in the scope of where you were called from!
I’m saying there is a way in which they can be used where there isn’t scope for numerical hallucinations at all. They can write sql queries, for example, without ever being allowed to even see numbers.
What invariably does and will happen though is they’ll inner join instead of left join and some data will get missed. Or there will be some missing context (users in this set already have a certain class of property by virtue of some selection bias and that will be mistreated as some signal etc).