Huh. I thought hosting one's own databases was still the norm. Guess I'm just stuck in the past, or don't consume cloud vendor marketing, or something.
Local inference users are all about sampling, but users addicted to commercial inference services are wary of sampling, because they have to pay by the token.
The author speculates that bigger/smarter models interpreting vague directives to utilize general-function tools will outperform more precise and detailed directives to utilize narrow-function tools:
> Granted to use a skill the agent needs to have general purpose access to a computer, but this is the bitter lesson in action. Giving an agent general purpose tools and trusting it to have the ability to use them to accomplish a task might very well be the winning strategy over making specialized tools for every task.
The academic community has been using the term "skill" for years, to refer to classes of tasks at which LLMs exhibit competence.
Now OpenAI has usurped the term to refer to these inference-guiding .md files.
I'm not looking forward to having to pick through a Google hit list for "LLM skills", figuring out which publications are about skills in the traditional sense and which are about the OpenAI feature. Semantic overload sucks.
How do we deal with this? Start using "competencies" (or similar) in academic papers? Or just resign ourselves to suffering the ambiguity?
Or maybe the OpenAI feature will fall flat and nobody will talk about it at all. That would frankly be the best outcome.
Yes, you will need to employ someone with basic system administration competence. That's a given.
Cloud infra is touted as obviating the need to hire system administrators, but that's a marketing fabrication. Trying to manage infrastructure without the necessary in-house skills is a recipe for disaster, whether it's in the cloud or on-prem.
Senior software engineer with 46 years of experience (since I was 7). LLM inference hasn't been too useful for me for writing code, but it has proven very useful for explaining my coworkers' code to me.
Recently I had Gemma3-27B-it explain every Python script and library in a repo with the command:
$ find -name '*.py' -print -exec /home/ttk/bin/g3 "Explain this code in detail:\n\n`cat {}`" \; | tee explain.txt
There were a few files it couldn't figure out without other files, so I ran a second pass with those, giving it the source files it needed to understand source files that used them. Overall, pretty easy, and highly clarifying.
My shell script for wrapping llama.cpp's llama-cli and Gemma3: http://ciar.org/h/g3
That script references this grammar file which forces llama.cpp to infer only ASCII: http://ciar.org/h/ascii.gbnf
Cost: electricity
I've been meaning to check out Aider and GLM-4, but even if it's all it's cracked up to be, I expect to use it sparingly. Skills which aren't exercised are lost, and I'd like to keep my programming skills sharp.
Its runtime is quite a bit faster and enjoys a smaller memory footprint. It is also more amenable to "bash on steroids" scripting, and has a quarter-million canned solutions in CPAN.
PyPi has caught up with CPAN in terms of sheer number of packages, but there are still a lot of differences in what those packages cover. You might find solutions in CPAN without a counterpart in PyPi.
More narrowly, every time I deal with databases in python, I find myself missing perl's DBI, which provides the same API for all databases with a corresponding DBD. Python should get a similar universal database interface someday.
Glad my employer is still one of the sane ones.