HackerTrans
トップ新着トレンドコメント過去質問紹介求人

stephendause

no profile record

投稿

Six Principles for More Rigorous Evaluation of Cognitive Capacities

aiguide.substack.com
2 ポイント·投稿者 stephendause·6 か月前·0 コメント

コメント

stephendause
·6 か月前·議論
I don't know; I've never done something like that. If no one else answers, you can always ask Claude itself (or another chatbot). This kind of thing seems tricky to get right, so be careful!
stephendause
·6 か月前·議論
There is an example of [dis]allowing certain bash commands here: https://code.claude.com/docs/en/settings

As for queries, you might be able to achieve the same thing with usage of command-line tools if it's a `sqlite` database (I am not sure about other SQL DBs). If you want even more control than the settings.json allows, you can use the claude code SDK.
stephendause
·6 か月前·議論
Your story sounds similar to mine. There are some parts of programming at which I know I will never excel. I also don't have time in my life to spends lots of hours outside of work developing my skills. I think it's important to realize that the median software engineer is probably not doing these things either. Maybe the top 10% are? Something like that would be my guess. It's okay to not be in the top 10%!
stephendause
·6 か月前·議論
Beauty is in the eye of the beholder. My personal taste for the presentation of a piece of writing is that less is more. I usually find artwork that accompanies a text to be distracting. I love reading work that can stand on its own, invoking images in the mind. I also dislike animations that seem to be made for a certain scroll speed.

Having said all of that, I certainly don't think it's bad, nor is it a commentary on the arguments being made. It's just not my cup of tea.
stephendause
·7 か月前·議論
Why? To me, hosting previous versions of an article in a public git repo adds transparency. Or perhaps you are talking about GitHub specifically?
stephendause
·7 か月前·議論
> - Is the work faster? It sounds like it’s not faster.

The author didn't discuss the speed of the work very much. It is certainly true that LLMs can write code faster than humans, and sometimes that works well. What would be nice is an analysis of the productivity gains from LLM-assisted coding in terms of how long it took to do an entire project, start to finish.
stephendause
·9 か月前·議論
This is total speculation, but my guess is that human reviewers of AI-written text (whether code or natural language) are more likely to think that the text with emoji check marks, or dart-targets, or whatever, are correct. (My understanding is that many of these models are fine-tuned using humans who manually review their outputs.) In other words, LLMs were inadvertently trained to seem correct, and a little message that says "Boom! Task complete! How else may I help?" subconsciously leads you to think it's correct.
stephendause
·9 か月前·議論
This is a good insight, but do you know of better ways to measure machines' abilities to solve problems in the "messy real world"?
stephendause
·9 か月前·議論
I think it's not only the potential for self-improvement of AGI that is revolutionary. Even having an AGI that one could clone for a reasonable cost and have it work nonstop with its clones on any number of economically-valuable problems would be very revolutionary.
stephendause
·10 か月前·議論
This is a key question in my opinion. It's one of the things that make benchmarking the SWE capabilities of LLMs difficult. It's usually impossible to know whether the LLM has seen a problem before, and coming up with new, representative problem sets is time-consuming.
stephendause
·10 か月前·議論
Jonathan Haidt has a lot of good material on this. He is leading the charge in encouraging parents to delay giving their child a phone until high school and not allowing them to have social media accounts until age 16.

https://www.goodmorningamerica.com/family/story/author-sugge...