I agree, and yet I think even with a well engineering agent harness, there are a lot of unknown unknowns out there.
I imagine the problem will persist if users continue to submit PRs that pass the harness without being able to validate for themselves that it actually works.
I've been working on a project myself over the last few weeks where the documentation is quite minimal. To no surprise the LLMs fell flat at being able to generate any sort of meaningful code. However, I realized that if I focused first on building out documentation and coding tools (linters, parsers, formatters, etc...), LLMs can do a decent job at solving fundamental problems.
I'm curious what percentage of PRs are just the AI blindly writing code and submitting a PR without testing, and which have at least been locally tested to some degree. Any OS maintainers have any insights on this?
I've used Claude Max awhile now, and I usually only get to around 50% usage in a 4/5hr block (using medium effort). Yesterday, I switched from high -> medium effort using the /model command, but afterwards it still felt like I was burning through tokens at the high effort rate.
Haha, I think the experience is a bit different at the higher levels (between ranks 50-1000), but overall people are quite a bit nicer than those playing League of Legends or Dota.
Does anyone have any recommendations on best practice security methods? As others have said, it sounds like there may be an order of magnitude more vulnerabilities found / exploited, and I'm wondering if security such as 2FA and Password Managers will be enough? Should people be getting on board with other protections such as security keys?
I think the key part here is the bootstrapping phase. You may not use a specific English word every week, but maybe you use it every 2-3 months. SRS is great for getting information to these different thresholds!
As someone who has used spaced repetition extensively I will just provide a few insights that might be helpful:
1. Decide on what's important. Just because you learn something doesn't mean that it should be logged to the system. I used to log a lot of minor details (like niche method signatures or command flags to the system). If you make cards for every detail like this then you will be trapped reviewing 100s of cards daily that you likely never use.
2. For the cards you deem are important, make sure you understand the concept. This often means making 2-5 cards for the concept that test your understanding from different angles (definition, pros, cons, how would I explain this to someone else, etc...). This helps to cement the concept at a foundational level.
3. Try to move from the existing flashcards to 2nd order flashcards or pure application after the first couple reviews. So your foundational cards are now set to review in 6 months or 1 year. At this timescale if you prioritized what was important and made sure that you understood the foundational concepts, then usually simply doing things related to the concepts will be the reviews (and sorry to say but if in 1 year you get a card related to what you are doing, but never used, chances are it probably wasn't that important). In addition to doing, you can also create 2nd order flashcards (which might compare 2 concepts). These types of cards test the foundational knowledge indirectly, and are helpful for higher order thinking.
In conclusion, I think spaced repetition is a very effective tool for efficient learning (especially in the first 60 days or so after learning something). I think the major pitfall is not prioritizing what cards get made and being stuck in review hell.
Honestly, it's a big conception that success in RTS games rely on high APM. Great strategy and macro will get you to the 99th percentile in most rts games. I notice that a lot of players that are in the lower percentiles are those who focus more on trying to improve APM rather than strategy or macro.
I imagine the problem will persist if users continue to submit PRs that pass the harness without being able to validate for themselves that it actually works.