It's a combination of different activity metrics (commits, PRs, comments, etc) with AI thrown in to figure out substance/complexity of the work. I originally thought the summaries were simple metrics without factoring quality. There are products that provide this for github and other tools. E.g. https://getdx.com/
With access to view usage for my org and conversations with developers, I think much of the high token usage is a result of people not knowing how to right size the model for the given task. The trend seems to be to pick the most powerful model and use it for everything. Based upon git metrics, I'm one of the top performing engineers at my org and I've yet to run into any overage or throttling on the $200/mo anthropic sub.
llm-wiki is a popular topic. I've found it to be very helpful with keeping details of the random web pages I visit and want to remember. I create a claude plugin designed to work with Obsidian's Web Clipper and qmd for search.
Create .claude/commands/ when you have well defined repeatable, multi-step actions you want to take. I have a command `/triage` where I provide it a bug report and it asks me questions to start scaffolding a work area, lookup related issues, create patches, etc.
Create .claude/agents/ when you have a narrow scoped thing that you want to happen, but not pollute the main context and have a well formed output. Also handy for reducing costs by using a different model for a class of actions you repeat often. For example, the "lookup related issues" from `/triage` is a Sonnet agent that finds Trac issues for a bug reports. Main context only needs the small list of matches and not the dozens of not relevant ones.
Create .claude/skills/ for job description like behaviors you want. For example, code reviewer or security researcher.
For any of these, install the `/plugin` skill-creator. Run `/init` if you don't have a claude.md and then ask the skill creator to help you improve the claude.md and create useful commands and skills.
There is a clear correlation between the rise in LLM use and the volume of PRs and bug reports. Unfortunately, this has predominately increased the volume of submissions and not the overall quality. My view of the security issues reported, many are clearly LLM generated and at face value don't seem completely invalid, so they must be investigated. There was a recent Django blog post about this [1].
The fellows and other volunteers are spending a much greater amount of time handling the increased volume.
Any take home test trivial enough to complete in under 20 minutes could be completed by an AI. The only signal you get from a take home test is whether or not they can submit answers. It doesn't let you know if the candidate is capable of passing the test unassisted.
Take home tests were never a worthwhile signal. Pre-AI, people would search for solutions or have another person complete it.
Creating, possessing, and distributing CSAM is illegal in the US and many other countries. Can you explain why you think it should be legal to create something that is illegal to possess or distribute?
Why? To avoid using Google search. It's been an inferior search product for years. Last time I used the default launcher on a pixel, I couldn't change it to a better search product, so I changed launcher.
Most hobby and small business sites can easily run on a $5-10/mo VPS. If you need a bit more, hetzner server auctions should suffice. It's always safer to use a fixed cost service, instead of the cloud hosts that don't let you set a hard quota on spend.
I've noticed a pattern in the security reports for a project I'm involved in. After a CVE is released, for the next month or so there will likely be additional reports targeting the same (or similar) areas of the framework. There is definitely a competitive spirit amongst security researchers as they try to get more CVEs credited to them (and potentially bounties).