(Not the original commenter.)

You can rack up quite a lot of tokens if you ask it to try out a lot of things, eg for performance investigations and trying out optimisation ideas.

is there a standard pattern for this? Like spawn an agent for each technique to try?

I regularly do a “go to DynaTrace, look at how this service gets used in production then use a profiler and see if there’s any low hanging optimisations we could make” on stuff. LLMs are really good at doing everything that was fun about software development.

Ha, I have a whole hobby work-stream going on about finding low-hanging fruit in various open source projects to turn into valuable contributions.

Two premier sources: (1) look at good contributions someone already tried to make, but that got stuck in review or were otherwise abandoned. (2) look at user reported bugs and see if we can find a user reported bugs, and see if we can reproduce and fix cheaply.

Both are explicitly scoped as best-effort affairs: move on, if you can't quickly make progress.

Most of the work I have to do as a human is review and navigating the submission process: tokens are cheap these days, so you really need to make sure the contribution is actually worth someone's time to review.

maybe this? https://github.com/karpathy/autoresearch

Not sure. I usually tell the agent to spawn subagents at will. (And they are doing that on their own anyway.)

Writing custom WMMA/MFMA kernels for an exact integer matrix multiplication library that uses RNS & CRT + Int8 GEMM to get ~90% of the theoretical i64 TOPS output from a 7900XTX (3.9 TOPS vs the 0.5 or so you get with naive hip-direct usage)

I'm not sure if you think that's a lot, but that was barely even 8 hours. I've had 200B+ months lol