garfij · HackerTrans

2 bulan yang laludiscuss

I've been playing around with this recently too and started getting much better results when I told it how to produce rendered PNGs of the file and to inspect it from several angles during iteration. I'm really only just getting going with it though, so if you have any tricks to share, I'd love to hear them!

garfij

3 bulan yang laludiscuss

FWIW I've only ever been on the API based plan at work and we never seem to run into the majority of the problems people seem to be very vocal about. Outages still affect us, and we do have the intermittent voodoo feeling of "Claude seems stupider today", but nothing persistent.

Of course it's a stupid amount of money sometimes, but I generally feel like we get what we're paying for.

garfij

8 bulan yang laludiscuss

Do your markdown files have frontmatter configuration?

garfij

10 bulan yang laludiscuss

We use them extensively in our agent framework at work for all sort of things. You can make up whatever you want, if the tags are semantic enough it just gets it, or you can add a bit of explanation about it in the system prompt or whatever.

  - Circuit breakers when it seem like it's stuck in a loop
  - Warnings about running low on context
  - Reminders about task lists (or anything)
  - All sorts of warnings about whatever

garfij

tahun laludiscuss

We've done varying forms of this to differing degrees of success at work.

Dynamic, on-the-fly generation & execution is definitely fascinating to watch in a sandbox, but is far to scary (from a compliance/security/sanity perspective) without spending a lot more time on guardrails.

We do however take note of hallucinated tool calls and have had it suggest an implementation we start with and have several such tools in production now.

It's also useful to spin up any completed agents and interrogate them about what tools they might have found useful during execution (or really any number of other post-process questionnaire you can think of).

garfij

tahun laludiscuss

Probably less sensible than you think. How many terms would they need to do this over? How many terms would they need to do it for _at once_? How many tokens would that add to every prompt that comes in?

Let alone that dynamically modifying the base system prompt would likely break their entire caching mechanism given that caching is based on longest prefix, and I can't imagine that the model's system prompt is somehow excluded from this.

garfij

tahun laludiscuss

Thank you for saying this out loud. I've been losing my mind wondering where the discussion on this was. LLMs without Tool Use/Function Calling is basically a non starter for anything I want to do.

garfij

tahun laludiscuss

What do you find difficult about distilling your own prompts?

After any back and forth session I have reasonably good results asking something like "Given this workflow, how could I have prompted this better from the start to get the same results?"

garfij

2 tahun yang laludiscuss

I believe if you read carefully, it's not blocked, it's rate limited to once daily, with very clear remediation steps included in the response.

garfij

2 tahun yang laludiscuss

Other people have said it in this thread, but I think your experience is being severely clouded by Oracle. Postgres, MySql, and all have the `TEXT` datatype which are variable length (though with some upper bound)

comments