okwhateverdude

3 個月前discuss

Robots struggle with syntax-in-syntax. Really easy to confuse them when asking it to write a SQL query that targets a JSON column but it must respond with a JSON envelope so the harness can parse the result. Lots of escaping that needs to happen. Deeply nested structures in JSON also end up with foibles like missing a ] or } in a string of }}]}]}. Aside from the prompt injection possibility, just the result being straight up broken and requiring another LLM call is tokens flushed.

okwhateverdude

4 個月前discuss

How can we opt-out of these tests? The behavior foibles I've been experiencing over the past month might be directly attributable to these experiments! It can be extreme frustrating. I don't want to be in the beta channel. Please change this to be opt-in.

okwhateverdude

6 個月前discuss

> However, for some reason, both Gemini and ChatGPT tend to argue with me

The trick here is: "Be succinct. No commentary."

And sometimes a healthy dose of expressing frustration or anger (cursing, berating, threatening) also gets them to STFU and do the thing. As in literally: "I don't give a fuck about your stupid fucking opinions on the matter. Do it exactly as I specified"

Also generally the very first time it expresses any of that weird shit, your context is toast. So even correcting it is reinforcing. Just regenerate the response.

okwhateverdude

10 個月前discuss

I work for a fintech. We're currently building agents to manage basic human labor tasks that are time consuming (think: initial credit analysis, initial web research for AML, etc.). Chatbots are a waste of the actual power behind engineering with LLMs. With tool usage, and detailed instructions, we're easily able to direct the agents to do very useful work. While it has been possible to build these kinds of automations previously, the costs (time and opportunity) would have been astronomical to capture and encode the reasoning in a programming language. Now we can simply ask the robot in plain language to do the task. It won't reduce our headcount, but it will enable scaling the business with way less hands. And the cost to develop is very minimal. The first MVP for the credit analysis robot was vibe coded in an afternoon using an off-the-shelf lib. Doing the actual engineering is trivial context management and tool integration. The challenge is having the kinds of problems we do where these kinds of solutions make sense. In finance and insurance, there are tons of these manual human tasks that could benefit from the robots. I expect this space to explode in the next couple of years.

okwhateverdude

10 個月前discuss

Do you have an iGPU+dGPU combo like in a gaming laptop? For my 2019 gaming laptop with a i915+RTX2060, if I leave the switchable option on in the hardware settings (meaning the OS can choose which GPU to use ie. iGPU for low power), I end up with similar behavior in Linux/Win10. The external port actually uses the dGPU so rendering must go through it but it will struggle with the switchable option and I end up with freezing or stutters. In Linux, I explicitly configure Xorg to only use the iGPU and reserve the dGPU for model inference which fixes the stuttering issue. When booting into Win10, I explicitly disable the iGPU and turn off the external monitor so my VR playing doesn't stutter or youtube doesn't freeze or stutter.

okwhateverdude

10 個月前discuss

10786 :-D https://www.johnayliff.com/games/seedship/index.html?bHcorjm...

okwhateverdude

去年discuss

Really good times on k5 all those many moons ago. It is weird to consider how ephemeral these internet communities are in the end.

okwhateverdude

去年discuss

This was some out there singularity fiction back in the day with some really vivid imagery including being raped by a zombie, skinning the protagonist alive and dumping a mound of fire ants on them, and some incest. I re-read this occasionally and still enjoy reading it if only because it feels truthy for how various shades of humanity would deal with an immortality giving, reality altering, techno-god. The shock value of various scenes mimics the darker corners of the internet and does a good job exploring the dichotomy of behavior on and off the internet in the guise of dealing with the reality foisted upon the characters.

If you've not read it, and aren't bothered by some extreme imagery, I definitely recommend.

okwhateverdude

2 年前discuss

It is default, but unless your activities are public, your data isn't included. It will still tell you where your recorded activity would rank (and a CTA to change the privacy). I have a couple of somewhat popular segments I ride. I try to stay in the top 100.

okwhateverdude

2 年前discuss

Not to put too fine of a point on it, but why not merge one of the simpler PRs for this feature, gate the feature behind an opt-in env var (ie. OLLAMA_EXPERIMENTAL_GRAMMAR=1), and sprinkle these caveats you've mentioned into the documentation? That should be enough to ward off the casuals that would flood the issue queue. Add more hoops if you'd like.

There seems to be enough interest in this specific feature that you don't need to make it perfect or provide a complicated abstraction. I am very willing to accept/mitigate the side effects for the ability to arbitrarily constrain generation. Not sure about others, but given there are half a dozen different PRs specifically for this feature, I am pretty sure they, too, are willing to accept the side effects.

okwhateverdude

2 年前discuss

I think this is a neat project, and use it a lot. My only complaint is the lack of grammar support. llama.cpp that they wrap will take a grammar. The dumbest patch to enable this is like two lines. And they seem to be willfully ignoring the (pretty trivial) feature for some reason. I'd rather not maintain a -but-with-grammars fork, so here we are.

https://github.com/ollama/ollama/pull/4525#issuecomment-2157...

okwhateverdude

3 年前discuss

I already struggle with waiting for the rube goldberg-esque build-deploy-to-dev loop at my current gig. Thanks for the additional disgust.

comments