meatmanek

vor 4 Tagendiscuss

Direction dependence also means your output drops even more on cloudy/overcast days.

vor 4 Tagendiscuss

It would be really cool if they could expose this information to customers somehow. Imagine:

   - having a log of the most prominent J-space tokens during your customer support chatbot's interactions with a user, so you can have more introspection into why a particular outcome happened
   - being able to detect certain thoughts associated with undesirable behavior (hallucinations, overstepping authority, lying, etc.) and trigger some sort of remediation (e.g. upgrading to a better model, redirecting to a human, forcing tool calls)

meatmanek

vor 14 Tagendiscuss

All you're saying is that the government under Biden talked to Meta.

meatmanek

vor 15 Tagendiscuss

https://youtu.be/zdA__2tKoIU?t=149

meatmanek

vor 16 Tagendiscuss

> You are missing the context of the ecosystem

That's one of several reasons why a citation is warranted.

meatmanek

vor 23 Tagendiscuss

This is super cool. Do you know if any of the inference backends (llama.cpp, vllm, etc) support this technique?

meatmanek

vor 25 Tagendiscuss

> speed dial options. That doesn’t happen if you don’t click on them directly.

It also seems to happen if you type the domain name in the address bar but hit enter when the suggested URL autofills. For me, typing out aliexpress.com fully will send me directly to AE, but typing aliexpress.c and hitting enter (with the autofill completing "om") redirects through vivaldi.com/bk/aliexpresscom-us

meatmanek

vor 29 Tagendiscuss

Frankly, it's not really more insecure than any other installation method. Apt packages and the like generally have the ability to specify pre/post-install scripts, so `sudo dpkg -i ./random.deb` is equivalent to `sudo bash ./random.sh`. Even if they didn't have pre/post-install scripts, they're still writing arbitrary files to arbitrary locations on your disk, so they can trigger execution the next time you boot or log in or whatever.

And at the end of the day, no matter the installation method (even just unpacking a tarball and executing the program directly from that directory), you're going to run their program on your computer, and then the program can do whatever it wants. Maybe you don't run it with sudo, but https://xkcd.com/1200/ seems relevant.

meatmanek

letzten Monatdiscuss

Freemium is a potentially interesting model for app developers. If you're small enough to be in the free tier, you can avoid the risk of someone turning your app into the next backend for Chipotlai Max and running up your AI bill.

Once you get enough downloads to where Apple starts wanting to charge you money, you can consider switching to OpenAI or Anthropic or Google or Deepseek or whatever. Sounds like they've even made that relatively easy to do in the Foundation Models framework -- just implement the LanguageModel protocol. I expect open-source or provider-written adapters to pop up that would let you use your vendor of choice.

meatmanek

letzten Monatdiscuss

I'm guessing they'll integrate with the double-tap-the-bottom-of-the-screen feature that pulls up siri in front of a screenshot. Currently it doesn't seem to hook into "visual intelligence", and needs to call out to ChatGPT to do anything with the screen contents.

meatmanek

letzten Monatdiscuss

"shape"

meatmanek

letzten Monatdiscuss

At least that somewhat aligns incentives between players and the game studio. If an old game has a long-lasting player base, then a modest subscription makes it more likely that the studio would keep the servers up and running, if not actively patching the game. With a game that you pay for up-front, a long-lived player base can be a liability for the company (ongoing costs without many new purchases.)

meatmanek

letzten Monatdiscuss

it's a joke: the author of the post is literally the guy who wrote PuTTY

meatmanek

letzten Monatdiscuss

I still use google as a verb even though I use Kagi.

meatmanek

letzten Monatdiscuss

Presumably asymmetric splits meaning A and B can talk, B and C can talk, but A and C can't talk.

meatmanek

vor 2 Monatendiscuss

> AI works great on it.

Define AI, and define "great"

meatmanek

vor 2 Monatendiscuss

> What they first received

Shouldn't every cell in this column be the same?

meatmanek

vor 2 Monatendiscuss

I think you're correct with the standard thinking approach (just generate a big stream of tokens before drafting your actual answer). After a while, additional thinking just results in loops.

The RSA approach from https://rsa-llm.github.io/, expanded on by https://www.zyphra.com/post/zaya1-8b, looks like a promising way to squeeze a bit more intelligence from a small model. As I understand it, running multiple independent thinking traces in parallel gives you a chance of one of them finding a different local optimum, whereas running a single trace for longer is likely to just circle around one optimum.

That said, at the end of the day, there's only so much information a small model can contain. If a model just doesn't know some key piece of information, no amount of thinking will help it figure out a solution that depends on that information.

meatmanek

vor 2 Monatendiscuss

> Doubleclicking random exe files off the internet is almost uniquely a Windows problem.

Tell that to my partner's grandfather, who managed to find and install malware chrome extensions on his chromebox.

meatmanek

vor 2 Monatendiscuss

Ok, I double-checked, and I get 21-22tps with lmstudio-community/LFM2-24B-A2B-Q4_K_M.gguf running under LM Studio on my i5-12400 with 2x32GB sticks of DDR4 3200. This is with small context (just "Write me a poem about a language model named Liquid" in `lms chat`)

    Prediction Stats:
      Stop Reason: eosFound
      Tokens/Second: 21.10
      Time to First Token: 1.827s
      Prompt Tokens: 42
      Predicted Tokens: 187
      Total Tokens: 229

Submissions

Preventing Kubernetes from pulling the pause image from the internet

comments