HackerLangs
TopNewTrendsCommentsPastAskShowJobs

eurekin

1,020 karmajoined il y a 14 ans
"thoughtful considerations about practical applications"

Self hosting, SBCs, AI/Vision/LLMs

comments

eurekin
·il y a 5 jours·discuss
Disclaimer, I only use it to grow the "knowledge hub".

It's a single git project at my $USER home, that is referenced in global memory. It contains as much information about work things, as possible, to be productive.

I found that, if I allowed Claude to create the notes, it actually became more and more useful, but without the guideline, I just could barely get through reading it manually.

I'd never publish anything with such origin.
eurekin
·il y a 6 jours·discuss
That's where the annotation in plannotator helps.

I'm asking to scan projects on gitlab, go through some docs to find more grounding material, write a subarticle (in the same style), scan logs on the test env, issue some curls, etc.; until the whole article is digestible - in the "backing knowledge graph" department.
eurekin
·il y a 6 jours·discuss
Oh, I know exactly what you mean. I use plannotator with claude a lot and have much better time, since I asked for a specific styleguide.

I used "CD era MSDN reference and Raymond Chen blogging style" as a starting prompt for the styleguide and my work ability to digest AI plans raised a lot.

Couldn't recommend it more. Humble, insightful and respecting the reader
eurekin
·il y a 13 jours·discuss
Similarly, I never try to imagine a bright sunshine. It can wake me up
eurekin
·il y a 16 jours·discuss
There never are. Those are going to be viewed as two discreet successful interventions.

One for lay-offs, because it was the best move at the time with the knowledge they had.

Second for quick correction, ability to pivot and execute quickly.

It's been always like that
eurekin
·il y a 18 jours·discuss
I remember seeing this popping up in discussions the first time, but never noticed any resolution (other than to train both sides). Has SOTA advanced?
eurekin
·il y a 21 jours·discuss
> A shift is happening among major AI labs, who are becoming increasingly skeptical of endless parameter count and training data scaling

I'm pretty sure it's mostly due to the training data quality. No idea, why this never gets mentioned in those discussions.

It was obvious right from the get go, that the scaling law just enabled some abilities, that were described by the underlying data and allowing the ANN to abstract it in the latent space.
eurekin
·il y a 23 jours·discuss
Noticed few cliffs. Sometimes it was a spurious stop (had to write "go on" or "continue" to restart), othertimes it was randomly saying: "Oh the user wants [the thing we already resolved]" and goes back in history. Cleared all out on fp16
eurekin
·il y a 23 jours·discuss
If I started today, with building a server, I'd jump right into verified set-ups and writeups, like this one:

https://github.com/noonghunna/club-3090

You can find info about running a patched version of vllm for 1x24gb, 2x and 4x. There's also quite a few "blackwell" subreddits, where people seem to share a lot of substantial information, if you're going the 6000 route.
eurekin
·il y a 23 jours·discuss
> The model is running so hot, that it shoots past the goal and starts looping

later:

> My latest experiment was setting up vLLM (the gold standard for production and concurrent serving) and even with an NVLink (175GBP) and tensor parallelism turned on, it was 3 tokens/second slower than llama.cpp during generation for an equivalent setup.

In all my tests, getting vllm to run is worth it. It was the single biggest thing, that helped for looping issues, agents going whack and losing focus on the task, long context being essentially useless.

FP8 model, unquantized cache in vllm an you have a league better overall experience, with any other stack I tested. Then, you can actually focus on using the model for other things and stop tinkering with settings.
eurekin
·il y a 26 jours·discuss
I'm still sour they had only one toast in, in a two slot toaster
eurekin
·il y a 26 jours·discuss
Also, supposedly biggest MMA event Rogan doesn't want to be on.

Something is up
eurekin
·il y a 27 jours·discuss
I kept getting recipes with "that one ingredient", which was either a major PITA to source or produced too much waste, even from a real world dietician consultation. Example, use 1/4th of a pumpkin for something. Those were good recipes, in terms of macronutrient composition, but doesn't work long term due to logistics.

I'm years after that strict diet needs, but that itch of fixing or easing some parts of the process stayed.
eurekin
·il y a 27 jours·discuss
Chrome driven by the OS accessibility API
eurekin
·il y a 28 jours·discuss
I keep finding more and more usecases for Q3.6 27b (same league) and the best performance is, when answers to my question is already in the context.

The moment I'm trying something open-ended or ambitious, Claude/ChatGPT clearly take you to the goal quicker.

For things, where there's a way to build a knowledgebase though, the local llm definitely can be a true contender. Plus, having a big context and no worries about filling it over and over - you can get quite far.

I'm writing this, literally in between cooking a pasta, that the local llm ordered products for me online. I've built a grocery shopping skill, so that it roughly knows what I have in fridge (losely), my last 10 representative orders (general preferences plus rich info about shops and skus around me) and actual real-time in stock info. The last part has been my personal pet peeve for every product that promised cooking ingredient delivery (that is not packaged specifically for that).

This is what has been promised to us by every big tech company with an agent, and now a local llms actually solved that for me fully.
eurekin
·il y a 30 jours·discuss
Yeah, it won't. SWE here with a sidegoal to tackle the deployment side through various means (homelabbing, grabbing sre/cloud/observability tasks at work).

The biggest observable improvement in my post and pre ai development is the ability to tackle two projects at once, if the agent is on track. If not, and I have to do a deep dive to debug, it basically regresses to plain old everything like before.
eurekin
·le mois dernier·discuss
It feels now like an alternative timeline, one which performance optimisations were first and foremost still. Sometimes I fantasize, thinking how would our current development ecosystem look like, if we never abandoned the "be very vigilant with all resources you use" approach, that includes the whole webdev liftoff, where we ship a few hundred mb chromium engine for a dock app
eurekin
·il y a 2 mois·discuss
Batching lowers that, since the model is read once from memory. Activation accumulation doesn't scale as nicely
eurekin
·il y a 2 mois·discuss
On my homelab. It really feels like a dream come true for my usecase. No more puppet agents. No more declarative syntax, that you have to work around to do basic imperative ways. Or use a module, that stopped being maintained 3 years ago. Just plop a file here and there through ssh.
eurekin
·il y a 2 mois·discuss
Yes, my goes either-or quite abruptly and, while that's not really annoying, I notice it doing way too often, than I'd like to. It shouldn't be done in a binary fashion as well.