HackerTrans
TopNewTrendsCommentsPastAskShowJobs

utopman

no profile record

Submissions

Show HN: Best setup local LLM found for a 5090 (llama.cpp fork + turboquant)

local-llm.utop.workers.dev
2 points·by utopman·geçen ay·2 comments

comments

utopman
·geçen ay·discuss
you are absolutely right. my title is very bad. I'll update it to a very less absolute statement. sorry for that.

I am now trying to sweet spot things with 27b model + tubo8 so I guess should have plenty quality context left.

the error I made in my tests is to stop with a working configuration that maximised my hardware use, and missing real deep software tests. The one shot 3D app I generated with previous setup is exactly telling this : I did not try my setup on real software development cases.

So thank you for guidance. I am not new using agentic code, but when it comes to proper setup with deep understanding of real trades off on inferences engines, I need more deep undestanding to make better decisions.

The 27b Q6_K turbo 8 for ~150K context should give me a real improvement on this stack. It's test party time :D

Edit : oops, I also found I cannot edit anymore my bold wrong title :/
utopman
·6 ay önce·discuss
Maybe it is just me, but : these emails are spam. Marking them as spam should be easy in a common email box nowdays. Marking these undesired emails as spam lowers email sender reputation, then finally gives real insight to the spamer soon or later. Meanwhile you have no more emails from them.

This is unperfect because of ressource waste and the underlaying unsolved law compliance of these services. But at least you get job done easily this way.

As many things in life this is compromise, not perfect solution. In between using this simple trick I can spend my time on more interesting things.

I respect anyway the fact that people try to fight against the intrusive AI default communication mindset. In the end, i think this post need to be heard rather than having a solution.
utopman
·7 ay önce·discuss
This.

I am mid career now.

High level langages like js or python have a lot of bad design / suboptimal code... as well as some java code in many places.

Some bad java code (it just needs to be a sql select in a loop) can easily perform thousand time worse than a clean python implementation of the same thing.

As said above, once c was high level programing language and still is in some places.

I do not code in python / go / js that much these days, but what made me a not so bad developper is my understanding of computing mechanism (why and how to use memory instead of disk, how to arange code so cpu can use it's cache efficiently...)

As said in many posts, code quality even for vibe coded stuff depends more on what was prompted and how many efforts the PR diff is human readable to get maintainable and efficient softwares at the end of the day.

Yet senior devs often spend more time reviewing code instead of actually writting some. Vibe coding ultimately feels the same for me at the moment.

I still love to write some code by hand, but I start to feel less and less productive with this approach while at the same time feeling I don't really lost my skills to do so.

I think I really feel and effectively am more efficient at delivering thing with appropriate quality level for my customers now that I have agentic coding skills in my belt.
utopman
·8 ay önce·discuss
Then the thing becomes :

Understand how to do the thing

Do the thing

Write down fresh documentation about the thing
utopman
·8 ay önce·discuss
Not sure it is your case but I think I sometimes had to solve many of them when I am in my daily task rush. My hypothesis is that I solve them too fast for "average human resolving duration" recaptcha seems to expect (I think solving it too fast triggers bot fingerprint). More recently when I fall on a recaptcha to solve, I consciently do not rush it and feel have no more to solve more than one anymore. I don't think I have super powers, but as tech guy I do a lot a computing things mechanically.