HackerTrans
TopNewTrendsCommentsPastAskShowJobs

imrehg

1,362 karmajoined 17 лет назад
Senior Machine Learning Engineer at Faculty https://faculty.ai

Contact: [email protected]

Based at http://gergely.imreh.net

Taipei Hackerspace co-founder https://taipeihack.org

[ my public key: https://keybase.io/imrehg; my proof: https://keybase.io/imrehg/sigs/Vqc30FUDkAzeR6b4jTTQown3Qjned_Xju7DqvvT6DfU ]

comments

imrehg
·5 часов назад·discuss
That's about the speed I get on a AMD Ryzen AI 9 HX 370 (inside a Framework 13), with Qwen3.6-35B-A3B, so doing the same on that much larger model...
imrehg
·6 дней назад·discuss
As long as places like Taiwan are effectively blocked from being in the UN[0], I don't believe we should be adding more to their power and responsibilities than what they have now, au contraire!

If it was a truly world representation, this might be different. But if things like health are sacrificed (ie. no WHO access either), I don't think they really deserve the benefit of the doubt.

[0]: https://en.wikipedia.org/wiki/Taiwan_and_the_United_Nations
imrehg
·8 дней назад·discuss
Should be, judging by their LinkedIn from the bottom of the page (Centaur Technology - Member of Technical Staff: 2005 - 2013)

This is a blast from the past. Centaur was doing VIA Technology's CPUs, and I was at VIA (in Taiwan) while this author was in Centaur (in the US). I was on the embedded side, but I remember some distinct collaborations with the US team, so there's a non-zero chance to have crossed path.
imrehg
·11 дней назад·discuss
From the Huggingface page, it is a fine-tuned version of the Qwen3.6-35B-A3B that is my alternative, and the benchmarks seems to be better. So I'm using as a "likely some quality gain over the other model, while the performance seems pretty much the same".

I've done a couple of checks, and it seems very marginally better on some local benchmarks I'm running, but it's not super scientific evaluation.
imrehg
·12 дней назад·discuss
On the MoE versions of these models the MTP versions have only marginal benefit. In my trials the speed-up is <20% (not the ~2x that happens with some other setup/models) and usually more like 10%. Ie. something like 13 -> 15 token/s... on my device.

I still use the MTP version as it _feels_ slightly better quality, and because the unsloth quantizations I can get have more variety to fit into the various systems at hand... but that's not for the MTP aspect, unfortunately.

In the article they did have ~2x performance on the 27B (which might be something to retry, though on my Framework that would bring it from 5 -> 10 token/s so still "excrutiating" speed, probably).

YMMV for sure.
imrehg
·12 дней назад·discuss
I'm having a decently good time time with `qwen3.6-35b-a3b-mtp` (unsloth's multi-token prediction version) and and `qwen-agentworld-35b-a3b`.

On a 2021 M1 Pro (32GB RAM) I can get either of them as `IQ4_NL` quantized models (the first with reduced context, around 160k; the second can do the whole 264k with RAM left over), running something like 30tokens/s.

On a Framework 13 AMD AI HX370 it can use the same, but both on Q8_0 quantization, full context window, parallelism. Speed is just ~15tokens/s so slower, but definitely smarter than the lower quantized siblings.

Both of them are good developer partners for an engineer who wants more of a second pair of eyes and a rubber duck, rather than a model to just do everything for them. Pretty good for my brain dumping, some commit reviews, sanity checks, just always assume that every claim has to be checked and re-checked.

The only problem is really the context loading, that's pretty slow (starts off around 300token/s on empty context, by the time we get to something like 70-80k which is just a bit of repo discovery, it can run around 80 prompt token/s or less, so there's always a lot more waiting around. Local tools need to bump all of their timeouts, and have to be mindful that there's unlikely to be really meaningful parallelism on these machines with local models.

I'm still figuring out how to approach these things, though. Definitely better than glorified autocomplete or search tool (and too slow for the former, pretty decent for the latter). Their limited skill and performance make it more in line with other tools like my IDE or editors, that they are still in the "tools" compartment of my thinking, rather than "independent, cognitively active entities". Which feels like a good thing.
imrehg
·17 дней назад·discuss
I value Open Source very highly, and enjoy contributing. Fix a bug here, add a small feature there, most OSS projects have low hanging opportunities. And these days, to "switch off" from my at-work code, try to do more OSS contribution as well.

What I see instead, really, is that most projects no longer, or very rarely look at any contribution, and e.g. any issue + PR/MR combo I make, has a much higher chance of never being looked at and some bot just closes it. Even though the rest of the project might be actually quite active.

It takes some getting used to, apparently being filed together with the "noise", when I try to go out of my way to be as much "signal" as possible. But well, if I really wanted to, I can just run my own changes locally, that's the beauty of OSS, but I hope we can get to some more balanced place over time (being forever the optimist).
imrehg
·18 дней назад·discuss
Above a certain size you cannot get the quota-based models.

In my org, as our Team account (quota-based) was growing, to onboard more people we _had to_ switch to Enterprise (and thus API-based costs). The switch and now cost visibility really drives home the "did this code review worth $15?" kinds of questions...

We are just building the monitoring and I think there will be a crack-down sooner rather than later.

I'm more "code as craft" person, so I'm mainly unaffected (plus found my favourite local LLMs to be the missing rubber ducky with no associated cost), butI cannot say the same for everyone else around.
imrehg
·22 дня назад·discuss
Oh, this was one that I played a lot as a kid! (Alongside F-19 Stealth Fighter, F-117A Nighthawk Stealth Fighter - the two that apparently came before and after this game - TIL, and to a lesser extent F-14 Tomcat)

I think, this needs the original game files to run, if I read things correctly. So probably just gonna read the dev journals, rather than fly this particular bird again...
imrehg
·в прошлом месяце·discuss
I encountered this when I was checking why my gluten-free bread came out the bread machine the way it did. I guess it latched onto some yeast-related points and it fell back to Opus...

Having said that, on this query I've seen very little difference in the quality, there's nothing to be "2x as good on" for the "2x quota usage", so shrugs?
imrehg
·2 месяца назад·discuss
Yeah, I've ended up mapping "M5" to "End" (so that "Home" has a counterpart). This current one still has the Ins/Del/PgUp/PgDn/Home for direct access, but certainly some of the lack is a head-scratcher.
imrehg
·2 месяца назад·discuss
I've just recently got a mechanical keyboard, after doing a workspace ergonomics assessment, and also cultural influence from various other developer friends, etc...

It's a Keychron Q10 Max (Alice Layout) - looks like a split keyboard but it's one piece. It's excellent typing, has both wired (USB-C) and wireless (Bluetooth, probably also radio too? I don't use that) connectivity too. I don't normally use the LED lights, but occasionally they are fun...

It's heavy, so not portable like the one the author uses. Had something like that before, the portability was nice, but then didn't use it much. This is not (practically) portable, but still has all the flexibility, and it's a joy typing with it. I wouldn't go as far as I love a piece of euqipment, but I do look forward using it every day.

I do think more about keyboards as I use Mac, Linux, built in laptop keyboards, this stand alone one, etc... And because of the variety it's really hard to build up some muscle-memory. Ctrl, Option, Alt, Fn, ... basically all the extra keys beside the alphabet are slightly different in all systems. So it's more conscious typing than I'd hope for, but not toooooo bad (and it's not the keyboard's problem, I might have to look into remapping stuff, but it's not that level of pain yet).

Happy typing, everyone!
imrehg
·2 месяца назад·discuss
Yup, that's exactly it:

- get the gravitational constant with these two known masses

- then can deduct the mass of the unknown Earth by its interaction with other masses (say the "g" gravitational acceleration value)

- then from the mass and the otherwise measured size of Earth the density pops out

More details in good ol' Wikipedia: https://en.wikipedia.org/wiki/Cavendish_experiment#Derivatio...
imrehg
·4 месяца назад·discuss
I haven't really seen many alternatives besides what's listed, and looking for it (FOSSin my personal development, closed source at $WORK).

Imho the best options there are

- GitLab (if you want someone else to host your things, and be full featured),

- cloud provider-based repos (GCP, AWS has git hosting, if e.g. you are already using them, but it's a subset of features, needing to tie in other services to be a full replacement), or

- go down the self-hosting route if you have the capacity. GitLab is pretty easy to self-host, and there's forgejo, both mentioned in the doc.

Either way you are looking at paying for hosting, one way or another, which for commercial projects should be a baseline. The question is what to pay for, versus what the team should do itself if they can.
imrehg
·9 месяцев назад·discuss
Reminds me of PoC||GTFO.

Loads of fun in this and in that. And does indeed make me sometimes question if I know anything about programming (in a good way:)
imrehg
·7 лет назад·discuss
Interesting to see the starting commit for that file, there's quite a bit of discussion and rationale too: https://github.com/git/git/commit/c8af66ab8ad7cd78557f0f9f5e...