traceroute66

5,338 karmajoined 7 лет назад

Submissions

Tailscale state file encryption no longer enabled by default

367 pointsby traceroute666 месяцев назад140 comments

comments

4 дня назадdiscuss

The blog author complains of "lack of/poor web search capabilities" in GLM, but you can always use it against an MCP of which there are many. For applications where I am not concerned about my queries being passed through a US provider, I have had success with exa[1]

There are also other ways to give it context without web-search. For example the various MCPs that make `man` pages available.

I've also found GLM to be quite strong for coding tasks without the need for web search. So it also depends what you're doing.

[1] https://exa.ai/

13 дней назадdiscuss

Don't forget the little `alpha` tag next to tangled !

If I'm working in software development, the last thing I want to do is trust my coding workflow to an alpha-state platform.

13 дней назадdiscuss

> is not secure against a potential attacker who has physical access to the hardware.

Well, yes, its the oldest adage in computing that "physical access == game over".

So I would argue it is more about reducing your risk to a more acceptable level.

And in that respect I would say using services such as Tinfoil or Privatemode is an enormous step up from "trust me dude, we won't look at your data".

Remotely verifiable attestation combined with independent audits of the company hosting is a large step up from a Zero Data Retention clause in your contract that you have no way of verifying is actually happening other than "trust me dude".

Clearly I absolutely agree, having it on your own infrastructure is best for confidentiality. But even then, what about evil-maid attacks in the datacentre ? Unless you have your own datacentre, you're going to be in a shared colo facility ...

13 дней назадdiscuss

> I believe Nvidia chips have a secure way to run your model on other infra.

Yes. And its already on offer today.

See Tinfoil(US)[1] and Privatemode(Germany)[2]

Tinfoil have not been independently audited, it is somewhere on their long-term radar.

Privatemode have been thoroughly independently audited with documentation available on request.

[1]https://tinfoil.sh/ [2] https://www.privatemode.ai/

14 дней назадdiscuss

> What if you're the employer ("first engineer" etc.), and there are no practices yet?

In that scenario the practices will still come first. You're not going to be doing any coding or systems engineering until you've got compliance signed off. You're going to be spending lots of time with lawyers and compliance people.

> Fintech almost by definition sometimes includes doing things from scratch

Yes, but cut through the noise of the typical Fintech fancy website and app and you're still staring straight down the barrel of spending 80% of your time on regulatory compliance.

Try as you might there are only so many ways you can re-invent the wheel for dealing with hard-facts legislation.

14 дней назадdiscuss

> any ideas and practices presented

Unless its your job to architect stuff, in a financial firm you don't go looking around for ideas and practices.

You comply with your employer's practices end of story.

If you like looking up ideas and other people's practices then a heavily regulated environment is probably not the place for you.

> how does one get there without having a conversation

"having a conversation" about new ideas/practices in a regulated firm will involve lawyers and the compliance department.

More than likely that "conversation" will be above most people's pay grade. So you're better off just not wasting your time and adhering to your employer's existing practices.

And for everyone else, its an expensive and high-friction conversation to have if you want to change existing practices.

14 дней назадdiscuss

> even bad advice

That's putting it politely. Honestly, I think this "handbook" was mostly written by an LLM.

For example, in the immutability section we have this:

   "Separating PII from financial data lets you honor erasure without losing the financial history you’re obliged to keep."

In a financial organisation the two go hand-in-hand for obvious KYC/AML reasons.

Keeping the financial data whilst trashing the customer names, addresses etc. instantly on-demand before the expiry of the relevant time periods is going to leave your entire organisation with a very bad day in the office if a $lawful_body comes knocking for the data to trace a crime.

People going to work in a Fintech should not be relying on a random "Handbook" written by an unknown person in an unknown jurisdiction.

People going to work in a Fintech should only ever work in accordance with their employer's internal handbooks/guidelines/etc which will have been written in conjunction with their firm's lawyers and compliance people to ensure it complies with the laws and reporting requirements in the jurisdiction(s) in which their employer operates.

17 дней назадdiscuss

> tinfoil doesn't seem to distinguish between cached and input tokens.

Correct.

Privatemode correctly distinguish[1][2].

But with Tinfoil it appears to be a fundamental architectural limitation that they cannot.[3]

[1] https://www.privatemode.ai/pricing [2] https://www.privatemode.ai/blog/secure-prompt-caching [3] https://docs.tinfoil.sh/resources/caching

18 дней назадdiscuss

> because their pricing is part of their brand

I know people usually say that about Apple, but to be fair to them on this occasion they have not hiked up their prices yet because they are clearly at present still under some old deals that they did a good job negotiating.

However, of course, at some point Apple will run out of both inventory and old-pricing manufacturing capacity. Yes, I am fully expecting some sort of price-hike like has been seen everywhere else. I am not naïve.

When that time comes it will remain a financial calculation, Apple boxes on one side versus hosted-option-costs on another, in relation to my specific use-cases.

Ultimately I still blame the chip-hoarding hyperscalers though. :)

19 дней назадdiscuss

> We do want privacy, and we also want to own the hardware so the US can't just turn it off whenever it feels like it.

I agree and I prefer on-prem where possible. The Apple Mac Studios have been great for that although I don't have enough of them to run GLM-5.2 without heavy quantization. I'm also waiting for the Apple next product refresh which I hope will enable me to do more with less.

Meanwhile there are hosted privacy-conscious options out there. Two names to look at are Tinfoil[1] and Privatemode (from Edgeless Systems)[2].

Tinfoil[1] is, sadly, US-based. EU-sovereignty-option is on their long-term radar. But they do have GLM-5.2 today.

Privatemode[2] is a German company (Edgeless Systems) with EU-based servers. But sadly no GLM-5.2 today, it is on their mid-long term radar though.

Both Tinfoil and Privatemode operate on the same concept of the LLM operating in a secure enclave and you have end-to-end attestation and encryption.

Tinfoil have not been independently audited, it is somewhere on their long-term radar.

Privatemode have been thoroughly independently audited with documentation available on request.

Both of them are API-tokens-only. So if you're currently one of those people throwing $200 a month down the pan at Anthropic/OpenAI for a so-called-alleged 'unlimited' plan, then neither Tinfoil or Privatemode will be the place for you.

[1]https://tinfoil.sh/ [2] https://www.privatemode.ai/

20 дней назадdiscuss

> I think they're just suggesting renting as a way to test

Well, yes, I understood that.

Which is why I started with the words "You don't even need to go that far.".

To re-phrase what I said in clearer terms:

Instead of renting an instance, then messing around with configuring Linux and whatever via SSH or Ansible or whatever. Just point a Hugging Face link at this magic service and get a ready-to-go API back. Enabling you to test your desired model spec with minimum fuss.

Ultimately the guy wants his own hardware. So why waste time messing around with someone else's VM if you just want to test a specific model spec. That is the TL;DR.

20 дней назадdiscuss

> You probably want to try renting some time on a dedicated box with roughly the specs you’re considering and running the open models

You don't even need to go that far. For example, with Exoscale Dedicated Inference[1] you just point it at the Hugging Face for the model and quantisation you want to test and it automagically spits out an OpenAI-compatible API endpoint.

[1] https://www.exoscale.com/ai-cloud-infrastructure/dedicated-i...

(I have no relationship with Exoscale, this particular product just crossed my radar recently)

20 дней назадdiscuss

> We started SlicerVM ....

Shame you did not mention once in your long post that you are based on Firecracker, because I'm sure I'm not the first who was about to post "why is this better than Firecracker".

Also it is a shame you've adopted the subscription billing model instead of allowing people to buy perpetual licenses.

I dislike the subscription model in a pure sense, but also I dislike the "but its 'only' $x a month" argument oft-used by developers. Sure, in theory that's the case. But like everyone else in the world, I also have $x a month of other monthly expenses in my life, and I simply do not need or want N+1 software subscriptions. It all adds up.

The same applies to business environments, except the cost becomes even more exponential because you have (X-employees * N-subscriptions)/month.

22 дня назадdiscuss

> I only walked a few of the main streets

Person walks along main roads in London and complains they see no trees. Meanwhile in other news .... :)

London is one of the most tree-ridden cities on this earth, so I dread to think what "main streets" you were walking along.

24 дня назадdiscuss

> My electric company gave me a number (UID, not phone number) to resume a call if the issue wasn't fixed within 24 hours

What planet do you live on ?

Seriously. Where I am it is guaranteed that a utility company would never even consider such a concept let alone have the technical competence to actually implement it.

I'm jealous. ;)

24 дня назадdiscuss

> I've never seen an hotline where you can call back and resume the call you were doing.

Assuming they even accept inbound calls to the CLI number in the first place.

I frequently encounter companies where I miss a call due to $reason, I then try to call back the CLI number and it just says "This was $megacorp trying to call you, we will try again later".

Or, if you're really lucky, the CLI will just dump you into IVR-hell which, of course, is "AI powered" today, so you have to spend 30 minutes telling the stupid robot they mis-intepreted your voice.

24 дня назадdiscuss

> even network services benefit from things like OpenAPI for type safety, and you don't get that from the Go stdlib.

Sure, but the point is for the majority of people writing stuff in Go, they can get 99% of the way there with the Go stdlib.

Then, if they need to import one or two things to help them, such as the AWS Go SDK or whatever, that's perfectly fine.

It still means you end up with a go.mod file that has literally only two or three third-party imports in it.

Meanwhile if you wanted to write the equivalent tool in Rust, if you don't care you'll quickly end up with a Cargo.toml measured in hundreds of lines.

And if you are willing to put in the hours to hand-cherry-pick and make your Rust imports "reasonably necessary", then you'll still have a whole bunch more third-party imports than the Go equivalent.

24 дня назадdiscuss

> and instead tries to provide comprehensive support for low-level operations that enable you to build a custom-tailored solution to whatever you need on top of it

So in other words Rust forces you to either (a) re-invent the wheel (yet again !) or (b) import yet-another-crate into your project.

Of course if you choose (b), then its having first wasted your time deciding which of hundreds of yet-another-crate-doing-the-same-thing you want to import.

Its a waste of time and effort for the majority of projects where you are not working with embedded systems or trying to squeeze every micro-second of performance out of your system.

Sadly the majority of Rust projects show exactly zero import discipline, both from a pure import perspective and a security perspective. Which is why many Rust projects end up importing a gazillion crates.

Import discipline in Rust is hard work. Sure you can reach "reasonably necessary" level of imports the majority of Rust projects simply don't bother because its such a pain in the backside.

Don't get me wrong, Rust has its place in this world. But for many people on many projects they would be better off using Go and only using Rust where there is actually a serious use-case for it in their environment.

24 дня назадdiscuss

> it's only limited compared to Python

Erm ....

Its limited compared to Go as well.

And that's a BIG deal because Go gives you single binaries with a stdlib that allows you to hit the ground running in a serious manner.

For example, making API calls which is the sort of thing many here do for their bread and butter. Everything you need to do can be don in Go stdlib without opening yourself up to supply chain vulnerabilities or having to choose which crate or having to keep track of crates versioning. The same could be said of crypto or hundreds of other things present in the Go stdlib.

27 дней назадdiscuss

Fun but hugely unrealistic simulation, so many "bugs":

    - Able to split log into unrealistically thin slices and they remain perfectly upright
    - Split a log into two, rotate 90 degrees, and by some miracle you can split the half further away from you whilst the piece nearest to you doesn't get hit or move an inch

etc.