dave_universetf

vorige maanddiscuss

The readme has strong LLM smells. Was the code written by an LLM as well?

What is your experience with cryptographic engineering, in particular avoiding common implementation pitfalls that bite first-time implementers of cryptographic primitives?

Are the primitives tested against Wycheproof vectors, and proofed against the common implementation mistakes they document?

dave_universetf

2 maanden geledendiscuss

It's vibe coded, so not surprising that it replicates the patterns of existing systems.

dave_universetf

3 maanden geledendiscuss

+1, the information content is nice, but the AI telltales and writing patterns were annoyingly distracting.

dave_universetf

3 maanden geledendiscuss

This is probably just a difference in how your brain and the author's work. A variety of home organizers have told me that people mean different things when they say they want an organized space. Some people want everything precisely labeled and sorted into narrow categories, and hidden away in drawers or closets. Others want everything visible and coarser categories. Each system looks and feels very distressing to brains of the other types.

It's especially a problem for people with ADHD, because the "very sorted and hidden" mode of organizing is heavily socialized as the _only_ way to be organized, but it's also the exact opposite of how (some) ADHD brains want to operate. OTOH the very exposed and "emergent" organization that works for an ADHD brain probably is mild torture to an OCD brain :)

For myself, the sorting system in this post looks pretty ideal. All the stuff is right there where I can see it and scan for what I'm after, it explicitly allows for emergent organization where classification happens incrementally over time, and the dots thing has near zero activation energy but still gives me long-term information I can use. It's much better than an electronic or "clean" inventory system precisely because I'll never be able to consistently keep using those, whereas slapping a dot on a box, even on bad brain days I can manage that!

dave_universetf

5 maanden geledendiscuss

Strictly speaking: no. The "forward pass" terminology does not imply that there exists a "reverse pass" that does the same kind of computation. Rather, it's describing two different kinds of computation, and the direction they occur in.

The forward pass is propagating from inputs to outputs, computing the thing the model was trained for. The reverse/backwards pass is propagating from outputs back to inputs, but it's calculating the gradients of parameters for training (rougly: how much changing each parameter in isolation affects the output, and whether it makes the output closer to the desired training output). The result of the "reverse pass" isn't a set of inputs, but a set of annotations on the model's parameters that guide their adjustment.

The computations of the forward pass are not trivially reversible (e.g. they include additions, which destroys information about the operand values). As a sibling thread points out, you can still probabilistically explore what inputs _could_ produce a given output, and get some information back that way, but it's a lossy process.

And of course, you could train a "reverse" model, one that predicts the prefix of a sequence given a suffix (trivially: it's the same suffix prediction problem, but you train it on reversed sequences). But that would be a separate model trained from scratch on that task, and in that model the prefix prediction would be its forward pass.

dave_universetf

5 maanden geledendiscuss

Both, with caveats. The attention computation is fundamentally quadratic: for every token in the sequence, you're doing a computation that has to compute over every other token in the sequence. So it's O(N) per token, O(N^2) for the whole sequence.

The big mitigation for this is that in causal transformers (i.e. all the chatbot type applications, where each token is only allowed to see tokens before it), you're running inference repeatedly on the same prefix in order to grow it by one token at a time. So if you cache the computations for tokens 0..N-1, on each inference pass you only have to compute O(N) for the newly added token at the end of the sequence.

That's why caching (and caching charges) appear so prominently everywhere in the pricing of inference.

In practice, caching is most beneficial at inference time, because you typically have relatively long conversations that start with the same cacheable prefix (the system prompt). At training time the same optimization can apply, but you're typically not pushing the same prefixes through the model repeatedly so you end up paying the quadratic cost more often.

The quadratic cost of attention is the fundamental compute bottleneck for transformer architectures, which is why there's research like this trying to find shortcuts in computing attention, as well as research into completely new primitives to replace attention (e.g. SSM, which is O(N) on a cold cache and O(1) on a warm cache).

dave_universetf

7 maanden geledendiscuss

I don't know about all hyperscalers, but I have knowledge of one of them that has a large enough fleet of atomic frequency standards to warrant dedicated engineering. Several dozen frequency standards at least, possibly low hundreds. Definitely not one per machine, but also not just one per datacenter.

As you say, the goal is to keep the system clocks on the server fleet tightly aligned, to enable things like TrueTime. But also to have sufficient redundancy and long enough holdover in the absence of GNSS (usually due to hardware or firmware failure on the GNSS receivers) that the likelihood of violating the SLA on global time uncertainty is vanishingly small.

The "global" part is what pushes towards having higher end frequency standards, they want to be able to freewheel for O(days) while maintaining low global uncertainty. Drifting a little from external timescales in that scenario is fine, as long as all their machines drift together as an ensemble.

The deployment I know of was originally rubidium frequency standards disciplined by GNSS, but later that got upgraded to cesium standards to increase accuracy and holdover performance. Likely using an "industrial grade" cesium standard that's fairly readily available, very good but not in the same league as the stuff NIST operates.

dave_universetf

11 maanden geledendiscuss

I wrote MetalLB, a bare metal load-balancer for Kubernetes, because I needed one for myself. It gained some popularity because for a couple years, it was the only way to get working L4 LB outside of clouds. These days I believe a couple of the CNIs added support for external BGP peering and integration with k8s's LB machinery, but that came years later.

As a result, I became network troubleshooting tech support for a large chunk of people trying to run kubernetes on bare metal. If you've not looked at k8s's networking, debugging even your own cluster's networking is a nightmare, never mind debugging someone else's over slack, while (usually) simultaneously having to give them a crash course in intermediate/advanced networking stuff like asymmetric routing and tracing packets through netfilter so that you can then tell them that networks can't do the thing they wanted and no amount of new features I can add will change that.

Meanwhile companies selling bare metal k8s services started bundling MetalLB, but kept sending their customers to my bugtracker instead of taking some of the load themselves.

The experience burned me out severely. It's been several years and I still have a visceral negative reaction to the idea of open-sourcing code I wrote, and when I infrequently do they come with a fairly blunt "contributions no welcome" message and a disabled issue tracker. I handed over the keys to MetalLB a long while back now. I hope the new maintainers and the project are doing okay.

I'll mention a positive of that time as well, to balance it out: as an experiment I opened a pinned issue asking happy users to drop me a note (https://github.com/metallb/metallb/issues/5), and many did. It was nice occasionally getting a notification that wasn't a complaint or demand for support. At one point someone left me a note that it was being used to support research projects at NASA JPL and DARPA. That was pretty neat.

dave_universetf

4 jaar geledendiscuss

Trust Report seems to be irrelevant to this (from what I can tell from the brochure without being a vanta customer), because it's a way for a company to publish claims about itself. Crucially, nowhere does it say that an independent auditor verified those claims.

SOC2 broadly contains:

  - A description of what the company claims to do

  - A statement that the description is complete and accurate

  - Auditor's testing procedure for verifying the company's claims

  - The results of the testing

  - The auditor's overall conclusion as to whether the company meets the bar for SOC2.

Trust Report seems to only cover the first point.

dave_universetf

4 jaar geledendiscuss

This is a restriction of SOC2 itself. SOC2 produces a "restricted use" report, meant for the client company who purchased it, and for limited access by third parties that do business with the client company.

AIUI, the intent is to ensure that the people reading the SOC2 report don't read into it more than what the auditor intended. This, according to the auditors, is fraught with difficulty, because you need a degree of familiarity with COSO principles and general SOC-ese to understand precisely what the auditor is making strong claims about, and what is _not_ being claimed.

In practice, the way this is accomplished is that the SOC2 report includes standard verbiage saying that the report is only for the client company, and for specific third-parties who must have "sufficient understanding" to parse the report correctly.

The client company then implements some process, with the guideline of "do something that won't piss off your auditor". For small companies, by and large the implementation is "you have to be an existing or prospective customer, email to ask for the report, and sign an NDA." The very act of requesting a SOC2 report implies that you know how to read a SOC2 correctly.

Some larger companies set up portals where you can help yourself to their various reports, sometimes without NDA - but in that case you have to click through a pinky-promise to not redistribute, and the report is watermarked with your identity to deter distribution. A very few publish the report at a hidden URL and hand out the URL to anyone who emails in to ask for it, though I personally think that's walking a bit too close to the edge of what they agreed to with auditors.

Companies with deeper pockets end up paying an auditor extra for a SOC3 report, which is "SOC2, but abridged and with unrestricted distribution rights". I believe the theory (besides giving more money to the auditor) is that the SOC3 removes all the information that might be misinterpreted, boiling it down to barely more than "I, a trustworthy auditor, confirm that the company is doing all the right things." You don't get much detail, but as long as you're willing to transitively trust the auditor (who are themselves scrutinized by their regulatory bodies), that gives you a "compliance is yes" document that you can publish far and wide.

If you want an example of what a SOC3 says, Github has one: https://github.githubassets.com/images/modules/site/security...

dave_universetf

4 jaar geledendiscuss

Our epic treatise on how NAT traversal works (in general, not specific to Tailscale) mentions this. IPv6 greatly reduces the amount of pain for p2p connections, but does not eliminate some of the fundamentals (stateful firewall traversal) if you want it to be zero-config: https://tailscale.com/blog/how-nat-traversal-works/

But until deployment hits 100%, and until ISPs start caring about IPv6 reliability the way they do about IPv4, "just use IPv6" can't be your answer. It's lovely when it works, but you need to do something other than "give up" when it doesn't. (also, as long as the internet is dual-stacked, doing IPv6 right also implies figuring out if NAT64 is in play, and wielding it correctly; so arguably IPv6 adds more complexity to the overall story, for now :) )

dave_universetf

6 jaar geledendiscuss

AUR package maintainer here (also tailscale employee). The poor experience was definitely on me. Relaynode's initial setup flow is a bit weird, and I didn't make the package explain anything. I think you were the first user of the Arch package other than me, so you got to experience the fun :)

Next release will have a better daemon, with a more typical setup flow. If you want to test drive it, `tailscale-unstable-bin` is the AUR package for it.

comments