rwiggins

no profile record

comments

hace 12 mesesdiscuss

Hmm. I stick to jq for basically any JSON -> JSON transformation or summarization (field extraction, renaming, etc.). Perhaps I should switch to scripts more. uv is... such a game changer for Python, I don't think I've internalized it yet!

But as an example of about where I'd stop using jq/shell scripting and switch to an actual program... we have a service that has task queues. The number of queues for an endpoint is variable, but enumerable via `GET /queues` (I'm simplifying here of course), which returns e.g. `[0, 1, 2]`. There was a bug where certain tasks would get stuck in a non-terminal state, blocking one of those queues. So, I wanted a simple little snippet to find, for each queue, (1) which task is currently executing and (2) how many tasks are enqueued. It ended up vaguely looking like:

    for q in $(curl -s "$endpoint/queues" | jq -r '.[]'); do
        curl -s "$endpoint/queues/$q" \
        | jq --arg q "$q" '
            {
                "queue": $q,
                "executing": .currently_executing_tasks,
                "num_enqueued": (.enqueued_tasks | length)
            }'
    done | jq -s

which ends up producing output like (assuming queue 0 was blocked)

    [
        {
            "queue": 0,
            "executing": [],
            "num_enqueued": 100
        },
        ...
    ]

I think this is roughly where I'd start to consider "hmm, maybe a proper script would do this better". I bet the equivalent Python is much easier to read and probably not much longer.

Although, I think this example demonstrates how I typically use jq, which is like a little multitool. I don't usually write really complicated jq.

hace 12 mesesdiscuss

Oh wow, that's fantastic. I love that it includes real values while still summarizing the doc's structure. I'm going to steal that. I'll probably keep jq-structure around because it's so easy to copy/paste paths I'm looking for, but yours is definitely better for understanding what the JSON doc actually contains.

hace 12 mesesdiscuss

Oh, fantastic. jq has become an integral part of work for me.

I'll use this opportunity to plug the one-liner I use all the time, which summarizes the "structure" of a doc in a jq-able way: https://github.com/stedolan/jq/issues/243#issuecomment-48470... (I didn't write it, I'm just a happy user)

For example:

    $ curl -s 'https://ip-ranges.amazonaws.com/ip-ranges.json' | jq -r '[path(..)|map(if type=="number" then "[]" else tostring end)|join(".")|split(".[]")|join("[]")]|unique|map("."+.)|.[]'
    .
    .createDate
    .ipv6_prefixes
    .ipv6_prefixes[]
    .ipv6_prefixes[].ipv6_prefix
    .ipv6_prefixes[].network_border_group
    .ipv6_prefixes[].region
    .ipv6_prefixes[].service
    .prefixes
    .prefixes[]
    .prefixes[].ip_prefix
    .prefixes[].network_border_group
    .prefixes[].region
    .prefixes[].service
    .syncToken

(except I have it aliased to "jq-structure" locally of course. also, if there's a new fancy way to do this, I'm all ears; I've been using this alias for like... almost a decade now :/)

In the spirit of trying out jqfmt, let's see how it formats that one-liner...

    ~  echo '[path(..)|map(if type=="number" then "[]" else tostring end)|join(".")|split(".[]")|join("[]")]|unique|map("."+.)|.[]' | ~/go/bin/jqfmt -ob -ar -op pipe
    [
        path(..) | 
        map(if type == "number" then "[]" else tostring end) | 
        join(".") | 
        split(".[]") | 
        join("[]")
    ] | 
        unique | 
        map("." + .) | 
        .[]%
    ~

Not bad! Shame that jqfmt doesn't output a newline at the end, though. The errant `%` is zsh's partial line marker. Also, `-ob -ar -op pipe` seems like a pretty good set of defaults to me - I would prefer that over it (seemingly?) not doing anything with no flags. (At least for this sample snippet.)

el año pasadodiscuss

> how clear and useful it is. Perhaps, but do you really need all that?

Do I need clear and useful things? Maybe not. Would I like to have them anyway? Yes.

Years ago, I configured a Java project's logging framework to automatically exclude all the "uninteresting" frames in stack traces. It was beautiful. Every stack trace showed just the path taken through our application. And we could see the stack of "caused-by" exceptions, and common frames (across exceptions) were automatically cut out, too.

Granted, I'm pretty sure logback's complexity is anathema to Go. But my goodness, it had some nice features...

And then you just throw the stack trace in IntelliJ's "analyze stacktrace" box and you get clickable links to each line in every relevant file... I can dream.

> the wrapping is very greppable when done well

Yeah, that's my other problem with it. _When done well._ Every time I write an `if err != nil {}` block, I need to decide whether to return the error as is (`return err`) or decorate it with further context (`return fmt.Errorf("stuff broke: %w", err)`). (Or use `%v` if I don't want to wrap. Yet another little nuance I find myself needing to explain to junior devs over and over. And don't get me started about putting that in the `fmt` package.)

So anyway, I've seen monstrosities of errors where there were 6+ "statement: statement: statement: statement: statement: final error" that felt like a dark comedy. I've also seen very high-level errors where I dearly wished for some intermediate context, but instead just had "failed to do a thing: EOF".

That all being said, stack traces are really expensive. So, you end up with some "fun" optimizations: https://stackoverflow.com/questions/58696093/when-does-jvm-s...

el año pasadodiscuss

It's been so long that I barely remember the details, but yes, we used a gnomon to calculate several things... the earth's circumference, our local solar noon, latitude and longitude, etc.

I specifically remember that we measured how far the shadow moved over time using chalk. I think this was in lieu of having a second triangle somewhere else, although, for all I know we might have used a second reference distance.

Our campus had this elaborate outdoor "observatory" that had all sorts of interesting features. The gnomon was one of them, but there were other cool things, like IIRC there was a metal sculpture where Polaris would be seen through a central hole (and a guide inscribed in the concrete to figure out where to stand for that to happen -- I think it was based on viewing height).

el año pasadodiscuss

> Have you ever done the experiments to prove the Earth is round?

I have, actually! Thanks, astronomy class!

I've even estimated the earth's diameter, and I was only like 30% off (iirc). Pretty good for the simplistic method and rough measurements we used.

Sometimes authorities are actually authoritative, though, particularly for technical, factual material. If I'm reading a published release date for a video game, directly from the publisher -- what is there to contest? Meanwhile, ask an LLM and you may have... mixed results, even if the date is within its knowledge cutoff.

hace 2 añosdiscuss

Setting aside that you can enable mouse mode in vim and use the mouse to your heart's desire --

and setting aside the "declarative" vs "imperative" nomenclature debate --

vim is pretty dang efficient. Typing `gg` to go to the beginning of a file, or `G` to go the end. Or all the other variety of motions. There are a lot.

Also, there is not necessarily any need to move the cursor anywhere. For example, if I'm in the middle of writing some string and I've decided to change the whole thing, `<ESC>ci"` does that, with no cursor movement required.

hace 2 añosdiscuss

Yeah, having the read the article, my conclusion was that you should avoid honeycrisps from like February through ~August. i.e. only buy them when they're vaguely in season, or not too long afterward.

Anecdotally, I had some pretty delicious honeycrisps last night (in WA).

hace 2 añosdiscuss

Errr, not in the rural area I grew up in. Gravel driveways are super common, gravel roads not so much.

To give some specifics: I only remember driving down an actual gravel road (like, for public use) a single time. In 18 years. Even my friends who lived >30min from the nearest "city" (~10k population) had paved roads all the way.

But that is just my own experience. Areas with a different climate or geography might be a totally different story. My hometown area is relatively flat, lots of farmland, and rarely gets severe winter weather.

hace 2 añosdiscuss

Maybe area-dependent? I grew up in an extraordinarily rural area in Tennessee. Most roads were paved (asphalt). Even ones out in the middle of nowhere.

The conditions of some of the remote roads might not have been great, mind you... and some seemed "thinner" almost, maybe paved a long time ago?

hace 2 añosdiscuss

I felt the same way initially, but the language has grown on me. The turning point was writing a lot of Go, like full-time project work for a few months.

But as I've gotten older, I've started striving more and more for simplicity above all else, especially in systems design (disclaimer: I'm an SRE). Go is pretty good at being simple.

There are some things that still annoy me a whole bunch, though. Like - just one example - `fmt.Errorf` not being a first-class syntactic construct (or the difference between `%v` and `%w` in `fmt.Errorf`).

hace 2 añosdiscuss

It's a link to source code on GitHub. It could be marketing too, but it's absolutely, incontrovertibly source code.

hace 2 añosdiscuss

> show us the code if it's that easy to give to others

See https://security.apple.com/documentation/private-cloud-compu...

disclosure: work at Apple, opinions absolutely my own.

hace 2 añosdiscuss

I haven't used Terraform in years (because I changed jobs, not because of the tech itself), but back in the day v0.12 solved most of my gripes. I have always wished they'd implement a better "if" syntax for blocks, because the language itself pseudo-supports it: https://github.com/hashicorp/terraform/issues/21512

But yeah, at $previous_job, Terraform enabled some really fantastic cross-SaaS integrations. Stuff like standing up a whole stack on AWS and creating a statuspage.io page and configuring Pingdom all at once. Perfect for customers who wanted their own instance of an application in an isolated fashion.

We also built an auto-approver for Terraform plans based on fingerprinting "known-good" (safe to execute) plans, but that's a story for a different day.

hace 2 añosdiscuss

Out of curiosity, how would you describe TCP in these terms? Does the TCP stack's handling of sequence numbers constitute processing (on the client and server both I assume)? Which part(s) of a TCP connection could be described as delivery?

hace 2 añosdiscuss

Page not found - is that a private repo?

hace 2 añosdiscuss

The source is on GitHub:

utils.js, which contains the (de)serialization code: https://github.com/ccbikai/loooooooooooooooooooooooooooooooo...

tool.js, which does the serialization: https://github.com/ccbikai/loooooooooooooooooooooooooooooooo...

display.js, which does the deserialization and redirect: https://github.com/ccbikai/loooooooooooooooooooooooooooooooo...

hace 2 añosdiscuss

Unlike SO, it's common to have very situation-specific questions posted on YAQS. In fact, my team preferred random one-off questions to go through YAQS (our contact golink pointed you to a monitored YAQS queue) precisely because they're much more searchable (and scalable) than point-to-point chats.

So yes, searching for your GShoe error, and (assuming you found nothing) asking about it on YAQS is not a bad way to get help from some random faraway team.

I suppose it's partially because most team chats are locked down (invite-only). In a company with a reasonably open slack, you might be able to ask in #gshoe-team or search it for relevant conversations, but not at Google in my experience - and this is setting aside the issue of message retention.

BTW, I agree 24h retention was truly ridiculous. Most of my colleagues hated it - fortunately (probably as a result of this legal case!) they disabled it and now the default is 30d everywhere.

Regarding promo, community contributions are still very much an expectation. Being active on YAQS counts toward that. True, the promo committee isn't going to go looking for it, so your manager needs to agree YAQS is a level-appropriate community contribution and include that in your promo packet.

Disclosure: I left Google like, a couple weeks ago

hace 2 añosdiscuss

Exactly this. It's not uncommon to ask a coworker how to do something, get a response, and then send them a follow-up CL adding that information to a playbook or doc somewhere so others can reference it.

Heck, sometimes I responded to questions with a CL adding that info to docs.

The internal search engine helps drive a lot of this, too - if you want to know how to do something, docs (via search) are like the #1 choice. So, everyone's pretty incentivized to make it a good resource.

hace 2 añosdiscuss

Best/worst. I worked at an e-learning company that, thanks to ancient e-learning standards, really tried hard to figure out when a user is leaving a web-page to handle session exits gracefully.

sendBeacon was dead on arrival, for that purpose at least; iirc it's blocked by most ad blockers. (Or was, who knows with Manifest v3.)

Similarly we're unlikely to see sendBeacon truly replace tracking redirects, unfortunately. This isn't an area where tech firms are happy to see fine user controls.