fmstephe · HackerTrans

vor 2 Monatendiscuss

    Location: New Zealand, Manawatu
    Remote: Yes
    Willing to relocate: No
    Technologies: Go, Java, C#, Git, Erlang, Postgres, Linux
    Resume: https://www.linkedin.com/in/francis-stephens/
    Email: [email protected]

I work primarily on backend systems, with a strong focus on performance and system stability/resilience. I worked as a performance engineer at the mobile add-attribution company Adjust. Some interesting open-source projects include https://github.com/fmstephe/memorymanager An exploratory manual memory allocator for building large in-memory data structures with near zero GC cost.

https://github.com/fmstephe/matching_engine A financial trading matching engine with a somewhat novel red+black tree implementation.

https://github.com/fmstephe/flib A set of packages primarily in support of a lock-free single-producer single-consumer queue.

My ideal position would be working on backend systems primarily in Go.

fmstephe

vor 3 Monatendiscuss

    Location: New Zealand, Manawatu
    Remote: Yes
    Willing to relocate: No
    Technologies: Go, Java, Git, Erlang, Postgres, Linux
    Resume: https://www.linkedin.com/in/francis-stephens/
    Email: [email protected]

I work primarily on backend systems, with a strong focus on performance and system stability/resilience. I worked as a performance engineer at the mobile add-attribution company Adjust. Some interesting open-source projects include

https://github.com/fmstephe/memorymanager An exploratory manual memory allocator for building large in-memory data structures with near zero GC cost.

https://github.com/fmstephe/matching_engine A financial trading matching engine with a somewhat novel red+black tree implementation.

https://github.com/fmstephe/flib A set of packages primarily in support of a lock-free single-producer single-consumer queue.

My ideal position would be working on backend systems primarily in Go.

fmstephe

vor 4 Monatendiscuss

    Location: New Zealand, Manawatu
    Remote: Yes
    Willing to relocate: No
    Technologies: Go, Java, Git, Erlang, Postgres, Linux
    Resume: https://www.linkedin.com/in/francis-stephens/
    Email: [email protected]

I work primarily on backend systems, with a strong focus on performance and system stability/resilience. I worked as a performance engineer at the mobile add-attribution company Adjust.

Some interesting open-source projects include

https://github.com/fmstephe/memorymanager An exploratory manual memory allocator for building large in-memory data structures with near zero GC cost.

https://github.com/fmstephe/matching_engine A financial trading matching engine with a somewhat novel red+black tree implementation.

https://github.com/fmstephe/flib A set of packages primarily in support of a lock-free single-producer single-consumer queue.

My ideal position would be working on backend systems primarily in Go.

fmstephe

vor 5 Monatendiscuss

Last time this was asked I was working on this

https://github.com/fmstephe/simd_explorer

A little TUI app for interactively running different SIMD instructions and seeing the outputs.

Since then I have completed the tool for AVX/2. At this stage that's as far as I intend to go.

It's potentially valuable as an interactive quick reference guide for SIMD instructions.

It works on Windows, Linux and with the right environment variables it will successfully pretend to be AMD64 running on an Apple M chip.

Arm NEON instructions are not supported at all, currently Go's assembler does not include these instructions directly, so I didn't attempt to build for them. Maybe one day.

Next up, learn Zig - be happy.

fmstephe

vor 6 Monatendiscuss

Can some clarify this part of the article for me

"if you search forward, you need to scan through the entire window to find where to split. you’d find a delimiter at byte 50, but you can’t stop there — there might be a better split point closer to your target size. so you keep searching, tracking the last delimiter you saw, until you finally cross the chunk boundary. that’s potentially thousands of matches and index updates."

So I understand that this is optimal if you want to make your chunks as large as possible for a given chunk size.

What I don't understand is why is it desirable to grab the largest chunk possible for a given chunk limit?

Or have I misunderstood this part of the article?

fmstephe

vor 7 Monatendiscuss

If anyone wants to try it out (the UI is a bit rough). I will try fix up any issues that are uncovered.

fmstephe

vor 7 Monatendiscuss

Working on a TUI tool which demonstrates the behaviour of X86 SIMD instructions. This is all done in Go assembly, and is probably most valuable for Go programmers.

The problem for me was trying to read and understand the implementation of a swiss map implementation. The SIMD instructions were challenging to understand and the documentation felt difficult to read. I thought that if I had an interactive tool where I could set the inputs to a SIMD instruction and then read the outputs, understanding the instructions would be much easier.

This turned out to be true.

Building this tool for all AVX/AVX2 instructions turned out to be a larger task than I had expected. Naively I just went off a Wikipedia page on AXV and assumed it had listed all the instructions (this was a bad assumption).

I am nearly there. Looking forward to completing this project so I can actually use it to do some fun stuff processing text and maybe even get back to that swiss map implementation.

https://github.com/fmstephe/simd_explorer

(This is also my first attempt at a TUI app)

fmstephe

vor 7 Monatendiscuss

Location: New Zealand, Manawatu Remote: Yes Willing to relocate: No Technologies: Go, Java, Git, Erlang, Postgres, Linux Resume: https://www.linkedin.com/in/francis-stephens/ Email: [email protected] I work primarily on backend systems, with a strong focus on performance and system stability/resilience. I worked as a performance engineer at the mobile add-attribution company Adjust. Some interesting open-source projects include https://github.com/fmstephe/memorymanager An exploratory manual memory allocator for building large in-memory data structures with near zero GC cost. https://github.com/fmstephe/matching_engine A financial trading matching engine with a somewhat novel red+black tree implementation. https://github.com/fmstephe/flib A set of packages primarily in support of a lock-free single-producer single-consumer queue. My ideal position would be working on backend systems primarily in Go.

fmstephe

vor 8 Monatendiscuss

Location: New Zealand, Manawatu Remote: Yes Willing to relocate: No Technologies: Go, Java, Git, Erlang, Postgres, Linux Resume: https://www.linkedin.com/in/francis-stephens/ Email: [email protected] I work primarily on backend systems, with a strong focus on performance and system stability/resilience. I worked as a performance engineer at the mobile add-attribution company Adjust. Some interesting open-source projects include

https://github.com/fmstephe/memorymanager An exploratory manual memory allocator for building large in-memory data structures with near zero GC cost.

https://github.com/fmstephe/matching_engine A financial trading matching engine with a somewhat novel red+black tree implementation.

https://github.com/fmstephe/flib A set of packages primarily in support of a lock-free single-producer single-consumer queue.

My ideal position would be working on backend systems primarily in Go.

fmstephe

letztes Jahrdiscuss

    Location: New Zealand, Manawatu
    Remote: Yes
    Willing to relocate: No
    Technologies: Go, Java, Git, Erlang, Postgres, Linux
    Resume: https://www.linkedin.com/in/francis-stephens/
    Email: [email protected]

I work primarily on backend systems, with a strong focus on performance and system stability/resilience. I worked as a performance engineer at the mobile add-attribution company Adjust.

Some interesting open-source projects include

https://github.com/fmstephe/memorymanager An exploratory manual memory allocator for building large in-memory data structures with near zero GC cost.

https://github.com/fmstephe/matching_engine A financial trading matching engine with a somewhat novel red+black tree implementation.

https://github.com/fmstephe/flib A set of packages primarily in support of a lock-free single-producer single-consumer queue.

My ideal position would be working on backend systems primarily in Go.

fmstephe

vor 9 Jahrendiscuss

Good question, my claim was _very_ vague. The current GC in Go is inefficient in its use of CPU.

Specifically because it uses a non-moving collector it has a sweep phase which frees each of the dead allocations.

In Java, or similarly .net, the use of a moving collector means that dead objects aren't freed. The live objects are moved out of the current memory region and the whole region is then 'free'. If you have few live allocations and lots of dead allocations in that region then your GC cycle is much more efficient.

I can't comment on memory usage experiences. I have written very careful Java programs that ran on the 64bit hotspot server in ~40mb or memory. I've written Java programs that used gigs, and I've written the same range in Go.

I would like to quickly note that I actually like the Go GC. I think they are on a very promising path to a potentially great GC. But they are also given to some public hyperbole which I find awkward.

fmstephe

vor 9 Jahrendiscuss

Can you expand on

"Also, the generational hypothesis has nothing to do with whether compaction is profitable. These are orthogonal things."

It is my understanding that these two are strongly linked, at least in practice.

The generational hypothesis states that most (even overwhelmingly most) allocations are unreachable very quickly.

So we employ moving/compacting collectors so that we no longer need to sweep all those dead allocations. These dead allocations are freed at no cost and we only pay for copying the live set.

Having written that, it occurs to me that the sentence I quote read oddly to me because I conflated 'compaction' with 'moving'.

Happy to be corrected on anything written above.

fmstephe

vor 9 Jahrendiscuss

I would like to add another choice quotation from that discussion.

"It is common to see early GC-based runtime implementations that do not move objects around succumb to an architectural "hole" that creates a reliance on that fact early in the maturity cycles. This usually happens by allowing native code extensions to assume object addresses are fixed (i.e. that objects will not move during their lifetime) and getting stuck with that "assumed quality" in the long run due to practical compatibility needs. From what I can tell, it is not too late for Go to avoid this problem."

Previously I was not concerned about the long-term outlook for Go's GC. It's low pause (with some pathological cases) and currently very inefficient. The long term plan had previously mentioned moving to a generational/moving collector in the future. Gil's endorsement was cheering.

But, Ian's comments on non-moving collector's on the Golang-nuts mailing list were alarming (and seemed technically confused). Time will tell.