HackerTrans
TopNewTrendsCommentsPastAskShowJobs

mwsherman

no profile record

Submissions

Making Unicode things fast in Go

clipperhouse.com
2 points·by mwsherman·9 tháng trước·0 comments

Show HN: Go Allocations Explorer for VS Code

marketplace.visualstudio.com
4 points·by mwsherman·10 tháng trước·0 comments

Allocations Are a Dependency

clipperhouse.com
1 points·by mwsherman·10 tháng trước·0 comments

I deal with copyright claims [video]

youtube.com
2 points·by mwsherman·10 tháng trước·0 comments

comments

mwsherman
·6 tháng trước·discuss
While this article is about perf — and trading off semantic precision by design — there is a Unicode standard for sentence boundaries, may be interesting: https://www.unicode.org/reports/tr29/#Sentence_Boundaries

I implemented the sentence boundaries, but also thought that the notion of a “phrase” might be useful for such applications: https://github.com/clipperhouse/uax29/tree/master/phrases
mwsherman
·7 tháng trước·discuss
Shameless plug, if one wishes to track down allocations in Go, an allocations explorer for VS Code: https://marketplace.visualstudio.com/items?itemName=Clipperh...
mwsherman
·7 tháng trước·discuss
I’ve found C#’s frozen dictionary to be useful: https://learn.microsoft.com/en-us/dotnet/api/system.collecti...

It’s optimized for fast reads in exchange for expensive creation.
mwsherman
·9 tháng trước·discuss
That’s true, converting in either direction will typically allocate. Which it must, semantically.

One can use unsafe for a zero-copy conversion, but now you are breaking the semantics: a string becomes mutable, because its underlying bytes are mutable.

Or! One can often handle strings and bytes interchangeably with generics: https://github.com/clipperhouse/stringish
mwsherman
·9 tháng trước·discuss
There is mention of how len() is bytes, not “characters”. A further subtlety: a rune (codepoint) is still not necessarily a “character” in terms of what is displayed for users — that would be a “grapheme”.

A grapheme can be multiple codepoints, with modifiers, joiners, etc.

This is true in all languages, it’s a Unicode thing, not a Go thing. Shameless plug, here is a grapheme tokenizer for Go: https://github.com/clipperhouse/uax29/tree/master/graphemes
mwsherman
·9 tháng trước·discuss
In Go, string effectively serves as a read-only slice, if we are talking about bytes.

ReadOnlySpan<T> in C# is great! In my opinion, Go essentially designed in “span” from the start.
mwsherman
·9 tháng trước·discuss
A Go allocations explorer for VS Code. Extension: https://marketplace.visualstudio.com/items?itemName=Clipperh...

Source: https://github.com/clipperhouse/go-allocations-vsix
mwsherman
·9 tháng trước·discuss
Shameless plug, you may wish to do Lucene-style tokenizing using the Unicode standard: https://github.com/clipperhouse/uax29/tree/master/words
mwsherman
·10 tháng trước·discuss
I move between Go and C#. I wrote a zero-allocation package in Go [1] and then ported to C# — and the allocations exploded!

I had forgotten, or perhaps never realized, that substrings in C# allocate. The solution was Spans.

Notably, it caused me to realize that Go had “spans” designed in from the start.

[1] https://github.com/clipperhouse/uax29