HackerTrans
TopNewTrendsCommentsPastAskShowJobs

nprescott

no profile record

comments

nprescott
·3 lata temu·discuss
This reminded me of a prior discussion[0] on bulk data generation in SQLite with Rust (vs Python vs PyPy) which previously led me to trying out two different techniques using just SQLite[1]. The approach here is so similar I tried my prior solution on the slowest VPS I have access to (1 vCPU core 2.4GHz, 512Mi, $2.50/month from vultr):

    sqlite> create table users (id blob primary key not null, created_at text not null, username text not null);
    sqlite> create unique index idx_users_on_id on users(id);
    sqlite> pragma journal_mode=wal;
    sqlite> .load '/tmp/uuid.c.so'
    sqlite> .timer on
    sqlite> insert into users(id, created_at, username)
            select uuid(), strftime('%Y-%m-%dT%H:%M:%fZ'), 'hello'
            from generate_series limit 100000;
    Run Time: real 1.159 user 0.572631 sys 0.442133
where the UUID extension comes from the SQLite authors[2] and generate_series is compiled into the SQLite CLI. It is possible further pragma-tweaking might eke out further performance but I feel like this representative of the no-optimization scenario I typically find myself in.

In the interest of finding where the bulk of the time is spent and on a hunch I tried swapping the UUID for plain auto-incrementing primary keys as well:

    sqlite> insert into users(created_at, username) select strftime('%Y-%m-%dT%H:%M:%fZ'), 'hello' from generate_series limit 100000;
    Run Time: real 0.142 user 0.068090 sys 0.025507
Clearly UUIDs are not free!

[0]: https://news.ycombinator.com/item?id=27872575

[1]: https://idle.nprescott.com/2021/bulk-data-generation-in-sqli...

[2]: https://sqlite.org/src/file/ext/misc/uuid.c
nprescott
·3 lata temu·discuss
PEP-508[0] explains the grammar for "extras":

> Optional components of a distribution may be specified using the extras field:

  identifier_end = letterOrDigit | (('-' | '_' | '.' )* letterOrDigit)
  identifier    = letterOrDigit identifier_end*
  name          = identifier
  extras_list   = identifier (wsp* ',' wsp* identifier)*
  extras        = '[' wsp* extras_list? wsp* ']'
as well as explaining their behavior, albeit briefly:

> Extras union in the dependencies they define with the dependencies of the distribution they are attached to.

The resolution on . is explained by the pip documentation[1]:

> pip looks for packages in a number of places: on PyPI (if not disabled via --no-index), in the local filesystem, and in any additional repositories specified via --find-links or --index-url. There is no ordering in the locations that are searched. Rather they are all checked, and the “best” match for the requirements (in terms of version number - see PEP 440 for details) is selected.

[0]: https://peps.python.org/pep-0508/#grammar

[1]: https://pip.pypa.io/en/stable/cli/pip_install/#finding-packa...
nprescott
·4 lata temu·discuss
ZFS has been on my list of things I should learn "some day" for a long time. I installed Fedora a while ago and ended up opted into Btrfs without really trying. I have been pleasantly surprised with it and have used it on 2 low-impact servers and 3 personal computers for about a year.

I don't tend to do anything very interesting to tax the filesystem so it is most important that things just work day to day and the less maintenance the better. I found a few use cases[0] for the copy-on-write abilities of Btrfs (built into systemd-nspawn) after switching but even that was a case where something I was previously doing was made instantly faster rather than working out a btrfs incantation.

I've recently been trying out a nifty backup tool[1] to try out a snapshot feature locally. This one did require two btrfs specific commands to restore a backup (`btrfs {send,receive}`). Once again though, this was an additional feature I decided to try out and not really required for my day to day. For me personally Btrfs has been easy to adopt because of how low-friction it has been to add and learn incrementally.

I've still not learned ZFS.

[0]: https://idle.nprescott.com/2022/systemd-nspawn-and-btrfs.htm...

[1]: https://digint.ch/btrbk/
nprescott
·4 lata temu·discuss
I've had success using https://git-rebase.io/ to help people understand the workflow.