redsaz · HackerTrans

năm ngoáidiscuss

Thankfully the episode's interpretation of a Good Samaritan Law is completely wrong, at least in many (most?) US states. They protect citizens acting in good faith to help somebody, not like it was interpreted in the episode to require somebody to help.

redsaz

năm ngoáidiscuss

How should you conduct the interview, then, if:

... Classic interviewing techniques of "explain how X algo works" or "write code to solve Y" will unfairly bias against interviewees that don't test well under pressure, but would otherwise be a good coworker.

... "Teach me something interesting to you" or "tell me stories about past experiences" will unfairly bias against interviewees that are shy, soft spoken, or are mildly socially awkward, but would otherwise be a good coworker.

Given the above awareness of where bias can emerge, how should the interview be done in order to get a candidate that knows what they're doing, and works well with the rest of the team?

Other comments mention relying more on recruiters and referrals, but that isn't always an option.

redsaz

2 năm trướcdiscuss

tl;dr: Neanderthals needed to go clubbing more.

redsaz

2 năm trướcdiscuss

> The only thing missing is a game

I've seen this story of "I want to make a game" [proceeds to make a game engine instead] happen in my own life (my engines were never any good or complete though), and in countless other programmers lives.

It may be the trap of thinking that "If I get the hard part out of the way first (which is writing the engine code, right? Right? Anyone?), then the rest of the game making process will be easy" that gets me.

Or maybe it's finding out along the way that it was more fun to make the engine than the game itself: "check it out, I completely redid the particle effects and I can now do 100x more particles at 60fps, how cool is that?"

There's way more easier-to-see improvements in the making of the engine, than in the making of the game itself, and so we (ok, I) keep optimizing the engine because those are quick dopamine payoffs compared to the slower payoff of having a polished game that's actually fun for the target audience to play. Sure, I might tell myself that the game I want is only possible once I have the engine first, so I'd better concentrate on that before making the actual game, and there's some logic to it. But without a clear idea of what the game will actually be, it's easy to fall into the trap of endlessly adding and refining features, rather than actually try and use those features in anything beyond a slick demo.

To combat the tendency of only making an engine rather than a game in my latest hobby project, I picked an already existing engine (Phaser js) and tried to get something interactive on the screen ASAP "with the stupidest, least designed code possible", and it mostly worked to get me a playable (ish) game! Granted, it's a knockoff puzzle game but hey, I sometimes find myself "playtesting" it instead of what I should really be doing, which is refactoring the code for what I'd like to have it do next, so I'm marking it as a win.

redsaz

2 năm trướcdiscuss

[flagged]

redsaz

2 năm trướcdiscuss

> empirically measured that completing a task... is twice as fast in [Rust] than in [C++]

I have not read up on which tasks you're referring to that are empirically measured, apologies. The reason I'm curious on what the tasks are, is that depending on the task, navigability may not matter.

For example, if the task is "build a tool that does X", then navigability of the code does not matter. Once built, the tool does X, and there's no reason to revisit the code, and thus no reason to navigate the code.

But if the task is "Given a tool that already does W, X, Y, make the tool also do X', Y', and Z", then navigability of the code matters. This is because the coder must understand what the tool already does, and where the changes need to be made.

Most of my professional life, (and I'm willing to bet, most other coders here as well) I more often find myself in the second task than the first.

But, I'm not interested in Rust vs C++. I'd be more interested in the results of "given a version that makes high use of type inference vs not, how quickly can someone new to the project add X', Y', and Z." That would be a more appropriate test for what the author describes here. And I'd imagine that probably, those that are using sufficiently advanced IDEs would beat out those without, regardless of if type inference used or not, and would probably be slightly faster when given the highly type-inferenced version.

redsaz

2 năm trướcdiscuss

> This post can definitely be considered a “religious” opinion piece

The author certainly has that right, because the post steps on two programming religion landmines, from how I read it:

1. strict static typing (without type inference) is good. 2. code should be written to allow IDEs to enhance navigability, rather than written on the assumption that IDEs will be the sole provider for navigability.

I believe there is a point to be made in the "when we don't know what we're getting back, that harms navigability" camp. But as another commenter posted, there's a point to be made in the "when we overspecify what we're getting back every time, that can harm readability, too" camp.

I can't express where this balance is. It's somewhere between poetry and a legal document, the prose where you can really get into a good book and enjoy the world that the author presents. Some people really like the beauty of a short poem. Other people may require precise wording that leaves no room for enjoyment or interpretation. The rest of us can have the majority of fun somewhere in between.

Where that "in between" equivalent would be in my day-to-day programming, I'm not entirely sure, because what I'm writing could be a short script where brevity is vital (poetry-ish) vs some section of unfortunately highly complex code with lots of tests for edge cases (legalese), and all the other code where I'm still world-building and conveying ideas (prose). And I believe that complexity should be spelt out as precisely as it can in the code itself, rather than rely on the hope that somebody else is using the same IDEs and features as me. I've tried using type inference where it seems fine to use, and then spelling out the exact type that a variable wants where it isn't clear what might get returned, all in the same app, but it comes across as sloppily inconsistent in my mind. Ah well.

redsaz

2 năm trướcdiscuss

We have a similar saying: "You can lead a horse to water but you can't make 'em drink." At least, it seems similar to me.

redsaz

2 năm trướcdiscuss

Answer: because they don't want to be promoted to manager, they'd rather be coding.

redsaz

3 năm trướcdiscuss

> But the ideal tool should also try to handle the opposite case well too... a huge collection of unique ~100KiB files.

and

> But the pathological case is even worse -- what if nearly all of your files are about the same size?... If you want to quickly verify none of the files are dups, reading one block from each is far more efficient than hashing them all.

I agree this is a worthy consideration. No sense in reading the entirety of each of those files when only reading the first block would do, in order to remove the uniques early. If I were to redo the util, it'd probably be something like:

1. Group all files into lists of same file sizes.

2. After all files are read, eliminate any groupings with just one file, these are unique files

3. Read first N bytes to pare down files in those lists, (so now the key is filesize and hash-of-first-N-bytes or even filesize and first-N-bytes if N is small enough, either way)

4. After each filesize-group is subgrouped by first-N-bytes evaluation, eliminate any subgroupings with just one file, these are unique.

5. What remains are files fairly likely to be duplicates.

5a. For users that consider this "good enough", allow them to stop here. (Some deduper tools do this)

5b. For everybody else, in order to make sure the files are dupes or not, the files can next be subgrouped by fully byte-for-byte comparison and/or hashed, whatever the user is good with.

6. The remaining groupings of two or more files are dupes.

In the end I opted not to go for this rewrite, at least not yet, because I got sidetracked thinking about how the whole reason I'm doing this in the first place is because the way I've backed up data across all my machines for years is pretty horrible, all things considered, and now I've got my wife's data to consider too, and I still want my data to be locally available on my laptop, and I don't want to entirely rely on cloud services for syncing, and, and, and... so now I'm making a tool for all that. And then when it is finished, somebody can come along and say "you could've just used owncloud and syncthing and rclone and a pair of external drives, good grief man."

Still though, I might rework the deduper logic anyway.

redsaz

3 năm trướcdiscuss

> I'm not concerned about malicious, forced collisions

Consider taking a look at xxh3, possibly. It seems a pretty decent contender, hashing speed-wise, to trade off of secure hashing: https://github.com/Cyan4973/xxHash

redsaz

3 năm trướcdiscuss

I like the strategy of only hashing the first part of a file as a multiphase approach of deduping in order to quickly eliminate unique files, I wish I had done it that way with my util. Maybe for v2!

redsaz

3 năm trướcdiscuss

> that tells you nothing about maliciously constructed files. Even if a hash function has no known collisions today, doesn't mean that they won't be found someday.

This is what I meant by "barring any discovered hash collisions" but in retrospect I didn't make that clear enough.

Though, if you're crafting your own malicious different-content-same-size files and storing them in your NAS to cause a hash collision to make them appear the same, then I bet several governments are willing to pay top dollar for your abilities :D

Or, different scenario, say you're hosting a Dropbox-like service and you're storing files for hundreds of thousands of users, then you shouldn't be using a duplicate-file-finding util anyway, it'd be better if it was implemented at a different layer anyway.

The scenario you describe (lots of big files of same sizes, few dupes) I agree hashing the entire file would be wasteful. From my experience on my file server, when I had two or more files of the same size, and the size was larger than a few MB, they likely had the same content.

Put another way, if multiple files of the same sufficiently-large size are encountered, expect to read the entirety of those files anyway, whether hashing or checking byte-for-byte, because they are likely dupes. So, there's still potential for perf gains by avoiding hashing, but I'm willing to bet it isn't as much as one would hope/expect.

(You do have me curious as to how much difference it could make, though)

Edit: I'm also willing to admit that I have so many dupes because my backup strategy is TRASH and I have dupes everywhere, and so my scenario could be more unusual than other people.

redsaz

3 năm trướcdiscuss

I've considered adding a "just check the first {user-configged-bytes}" mode, which would offer enough of the speed up you describe, which I think jdupes does (maybe? It's been awhile since I looked at it). I think the speed of opening the file and reading the first buffered page of bytes would dominate the time of the operation, especially if one were to do async reads (which my program does NOT do. I should look into it.)

Worth some performance measurements, anyway!

redsaz

3 năm trướcdiscuss

This isn't the first time I've heard concerns about using hashes for checking file equality. I've considered adding a "paranoid mode" to do the direct byte-for-byte checks for such folks that don't even want to introduce a so-remote-it's-virtually-impossible theoretical chance for a collision to occur.

I'd go into the math about how remote of a chance it would be (barring any discovered hash collisions) but others have explained it better than me elsewhere.

redsaz

3 năm trướcdiscuss

Writing a program like this is one of the first exercises I give myself when learning a new programming language, because it touches a little bit of everything (reading files, output, CLI, using libraries, hashmaps, functions, loops, conditionals, etc) and isn't too onerous to implement.

My latest (it's a few years old at this point) is lsdup (rust version) using blake3 for hashing the content: https://github.com/redsaz/lsdup/

All it does is list the groups of duplicate files, grouped by hash, groups ordered by size. I'll usually pipe the output to a file, then do whatever I want to the list, and run a different script to process the resulting list. It works fine enough.

redsaz

3 năm trướcdiscuss

I've not heard that version before, I like it. The way I usually hear it end:

So, he yells out "102!" and... Crickets.

"What'd I do wrong?"

"Ehh, you must not have told it right."

(...Or, in this case, "you're probably not using the right model GPU")