If you can recreate a file so it’s hash matches known CP then that file is CP my dude. The probability of just two hashes accidentally colliding is approximately: 4.3*10-60
Even if you do a content aware hash where you break the file into chunks and hash each chunk, you still wouldn’t be able to magically recreate the hash of a CP file without also producing part of the CP.
The overhead of stream abstractions is negligible if your goal is security when processing arbitrary input files provided from a zero-trust environment.
In environments where you’re prioritizing performance I’d still argue streams are likely your best bet when the size of the file to be parsed is not a constant. You wouldn’t want to load 50 large files into ram on a server environment let alone a phone.
If your input buffer is a bunch of tiny 10 KB files and you trust them? Sure, load them into memory and access their indices on the stack. Make sure you reuse the buffer to avoid unnecessary allocations.
If you want parallel processing with zero-allocations then streams with an array pool for their backing buffer are the best bet.
Not loading arbitrary files into memory will always be safer than doing so.
As for decoding - I believe the functions for validating if an array of bytes is an image should be far removed from the decoding and presentation of those bytes to the frame buffer. You don’t need to decode a JPG to validate that a file is a JPG. It either conforms to the standard or it doesn’t; the pixel data is irrelevant.
If your goal is validation (i.e. this is a JPG/PNG) and stripping of EXIF data it is entirely possible to write your own parser in a managed and safe language in less than 500 lines of code without sacrificing any performance.
Don’t load them into memory, parse them as a stream byte-by-byte in accordance with the standard for the codec, check every offset before seeking, and reject images that don’t conform to the standard.
We're unleashing the future of computing, collaboration, productivity, and development by using real-time video streaming to change the way that desktop software is built and distributed.
The political speech some wish to see silenced revolves around the racial and socioeconomic inequality they benefit from each day. Turning a blind eye to systemic racism does not mean it does not exist, and inherently saying Black lives matter is not a political statement. The ideological aggression against it however is.
> I rarely see talented developers doing this because they're too busy working.
I love the privilege on display here. You can ignore racial inequality issues because "they are too distracting," meanwhile I have to wake up each morning, read yet another one of my brothers and sisters has been killed by police, and I still have to run my business. Your mediocrity exist in your complacently to the status-quo. On a daily basis we innovate, build, ship, and push tech more than you could ever hope -- and we don't need to ban "political discussions" at work.
If a business chooses to condemn something without backing up that solidarity with meaningful resources and capital, they should be criticized.
They are impossible to benchmark against each other without making an assumption about how often you want to access the data, and which parts of it you want to access.
but this means Bebop and FlatBuffers can exist side-by-side / solve different problems: what FlatBuffers does makes sense if you want to access only parts of your objects in limited specific ways, what we do is better if you're always interested in the whole packet.
- The benchmark code is present in the laboratory directory of the repository.
- We don’t compare to Capt’n Proto because it does not have a stable web-based implementation, at least not one that has the features that make it so fast natively, so there is nothing to compare.
- Flatbuffers are fast but have a notoriously awful API to work with while also creating their own non-standard data structures in languages like C++. Bebop generates standard type-safe code.
- Bebop doesn’t try to compress data other than strings. This is because we don’t want to be responsible for compressing trailing zeroes when faster compression algorithms exist that can be down after encoding. Also most data is tiny.
- Bebop supports discriminated unions and has a much more robust type system than Flatbuffers.
- We’re not convincing anyone to use our stuff. It was made for us and open sourced because it was useful; we don’t need people ripping out their current serializers if there’s no pressure to do so.
Your inability to accept reality is not an assertion of truth unto others. People can’t pull themselves up by the bootstraps if they have a boot on their neck.
I’m sorry I’m not a model minority in your eyes, but it’s rich you’re telling a black person to spend more time with black people. These aren’t just my lived experiences, it’s subject matter that comes from thousands of hours of talking with the very people you claim I should spend more time with.
You’re so uncomfortable with race and acknowledging racism you’d rather pretend you are the expert. I’m not “too far gone,” I simply live a life you can’t bring yourself to understand.
If it’s such a ridiculous statement then I invite you to provide anything backing up that claim. Because all those resources you claim are available to Black Americans are often the very things lacking in Black communities: https://www.epi.org/blog/the-racial-wealth-gap-how-african-a...
I mean Christ, have you even heard of a food desert?
Ignoring the blatant racist generalization of Black people in your comment, recent Fed data shows your assumption is categorically false.
Systemic Racism has always slowed the economic progress of those whose family roots are tied back to the first generation of slaves. It’s easier to build wealth as an immigrant than it is to build wealth after generations in a system that churns and spits out Black bodies without a second thought.
This essay talks specifically about the tax of being successful and Black in America: what did your comment aim to do other than try to dismiss that people with a different skin color have difficult lived experiences?
That’s correct! You could call it a brown tax and it still holds universal truth. A really good book on that subject is The Divide: Global Inequality from Conquest to Free Markets.
Author here. I think you missed the point of the essay; my startup at the time didn’t get funded because the idea wasn’t a good fit for venture capital. We still ended up up making close to half a million in revenue, pivoted, and have raised over $5M to date while grossing close to $10M last year in revenue.
Having experienced actual racism I can infer when someone is making a comment from a place of privilege vs. being malicious. The entire point of that section was to frame things.
Author here. This comment section is almost self-aware, which is nice to see. While it’s nice to believe everyone would have the same struggles from the same economic starting point, the reality is that as a poor white person it is much easier to find your way out of poverty.
You’ll have more job opportunities presented to you, more people willing to take risk. Of course it also helps not to have hundreds of years to systemic racism on your next as well which allows other members of your family to likely not be poor — thus able to support you.
You can’t erase race because you don’t understand it and it makes you uncomfortable to acknowledge that being white has its benefits. That’s why the American Dream is so far out of reach to many: https://andrew.im/essays/a-journey-to-the-american-dream