Just had a look (https://github.com/duckdb/duckdb/issues/9399). Yeah it's worrying that such a trivial query returned incorrect results - but credit to the Devs for getting it fixed quickly.
To my knowledge the only databases that can be described as "military-grade" in terms of testing are SQLite and Postgres.
Both the Brier score and log loss are proper scoring rules (i.e. optimized when the predicted probabilities are the true outcome probabilities), and the choice between the two seems to have minimal impact on the conclusions that can be drawn (https://pubsonline.informs.org/doi/abs/10.1287/deca.2013.028...). I covered the Brier score in the post as I thought it would be easier to digest for a general audience.
As Frank Harrell wrote on his blog (https://www.fharrell.com/post/class-damage/), one advantage of the Brier score could be its interpretability and the ability to break it decompose it into discrimination and calibration components.
Agreed. It would be great to hear your views on some of the key gaps in modern data science curricula that could be covered in the blog - would you be able to drop me a line at [email protected]? Thanks!
I agree that the post lacks depth, but it was intended to be a gentle article accessible to a general audience, so they can start applying it in practice in their day to day lives. I would, however, really love to hear your views on what might be a more rigorous treatment of similar topics that can be introduced in an accessible way - would you be able to drop me a line at [email protected]? Thanks!
To my knowledge the only databases that can be described as "military-grade" in terms of testing are SQLite and Postgres.