HackerTrans
トップ新着トレンドコメント過去質問紹介求人

dmoura

no profile record

投稿

Open Dataset: Vehicle Accidents

huggingface.co
2 ポイント·投稿者 dmoura·昨年·0 コメント

Nexar Dashcam Crash Prediction Challenge

kaggle.com
10 ポイント·投稿者 dmoura·昨年·4 コメント

Limitations of 'Understanding the Limitations of Mathematical Reasoning in LLMs'

desirivanova.com
1 ポイント·投稿者 dmoura·2 年前·0 コメント

The Five Dimensions of Sustainable Software Engineering

luiscruz.github.io
2 ポイント·投稿者 dmoura·4 年前·0 コメント

Kangas: Explore Multimedia Datasets at Scale

github.com
9 ポイント·投稿者 dmoura·4 年前·2 コメント

Neural Geometry and Rendering ECCV2022

ngr-co3d.github.io
2 ポイント·投稿者 dmoura·4 年前·0 コメント

Reconstructing Training Data from Trained Neural Networks

giladude1.github.io
11 ポイント·投稿者 dmoura·4 年前·0 コメント

Command-line data analytics

danielcmoura.com
103 ポイント·投稿者 dmoura·4 年前·25 コメント

SPyQL – SQL Powered by Python

spyql.readthedocs.io
3 ポイント·投稿者 dmoura·4 年前·0 コメント

コメント

dmoura
·3 年前·議論
Thank you for sharing your learnings and for your transparency! Congrats!
dmoura
·3 年前·議論
I prefer a SQL-like format. It’s not as complete but it cover most of the day-to-day use cases. Take a look at https://github.com/dcmoura/spyql (I am the author). Congrats on fq!
dmoura
·4 年前·議論
DuckDB is great! I love what you guys are building. The main gap for me is native support of JSON (lines), like you have for CSV and Parquet.
dmoura
·4 年前·議論
updated, thank you
dmoura
·4 年前·議論
Things you can do with SPyQL CLI that you can't with clickhouse local (AFAIK, top of my mind, not exhaustive):

- use python code in your queries

- import python libs (just install them with pip/conda)

- write your one UDFs in Python

- run OS commands from within the query (using os.system)

- have guaranty of row order (like in grep, sed, etc)

And there is more, please take a look at: https://spyql.readthedocs.io/en/latest/distinctive.html
dmoura
·4 年前·議論
Author of the benchmark and of SPyQL here. ClickHouse is fantastic. Amazing performance. SPyQL is built on top of Python but still can be faster than jq and several other tools as shown in the benchmark. SPyQL can handle large datasets but Clickhouse local should always show better performance.

SPyQL CLI is more oriented to work in harmony with the shell (piping), to be very simple to use and to leverage the Python ecosystem (you can import Python libs and use them in your queries).
dmoura
·4 年前·議論
This is great!

I am the author of SPyQL [1]. Combining JC with SPyQL you can easily query the json output and run python commands on top of it from the command-line :-) You can do aggregations and so forth in a much simpler and intuitive way than with jq.

I just wrote a blogpost [2] that illustrates it. It is more focused on CSV, but the commands would be the same if you were working with JSON.

[1] https://github.com/dcmoura/spyql [2] https://danielcmoura.com/blog/2022/spyql-cell-towers/