HackerTrans
TopNewTrendsCommentsPastAskShowJobs

abadid

148 karmajoined 9 ปีที่แล้ว

Submissions

Will GPUs Accelerate Your Traditional SQL Workload?

starburst.io
2 points·by abadid·10 วันที่ผ่านมา·1 comments

Benchmarking the JDBC Bottleneck in Trino

starburst.io
1 points·by abadid·6 เดือนที่ผ่านมา·1 comments

The dangers of the JDBC bottleneck in Trino

starburst.io
2 points·by abadid·11 เดือนที่ผ่านมา·1 comments

How to Avoid Data Access Bottlenecks When Using Trino

starburst.io
2 points·by abadid·ปีที่แล้ว·1 comments

Parquet and ORC's many shortfalls for machine learning, and what to do about it?

starburst.io
6 points·by abadid·2 ปีที่แล้ว·1 comments

comments

abadid
·10 วันที่ผ่านมา·discuss
The hype doesn't always live up to reality.
abadid
·6 เดือนที่ผ่านมา·discuss
More proof that it is really slow to access data in a database over JDBC.
abadid
·11 เดือนที่ผ่านมา·discuss
Short summary: Trino uses JDBC in ways in which it was not designed and this can result in massive performance bottlenecks. Sometimes partitioning source data can mitigate the bottleneck.
abadid
·ปีที่แล้ว·discuss
A lot of people don't realize how slow it is to access data in bulk from a database using JDBC. JDBC was never designed for the use case that Trino uses it for.
abadid
·2 ปีที่แล้ว·discuss
This article summarizes research from my lab in collaboration with ByteDance published in CIDR (a computer science conference held in Amsterdam two weeks from now) on a new columnar format designed for ML workloads.
abadid
·2 ปีที่แล้ว·discuss
Why use PostgreSQL instead of columnar databases that are likely to perform way better for these types of analytical workloads?
abadid
·5 ปีที่แล้ว·discuss
I agree that in theory they could both co-exist for the reasons you state, but in practice I think it's unlikely a company that invests in a data fabric (which is largely a technology cost) is going to simultaneously invest in the incentives for the data product creators that are necessary for the data mesh not to become the wild west.
abadid
·5 ปีที่แล้ว·discuss
In general, whenever you need to perform a join (of multiple datasets), that ends the pipeline of local operations on a partition. Other operators as well that necessarily require data from other partitions end local pipelines. This is why linear scalability is not completely achieved in practice. Most interactions with data cannot be performed in a completely partitionable way.

Usually those other operations which force the local pipeline to end occur in a query plan prior to hitting any kind of tradeoff of doing too much in a local pipeline, since local pipelines are SO much faster than what happens when communication is required.
abadid
·5 ปีที่แล้ว·discuss
I'm the author of this piece. I'm happy to respond to comments in this thread.
abadid
·6 ปีที่แล้ว·discuss
IMO, it's hard to put a price on strong isolation and consistency. Being able to write an app that that uses atomic transactions, that are isolated from concurrently running transactions, and that see the correct data is something that translates to reduced programmer time and effort, and improves user experience. Many programmers discount those important features when they start out, but they'd be better served including them in the price comparisons of different products that are out there.