Photon powered Engine (enhance Apache Spark 3.0’s performance by up to 20x)(techcommunity.microsoft.com)

7 pointsby techempower6 ปีที่แล้ว2 comments

techcommunity.microsoft.com

Photon powered Engine (enhance Apache Spark 3.0’s performance by up to 20x)

https://techcommunity.microsoft.com/t5/azure-databricks/turbocharge-azure-databricks-with-photon-powered-delta-engine/ba-p/1694929

2 comments

alphachloride6 ปีที่แล้ว

For just a moment I was pleasantly surprised that I would be reading about an interesting propulsion technology. Seeing as that current propulsion approaches rely on ejecting Fermion composites at high relative velocities to the engine.

I am not pleasantly surprised anymore.

legg0myegg06 ปีที่แล้ว

DuxkDB's query engine is inspired by the same paper! You'll be surprised what you can process on a single node with DuckDB - it takes 33 Spark nodes to match performance! That still puts it ahead of Photon, and it is open source.