HackerTrans
TopNewTrendsCommentsPastAskShowJobs

contol-m

no profile record

Submissions

BigQuery Tables for Apache Iceberg

cloud.google.com
2 points·by contol-m·2 ปีที่แล้ว·0 comments

Unify data lakes and warehouses with BigLake

cloud.google.com
1 points·by contol-m·4 ปีที่แล้ว·0 comments

BigLake: Unifying data lakes and data warehouses across clouds

cloud.google.com
2 points·by contol-m·4 ปีที่แล้ว·0 comments

comments

contol-m
·4 ปีที่แล้ว·discuss
[I work on BigQuery]

BigQuery also supports in-place querying of datasets on GCS (or S3/Azure using Omni) via external/BigLake tables. https://cloud.google.com/bigquery/docs/query-cloud-storage-u...
contol-m
·4 ปีที่แล้ว·discuss
Currently the main benefit of BigLake over the current external tables is governance: you get row and column level security over cloud storage data. The governance is uniformly enforced across BigQuery and also the BigQuery storage API. The storage API can be used by any engine and we have pre-built open source connectors available for Spark, Presto/Trino, Dataflow and Tensorflow.

We're constantly working on improving BigQuery performance over open file formats on cloud storage. Some of these features will be specific to BigLake. Please stay tuned.
contol-m
·4 ปีที่แล้ว·discuss
I work on BigQuery. All of these are great points: just wanted to point out that BigQuery can federate into external data sources as well: e.g. files on cloud storage and BigTable. Relevant feature is BigLake: https://cloud.google.com/bigquery/docs/biglake-intro