Ask HN: Should data warehouses be the back end of data apps?
2 comments
Data warehouses are for strategic data use cases, that is, data that can be updated once per day or less. Think things like “what is my year on year profit for the footwear segment?” Or “how often in the past 5 years have customers from segment x contacted us for tender, what % did we win, and what was the net profit?”
If you need near real time data source for operational / transactional types of reporting then it is apparently not the right solution.
My company has no suitable in built reporting options in source systems so unfortunately we will be trying to serve near real time data via a data warehouse. We may be able to get the refresh pipeline down to about 1 hr by throwing resources at it.
If you need near real time data source for operational / transactional types of reporting then it is apparently not the right solution.
My company has no suitable in built reporting options in source systems so unfortunately we will be trying to serve near real time data via a data warehouse. We may be able to get the refresh pipeline down to about 1 hr by throwing resources at it.
Interesting. Which warehouse are you using?
Or if we're able to improve ingestion latency, query response time, and concurrency/scaleability, maybe it can be a good choice?