I just did the exact same thing for my company. I didn’t do the sql lite approach for gdrive though just a direct search.
The one part that is still difficult is the data modeling and table level descriptions etc. Maybe you make an update to a table - remove a column, etc. The 3rd party systems all have their schemas defined but the data warehouse is a bit more loose. So solving that really helps. Did you just use dbt schema to describe tables and columns then sync that to your bot? How did you keep it updated? And end of the day - worth building or buying? Also how did you track costs? I let users choose their model - but have learned it can get expensive fast. As I can see there are a lot of providers trying to solve this one thing. That said the data warehouse aspect is the loosely defined area and I can see dbt or one of those players try to build something.
Still can’t believe there is no true open source cms that sits on top of a Postgres db. Like. Yea you have payload but you are locked into nextjs. Just a cms that hooks into openapi or Postgres natively would be great. I still prefer Django over anything else now days.
Looks good. Question - is it always better to use a monorepo in this new AI world? Vs breaking your app into separate repos? At my company we have like 6 repos all separate nextjs apps for the same user base. Trying to consolidate to one as it should make life easier overall.
The one part that is still difficult is the data modeling and table level descriptions etc. Maybe you make an update to a table - remove a column, etc. The 3rd party systems all have their schemas defined but the data warehouse is a bit more loose. So solving that really helps. Did you just use dbt schema to describe tables and columns then sync that to your bot? How did you keep it updated? And end of the day - worth building or buying? Also how did you track costs? I let users choose their model - but have learned it can get expensive fast. As I can see there are a lot of providers trying to solve this one thing. That said the data warehouse aspect is the loosely defined area and I can see dbt or one of those players try to build something.