Nice! I think this space is growing. There are a few others Im aware off in the space worth checking out: https://julius.ai/, https://cipher42.ai (I've built the early version of this).
complementary hitler uses docker https://www.youtube.com/watch?v=PivpCKEiQOQ . but yeah I think infrastructure is hard but at the end it's just a tool for a job, you can use terraform, cdk, pulumi, yoke - the cycle of X for Infrastructure is bad continues :)
This is why we built CloudQuery (https://github.com/cloudquery/cloudquery) an open source high performance ELT framework powered by Apache Arrow (framework is open source, our connectors closed source). You can run local pipeline and write plugins (extractors) in Go, Python, Javascript and any other language and save data to any destination (files, SQLite, DuckDB, PostgreSQL, ...)
I wish you all the best but unfortunately I think you will find out that this business model doesn't work and if you give something for free most companies won't pay for it even if they have the money as it doesn't even worth to go through procurement if it is free. You will have to lock some features in an open core way or other way to paywall companies to pay-up.
Hope Im wrong but I think in the last few years all companies realised that FOSS is not a business model.
Congrats! What is the business model? Also, I see that the license is permissive which is great but how do you protect yourself from other companies hosting your solution and competing your cloud offering?
Here is a free tip to fix terraform business model (no MBA required):
- Terraform is one of the most valuable tools for IaC - it's all free and it shouldn't - it is not a consumer product, it is not running ads and 99% of the users are business users with money.
- Keep Terraform SDK free and open source, close source plugins, charge per resource.
- Invest much more into Core and plugins.
all forks are doomed with the same destiny, business model is not sustainable with negative margin - as long as you have such a generous free tier and storage can run on S3 the price you can ask for a managed version is incredibly low.
Firstly, congrats :) (Generalized) ingestion is a very hard problem because any abstraction that you come up with will always some limitations where you might need to fallback to writing code and have full access to the 3rd party APIs. But definitely in some cases generalized ingestion is much better then re-writing the same ingestion piece especially for complex connectors. Take a look at CloudQuery (https://github.com/cloudquery/cloudquery) open source high performance ELT framework powered by Apache Arrow (so you can write plugins in any language). (Maintainer here)
Firstly, Big Kudos if you were able to bootstrap a big business this is very very hard (actually doing this with VC money is also hard). As someone who did both VC funded startup and self-funded (fuzzit.dev acquired by GitLab), I would say doing a self-funded startup even though it sounds good (because you are an underdog and you win on the hardest level if you succeed), a lot of time is really impossible or if possible there is information people are not sharing - for example for self-funded - did you include 300k-600k you had as saving that you used while working on it (or did you consulting while working on it). Did you take loans to grow faster? - the details are really important.
Personally I got to a conclusion that the common ground is a lot of time what makes sense. If you can validate as much as possible before raising the first round, raising reasonable amount of what you need to prove or get to the next milestone and do a lot of first principal thinking that could be better then beiing on each side of the extreme - let's raise as much as possible or Im not taking any VC money because it is all bad. Just my 2 cents but yes if you can grow a business with your own money - then it's amazing - just think this is not possible in a lot of cases and products.
Unified API is a holly grail but as many said quite difficult to abstract every use case in a scalable way that won't break. At CloudQuery (https://github.com/cloudquery/cloudquery) we focus solely on the ELT use-case(Founder/Maintainer here).
Congrats!! How does it compare to the ELT space and the modern data stack where you have ingestion/storage/visualization layers decoupled?
Asking as the founder of CloudQuery (https://github.com/cloudquery/cloudquery), Saw Datasette quite a few times around data exploration but curious to hear about the most popular use-cases of Datasette!
I think it's not only the issue with terraform but also the underlying infrastructure. AWS should've never have imperative APIs in the first place. Or at least it's time for AWS V2 APIs
IMO Terraform providers should've never been free. It should've been open-core, whether you are running on prem or on the cloud. There are multi-billion dollars companies using Terraform and pay exactly 0$ (yes, some that are generous are paying for support, great but you don't build a business on charity). Maintaining 3000 APIs for GCP, AWS, and Azure is costing at the very list $20M/year - trying to drive everyone to the cloud offering instead of charging for whatever people already use is the wrong way around imo. You can charge less but charging nothing doesn't gonna work. Heck, even a restaurant is charging a bit less for food and then charges more on beverage but it never gives the food for free.
Congrats!! We also focus on performance at CloudQuery (https://github.com/cloudquery/cloudquery) by using Golang, gRPC and still trying to be abstract enough to support different databases :)