Metaplane – A metadata catalog for modern data teams(metaplane.dev)
metaplane.dev
Metaplane – A metadata catalog for modern data teams
https://metaplane.dev
20 comments
A lot of people want a product like this, or at least think they do, and many have also attempted to build parts of this. The issue, of course, with this product is that there is no evidence of a product nor an ability to assess it in abstract of a sales person presumably emailing you.
Questions I think would be important to answer:
- What types of queries are supported?
- Does data size or update frequency have performance implications?
- Is discovery embedded in this product?
- Is it available on-prem or only as an API?
- What are the data security guarantees?
- What workflows are easy to do in the UI?
- Does this support alerting on queries?
I could go on, but I think the point is made. There’s a reason a lot of data tooling companies use a freemium model — users want to feel the system before investing in it, maybe only reaching out when they’re ready to load test.
Questions I think would be important to answer:
- What types of queries are supported?
- Does data size or update frequency have performance implications?
- Is discovery embedded in this product?
- Is it available on-prem or only as an API?
- What are the data security guarantees?
- What workflows are easy to do in the UI?
- Does this support alerting on queries?
I could go on, but I think the point is made. There’s a reason a lot of data tooling companies use a freemium model — users want to feel the system before investing in it, maybe only reaching out when they’re ready to load test.
The old guard is very enterprise-y for a reason. Their products are complex monoliths that require mid-six figure investments to get moving. Either you have the staff/dollars/organizational momentum to buy their crap and follow-through or it's not gonna work.
I've spent more time than I'm comfortable admitting with Gartner's favorites in the Data Governance and Data Quality areas and they all suck. Desperately looking forward to having someone not-tied to legacy technology enter this space.
I've spent more time than I'm comfortable admitting with Gartner's favorites in the Data Governance and Data Quality areas and they all suck. Desperately looking forward to having someone not-tied to legacy technology enter this space.
I am curious how this compares to http://dataframe.ai or the open source metaframe ( http://github.com/rsyi/metaframe ) tools I've looked at in the past?
It might be helpful to have more specific information about how it compares to other solutions aimed at a technical audience.
It might be helpful to have more specific information about how it compares to other solutions aimed at a technical audience.
If you're looking for assistance developing the tool and/or selling to enterprise, reach out (username minus the 2 at gmail), I've got a virtual Rolodex full of F500 clients looking for better answers in this space.
Disclosure: I am part of the DataHub team at LinkedIn
There have been a number of recent open source offerings in this space, one of them being DataHub [1] which is a product of our evolution in the metadata space over the years.
An important lesson we learnt as part of this evolution is that monolithic/centralized architecture just doesn't scale with new data and users. Individual teams/owners must have the power and flexibility to decide what they care about while still being able to tap into the global metadata.
Join our community if you are interested.
[1] https://github.com/linkedin/datahub
There have been a number of recent open source offerings in this space, one of them being DataHub [1] which is a product of our evolution in the metadata space over the years.
An important lesson we learnt as part of this evolution is that monolithic/centralized architecture just doesn't scale with new data and users. Individual teams/owners must have the power and flexibility to decide what they care about while still being able to tap into the global metadata.
Join our community if you are interested.
[1] https://github.com/linkedin/datahub
Aren't these abstractions often so complex, company-specific and leaky that most companies are better off building their own in-house solutions? Atleast that's what I've seen at most FAANG companies but then again they're notorious for having NIH syndrome and have the resources which your regular IT shop might not.
IME the issue is that startup data modelling ends up in some ad-hoc framework that enforces a specific paradigm. When you try to adopt a standardized tool they don't ask "does this produce the right answer" but "can this do exactly what the old tool did", and usually it's not the case. Nobody can actually reason about the whole set of data from first principles anymore so they're forced to mechanically repeat the same exact process to get consistent results. Hell, the results might be wrong, or nobody uses them, but you don't have the tooling to detect that and you're too afraid to adopt it
I went to that link, and still don't have a great idea of what the tool actually is
I'd suggest that you're not the target market then.
There is no contact on your profile or your site.
I am interested and actively in the market for a solution of this type.
I am interested and actively in the market for a solution of this type.
We are actively developing a solution to this at Splitgraph [0].
We’re looking for our first customers / design partners for the self-hosted / on-prem product.
I would love to talk to you, but you’ve got no contact in your profile. Can you email me at [email protected] and we can set up a time to chat?
[0] https://www.splitgraph.com
We’re looking for our first customers / design partners for the self-hosted / on-prem product.
I would love to talk to you, but you’ve got no contact in your profile. Can you email me at [email protected] and we can set up a time to chat?
[0] https://www.splitgraph.com
Would be curious to better understand your needs in this space. I work at an early stage VC firm and have met probably ever single catalogue/metadata startup - shoot me an email at davis (at) innovationendeavors.com if you want to share notes on the space. I can potentially recommend a few options.
Looks interesting. I'm finding that using dbt and the automatically generated docs helps with quite a lot of this. There are other sources of metadata though, so I can see why some might want a tool like this.
The problem with most metadata catalog is not technical, but the human: Catalog are documents for data, and developers hate writing documents.
OP do you work here? or if anyone that works on this project sees this comment, DM me, we could definitely use this at my company
Would be curious to better understand your needs in this area. I work at an early stage VC firm and have met probably ever single catalogue/metadata startup - shoot me an email at davis (at) innovationendeavors.com if you want to share notes on the space.
Interesting product. I am wondering how does this fit into typical workflows.
Can this be hooked up to Hive to replace the metastore? Or Presto etc?
this looks cool but what is the list of connectors that it currently has.
What sort of pricing is this targeting?