Great question! Short answer is that Gretel supports multiple deployment options, depending on your specific circumstances and needs. If you want a more detailed technical answer, I recommend joining our Discord and asking there. One of our engineers will follow up. Hope that's helpful!
https://grtl.ai/discord
If you’re a Vertex AI user, here’s how you can utilize Gretel to create high-quality synthetic tabular data that you can use as training data for a classification model.
Gretel introduces Reinforcement Learning from Privacy Feedback (RLPF), a method that can be used to align large language models (LLMs) to improve generative quality while also making them more privacy-preserving. Language models leaking proprietary data or custom prompts is a problem that's currently plaguing many generative AI applications. We propose RLPF to mitigate some of these issues. We also suggest future directions to reduce bias, discrimination, and other harmful characteristics that might exist in today’s language models.
In case you missed it, several sessions for _synthesize2023 were announced this week. This event is free and open to all who are interested in learning about state-of-the-art applications for synthetic data and generative AI.
Here are some of the speaker highlights:
- Keynote speaker: Sridhar Ramaswamy, CEO and Cofounder at Neeva and n.xyz, former SVP of Engineering and Ads at Google
- Google research scientist Peter Kairouz will discuss how privacy-enhancing technologies (PETs) like synthetic data and federated learning are helping advance the science and safe application of foundation models.
- Illumina's Senior Director of Emerging Solutions, Pam Cheng, will highlight how synthetic data enables medical and life science research and product development.
- NVIDIA product manager Nyla Worker will demonstrate how to train a perception model, an SDK for creating 3D synthetic data.
Data sharing is central to modern business but entails risks. Synthetic data can enable data sharing while reducing the risk of privacy-compromising linkage attacks.
- Performance metrics for evaluating the quality of data
- How to interpret data quality scores
- Use cases for both low fidelity and high fidelity synthetic data
“ Data is the lifeblood of modern artificial intelligence. Getting the right data is both the most important and the most challenging part of building powerful AI. Collecting quality data from the real world is complicated, expensive and time-consuming. This is where synthetic data comes in.”
How we implemented a practical attack on a synthetic data model to validate its ability to protect sensitive information under different parameter settings.
It’s been trained on millions of public datasets and allows developers and data scientists to create new variations of synthetic text and labels for their datasets. Enjoy!
How to de-identify a relational database for demo or pre-production testing environments while keeping the referential integrity of primary and foreign keys intact.
Synthetic data is algorithmically generated data that mirrors the statistical properties of the dataset it’s based on. Learn how to make high-quality synthetic data.
This paper investigated the existing artificial intelligence tool, the Gretel.ai, for generating synthetic data and intelligent data analysis and applications. Using the ASD Toddler dataset, which is publicly available, the proposed framework can generate synthetic data ranging from 1054 to 5000 records without changing the original features. By viewing the graphics using the ASD dataset, the tool also provides a quick report, namely the Gretel Synthetic Report, which can help quantify their utility on exploratory data analysis. With these benefits and the availability of synthetic data, it will likely become the future of Artificial Intelligence. In due course, the synthetic data will replace actual data to become primary data generation for future references.
We published a notebook and a GitHub repo that helps you train synthetic models on highly dimensional datasets (e.g. 1000's of columns, and millions of records). It works by using Gretel's open source header clustering to group correlated data and parallelize training across multiple GPUs.
One of the biggest bottlenecks to innovation that developers and data scientists face today is getting access to data, or creating the data that you need to test an idea or build a new feature. That’s where synthetic data comes in.
In this study, we discuss the creation of high-quality synthetic time-series datasets for one of the largest financial institutions in the world, and the methods we designed to assess the accuracy and privacy of our models and data. The temporal, ordered nature of time series data can help track and forecast future trends, which unsurprisingly, has enormous utility for business planning and investing. However, due to regulations and the inherent security risks that come with sharing data between individuals and organizations, much of the value that could be gleaned from it remains inaccessible.
Gretel’s work demonstrates that synthetic data can help close this gap while preserving privacy. By generating synthetic time-series data that are generalizable and shareable amongst diverse teams, we can give financial institutions a competitive edge and the power to explore a whole new world of opportunities.
Developers can test our methods by following along with the 3-step process outlined in the blog post!
This is the proof of concept code from a Gretel.ai & Illumina project which used generative neural networks to create synthetic versions of mouse genotype and phenotype data.