Try pouring a torrent of data onto a person, and it becomes a chaos of information. Yet, feed the same surge to a computer, and voila! You get machine learning models that can intuitively complete sentences mid-way, or even detect elusive tumors in medical scans that might elude a human gaze.

Think of data as the catalyst that’s driving the massive strides we are witnessing in the artificial intelligence landscape, yielding ground-breaking insights, innovative discoveries, and evidence-supported decisions. Data has grown so integral to the global economy that the demand for authentic, high-grade data has skyrocketed. However, heightened data privacy regulations and the growth in AI model complexity have made the task of collecting and labeling original data progressively challenging and often unfeasible.

Enter synthetic data – a cost-effective, computer-crafted alternative for testing and training AI models, that’s proving to be a game-changer in this era fueled by data. It is inexpensive to generate, arrives with automatic labels, and deftly maneuvers around the myriad of logistical, ethical, and privacy hurdles that come with training deep learning models using real-world data. According to a forecast by the research firm Gartner, by the year 2030, synthetic data will outstrip actual data in the race to train AI models.

What is Synthetic Data?

Synthetic data can be described as data that doesn’t originate from real-world events, but is instead artificially manufactured. Algorithms are used to generate it, with the primary aim being to scrutinize operational data datasets. Its primary use is in affirming the accuracy of mathematical models and priming this artificial data for use in deep learning models.

One of the main merits of utilizing synthetic data is that it mitigates the constraints often encountered when dealing with regulated or sensitive data. Furthermore, it allows the creation of data sets according to particular requirements, which might not be achievable with real data. Commonly, synthetic data sets are produced for ensuring the quality and testing of software.

However, the use of synthetic data also has its downsides. Among these are the inconsistencies that may arise as one tries to mimic the complexity inherent in original data. Another challenge lies in the fact that synthetic data cannot directly replace authentic data since accurate data is still essential for producing valuable results.

The Significance of Synthetic Data

Synthetic data offers businesses considerable advantages for three primary reasons: safeguarding privacy, speeding up product testing, and training machine learning models. Data privacy regulations often impose strict constraints on how companies handle sensitive information.

Any unauthorized disclosure or sharing of personally identifiable customer data can result in hefty lawsuits, which in turn tarnish the corporate image. As such, the foremost incentive for companies to adopt synthetic data generation techniques is the reduction of privacy-related risks.

For entirely novel products, data is often non-existent. Additionally, data annotation by human operators is an expensive and lengthy undertaking. These hurdles can be bypassed if companies opt to invest in synthetic data. Such data can be swiftly produced and utilized in the creation of reliable machine learning models, thus saving time and resources.

Benefits of Synthetic Data

Infinite Stock of Annoted Data

The allure of generating synthetic data on a computer is the ability to obtain it instantaneously, tailored to your precise needs, and in nearly infinite volumes. Computer simulations are a frequently used method to produce synthetic datasets. Leveraging a graphics engine, you can produce an endless stream of lifelike images and videos in a digital environment.

AI itself provides another avenue for the creation of artificial data, employing generative models to produce realistic text, images, tables, and other data types. Models under the generative AI category include transformer-based foundation models, diffusion models, and GANs, which learn the data’s fundamental characteristics to generate similar styles. Notable models include DALL-E for image generation and GPT for text.

A significant advantage of synthetic data is that it comes pre-tagged. The collection and manual annotation of real data are time-consuming, costly, and often unfeasible. Machines generating digital replicas already comprehend the data, eliminating the need for humans to meticulously detail each image, sentence, or audio file.

Safeguarding Sensitive Data

Synthetic data offers another benefit: it allows companies to circumvent some of the regulatory issues linked to handling personal data. Privacy and copyright laws protect healthcare records, financial data, and web content, making large-scale analysis challenging for companies.

Financial services frequently depend on sensitive customer data for internal tasks like software testing, fraud detection, and stock market trend prediction. To protect this information, companies adhere to strict internal procedures for data management. As a result, it can take months for employees to gain access to anonymized data, and the process of anonymization can introduce errors that greatly impair the product or prediction’s quality.

The challenge, therefore, is to generate synthetic financial datasets that cannot be linked to individuals but preserve the original data’s statistical properties. IBM’s Kate Soule and Akash Srivastava lead Project Synderella, a privacy-preserving synthetic data product aimed at creating synthetic tabular data for enterprises like banks, accelerating product development and enabling customers to discover new insights.

Accelerating AI Model Training

Training foundation models with billions of parameters requires substantial time and money. Substituting a portion of real-world training data with synthetic data can expedite and reduce the cost of training and deploying AI models of all sizes.

Synthetic images can be produced in multiple ways. IBM researchers have employed the ThreeDWorld simulator and related Task2Sim platform to create images of realistic scenes and objects for pretraining image classifiers. Not only does this method reduce the amount of real training data required, but it can be as effective as real images in pretraining a model for tasks like detecting cancer in a medical scan.

Generative AI can produce synthetic images even faster. A recent collaboration between MIT and IBM researchers merged thousands of small image-generating programs to create simple, colored, and textured images. A classifier pretrained on these basic images outperformed models trained on more detailed synthetic data.

Using more synthetic data and less real data also reduces the risk of a model pretrained on raw data scraped from the internet veering into discriminatory biases. Artificial data generated to order comes pre-checked with fewer biases.

David Cox, co-director of the MIT-IBM Watson AI Lab and head of Exploratory AI Research, stated, “Maximizing the use of synthetic data before leveraging real-world data could potentially sanitize the reckless phase we’re currently in.”

Enriching Dataset Diversity

The autonomous vehicle industry was quick to adopt synthetic data. It would be impractical, if not impossible, to gather examples of all potential road scenarios, including rare, edge cases. Synthetic data can bridge this gap by creating custom data.

Chatbots also experience diversity in accents, rhythm, and communication styles. It could take years for a chatbot to understand every customer’s request nuances and respond effectively. As a result, synthetic data has become critical to enhancing chatbot performance.

IBM Research’s LAMBADA algorithm generates fake sentences to fill a chatbot’s knowledge gaps. LAMBADA produces the sentences with GPT and then screens them for accuracy. As IBM’s Ateret Anaby-Tavor, an expert in natural language processing, stated, “With just a push of a button, you can generate thousands of sentences and simply evaluate and filter them.”

However, sometimes, there isn’t enough data to create a fake sentence, especially for thousands of languages spoken by relatively few people worldwide. To train AI models on these low-resource languages, IBM researchers have tested pretraining language models on image-grounded gibberish.

Chuang Gan, an IBM researcher, stated that a model pretrained on complete nonsense performed nearly as well on a fill-in-the-blank fluency test as a model pretrained on Spanish. “Teaching the model an emergent language first can help it learn non-Indo-European languages while avoiding some of the cultural biases that come with pretraining on a Western language,” he said.

Mitigating Vulnerability and Bias

Synthetic data is also regularly used to test AI models for security flaws and biases. AI models that excel on benchmarks can be easily fooled with adversarial examples — subtly altered images and text designed to provoke errors.

Using public data, IBM researchers recently developed a tool to create fabricated quote tweets on Twitter to test the robustness of stock prediction models that trawl social media for insights. After ingesting the fake tweet, an AI stock picker might reverse its decision, suggesting that investors buy rather than sell.

Large models often contain hidden biases, too, absorbed from the articles and images they have processed. IBM researchers recently developed a tool to locate these flaws and create fake text to correct the model’s discriminatory assumptions. The tool generates a counterfactual conditioned on the class you want to test — a topic, tense, or sentiment — to flip the model’s decision.

For instance, take the statement: “my boss is a man.” The tool generates a hypothetical statement with the gender reversed: “my boss is a woman.” Such a minor change should not cause a classifier to change its “positive” sentiment rating to “negative,” but it does in this case. To alleviate the bias, the model could be retrained on a dataset augmented with counterfactuals, teaching it to classify similar statements equivalently.

Characteristics of Synthetic Data

For data scientists, the nature of the data—real or synthetic—is less significant than the data’s quality, the underlying trends, patterns it reveals, and biases it may harbor.

Here are some defining characteristics of synthetic data:

Enhanced data quality: The acquisition of real-world data can be costly, challenging, and prone to human error, inaccuracies, and biases. These issues can directly affect the quality of a machine learning model. However, when creating synthetic data, companies can have greater assurance in the data’s quality, diversity, and equilibrium.

Scalable data: Given the escalating need for training data, data scientists are increasingly turning to synthetic data. Its volume can be adjusted to meet the training requirements of machine learning models.

Simplicity and efficacy: The use of algorithms makes the process of fabricating data straightforward. However, it’s essential to guarantee that the synthetic data does not expose any connections to the actual data, is free from errors, and does not introduce new biases. Data scientists have total control over the organization, presentation, and labeling of synthetic data. This means that companies can access a ready-to-go source of high-quality, reliable data with just a few clicks.

Categories of Synthetic Data

When choosing the most suitable approach to generate synthetic data, it’s crucial to understand the kind of synthetic data needed to address a business issue. There are two primary types of synthetic data: entirely synthetic and semi-synthetic data.

Entirely synthetic data has no ties to real data. This means that all necessary variables are present, but the data is not identifiable. On the other hand, semi-synthetic data preserves all information from the original data, excluding sensitive details. It is derived from actual data, and as such, there’s a possibility that some true values may persist in the resulting synthetic data set.

Different Forms of Synthetic Data

Below are some distinct forms of synthetic data: