AI model DiffsFormer augmenting stock data for improved financial forecasting.

Can AI Fix Wall Street's Data Problem? New Tool Promises Better Stock Predictions

Jordan Keane in Tech & Innovation December 2025 • 4 min read.

"A new AI model called DiffsFormer uses diffusion models to generate synthetic stock data, potentially overcoming data scarcity and improving forecasting accuracy."

Accurate stock forecasting is the holy grail of asset management and investment. The ability to predict future stock behavior, like return ratios or price movements, is crucial for making informed decisions and maximizing profits. Traditionally, analysts have relied on historical data and various machine learning techniques to achieve this goal.

However, a significant challenge plagues the financial industry: data scarcity. High-quality financial data is often limited, noisy, and homogenous, making it difficult for models to learn effectively and produce reliable forecasts. This scarcity stems from factors like low signal-to-noise ratios in stock data and the tendency for stocks within the same industry sector to behave similarly.

Now, a new approach is emerging that leverages the power of artificial intelligence to tackle the data scarcity problem. Enter DiffsFormer, an AI model that uses diffusion models and a Transformer architecture to generate synthetic stock data, effectively augmenting existing datasets and improving the accuracy of stock forecasting models.

DiffsFormer: AI-Powered Stock Data Augmentation

AI model DiffsFormer augmenting stock data for improved financial forecasting.

DiffsFormer, short for Diffusion Transformer, represents a significant advancement in the application of AI to financial forecasting. It addresses the core issue of data scarcity by generating artificial intelligence-generated samples (AIGS) to enhance training procedures. The model is trained using a diffusion model, a type of generative AI, to create new stock factors, incorporating a Transformer architecture to capture complex patterns in the data.

The DiffsFormer model is initially trained on a large-scale source domain, capturing global joint distributions with conditional guidance. This allows the model to understand the relationships between different factors influencing stock prices. When faced with a specific downstream task, DiffsFormer augments the training data by editing existing samples. This editing process is crucial, as it allows control over the extent to which the generated data deviates from the target domain, ensuring relevance and accuracy.

Here's how DiffsFormer works:

Diffusion Process: The model progressively adds noise to existing stock factor data, eventually transforming it into a state of pure noise.
Denoising Process: The model then learns to reverse this process, predicting and removing the noise to reconstruct the original stock factors.
Transformer Architecture: This architecture allows the model to capture long-range dependencies and complex relationships within the time-series data of stock factors.
Conditional Guidance: The model uses labels and other information to guide the data generation process, ensuring that the generated data is relevant and realistic.
Transfer Learning: The model leverages knowledge gained from a large source domain to improve performance on specific target domains with limited data.

To assess the effectiveness of DiffsFormer, researchers conducted experiments on the CSI300 and CSI800 datasets, employing eight commonly used machine learning models. The results were promising, with the proposed method achieving relative improvements of 7.2% and 27.8% in annualized return ratio for the respective datasets. These results demonstrate the potential of DiffsFormer to significantly enhance the performance of stock forecasting models.

The Future of Financial Forecasting with AI

DiffsFormer represents a significant step forward in addressing the challenges of data scarcity in stock forecasting. By leveraging AI to generate synthetic data and augment existing datasets, this approach has the potential to improve the accuracy and reliability of financial forecasting models. As AI continues to evolve, we can expect even more sophisticated techniques to emerge, transforming the way investment decisions are made and shaping the future of finance. The ability to generate realistic stock factors, coupled with controlled editing, offers a promising avenue for improved forecasting accuracy.

About this Article -

This article was crafted using a human-AI hybrid and collaborative approach. AI assisted our team with initial drafting, research insights, identifying key questions, and image generation. Our human editors guided topic selection, defined the angle, structured the content, ensured factual accuracy and relevance, refined the tone, and conducted thorough editing to deliver helpful, high-quality information.See our About page for more information.

This article is based on research published under:

DOI-LINK: https://doi.org/10.48550/arXiv.2402.06656,

Title: Diffsformer: A Diffusion Transformer On Stock Factor Augmentation

Subject: q-fin.st cs.ai cs.lg

Authors: Yuan Gao, Haokun Chen, Xiang Wang, Zhicai Wang, Xue Wang, Jinyang Gao, Bolin Ding

Published: 04-02-2024

Everything You Need To Know

What is DiffsFormer and how does it aim to improve stock forecasting?

DiffsFormer is an innovative AI model designed to tackle data scarcity in financial markets. It utilizes diffusion models and a Transformer architecture to generate synthetic stock data, effectively augmenting existing datasets. By doing so, it aims to enhance the accuracy and reliability of stock forecasting models, addressing a significant challenge in the financial industry where limited, noisy, and homogenous data often hinders the performance of traditional forecasting methods.

How does DiffsFormer use diffusion models to generate synthetic stock data?

DiffsFormer uses diffusion models through a two-step process: First, the model progressively adds noise to existing stock factor data until it becomes pure noise. Second, it learns to reverse this process, predicting and removing the noise to reconstruct the original stock factors. This denoising process enables DiffsFormer to generate new, realistic stock factors that augment the original dataset, addressing data scarcity and enhancing the training of forecasting models.

What is the role of the Transformer architecture in the DiffsFormer model?

The Transformer architecture in DiffsFormer plays a crucial role in capturing complex patterns within stock market data. Specifically, it enables the model to identify and learn long-range dependencies and intricate relationships within the time-series data of stock factors. This capability is essential for understanding how different factors influence stock prices over time, contributing to the generation of more accurate and relevant synthetic data for improving forecasting accuracy.

What is 'conditional guidance' in the context of DiffsFormer, and why is it important?

Conditional guidance in DiffsFormer refers to the model's use of labels and other relevant information to direct the data generation process. This ensures that the synthetic data produced is not only realistic but also relevant to the specific forecasting task at hand. By guiding the data generation, DiffsFormer maintains control over the characteristics of the generated data, preventing it from deviating too far from the target domain and ensuring its usefulness in improving the performance of stock forecasting models. This also incorporates transfer learning, leveraging knowledge from other domains.

How effective is DiffsFormer in enhancing stock forecasting, and what were the results of its experimental evaluation?

DiffsFormer demonstrates significant effectiveness in enhancing stock forecasting by generating realistic stock factors. In experimental evaluations using the CSI300 and CSI800 datasets, DiffsFormer achieved relative improvements of 7.2% and 27.8% in annualized return ratio, respectively, when compared to existing models. These results highlight the potential of DiffsFormer to substantially improve the performance of machine learning models in predicting stock behavior and underscore the value of AI-driven data augmentation in addressing data scarcity challenges in finance.