Elon Musk acknowledges that the available AI training data has been fully utilized. Technology news.

Elon Musk acknowledges that the available AI training data has been fully utilized.

Elon Musk agrees with other AI experts that real-world data for training AI models is running out. 

“We’ve now essentially exhausted the cumulative sum of human knowledge … in AI training,” Musk stated during a livestreamed conversation with Stagwell chairman Mark Penn on X late Wednesday. “That basically happened last year.”

Musk, who leads AI company xAI, echoed themes previously discussed by former OpenAI chief scientist Ilya Sutskever at the NeurIPS machine learning conference in December. Sutskever described the industry as having reached “peak data,” predicting that the scarcity of training data would prompt a shift in how AI models are developed.

To address this challenge, Musk proposed synthetic data—data generated by AI itself—as the solution. “The only way to supplement [real-world data] is with synthetic data, where the AI creates [training data],” he said. “With synthetic data … [AI] will essentially grade itself and engage in self-learning.”

Other companies, including Microsoft, Meta, OpenAI, and Anthropic, are already incorporating synthetic data into their AI training processes. Gartner estimates that by 2024, 60% of the data used in AI and analytics projects will be synthetically generated. 

For example, Microsoft’s recently open-sourced Phi-4 model was trained on a mix of synthetic and real-world data, as were Google’s Gemma models. Similarly, Anthropic relied on synthetic data for its Claude 3.5 Sonnet system, and Meta fine-tuned its latest Llama models using AI-generated data.

Training on synthetic data offers benefits, such as reduced costs. AI startup Writer developed its Palmyra X 004 model almost entirely using synthetic sources at a cost of just $700,000—significantly less than the estimated $4.6 million required for a comparable OpenAI model.

However, synthetic data comes with challenges. Research suggests it can lead to “model collapse,” where an AI becomes less creative and more biased, compromising its functionality. Since synthetic data is generated by models themselves, any biases or limitations in the original data may be perpetuated and amplified in future outputs.

see more.........

What is Elon Musk saying about AI? Does Twitter use your data for AI? What does Elon Musk have to do with OpenAI? What is the AI chatbot from Elon Musk?   amazing inventions that are on another level,artificial intelligence training,amazing tools that are on another level,i challenged my ai clone,how i pick the best stocks: investing for beginners,how to invest in the stock market,biden administration,order block trading strategy,squawk on the street,the behavioral arts,how i pick the best stocks,solving the money problem,order block trading,tesla data,the future of ai,the behavior panel,the body language guy

Topics

AIAIElon MuskGenerative AIsynthetic datatraining data