- OpenAI’s upcoming model, Orion, reportedly shows less improvement over existing models than prior releases, with limited gains in tasks like coding.
- To address improvement slowdowns, OpenAI is considering training models on synthetic AI-generated data and enhancing models post-training.
According to a recent report from The Information, OpenAI’s forthcoming model, code-named Orion, might not be the substantial leap forward that its predecessors were.
The latest internal tests reportedly show Orion outperforming existing models but with less significant gains compared to the advancements seen in the shift from GPT-3 to GPT-4. This suggests a slowdown in the rate of improvement that has characterized OpenAI’s past flagship releases.
Employees familiar with Orion’s capabilities suggest that, while it does excel over previous models in some areas, it may not consistently outperform them in specific tasks, such as coding.
This observation marks a potential shift in OpenAI’s model development trajectory and highlights a new challenge: limited availability of fresh, high-quality training data, which has traditionally fueled the rapid advancements in OpenAI’s generative AI capabilities.
In response, OpenAI has reportedly created a foundations team tasked with exploring ways to sustain and enhance model performance. Among the strategies under consideration are training Orion on synthetic data generated by existing AI models and implementing improvements during the post-training phase. These approaches represent a shift from traditional reliance on vast quantities of new, natural training data.
OpenAI has yet to comment on these developments, but previously noted regarding its flagship models that, “We don’t have plans to release a model code-named Orion this year.”
Edited by Harshajit Sarmah
