Revolutionary AI model eliminates barriers to creativity, enabling anyone to produce stunning images instantly with just a single GPU.
Our one-step diffusion pipeline generates vibrant and photorealistic images with exceptional detail in a single inference step, broadening the potential for text-to-image synthesis in applications like real-time interactive systems.
Our one-step diffusion pipeline generates vibrant and photorealistic images with exceptional detail in a single inference step, broadening the potential for text-to-image synthesis in applications like real-time interactive systems. Research: NitroFusion: High-Fidelity Single-Step Diffusion through Dynamic Adversarial Training
*Important notice: arXiv publishes preliminary scientific reports that are not peer-reviewed and, therefore, should not be regarded as definitive, used to guide development decisions, or treated as established information in the field of artificial intelligence research.
The Surrey Institute for People-Centred Artificial Intelligence (PAI) at the University of Surrey has announced a groundbreaking AI model that creates images as the user types, using only modest and affordable hardware.
The model, NitroFusion, represents a world first and has been made open source by its developers – SketchX, a lab within PAI – a move that fundamentally transforms access to AI-enabled image creation models for creative professionals.
Professor Yi-Zhe SonG, Director of SketchX and Co-Director of PAI, said:
"NitroFusion represents a paradigm shift in making AI accessible to everyone, eliminating the need for large compute resources and the long waiting times between prompt and result that are common with most image generation platforms."
Typically, similar technology is available only to corporate giants with vast computing resources. However, NitroFusion runs on a single consumer-grade graphics card – marking a decisive step forward in bringing advanced AI capabilities to individual creators, small studios, and educational institutions. The system supports a flexible process, allowing users to select between one and four refinement steps to balance generation speed and image quality according to their needs.
Dar-Yen Chen, the PhD researcher who helped to develop the project at PAI, said:
"NitroFusion leverages a novel dynamic adversarial framework that works like a panel of specialised art critics, each evaluating different aspects of the generated image to ensure high quality in a single step. The 'dynamic discriminator pool' in this framework introduces fresh evaluators during training to prevent overfitting and maintain diversity in image quality. The system's flexible architecture allows users to optionally use between one to four refinement steps, providing direct control over the balance between generation speed and image quality."
Professor SonG added:
"With NitroFusion, we're not just releasing another image generation model – we're pioneering an entirely new approach that democratizes AI interaction. Following our DemoFusion release last year, which provided a new way to upscale AI-generated images, this innovation further establishes our position at the forefront of making powerful AI technology accessible to all."
This breakthrough delivers multiple leaps for the users and industry:
Instant image generation that responds as users type – a first in the field – enabling rapid iteration, greater control, and better experimentation
Improved sustainability through significantly reduced energy consumption
Consumer-grade affordable hardware requirements (e.g., a single high-performance GPU) that mean individuals and small studios can create imagery affordably
Open-source availability enables global innovation, adaptation, and variations
Comprehensive benchmarking shows that NitroFusion outperforms other state-of-the-art models, including SDXL-Turbo and Hyper-SDXL, in areas like photorealism, vibrancy, and text alignment.
No cloud dependencies or subscription fees.
Professor Adrian Hilton, Director of the Institute for People-Centred AI at the University of Surrey, said:
"We believe we're the first in the world to achieve interactive image generation at this scale and efficiency. This opens up access to state-of-the-art AI for image generation and is just the beginning of our commitment to democratizing creative AI tools. Our Institute will continue to develop open-source, groundbreaking technologies that put professional-grade AI capabilities into the hands of creators everywhere.
"We're particularly proud of the great work that our SketchX Lab, creating new concepts and advancing the science of generative AI. Our research is focused on ensuring that the future of creative AI technology is inclusive, responsible and accessible to all, and we're keen to continue to work with organisations that share this ethos. We are also transparent about challenges, such as NitroFusion's current lack of support for Classifier-Free Guidance (CFG), which could improve control over complex prompts. However, this is an area of ongoing research."
The technology is available immediately through https://chendaryen.github.io/NitroFusion.github.io/, with comprehensive documentation and community support resources.
*Important notice: arXiv publishes preliminary scientific reports that are not peer-reviewed and, therefore, should not be regarded as definitive, used to guide development decisions, or treated as established information in the field of artificial intelligence research.
Journal reference:
Preliminary scientific report. Chen, D., Bandyopadhyay, H., Zou, K., & Song, Y. (2024). NitroFusion: High-Fidelity Single-Step Diffusion through Dynamic Adversarial Training. ArXiv. https://arxiv.org/abs/2412.02030