AsyncDiff

Preview：

Introduce：

AsyncDiff is an asynchronous de-noising acceleration scheme for parallelizing diffusion models. It realizes the parallel processing of noise prediction models by splitting them into multiple components and distributing them to different devices. This approach significantly reduces inference latency while having little impact on the quality of the generation. AsyncDiff supports a variety of diffusion models, Stable Diffusion 2.1, Stable Diffusion 1.5, Stable Diffusion x4 Upscaler, Stable Diffusion XL 1.0, ControlNet, Stable Video Diffusion and AnimateDiff.

Stakeholders:
AsyncDiff is suitable for researchers and developers who need efficient image generation and video generation. It is especially suitable for application scenarios that seek to reduce the reasoning time of deep learning models while maintaining the quality of the generated content.
Usage Scenario Examples:

• Use AsyncDiff to speed up the image generation process of Stable Diffusion XL
• AsyncDiff parallelizes the ControlNet model to improve video generation efficiency
• Accelerate the Stable Diffusion x4 Uliscaler with AsyncDiff to quickly generate high-resolution images

The features of the tool:

• It supports parallel acceleration of various Diffusion models, such as Stable Diffusion 2.1, Stable Diffusion 1.5, and Stable Diffusion x4 Uliscaler.
• The parallel calculation between devices is realized by segmentation noise prediction model, and the inference delay is reduced effectively.
• Reduce inference latency and maintain generation quality for efficient image and video generation.
• Provide detailed scripts to speed up the reasoning process for specific models for user customization optimization.
• Support ControlNet, Stable Diffusion XL and other models, flexible to adapt to different application scenarios.
• It provides flexible configuration options to adapt to different parallel computing needs, making asynchronous parallel inference more convenient.
• Easy to integrate, only a small amount of code is added to enable asynchronous parallel reasoning and reduce development costs.

Steps for Use:

• Install the necessary environments and dependencies, including NVIDIA Gpus, CUDA, and CuDNN, to ensure the system supports parallel computing.
• Create a Python environment and activate it, then install AsyncDiff’s dependency packages for asynchronous parallel reasoning.
• Introduce AsyncDiff into the existing diffusion model code and make the necessary configurations, such as the number of splits and the size of the denoising step.
• The number of model segments, denoising step length and preheating stage are selected and configured according to the requirements to meet different parallel computing needs.
• Run the provided sample scripts or custom scripts to perform parallel reasoning and evaluate the speedup.
• Evaluate the AsyncDiff acceleration against the output and make the necessary adjustments to achieve the best performance.

Tool’s Tabs: Distributed computing, text to image