Unraveling the smoothness properties of diffusion models: A gaussian mixture perspective

Y Liang, Z Shi, Z Song, Y Zhou - arXiv preprint arXiv:2405.16418, 2024 - arxiv.org
Diffusion models have made rapid progress in generating high-quality samples across
various domains. However, a theoretical understanding of the Lipschitz continuity and …

Exploring the frontiers of softmax: Provable optimization, applications in diffusion model, and beyond

J Gu, C Li, Y Liang, Z Shi, Z Song - arXiv preprint arXiv:2405.03251, 2024 - arxiv.org
The softmax activation function plays a crucial role in the success of large language models
(LLMs), particularly in the self-attention mechanism of the widely adopted Transformer …

Unifying Bayesian Flow Networks and Diffusion Models through Stochastic Differential Equations

K Xue, Y Zhou, S Nie, X Min, X Zhang, J Zhou… - arXiv preprint arXiv …, 2024 - arxiv.org
Bayesian flow networks (BFNs) iteratively refine the parameters, instead of the samples in
diffusion models (DMs), of distributions at various noise levels through Bayesian inference …

IBD-PSC: Input-level Backdoor Detection via Parameter-oriented Scaling Consistency

L Hou, R Feng, Z Hua, W Luo, LY Zhang… - arXiv preprint arXiv …, 2024 - arxiv.org
Deep neural networks (DNNs) are vulnerable to backdoor attacks, where adversaries can
maliciously trigger model misclassifications by implanting a hidden backdoor during model …

Lightweight diffusion models: a survey

W Song, W Ma, M Zhang, Y Zhang, X Zhao - Artificial Intelligence Review, 2024 - Springer
Diffusion models (DMs) are a type of potential generative models, which have achieved
better effects in many fields than traditional methods. DMs consist of two main processes …

Improved ddim sampling with moment matching gaussian mixtures

P Gabbur - arXiv preprint arXiv:2311.04938, 2023 - arxiv.org
We propose using a Gaussian Mixture Model (GMM) as reverse transition operator (kernel)
within the Denoising Diffusion Implicit Models (DDIM) framework, which is one of the most …

Mix-DDPM: Enhancing Diffusion Models through Fitting Mixture Noise with Global Stochastic Offset

H Wang, D Zhai, X Zhou, J Jiang, X Liu - ACM Transactions on …, 2024 - dl.acm.org
Denoising diffusion probabilistic models (DDPM) have shown impressive performance in
various domains as a class of deep generative models. In this article, we introduce the …

On Statistical Rates of Conditional Diffusion Transformers: Approximation, Estimation and Minimax Optimality

JYC Hu, W Wu, YC Lee, YC Huang, M Chen… - arXiv preprint arXiv …, 2024 - arxiv.org
We investigate the approximation and estimation rates of conditional diffusion transformers
(DiTs) with classifier-free guidance. We present a comprehensive analysis for``in …

[HTML][HTML] Average entropy of Gaussian mixtures

B Joudeh, B Škorić - Entropy, 2024 - mdpi.com
We calculate the average differential entropy of aq-component Gaussian mixture in R n. For
simplicity, all components have covariance matrix σ 2 1, while the means {W i} i= 1 q are iid …

DC-DPM: A Divide-and-Conquer Approach for Diffusion Reverse Process

YJ Dong, H Yin, F Wang, Y Zhao, C Zhang, C Li… - openreview.net
Diffusion models have achieved great success in generative tasks\textblue {, with the quality
of generated samples guaranteed by their convergence properties, typically derived within …