Can continuous diffusion models bring the same performance breakthrough on natural
language they did for image generation? To circumvent the discrete nature of text data, we
can simply project tokens in a continuous space of embeddings, as is standard in language
modeling. We propose Self-conditioned Embedding Diffusion, a continuous diffusion
mechanism that operates on token embeddings and allows to learn flexible and scalable
diffusion models for both conditional and unconditional text generation. Through qualitative …