metadata
license: mit
library_name: diffusers
pipeline_tag: text-to-image
LightGen: Efficient Image Generation through Knowledge Distillation and Direct Preference Optimization
About
This model (LightGen) introduces a novel pre-train pipeline for text-to-image models. It uses knowledge distillation (KD) and Direct Preference Optimization (DPO) to achieve efficient image generation. Drawing inspiration from data KD techniques, LightGen distills knowledge from state-of-the-art text-to-image models into a compact Masked Autoregressive (MAR) architecture with only $0.7B$ parameters.
It is based on this paper, code release on this github repo.
Currently, we just release some checkpoint without DPO
🦉 ToDo List
- Release Complete Checkpoint.