Transformer

Training a small transformer to generate 16x16 pixel art characters by de-masking pixels step by step.

Building a discrete diffusion transformer (DiT) from scratch and training it to generate Shakespeare-like text, one denoising jump at a time.