ByteDance Seed releases GenFirst, an end-to-end latently trained image model

On 3 November, ByteDance Seed published a repository called GenFirst on Hugging Face, listed under the text-to-image pipeline and the PyTorch framework. The model card gives the fuller name: “Generation Before Reconstruction for Stable End-to-End Latent Generative Modeling.” At the time of listing the page showed zero downloads and one like, which marks the start of a public release rather than a model already in circulation.

GenFirst targets an old problem in latent generative modelling. When a variational autoencoder and a diffusion prior are trained end to end together, the latent space tends to collapse: the prior learns something usable while the decoder stops recovering detail. According to the model card, GenFirst applies a two-phase schedule to the prior and entropy weights. The first phase lets the prior shape the geometry of the latent space; the second drops both weights tenfold so the decoder can recover detail. The result is meant to be a latent space that a diffusion prior fits well and that still works as an autoencoder.

The repository ships two kinds of artifact that are used together. One is the set of encoders and decoders produced by the end-to-end stage, each accompanied by its own latent normalisation statistics. The other is the generative priors: SiT models for class-conditional generation and MMDiT models for text-to-image, both trained on a frozen VAE. The card warns that a VAE and its statistics must be kept together, because pairing them wrongly produces noise rather than a slightly worse image.

Only the EiT diffusion-prior instantiation is released; the autoregressive EAR version from the paper is not. Usage is also not turnkey. The release consists of raw .pt checkpoints with no diffusers integration and no from_pretrained entry point, so users must clone the training repository, install its dependencies and sample through the provided scripts.

Architecturally the priors are SiT-XL and MMDiT-XL: 28 blocks, width 1152, 16 attention heads, roughly 675 million parameters. The text-to-image models use a frozen CLIP-L text encoder, the weights ship in fp32 and sampling was tested in bf16. That is a research-scale model, and the emphasis is on validating a training recipe rather than competing on parameter count.

The reported numbers are specific. On class-conditional ImageNet at 256×256, the best prior checkpoint at 800 epochs reports a gFID of 1.45, or 0.969 with guidance, while a 400-epoch 512×512 checkpoint reaches 1.23 guided. For text-to-image, the 512×512 EiT variant scores 0.8987 on GenEval and 82.60 on DPG-Bench without prompt rewriting. A separate f8d32 VAE initialised from the FLUX.2 VAE reports a reconstruction rFID of 0.041 and a PSNR of 36.46.

More interesting is the shared-latent set of experiments. Using a Qwen3-VL ViT autoencoder shaped end to end and adding representation supervision from SigLIP or VLM-NLL, one latent space serves generation at 0.90 GenEval and 83.76 DPG-Bench while keeping linearly probeable semantics, with top-1 accuracy of 79.25% and 81.69% respectively. A JointMMDiT trained on jointly shaped text and image VAEs reaches 86.57 on GenEval. The interesting part is not another text-to-image checkpoint but the claim that one latent space can serve generation and understanding at the same time.

Licensing is the most easily missed constraint. The weights are released under CC BY-NC 4.0, for non-commercial research use only, which the card notes is stricter than the MIT licence on the code. Teams hoping to put GenFirst into a commercial product would need separate terms or would be limited to treating it as a methods reference.

The card also declines to present its benchmark numbers as settled. FID is sensitive to reference statistics and to the resizing path, so comparisons only hold under the same protocol, and the published figures were produced with pinned environment versions. A listing that starts at zero downloads and one like also means the community has not yet tested any of it. What to watch next: whether a diffusers port appears, whether the EAR variant is released, and whether the end-to-end recipe still holds at larger scale.