Validation — imagehub 0.1¶
Validated on CPU on October 4, 2026. Environment: Python 3.13.11 / PyTorch 2.14.1.
Reference: diffusers 0.40.0, transformers 5.18.0. Machine: Apple M2 Max, 96 GB unified memory,
12 CPU threads. Sources, commits and file fingerprints are in validation.json.
References are recorded per model in validation.json under reference.
LoRA training¶
Validated on CPU on November 4, 2026 against the step of diffusers 0.40.0
examples/text_to_image/train_text_to_image_lora.py with peft 0.21.2 (transformers 5.18.0,
PyTorch 2.15.0, torchvision 0.29.1).
- Both sides start from the same LoRA weights and get the same pixels, token ids, posterior noise, noise and timesteps. Every loss, gradient and parameter below is bit-identical.
- The reference script passes an already-consumed parameter generator to
clip_grad_norm_, so it never clips. imagehub clips; the multi-step comparison clips the reference explicitly.
Comparisons (CPU, float32)¶
| Comparison | Model | Largest observed difference | Tolerance |
|---|---|---|---|
| Image transform (resize, center / random crop, flip, normalize) vs torchvision | 4 image sizes | identical | identical |
| LR multipliers (constant, constant with warmup, linear, cosine), 20 steps | — | identical | identical |
| First-step loss and LoRA gradients, epsilon and v-prediction, with and without min-SNR | tiny SD 1.x | 0 | 0 |
| First-step loss and LoRA gradients, epsilon, 256×256, batch 2 | sd-v1-5 |
0 | 0 |
| Four trainer steps (AdamW, cosine with warmup, clipping): losses and parameters | tiny SD 1.x | 0 | 0 |
Saved adapter loaded by diffusers load_lora_weights vs imagehub, 3 steps |
tiny SD 1.x | 0 (pixels) | 0 |
Exact resume (six straight steps against three, resume, three) and mixed-precision runs are covered by the unit tests.
Stable Diffusion XL¶
Against examples/text_to_image/train_text_to_image_lora_sdxl.py (diffusers 0.40.0, peft 0.21.2):
| Comparison | Model | Largest observed difference | Tolerance |
|---|---|---|---|
| Image transform (resize, crop, original size and crop corner) vs the script, 4 image sizes | — | identical | identical |
| First-step loss, epsilon and v-prediction, with and without min-SNR | tiny SDXL | 0 | 0 |
| First-step LoRA gradients | tiny SDXL | 0 | 0 |
Saved adapter loaded by diffusers load_lora_weights vs imagehub, 3 steps |
tiny SDXL | 0 (pixels) | 0 |
| Gradients with and without gradient checkpointing (SD 1.x and SDXL) | tiny | identical | identical |
Like diffusers, the U-Net's resnet blocks make the shortcut input contiguous in training mode; the copy does not change outputs but does change the backward summation order, and with it the gradients are bit-identical.
Stable Diffusion 3 / 3.5¶
Validated on CPU on November 7, 2026 against examples/dreambooth/train_dreambooth_lora_sd3.py
(diffusers 0.40.0, peft 0.21.2), with the script's default LoRA targets (the eight
joint-attention projections):
| Comparison | Model | Largest observed difference | Tolerance |
|---|---|---|---|
Training timesteps and sigmas vs FlowMatchEulerDiscreteScheduler, shift 1 and 3 |
— | identical | identical |
Timestep density (logit_normal, mode, sigma_sqrt, cosmap) and loss weighting vs diffusers.training_utils, same generator |
— | identical | identical |
| Image transform (resize, flip, center / random crop) vs the script, 4 image sizes | — | identical | identical |
| CLIP-L, CLIP-G and T5 token ids vs the script's tokenization | tiny SD3 | identical | identical |
First-step draws (posterior noise, noise, timesteps): each weighting scheme with preconditioned outputs, and logit_normal without |
tiny SD3 | identical | identical |
| First-step loss and LoRA gradients, same five cases | tiny SD3 | 0 | 0 |
Saved adapter loaded by diffusers load_lora_weights vs imagehub, 3 steps, guidance 4 |
tiny SD3 | 0 (pixels) | 0 |
| Gradients with and without gradient checkpointing | tiny SD3 | identical | identical |
The script tosses the flip coin with Python's random, and its RandomHorizontalFlip(p=1.0)
draws one more torch number before the crop offsets; imagehub draws the coin from its generator.
The comparison replays imagehub's coin on both sides. sd3.5-large-turbo is few-step distilled
and stays inference_only.
FLUX.1¶
Validated on CPU on November 7, 2026 against diffusers 0.40.0
train_dreambooth_lora_flux.py and train_dreambooth_lora_flux_kontext.py (peft 0.21.2).
Tiny random snapshots cover FLUX.1 with and without guidance embeddings and text-only Kontext.
The Kontext script takes the same plain FLUX path when no conditioning-image column is supplied;
image-conditioned training is out of scope.
| Comparison | Model | Largest observed difference | Tolerance |
|---|---|---|---|
| Image transform and CLIP / T5 token ids | tiny FLUX.1 | identical | identical |
| First-step posterior noise, noise, timesteps, loss and every LoRA gradient, all weighting schemes | tiny FLUX.1 | 0 | 0 |
| Guidance embedding on / off | tiny FLUX.1 | 0 | 0 |
| Gradient checkpointing over double- and single-stream blocks | tiny FLUX.1 | identical | identical |
Saved adapter loaded by diffusers load_lora_weights vs imagehub, 3 steps |
tiny FLUX.1 | 0 (pixels) | 0 |
Saved Kontext adapter (with proj_mlp) in FluxKontextPipeline vs imagehub, with a condition image |
tiny Kontext | 0 (pixels) | 0 |
| Random PEFT adapters (transformer + CLIP), single, stacked 0.8 / 0.5 and fused, FLUX.1 and Kontext with a condition image | tiny | 0 (pixels) | 0 |
| CLIP pooled embedding with a CLIP adapter | tiny FLUX.1 | 0 | 0 |
Sana¶
Validated on CPU on November 7, 2026 against diffusers 0.40.0
train_dreambooth_lora_sana.py and PEFT 0.21.2. Tiny random snapshots cover the standard Sana
transformer and deterministic DC-AE path; Sana-Sprint remains inference-only because its
checkpoint is distilled.
| Comparison | Model | Largest observed difference | Tolerance |
|---|---|---|---|
| Default arguments vs the script's argparse defaults (weighting scheme, logit mean / std, mode scale, sequence length 300, no complex-human instruction, rank / alpha, targets) | — | identical | identical |
| Image transform; Gemma 2 token ids and attention mask without a complex-human instruction, as the script's default | tiny Sana | identical | identical |
| First-step noise, timestep, loss and every LoRA gradient, all weighting schemes | tiny Sana | 0 | 0 |
| Gradient checkpointing over transformer blocks | tiny Sana | identical | identical |
Saved adapter loaded by diffusers load_lora_weights vs imagehub, 3 steps |
tiny Sana | 0 (pixels) | 0 |
Random PEFT transformer adapters: to_k, to_q, to_v, to_out.0, linear, linear_1, linear_2, proj_out; single, stacked 0.8 / 0.5 and fused |
tiny Sana | 0 (pixels) | 0 |
Qwen-Image training¶
Validated on CPU float32 on November 7, 2026 against diffusers 0.40.0
train_dreambooth_lora_qwen_image.py with PEFT 0.21.2. Qwen-Image 2.1 is validated against
train_dreambooth_lora_qwenimage21.py at diffusers commit 7997e4eab (no release contains it
yet), in a separate environment with that commit, PEFT 0.21.2 and PyTorch 2.14.1, as for its
inference validation.
| Comparison | Model | Largest observed difference | Tolerance |
|---|---|---|---|
| Default arguments vs the script's argparse defaults: objective, collator, rank / alpha and transformer LoRA targets | tiny Qwen-Image | identical | identical |
| Image transform; prompt token ids and attention mask after dropping the chat-template prefix | tiny Qwen-Image | identical | identical |
| First-step posterior noise, noise, timestep, loss and every LoRA gradient, all weighting schemes | tiny Qwen-Image | 0 | 0 |
| Random PEFT transformer adapters: single, stacked 0.8 / 0.5 and fused | tiny Qwen-Image | 0 (latents and pixels) | 0 |
Three-step trained adapter loaded by diffusers load_lora_weights |
tiny Qwen-Image | 0 (latents and pixels) | 0 |
| Default arguments vs the script's argparse defaults: weighting, logit mean / std, mode scale, rank / alpha 16 / 16 and transformer LoRA targets | tiny Qwen-Image 2.1 | identical | identical |
| Image transform; prompt token ids and attention mask after dropping the system-template prefix | tiny Qwen-Image 2.1 | identical | identical |
| First-step posterior noise, noise, timestep, loss and every LoRA gradient, all weighting schemes; prompt/image-slot mask and flat packing | tiny Qwen-Image 2.1 | 0 | 0 |
| Random PEFT transformer adapters: single, stacked 0.8 / 0.5 and fused | tiny Qwen-Image 2.1 | 0 (latents and pixels) | 0 |
Three-step trained adapter loaded by diffusers load_lora_weights |
tiny Qwen-Image 2.1 | 0 (latents and pixels) | 0 |
Z-Image training¶
Validated on CPU float32 on November 7, 2026 against diffusers 0.40.0
train_dreambooth_lora_z_image.py with PEFT 0.21.2. The tiny random snapshot covers Z-Image's
chat-template prompt encoding, deterministic KL posterior mean and published latent shift / scale.
| Comparison | Model | Largest observed difference | Tolerance |
|---|---|---|---|
| Default arguments vs the script's argparse defaults: weighting, logit mean / std, mode scale, sequence length 512, rank / alpha and transformer targets | tiny Z-Image | identical | identical |
| Image transform; chat-template token ids and attention mask | tiny Z-Image | identical | identical |
| First-step noise, timestep, loss and every LoRA gradient, all weighting schemes | tiny Z-Image | 0 | 0 |
| Gradient checkpointing over transformer blocks | tiny Z-Image | identical | identical |
| Random PEFT transformer adapters: single, stacked 0.8 / 0.5 and fused | tiny Z-Image | 0 (latents and pixels) | 0 |
Three-step trained adapter loaded by diffusers load_lora_weights |
tiny Z-Image | 0 (latents and pixels) | 0 |
FLUX.2 klein training (November 7, 2026)¶
FLUX.2 klein LoRA training is checked on CPU float32 against diffusers 0.40.0 and PEFT 0.21.2.
The tiny random snapshot compares rank / alpha 4 / 4, dropout 0, "none" weighting, logit mean /
std 0 / 1, mode scale 1.29, 512-token prompts, Qwen layers 9 / 18 / 27 (the official FLUX.2 layers, passed to the script, whose default is the FLUX.2 [dev] set 10 / 20 / 30), guidance 3.5 and the
literal target list. It also compares the fixed resize/crop/flip transform, chat-template token
ids and attention mask. For every weighting scheme, first-step noise, timestep, sigma, loss and
every LoRA gradient are exact. Random single, stacked (0.8 / 0.5) and fused PEFT adapters, plus a
three-step trained adapter, first prove their registered targets and changed diffusers output, then
compare final latents and uint8 pixels exactly.
| Comparison | Model | Largest observed difference | Tolerance |
|---|---|---|---|
| Defaults, literal target-module selection, image transform, token ids and mask | tiny FLUX.2 klein | identical | exact |
| First-step noise, timestep, sigma, loss and every LoRA gradient, all weighting schemes | tiny FLUX.2 klein | 0 | 0 |
| Gradient checkpointing over double- and single-stream blocks | tiny FLUX.2 klein | identical | exact |
Validation image vs pipeline(..., lora=adapter) |
tiny FLUX.2 klein | exact pixels | exact |
| Random PEFT adapter: single, stacked 0.8 / 0.5 and fused; registered targets, changed output, final latents and uint8 pixels | tiny FLUX.2 klein | 0 / 0 | 0 / 0 |
| Three-step trained adapter loaded in diffusers; registered targets, changed output, final latents and uint8 pixels | tiny FLUX.2 klein | 0 / 0 | 0 / 0 |
Krea 2 training (November 7, 2026)¶
Krea 2 training has CPU float32 coverage against diffusers 0.40.0. The non-distilled Raw training snapshot uses rank / alpha 32 / 32,
"none" weighting, logit mean / std 0 / 1, mode scale 1.29 and a 512-token prompt. Turbo is
refused before weights load, and a unit test trains an adapter on Raw and loads it on the
same-weight Turbo layout.
| Comparison | Model | Largest observed difference | Tolerance |
|---|---|---|---|
| Profile defaults, literal target-module registration, image transform, prompt token ids / mask, and diffusers prompt embeddings / mask | tiny Krea 2 | identical | exact |
| First-step posterior noise, noise, timestep, sigma, loss and every LoRA gradient for all five weighting schemes | tiny Krea 2 | 0 | 0 |
| Random PEFT adapter: single, stacked 0.8 / 0.5 and fused; registered targets, changed output, final latents and uint8 pixels | tiny Krea 2 | 0 / 0 | 0 / 0 |
| Three-step trained adapter loaded in diffusers; registered targets, changed output, final latents and uint8 pixels | tiny Krea 2 | 0 / 0 | 0 / 0 |
| Distilled-snapshot rejection before weight loading; one-step Raw training and Turbo adapter load | tiny Krea 2 | covered by unit test | — |
Gradient checkpointing; validation image vs pipeline(..., lora=adapter) |
tiny Krea 2 | identical pixels / gradients | exact |
Training runtime (CPU)¶
| Check | Result |
|---|---|
| Two-process gloo run vs one-process replay of both processes' batches and noise with averaged gradients (cosine schedule, clipping) | adapter identical; logged loss equal |
| Both processes after a two-process run | identical adapters and logs |
| Two-process resume from step 2 vs uninterrupted 4 steps | identical |
| Resuming a two-process checkpoint in one process | refused |
eval_loss of two processes vs one process |
equal (to float rounding of the sum) |
evaluate() twice on the same adapter; training with and without evaluation |
identical |
Validation image vs pipeline(..., lora=adapter) |
pixels identical |
A multi-GPU NCCL run has not been made yet.
LoRA adapters¶
Validated on CPU on November 4, 2026 against diffusers 0.40.0 load_lora_weights with
peft 0.21.2 (transformers 5.18.0, PyTorch 2.15.0).
- Random adapters were written in each supported layout for tiny random-weight Stable Diffusion
1.x and SDXL snapshots, covering the U-Net attention,
proj_in/proj_out, feed-forward and CLIP text-encoder layers. Both sides generated 3 steps at 32×32 from seed 42. - On November 7, 2026, diffusers / PEFT adapters for a tiny SD3 snapshot covered the transformer's joint attention and feed-forward layers and both CLIP text encoders. Both sides generated 3 steps at 32×32 from seed 42 with guidance 4. The guided prompt embeddings were compared too, because text-encoder adapters barely move the tiny model's pixels.
- On November 7, 2026, tiny Sana PEFT transformer adapters covered the training defaults
(
to_k,to_q,to_v) plusto_out.0,linear,linear_1,linear_2, andproj_out. Both sides generated 3 steps at 32×32 from seed 42 with guidance 4 and the same explicit complex-human instruction. Single, stacked (0.8 / 0.5), fused, and 3-step trained adapters were compared. - On November 7, 2026, tiny Z-Image PEFT transformer adapters covered
to_k,to_q,to_vandto_out.0. Both sides generated 3 steps at 32x32 from seed 42 with guidance 4. Single, stacked (0.8 / 0.5), fused and 3-step trained adapters were compared, including final latents when uint8 pixels did not move. - On November 7, 2026, tiny Lumina 2 PEFT transformer adapters covered
to_k,to_q,to_vandto_out.0. Both sides generated 3 steps at 32x32 from the same explicit latents with guidance 4 and the pipeline's default system prompt. Single, stacked (0.8 / 0.5), fused and 3-step trained adapters were compared; each loaded adapter registered those targets and changed the diffusers output before final latents and uint8 pixels were checked. - On November 7, 2026, tiny HiDream-I1 PEFT transformer adapters covered the image-stream
to_k,to_q,to_vandto_outtargets. Both sides generated 3 steps from the same explicit latents with guidance 4 and sequence length 8. Single, stacked (0.8 / 0.5), fused and 3-step trained adapters were compared; each diffusers-loaded adapter registered only those image-stream targets, changed the diffusers output, then matched final latents and uint8 pixels exactly. - On November 7, 2026, tiny FLUX.2 klein PEFT transformer adapters covered the literal training
targets:
to_k,to_q,to_v,to_out.0,to_qkv_mlp_proj, and matching first-24 single-blockattn.to_outpaths. Single, stacked (0.8 / 0.5), fused and 3-step trained adapters registered the intended targets, changed the diffusers output, then matched final latents and uint8 pixels exactly. - One published adapter,
latent-consistency/lcm-lora-sdv1-5atcf2fced511dbe7e26c8d1d397e728fbab875db4b(kohya layout, per-module alphas, 278 layers of which 96 are convolutions, 50 of them 3×3), was loaded onsd-v1-5and generated 2 steps at 256×256 from seed 0. - Every image is bit-identical to the diffusers one.
Comparisons (CPU, float32)¶
| Comparison | Family | Pixels max abs | Tolerance |
|---|---|---|---|
| diffusers / PEFT layout, alpha in the header | SD 1.x, SDXL, SD3, FLUX.1, Sana, Z-Image | 0 | 0 |
Random PEFT transformer adapter: to_k, to_q, to_v, to_out.0, linear, linear_1, linear_2, proj_out; 3 steps at 32×32, seed 42, guidance 4, explicit complex-human instruction |
Sana | 0 | 0 |
| 3-step trained transformer adapter: 3 steps at 32×32, seed 42, guidance 4, explicit complex-human instruction | Sana | 0 | 0 |
Random PEFT transformer adapter: to_k, to_q, to_v, to_out.0; single, stacked 0.8 / 0.5 and fused |
Z-Image | 0 | 0 |
| 3-step trained transformer adapter: 3 steps at 32×32, seed 42, guidance 4 | Z-Image | 0 | 0 |
Random PEFT transformer adapter: to_k, to_q, to_v, to_out.0; single, stacked 0.8 / 0.5 and fused; final latents and uint8 pixels |
Lumina 2 | 0 | 0 |
| 3-step trained transformer adapter: 3 steps at 32×32 from explicit latents, guidance 4, default system prompt; final latents and uint8 pixels | Lumina 2 | 0 | 0 |
Random PEFT transformer adapter: to_k, to_q, to_v, to_out; single, stacked 0.8 / 0.5 and fused; final latents and uint8 pixels |
HiDream-I1 | 0 / 0 | 0 / 0 |
| 3-step trained transformer adapter: 3 steps from explicit latents, guidance 4, sequence length 8; final latents and uint8 pixels | HiDream-I1 | 0 / 0 | 0 / 0 |
| Random PEFT transformer adapter: literal targets; single, stacked 0.8 / 0.5 and fused; final latents and uint8 pixels | FLUX.2 klein | 0 / 0 | 0 / 0 |
| 3-step trained transformer adapter; final latents and uint8 pixels | FLUX.2 klein | 0 / 0 | 0 / 0 |
| Random PEFT transformer adapter: literal targets; single, stacked 0.8 / 0.5 and fused; final latents and uint8 pixels | Krea 2 | 0 / 0 | 0 / 0 |
| 3-step trained transformer adapter (trained on the raw layout); final latents and uint8 pixels | Krea 2 | 0 / 0 | 0 / 0 |
Random PEFT transformer adapters in diffusers layout on dequantized fp8 weights: split q/k/v onto fused qkv, to_out.0 onto o, MLP w1 / w2 / w3; single, stacked 0.8 / 0.5, fused and q-only; conditional transformer output |
Ideogram 4 | 0 | 0 |
| diffusers / PEFT layout, no alpha | SD 1.x, SD3 | 0 | 0 |
| Prompt embeddings with text-encoder adapters (CLIP-L, CLIP-G, pooled, negative prompt) | SD3 | 0 | 0 |
kohya layout with .alpha |
SD 1.x, SDXL | 0 | 0 |
| kohya layout, SDXL original block names | SDXL | 0 | 0 |
| legacy diffusers layout | SD 1.x | 0 | 0 |
| Two stacked adapters, scales 0.8 / 0.5 | SD 1.x, SDXL, SD3, FLUX.1, Sana, Z-Image | 0 | 0 |
| Two stacked adapters, fused | SD 1.x, SDXL, SD3, FLUX.1, Sana, Z-Image | 0 | 0 |
Published LCM-LoRA on sd-v1-5 |
SD 1.x | 0 | 0 |
diffusers 0.40 rewrites kohya and legacy text-encoder keys to the transformers 4 module names
(text_model.encoder...), which transformers 5 CLIP models no longer have, so it cannot load those
text-encoder adapters. Those two layouts are therefore compared on the U-Net; imagehub reads their
text-encoder keys in both spellings, which the unit tests cover.
Cosmos-Predict2¶
cosmos-predict2-2b-t2i is pinned to NVIDIA revision acdb5fd. The offline suite strictly
matches the 195 T5, 567 transformer, and 194 WAN VAE published tensor names and shapes. Against
diffusers 0.40.0 and transformers 5.18.0, random-weight T5, transformer, VAE, tokenizer, Karras
schedule, and tiny whole-pipeline comparisons are exact. The tiny end-to-end cases cover
classifier-free guidance on and off, non-square output, seed 0 and seed 42, including the
reference pipeline's uint8 conversion.
The opt-in real-checkpoint suite separately compares tokenization, T5 conditioning, one denoiser
step, VAE decoding, and two public pipeline(...) generations recorded as
e2e_cosmos-predict2-2b-t2i and seed_e2e_cosmos-predict2-2b-t2i. All 8 real tests passed, and
the observed difference in every recorded comparison is exactly 0 (0 means exact): on CPU in
float32, the local implementation produces a bit-identical image to the one diffusers produces.
The gated checkpoint's T5 shards are bfloat16 and are upcast to float32 on both sides. The
real-checkpoint run took 1 minute on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| Tensor header check (T5, transformer, WAN VAE; 195 / 567 / 194 tensors) | 0 missing / extra / mismatched | exact |
| T5 tokenizer token ids | identical | exact |
| T5 text encoder | 0 | 0 |
| Transformer single step | 0 | 0 |
| WAN VAE decode | 0 | 0 |
| Karras schedule | exact | exact |
| End to end: final pixels (seed 0) | 0 | 0 |
| End to end, seed 42 (pixels) | 0 | 0 |
Records: cosmos2_fixture_header, cosmos2_tokenizer, cosmos2_real_tokenizer,
cosmos2_real_text_encoder, cosmos2_real_transformer, cosmos2_real_vae, cosmos2_scheduler,
e2e_cosmos-predict2-2b-t2i and seed_e2e_cosmos-predict2-2b-t2i,
validation-measurements.json.
PixelDiT¶
pixeldit is pinned to nvidia/PixelDiT-1300M-1024px revision 7c63b99 and the official
NVlabs/PixelDiT implementation revision f7d7c35. Offline validation strictly checks all 391
published denoiser tensors plus the Gemma-2 shards. Exact random-weight comparisons cover the
dual-level transformer, the second-order FLOW DPM-Solver++ trajectory, and a tiny whole generation
ending in uint8 pixels. The real-checkpoint suite separately records pixeldit_text_encoder,
pixeldit_denoiser_step, e2e_pixeldit (seed 0) and seed_e2e_pixeldit (seed 42, a separate public
pipeline(...) run). Both end-to-end cases use two steps and 16×16 pixels to bound CPU time; model
defaults remain 50 steps and 1024×1024.
Validated on CPU in the same environment as above. All 8 real tests passed, and the observed difference in every recorded comparison is exactly 0 (0 means exact): on CPU in float32, the local implementation produces a bit-identical image to the one the official implementation produces. The published denoiser weights are float32; the published Gemma-2 text-encoder weights are bfloat16 and are upcast to float32 on both sides. The real-checkpoint run took less than a minute on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| Tensor header check (published denoiser; 391 tensors) | 0 missing / extra / mismatched | exact |
| Gemma tokenizer token ids and text-encoder hidden state (4 prompts) | identical / 0 | exact / 0 |
| Denoiser single step (published weights, 16×16 input, t = 0.5) | 0 | 0 |
| Random-weight transformer (tiny configuration) | 0 | 0 |
| Random-weight second-order flow DPM-Solver++ trajectory (4 steps) | 0 | 0 |
| Random-weight tiny end to end (uint8 pixels) | 0 | 0 |
| End to end: final pixels (16×16, 2 steps, seed 0) | 0 | 0 |
| End to end, seed 42 (pixels) | 0 | 0 |
Records: pixeldit_fixture_header, pixeldit_text_encoder, pixeldit_denoiser_step,
pixeldit_transformer_random, pixeldit_flow_dpm_random, pixeldit_tiny_e2e, e2e_pixeldit
and seed_e2e_pixeldit,
validation-measurements.json; components_pixeldit and
tests.reference_summary, validation.json.
Scope¶
Latent Diffusion Model¶
ldm-text2im-large-256 was loaded strictly from CompVis/ldm-text2im-large-256 at revision
30de525 and compared with diffusers 0.40.0 LDMTextToImagePipeline on CPU in float32. All 5
real-checkpoint tests passed, and the observed difference in every recorded comparison is
exactly 0 (0 means exact): on CPU in float32, the local implementation produces a bit-identical
image to the one diffusers produces. The real-checkpoint run took 1 minute on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
Tensor header check (bert, unet, vqvae; 422 / 686 / 248 tensors) |
0 missing / extra / mismatched | exact |
| BERT tokenizer token ids (prompt at 77 tokens) | identical | exact |
| LDM BERT text encoder | 0 | 0 |
| U-Net single step (t = 981) | 0 | 0 |
| VQ-VAE decode | 0 | 0 |
| Random-weight BERT encoder (tiny configuration) | 0 | 0 |
| End to end: final pixels (50 steps, guidance 1.0, seed 0) | 0 | 0 |
| End to end, seed 42 (pixels) | 0 | 0 |
The end-to-end prompt was "a photograph of an astronaut riding a horse"; both runs use the
defaults of 50 steps, guidance 1.0 and 256×256. Records: header_ldm-text2im-large-256,
ldm_tokenizer, ldm_text_encoder, ldm_unet, ldm_vqvae_decode, ldm_random_bert,
e2e_ldm-text2im-large-256 and seed_e2e_ldm-text2im-large-256,
validation-measurements.json; tensor_header_check, end_to_end
and seed_end_to_end, validation.json.
DreamLite¶
Validated on CPU in the same environment as above. dreamlite-base and dreamlite-mobile were
loaded strictly from carlofkl/DreamLite-base at revision a9a0f15 and
carlofkl/DreamLite-mobile at revision a48c656 and compared with diffusers 0.40.0
DreamLitePipeline and DreamLiteMobilePipeline and transformers 5.18.0 on CPU in float32. Both
checkpoints publish bfloat16 weights, which are upcast to float32 on both sides. Their 1,075 U-Net
and 134 TinyVAE published tensor names and shapes match the native modules strictly. Random-weight
U-Net and TinyVAE comparisons are exact, as are tiny end-to-end comparisons against both diffusers
pipelines with guidance on and off, non-square output, and two seeds. All 8 real-checkpoint tests
passed, and the observed difference in every recorded comparison is exactly 0 (0 means exact):
on CPU in float32, the local implementation produces a bit-identical image to the one diffusers
produces. The real-checkpoint run took 2 minutes on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Checkpoint | Largest observed difference | Tolerance |
|---|---|---|---|
Tensor header check (unet, vae; 1,075 / 134 tensors) |
both | 0 missing / extra / mismatched | exact |
| Qwen3-VL text encoder through the published prompt template | both | 0 | 0 |
| U-Net single step (t = 500, 8×12 latent grid) | both | 0 | 0 |
| TinyVAE decode (4×6 latents) | both | 0 | 0 |
FlowMatch Euler timesteps (4 steps, mu for 1024×768) |
both | 0 | 0 |
End to end: final pixels (1024×768, 4 steps, initial noise from seed 1234, guidance 3.5 for dreamlite-base) |
both | 0 | 0 |
| End to end, seed 42 (pixels) | both | 0 | 0 |
The end-to-end prompt was "a photograph of an astronaut riding a horse". Both runs use 4 steps
instead of dreamlite-base's 30-step default only to shorten the CPU run; dreamlite-mobile is
the four-step distilled checkpoint and does not use classifier-free guidance. Records:
header_dreamlite-base, header_dreamlite-mobile, components_dreamlite-base,
components_dreamlite-mobile, e2e_dreamlite-base, e2e_dreamlite-mobile,
seed_e2e_dreamlite-base and seed_e2e_dreamlite-mobile,
validation-measurements.json; tensor_header_check,
end_to_end and seed_end_to_end under dreamlite-base and dreamlite-mobile, and
components_dreamlite, validation.json.
Stable Diffusion 1.x¶
- 5 checkpoints,
full:sd-v1-1,sd-v1-2,sd-v1-3,sd-v1-4,sd-v1-5. The published tensors were loaded strictly. A 50-step generation from the same initial noise was compared against diffusers. - The observed difference in every comparison is exactly 0. On CPU in float32, the local implementation produces a bit-identical image to the one diffusers produces.
- This statement is not a quality benchmark.
Comparisons (CPU, float32)¶
| Comparison | Checkpoint | Largest observed difference | Tolerance |
|---|---|---|---|
| Tokenizer token ids | v1.4 (14 prompts) and v1.1–v1.5 (1,000 random prompts each) | identical | identical |
| CLIP text encoder | v1.4 | 0 | 0 |
| U-Net single step (t = 981, 501, 1) | v1.4 | 0 | 0 |
| VAE decode / encode | v1.4 | 0 / 0 | 0 |
| Initial noise from seed | — | identical | identical |
| End to end: final latent / pixels | v1.1 | 0 / 0 | 0 / 0 |
| End to end: final latent / pixels | v1.2 | 0 / 0 | 0 / 0 |
| End to end: final latent / pixels | v1.3 | 0 / 0 | 0 / 0 |
| End to end: final latent / pixels | v1.4 | 0 / 0 | 0 / 0 |
| End to end: final latent / pixels | v1.5 | 0 / 0 | 0 / 0 |
| End to end, seed 42 (pixels) | v1.5 | 0 | 0 |
End-to-end test settings: prompt "a photograph of an astronaut riding a horse", 50 steps,
guidance 7.5, 512×512, initial noise seed=1234. On the diffusers side, safety_checker=None
and feature_extractor=None were used.
Tolerances are set by this rule: if the measured difference is 0, the tolerance is 0; otherwise it is the next order of magnitude up. Since every measurement came out 0, no tolerance needed to be loosened.
Tensor header check¶
For each of the five checkpoints, the tensor names and shapes of three components (text_encoder,
unet, vae) were read without loading the weights into memory (header only for safetensors,
map_location="meta" for .bin) and compared with the model's state_dict after
prepare_state_dict. Across all 5 × 3 components, missing, extra and shape-mismatched tensors are
0. Tensor counts: 196 / 686 / 248 (after dropping text_model.embeddings.position_ids).
Record: tensor_header_check, validation.json.
DreamShaper 8¶
Validated on CPU in the same environment as above. dreamshaper-8, full in the catalog, reuses
the Stable Diffusion 1.x components and the published second-order DEIS scheduler. Its published
float32 safetensors headers contain the supported CLIP, U-Net and VAE layouts and were loaded
strictly. The reference suite checks strict headers, tokenizer ids, CLIP output, one U-Net step, VAE
decode, the DEIS trajectory, and separate default 512×512 public-pipeline generations at seeds 0
and 42. All 4 real tests passed, and the observed difference in every recorded comparison is
exactly 0 (0 means exact): on CPU in float32, the local implementation produces a bit-identical
image to the one diffusers produces. The published weights are float32, so no upcast is involved.
The real-checkpoint run took 5 minutes on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
Tensor header check (text_encoder, unet, vae; 196 / 686 / 248 tensors) |
0 missing / extra / mismatched | exact |
| Tokenizer token ids (4 prompts) | identical | exact |
| CLIP text encoder | 0 | 0 |
| U-Net single step (t = 500) | 0 | 0 |
| VAE decode | 0 | 0 |
| Published DEIS trajectory (4 steps) | 0 | 0 |
| End to end: final pixels (512×512, 50 steps, guidance 7.5, seed 0) | 0 | 0 |
| End to end, seed 42 (pixels) | 0 | 0 |
Records: header_dreamshaper-8, sd_tokenizer_dreamshaper-8, sd_text_encoder_dreamshaper-8,
sd_unet_dreamshaper-8, sd_vae_decode_dreamshaper-8, sd_scheduler_dreamshaper-8,
e2e_dreamshaper-8 and seed_e2e_dreamshaper-8,
validation-measurements.json.
SDXS-512¶
sdxs-512-0.9 uses the existing Stable Diffusion pipeline with its published one-step Euler
schedule and TinyVAE decoder. Offline tests cover the pinned full-precision tensor headers (373
published text-encoder, 358 U-Net, and 134 VAE tensors), exact random-weight U-Net parity, and
exact tiny end-to-end parity at the defaults and with guidance at a non-square size. Dedicated
real-checkpoint tests compare tokenizer and text-encoder outputs, one denoiser step, VAE decoding,
the scheduler, and separate 512×512 public-pipeline generations at seed 0 and seed 42. CPU float32
comparisons require exact tensor and uint8 equality. Validated on CPU in the same environment as
above. All 5 real tests passed, and the observed difference in every recorded comparison is
exactly 0 (0 means exact): on CPU in float32, the local implementation produces a bit-identical
image to the one diffusers produces. The published weights are float32. The real-checkpoint run
took 0 minutes on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
Tensor header check (text_encoder, unet, vae; 372 / 358 / 134 tensors) |
0 missing / extra / mismatched | exact |
| Tokenizer token ids | identical | exact |
| CLIP text encoder | 0 | 0 |
| U-Net single step | 0 | 0 |
| VAE decode | 0 | 0 |
| Euler scheduler timesteps / sigmas | 0 / 0 | 0 |
| End to end: final pixels (512×512, 1 step, guidance 0.0, seed 0) | 0 | 0 |
| End to end, seed 42 (pixels) | 0 | 0 |
Records: header_sdxs-512-0.9, sdxs_tokenizer_text_encoder_sdxs-512-0.9,
sdxs_components_sdxs-512-0.9, e2e_sdxs-512-0.9 and seed_e2e_sdxs-512-0.9,
validation-measurements.json.
Segmind Small-SD and Tiny-SD¶
Validated on CPU in the same environment as above. segmind-small-sd and segmind-tiny-sd, both
full in the catalog, were loaded strictly and compared with diffusers 0.40.0
StableDiffusionPipeline. Validation covers strict published tensor layouts, tokenizer and CLIP
output, a U-Net step, VAE decode, the published DEIS/DPM-Solver++ trajectory, and four-step 64×64
public-pipeline runs at seed 0 and seed 42; the random-weight U-Net and tiny end-to-end tests
compare both pruned layouts exactly with diffusers. CPU float32 comparisons require exact tensor
and uint8 image equality. All 14 real tests passed, and the observed difference in every recorded
comparison is exactly 0 (0 means exact): on CPU in float32, the local implementation produces a
bit-identical image to the one diffusers produces. Small-SD's published tensors are float32, and
Tiny-SD's published float16 tensors are upcast to float32 by both implementations. The
real-checkpoint run took 1 minute on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Checkpoint | Largest observed difference | Tolerance |
|---|---|---|---|
Tensor header check (text_encoder, unet, vae; 196 / 442 / 248 tensors) |
small-sd | 0 missing / extra / mismatched | exact |
Tensor header check (text_encoder, unet, vae; 196 / 358 / 248 tensors) |
tiny-sd | 0 missing / extra / mismatched | exact |
| Tokenizer token ids (4 prompts) | both | identical | exact |
| CLIP text encoder | both | 0 | 0 |
| U-Net single step (t = 500) | both | 0 | 0 |
| VAE decode | both | 0 | 0 |
| Published scheduler (DEIS / DPM-Solver++, 4 steps) | both | 0 | 0 |
| End to end: final pixels (64×64, 4 steps, guidance 5.0, seed 0) | both | 0 | 0 |
| End to end, seed 42 (pixels) | both | 0 | 0 |
Records: header_segmind-small-sd, header_segmind-tiny-sd, sd_tokenizer_*, sd_text_encoder_*,
sd_unet_*, sd_vae_decode_*, sd_scheduler_*, e2e_segmind-small-sd, e2e_segmind-tiny-sd,
seed_e2e_segmind-small-sd and seed_e2e_segmind-tiny-sd,
validation-measurements.json; components_segmind_sd and
tests.reference_summary, validation.json.
The end-to-end comparisons are deliberately short: 64×64 for four steps. e2e_segmind-small-sd and
e2e_segmind-tiny-sd use seed 0, seed_e2e_segmind-small-sd and seed_e2e_segmind-tiny-sd use
seed 42, and all four runs use guidance 5.0 and the same negative prompt. Their final pixels differ
by exactly 0 in every run. This statement is not a quality benchmark.
BK-SDM¶
bk-sdm-base, bk-sdm-small and bk-sdm-tiny use the existing Stable Diffusion pipeline with
block-pruned U-Nets. They are pinned to the three author repositories at revisions 0c03b7a,
572238d and 0364108 and are compared with diffusers 0.40.0 StableDiffusionPipeline and
transformers 5.18.0 on CPU in float32. All 27 real tests passed, and the observed difference in
every recorded comparison is exactly 0 (0 means exact): on CPU in float32, the local
implementation produces a bit-identical image to the one diffusers produces. The real-checkpoint
run took less than a minute on an M2 Max (CPU).
The dedicated reference suite checks every published tensor header; tokenizer and text-encoder
outputs; one U-Net step; VAE encode and decode; PNDM timesteps; exact random-weight U-Net parity at
tiny and published layouts; and tiny guided and unguided end-to-end generation at non-square sizes.
For each checkpoint, e2e_<model-id> is a four-step 64×64 public-pipeline run at seed 0 and
seed_e2e_<model-id> is a separate run at seed 42. Both use guidance 7.5 and the prompt
"a photograph of an astronaut riding a horse".
Comparisons (CPU, float32)¶
| Comparison | Checkpoint | Largest observed difference | Tolerance |
|---|---|---|---|
Tensor header check (text_encoder, unet, vae; 196 / 442 / 248 tensors) |
bk-sdm-base | 0 missing / extra / mismatched | exact |
Tensor header check (text_encoder, unet, vae; 196 / 396 / 248 tensors) |
bk-sdm-small | 0 missing / extra / mismatched | exact |
Tensor header check (text_encoder, unet, vae; 196 / 358 / 248 tensors) |
bk-sdm-tiny | 0 missing / extra / mismatched | exact |
| Tokenizer token ids (4 prompts) | all three | identical | exact |
| CLIP text encoder | all three | 0 | 0 |
| U-Net single step (t = 500) | all three | 0 | 0 |
| VAE decode / encode | all three | 0 / 0 | 0 / 0 |
| PNDM scheduler timesteps (4 steps) | all three | 0 | 0 |
| End to end: final pixels (64×64, 4 steps, guidance 7.5, seed 0) | all three | 0 | 0 |
| End to end, seed 42 (pixels) | all three | 0 | 0 |
Records: header_bk-sdm-base, header_bk-sdm-small, header_bk-sdm-tiny,
bk_sdm_*_bk-sdm-base, bk_sdm_*_bk-sdm-small, bk_sdm_*_bk-sdm-tiny,
e2e_bk-sdm-base, e2e_bk-sdm-small, e2e_bk-sdm-tiny, seed_e2e_bk-sdm-base,
seed_e2e_bk-sdm-small and seed_e2e_bk-sdm-tiny.
BK-SDM v2¶
bk-sdm-v2-base, bk-sdm-v2-small and bk-sdm-v2-tiny reuse the Stable Diffusion pipeline with
the BK-SDM pruning layouts and Stable Diffusion 2.1-base conditioning. They are pinned to the author
repositories at revisions 21915be, ad82c51 and 68277af and are compared with diffusers 0.40.0
StableDiffusionPipeline and transformers 5.18.0 on CPU in float32. All 27 real tests passed, and
the observed difference in every recorded comparison is exactly 0 (0 means exact): on CPU in
float32, the local implementation produces a bit-identical image to the one diffusers produces. The
real-checkpoint run took less than a minute on an M2 Max (CPU).
The dedicated suite checks the published tensor headers; tokenizer and OpenCLIP-H text-encoder
outputs; one U-Net step; VAE encode and decode; PNDM timesteps; exact random-weight U-Net parity at
tiny and published layouts; and exact tiny guided and unguided end-to-end generation at non-square
sizes. For each checkpoint, e2e_<model-id> is a separate four-step 64×64 public-pipeline
comparison at seed 0 and seed_e2e_<model-id> is a run at seed 42; both use guidance 7.5 and the
prompt "a photograph of an astronaut riding a horse".
Comparisons (CPU, float32)¶
| Comparison | Checkpoint | Largest observed difference | Tolerance |
|---|---|---|---|
Tensor header check (text_encoder, unet, vae; 372 / 442 / 248 tensors) |
bk-sdm-v2-base | 0 missing / extra / mismatched | exact |
Tensor header check (text_encoder, unet, vae; 372 / 396 / 248 tensors) |
bk-sdm-v2-small | 0 missing / extra / mismatched | exact |
Tensor header check (text_encoder, unet, vae; 372 / 358 / 248 tensors) |
bk-sdm-v2-tiny | 0 missing / extra / mismatched | exact |
| Tokenizer token ids (4 prompts) | all three | identical | exact |
| CLIP (OpenCLIP-H) text encoder | all three | 0 | 0 |
| U-Net single step (t = 500) | all three | 0 | 0 |
| VAE decode / encode | all three | 0 / 0 | 0 / 0 |
| PNDM scheduler timesteps (4 steps) | all three | 0 | 0 |
| End to end: final pixels (64×64, 4 steps, guidance 7.5, seed 0) | all three | 0 | 0 |
| End to end, seed 42 (pixels) | all three | 0 | 0 |
Records: header_bk-sdm-v2-base, header_bk-sdm-v2-small, header_bk-sdm-v2-tiny,
bk_sdm_v2_*_bk-sdm-v2-base, bk_sdm_v2_*_bk-sdm-v2-small, bk_sdm_v2_*_bk-sdm-v2-tiny,
e2e_bk-sdm-v2-base, e2e_bk-sdm-v2-small, e2e_bk-sdm-v2-tiny, seed_e2e_bk-sdm-v2-base,
seed_e2e_bk-sdm-v2-small and seed_e2e_bk-sdm-v2-tiny,
validation-measurements.json; components_bk_sdm_v2,
validation.json.
LCM¶
Validated on CPU in the same environment as above. lcm-dreamshaper-v7 was loaded strictly from
SimianLuo/LCM_Dreamshaper_v7. The observed difference in every comparison is exactly 0: on
CPU in float32, the local implementation produces a bit-identical result to the reference. The
full run took 3 minutes.
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| CLIP text encoder | 0 | 0 |
| U-Net single step with guidance embedding (t = 999) | 0 | 0 |
| VAE decode / encode | 0 / 0 | 0 |
| LCM scheduler (4 steps, seeded noise between steps) | 0 | 0 |
| End to end: final latent / pixels (768×768, 4 steps, guidance 8.0) | 0 / 0 | 0 / 0 |
| End to end, seed 42 (pixels) | 0 | 0 |
The end-to-end prompt was "a photograph of an astronaut riding a horse". The default run uses
768×768, 4 steps and guidance 8.0. The LCM scheduler config's steps_offset is accepted and
ignored because the reference never uses it.
The header-only tensor check covered text_encoder, unet and vae: tensor counts
196 / 687 / 248, with missing, extra and shape-mismatched tensors all 0. Record:
tensor_header_check, validation.json.
Stable Diffusion 2.1¶
sd-v2-1 was loaded strictly from sd2-community/stable-diffusion-2-1 (revision bb21548, a copy of the withdrawn
stabilityai/stable-diffusion-2-1 release) and compared with diffusers 0.40.0 StableDiffusionPipeline on CPU in
float32. Every comparison is exact. The real-checkpoint run takes about 14 minutes on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| Tokenizer token ids (5 prompts) | identical | exact |
| OpenCLIP-H text encoder | 0 | 0 |
| U-Net single step (t = 981, 481, 1) | 0 | 0 |
| VAE decode / encode (768×768) | 0 | 0 |
| DDIM timesteps with leading spacing (25 and 50 steps) | 0 | 0 |
| End to end: final latent / pixels (768×768, 50 steps, guidance 7.5) | 0 / 0 | 0 / 0 |
| End to end, seed 42 (pixels) | 0 | 0 |
The tokenizer exposed a loading rule that now applies to every CLIP tokenizer: when tokenizer_config.json has no
added_tokens_decoder, transformers merges special_tokens_map.json over it (this repository pads with !, which
only the map names), and it registers the named special tokens in the order bos, eos, unk, pad.
Without weights, a tiny random v-prediction model with the DDIM scheduler and upcast_attention
matches StableDiffusionPipeline exactly, and the DDIM scheduler matches diffusers exactly for
epsilon, sample and v-prediction trajectories at 25 and 50 steps.
SD-Turbo¶
Validated on CPU in the same environment as above. sd-turbo was loaded strictly from
stabilityai/sd-turbo. The observed difference in every comparison is exactly 0: on CPU in
float32, the local implementation produces a bit-identical result to the reference. The full run
took 1 minute.
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| Tokenizer token ids (5 prompts) | identical | exact |
| OpenCLIP-H text encoder | 0 | 0 |
| U-Net single step (t = 999, 499) | 0 | 0 |
| VAE decode / encode | 0 / 0 | 0 |
| Euler timesteps / sigmas with trailing spacing (1, 2 and 4 steps) | 0 / 0 | 0 |
| End to end: final latent / pixels (512×512, 1 step, guidance 0.0) | 0 / 0 | 0 / 0 |
| Guided end to end: final latent / pixels (512×512, 4 steps, guidance 2.0, negative prompt) | 0 / 0 | 0 / 0 |
| End to end, seed 42 (pixels) | 0 | 0 |
The end-to-end prompt was "a photograph of an astronaut riding a horse" with initial noise
seed=1234. The default run uses 512×512, 1 step and guidance 0.0; the guided run uses 4 steps,
guidance 2.0 and a negative prompt.
The header-only tensor check covered text_encoder, unet and vae: tensor counts
372 / 686 / 248, with missing, extra and shape-mismatched tensors all 0. Record:
tensor_header_check, validation.json.
SDXL¶
Validated on CPU on October 5, 2026, in the same environment as above. Eight checkpoints, full:
sdxl-base-1.0, sdxl-lightning-2step, sdxl-lightning-4step, sdxl-lightning-8step,
sdxl-turbo, ssd-1b, playground-v2 and playground-v2.5. The published tensors were loaded strictly. A
generation at each model's native size was compared against diffusers from the same initial noise.
Comparisons (CPU, float32)¶
| Comparison | Checkpoint | Largest observed difference | Tolerance |
|---|---|---|---|
| Tokenizer token ids (2 tokenizers, 18 prompts) | sdxl-base-1.0 | identical | identical |
| CLIP ViT-L/14 text encoder (penultimate layer) | sdxl-base-1.0 | 0 | 0 |
| OpenCLIP bigG text encoder (penultimate layer and pooled output) | sdxl-base-1.0 | 0 / 0 | 0 / 0 |
| U-Net single step (t = 999, 500, 1) | sdxl-base-1.0 | 0 | 0 |
| VAE decode / encode | sdxl-base-1.0 | 0 / 0 | 0 |
| Euler timesteps / sigmas (50 and 30 steps) | sdxl-base-1.0 | 0 / 0 | 0 |
| End to end: final latent / pixels | sdxl-base-1.0 | 0 / 0 | 0 / 0 |
| End to end, seed 42 (pixels) | sdxl-base-1.0 | 0 | 0 |
| U-Net single step (1024×1024, t = 999) | sdxl-lightning-2step | 0 | 0 |
| End to end: final latent / pixels (1024×1024, 2 steps, guidance 0.0) | sdxl-lightning-2step | 0 / 0 | 0 / 0 |
| End to end, seed 42 with the defaults (pixels) | sdxl-lightning-2step | 0 | 0 |
| U-Net single step (1024×1024, t = 999) | sdxl-lightning-4step | 0 | 0 |
| End to end: final latent / pixels (1024×1024, 4 steps, guidance 0.0) | sdxl-lightning-4step | 0 / 0 | 0 / 0 |
| End to end, seed 42 with the defaults (pixels) | sdxl-lightning-4step | 0 | 0 |
| U-Net single step (1024×1024, t = 999) | sdxl-lightning-8step | 0 | 0 |
| End to end: final latent / pixels (1024×1024, 8 steps, guidance 0.0) | sdxl-lightning-8step | 0 / 0 | 0 / 0 |
| End to end, seed 42 with the defaults (pixels) | sdxl-lightning-8step | 0 | 0 |
| Tokenizer token ids (2 tokenizers, 18 prompts) | sdxl-turbo | identical | identical |
| CLIP ViT-L/14 text encoder (penultimate layer) | sdxl-turbo | 0 | 0 |
| OpenCLIP bigG text encoder (penultimate layer and pooled output) | sdxl-turbo | 0 / 0 | 0 / 0 |
| U-Net single step (t = 999, 500, 1) | sdxl-turbo | 0 | 0 |
| VAE decode / encode | sdxl-turbo | 0 / 0 | 0 |
| Euler ancestral timesteps / sigmas (50 and 30 steps) | sdxl-turbo | 0 / 0 | 0 |
| End to end: final latent / pixels (1 step, guidance 0.0) | sdxl-turbo | 0 / 0 | 0 / 0 |
| End to end: final latent / pixels (4 steps, guidance 2.0) | sdxl-turbo | 0 / 0 | 0 / 0 |
| End to end, seed 42 (pixels) | sdxl-turbo | 0 | 0 |
| Tokenizer token ids (2 tokenizers, 18 prompts) | ssd-1b | identical | identical |
| CLIP ViT-L/14 text encoder (penultimate layer) | ssd-1b | 0 | 0 |
| OpenCLIP bigG text encoder (penultimate layer and pooled output) | ssd-1b | 0 / 0 | 0 / 0 |
| U-Net single step (t = 999, 500, 1) | ssd-1b | 0 | 0 |
| VAE decode / encode | ssd-1b | 0 / 0 | 0 |
| Euler timesteps / sigmas (50 and 30 steps) | ssd-1b | 0 / 0 | 0 |
| End to end: final latent / pixels | ssd-1b | 0 / 0 | 0 / 0 |
| End to end, seed 42 (pixels) | ssd-1b | 0 | 0 |
| Tokenizer token ids (2 tokenizers, 18 prompts) | playground-v2 | identical | identical |
| CLIP ViT-L/14 text encoder (penultimate layer) | playground-v2 | 0 | 0 |
| OpenCLIP bigG text encoder (penultimate layer and pooled output) | playground-v2 | 0 / 0 | 0 / 0 |
| U-Net single step (t = 999, 500, 1) | playground-v2 | 0 | 0 |
| VAE decode / encode | playground-v2 | 0 | 0 |
| Euler ancestral timesteps / sigmas (50 and 30 steps) | playground-v2 | 0 / 0 | 0 |
| End to end: final latent / pixels | playground-v2 | 0 / 0 | 0 / 0 |
| End to end, seed 42 (pixels) | playground-v2 | 0 | 0 |
| Tokenizer token ids (2 tokenizers, 18 prompts) | playground-v2.5 | identical | identical |
| CLIP ViT-L/14 text encoder (penultimate layer) | playground-v2.5 | 0 | 0 |
| OpenCLIP bigG text encoder (penultimate layer and pooled output) | playground-v2.5 | 0 / 0 | 0 / 0 |
| U-Net single step (t = 999, 500, 1) | playground-v2.5 | 0 | 0 |
| VAE decode / encode | playground-v2.5 | 0 / 0 | 0 |
| EDM DPM-Solver++ timesteps / sigmas (50 and 30 steps) | playground-v2.5 | 0 / 0 | 0 |
| End to end: final latent / pixels | playground-v2.5 | 0 / 0 | 0 / 0 |
| End to end, seed 42 (pixels) | playground-v2.5 | 0 | 0 |
The observed difference in every comparison is exactly 0: on CPU in float32, the local implementation produces a bit-identical image to the one diffusers produces. The same tolerance rule applies: since every measurement came out 0, no tolerance needed to be loosened.
End-to-end test settings for sdxl-base-1.0 and ssd-1b: prompt "a photograph of an astronaut
riding a horse", 1024×1024, guidance 5.0, initial noise seed=1234 (the seed 42 run uses the same
settings). 30 steps are used instead of the default 50 to keep CPU runtime reasonable. SDXL Turbo's
native size is 512×512. Its default run uses 1 step and guidance 0.0; the guided run uses 4 steps
and guidance 2.0. Its Euler ancestral scheduler draws the per-step ancestral noise from the same
seeded CPU generator as the initial noise. playground-v2 and playground-v2.5 use 1024×1024, 30 steps and guidance
3.0; 30 steps are used instead of the default 50 to keep CPU runtime reasonable. The playground-v2
real-checkpoint run took 21 minutes on an M2 Max (CPU).
Community fine-tunes¶
juggernaut-xl-v9, animagine-xl-4.0 and dreamshaper-xl-1.0 target diffusers 0.40.0
StableDiffusionXLPipeline. Their pinned published headers match the existing SDXL text encoders,
U-Net and VAE layouts. The suite also covers each published scheduler: trailing DDPM for
Juggernaut, Euler for Animagine and leading DEIS for DreamShaper.
The real-checkpoint run covers tokenizers, both text encoders, one U-Net denoising step, VAE encode/decode, a four-step scheduler trajectory, and separate 64×64 public-pipeline generations at seed 0 and seed 42 for each checkpoint. All 24 real tests passed (8 per checkpoint), and the observed difference in every recorded comparison is exactly 0 (0 means exact): on CPU in float32, the local implementation produces a bit-identical image to the one diffusers produces. The real-checkpoint run took 1 minute on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Checkpoint | Largest observed difference | Tolerance |
|---|---|---|---|
Tensor header check (text_encoder, text_encoder_2, unet, vae; 196 / 517 / 1680 / 248 tensors) |
all three | 0 missing / extra / mismatched | exact |
| Tokenizer token ids (2 tokenizers, 4 prompts) | all three | identical | exact |
| CLIP text encoder (penultimate layer) | all three | 0 | 0 |
| OpenCLIP bigG text encoder (penultimate layer and pooled output) | all three | 0 / 0 | 0 / 0 |
| U-Net single step (t = 500) | all three | 0 | 0 |
| VAE decode / encode | all three | 0 / 0 | 0 / 0 |
| Scheduler trajectory (4 steps) | all three | 0 | 0 |
| End to end: final pixels (64×64, 4 steps, guidance 5.0, seed 0) | all three | 0 | 0 |
| End to end, seed 42 (pixels) | all three | 0 | 0 |
Records: header_<id>, sdxl_tokenizers_<id>, sdxl_text_encoder_<id>,
sdxl_text_encoder_2_<id>, sdxl_unet_<id>, sdxl_vae_decode_<id>,
sdxl_vae_encode_<id>, sdxl_scheduler_<id>, e2e_<id> and seed_e2e_<id>,
validation-measurements.json.
The end-to-end runs use 64×64 output, four steps and guidance 5.0. e2e_<id> uses seed 0 and
seed_e2e_<id> uses seed 42; their recorded pixel difference is exactly 0 in every run.
Juggernaut and Animagine are validated after their published float16 tensors are upcast to CPU float32. DreamShaper publishes float32 component tensors. Safety checking, watermarking, fp16 arithmetic parity and duplicate single-file checkpoints are excluded.
Segmind-Vega¶
The dedicated Segmind-Vega suite targets diffusers 0.40.0 StableDiffusionXLPipeline. Published
header fixtures cover both text encoders, the U-Net and VAE. Random-weight tests cover the tiny and
published U-Net layouts, and tiny-pipeline tests cover classifier-free guidance on and off at
non-square sizes. The real-checkpoint run checks both tokenizers, both text encoders, a U-Net step,
VAE encode/decode, Euler scheduling and two separate public-API generations. All 9 real tests
passed, and the observed difference in every recorded comparison is exactly 0 (0 means exact):
on CPU in float32, the local implementation produces a bit-identical image to the one diffusers
produces. The real-checkpoint run took less than a minute on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
Tensor header check (text_encoder, text_encoder_2, unet, vae; 196 / 517 / 564 / 248 tensors) |
0 missing / extra / mismatched | exact |
| Tokenizer token ids (2 tokenizers, 4 prompts) | identical | exact |
| CLIP text encoder | 0 | 0 |
| OpenCLIP bigG text encoder | 0 | 0 |
| U-Net single step (t = 500) | 0 | 0 |
| VAE decode / encode | 0 / 0 | 0 / 0 |
| Euler scheduler (4 steps) | 0 | 0 |
| End to end: final pixels (64×64, 4 steps, guidance 5.0, seed 0) | 0 | 0 |
| End to end, seed 42 (pixels) | 0 | 0 |
Records: header_segmind-vega, sdxl_tokenizers_segmind-vega,
sdxl_text_encoder_segmind-vega, sdxl_text_encoder_2_segmind-vega,
sdxl_unet_segmind-vega, sdxl_vae_decode_segmind-vega, sdxl_vae_encode_segmind-vega,
sdxl_scheduler_segmind-vega, e2e_segmind-vega and seed_e2e_segmind-vega,
validation-measurements.json.
The real end-to-end comparisons are deliberately short: 64×64 for four steps.
e2e_segmind-vega uses seed 0 and seed_e2e_segmind-vega uses seed 42; both use guidance 5.0 and
the same negative prompt. Their final pixels differ by exactly 0 in both runs.
KOALA¶
The dedicated KOALA suite targets diffusers 0.40.0 StableDiffusionXLPipeline for koala-700m
and koala-1b. Published header fixtures cover both text encoders, the U-Net and VAE for each
checkpoint. Random-weight tests cover tiny and published U-Net layouts, including the absent
KOALA-700M mid-block, and tiny-pipeline tests cover classifier-free guidance on and off at
non-square sizes. The real-checkpoint comparison covers both tokenizers, both text encoders, one
U-Net step, VAE encode/decode, the four-step Euler schedule and two separate public-API
generations for each checkpoint. All 18 real tests passed, and the observed difference in every
recorded comparison is exactly 0 (0 means exact): on CPU in float32, the local
implementation produces a bit-identical image to the one diffusers produces. The published
weights are float32. The real-checkpoint run took less than a minute on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Checkpoint | Largest observed difference | Tolerance |
|---|---|---|---|
Tensor header check (text_encoder, text_encoder_2, unet, vae; 196 / 517 / 584 / 248 tensors) |
koala-700m | 0 missing / extra / mismatched | exact |
Tensor header check (text_encoder, text_encoder_2, unet, vae; 196 / 517 / 790 / 248 tensors) |
koala-1b | 0 missing / extra / mismatched | exact |
| Tokenizer token ids (2 tokenizers, 4 prompts) | both | identical | exact |
| CLIP text encoder | both | 0 | 0 |
| OpenCLIP bigG text encoder | both | 0 | 0 |
| U-Net single step (t = 500) | both | 0 | 0 |
| VAE decode / encode | both | 0 / 0 | 0 / 0 |
| Euler scheduler (4 steps) | both | 0 | 0 |
| End to end: final pixels (64×64, 4 steps, guidance 5.0, seed 0) | both | 0 | 0 |
| End to end, seed 42 (pixels) | both | 0 | 0 |
The end-to-end runs use 64×64 output, four steps, guidance 5.0 and the same negative prompt.
e2e_koala-700m and e2e_koala-1b use seed 0, seed_e2e_koala-700m and
seed_e2e_koala-1b use seed 42, and their final pixels differ by exactly 0 in every run.
Records: header_koala-700m, header_koala-1b, sdxl_tokenizers_<id>,
sdxl_text_encoder_<id>, sdxl_text_encoder_2_<id>, sdxl_unet_<id>, sdxl_vae_decode_<id>,
sdxl_vae_encode_<id>, sdxl_scheduler_<id>, e2e_<id> and seed_e2e_<id>, where <id> is
koala-700m or koala-1b, validation-measurements.json.
KOALA-Lightning¶
The dedicated KOALA-Lightning suite targets diffusers 0.40.0
StableDiffusionXLPipeline for koala-lightning-700m and koala-lightning-1b. Published header
fixtures cover both text encoders, the U-Net and VAE for each checkpoint. The layouts contain
196 / 517 / 584 / 248 tensors for the 700M checkpoint and 196 / 517 / 790 / 248 tensors for the
1B checkpoint. Random-weight tests cover both U-Net layouts, and tiny-pipeline tests cover
classifier-free guidance on and off at non-square sizes with exact uint8 output comparison.
The real-checkpoint suite separately compares both tokenizers, both text encoders, one U-Net step,
VAE encode/decode and the 10-step trailing Euler schedule. Its two public-API generations per
checkpoint use the published 10-step and guidance 3.5 defaults at 64×64: e2e_<id> uses seed 0
and seed_e2e_<id> uses seed 42. All 16 real tests passed, and the observed difference in every
recorded comparison is exactly 0 (0 means exact): on CPU in float32, the local implementation
produces a bit-identical image to the one diffusers produces. The published weights are float32.
The real-checkpoint run took 0 minutes on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Checkpoint | Largest observed difference | Tolerance |
|---|---|---|---|
Tensor header check (text_encoder, text_encoder_2, unet, vae; 196 / 517 / 584 / 248 tensors) |
koala-lightning-700m | 0 missing / extra / mismatched | exact |
Tensor header check (text_encoder, text_encoder_2, unet, vae; 196 / 517 / 790 / 248 tensors) |
koala-lightning-1b | 0 missing / extra / mismatched | exact |
| Tokenizer token ids (2 tokenizers, 4 prompts) | both | identical | exact |
| CLIP text encoder | both | 0 | 0 |
| OpenCLIP bigG text encoder | both | 0 | 0 |
| U-Net single step (t = 500) | both | 0 | 0 |
| VAE decode / encode | both | 0 / 0 | 0 / 0 |
| Euler scheduler (10 steps) | both | 0 | 0 |
| End to end: final pixels (64×64, 10 steps, guidance 3.5, seed 0) | both | 0 | 0 |
| End to end, seed 42 (pixels) | both | 0 | 0 |
The end-to-end runs use 64×64 output, ten steps and guidance 3.5, the published KOALA-Lightning
defaults. e2e_koala-lightning-700m and e2e_koala-lightning-1b use seed 0,
seed_e2e_koala-lightning-700m and seed_e2e_koala-lightning-1b use seed 42, and their final
pixels differ by exactly 0 in every run.
Records: header_koala-lightning-700m, header_koala-lightning-1b, sdxl_tokenizers_<id>,
sdxl_text_encoder_<id>, sdxl_text_encoder_2_<id>, sdxl_unet_<id>, sdxl_vae_decode_<id>,
sdxl_vae_encode_<id>, sdxl_scheduler_<id>, e2e_<id> and seed_e2e_<id>, where <id> is
koala-lightning-700m or koala-lightning-1b,
validation-measurements.json.
SDXL-Lightning¶
The SDXL-Lightning reference is built exactly as described by the model card: a float32 U-Net from the SDXL base configuration loaded with the published U-Net file, placed in the base pipeline, and an Euler scheduler with trailing timestep spacing. The text encoders and VAE are the validated SDXL base components. The three defaults are respectively 2, 4 and 8 steps at 1024×1024 with guidance 0.0. Each validation run took 2–4 minutes.
Hyper-SD¶
hyper-sdxl-1step was validated on CPU in float32 from the composite snapshot: SDXL base at
revision 4621659 plus Hyper-SDXL-1step-Unet.safetensors from ByteDance/Hyper-SD at revision
bc08d97. Every comparison was exact.
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| LCM explicit-timestep schedules and trajectories (1 and 4 steps; unit and reference) | 0 | 0 |
| Tiny-snapshot end to end with LCM and explicit timesteps (1 and 4 steps) | 0 | 0 |
| Overlay U-Net header (1680 tensors) | 0 missing / extra / mismatched | exact |
| U-Net single step (1024×1024, t = 800) | 0 | 0 |
End to end: final latent / pixels (1024×1024, 1 step, guidance 0.0, timesteps [800]) |
0 / 0 | 0 / 0 |
| End to end, seed 42 with the defaults (pixels) | 0 | 0 |
The reference pipeline loads the composite snapshot with
StableDiffusionXLPipeline.from_pretrained. Loading the U-Net in the model card's other documented
way—an empty model followed by load_state_dict—places the weights in freshly allocated memory
instead of preserving the file's memory-mapped layout. CPU summation order then differs, and one
U-Net step differs by about 1e-4; the same effect occurs with the SDXL base U-Net. The parity
claim is therefore against from_pretrained. The real-checkpoint run took 2 minutes.
DMD2¶
dmd2-sdxl-4step and dmd2-sdxl-1step were validated on CPU in float32 from composite snapshots:
SDXL base at revision 4621659 plus dmd2_sdxl_4step_unet.bin or
dmd2_sdxl_1step_unet.bin from tianweiy/DMD2 at revision be22767. Every comparison was exact.
| Comparison | Checkpoint | Largest observed difference | Tolerance |
|---|---|---|---|
| Overlay U-Net header (1680 tensors) | dmd2-sdxl-4step | 0 missing / extra / mismatched | exact |
| U-Net single step (1024×1024, t = 999) | dmd2-sdxl-4step | 0 | 0 |
End to end: final latent / pixels (1024×1024, 4 steps, guidance 0.0, timesteps [999, 749, 499, 249]) |
dmd2-sdxl-4step | 0 / 0 | 0 / 0 |
| End to end, seed 42 with the defaults (pixels) | dmd2-sdxl-4step | 0 | 0 |
| Overlay U-Net header (1680 tensors) | dmd2-sdxl-1step | 0 missing / extra / mismatched | exact |
| U-Net single step (1024×1024, t = 399) | dmd2-sdxl-1step | 0 | 0 |
End to end: final latent / pixels (1024×1024, 1 step, guidance 0.0, timesteps [399]) |
dmd2-sdxl-1step | 0 / 0 | 0 / 0 |
| End to end, seed 42 with the defaults (pixels) | dmd2-sdxl-1step | 0 | 0 |
The end-to-end comparisons used fixed initial noise with seed 1234 and a scheduler generator seeded
with 42 for the LCM noise drawn between steps. Zip-format PyTorch pickles are memory-mapped with
torch.load(..., weights_only=True, mmap=True), as in diffusers. Loading the weights into fresh
memory changed CPU summation order and made one U-Net step differ by about 6e-5. The
real-checkpoint run for both models took 4 minutes.
For sdxl-base-1.0, the same header-only tensor check covered the four components
(text_encoder, text_encoder_2, unet, vae): tensor counts 196 / 517 / 1680 / 248, and
missing, extra and shape-mismatched tensors are 0. Record: tensor_header_check,
validation.json.
For each SDXL-Lightning checkpoint, the header-only tensor check covered its distilled U-Net: 1680
tensors, with missing, extra and shape-mismatched tensors all 0. Record:
tensor_header_check, validation.json.
For sdxl-turbo, the same header-only tensor check covered the four components (text_encoder,
text_encoder_2, unet, vae): tensor counts 196 / 517 / 1680 / 248, and missing, extra and
shape-mismatched tensors are 0. Record: tensor_header_check,
validation.json.
For ssd-1b, the same header-only tensor check covered the four components (text_encoder,
text_encoder_2, unet, vae): tensor counts 196 / 517 / 944 / 248, and missing, extra and
shape-mismatched tensors are 0. Record: tensor_header_check,
validation.json.
For segmind-vega, the header-only tensor check covers the four components (text_encoder,
text_encoder_2, unet, vae): tensor counts 196 / 517 / 564 / 248, with no missing, extra or
shape-mismatched tensors. The published full-precision safetensors are used; fp16 and single-file
duplicates are excluded. Record: header_segmind-vega,
validation-measurements.json.
For koala-700m and koala-1b, dedicated header checks cover the four components
(text_encoder, text_encoder_2, unet, vae) against pinned published safetensors: tensor
counts 196 / 517 / 584 / 248 and 196 / 517 / 790 / 248, with missing, extra and shape-mismatched
tensors all 0. Records: header_koala-700m and header_koala-1b,
validation-measurements.json.
For playground-v2, the same header-only tensor check covered the four components
(text_encoder, text_encoder_2, unet, vae): tensor counts 196 / 517 / 1680 / 248, and
missing, extra and shape-mismatched tensors are 0. Record: tensor_header_check,
validation.json.
For playground-v2.5, the same header-only tensor check covered the four components
(text_encoder, text_encoder_2, unet, vae): tensor counts 196 / 517 / 1680 / 248, and
missing, extra and shape-mismatched tensors are 0. Record: tensor_header_check,
validation.json.
SDXL refiner¶
Validated on CPU in the same environment as above. sdxl-refiner-1.0 was loaded strictly from
stabilityai/stable-diffusion-xl-refiner-1.0 (revision 5d4cfe8) and compared against diffusers
0.40.0 StableDiffusionXLImg2ImgPipeline on CPU in float32. The observed difference in every
comparison is exactly 0: on CPU in float32, the local implementation produces a bit-identical
result to the reference. The real-checkpoint run took 10 minutes on an Apple M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| Tensor header check: text_encoder_2, unet, vae (517 / 1220 / 248 tensors) | 0 missing / extra / mismatched | exact |
| Tokenizer token ids (8 prompts) | identical | exact |
| OpenCLIP bigG text encoder (penultimate layer and pooled output) | 0 / 0 | 0 / 0 |
| Refiner U-Net single step (t = 999, 500, 1) | 0 | 0 |
| VAE decode / encode | 0 / 0 | 0 |
| VAE posterior sample | 0 | 0 |
| Input preprocessing (640×963, 1000×700, 1024×1024) | 0 | 0 |
| Euler image-to-image scheduler (30 steps strength 0.5, 50 steps strength 0.3) | exact | exact |
| End to end: final latent / pixels (1024×1024, 50 steps, strength 0.3, guidance 5.0, seed 0) | 0 / 0 | 0 / 0 |
| End to end, seed test (pixels) | 0 | 0 |
What the tests cover:
- Random weights, no checkpoint: the refiner U-Net (five time ids) with a tiny configuration and with
the published configuration at small spatial sizes; the Euler image-to-image trajectory (begin index,
add_noise, scaling and steps) for several step counts and strengths; a tiny end to end against the reference pipeline (defaults, odd and non-square sizes, negative prompt, guidance off, full strength, two images per prompt, RGBA input, micro-conditioning and aesthetic scores); input preprocessing againstVaeImageProcessor; the VAE posterior sample and the generator state it leaves. - Real checkpoint (
tests/reference/test_ref_sdxl_refiner.py): tensor headers, tokenizer, text encoder 2, U-Net single step, VAE decode, encode and sample, preprocessing, scheduler with begin index, end to end at the defaults (50 steps, strength 0.3, guidance 5.0, seed 0) with a synthetic 1024x1024 input image, and the seed test through the public API.
Existing SDXL tests are unchanged and still exact.
Z-Image¶
Validated on CPU on October 5, 2026, in the same environment as above. z-image-turbo and z-image
were loaded strictly from Tongyi-MAI/Z-Image-Turbo and Tongyi-MAI/Z-Image, respectively. The
observed difference in every comparison is exactly 0: on CPU in float32, the local implementation
produces a bit-identical result to the reference.
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| Tokenizer token ids and attention masks (3 fixed prompts, seeded fuzz, exhaustive per-code-point check) | identical | exact |
z-image tokenizer token ids and attention masks (3 fixed prompts) |
identical | exact |
Qwen3 text encoder (hidden_states[-2]) |
0 | 0 |
z-image Qwen3 text encoder (hidden_states[-2]) |
0 | 0 |
| Transformer single step (1024×1024, t = 0.5 and 0.25) | 0 | 0 |
z-image transformer single step (1024×1024, t = 0.5 and 0.25) |
0 | 0 |
| VAE decode / encode | 0 / 0 | 0 |
z-image VAE decode / encode |
0 / 0 | 0 |
| Flow-match scheduler timesteps / sigmas | 0 / 0 | 0 |
z-image flow-match scheduler timesteps / sigmas (shift 6.0) |
0 / 0 | 0 |
| End to end: final latent / pixels (1024×1024, 9 steps, guidance 0.0) | 0 / 0 | 0 / 0 |
z-image end to end: final latent / pixels (1024×1024, 8 steps, guidance 4.0) |
0 / 0 | 0 / 0 |
| Guided end to end: final latent / pixels (1024×1024, 4 steps, guidance 4.0, negative prompt) | 0 / 0 | 0 / 0 |
| End to end, seed 42 (1024×1024, 9 steps, guidance 0.0; pixels) | 0 | 0 |
z-image end to end, seed 42 (1024×1024, 8 steps, guidance 4.0; pixels) |
0 | 0 |
The end-to-end prompt was "a photograph of an astronaut riding a horse" with initial noise
seed=1234; the guided run uses a negative prompt. The tokenizer check covers fixed prompts, seeded
fuzz and every Unicode code point. The reference tokenizer's Unicode tables are older than Python's
for 108 combining marks, which it treats as unassigned.
The z-image end-to-end checks use 8 steps instead of the default 50 only to shorten the CPU run.
To fit the 96 GB machine, each component pair (ours and the reference) is loaded and freed one at a time, and in the end-to-end checks the reference pipeline runs first and is freed before ours runs.
For each checkpoint, the header-only tensor check covered text_encoder, transformer and vae:
tensor counts 398 / 521 / 244, with missing, extra and shape-mismatched tensors all 0. Record:
tensor_header_check, validation.json.
The full validation run took 47 minutes.
FLUX.1¶
Validated on CPU in the same environment as above. flux1-schnell, flux1-dev and flux1-krea-dev
were loaded strictly from black-forest-labs/FLUX.1-schnell at revision 741f7c3,
black-forest-labs/FLUX.1-dev at revision 3de623f and black-forest-labs/FLUX.1-Krea-dev at
revision 8162a9c, respectively. flux1-krea-dev shares the flux1-dev architecture and configs and is
checked with the same comparisons below (guidance 4.5, 512 tokens). The observed difference in
every comparison is exactly 0: on CPU in float32, the local implementation produces a
bit-identical result to the reference.
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
flux1-schnell CLIP tokenizer / pooled CLIP-L |
identical / 0 | exact / 0 |
flux1-schnell T5 tokenizer / T5-XXL encoder (256 tokens) |
identical / 0 | exact / 0 |
flux1-schnell transformer single step (1024×1024) |
0 | 0 |
flux1-schnell VAE decode / encode |
0 / 0 | 0 / 0 |
flux1-schnell flow-match Euler scheduler (static shift 1.0) |
exact | exact |
flux1-schnell end to end: final latent / pixels (1024×1024, 4 steps, guidance 0.0) |
0 / 0 | 0 / 0 |
flux1-schnell end to end, seed 42 (pixels) |
0 | 0 |
flux1-dev CLIP tokenizer / pooled CLIP-L |
identical / 0 | exact / 0 |
flux1-dev T5 tokenizer / T5-XXL encoder (512 tokens) |
identical / 0 | exact / 0 |
flux1-dev transformer single step (1024×1024, guidance-distilled) |
0 | 0 |
flux1-dev VAE decode / encode |
0 / 0 | 0 / 0 |
flux1-dev flow-match Euler scheduler (dynamic shifting) |
exact | exact |
flux1-dev end to end: final latent / pixels (1024×1024, 4 steps, guidance 3.5) |
0 / 0 | 0 / 0 |
flux1-dev end to end, seed 42 (pixels) |
0 | 0 |
flux1-krea-dev CLIP tokenizer / pooled CLIP-L |
identical / 0 | exact / 0 |
flux1-krea-dev T5 tokenizer / T5-XXL encoder (512 tokens) |
identical / 0 | exact / 0 |
flux1-krea-dev transformer single step (1024×1024, guidance-distilled) |
0 | 0 |
flux1-krea-dev VAE decode / encode |
0 / 0 | 0 / 0 |
flux1-krea-dev flow-match Euler scheduler (dynamic shifting) |
exact | exact |
flux1-krea-dev end to end: final latent / pixels (1024×1024, 4 steps, guidance 4.5) |
0 / 0 | 0 / 0 |
flux1-krea-dev end to end, seed 42 (pixels) |
0 | 0 |
The end-to-end prompt was "a photograph of an astronaut riding a horse" with initial noise
seed=42. Schnell was tested at its 4-step, guidance 0.0 defaults. Dev used 4 steps and guidance
3.5 instead of its 50-step default only to shorten the CPU run, and Krea [dev] likewise used 4 steps
and guidance 4.5.
The header-only tensor check covered text_encoder, text_encoder_2, transformer and vae:
flux1-schnell had 196 / 219 / 1156 / 244 tensors, and flux1-dev and flux1-krea-dev each had
196 / 219 / 1160 / 244 tensors, with missing, extra and shape-mismatched tensors all 0. Record:
tensor_header_check, validation.json.
The reference and native transformers, about 48 GB each in float32, were never loaded together. Each test ran the reference first, saved its outputs, freed it, then ran the native model. Schnell's component and seed tests took 11 minutes and its fixed-latent end-to-end test took 13 minutes. The full dev run took 25 minutes. The Krea [dev] real-checkpoint run took about 30 minutes on an M2 Max (CPU).
Flux.1-Lite 8B¶
The dedicated Flux.1-Lite suite targets diffusers 0.40.0 FluxPipeline and the ungated
Freepik/flux.1-lite-8B checkpoint at revision 16b9373. The published bfloat16 weights are
upcast to float32 on both sides. Config and safetensors-header fixtures verify the reduced
8-double-stream / 38-single-stream transformer layout; the existing FLUX.1 random-weight and tiny
pipeline tests cover exact configurable-module and generation-loop parity.
The real-checkpoint run compared strict headers for all four weighted components, tokenizer ids,
CLIP and T5 outputs, a transformer denoising step, VAE encode/decode, the dynamically shifted
scheduler, and two separate public-pipeline image runs. e2e_flux-1-lite-8b
uses seed 0 and seed_e2e_flux-1-lite-8b uses seed 42; both use two steps, guidance 3.5 and 64x64
output. The reference images are produced in a subprocess that exits before the native pipeline
loads.
All 6 real tests passed, and the observed difference in every recorded comparison is exactly 0 (0 means exact): on CPU in float32, on the upcast bfloat16 weights, the local implementation produces a bit-identical image to the one diffusers produces. The real-checkpoint run took 3 minutes on an M2 Max (CPU).
Comparisons (CPU, float32, upcast bfloat16 weights)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
Tensor header names and shapes (text_encoder, text_encoder_2, transformer, vae; 196 / 219 / 808 / 244 tensors) |
0 missing / extra / mismatched | exact |
| CLIP and T5 tokenizer token ids (77 and 512 tokens) | identical | exact |
| CLIP pooled output / T5 last hidden state | 0 / 0 | 0 / 0 |
| Transformer single step (random conditioning, timestep 0.5, guidance 3.5) | 0 | 0 |
| VAE decode / encode (8×8 latents, 64×64 image) | 0 / 0 | 0 / 0 |
| Flow-match Euler scheduler (dynamic shifting, mu 1.15) | exact | exact |
| End to end: final pixels (64×64, 2 steps, guidance 3.5, seed 0) | 0 | 0 |
| End to end, seed 42 (pixels, same settings) | 0 | 0 |
The end-to-end prompt was "a photograph of an astronaut riding a horse" with 512 text tokens. Two steps and 64x64 output are used instead of the 28-step, 1024x1024 model-card defaults only to shorten the CPU run.
Records: header_flux-1-lite-8b, flux1_tokenizers_flux-1-lite-8b,
flux1_clip_flux-1-lite-8b, flux1_t5_flux-1-lite-8b,
flux1_transformer_flux-1-lite-8b, flux1_vae_decode_flux-1-lite-8b,
flux1_vae_encode_flux-1-lite-8b, flux1_scheduler_flux-1-lite-8b,
e2e_flux-1-lite-8b and seed_e2e_flux-1-lite-8b,
validation-measurements.json.
JoyAI-Image¶
joyai-image-edit and joyai-image-edit-plus use the pinned Apache-2.0 checkpoints from
jdopensource at revisions 4b41fb25d961f37668750178ccbb380da326201c and
c2686460c7b64d8aa11bc4d0da423fb316b33f9e and were compared with diffusers 0.40.0
JoyImageEditPipeline and JoyImageEditPlusPipeline and transformers 5.18.0 on CPU in float32
in the same environment as above. The repositories publish bfloat16 safetensors only; the weights
are upcast to float32 on both sides. All 23 real tests passed, and the observed difference in
every recorded comparison is exactly 0 (0 means exact): on CPU in float32, the local
implementation produces a bit-identical image to the one diffusers produces. The real-checkpoint
run took 123 minutes on an M2 Max (CPU).
Comparisons (CPU, float32, upcast bfloat16 weights)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
Tensor header names and shapes (text_encoder, transformer, vae; 750 / 894 / 194 tensors, both checkpoints) |
0 missing / extra / mismatched | exact |
| Qwen3-VL tokenizer and vision preprocessing: token ids, image grid and pixels (both checkpoints) | identical | exact |
| Qwen3-VL text and vision conditioning, last hidden state (both checkpoints) | 0 | 0 |
joyai-image-edit transformer single step (timestep 500) |
0 | 0 |
joyai-image-edit-plus transformer single step (timestep 500, image-token mask and shape list) |
0 | 0 |
| VAE encode / decode (64×80 input, both checkpoints) | 0 / 0 | 0 / 0 |
| End to end with the input image: final pixels (512×512, 2 steps, guidance 4.0, seed 0, both checkpoints) | 0 | 0 |
| End to end, seed 42 (pixels, both checkpoints) | 0 | 0 |
The end-to-end prompt was "Turn the square blue." with a fixed synthetic input image resized to 512×512. Two steps are used instead of the 40-step (Edit) and 30-step (Edit Plus) defaults only to shorten the CPU run. The reference pipelines and the native models are loaded one at a time, and each reference end-to-end run executes in a subprocess before the native model runs. Random-weight parity for the Qwen3-VL vision encoder, both transformer paths, the WAN VAE and the tiny pipeline is unchanged and exact.
Records: header_joyai-image-edit, tokenizer_vision_joyai-image-edit,
conditioning_joyai-image-edit, transformer_joyai-image-edit, vae_encode_joyai-image-edit,
vae_decode_joyai-image-edit, e2e_joyai-image-edit and seed_e2e_joyai-image-edit, and the
same eight keys under the joyai-image-edit-plus id,
validation-measurements.json.
FLUX.1 Kontext¶
Validated on CPU in the same environment as above, against diffusers 0.40.0 FluxKontextPipeline.
flux1-kontext-dev was loaded strictly from black-forest-labs/FLUX.1-Kontext-dev at revision
24e9ded. The observed difference in every comparison is exactly 0: on CPU in float32, the
local implementation produces a bit-identical result to the reference, with identical bytes for
images.
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| Input preprocessing over several sizes (300×200, 512×768, 1024×1024, 1365×768, 2000×1100) and modes (RGB, L, RGBA), with and without the automatic resolution choice: resized image bytes and normalized tensor | 0 | 0 |
| VAE encode of the preprocessed input (same sizes and modes) | 0 | 0 |
| Tiny end to end, with an input image (square and non-square, several seeds, a rescaled request): final latent / pixels | 0 / 0 | 0 / 0 |
| Tiny end to end, without an input image: final latent / pixels | 0 / 0 | 0 / 0 |
| Tiny end to end, with true CFG and a negative prompt: final latent / pixels | 0 / 0 | 0 / 0 |
| Input preprocessing and VAE encode, real checkpoint (1024×768 and 640×960 inputs) | 0 | 0 |
| Transformer single step with the image tokens, real checkpoint (two timesteps) | 0 | 0 |
| End to end with an input image at the defaults: final latent / pixels (1024×768 input, 28 steps, guidance 3.5, seed 0) | 0 / 0 | 0 / 0 |
| End to end without an input image: final latent / pixels (4 steps) | 0 / 0 | 0 / 0 |
| End to end, seed 1234, non-square input image (640×960) and non-square output (1360×768, 4 steps) (pixels) | 0 | 0 |
The tiny end-to-end tests use random weights (2 steps) and compare the sequence layout (image tokens, their
position ids and the slicing of the output) against FluxKontextPipeline. The real-checkpoint
runs use fixed synthetic input images; the run with an image uses the 28-step, guidance 3.5
defaults, while the run without an image and the seed test use 4 steps to shorten the CPU run. Existing FLUX.1 tests are unchanged and still exact.
The header-only tensor check covered text_encoder, text_encoder_2, transformer and vae:
tensor counts 196 / 219 / 1160 / 244, with missing, extra and shape-mismatched tensors all 0.
Record: tensor_header_check, validation.json.
The reference and native transformers were never held in one process (the transformer comparison saves the reference output first, then runs the native model). The real-checkpoint run took about 2 hours 7 minutes on an Apple M2 Max CPU.
VisualCloze¶
Validated on CPU on November 9, 2026 (Python 3.13, PyTorch 2.15.0) against diffusers 0.40.0
VisualClozePipeline, transformers 5.18.0. visualcloze-384 was loaded strictly from
VisualCloze/VisualClozePipeline-384 at revision 59c469d. The observed difference in every
comparison is exactly 0.
imagehub follows the authors' official code in two places where the diffusers code has a defect:
- Layout prompt. diffusers'
get_layout_promptreturns a one-element tuple, so the encoders receive the literal text('A grid layout ...',). imagehub uses the official plain sentence. The reference tests replace that function on the reference processor with the plain sentence, and also assert that the unpatched function still returns a tuple. - Several targets. diffusers' "consistent width" pass swaps width and height on non-square images. The official code has no such pass, and imagehub omits it. The pass does nothing on square images, so several-target references use square images. Non-square several-target input has a shape test only.
Default settings (30 steps, guidance 30, upsampling strength 0.3) come from the authors' published example, and the default upsampling size follows the official code. The tests pass every setting explicitly to the reference.
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| Transformer with 384 input channels, tiny random weights | 0 | 0 |
| Grid preprocessing, target positions and image sizes over awkward sizes | 0 | 0 |
Prompt embeddings, with and without content_prompt, tiny |
0 | 0 |
Generation stage, tiny: one and several targets, two example rows, non-square single target, last row of only None, with and without content_prompt: final latents / pixels |
0 / 0 | 0 / 0 |
Both stages, tiny: strengths 0.3, 1.0 and 0, upsampling to the same and to other sizes, non-square, several targets, content_prompt (pixels) |
0 | 0 |
| CLIP and T5 token ids and embeddings, real checkpoint | 0 | 0 |
| Transformer on a 2 × 2 grid-sized input, three timesteps, real checkpoint | 0 | 0 |
| VAE encode and decode, real checkpoint | 0 | 0 |
| End to end, 2 rows × 2 columns, 8 steps, guidance 30, seed 0: generation latents / generation pixels | 0 / 0 | 0 / 0 |
| Same run, upsampling strength 0.3 to 512 × 512 (pixels) | 0 | 0 |
| Seed test, seed 7, 4 steps, no upsampling (pixels) | 0 | 0 |
The real-checkpoint end-to-end run is shortened to 8 steps, and the seed test to 4, to bound the CPU run; the default is 30 steps. Real-checkpoint input images are fixed synthetic images.
The header-only tensor check covered text_encoder, text_encoder_2, transformer and vae:
tensor counts 196 / 219 / 1160 / 244, with missing, extra and shape-mismatched tensors all 0.
Record: tensor_header_check, validation.json.
The reference and native transformers were never held in memory at the same time. The real-checkpoint run took about 20 minutes on an Apple M2 Max CPU.
The published weights carry an Apache-2.0 tag and are built on FLUX.1-Fill-dev. Not covered: MPS and CUDA, float16 and bfloat16, LoRA options, and the official code's own sampler, whose upsampling schedule and random draws differ from diffusers'.
Chroma¶
Validated on CPU in the same environment as above, against diffusers 0.40.0 ChromaPipeline.
chroma1-hd was loaded strictly from lodestones/Chroma1-HD at revision
0e0c60ece1e82b17cb7f77342d765ba5024c40c0. The observed difference in every comparison is
exactly 0: on CPU in float32, the local implementation produces a bit-identical result to the
reference. The random-weight and tiny-snapshot comparisons below are exact.
chroma1-flash was loaded strictly from lodestones/Chroma1-Flash at revision
093c4f63b60507234aa53a073f1f9f565b3f4336; its published bfloat16 weights are upcast to float32
on both sides.
chroma1-base is loaded strictly from lodestones/Chroma1-Base at revision
50ebe125b67794f4ed8e4985d26a4414ca62aed3; its published bfloat16 weights are upcast to float32
on both sides. Its flow-match Euler scheduler uses the published Beta(0.6, 0.6) sigma schedule;
the float64 quantiles and resulting float32 sigmas match SciPy and diffusers exactly.
Comparisons without published weights (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
Tiny Chroma transformer vs ChromaTransformer2DModel |
0 | 0 |
| Tiny Chroma transformer with the published config at small spatial size | 0 | 0 |
| Flow-match Euler scheduler with exponential shifting | exact | exact |
| Tiny end to end (128×128, 2 steps, guidance 5.0): final latent / pixels | 0 / 0 | 0 / 0 |
| Tiny end to end without guidance (guidance 1.0): final latent / pixels | 0 / 0 | 0 / 0 |
| Tiny end to end, seed 42, non-square 128×160 (pixels) | 0 | 0 |
The tiny end-to-end prompt was "a photograph of an astronaut riding a horse" with initial noise
seed=1234. The tests use random weights, several seeds, square and non-square latent sizes, and
guidance on and off. They also verify the attention mask: all real text tokens plus exactly one
padding token are unmasked, while the remaining padding tokens are masked.
Real-checkpoint validation¶
chroma1-hd was compared with diffusers 0.40.0 ChromaPipeline on CPU in float32 in the same
environment as above. All 14 real tests passed, and the observed difference in every recorded
comparison is exactly 0 (0 means exact). The real-checkpoint run took 35 minutes on an M2 Max
(CPU).
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
Tensor header names and shapes (text_encoder, transformer, vae; 219 / 1023 / 244 tensors) |
0 missing / extra / mismatched | exact |
| SentencePiece tokenizer token ids | identical | exact |
| T5-XXL text encoder (512 tokens) | 0 | 0 |
| Chroma transformer single step (timestep 0.5) | 0 | 0 |
| VAE decode | 0 | 0 |
| Flow-match Euler scheduler with exponential shifting (mu 1.15) | exact | exact |
| End to end: final latent / pixels (1024×1024, 2 steps, guidance 5.0) | 0 / 0 | 0 / 0 |
| End to end, seed 42 (pixels) | 0 | 0 |
The end-to-end prompt was "a photograph of an astronaut riding a horse" with an empty negative
prompt, max_sequence_length 512 and initial noise seed=42. Two steps are used instead of the
35-step default only to shorten the CPU run. The reference and native transformers are loaded one
at a time. Record: tensor_header_check, end_to_end and seed_end_to_end,
validation.json; component comparisons under the chroma1-hd_* keys,
validation-measurements.json.
chroma1-flash was compared with diffusers 0.40.0 ChromaPipeline on CPU in float32 in the same
environment. All 16 real tests passed, and the observed difference in every recorded comparison is
exactly 0 (0 means exact): on CPU in float32, the local implementation produces a bit-identical
image to the one diffusers produces. The recorded comparisons are the tensor header check
(text_encoder, transformer, vae; 219 / 1023 / 244 tensors), the SentencePiece tokenizer token
ids, the T5-XXL text encoder, one transformer step, the VAE decode, the flow-match Euler scheduler
with exponential shifting (mu 1.15), and both end-to-end runs (2 steps, guidance 5.0, and seed 42).
The real-checkpoint run took 24 minutes on an M2 Max (CPU). Records: header_chroma1-flash,
chroma1-flash_text_encoder, chroma1-flash_tokenizer, chroma1-flash_transformer,
chroma1-flash_vae_decode, chroma1-flash_scheduler, e2e_chroma1-flash,
e2e_chroma1-flash_latents and seed_e2e_chroma1-flash,
validation-measurements.json; tensor_header_check, end_to_end
and seed_end_to_end, validation.json.
chroma1-base was compared with diffusers 0.40.0 ChromaPipeline on CPU in float32 in the same
environment. All 18 real tests passed, and the observed difference in every recorded comparison is
exactly 0 (0 means exact): on CPU in float32, the local implementation produces a bit-identical
image to the one diffusers produces. The recorded comparisons are the tensor header check
(text_encoder, transformer, vae; 219 / 1023 / 244 tensors, with no missing, extra or
shape-mismatched tensors), the SentencePiece tokenizer token ids, the T5-XXL text encoder, one
transformer step, the VAE decode, the flow-match Euler scheduler with the published Beta(0.6, 0.6)
sigma schedule (recorded mu 1.15), and both end-to-end runs (2 steps, guidance 5.0, and seed 42).
The real-checkpoint run took 22 minutes on an M2 Max (CPU). Records: header_chroma1-base,
header_chroma1-base_fixtures, chroma1-base_text_encoder, chroma1-base_tokenizer,
chroma1-base_transformer, chroma1-base_vae_decode, chroma1-base_scheduler,
e2e_chroma1-base, e2e_chroma1-base_latents and seed_e2e_chroma1-base,
validation-measurements.json; tensor_header_check, end_to_end
and seed_end_to_end, validation.json.
FLUX.2 klein¶
Validated on CPU in the same environment as above. flux2-klein-4b and flux2-klein-9b were loaded
strictly from black-forest-labs/FLUX.2-klein-4B and black-forest-labs/FLUX.2-klein-9B,
respectively. The observed difference in every comparison is exactly 0: on CPU in float32, the
local implementation produces a bit-identical result to the reference.
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| Prompt string / token ids / attention masks | identical | exact |
| Qwen3-4B text encoder (layers 9, 18 and 27 stacked) | 0 | 0 |
| FLUX.2 transformer single step | 0 | 0 |
| VAE decode / encode | 0 / 0 | 0 |
| Flow-match Euler scheduler with empirical shift | exact | exact |
| End to end: final latent / pixels (1024×1024, 4 steps, guidance 1.0) | 0 / 0 | 0 / 0 |
| End to end, seed 42 (pixels) | 0 | 0 |
flux2-klein-9b prompt string / token ids / attention masks |
identical | exact |
flux2-klein-9b Qwen3-8B text encoder (layers 9, 18 and 27 stacked) |
0 | 0 |
flux2-klein-9b FLUX.2 transformer single step |
0 | 0 |
flux2-klein-9b VAE decode / encode |
0 / 0 | 0 / 0 |
flux2-klein-9b flow-match Euler scheduler with empirical shift |
exact | exact |
flux2-klein-9b end to end: final latent / pixels (1024×1024, 4 steps, guidance 1.0) |
0 / 0 | 0 / 0 |
flux2-klein-9b end to end, seed 42 (pixels) |
0 | 0 |
The end-to-end prompt was "a photograph of an astronaut riding a horse" with initial noise
seed=1234.
The header-only tensor check covered text_encoder, transformer and vae: flux2-klein-4b had
398 / 169 / 251 tensors and flux2-klein-9b had 399 / 233 / 251 tensors, with missing, extra and
shape-mismatched tensors all 0. Record: tensor_header_check,
validation.json.
For the 9B run, one reference/native component pair was loaded at a time; peak memory use was about 65 GB. The 4B validation run took 12 minutes and the 9B validation run took 29 minutes.
HunyuanDiT¶
Validated on CPU in the same environment as above. hunyuan-dit-v1.1-distilled was loaded strictly
from Tencent-Hunyuan/HunyuanDiT-v1.1-Diffusers-Distilled at revision 527cf2e. The observed
difference in every comparison is exactly 0: on CPU in float32, the local implementation
produces a bit-identical result to the reference.
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| BERT and mT5 tokenizers: fixed prompts and 2500 seeded fuzz strings, ids / masks at 77 / 256 tokens | identical | exact |
| BERT-large encoder (3 prompts, including Chinese and the empty negative prompt) | 0 | 0 |
| mT5-XL encoder (3 prompts) | 0 | 0 |
| HunyuanDiT transformer single step (1024×1024, batch 2, timestep 500) | 0 | 0 |
| VAE decode / encode | 0 / 0 | 0 / 0 |
| DDPM timesteps (25 / 50 steps) | exact / exact | exact / exact |
| DDPM 25-step trajectory with generator noise (v-prediction / epsilon) | 0 / 0 | 0 / 0 |
| End to end: final latent / pixels (1024×1024, 25 steps, guidance 5.0) | 0 / 0 | 0 / 0 |
| End to end, seed 42 (pixels) | 0 | 0 |
The end-to-end prompt was "a photograph of an astronaut riding a horse" with fixed initial noise
seed=7 and scheduler generator seed=42. It used the 25-step, guidance 5.0 defaults. The seed
end-to-end check used seed=42.
The header-only tensor check covered text_encoder, text_encoder_2, transformer and vae:
tensor counts 391 / 219 / 1545 / 248, with missing, extra and shape-mismatched tensors all 0.
Record: tensor_header_check, validation.json.
The reference tokenizers require sentencepiece and protobuf, which are included in the
reference extra; imagehub reads spiece.model natively at runtime. The reference and native
components were loaded together: in float32 the transformer used about 6.1 GB, mT5-XL about 6.7 GB
and BERT about 1.4 GB. The end-to-end reference test file took 52 minutes.
HunyuanDiT v1.2¶
hunyuan-dit-v1.2 (Tencent-Hunyuan/HunyuanDiT-v1.2-Diffusers, revision 5e96094) and
hunyuan-dit-v1.2-distilled (Tencent-Hunyuan/HunyuanDiT-v1.2-Diffusers-Distilled, revision
ba991d1) share the layout, text encoders, tokenizers and VAE of v1.1. They differ in two published
settings that the same code handles: the transformer config sets
use_style_cond_and_image_meta_size to false (no style embedder, 1544 transformer tensors, a
narrower extra embedder input) and the scheduler uses beta_end 0.018. The observed difference in
every comparison is exactly 0: on CPU in float32, the local implementation produces a
bit-identical result to the reference, with identical bytes for images.
| Comparison | Release | Largest observed difference | Tolerance |
|---|---|---|---|
Header-only tensor check (text_encoder, text_encoder_2, transformer, vae): 391 / 219 / 1544 / 248 tensors |
hunyuan-dit-v1.2 | missing / extra / mismatched 0 / 0 / 0 | exact |
Header-only tensor check (text_encoder, text_encoder_2, transformer, vae): 391 / 219 / 1544 / 248 tensors |
hunyuan-dit-v1.2-distilled | missing / extra / mismatched 0 / 0 / 0 | exact |
| DDPM timesteps (25 steps) | both v1.2 releases | exact | exact |
| Transformer single step (1024×1024, batch 2, timestep 500) | both v1.2 releases | 0 | 0 |
| End to end: final latent / pixels (1024×1024, 10 steps, guidance 5.0) | both v1.2 releases | 0 / 0 | 0 / 0 |
| End to end, seed 42 (10 steps, pixels) | both v1.2 releases | 0 | 0 |
The end-to-end prompt was "a photograph of an astronaut riding a horse" with fixed initial noise
seed=7 and scheduler generator seed=42, as in v1.1; the seed end-to-end check used seed=42.
The checks per release are the header comparison for all four components, the DDPM timesteps, one
transformer step at 1024×1024 (batch 2, timestep 500), the end-to-end comparison with fixed initial
noise and the seed end-to-end comparison. The end-to-end runs use 10 steps instead of the defaults
to keep the CPU float32 runs short. Tiny random-weight pipelines of both releases, including a
non-square size and guidance switched off, are compared with diffusers in the regular reference
tests, together with a DDPM trajectory using the published v1.2 scheduler configuration. Records:
tensor_header_check and the comparison measurements, validation.json.
The real-checkpoint run of the two v1.2 releases (10 real tests) took 45 minutes on an M2 Max (CPU).
Kandinsky 2.1¶
Validated on CPU in the same environment as above. kandinsky-2-1 was loaded strictly from the
decoder repository kandinsky-community/kandinsky-2-1 at revision 04b3666 and the prior
repository kandinsky-community/kandinsky-2-1-prior at revision b726af6, and compared with
diffusers 0.40.0 KandinskyPriorPipeline plus KandinskyPipeline on CPU in float32. All 4
real-checkpoint tests passed, and the observed difference in every recorded comparison is
exactly 0 (0 means exact): on CPU in float32, the local implementation produces a bit-identical
result to the reference. The real-checkpoint run took 10 minutes on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| Tensor header check: prior text encoder, prior image encoder, prior, MultilingualCLIP text encoder, text-image U-Net and MoVQ (198 / 393 / 338 / 394 / 726 / 431 tensors) | 0 missing / extra / mismatched | exact |
| XLM-RoBERTa tokenizer token ids and attention masks (4 prompts) | identical | exact |
| MultilingualCLIP text encoder (pooled and hidden states) | 0 / 0 | 0 / 0 |
| PriorTransformer single step (t = 500) | 0 | 0 |
| Text-image-conditioned U-Net single step (t = 500, batch 2) | 0 | 0 |
| MoVQ decode | 0 | 0 |
| End to end: final pixels (512×512, 25 prior steps, 100 decoder steps, guidance 4.0, fixed prior latents) | 0 | 0 |
| End to end, seed 42 (pixels) | 0 | 0 |
The end-to-end comparison used the prompt "a photograph of an astronaut riding a horse", fixed
prior latents from seed 7 and a scheduler generator seeded 42; the seed comparison ran the public
pipeline with seed=42. The defaults are 25 prior steps with guidance 4.0 followed by 100
decoder steps with guidance 4.0 at 512×512.
Published safetensors header fixtures cover the prior text encoder, prior image encoder,
PriorTransformer, MultilingualCLIP text encoder, text-image-conditioned U-Net and MoVQ on the meta
device. Random-weight reference tests match exactly for the XLM-RoBERTa tokenizer and
MultilingualCLIP encoder, the text-image U-Net, the DDIM trajectory, and the tiny combined
pipeline. The tiny end-to-end cases cover guidance on and off, an explicit negative prompt,
non-square output and two seeds. Records: tensor_header_check, end_to_end and
seed_end_to_end, validation.json; header_kandinsky-2-1, the kandinsky21_*
component comparisons, e2e_kandinsky-2-1 and seed_e2e_kandinsky-2-1,
validation-measurements.json.
Kandinsky 2.2¶
Validated on CPU in the same environment as above. kandinsky-2.2 was loaded strictly from the
decoder repository kandinsky-community/kandinsky-2-2-decoder at revision 9ae140d and the prior
repository kandinsky-community/kandinsky-2-2-prior at revision 9fc51ad. The prior is stored in
the snapshot's prior/ subfolder. The observed difference in every comparison is exactly 0: on
CPU in float32, the local implementation produces a bit-identical result to the reference.
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| Token ids | identical | exact |
| CLIP text embeddings | 0 | 0 |
| CLIP vision zero image embedding | 0 | 0 |
| PriorTransformer single step | 0 | 0 |
| Prior output after 25 steps | 0 | 0 |
| Decoder U-Net single step (512×512) | 0 | 0 |
| MoVQ decode | 0 | 0 |
| End to end at the defaults (pixels) | 0 | 0 |
| End to end, seed 42 (pixels) | 0 | 0 |
The defaults are 25 prior steps with guidance 4.0, followed by 100 decoder steps with guidance 4.0
at 512×512. Random-weight tests also established exact parity for the CLIP vision model,
PriorTransformer, image-conditioned U-Net, MoVQ, UnCLIP scheduler and the Kandinsky DDPM options.
A tiny end-to-end test matched KandinskyV22CombinedPipeline exactly.
The header-only tensor check covered movq, prior/image_encoder, prior/prior,
prior/text_encoder and unet: tensor counts 431 / 777 / 338 / 518 / 724, with missing, extra and
shape-mismatched tensors all 0. Record: tensor_header_check,
validation.json.
The reference and native models were loaded together and used about 16 GB in float32. The real-checkpoint run took about 13 minutes.
Stable Cascade¶
Validated on CPU in the same environment as above. stable-cascade and stable-cascade-lite were
loaded strictly from the decoder repository stabilityai/stable-cascade at revision a89f66d and
the prior repository stabilityai/stable-cascade-prior at revision 7ca32c2. The prior is stored
in the snapshot's prior/ subfolder. The reference is diffusers 0.40.0
StableCascadeCombinedPipeline. The observed difference in every comparison is exactly 0: on
CPU in float32, the local implementation produces a bit-identical result to the reference.
Comparisons (CPU, float32, both weight sets)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| Masked CLIP text embeddings, pooled embeddings and negative branch | 0 | 0 |
| Stage C U-Net single step (ratio 1 and 0.5) | 0 | 0 |
| Stage C latents after 60 steps | 0 | 0 |
| Stage B U-Net single step (ratio 1 and 0.5) | 0 | 0 |
| Stage B latents after 12 steps | 0 | 0 |
| Stage A decode | 0 | 0 |
| End to end at the defaults (pixels) | 0 | 0 |
| End to end, second seed (pixels) | 0 | 0 |
The defaults are 60 prior steps with guidance 4.0, followed by 12 decoder steps with guidance 0.0 at
512×512. Random-weight tests also established exact parity for both U-Net configurations (tiny and
with the published layer layout, including odd latent sizes), the DDPM Wuerstchen scheduler with
generator noise and several scaler values, Paella decoding and encoding, and the CLIP text model
with an attention mask (GELU and QuickGELU, all-ones and padded masks). A tiny end-to-end test
matched StableCascadeCombinedPipeline exactly, including a non-square size and decoder guidance.
The header-only tensor check covered the stage C U-Net, the stage B U-Net, the text encoder and stage
A. For stable-cascade-lite: prior/prior_lite 782, decoder_lite 970, prior/text_encoder 517
and vqgan 122 tensors. For stable-cascade: prior/prior 1550 and decoder 1726, with the same
text encoder and stage A. Missing, extra and shape-mismatched tensors were all 0. Record:
tensor_header_check, validation.json.
The stages were compared one at a time so that the reference and native models were never all resident together.
Würstchen v2¶
wuerstchen-v2 is loaded strictly from the decoder repository warp-ai/wuerstchen at revision
c3da414 and the prior repository warp-ai/wuerstchen-prior at revision c41be63; the prior is
stored in the snapshot's prior/ subfolder. The reference is diffusers 0.40.0
WuerstchenCombinedPipeline. The real-checkpoint tests (tests/reference/test_ref_wuerstchen.py)
assert the tolerances below. All 7 real-checkpoint tests passed with exactly 0 difference in every
comparison (see the table), and the run took about 1 h 2 min on an M2 Max (CPU). The random-weight
and tiny-snapshot comparisons need no downloaded weights and also pass with exactly 0 difference.
Comparisons (CPU, float32)¶
| Comparison | Tolerance | Measured |
|---|---|---|
| Tensor header check: both text encoders, prior, decoder, VQGAN (names and shapes) | 0 missing, extra or mismatched | 0 missing, extra or mismatched (388 and 517 tensors in the text encoders, 648 prior, 990 decoder, 122 VQGAN) |
| Masked CLIP text states of the prior (ViT-bigG) and of the decoder (ViT-H), prompt and negative | 0 | exactly 0 (max abs and max rel) |
| Prior single step (batch 2, three timestep ratios, real text states) | 0 | exactly 0 at every ratio |
| Decoder single step (two timestep ratios, real text states) | 0 | exactly 0 at both ratios |
| VQGAN decode | 0 | exactly 0 |
| End to end: prior latents / decoder latents / pixels (512×512, 60 and 12 steps, seed 0) | 0 / 0 / 0 | 0 / 0 / 0 |
| End to end, seed 1234 (pixels) | 0 | 0 |
The tensor layout of the published files was also checked against the recorded headers in
tests/fixtures/wuerstchen_configs/: 648 tensors for the prior, 990 for the decoder and 122 for
the VQGAN, with no missing, extra or mismatched names or shapes.
Random-weight parity covers the prior and the decoder with tiny configurations and with the
published configurations at small latent sizes, including the optional x_cat input and the split
output of the decoder. Tiny-snapshot end-to-end parity against WuerstchenCombinedPipeline covers
the defaults, decoder guidance with a negative prompt, decoder guidance with the prior's guidance
switched off, a non-square size, a second seed, and a prompt longer than the 77 tokens the text
encoders accept; it compares the prior's image embeddings, the decoder's latents and the final
bytes.
GLM-Image¶
glm-image is loaded strictly from zai-org/GLM-Image at revision 2c433cc; the reference is
diffusers 0.40.0 GlmImagePipeline with transformers 5.18.0 GlmImageForConditionalGeneration,
GlmImageProcessor, T5EncoderModel and ByT5Tokenizer. The transformer, the autoregressive
model and the VAE are published in bfloat16 and are widened to float32 on both sides. The model has
about 16 billion parameters, so the real-checkpoint tests
(tests/reference/test_ref_glm_image.py) run each reference stage in a fresh process, save its
result and compare it with the native code afterwards, and the end-to-end test shortens the
diffusion stage to 4 steps at 1024×1024 (the autoregressive stage runs in full) and uses a second
seed at 512×512 with 2 steps. All 8 real-checkpoint tests passed, and every comparison in the
table below is exact: the difference is 0 or the tokens are identical. The real-checkpoint run took
64 minutes on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Measured | Tolerance |
|---|---|---|
| Tensor header check: autoregressive model (vision tower and VQ encoder excluded), ByT5 encoder, transformer, VAE | 0 missing, extra or mismatched (1011 autoregressive model, 111 ByT5 encoder, 443 transformer, 288 VAE tensors) | 0 missing, extra or mismatched |
| Prompt tokenization with the processor's chat template and target grids, fuzz against the processor tokenizer, ByT5 ids | identical (5 prompts at three target grids, 300 fuzz strings) | exact |
| Glyph embeddings for the prompt and the empty string | 0 (max abs and max rel) | 0 |
| Prior tokens (512×512, seed 0) sampled with the published generation settings | exact | exact |
| Transformer conditional and unconditional single step | 0 | 0 |
| VAE decode | 0 | 0 |
| End to end (1024×1024, 4 steps, seed 0; 512×512, 2 steps, seed 1): pixels | 0 in both runs | 0 |
The comparisons that need no downloaded weights also pass with exactly 0 difference: random-weight
transformer parity (tiny, medium and published configurations, with and without the dropped prior),
random-weight sampling parity of the autoregressive model against generate (tiny and published
head geometry, several seeds, temperature, top-k and top-p settings, one-token prompts), the
sigma and timestep schedule against diffusers for several step counts and resolutions, the ByT5
tokenizer against transformers on a fuzz set that includes the special tokens, and a tiny
end-to-end comparison with the whole diffusers pipeline (processor, sampling, glyph embeddings,
guidance on and off, non-square sizes, several seeds).
PixArt¶
PixArt-LCM¶
Validated on CPU in the same environment as above. pixart-lcm-xl-1024 uses
PixArt-alpha/PixArt-LCM-XL-2-1024-MS at revision e153060b with the published float32
transformer, VAE and four T5 shards. It reuses the PixArt Alpha components and the LCM scheduler,
defaults to 4 steps and guidance 0.0, and passes strict fixture-header and tiny-snapshot parity
tests. All 7 real-checkpoint tests passed, and the observed difference in every recorded
comparison is exactly 0 (0 means exact). The real-checkpoint run took 5 minutes on an M2 Max
(CPU).
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| Tensor header check: text encoder, transformer, VAE (219 / 612 / 248 tensors) | 0 missing / extra / mismatched | exact |
| T5 tokenizer ids / attention masks | identical / identical | exact / exact |
| T5-v1.1-XXL encoder | 0 | 0 |
| Transformer single step (1024×1024) | 0 | 0 |
| VAE decode / encode | 0 / 0 | 0 / 0 |
| LCM scheduler (4 steps) | exact | exact |
| End to end: final latent / pixels (1024×1024, 4 steps, guidance 0.0) | 0 / 0 | 0 / 0 |
| End to end, seed 42 (pixels) | 0 | 0 |
The end-to-end prompt was "a photograph of an astronaut riding a horse" with fixed initial noise
seed=7 and scheduler generator seed=42. The seed end-to-end check used seed=42. Record:
tensor_header_check, end_to_end and seed_end_to_end, validation.json;
the pixart-lcm-xl-1024_* component keys plus e2e_pixart-lcm-xl-1024 and
seed_e2e_pixart-lcm-xl-1024, validation-measurements.json.
Further resolutions¶
pixart-alpha-xl-512 (PixArt-alpha/PixArt-XL-2-512x512, revision 50f7021) and the transformer-only
pixart-sigma-xl-256, pixart-sigma-xl-512 and pixart-sigma-xl-2k (revisions 495cb1b, 76fb7eb and
cd274f4, with the remaining components from PixArt-Sigma-XL-2-1024-MS at e102b35) share the
PixArt code. Their published configurations and tensor headers are checked in as fixtures, and the
real-checkpoint tests in tests/reference/test_ref_pixart.py and
tests/reference/test_ref_random_pixart.py are parametrized over them: header, tokenizer and encoder,
one transformer step at the native latent size, VAE decode and encode, scheduler timesteps, end to end
at the native size with fixed latents and the seed test. The 2K model's single-step and decode checks use
1024-pixel inputs, and its end-to-end runs use 4 steps instead of 20 to keep the 2048×2048 runs short.
Measured on CPU in the same environment as above, every recorded comparison for the four models is
exact: identical tokenizer ids and attention masks, difference 0 for the T5-v1.1-XXL encoder, the
transformer step, VAE decode and encode, the end-to-end latent and pixels and the seed test, exact
DPM-Solver++ timesteps, and 0 missing, extra and shape-mismatched tensors in the header check
(text_encoder / transformer / vae: 219 / 603 / 248 for the three Sigma models and 219 / 604 /
248 for pixart-alpha-xl-512). The real-checkpoint run took 59 minutes on an M2 Max (CPU).
Validated on CPU in the same environment as above. pixart-sigma-xl-1024 and
pixart-alpha-xl-1024 were loaded strictly from PixArt-alpha/PixArt-Sigma-XL-2-1024-MS at
revision e102b35 and PixArt-alpha/PixArt-XL-2-1024-MS at revision b89adad, respectively; the
four further-resolution checkpoints were loaded as described above. The observed difference in every
comparison is exactly 0: on CPU in float32, the local implementation produces a bit-identical
result to the reference.
Comparisons (CPU, float32)¶
| Comparison | Model | Largest observed difference | Tolerance |
|---|---|---|---|
| T5 tokenizer ids / attention masks (300 tokens) | pixart-sigma-xl-1024 |
identical / identical | exact / exact |
| T5-v1.1-XXL encoder | pixart-sigma-xl-1024 |
0 | 0 |
| Transformer single step (1024×1024, batch 2) | pixart-sigma-xl-1024 |
0 | 0 |
| VAE decode / encode | pixart-sigma-xl-1024 |
0 / 0 | 0 / 0 |
| DPM-Solver++ timesteps (20 steps) / trajectories | pixart-sigma-xl-1024 |
exact / 0 | exact / 0 |
| End to end: final latent / pixels (1024×1024, 20 steps, guidance 4.5) | pixart-sigma-xl-1024 |
0 / 0 | 0 / 0 |
| End to end, seed 42 (pixels) | pixart-sigma-xl-1024 |
0 | 0 |
| T5 tokenizer ids / attention masks (120 tokens) | pixart-alpha-xl-1024 |
identical / identical | exact / exact |
| T5-v1.1-XXL encoder | pixart-alpha-xl-1024 |
0 | 0 |
| Transformer single step (1024×1024, batch 2) | pixart-alpha-xl-1024 |
0 | 0 |
| VAE decode / encode | pixart-alpha-xl-1024 |
0 / 0 | 0 / 0 |
| DPM-Solver++ timesteps (20 steps) / trajectories | pixart-alpha-xl-1024 |
exact / 0 | exact / 0 |
| End to end: final latent / pixels (1024×1024, 20 steps, guidance 4.5) | pixart-alpha-xl-1024 |
0 / 0 | 0 / 0 |
| End to end, seed 42 (pixels) | pixart-alpha-xl-1024 |
0 | 0 |
| T5 tokenizer ids / attention masks | pixart-alpha-xl-512 |
identical / identical | exact / exact |
| T5-v1.1-XXL encoder | pixart-alpha-xl-512 |
0 | 0 |
| Transformer single step (native latent size) | pixart-alpha-xl-512 |
0 | 0 |
| VAE decode / encode | pixart-alpha-xl-512 |
0 / 0 | 0 / 0 |
| DPM-Solver++ timesteps (20 steps) | pixart-alpha-xl-512 |
exact | exact |
| End to end: final latent / pixels (512×512, 20 steps, guidance 4.5) | pixart-alpha-xl-512 |
0 / 0 | 0 / 0 |
| End to end, seed 42 (pixels) | pixart-alpha-xl-512 |
0 | 0 |
| T5 tokenizer ids / attention masks | pixart-sigma-xl-256 |
identical / identical | exact / exact |
| T5-v1.1-XXL encoder | pixart-sigma-xl-256 |
0 | 0 |
| Transformer single step (native latent size) | pixart-sigma-xl-256 |
0 | 0 |
| VAE decode / encode | pixart-sigma-xl-256 |
0 / 0 | 0 / 0 |
| DPM-Solver++ timesteps (20 steps) | pixart-sigma-xl-256 |
exact | exact |
| End to end: final latent / pixels (256×256, 20 steps, guidance 4.5) | pixart-sigma-xl-256 |
0 / 0 | 0 / 0 |
| End to end, seed 42 (pixels) | pixart-sigma-xl-256 |
0 | 0 |
| T5 tokenizer ids / attention masks | pixart-sigma-xl-512 |
identical / identical | exact / exact |
| T5-v1.1-XXL encoder | pixart-sigma-xl-512 |
0 | 0 |
| Transformer single step (native latent size) | pixart-sigma-xl-512 |
0 | 0 |
| VAE decode / encode | pixart-sigma-xl-512 |
0 / 0 | 0 / 0 |
| DPM-Solver++ timesteps (20 steps) | pixart-sigma-xl-512 |
exact | exact |
| End to end: final latent / pixels (512×512, 20 steps, guidance 4.5) | pixart-sigma-xl-512 |
0 / 0 | 0 / 0 |
| End to end, seed 42 (pixels) | pixart-sigma-xl-512 |
0 | 0 |
| T5 tokenizer ids / attention masks | pixart-sigma-xl-2k |
identical / identical | exact / exact |
| T5-v1.1-XXL encoder | pixart-sigma-xl-2k |
0 | 0 |
| Transformer single step (native latent size) | pixart-sigma-xl-2k |
0 | 0 |
| VAE decode / encode | pixart-sigma-xl-2k |
0 / 0 | 0 / 0 |
| DPM-Solver++ timesteps (20 steps) | pixart-sigma-xl-2k |
exact | exact |
| End to end: final latent / pixels (2048×2048, 4 steps, guidance 4.5) | pixart-sigma-xl-2k |
0 / 0 | 0 / 0 |
| End to end, seed 42 (pixels) | pixart-sigma-xl-2k |
0 | 0 |
The end-to-end prompt was "a photograph of an astronaut riding a horse" with fixed initial noise
seed=7 and scheduler generator seed=42. It used the 20-step, guidance 4.5 defaults, except
pixart-sigma-xl-2k, which ran 4 steps. The seed end-to-end check used seed=42.
Random-weight parity also covered Sigma- and Alpha-style tiny and medium transformer configurations, and tiny-snapshot end-to-end parity covered both the PixArt Sigma and PixArt Alpha pipelines. Every comparison was exact.
The header-only tensor check covered text_encoder, transformer and vae: tensor counts were
219 / 603 / 248 for pixart-sigma-xl-1024 and 219 / 612 / 248 for pixart-alpha-xl-1024, with
missing, extra and shape-mismatched tensors all 0. Record: tensor_header_check,
validation.json.
Caption cleaning requires bs4 and ftfy and is not implemented. Parity was measured with
clean_caption=False, which is what diffusers uses when those packages are unavailable. The
reference and native components were loaded together; T5-XXL used about 19 GB in float32. The
real-checkpoint run for both models (19 tests) took 45 minutes.
Nucleus-Image¶
Validated on CPU in the same environment as above. nucleus-image was loaded strictly from
NucleusAI/Nucleus-Image at revision 5e963db4fd0a65c7e4faf53ca2d4eca567c4dcfa and compared with
diffusers 0.40.0 NucleusMoEImagePipeline and transformers 5.18.0. The published bfloat16 weights
were upcast to float32. All 4 real-checkpoint tests passed, and the observed difference in every
comparison is exactly 0.
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| Transformer / VAE tensor names and shapes (610 / 194 tensors, strict meta-device load) | identical | exact |
| Qwen3-VL text states (layer -8, valid positions) and attention mask | 0 / identical | exact |
| VAE decode (1024×1024) | 0 | 0 |
| End to end, seed 0 (1024×1024, 2 steps, guidance 4.0; pixels) | 0 | 0 |
| End to end, seed 42 (1024×1024, 2 steps, guidance 4.0; pixels) | 0 | 0 |
The end-to-end prompt was "a photograph of an astronaut riding a horse". The two steps only bound the CPU run; the default is 50 steps. The reference pipeline runs in its own process, because the float32 reference and native pipelines do not fit in memory together. The run took 29 minutes on an M2 Max (CPU) with 12 threads.
The published transformer config sets use_grouped_mm, so the expert layers use grouped matrix
products on CPU and CUDA, as diffusers does. With several CPU threads, one product per expert
differs from them in the last bit at some token counts. At 1024×1024 the first two expert blocks
take 256 tokens per expert, and the difference changed the image. A random-weight test at the
published widths and 4,096 image tokens covers this case. On MPS, where grouped products fail for
small shapes, the backend computes one product per expert.
Random-weight parity covers the transformer with and without text masks, and tiny-snapshot
pipelines compare guidance on and off, non-square output, token dropping and two seeds.
Records: header_nucleus-image, nucleus-image_text_encoder, nucleus-image_vae_decode,
e2e_nucleus-image and seed_e2e_nucleus-image,
validation-measurements.json.
Krea 2¶
Validated on CPU in the same environment as above. krea2-turbo was loaded strictly from
krea/Krea-2-Turbo at revision 98e0fe1. The weights are published in bfloat16 and were upcast
to float32. The observed difference in every comparison is exactly 0: on CPU in float32, the
local implementation produces a bit-identical result to the reference.
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| Tensor header names and shapes | identical | exact |
| Prompt template token ids / attention masks | identical / identical | exact / exact |
| Qwen3-VL text encoder (12 selected hidden layers, stacked) | 0 | 0 |
| Transformer single step (1024×1024) | 0 | 0 |
| VAE decode / encode | 0 / 0 | 0 / 0 |
Flow-match Euler scheduler with exponential dynamic shifting, including mu |
exact | exact |
| End to end: final latent / pixels (1024×1024, 8 steps, guidance 0.0) | 0 / 0 | 0 / 0 |
| End to end, seed 42 (pixels) | 0 | 0 |
The end-to-end prompt was "a photograph of an astronaut riding a horse". The fixed-noise and seed checks used the 8-step, guidance 0.0, 1024×1024 defaults.
The header-only tensor check covered text_encoder, transformer and vae: tensor counts were
713 / 430 / 194, with missing, extra and shape-mismatched tensors all 0. Record:
tensor_header_check, validation.json.
The reference pipeline calls the text encoder with transformers' default key-value cache. Its
contiguous key/value copies and masked grouped-query attention on hosts with MPS set the SDPA
summation order. The transformer's float32 tensors remain at their file offsets as diffusers loads
them (clone_shards = False). Latents are de-normalized by dividing by the reciprocal standard
deviation, matching the reference. The reference transformer was loaded without
_keep_in_fp32_modules because that path requires accelerate; all components ran in float32,
and the transformer was cast to float32 after loading.
The roughly 13B transformer uses about 52 GB in float32, so the reference and native transformers were never loaded together. Each comparison ran the reference first, saved its outputs, freed it, then ran the native model. The full real-checkpoint run on the Apple M2 Max took about 1 hour.
ERNIE-Image¶
ernie-image is loaded strictly from baidu/ERNIE-Image at revision 5346b31. The weights are
published in bfloat16 and are upcast to float32. The observed difference in every comparison is
exactly 0: on CPU in float32, the local implementation produces a bit-identical result to the
reference. The real-checkpoint run took 36 minutes on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| Tensor header names and shapes (transformer, VAE, Ministral 3 language model) | 0 missing / extra / mismatched | exact |
| Tokenizer against transformers (fixed prompts, 500 random strings, truncation) | identical | exact |
| Ministral 3 text encoder (second-to-last hidden state) | 0 | 0 |
| Transformer single step (two latents, ragged text lengths) | 0 | 0 |
| VAE decode with latent de-normalisation | 0 | exact pixels |
| Flow-match Euler scheduler (static shift, 1, 4 and 50 steps) | exact | exact |
| End to end: final latent / pixels (1024×1024, 4 steps, guidance 4.0) | 0 / 0 | 0 / 0 |
| End to end, seed 0 (pixels) | 0 | 0 |
Without weights, the same code is compared against diffusers and transformers with random weights: the transformer and the Ministral 3 language model (tiny and published-style configs, including YaRN positions beyond the original window), and the whole pipeline on a tiny snapshot (guidance on and off, non-square sizes, two seeds, several images per prompt). All of these are exact.
The header-only tensor check covered text_encoder, transformer and vae: tensor counts were
236 / 409 / 251, with missing, extra and shape-mismatched tensors all 0. Record:
tensor_header_check, validation.json.
The reference runs with use_pe=False and without the pe components (the prompt enhancer is out
of scope). The end-to-end runs use 4 steps instead of the default 50 because the 8B transformer is
slow on CPU. The transformer alone needs about 32 GB in float32 and the text encoder about 14 GB,
so every comparison runs the reference first, saves its outputs, frees it, then runs the native
model.
HunyuanImage 2.1¶
Validated on CPU in the same environment as above. hunyuanimage-2-1 was loaded strictly from
hunyuanvideo-community/HunyuanImage-2.1-Diffusers at revision
7e7b7a177de58591aeaffca0929f4765003d7ced and compared with diffusers 0.40.0
HunyuanImagePipeline and transformers 5.18.0. The published bfloat16 transformer, VAE and Qwen
text encoder were upcast to float32; the ByT5 encoder is published in float32. All 8
real-checkpoint tests passed, and the observed difference in every comparison is exactly 0.
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| Qwen and ByT5 tokenizers (4 prompts, with and without quoted text) | identical | exact |
| Qwen2.5-VL text embeddings (prompt / negative prompt) | 0 / 0 | 0 |
| ByT5 glyph encoder | 0 | 0 |
| Transformer single step | 0 | 0 |
| VAE decode / encode (1024×1024) | 0 / 0 | 0 |
| Flow-match Euler schedule | exact | exact |
| End to end with fixed noise: text embeddings / final latent / pixels (512×512, 8 steps) | 0 / 0 / 0 | 0 |
| End to end, seed 42 (256×256, 3 steps; pixels) | 0 | 0 |
The end-to-end runs used the prompt "A cute cartoon penguin plush toy standing in a painting studio, photorealistic" and the guider configuration from the snapshot. The shortened step counts and sizes only bound the CPU run; the defaults are 50 steps at 2048×2048. Each reference stage runs in a subprocess that exits before the native stage, so the two 68 GB float32 transformers are never resident together. The run took 25 minutes on an M2 Max (CPU).
The header-only tensor check covered text_encoder, text_encoder_2, transformer and vae:
tensor counts 729 / 111 / 1264 / 324, with missing, extra and shape-mismatched tensors all 0.
The catalog uses the community diffusers conversion. scripts/check_hunyuanimage_conversion.py
converts Tencent's official checkpoint (tencent/HunyuanImage-2.1 at revision
e435da11d9e8795a25e224c5ba27b099ed45c55b) with diffusers' conversion functions and compares every
tensor: the transformer is an exact copy of the official bfloat16 weights, while the community VAE
is the official float32 VAE rounded to bfloat16.
Random-weight parity covers the transformer, VAE, text encoder, tokenizer wrapper, guider,
scheduler and a tiny end-to-end snapshot with exact PyTorch equality. Records:
header_hunyuanimage_2_1, the hunyuanimage_* component comparisons, e2e_hunyuanimage_2_1 and
seed_e2e_hunyuanimage_2_1, validation-measurements.json.
LongCat-Image¶
Validated on CPU in the same environment as above. longcat-image was loaded strictly from
meituan-longcat/LongCat-Image at revision d2ea50b. The published bfloat16 weights were upcast to
float32. The observed difference in every comparison is exactly 0: on CPU in float32, the local
implementation produces a bit-identical result to the reference.
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| Rewritten prompt string and token ids | identical | exact |
| Qwen2.5-VL text encoder | 0 | 0 |
| LongCat-Image transformer single step (1024×1024) | 0 | 0 |
| VAE decode / encode | 0 / 0 | 0 / 0 |
Flow-match Euler scheduler with dynamic shifting, including mu |
exact | exact |
| End to end: final latent / pixels (1024×1024, 8 steps, guidance 4.5) | 0 / 0 | 0 / 0 |
| End to end, seed 42 (pixels) | 0 | 0 |
The end-to-end run used 8 steps instead of the 50-step default and fixed initial noise. Prompt
rewrite and CFG renormalization were enabled. The seed end-to-end check used seed=42.
Random-weight parity covered the Qwen2.5-VL text model, including per-step generation logits, and the transformer. Tiny-snapshot end-to-end parity covered configurations with and without prompt rewrite and CFG renormalization. Every comparison was exact.
The header-only tensor check covered text_encoder, transformer and vae: tensor counts were
729 / 612 / 244, with missing, extra and shape-mismatched tensors all 0. Record:
tensor_header_check, validation.json.
Reference-matching details: rewrite generation uses a key-value cache with one new token per step,
as transformers' generate does; recomputing the whole sequence changed logits and flipped tokens.
The Qwen2-VL chat template adds a default system turn. Published Qwen2.5-VL checkpoints use
transformers 4.x model.layers.* key names and the legacy "type": "mrope", which are read as
transformers 5 does. The text encoder uses the default key-value cache, with contiguous keys and
values and the masked GQA rule on MPS hosts.
In float32 the text encoder used about 33 GB and the transformer about 25 GB. End-to-end validation used the save-then-compare sequence. The real-checkpoint runs took about 26, 25 and 47 minutes.
CogView4¶
Validated on CPU in the same environment as above. cogview4-6b was loaded strictly from
zai-org/CogView4-6B at revision 63a52b7. The published bfloat16 weights were upcast to float32.
The observed difference in every comparison is exactly 0: on CPU in float32, the local
implementation produces a bit-identical result to the reference.
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| GLM tokenizer: 8 fixed prompts and 2000 seeded fuzz strings | identical | exact |
| GLM token ids | identical | exact |
| GLM-4-9B text encoder, second-to-last hidden state | 0 | 0 |
| CogView4 transformer single step (1024×1024) | 0 | 0 |
| VAE decode / encode | 0 / 0 | 0 / 0 |
Flow-match Euler scheduler with linear dynamic shifting, including mu |
exact | exact |
| End to end: final latent / pixels (1024×1024, 8 steps, guidance 5.0) | 0 / 0 | 0 / 0 |
| End to end, seed 42 (pixels) | 0 | 0 |
The end-to-end prompt was "a photograph of an astronaut riding a horse" with fixed initial noise.
It used 8 steps instead of the 50-step default to shorten the CPU run, with guidance 5.0. The seed
end-to-end check used seed=42.
Random-weight parity also covered the GLM text model and CogView4 transformer, and tiny-snapshot end-to-end parity covered the complete pipeline. Every comparison was exact. The reference text encoder was called exactly as the pipeline calls it: with the default key-value cache, contiguous keys and values, and the masked grouped-query attention rule.
The header-only tensor check covered text_encoder, transformer and vae: tensor counts were
442 / 408 / 288, with missing, extra and shape-mismatched tensors all 0. Record:
tensor_header_check, validation.json.
The text encoder used about 36 GB and the transformer about 25 GB in float32. End-to-end tests used the save-then-compare sequence: the reference pipeline ran first, saved its outputs and was freed before the native pipeline ran. The real-checkpoint component and end-to-end runs took about 31 and 53 minutes, respectively.
CogView3-Plus¶
cogview3-plus-3b (zai-org/CogView3-Plus-3B, revision 5d70e40) was loaded strictly and
validated on CPU in float32 against diffusers 0.40.0 (CogView3PlusPipeline) and transformers
5.18.0. The published transformer and VAE are bfloat16 and are upcast to float32 by both
implementations; the T5-XXL text encoder is published in float32. The observed difference in every
comparison is exactly 0: on CPU in float32, the local implementation produces a bit-identical
result to the reference. The real-checkpoint run took 37 minutes on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| Tensor header check: text encoder, transformer, VAE (219 / 436 / 288 tensors) | 0 missing / extra / mismatched | exact |
| T5 token ids | identical | exact |
| T5-XXL text encoder | 0 | 0 |
Transformer single step (1024×1024, batch 2, timesteps [999, 999] and [1, 501]) |
0 | 0 |
| VAE decode / encode | 0 / 0 | 0 |
| Shifted-SNR DDIM scheduler timesteps (8 and 50 steps) | exact | exact |
| End to end: final latent (1024×1024, 8 steps, guidance 5.0) | 0 | 0 |
| End to end: pixels (1024×1024, 8 steps, guidance 5.0) | 0 | 0 |
| End to end, seed 0 (1024×1024, 8 steps, guidance 5.0; pixels) | 0 | 0 |
Tests that need no weights (all exact, tolerance 0):
- Scheduler:
CogVideoXDDIMSchedulertimesteps,alphas_cumprodandfinal_alpha_cumprodfor six step counts and seven configuration variants (leading, linspace and trailing spacing, linear, scaled-linear and cosine betas, zero-terminal-SNR rescaling), and 25- and 50-step trajectories for all three prediction types. - Tokenizer: the published SentencePiece model (kept as a fixture) against transformers on fixed
prompts with the pipeline's
max_lengthpadding and truncation, 3000 seeded fuzz strings and every Unicode code point. - Random-weight parity of the transformer: a tiny configuration, a wider one and the published configuration with two layers.
- Tiny-snapshot end to end against
CogView3PlusPipeline: seed andlatentssources, the zero negative embedding and an explicit negative prompt, guidance on and off, square and non-square sizes, and the SDXL-styleoriginal_sizeandcrops_coords_top_leftconditions. - Header checks of the text encoder (219 tensors), transformer (436) and VAE (288) against the published safetensors headers, kept as fixtures.
The real-checkpoint file tests/reference/test_ref_cogview3_plus.py (skipped without the snapshot
IMAGEHUB_SNAPSHOT_COGVIEW3_PLUS_3B) covered the tensor headers, tokens and the T5 text encoder, the
transformer at two timesteps (1024×1024, batch 2, 224 text tokens), VAE decode and encode, the
scheduler at 8 and 50 steps, and end-to-end and seed runs at 1024×1024 with guidance 5.0, seed 0 and
8 steps instead of the 50-step default to shorten the CPU run. The reference result is computed,
saved and freed before the native model is loaded. Record: tensor_header_check,
validation.json.
Ideogram 4¶
ideogram4-fp8 was loaded strictly from ideogram-ai/ideogram-4-fp8 at revision ee79a72 and
validated on CPU in float32. The reference is the official ideogram-oss/ideogram4 implementation
at commit 990fe1c4e950bb9e9dc90e01c0ad98ba434f83c2, not diffusers: the model card lists no diffusers
support for the fp8 release. The official reference ran with PyTorch 2.14.1 and transformers 5.18.0
in its own Python 3.13 environment. Every comparison against that official implementation is
exactly 0.
The official reference environment was created with:
uv venv -p 3.13 .venv-ref-ideogram4
uv pip install --python .venv-ref-ideogram4 git+https://github.com/ideogram-oss/ideogram4@990fe1c4e950bb9e9dc90e01c0ad98ba434f83c2 "transformers==5.18.0" pytest numpy
The official-reference tests run with:
IMAGEHUB_REFERENCE_TESTS=1 PYTHONPATH=$PWD .venv-ref-ideogram4/bin/python -m pytest tests/reference/test_ref_ideogram4.py tests/reference/test_ref_tiny_ideogram4.py
Comparisons with the official implementation (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| Published headers: fp8 tensors and per-row scales for every quantized component | identical | exact |
| Prompt token ids | identical | exact |
| Qwen3-VL text features (53,248 wide from stacked hidden states) | 0 | 0 |
| Conditional transformer single step (1024×1024) | 0 | 0 |
| Unconditional transformer single step (1024×1024) | 0 | 0 |
| Autoencoder decode / encode | 0 / 0 | 0 / 0 |
| Official sampler intervals and per-step guidance schedule | identical | exact |
End to end, seed 0 (V4_TURBO_12, 1024×1024 pixels) |
0 | 0 |
End to end, seed 42 (V4_TURBO_12, 1024×1024 pixels) |
0 | 0 |
The published header check covered text_encoder, transformer, unconditional_transformer and
vae: 1117 / 669 / 669 / 251 tensors. The three quantized components contained 368 / 211 / 211
float8 e4m3 weights and the same number of float32 scales; the VAE contained neither. imagehub
keeps the published fp8 weights and dequantizes each forward pass as
weight.to(dtype) * weight_scale.to(dtype).unsqueeze(1), exactly like the official implementation.
The float32 result is bit-identical. Record: tensor_header_check,
validation.json.
Random-weight parity covered fp8 linears and the transformer, and a tiny end-to-end test covered
the complete pipeline against the official implementation. Every comparison was exact. The
real-checkpoint end-to-end comparisons used the official V4_TURBO_12 preset rather than the
48-step V4_QUALITY_48 default to shorten the CPU run.
The adapter reference uses diffusers 0.40.0 on a tiny float32 transformer whose weights are made
by dequantizing each fp8 row (weight * weight_scale), splitting fused attention.qkv into
to_q, to_k and to_v, and renaming attention.o to to_out.0. It checks single, stacked
0.8 / 0.5, fused, and q-only PEFT adapters against the native conditional transformer exactly.
The test does not make an end-to-end diffusers pipeline comparison: the tiny reference fixture
only constructs transformer inputs, not all pipeline components.
The official pipeline peaks near 52 GB while loading in float32 and the native pipeline near 40 GB. The official reference therefore ran in a child process with the same thread count, saved its outputs and exited before the native outputs were compared. The real-checkpoint run took about two hours.
Qwen-Image¶
qwen-image was loaded strictly from Qwen/Qwen-Image at revision
75e0b4be04f60ec59a75f475837eced720f823b6 and compared with diffusers 0.40.0 QwenImagePipeline
and transformers 5.18.0 Qwen2_5_VLForConditionalGeneration on CPU in float32, on the float32
upcast of the published bfloat16 weights. All 6 real-checkpoint tests passed, and the observed
difference in every recorded comparison is exactly 0 (0 means exact): on CPU in float32, the
local implementation produces a bit-identical image to the one diffusers produces. The
real-checkpoint run took 30 minutes on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
Tensor header names and shapes (text_encoder, transformer, vae; 729 / 1933 / 194 tensors) |
0 missing / extra / mismatched | exact |
| Qwen2 tokenizer ids for the templated prompt | identical | exact |
| Qwen2.5-VL text encoder (last hidden state) | 0 | 0 |
| Qwen-Image transformer single step (1×4×64 latents, 1×8×3584 context, timestep 0.75, one call) | 0 | 0 |
| VAE decode (16-channel latents) | 0 | 0 |
End to end: final latent / pixels (32×32, 2 steps, true_cfg 4.0) |
0 / 0 | 0 / 0 |
End to end, seed 42 (32×32, 2 steps, true_cfg 4.0; pixels) |
0 | 0 |
The end-to-end prompt was "a photograph of an astronaut riding a horse" with the negative prompt
"blurry, low quality". Two steps at 32×32 only shorten the CPU run; the backend defaults are 50
steps at 1024×1024. Records: header_qwen-image, qwen_image_tokenizer, qwen_image_text_encoder,
qwen_image_transformer, qwen_image_vae, e2e_qwen-image and seed_e2e_qwen-image,
validation-measurements.json.
Qwen-Image-Edit¶
qwen-image-edit was loaded strictly from Qwen/Qwen-Image-Edit at revision
ac7f9318f633fc4b5778c59367c8128225f1e3de, and qwen-image-edit-2509 from
Qwen/Qwen-Image-Edit-2509 at revision d3968ef930e841f4c73640fb8afa3b306a78167e. They were
compared with diffusers 0.40.0 QwenImageEditPipeline and QwenImageEditPlusPipeline and
transformers 5.18.0 on CPU in float32, on the float32 upcast of the published bfloat16 weights.
All 20 real-checkpoint tests passed for both releases, and the observed difference in every
recorded comparison is exactly 0 (0 means exact).
The suite checks every published tensor header, image preprocessing and Qwen2.5-VL conditioning (including a long vision sequence through the first transformer block), a transformer step, VAE encode/decode, the source-image latent stages, the scheduler, and separate public-pipeline end-to-end runs for seeds 0 and 42. 2509 runs with two source images. The end-to-end cases use two steps at 32×32 to keep the CPU run bounded; production defaults remain 50 steps at 1024×1024. Reference components are loaded directly, and prompt embeddings are computed before the diffusion components are loaded, because the transformers CPU result is allocation-sensitive for this long vision sequence when Accelerate is unavailable.
The seed-42 run of qwen-image-edit found that the latents handed to the VAE decoder must be
rescaled the way the pipeline does it, latents / (1 / std) + mean: latents * std + mean
differs in the last bit and moved one pixel that sits on a rounding boundary. A tiny test with the
published latent statistics now compares the decoder input bit for bit for qwen-image and both
edit releases.
Comparisons (CPU, float32)¶
Each row was measured for both qwen-image-edit and qwen-image-edit-2509; the values are the same
for both.
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
Published tensor headers (text_encoder / transformer / vae: 729 / 1933 / 194 tensors; missing, extra, mismatched) |
0 | exact |
| Image preprocessing and Qwen2.5-VL conditioning (token ids, pixel values, prompt embeddings) | identical / 0 / 0 | exact / 0 / 0 |
| Conditioning at the published condition resolution (token ids, pixel values, hidden states) | identical / 0 / 0 | exact / 0 / 0 |
| First transformer block on the long vision sequence (image / text stream) | 0 / 0 | 0 |
| Transformer step with source latents | 0 | 0 |
| VAE encode / decode | 0 / 0 | 0 |
| Source-image stages (preprocessed pixels, source latents, seed-42 noise, decoded pixels) | 0 / 0 / 0 / 0 | 0 |
| Flow-match Euler scheduler with dynamic shifting (sigmas / timesteps) | 0 / 0 | 0 |
End to end, seed 0 (32×32, 2 steps, true_cfg 4.0; pixels) |
0 | 0 |
End to end, seed 42 (32×32, 2 steps, true_cfg 4.0; pixels) |
0 | 0 |
Records: header_<id>, qwen_image_edit_conditioner_<id>,
qwen_image_edit_real_conditioning_<id>, qwen_image_edit_transformer_<id>,
qwen_image_edit_long_transformer_first_block_<id>, qwen_image_edit_vae_<id>,
qwen_image_edit_image_stages_<id>, qwen_image_edit_scheduler_<id>, e2e_<id> and
seed_e2e_<id> for <id> in qwen-image-edit and qwen-image-edit-2509,
validation-measurements.json.
Qwen-Image 2.1¶
The reference is the implementation the Qwen team contributed to diffusers. No released diffusers
version contains it yet: 0.40.0 predates it. The reference was pinned to commit
7997e4eabb1f880fa5697e2b24de09a892fffad5 in a separate environment with the same PyTorch 2.14.1
and transformers 5.18.0. The comparison will be re-run when a diffusers release includes the model.
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| Prompt token ids (2 prompts) | identical | exact |
| Qwen3-VL text encoder (last decoder layer before the final norm) | 0 | 0 |
| Transformer cached-flow calls (1024×1024) | 0 | 0 |
| VAE decode / encode | 0 / 0 | 0 |
| Flow-match Euler scheduler with dynamic shifting | 0 | 0 |
End to end: final latent / pixels (1024×1024, 8 steps, true_cfg 1.0) |
0 / 0 | 0 / 0 |
Guided end to end: final latent / pixels (1024×1024, 4 steps, true_cfg 4.0, negative prompt) |
0 / 0 | 0 / 0 |
End to end, seed 42 (1024×1024, 8 steps, true_cfg 1.0; pixels) |
0 | 0 |
The end-to-end prompt was "a photograph of an astronaut riding a horse" with initial noise
seed=1234; the guided run used the negative prompt "blurry". The model card uses 40 steps at
2048×2048. The reduced step counts and 1024×1024 size used here only shorten the CPU run.
The header-only tensor check covered text_encoder, transformer and vae: tensor counts
750 / 297 / 238, with missing, extra and shape-mismatched tensors all 0. Understanding-only
text-encoder tensors were skipped by an explicit allow-list. Record: tensor_header_check,
validation.json.
To fit the 96 GB machine, each native/reference component pair was loaded and freed one at a time. The full validation run took 42 minutes.
Janus-Pro¶
Validated on CPU on October 5, 2026, in the same environment as above. janus-pro-1b and
janus-pro-7b were loaded strictly and compared with transformers 5.18.0
JanusForConditionalGeneration.generate(generation_mode="image"). The observed difference in
every numeric comparison is exactly 0: on CPU in float32, the local implementation produces
bit-identical image tokens and pixels.
Comparisons (CPU, float32)¶
| Comparison | Checkpoint | Largest observed difference | Tolerance |
|---|---|---|---|
| DeepSeek LLM tokenizer (fixed prompts, seeded fuzz and every code point; processor prompt ids) | janus-pro-1b | identical | exact |
| Language model (prompt prefill and 8 decode steps with the static cache) | janus-pro-1b | 0 | 0 |
| LlamaGen VQ-16 decode | janus-pro-1b | 0 | 0 |
| End to end: 576 image tokens / pixels (1 image, seeded sampling, guidance 5) | janus-pro-1b | identical / identical | exact / exact |
| End to end: 576 image tokens / pixels (batch of 2, seeded sampling, guidance 5) | janus-pro-1b | identical / identical | exact / exact |
| End to end, seed 42 (pixels) | janus-pro-1b | identical | exact |
| DeepSeek LLM tokenizer (13 fixed prompts + 3,000 fuzz + every code point: 3,336,096 strings; 0 mismatches) | janus-pro-7b | identical | exact |
| Language model (prompt prefill and 8 decode steps with the static cache) | janus-pro-7b | 0 | 0 |
| LlamaGen VQ-16 decode | janus-pro-7b | 0 | 0 |
| End to end: 576 image tokens / pixels (1 image, seeded sampling, guidance 5) | janus-pro-7b | identical / identical | exact / exact |
| End to end: 576 image tokens / pixels (batch of 2, seeded sampling, guidance 5) | janus-pro-7b | identical / identical | exact / exact |
| End to end, seed 42 (pixels) | janus-pro-7b | identical | exact |
The end-to-end prompt was "a photograph of an astronaut riding a horse". Both the transformers
reference and imagehub use seeded sampling, guidance 5, temperature 1.0 and 576 autoregressive
image tokens. The header checks covered 964 tensors for janus-pro-1b and 1018 tensors for
janus-pro-7b. Understanding-only tensors are skipped by an explicit allow-list; missing, extra
and shape-mismatched generation tensors are all 0. Record: tensor_header_check,
validation.json.
Checkpoint provenance. The catalog uses the transformers conversions
deepseek-community/Janus-Pro-1B and deepseek-community/Janus-Pro-7B (provenance
community-conversion) because the authors' repositories ship weights for the original janus
package. For 1B, all 819 tensors used for image generation are byte-identical to the authors'
deepseek-ai/Janus-Pro-1B file; the 145 tensors that differ belong to the understanding-only
vision_model. For 7B, 873 of 1018 converted tensors are byte-identical to the authors'
deepseek-ai/Janus-Pro-7B shards; the 145 that differ are the understanding-only vision_model.
The 7B checkpoint is approximately 28 GB per float32 copy. To fit the 96 GB machine, one reference and one native copy were loaded at a time. The full 7B validation run took 68 minutes.
BAGEL¶
The bagel-7b-mot suite targets the author checkpoint at revision 5019f57 and the official
repository at a2fa77d. The published-header checks cover 1,223 bfloat16 EMA tensors and 244
float32 autoencoder tensors. Random-weight parity exercises both routes of the Qwen2
Mixture-of-Transformers exactly, and tokenizer comparisons cover ordinary Unicode and BAGEL's
image boundary tokens.
The real-checkpoint suite loads the official InterleaveInferencer / Bagel.generate_image
result in a subprocess before loading the native model. All 6 real tests passed, and the observed
difference in every recorded comparison is exactly 0 (0 means exact). The published header
check covers 1,467 tensors with 0 missing, extra and shape-mismatched entries
(header_bagel-7b-mot); the tokenizer produces identical ids on three fixed prompts
(bagel_tokenizer); the prompt embedding, one flow step and the VAE decode are exact
(bagel_text_encoder, bagel_denoiser, bagel_vae_decode). The two end-to-end records are
e2e_bagel-7b-mot at seed 0 and seed_e2e_bagel-7b-mot at seed 42, each shortened to 64×64 and
two timesteps, and both produce uint8 images identical to the reference. The published default
remains 1024×1024 and 50 timesteps. The real-checkpoint run took 15 minutes on an M2 Max (CPU).
Stable Diffusion 3 and 3.5¶
Validated on CPU in the same environment as above against diffusers 0.40.0
StableDiffusion3Pipeline and transformers 5.18.0. sd3-medium, sd3.5-medium, sd3.5-large
and sd3.5-large-turbo were loaded strictly. Every numeric comparison was exactly 0, so the
native implementation produced bit-identical results to the reference in CPU float32.
Comparisons (CPU, float32)¶
| Comparison | Checkpoint | Largest observed difference | Tolerance |
|---|---|---|---|
| Three tokenizers (3 fixed prompts) | sd3-medium | identical | exact |
| CLIP-L / CLIP-G / T5-XXL text encoders | sd3-medium | 0 / 0 / 0 | 0 |
| 24-block MMDiT without q/k normalization, single-step calls (1024×1024, t = 1000 and 500) | sd3-medium | 0 | 0 |
| VAE decode / encode | sd3-medium | 0 / 0 | 0 |
| Flow-match Euler timesteps / sigmas (28 steps) | sd3-medium | 0 / 0 | 0 |
| End to end: final latent / pixels (1024×1024, 12 steps, guidance 7.0) | sd3-medium | 0 / 0 | 0 / 0 |
| End to end, seed 42 with the same settings (pixels) | sd3-medium | 0 | 0 |
| Three tokenizers (3 fixed prompts) | sd3.5-medium | identical | exact |
| CLIP-L / CLIP-G / T5-XXL text encoders | sd3.5-medium | 0 / 0 / 0 | 0 |
| MMDiT-X single-step calls (1024×1024, t = 1000 and 500) | sd3.5-medium | 0 | 0 |
| VAE decode / encode | sd3.5-medium | 0 / 0 | 0 |
| Flow-match Euler timesteps / sigmas (40 steps) | sd3.5-medium | 0 / 0 | 0 |
| End to end: final latent / pixels (1024×1024, 12 steps, guidance 4.5) | sd3.5-medium | 0 / 0 | 0 / 0 |
| End to end, seed 42 with the same settings (pixels) | sd3.5-medium | 0 | 0 |
| Three tokenizers (3 fixed prompts) | sd3.5-large | identical | exact |
| CLIP-L / CLIP-G / T5-XXL text encoders | sd3.5-large | 0 / 0 / 0 | 0 |
| MMDiT single-step calls (1024×1024, t = 1000 and 500) | sd3.5-large | 0 | 0 |
| VAE decode / encode | sd3.5-large | 0 / 0 | 0 |
| Flow-match Euler timesteps / sigmas (28 steps) | sd3.5-large | 0 / 0 | 0 |
| End to end: final latent / pixels (1024×1024, 8 steps, guidance 3.5) | sd3.5-large | 0 / 0 | 0 / 0 |
| End to end, seed 42 with the same settings (pixels) | sd3.5-large | 0 | 0 |
| Three tokenizers (3 fixed prompts) | sd3.5-large-turbo | identical | exact |
| CLIP-L / CLIP-G / T5-XXL text encoders | sd3.5-large-turbo | 0 / 0 / 0 | 0 |
| MMDiT single-step calls (1024×1024, t = 1000 and 500) | sd3.5-large-turbo | 0 | 0 |
| VAE decode / encode | sd3.5-large-turbo | 0 / 0 | 0 |
| Flow-match Euler timesteps / sigmas (4 steps) | sd3.5-large-turbo | 0 / 0 | 0 |
| End to end: final latent / pixels (1024×1024, 4 steps, guidance 0.0) | sd3.5-large-turbo | 0 / 0 | 0 / 0 |
| Guided end to end: final latent / pixels (1024×1024, 4 steps, guidance 3.5, negative prompt) | sd3.5-large-turbo | 0 / 0 | 0 / 0 |
| End to end, seed 42 with the defaults (pixels) | sd3.5-large-turbo | 0 | 0 |
For all four checkpoints, all three tokenizers matched. The T5 tokenizer additionally passed seeded
fuzzing and an every-code-point check against the snapshot's tokenizer_3.
The end-to-end prompt was "a photograph of an astronaut riding a horse" with initial noise
seed=1234. The guidance values are the model-card settings. sd3-medium used 12 steps instead of
its 28-step default, sd3.5-medium used 12 steps instead of its 40-step default, and large used 8
instead of its 28-step default; these reductions only shorten the CPU validation run. Large-turbo
used its defaults of 4 steps and guidance 0.0, with no reduction. Its guided comparison used
guidance 3.5 and the negative prompt "blurry". The seed 42 comparisons used the same size, step
count and guidance.
The header-only checks covered text_encoder, text_encoder_2, text_encoder_3, transformer
and vae. sd3-medium had 197 / 517 / 219 / 683 / 244 tensors; sd3.5-medium had
197 / 517 / 219 / 909 / 244; large had 197 / 517 / 219 / 1227 / 244; large-turbo had
197 / 517 / 219 / 1227 / 244. Missing, extra and shape-mismatched tensors were all 0. Record:
tensor_header_check, validation.json.
To fit the 96 GB machine, each native/reference component pair was loaded and freed one at a time.
For end-to-end validation, the reference pipeline ran first and was freed before the native
pipeline ran. The sd3-medium run took 13 minutes, the sd3.5-medium run took 16 minutes and the
large run took 27 minutes. The large-turbo run took 17 minutes.
Sana¶
Validated on CPU in the same environment as above. sana-1600m-1024 and sana-600m-1024 are
loaded strictly from Efficient-Large-Model/Sana_1600M_1024px_diffusers at revision ac0da2f and
Efficient-Large-Model/Sana_600M_1024px_diffusers at revision 28f3af7. Every comparison below is
exactly 0 (tolerance 0; identical bytes for images). The real-checkpoint run for both weight sets takes about 2 h 15 min on an M2 Max (CPU).
Comparisons (CPU, float32, both weight sets)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| Tensor header check: text encoder, transformer, DC-AE (names and shapes) | 0 missing, extra or mismatched | 0 |
Gemma tokenizer ids / attention masks (prompt with the complex human instruction, and "") |
identical / identical | exact / exact |
SanaPipeline.encode_prompt embeddings and masks (prompt and negative) |
0 | 0 |
| Transformer single step (batch 2, 32×32 latents, two timesteps, real conditioning) | 0 | 0 |
| DC-AE decode (32×32 latents, scaled as the pipeline does) | 0 | 0 |
| End to end: final latent / pixels (1024×1024, 20 steps, guidance 4.5, seed 0) | 0 / 0 | 0 / 0 |
| End to end, seed 1234, 1360×768 request (binning, resize and crop back; pixels) | 0 | 0 |
The end-to-end prompt was "a photograph of an astronaut riding a horse". The reference text encoder
was loaded exactly as SanaPipeline.from_pretrained(..., torch_dtype=torch.float32) loads it. The
non-square seed test exercises resolution binning and the resize and centre crop of the decoded
image back to the requested size.
Random-weight parity covered the Gemma-2 text model, the transformer and DC-AE with tiny and with the
published configurations at small spatial sizes, and tiny-snapshot end-to-end parity covered binning
on and off, guidance on and off and non-square sizes. The Gemma tokenizer is additionally covered by
fixed-prompt and seeded fuzz tests. The scheduler is checked against diffusers with flow sigmas and
flow_prediction.
The reference and native components were loaded one model id at a time; the float32 reference needs about 6.4 GB for the transformer and 10.5 GB for Gemma-2.
Sana-Sprint 1.6B and Sana 1.5 4.8B¶
SanaSprintPipeline and the SCM scheduler are validated for sana-sprint-1.6b-1024 (revision
19683c5), and SanaPipeline with qk normalization for sana-1.5-4.8b-1024 (revision
9468102); both were loaded strictly and compared with diffusers 0.40.0 on CPU in float32. All
19 real tests passed and the observed difference in every recorded comparison is exactly 0
(0 means exact). The real-checkpoint run took 240 minutes on an M2 Max (CPU).
| Comparison | Model | Largest observed difference | Tolerance |
|---|---|---|---|
| Tensor header check: text encoder, transformer, DC-AE (288 / 480 / 375 tensors) | sana-sprint-1.6b-1024 |
0 missing, extra or mismatched | exact |
| Tensor header check: text encoder, transformer, DC-AE (288 / 1396 / 375 tensors) | sana-1.5-4.8b-1024 |
0 missing, extra or mismatched | exact |
| Gemma tokenizer ids / attention masks (prompt with the complex human instruction, 506 tokens, and a 300-token prompt) | sana-sprint-1.6b-1024 |
identical / identical | exact / exact |
| Gemma tokenizer ids / attention masks (prompt with the complex human instruction, 506 tokens, and a 300-token prompt) | sana-1.5-4.8b-1024 |
identical / identical | exact / exact |
SanaPipeline.encode_prompt prompt embeddings / masks |
sana-sprint-1.6b-1024 |
0 / identical | 0 / exact |
SanaPipeline.encode_prompt prompt embeddings / masks |
sana-1.5-4.8b-1024 |
0 / identical | 0 / exact |
SanaPipeline.encode_prompt negative embeddings |
sana-1.5-4.8b-1024 |
0 | 0 |
| Transformer single step (batch 2, 32×32 latents, timesteps 800 and 300, real conditioning) | sana-sprint-1.6b-1024 |
0 | 0 |
| Transformer single step (batch 2, 32×32 latents, timesteps 800 and 300, real conditioning) | sana-1.5-4.8b-1024 |
0 | 0 |
| DC-AE decode (32×32 latents, scaled as the pipeline does) | sana-sprint-1.6b-1024 |
0 | 0 |
| DC-AE decode (32×32 latents, scaled as the pipeline does) | sana-1.5-4.8b-1024 |
0 | 0 |
| End to end: final latent / pixels (1024×1024, 2 steps, guidance 4.5, seed 0) | sana-sprint-1.6b-1024 |
0 / 0 | 0 / 0 |
| End to end: final latent / pixels (1024×1024, 20 steps, guidance 4.5, seed 0) | sana-1.5-4.8b-1024 |
0 / 0 | 0 / 0 |
| End to end, seed 1234, 1360×768 request (binning, resize and crop back; pixels) | sana-sprint-1.6b-1024 |
0 | 0 |
| End to end, seed 1234, 1360×768 request (binning, resize and crop back; pixels) | sana-1.5-4.8b-1024 |
0 | 0 |
The end-to-end prompt was "a photograph of an astronaut riding a horse" with initial noise
seed=0. Sprint's default run uses 2 steps and embeds the guidance in the transformer, so it has
no negative text-encoder branch; the 4.8B model's default run uses 20 steps. Unit tests still
establish exact parity for the SCM scheduler and the guidance-aware transformer against diffusers
0.40.0, a tiny Sprint snapshot matches SanaSprintPipeline on CPU float32, and the qk-norm
transformer matches the reference in random-weight parity tests. The checked-in header fixtures
for both models recorded 0 missing, extra or shape-mismatched tensors as well
(fixture_header_sana-sprint-1.6b-1024, fixture_header_sana-1.5-4.8b-1024,
validation-measurements.json).
HiDream-I1¶
Training (November 7, 2026)¶
HiDream-I1 Dev and Full LoRA training is checked on CPU float32 against diffusers 0.40.0 and
PEFT 0.21.2. The tiny random snapshot compares the script defaults, resize/crop/normalize image
transform, four tokenizer id streams and T5/Llama sequence masks. For every weighting scheme it
compares posterior noise, noise, timestep and sigma draws, the loss and every LoRA gradient with
exact equality. The reference transformer is loaded with force_inference_output=True, then put
in training mode, so the test covers the inference MoE and unpatchify route used by the script.
Random single, stacked (0.8 / 0.5) and fused PEFT adapters, plus a 3-step trained adapter, are
loaded into diffusers from the same explicit starting latents; their final latents and uint8 pixels
match the native backend exactly.
| Comparison | Model | Largest observed difference | Tolerance |
|---|---|---|---|
| Default arguments: weighting, logit mean / std, mode scale, sequence length 128, rank / alpha and image-stream targets | tiny HiDream-I1 | identical | identical |
| Image transform; CLIP, T5 and Llama tokenizer ids and masks | tiny HiDream-I1 | identical | identical |
| First-step posterior noise, noise, timestep, sigma, loss and every LoRA gradient, all weighting schemes | tiny HiDream-I1 | 0 | 0 |
| Gradient checkpointing over double- and single-stream transformer blocks | tiny HiDream-I1 | identical | identical |
Validation image vs pipeline(..., lora=adapter) |
tiny HiDream-I1 | exact pixels | exact |
| Random PEFT adapter: single, stacked 0.8 / 0.5 and fused; registered image-stream targets, changed diffusers output, final latents and uint8 pixels | tiny HiDream-I1 | 0 / 0 | 0 / 0 |
| 3-step trained adapter loaded in diffusers; registered image-stream targets, changed diffusers output, final latents and uint8 pixels | tiny HiDream-I1 | 0 / 0 | 0 / 0 |
hidream-i1-fast is distilled and is intentionally inference_only; hidream-o1-image has a
different architecture and is unchanged.
Lumina-Image 2.0¶
Training (November 7, 2026)¶
Lumina 2 LoRA training is checked on CPU float32 against diffusers 0.40.0. The reference tests
compare the script defaults for the objective, collator and adapter; the resize/crop/normalize
transform; and Gemma token ids and attention masks with the pipeline's default system prompt.
For every weighting scheme, they compare posterior, noise, timestep and sigma draws, loss and every
adapter gradient. Random transformer adapters (to_k, to_q, to_v, to_out.0) are tested
single, stacked at 0.8 / 0.5 and fused; each first proves that diffusers registered the intended
targets and changed its output, then compares final latents and uint8 pixels. A three-step trained
adapter is loaded through load_lora_weights and checked the same way. Every tensor comparison uses
exact equality.
lumina-image-2.0 is loaded strictly from Alpha-VLLM/Lumina-Image-2.0 at revision 53504ab and
compared with diffusers 0.40.0 Lumina2Pipeline on CPU in float32. Every comparison below measured
exactly 0. The real-checkpoint run takes about 50 minutes on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Measured difference |
|---|---|
| Tensor header check: text encoder, transformer, autoencoder (names and shapes) | 0 missing, extra or mismatched |
Gemma tokenizer ids / attention masks (system prompt and prompt, a truncated prompt, and "") |
exact / exact |
Lumina2Pipeline.encode_prompt embeddings and masks (prompt and negative) |
0 |
| Transformer single step (batch 2, 64×64 latents, two timesteps, real conditioning) | 0 |
| Autoencoder decode (64×64 latents, scaled and shifted as the pipeline does) | 0 |
| End to end: final latent / pixels (1024×1024, 30 steps, guidance 4.0, seed 0) | 0 / 0 |
| End to end, seed 1234, 1360×768 request (pixels) | 0 |
The reference text encoder is loaded exactly as Lumina2Pipeline.from_pretrained(...,
torch_dtype=torch.float32) loads it, and its second-to-last hidden state is the conditioning. The
end-to-end reference run decodes its own latents with the pipeline's autoencoder and image processor.
Random-weight parity covers the transformer with tiny and published configurations and the Gemma-2
text model, including every entry of the hidden-state tuple. Tiny-snapshot end-to-end parity covers
the default settings, cfg_trunc_ratio below 1, cfg_normalization off, guidance 1.0, non-square
sizes, a custom system prompt and negative prompt, and several seeds.
The end to end first differed by about 3e-4 in the latents. The cause was the memory layout of the
loaded weights: when diffusers loads a sharded checkpoint without accelerate, it keeps the shard that
holds the model's first tensor at its file offsets and copies the other shards into aligned buffers,
and the CPU kernels sum in a layout-dependent order. The transformer now loads its shards the same
way (keep_first_shard_mapped).
Lumina-Next¶
lumina-next is loaded strictly from Alpha-VLLM/Lumina-Next-SFT-diffusers at revision 0ee5ec9 and
is compared with diffusers 0.40.0 LuminaPipeline on CPU in float32 (the weights are stored in
bfloat16 for the text encoder and the transformer, and in float32 for the autoencoder; the reference
upcasts the bfloat16 tensors exactly as we do). The observed difference in every comparison is
exactly 0: on CPU in float32, the local implementation produces a bit-identical result to the
reference. All 11 real-checkpoint tests in tests/reference/test_ref_lumina_next.py passed. The
real-checkpoint run took 42 minutes on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Measured difference |
|---|---|
| Tensor header check: text encoder, transformer, autoencoder (names and shapes) | 0 missing, extra or mismatched (164 / 639 / 248 tensors) |
| Gemma tokenizer ids and masks (prompt padding to a multiple of eight, negative prompt padding to the prompt length, fixed prompts and a seeded fuzz test) | exact (0 mismatches over 15 fixed prompts and 600 seeded fuzz prompts) |
LuminaPipeline.encode_prompt embeddings and masks (prompt and negative) |
0 |
| Transformer single step (batch 2, 64×64 latents, two timesteps, real conditioning; three rotary scalings, proportional attention on and off) | 0 |
| Autoencoder decode (64×64 latents scaled by the pipeline's factor) | 0 |
| End to end: final latent / pixels (1024×1024, 30 steps, guidance 4.0, seed 0) | 0 / 0 |
End to end with the time-aware watershed inside the schedule (1280×768, 8 steps, scaling_watershed 0.3): latent / pixels |
0 / 0 |
| End to end, seed 1234, 1360×768 request, 12 steps (pixels) | 0 |
The reference text encoder is loaded exactly as LuminaPipeline.from_pretrained(...,
torch_dtype=torch.float32) loads it, and its second-to-last hidden state is the conditioning. The
end-to-end reference run decodes its own latents with the pipeline's autoencoder and image
processor. The reference runs with clean_caption=False, which is what the pipeline falls back to
when beautifulsoup4 and ftfy are not installed. The seed test and the watershed test use fewer
steps than the default to keep the run short.
Without weights, the following run and pass with exactly 0 difference: random-weight parity
of the Gemma text model (tiny and with the published attention geometry, left and right padding, no
padding, every entry of the hidden-state tuple), of the transformer (tiny and the published
configuration with shortened depth, proportional attention on and off, three rotary scalings, with
and without query/key normalization) and of the rotary tables; tiny-snapshot end-to-end parity
(default settings, guidance 1.0, negative prompts including one longer than the prompt,
non-square sizes, five scaling_watershed values, proportional attention off, several seeds, one
step); the scheduler timesteps and sigmas; and the tokenizer on a tiny vocabulary.
Lumina-DiMOO¶
lumina-dimoo is loaded strictly from Alpha-VLLM/Lumina-DiMOO revision f014549 and compared
on CPU in float32 with the official LLaDAForMultiModalGeneration and generate_image at code
revision 81344e6. The published LLaDA weights are BF16 and are upcast on both sides; the VQ
weights are float32. All 11 real-checkpoint tests passed, and the observed difference in every
recorded comparison is exactly 0 (0 means exact): on CPU in float32, the local implementation
produces a bit-identical image to the one the official implementation produces. The real-checkpoint
run took 3 minutes on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Measured difference |
|---|---|
| Tensor headers: LLaDA / VQ model (291 / 277 tensors) | 0 missing, extra or mismatched |
| Tokenizer ids (4 formatted prompts and 200 seeded fuzz strings; Unicode and multimodal specials) | 0 mismatches |
| Random-weight LLaDA logits (published geometry and random weights) | 0 |
| Tiny end-to-end token trajectories (guidance 0 and 2) | identical |
| Real LLaDA logits | 0 |
| Real VQ decode | 0 |
| End to end (32x32, two steps, guidance 4.0, seed 0) | identical uint8 image (passed) |
| Separate public-pipeline seed run (seed 42) | identical uint8 image (passed) |
The real end-to-end cases use 32x32 output and two steps rather than the 1024x1024, 64-step
defaults because the 8B transformer evaluates the full image-token sequence on every step. Records:
header_lumina-dimoo, lumina_dimoo_tokenizer, lumina_dimoo_tokenizer_fuzz,
lumina_dimoo_llada_published_geometry, lumina_dimoo_llada_random_weights,
lumina_dimoo_real_llada, lumina_dimoo_real_vq, lumina_dimoo_tiny_e2e_g0,
lumina_dimoo_tiny_e2e_g2, e2e_lumina-dimoo and seed_e2e_lumina-dimoo,
validation-measurements.json.
aMUSEd¶
amused-256 and amused-512 are loaded strictly from amused/amused-256 at revision 09b6259 and
amused/amused-512 at revision 6f3f309, and are compared with diffusers 0.40.0 AmusedPipeline on
CPU in float32. The real-checkpoint tests (tests/reference/test_ref_amused.py) assert the exact
equalities below for both models. All 14 real-checkpoint tests (7 per model) passed, and every
comparison matched exactly. The real-checkpoint run for both models took about 5 minutes on an
Apple M2 Max (CPU). The tiny-snapshot and random-weight comparisons need no downloaded weights and
pass with exactly 0 difference.
Comparisons (CPU, float32)¶
| Comparison | Measured (amused-256, amused-512) |
|---|---|
| Tensor header check: text encoder, transformer, VQGAN (names and shapes) | amused-256: 197, 474 and 277 tensors; amused-512: 198, 478 and 277 tensors; 0 missing, extra or mismatched in both |
CLIP tokenizer ids (prompt, "", punctuation and case, a truncated prompt) |
all 4 cases exact for both |
Text encoder: projected pooled embedding and second-to-last hidden state (prompt and "") |
max abs 0, max rel 0 for both |
| One transformer step on a partly masked token grid with micro-conditioning | max abs 0, max rel 0 for both |
| VQGAN decode of fixed token ids (codebook lookup, group normalization) | max abs 0, max rel 0 for both |
| End to end at the model defaults (12 steps, guidance 10, seed 0): final token ids and uint8 image | token ids equal, max pixel difference 0 (256×256 and 512×512) |
| Seeded pipeline run at a different, non-square size (seed 1234): uint8 image | max pixel difference 0 (amused-256 at 320×192, amused-512 at 640×384) |
Random-weight comparisons, which pass with exactly 0 difference: the U-ViT with a tiny configuration (with and without bias, with and without the downsampling and upsampling of the 512 model) and with the published 256 configuration at full size; the scheduler trajectory with seeded generators (cosine and linear schedules, number and pair temperatures, the generator-free random state); and the VQGAN with group normalization, with and without the attention block and the codebook lookup, and with the published configuration. Tiny-snapshot end-to-end runs compare the token ids and the image bytes for both transformer variants, with the defaults, guidance 1.0 (no guidance), a negative prompt, micro-conditioning and temperature settings, a larger grid, and several seeds.
Kolors¶
kolors is loaded strictly from Kwai-Kolors/Kolors-diffusers at revision 7e091c7 and compared
with diffusers 0.40.0 KolorsPipeline on CPU in float32, on the float32 upcast of the published
float16 weights (only float16 weights are published). Every comparison below measured exactly 0.
The real-checkpoint run takes about 1 h 20 min on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Measured difference |
|---|---|
| Tensor header check: text encoder (200 tensors), U-Net (1682), autoencoder (248) (names and shapes) | 0 missing, extra or mismatched |
| ChatGLM tokenizer ids / attention masks (11 prompts, a 2000-prompt fuzz, 3 snapshot cases) | identical / identical |
KolorsPipeline.encode_prompt embeddings and pooled embeddings (prompt and negative) |
0 |
| U-Net single step (real conditioning) | 0 |
| Autoencoder decode | 0 |
| End to end: final latent / pixels (1024×1024, 50 steps, guidance 5.0, seed 0) | 0 / 0 |
| End to end, seed 1234, 1360×768 request (pixels) | 0 |
The reference pipeline was loaded with variant="fp16" and torch_dtype=torch.float32, so both
sides compute on the same upcast weights.
Random-weight parity covers the ChatGLM encoder in five configurations (the published layout,
post-residual layer norm, multi-head attention without the final norm, linear biases, and a wider
model), on left-padded batches. Tiny-snapshot end-to-end parity covers four cases (the default
settings, guidance 1.0, a non-square size with a negative prompt, and micro-conditioning with
explicit original size, crop and target size) with two sets of random weights each, plus a check of
the latents. The Euler scheduler ignores the configuration keys that diffusers' Euler scheduler does
not read (thresholding, dynamic_thresholding_ratio, clip_sample_range), which this release's
configuration contains.
DeepFloyd IF¶
Validated on CPU in the same environment as above. deepfloyd-if-i-xl combines the pinned gated
IF-I-XL and IF-II-L repositories with diffusers 0.40.0 IFPipeline and IFSuperResolutionPipeline
on CPU in float32. The published weights are float32. Unit and opt-in reference coverage exercises
the IF U-Net additions, dynamic thresholding, image noising, guidance on and off, and the combined
256×256 path. All 7 real-checkpoint tests passed, and the observed difference in every recorded
comparison is exactly 0 (0 means exact): on CPU in float32, the local implementation produces
a bit-identical image to the one diffusers produces. The real-checkpoint run took 4 minutes on an
M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| Tensor header check: stage I U-Net (775 tensors), stage II U-Net (939) (names and shapes) | 0 missing, extra or mismatched | exact |
| T5 tokenizer token ids / attention masks (1 prompt at 77 tokens) | identical / identical | exact |
| Shared T5 text encoder last hidden state (the same encoder conditions both stages) | 0 | 0 |
| Stage I U-Net single step (64×64, real conditioning, timestep 999) | 0 | 0 |
| Stage II U-Net single step (256×256, real conditioning, noise level 250, timestep 999) | 0 | 0 |
| End to end: final pixels (256×256, 4 steps per stage, guidance 7.0 and 4.0, seed 0) | 0 | 0 |
| End to end, seed 42 through the public pipeline | passed | exact |
The end-to-end prompt was "a quiet landscape". Both runs use 4 steps per stage instead of the
50-step defaults only to shorten the CPU run; guidance stays at the pipeline defaults 7.0 and 4.0
with noise_level 250. Stage III, safety checking and watermarking are excluded. Records:
header_deepfloyd_if_stage_i, header_deepfloyd_if_stage_ii,
deepfloyd_if_tokenizer, deepfloyd_if_t5, deepfloyd_if_stage_i_unet,
deepfloyd_if_stage_ii_unet, e2e_deepfloyd-if-i-xl and seed_e2e_deepfloyd-if-i-xl,
validation-measurements.json; components_deepfloyd_if,
validation.json.
Kandinsky 3¶
kandinsky-3 is loaded strictly from kandinsky-community/kandinsky-3 at revision bf79e6c and
compared with diffusers 0.40.0 Kandinsky3Pipeline on CPU in float32, on the float32 upcast of the
published float16 weights (only float16 weights are published). The observed difference in every
comparison is exactly 0: on CPU in float32, the local implementation produces a bit-identical
result to the reference (identical bytes for images). The real-checkpoint run of 7 tests took about
4 minutes on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference |
|---|---|
| Tensor header check: text encoder (291 tensors), U-Net (1071), MoVQ (431) (names and shapes) | 0 missing, extra or mismatched |
| T5 tokenizer ids / attention masks (6 prompts, 128 tokens) | identical / identical |
Kandinsky3Pipeline.encode_prompt embeddings and masks (no negative prompt, a negative prompt, no guidance) |
0 |
| U-Net single step (batch 2, 64×64 latents, masked context, three timesteps) | 0 |
| MoVQ decode (64×64 latents, no quantization) | 0 |
| End to end: final latent / pixels (1024×1024, 4 steps, guidance 3.0, seed 0) | 0 / 0 |
| End to end, seed 1234, 768×512 with a negative prompt, 3 steps (pixels) | 0 |
The text encoder needs about 35 GB in float32, so the reference conditioning is computed, saved to
disk and freed before the native text encoder is loaded, and the reference denoiser is run without
a text encoder on the saved conditioning. The end-to-end runs use 4 and 3 steps instead of 25 to
keep the run short; the sampler is stochastic at every step, so each step exercises the generator.
The reference pipeline is loaded with variant="fp16" and torch_dtype=torch.float32.
Random-weight parity covers the T5 encoder with the gated SiLU feed-forward (a small model and the Flan-UL2 widths with two layers, with and without a padding mask), the MoVQ decoder at the published configuration, the U-Net (small and published widths, masked, unmasked and fully masked rows) and the squared-cosine DDPM schedule. Tiny-snapshot end-to-end parity covers seven cases (the default settings, guidance 1.0, a non-square size rounded up to 64 pixels with a negative prompt, an empty negative prompt, two images, a prompt longer than the context and a single step) with two sets of random weights each, plus checks of the latents and of the cut context.
Kandinsky 5¶
kandinsky-5-t2i-lite is loaded strictly from
kandinskylab/Kandinsky-5.0-T2I-Lite-sft-Diffusers at revision 25da1e82e6adefdbc83c02a71f762287c5e471a4
and compared with diffusers 0.40.0 Kandinsky5T2IPipeline on CPU in float32, on the float32 upcast
of the published bfloat16 weights. The observed difference in every comparison is exactly 0: on
CPU in float32, the local implementation produces a bit-identical result to the reference
(identical bytes for images). The real-checkpoint tests (tests/reference/test_ref_kandinsky5.py)
check the tensor headers of all components, the Qwen and CLIP tokenizers, the Qwen text-encoder
outputs and CLIP pooled embedding the pipeline uses, one transformer step with real conditioning,
the VAE decode, and end-to-end generation with a shortened run and a non-square seed test. The
real-checkpoint run of 6 tests took about 27 minutes on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference |
|---|---|
| Tensor header check: Qwen text encoder (729 tensors), CLIP text encoder (197), 3D transformer (1246), VAE (244) (names and shapes) | 0 missing, extra or mismatched |
| Qwen and CLIP tokenizer ids for the prompt | identical |
| Qwen text-encoder last hidden state (templated prompt) | 0 |
| CLIP pooled text embedding used by the pipeline | 0 |
| 3D transformer single step (128×128 latent grid, real conditioning, timestep 500) | 0 |
| VAE decode (128×128 latent grid, 16 channels) | 0 |
| End to end: final pixels (1024×1024, 4 steps, guidance 3.5, seed 0) | 0 |
| End to end, 640×1408, 4 steps, guidance 3.5, seed 7 (pixels) | 0 |
Random-weight parity covers the Kandinsky 5 3D transformer with a tiny and with the published configuration at small spatial sizes. Tiny-snapshot end-to-end parity covers the default settings, guidance on and off, non-square sizes, two seeds and several weight seeds, using a random snapshot in the published diffusers layout.
Meissonic¶
meissonic is pinned to MeissonFlow/Meissonic revision 7ee5068 and compared on CPU in
float32 with the official viiika/Meissonic src.pipeline.Pipeline at commit a9de06c.
Offline fixtures cover 389 text-encoder tensors, 867 transformer tensors, and 277 VQ-VAE tensors
with no missing, extra, or mismatched names or shapes. Random-weight tests compare the transformer
and scheduler exactly, and tiny whole-pipeline tests compare guidance off and on, a non-square
image, multiple seeds, final tokens, and uint8 pixels exactly.
The real-checkpoint suite is in tests/reference/test_ref_meissonic.py. All 6 real tests passed,
and the observed difference in every recorded comparison is exactly 0 (0 means exact): on CPU
in float32, the local implementation produces a bit-identical image to the one the official
pipeline produces. The recorded comparisons are the tensor header check (text_encoder,
transformer, vqvae; 389 / 867 / 277 tensors, with no missing, extra or shape-mismatched
tensors), the CLIP tokenizer token ids (4 prompts), the projected and penultimate CLIP text
encoder states, one transformer step (masked token grid at 1024×32, timestep 1), the VQ-VAE
decode, and separate public-pipeline runs at seeds 0 and 42. Both end-to-end cases use 2 steps at
1024×32 instead of the 48-step 1024×1024 default to keep the CPU validation short while
exercising the published transformer's compression path. The real-checkpoint run took less than a
minute on an M2 Max (CPU). Records: header_meissonic, meissonic_tokenizer,
meissonic_text_embeds, meissonic_text_hidden, meissonic_transformer_step,
meissonic_vq_decode, e2e_meissonic and seed_e2e_meissonic,
validation-measurements.json; tensor_header_check,
end_to_end and seed_end_to_end, validation.json.
AuraFlow¶
Validated on CPU in the same environment as above. auraflow-v0.3 is loaded strictly from
fal/AuraFlow-v0.3 at revision 2cd8588. Every comparison below is required to be exactly 0
(tolerance 0; identical bytes for images). The real-checkpoint run took about 3 h 41 min on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| Tensor header check: text encoder, transformer, VAE (names and shapes) | 0 missing, extra or mismatched | 0 |
Tokenizer ids / attention masks (prompt and "", 256 tokens) |
identical / identical | exact / exact |
AuraFlowPipeline.encode_prompt embeddings (prompt and negative) |
0 | 0 |
| Transformer single step (batch 2, 32×48 latents, two timesteps, real conditioning) | 0 | 0 |
| VAE decode (32×48 latents, scaled as the pipeline does) | 0 | 0 |
| End to end: final latent / pixels (1024×1024, 50 steps, guidance 3.5, seed 0) | 0 / 0 | 0 / 0 |
| End to end, seed 1234, 1360×768 (pixels) | 0 | 0 |
The end-to-end prompt was "a photograph of an astronaut riding a horse". The reference was loaded
exactly as AuraFlowPipeline.from_pretrained(..., torch_dtype=torch.float32) loads it, with the
default negative prompt.
Random-weight parity covered the UMT5 encoder and the transformer with tiny and with the published configuration at small spatial sizes, and tiny-snapshot end-to-end parity covered the default settings, guidance on and off, non-square sizes, a truncated prompt and several seeds. The tokenizer is additionally covered by fixed-prompt and seeded fuzz tests on a synthetic folder.
The float32 transformer needs about 27 GB, so the reference and native transformers were loaded one after the other and each was freed before the next.
LlamaGen¶
llamagen-t2i-xl-256 and llamagen-t2i-xl-512 are loaded strictly from peizesun/llamagen_t2i at
revision 276f5c5 (GPT, VQ-16) and google/flan-t5-xl at revision 7d6315d (text encoder). They are
compared on CPU in float32 with the official LlamaGen code (commit ce98ec4: GPT_XL, generate,
VQ_16) and transformers 5.18.0 for the Flan-T5 encoder and tokenizer, driven as
autoregressive/sample/sample_t2i.py drives them without its CUDA-only settings. The random
generator is seeded right before sampling. The real-checkpoint tests
(tests/reference/test_ref_llamagen.py) assert the exact equalities below for both models. All 63
tests passed for the two models in a single run of about 19 minutes on an M2 Max (CPU), with
exactly 0 difference in every comparison. The tiny-snapshot and random-weight comparisons need no
downloaded weights and pass with exactly 0 difference.
Comparisons (CPU, float32)¶
| Comparison | Tolerance |
|---|---|
| Tensor header check: GPT (258 tensors), VQ-16 (344) and Flan-T5 encoder (219 used, 339 skipped) (names and shapes), both models | 0 missing, extra or mismatched |
Tokenizer ids and attention masks (prompts, "", punctuation and case, a truncated prompt; 120 tokens) |
exact |
| Caption embeddings after the driver's left padding and masking | 0 |
| GPT prefill logits (conditional and unconditional batch, left-padded caption mask) | 0 |
| VQ-16 decode of fixed token ids | 0 |
End to end at the defaults (cfg_scale 7.5, top_k 1000, seed 0; 256×256 and 512×512): token ids and uint8 image |
identical (max pixel difference 0) |
Seeded pipeline run with other sampling settings (cfg_scale 4.0, top_k 500, top_p 0.95, temperature 0.9, seed 1234): uint8 image |
identical (max pixel difference 0) |
Random-weight comparisons, which pass with exactly 0 difference: the GPT with a tiny configuration
and with the published GPT-XL configuration (prefill and cached decoding) and the rotation table of
both sizes; the VQ decoder with a tiny and with the published configuration; and the sampling
functions (temperature, top-k and top-p filtering and the draws) with random logits and seeds.
Tiny-snapshot end-to-end runs compare the token ids and the image bytes with the defaults,
cfg_scale 1.0 (no guidance), top-k, top-p and temperature settings, a guidance interval and
several seeds. Images are converted to uint8 as torchvision.utils.save_image(normalize=True,
value_range=(-1, 1)) does, which a test checks against torchvision.
The prompt is lower-cased and stripped before tokenization; the reference's default caption
cleaning (ftfy, BeautifulSoup) is not implemented, and the comparisons use the script's
use_text_preprocessing=False behavior.
Infinity¶
Validated on CPU in the same environment as above. infinity-2b loads infinity_2b_reg.pth and
infinity_vae_d32reg.pth from FoundationVision/Infinity revision e710892, plus Flan-T5-XL
revision 7d6315d. The reference is the original Infinity, Infinity.autoregressive_infer_cfg
and bitwise VAE decoder at commit 9f9fcd1. Tests are in tests/reference/test_ref_infinity_2b.py;
the original real-checkpoint pipeline runs in a subprocess so its large modules are freed before
imagehub loads its copies.
All 8 real tests passed, and the observed difference in every recorded comparison is exactly 0
(0 means exact): on CPU in float32, the local implementation produces a bit-identical image to the
one the original implementation produces. The published transformer, VAE and Flan-T5-XL weights are
float32. The tensor headers match with 0 missing, extra and mismatched entries for the transformer
(694 tensors), the VAE archive (302 tensors) and the Flan-T5 encoder (219 tensors)
(header_infinity-2b); the tokenizer and text-encoder outputs are equal and the VAE decoder and the
first scale load strictly (components_infinity-2b); the random-weight self-attention,
compact cross-attention, transformer scale and VAE decoder comparisons all record max_abs 0.0
(infinity_self_attention_random_weights, infinity_cross_attention_random_weights,
infinity_transformer_random_weights, infinity_vae_decoder_random_weights); and both
end-to-end runs (256x256, seven scales, seed 0 and seed 42) pass (e2e_infinity-2b,
seed_e2e_infinity-2b). The real-checkpoint run took 1 minute on an M2 Max (CPU). Records:
validation-measurements.json.
Comparisons (CPU, float32)¶
| Comparison | Tolerance |
|---|---|
| Tensor headers: transformer (694 tensors), VAE archive (302 tensors; 33 training-only perceptual tensors and one deterministic mask excluded), Flan-T5 encoder | 0 missing, extra or mismatched |
| Flan-T5 tokenizer ids, attention mask and encoder output | exact |
| Random-weight self-attention and compact cross-attention | 0 |
| Random-weight bitwise VAE decoder | 0 |
| Strict real transformer, text encoder and VAE load; first scale exercised by end to end | exact |
| End to end, official seven-scale 256x256 schedule, guidance 3.0, temperature 0.5, top-k 900, top-p 0.97, seed 0 | identical uint8 image |
| Separate public-pipeline seed run with seed 42 and the same shortened schedule | identical uint8 image |
The 256x256 comparison is the smallest schedule published for Infinity-2B and is intentionally used instead of the 13-scale 1024x1024 default so a CPU reference run finishes in minutes. Unit tests additionally cover a random tiny snapshot, strict loading, sampling option validation, repeatability, request isolation and restoration of the caller's autograd mode.
HiDream-I1¶
hidream-i1-fast, hidream-i1-dev (revision 0fad2ea) and hidream-i1-full (revision 8ccbbfb) are the HiDream-I1 repositories plus
meta-llama/Llama-3.1-8B-Instruct at revision 0e9e39f. The real-checkpoint tests
(tests/reference/test_ref_hidream.py) compare against diffusers 0.40.0 and transformers 5.18.0 on
CPU in float32 and require exactly 0 everywhere: the tensor headers of all six components, the
tokenizers (CLIP, T5, Llama), the pooled CLIP, T5 and Llama layer outputs the pipeline uses, one
transformer step with real conditioning, the VAE decode, the scheduler, and end to end with a
shortened run (4 steps, seed 0) and a non-square seed-42 test. All 8 real-checkpoint tests of
hidream-i1-fast passed with exactly 0 difference in every comparison (see the table), and that run
took 21 minutes on an M2 Max (CPU).
The published CLIP encoders are float32, the T5 encoder and the VAE are bfloat16 for every
checkpoint, and the transformer is bfloat16 for hidream-i1-fast and hidream-i1-dev and float16 for
hidream-i1-full; both implementations upcast all of it to float32.
hidream-i1-dev and hidream-i1-full were validated with the same eight cases each. All 16
real-checkpoint tests passed, and the observed difference in every recorded comparison is exactly
0 (0 means exact): on CPU in float32, the local implementation produces a bit-identical image to
the one diffusers produces. The recorded comparisons are the tensor headers of the same six
components (197 / 517 / 219 / 291 / 1615 / 244 tensors), the tokenizers on the 6 prompts, the pooled
CLIP, T5 and Llama layer outputs, one transformer step at t = 1000 and 562.5, the VAE decode, the
scheduler timesteps and sigmas (flow-match LCM for Dev, UniPC for Full, 16 steps), and both
end-to-end runs (4 steps, seed 0, and seed 42). The Dev and Full run took 51 minutes on an M2 Max
(CPU).
Records: header_hidream-i1-dev, header_hidream-i1-full, tokenizers_hidream-i1-dev,
tokenizers_hidream-i1-full, hidream_pooled_clip, hidream_t5, hidream_llama_layers,
hidream_transformer_t0, hidream_transformer_t1, hidream_vae_decode,
hidream_scheduler_timesteps, hidream_scheduler_sigmas, hidream_e2e_latents,
e2e_hidream_i1_dev, e2e_hidream_i1_full, seed_e2e_hidream_i1_dev and
seed_e2e_hidream_i1_full, validation-measurements.json.
Comparisons (CPU, float32)¶
All three checkpoints recorded the same values in every comparison.
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
Tensor header check: text_encoder, text_encoder_2, text_encoder_3, text_encoder_4, transformer, vae (197 / 517 / 219 / 291 / 1615 / 244 tensors) |
0 missing / extra / mismatched | exact |
| Tokenizers (CLIP ×2, T5, Llama; 6 prompts) | identical | exact |
| Pooled CLIP embedding | 0 | 0 |
| T5 encoder outputs used by the pipeline | 0 | 0 |
| Llama layer outputs used by the pipeline | 0 | 0 |
| Transformer single step (t = 1000 and 562.5) | 0 | 0 |
| VAE decode | 0 | 0 |
| Scheduler timesteps / sigmas (16 steps; flow-match LCM for Fast and Dev, UniPC for Full) | 0 / 0 | 0 / 0 |
| End to end: final latent / pixels (4 steps, seed 0) | 0 / 0 | 0 / 0 |
| End to end, seed 42, non-square request (pixels) | 0 | 0 |
What has been checked with random weights: the Llama text encoder against transformers (grouped
key/value heads, llama3 rotary scaling, padding), the Llama 3 tokenizer against the reference
fast tokenizer on a synthetic vocabulary (and, once, on the Llama 3.1 tokenizer file from a public
mirror with fuzzed text), and the whole backend on a tiny snapshot in the published layout against HiDreamImagePipeline
(default settings, guidance with a negative prompt, non-square sizes, two images, two weight seeds):
identical latents and identical bytes.
HiDream-O1¶
hidream-o1-image was loaded strictly from HiDream-ai/HiDream-O1-Image at revision
0b0901d; the reference is the official implementation at commit 2c2d29f and, for the chat
template and token ids, transformers 5.18.0. All 5 real-checkpoint tests passed on CPU in
float32 with the published float32 weights, and the observed difference in every recorded
comparison is exactly 0 (0 means exact): on CPU in float32, the local implementation
produces a bit-identical image to the one the official implementation produces. The
real-checkpoint run took 1 minute on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| Tensor header check (759 tensors; names and shapes) | 0 missing, extra or mismatched | exact |
| Qwen2 chat template and token ids (3 prompts) and the 24-position sample layout | identical | exact |
| Denoiser single step (t = 0.4, 2 image patches, one call) | 0 (max abs and max rel) | 0 |
| End to end: final pixels (32×64, 2 steps, guidance 5.0, shift 3.0, seed 0) | 0 | 0 |
| End to end, seed 42 (32×64, 2 steps, guidance 5.0, shift 3.0; pixels) | 0 | 0 |
Both end-to-end runs use the prompt "A small red sailboat on a calm lake at sunrise." with
classifier-free guidance and the published defaults; two steps at 32×64 only shorten the CPU run
because the backend defaults to 50 steps at 2560×1440. Records: header_hidream-o1-image,
hidream_o1_tokenizer_positions, hidream_o1_denoiser_step, e2e_hidream-o1-image and
seed_e2e_hidream-o1-image,
validation-measurements.json.
Offline tests already check the published 759 tensor names and shapes and compare the new pixel
projection, timestep embedding, mixed causal/generation decoder, fixed-point rotary positions and
Flow-UniPC scheduler against the pinned source. A tiny full sampling test runs the official sampling
code in CPU float32 with classifier-free guidance on and off, two seeds and both 32×64 orientations;
the resulting uint8 images are exactly equal. The real-checkpoint suite
(tests/reference/test_ref_hidream_o1.py) skips unless
IMAGEHUB_SNAPSHOT_HIDREAM_O1_IMAGE points to a local snapshot and records the header,
tokenizer/position, denoiser-step, e2e_hidream-o1-image and separate seed-42
seed_e2e_hidream-o1-image comparisons. The official result is saved and freed before the native
model is loaded, and the end-to-end runs use two steps and 32×64 output to keep the 8B checkpoint
run bounded.
Reference-image conditioning is not implemented or validated yet.
OmniGen¶
omnigen-v1 is loaded strictly from Shitao/OmniGen-v1-diffusers at revision 016e2f6 and is compared
with diffusers 0.40.0 OmniGenPipeline on CPU in float32. The published transformer is stored in
bfloat16 and converted to float32 on load by both implementations; the autoencoder is float32. The
real-checkpoint tests are in tests/reference/test_ref_omnigen.py. All 12 of them passed, and
every recorded comparison is exactly 0: on CPU in float32, the local implementation produces a
bit-identical result to the reference. The end-to-end runs use shortened step counts
because a 1024×1024, 50-step run takes hours on CPU. The real-checkpoint run took 18 minutes on
an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Measured difference | Tolerance |
|---|---|---|
| Tensor header check: transformer (275 tensors), autoencoder (248 tensors) (names and shapes) | 0 missing, extra or mismatched | exact |
Tokenizer ids (instruction prompt, edit prompt, long prompt, ""; 4 cases) and byte equality of the tokenizer files with the fixtures |
identical | exact |
| Transformer single step (timesteps 0.0, 0.31 and 0.87, with and without an input image, real conditioning; 6 calls) | 0 (max abs and max rel) | 0 |
| Autoencoder encode of the seeded posterior sample / decode | 0 / 0 | 0 / 0 |
| End to end text-to-image: final latent / pixels (1024×1024, 4 steps, guidance 2.5, seed 0) | 0 / 0 | 0 / 0 |
| End to end with an input image: final latent / pixels (512×512, 4 steps, 256×320 input, seed 0) | 0 / 0 | 0 / 0 |
| End to end, seed 1234, 1360×768 request (pixels, 2 steps) | 0 | 0 |
Tests that need no downloaded weights: random-weight parity of the transformer with a tiny and with the
published configuration (two layers), with and without input-image tokens and with sequences long
enough to switch the rotary embedding to its long factors; tokenizer parity with transformers on the
published tokenizer files (fixed prompts, a seeded fuzz test and random Unicode); parity of the image
preprocessing, token layout, attention masks and position ids with the diffusers processor; parity of
the inverted flow-match schedule and Euler steps; and tiny-snapshot end-to-end parity covering the
defaults, guidance 1.0 and above, non-square sizes, several seeds, one step, an input image,
use_input_image_size_as_output, a small max_input_image_size and RGBA and grayscale inputs.
Where the reference pipeline cannot run on its own, parity is defined as follows. The reference fails on
non-RGB input images, so the comparison uses the RGB conversion, as for FLUX.1 Kontext. The reference
draws the input-image posterior from the global torch generator, so the reference runs call
torch.manual_seed(seed) before the pipeline. The reference rotary embedding builds its table from the
first sequence of the batch only; the native module reproduces that.
Device smoke tests (no parity claim)¶
| Device | Result | Largest latent difference from CPU | Largest pixel difference | PSNR |
|---|---|---|---|---|
| MPS (Apple M2 Max) | finite, 512×512, 20 steps | 2.7e-3 | 1 | 72.8 dB |
| CUDA | not run yet |
Limits¶
- The parity claim holds only for float32 on CPU, against diffusers 0.40.0 and transformers 5.18.0.
- MPS and CUDA runs are smoke tests: they check for finite output and the correct size, and record the difference from CPU. There is no parity claim on these devices.
- There is no parity claim for float16 or bfloat16.
- There is no safety checker or invisible watermark. The reference was run with
safety_checker=None. - Only the scheduler published with each checkpoint is implemented: PNDM (PLMS) for SD 1.x and BK-SDM v2, Euler with trailing spacing for SD-Turbo, LCM for LCM Dreamshaper v7, Euler for SDXL base, SSD-1B and Kolors, LCM for Hyper-SDXL and DMD2, Euler ancestral for SDXL Turbo, EDM DPM-Solver++ for Playground v2.5, flow-match Euler for Z-Image and Stable Diffusion 3 and 3.5, flow-match Euler with a static shift for FLUX.1 schnell, Lumina-Image 2.0, Lumina-Next and AuraFlow, flow-match Euler with inverted sigmas for OmniGen, and dynamic shifting for FLUX.1 dev, Qwen-Image 2.1 and LongCat-Image, flow-match Euler with exponential shifting for Krea 2 and linear shifting for CogView4, shifted-SNR DDIM for CogView3-Plus, DDPM for HunyuanDiT, UnCLIP and DDPM for Kandinsky 2.2, DDPM Wuerstchen for Stable Cascade and Wuerstchen v2, DPM-Solver++ for PixArt and Sana (flow sigmas), and cosine-schedule masked-token schedulers for aMUSEd and Lumina-DiMOO, and flow-match LCM with re-noising for HiDream-I1 Fast and Dev, and UniPC flow prediction for HiDream-I1 Full. LlamaGen samples image tokens autoregressively and has no scheduler.
- No image quality benchmark (FID, CLIP score) was run.
- Only text-to-image generation is supported. SD 1.x img2img is not implemented.
Automated tests¶
- Regular tests: 630 passed, 500 skipped (the skipped ones are the optional reference tests). Tests do not download models from the network.
- Reference tests (CPU): 34 passed for SD 1.x, 4 passed for
dreamshaper-8, and 25 passed, 2 skipped for the SDXL family (sdxl-base-1.0,ssd-1b,sdxl-turbo), 24 passed forjuggernaut-xl-v9,animagine-xl-4.0anddreamshaper-xl-1.0, 14 passed forsegmind-small-sdandsegmind-tiny-sdtogether, 9 passed forsegmind-vega, 18 passed forkoala-700mandkoala-1btogether, 16 passed forkoala-lightning-700mandkoala-lightning-1btogether, 12 passed for the three SDXL-Lightning models, 11 passed forsdxl-refiner-1.0, 4 passed forhyper-sdxl-1step, 8 passed fordmd2-sdxl-4stepanddmd2-sdxl-1step, 8 passed forplayground-v2, 8 passed, 1 skipped forplayground-v2.5, 9 passed forz-image-turbo, 8 passed forz-image, 6 passed forjanus-pro-1b, 9 passed forjanus-pro-7b, 9 passed forqwen-image-2.1, 6 passed forqwen-image, 7 passed forflux2-klein-4b, 7 passed forflux2-klein-9b, 8 passed each forflux1-schnell,flux1-devandflux1-krea-dev, 6 passed forflux-1-lite-8b, 7 passed forflux1-kontext-dev, 15 passed forhunyuan-dit-v1.1-distilled, 10 passed each forhunyuan-dit-v1.2andhunyuan-dit-v1.2-distilled, 4 passed forkandinsky-2-1, 5 passed forkandinsky-2.2, 7 passed forkandinsky-3, 6 passed forkandinsky-5-t2i-lite, 7 passed each forstable-cascadeandstable-cascade-lite, 19 passed forpixart-sigma-xl-1024andpixart-alpha-xl-1024, 7 passed forpixart-lcm-xl-1024, 28 passed forpixart-alpha-xl-512,pixart-sigma-256,pixart-sigma-512andpixart-sigma-2ktogether, 7 passed forkrea2-turbo, 8 passed forlongcat-image, 9 passed forcogview4-6b, 8 passed forcogview3-plus-3b, 7 passed forideogram4-fp8, 9 passed forsd-turbo, 8 passed forlumina-image-2.0, 11 passed forlumina-next, 11 passed forlumina-dimoo, 7 passed forkolors, 7 passed forauraflow-v0.3, 7 passed forwuerstchen-v2, 63 passed forllamagen-t2i-xl-256andllamagen-t2i-xl-512together, 8 passed forsd-v2-1, 7 passed each forsana-600m-1024andsana-1600m-1024, 19 passed forsana-sprint-1.6b-1024andsana-1.5-4.8b-1024together, 7 passed each foramused-256andamused-512, 12 passed foromnigen-v1, 8 passed forglm-image, 7 passed forlcm-dreamshaper-v7, 5 passed forldm-text2im-large-256, 8 passed forernie-image, 8 passed forhidream-i1-fast, 8 passed each forhidream-i1-devandhidream-i1-full, 5 passed forhidream-o1-image, and 14 passed forchroma1-hd, 16 passed forchroma1-flash, 18 passed forchroma1-base, 7 passed forovis-image-7b, 8 passed forprx-1024-t2i-betaandprx-512-t2i-sfttogether, 8 passed fordreamlite-baseanddreamlite-mobiletogether, 23 passed forjoyai-image-editandjoyai-image-edit-plustogether, 7 passed fordeepfloyd-if-i-xl, 6 passed formeissonic, 8 passed each forhartandpixeldit, and 27 passed forbk-sdm-base,bk-sdm-smallandbk-sdm-tiny, and 27 passed forbk-sdm-v2-base,bk-sdm-v2-smallandbk-sdm-v2-tiny, 6 passed forshow-o-512, 5 passed forsdxs-512-0.9, 7 passed foremu3-gen, 8 passed forcosmos-predict2-2b-t2i, 5 passed fornextstep-1-large, 6 passed forbagel-7b-mot, and 8 passed forinfinity-2b. These include structural parity with random weights, end-to-end parity with a small model, real-checkpoint parity and tokenizer fuzz tests. - Stable Diffusion 3 and 3.5 reference tests (CPU): 10 passed for
sd3-medium, 10 passed each forsd3.5-mediumandsd3.5-large, and 11 passed forsd3.5-large-turbo. - MPS smoke test: 1 passed.
PRX¶
Validated on CPU in the same environment as above. prx-1024-t2i-beta and prx-512-t2i-sft were
loaded strictly from Photoroom/prx-1024-t2i-beta at revision 0370504 and
Photoroom/prx-512-t2i-sft at revision 2996423 and compared with diffusers 0.40.0
PRXPipeline and transformers 5.18.0 on CPU in float32. The 1024 checkpoint's published bfloat16
text-encoder weights are upcast to float32 on both sides; the other published weights are
float32. All 8 real tests passed, and the observed difference in every
recorded comparison is exactly 0 (0 means exact): on CPU in float32, the local implementation
produces a bit-identical image to the one diffusers produces. The real-checkpoint run took 7
minutes on an M2 Max (CPU).
The native PRX transformer, T5Gemma encoder, Gemma tokenizer behavior, flow-match loop, and FLUX VAE path have exact random-weight and tiny end-to-end parity tests against diffusers 0.40.0 and transformers 5.18.0.
Comparisons (CPU, float32)¶
| Comparison | Checkpoint | Largest observed difference | Tolerance |
|---|---|---|---|
| Tensor header check: text encoder, transformer, VAE (288 / 188 / 244 tensors) | both | 0 missing / extra / mismatched | exact |
| Gemma tokenizer token ids / attention masks (5 fixed prompts and 600 seeded fuzz strings, 256 tokens) | both | identical / identical | exact / exact |
| T5Gemma text encoder embeddings / masks (prompt and negative) | both | 0 / identical | 0 / exact |
| Transformer single step (16×16 latent grid, timesteps 0.8 and 0.3, real conditioning) | both | 0 | 0 |
| VAE decode | both | 0 | 0 |
| End to end: final pixels (native size, 2 steps, guidance 4.0, initial noise from seed 0) | both | 0 | 0 |
| End to end, seed 42 through the public pipeline (pixels) | both | 0 | 0 |
The end-to-end prompt was "a photograph of an astronaut riding a horse" with an empty negative
prompt, max_sequence_length 256 and resolution binning disabled. The runs use the native
1024×1024 and 512×512 sizes and 2 steps instead of the 28-step default only to shorten the CPU
run. Records: header_prx-1024-t2i-beta and header_prx-512-t2i-sft, the
prx_prx-1024-t2i-beta_* and prx_prx-512-t2i-sft_* component keys, e2e_prx-1024-t2i-beta,
e2e_prx-512-t2i-sft, seed_e2e_prx-1024-t2i-beta and seed_e2e_prx-512-t2i-sft,
validation-measurements.json; components_prx,
validation.json.
HART¶
Validated on CPU in the same environment as above. The hart checkpoint was loaded strictly from
mit-han-lab/hart-0.7b-1024px at revision b1521ff (with the Qwen2 text encoder
mit-han-lab/Qwen2-VL-1.5B-Instruct at revision 6e5b539) and compared with the official
HARTForT2I.autoregressive_infer_cfg implementation at revision e28a41f on CPU in float32. All
8 real-checkpoint tests passed, and the observed difference in every recorded comparison is
exactly 0 (0 means exact): on CPU in float32, the local implementation produces a bit-identical
image to the one the official implementation produces. The published HART generator and hybrid
tokenizer weights are float32; the published Qwen2 text-encoder weights are bfloat16 and are upcast
to float32 on both sides. The real-checkpoint run took 3 minutes on an M2 Max (CPU).
Offline tests strictly match the published tensor layouts (317 generator tensors, 190
inference-path autoencoder tensors, and 338 Qwen2 text tensors). Random-weight comparisons cover
Qwen2 padding behavior, HART rotary attention and cached stage execution, the multi-scale quantizer,
the residual-diffusion MLP, and the hybrid-tokenizer decoder; all five comparisons are bit-exact.
The real-checkpoint suite additionally checks the Qwen2 tokenizer on 4 fixed prompts and the encoded
prompt tensors, and runs separate public-pipeline comparisons at seed 0 and seed 42, recording
header_hart, hart_tokenizer, hart_text_encoder, e2e_hart and seed_e2e_hart with passed=
fields. The end-to-end checks keep the fixed 1024×1024 output, one final MaskGIT pass, and eight
residual steps because those dimensions are intrinsic to the published weights; both runs use the
prompt "A blue Porsche 356 parked in front of a yellow brick wall." with the published defaults
(guidance 4.5, the official system prompt enabled, top-p disabled).
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| Tensor header check: generator and inference autoencoder (317 / 190 tensors) | 0 missing / extra / mismatched | exact |
| Qwen2 tokenizer token ids / attention masks (4 fixed prompts, 300 tokens) | identical / identical | exact / exact |
Qwen2 prompt encoding: context, mask, positions (official encode_prompts) |
0 / identical / identical | 0 / exact / exact |
| End to end: final pixels (1024×1024, one MaskGIT pass, eight residual steps, guidance 4.5, seed 0) | 0 | 0 |
| Separate end to end, seed 42 (pixels) | 0 | 0 |
Records: header_hart, hart_tokenizer, hart_text_encoder, e2e_hart and seed_e2e_hart,
validation-measurements.json; components_hart,
validation.json.
Differences found and fixed during validation¶
The first run with real checkpoints exposed two tokenizer differences that tests with random weights could not show. Both were fixed and are guarded by permanent fuzz tests:
- The
normalizedflag of special tokens.<|startoftext|>matches in normalized text, while<|endoftext|>matches only as its exact spelling in the raw text. This is because transformers registers special tokens one after another and the last registration takes effect. - Splitting of sequences not recognized as special tokens. These sequences are split into three
pieces,
<|,endoftext,|>, as in the reference pre-tokenizer.
The SDXL validation found three more differences:
- Memory-mapped safetensors files. CPU BLAS picked a different summation order for differently aligned weights, so a 4.8e-7 difference in the pooled text embedding grew to a 29-pixel difference over sampling. Safetensors files are now memory-mapped, so the weights are read from the file at their published alignment.
nullfor disabled options. The published SDXL config usesnullfor disabled options, for exampleclass_embed_typeandcross_attention_normin the U-Net config.- Special-token metadata in
added_tokens_decoder. SDXL Turbo's tokenizer configs define special-token normalization there, so those entries now take precedence over the namedunk_token,bos_token,eos_tokenandpad_tokenvalues. - Older Unicode tables in the reference tokenizer. For 108 combining marks, the reference tokenizer treats code points Python recognizes as assigned as unassigned.
- Sharded checkpoint alignment. The reference loaders copy sharded checkpoints into fresh 64-byte aligned tensors, unlike single-file checkpoints, and BLAS summation order depends on alignment. The first run therefore differed by 4.2e-5 in the transformer, 3.1e-3 in unguided latents and 1 pixel.
- SDPA attention in the Janus reference. The reference loads the language model with SDPA attention. Eager attention differed by 5.5e-6 in the last hidden state.
- Janus return-sequence configuration. In transformers 5.18.0, setting
model.generation_config.num_return_sequencesas the model card shows fails validation. The equivalentgenerate(num_return_sequences=...)argument is used instead. - Euler initial noise scale with trailing spacing. With
timestep_spacing="trailing", the Euler scheduler's initial noise scale is the largest sigma itself, notsqrt(sigma² + 1).
Ovis-Image¶
Validated on CPU in the same environment as above. ovis-image-7b is pinned to
ATH-MaaS/Ovis-Image-7B revision 41be1c5821a92c970d63d7eb595a2fd3fe32b22e and compared with
diffusers 0.40.0 OvisImagePipeline and transformers 5.18.0 on CPU in float32. The published
transformer and text encoder are float32; the published VAE is bfloat16 and is upcast to float32
on both sides. Offline validation checks every published Ovis transformer tensor name and shape
and executes a tiny random-weight transformer. All 7 real-checkpoint tests passed, and the
observed difference in every recorded comparison is exactly 0 (0 means exact): on CPU in
float32, the local implementation produces a bit-identical image to the one diffusers produces.
The real-checkpoint run took 12 minutes on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
Tensor header check: text_encoder, transformer, vae (310 / 583 / 244 tensors) |
0 missing / extra / mismatched | exact |
| Qwen tokenizer token ids (4 prompts: empty, ASCII, accented and CJK, newline and box-drawing characters) | identical | exact |
| Qwen3 text encoder last hidden state (prompt at 64 tokens) | 0 | 0 |
| Transformer single step (t = 0.5, 8×8 image grid, 32 text tokens) | 0 | 0 |
| VAE decode (16×16×16 latents) | 0 | 0 |
| End to end: uint8 image (1024×1024, 2 steps, guidance 5.0, seed 0) | 0 | 0 |
| End to end, seed 42, different prompt (uint8 image) | 0 | 0 |
The end-to-end runs keep every pipeline default (guidance 5.0, 1024×1024) and use 2 steps instead
of the 50-step default only to shorten the CPU run. Records: header_ovis-image-7b,
ovis_image_tokenizer, ovis_image_text_encoder, ovis_image_transformer_step,
ovis_image_vae_decode, e2e_ovis-image-7b and seed_e2e_ovis-image-7b,
validation-measurements.json; tensor_header_check,
end_to_end and seed_end_to_end under ovis-image-7b and tests.reference_summary,
validation.json.
Show-o¶
show-o-512 is pinned to showlab/show-o-512x512 revision dbe757d951099968fb102d249d35a764528df2dd,
with showlab/magvitv2 revision 5c3fa78f8b3523347c5cd1a4c97f3c4e96f33d5d and the
microsoft/phi-1_5 tokenizer at revision 77aa61eeac94fbf33d492b9f2744c98b42d5b5eb.
The CPU float32 reference is the official Showo class and its t2i_generate method at repository
revision 45a5a2de01d1ebd10cd5864d29310a76476cdf23.
Offline validation checks all 437 Show-o and 382 MAGVIT-v2 published tensor names and shapes, exact CodeGen tokenizer output, random-weight Phi attention and MLP, exact MAGVIT-v2 decoding, and exact tiny discrete-denoising output. Real-checkpoint tests separately compare Phi logits, VQ decoding, and public-pipeline uint8 images at seed 0 and seed 42. Both end-to-end cases use two steps rather than the 18-step default to keep the CPU run short; guidance 10 and 512×512 output remain unchanged. All 6 real-checkpoint tests passed, and the observed difference in every recorded comparison is exactly 0 (0 means exact): on CPU in float32, the local implementation produces a bit-identical image to the one the official Show-o implementation produces. The real-checkpoint run took 1 minute on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
Tensor header check: showo, vq (437 / 382 tensors) |
0 missing / extra / mismatched | exact |
| CodeGen tokenizer token ids (5 prompts) | identical | exact |
| Phi logits | 0 | 0 |
| MAGVIT-v2 decode | 0 | 0 |
| End to end: uint8 image (512×512, 2 steps, guidance 10, seed 0) | 0 | 0 |
| End to end, seed 42 (uint8 image) | 0 | 0 |
The end-to-end runs keep every pipeline default (guidance 10, 512×512) and use 2 steps instead of
the 18-step default only to shorten the CPU run. Records: header_show-o-512, show_o_tokenizer,
show_o_phi_logits, show_o_vq_decode, e2e_show-o-512 and seed_e2e_show-o-512,
validation-measurements.json; tensor_header_check,
end_to_end and seed_end_to_end under show-o-512 and tests.reference_summary,
validation.json.
Emu3-Gen¶
emu3-gen pins BAAI/Emu3-Gen revision
40b3bb89a9cee99ab0b9deb35dead2e50d6a0d72, its vision tokenizer revision
c81f916ad371289e205310a7539255e8a9396488, and the official Emu3 implementation revision
dbbf9858194d70b8c58293e219ecffe22df0f9c7.
Offline validation checks every published generator and vision-tokenizer tensor name and shape and
compares tiny random-weight language-model and VQ-decoder outputs exactly. The real-checkpoint suite
checks a language-model step and VQ decode, then performs separate seed-0 and seed-42 runs through
pipeline("t2i", ...). End-to-end runs use a 2×3 visual-token grid (24×16 pixels), top-k 1 and the
published guidance path. This shortened case exercises prompt formatting, positive/negative passes,
forced row and image markers, autoregressive cache updates, visual lookup and decoding without an
impractical full 90×90 CPU generation.
Validated on CPU in the same environment as above. emu3-gen was loaded strictly from
BAAI/Emu3-Gen and compared with the official Emu3 implementation on CPU in float32. The
published generator and vision-tokenizer weights are float32. All 7 real tests passed, and the
observed difference in every recorded comparison is exactly 0 (0 means exact): on CPU in
float32, the local implementation produces a bit-identical image to the one the official
implementation produces. The real-checkpoint run took 2 minutes on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| Tensor header check (291 generator / 495 vision-tokenizer tensors) | 0 missing / extra / mismatched | exact |
| Random-weight language model with KV cache | 0 | 0 |
| Random-weight vision decode | 0 | 0 |
| Real checkpoint: language-model step and VQ decode | 0 | 0 |
| Tiny end to end (uint8 pixels) | 0 | 0 |
| End to end: final pixels (24×16, seed 0) | 0 | 0 |
| End to end, seed 42 (pixels) | 0 | 0 |
Records: emu3_gen_headers, emu3_gen_language_model_random, emu3_gen_vision_decode_random,
emu3_gen_real_components, emu3_gen_tiny_e2e, e2e_emu3-gen and seed_e2e_emu3-gen,
validation-measurements.json; the validation grouping is
components_emu3, validation.json.
NextStep-1 Large¶
nextstep-1-large pins stepfun-ai/NextStep-1-Large revision
97dd0b9e097f60e39e3ce3bbac1a3343a09547cd and the official implementation revision
a3303b62e11fc7cd473ba0e4b1f32eda73f4acc0.
Offline validation checks every published transformer tensor name and shape and compares the tiny
random-weight Qwen2 decoder and flow-matching head exactly. Unit coverage also checks patch packing,
flow output shape, validation before loading and seeded repeatability. The real-checkpoint suite
defines separate seed-0 and seed-42 runs through pipeline("t2i", ...) at 64×64 with one flow step.
The shortened case exercises the complete prompt-to-uint8 path while bounding the large CPU run.
Validated on CPU in the same environment as above. All 5 real tests passed, and the observed difference in every recorded comparison is exactly 0 (0 means exact): on CPU in float32, the local implementation produces a bit-identical image to the one the official implementation produces. The published token embedding is bfloat16 and the checkpoint config declares bfloat16, so both implementations load the transformer weights as bfloat16 and upcast them to float32 for this run; the published VAE checkpoint is float32. The real-checkpoint run took 6 minutes on an M2 Max (CPU).
Comparisons (CPU, float32)¶
| Comparison | Largest observed difference | Tolerance |
|---|---|---|
| Published transformer layout | 0 missing / extra / mismatched | exact |
| Random-weight Qwen2 decoder | 0 | 0 |
| Random-weight flow-matching head | 0 | 0 |
| End to end: uint8 image (64×64, 1 step, seed 0) | 0 | 0 |
| End to end, seed 42 (uint8 image) | 0 | 0 |
Records: nextstep_qwen_random, nextstep_flow_random, e2e_nextstep-1-large and
seed_e2e_nextstep-1-large,
validation-measurements.json.