Ana içeriğe geç

HiDream-I1

Model Source Revision Weight license Parity (CPU, fp32)
hidream-i1-dev HiDream-ai/HiDream-I1-Dev 0fad2ea MIT Verified
hidream-i1-fast HiDream-ai/HiDream-I1-Fast 4856a5d MIT Verified
hidream-i1-full HiDream-ai/HiDream-I1-Full 8ccbbfb MIT Verified

Catalog: hidream-i1 · Models: hidream-i1-dev, hidream-i1-fast, hidream-i1-full · Task: t2i

Architecture. Four text encoders condition a 17B-parameter sparse flow-matching transformer. Two CLIP text encoders with projection (CLIP-L and CLIP-G, 128 tokens) give a pooled vector of width 2048 that is added to the timestep embedding. A T5-XXL encoder (128 tokens, width 4096) and a Llama-3.1-8B-Instruct decoder supply the token-level context: the Llama encoder contributes the output of every one of its 32 layers (the last one after the final norm), and the transformer projects the layers named in llama_layers separately, one per block. The transformer has 16 dual-stream blocks followed by 32 single-stream blocks of width 2560 (20 heads of 128), 2×2 patches and rotary embeddings; every feed-forward layer is a mixture of four routed SwiGLU experts (two activated per token) plus a shared expert. Images are decoded by the 16-channel, 8× FLUX VAE (scaling factor 0.3611, shift factor 0.1159). The scheduler is the flow-match LCM scheduler with a static shift of 3.0: each step predicts the clean sample and re-noises it to the next sigma with fresh Gaussian noise.

Usage.

export HF_TOKEN=...   # an account with access to meta-llama/Llama-3.1-8B-Instruct
python -m imagehub.pretrained hidream-i1-fast checkpoints/hidream-i1-fast
from imagehub import pipeline

with pipeline(
    "t2i", backend="hidream-i1-fast", model="checkpoints/hidream-i1-fast", device="mps"
) as pipe:
    result = pipe("a cat, oil painting", seed=42)
    result.images[0].save("cat.png")

Gated Llama encoder. The HiDream repository does not contain the Llama encoder. The catalog entry therefore also downloads meta-llama/Llama-3.1-8B-Instruct (revision 0e9e39f) into the snapshot's text_encoder_4/ folder, together with the tokenizer files the pipeline needs. That repository is gated: accept the Llama 3.1 Community License on Hugging Face and pass a token that has access. The pipeline reads the tokenizer and the model from this folder; no chat template is applied, the prompt is tokenized as plain text with a leading begin-of-text token and padded with the end-of-turn token.

Settings. Fast defaults to 16 steps and guidance 0.0; Dev to 30 and 0.0; Full to 50 and 5.0 (classifier-free guidance), all at 1024×1024. The supported options are seed, latents, steps, guidance, width, height, num_images, negative_prompt and max_sequence_length (default 128 for every encoder, at most 512). All four encoders see the same prompt. Guidance above 1.0 runs the unconditional pass with the negative prompt (default empty) and combines the two predictions as usual; at 1.0 or below one pass is made. Like the diffusers pipeline, the requested width and height are only used for the aspect ratio: the image is rescaled to about one megapixel and rounded down to a multiple of 16, so width=1024, height=512 gives a 1440×720 image. metadata reports the size actually used. Unrecognized settings raise an error.

Random numbers. The scheduler adds fresh noise at every step. The diffusers pipeline draws the initial latents from the generator it is given but calls the scheduler without one, so the step noise comes from the global random state. imagehub never touches global random state. With a seed it uses two generators seeded with that value, one for the initial latents and one for the step noise; the result equals the pipeline's when its caller runs torch.manual_seed(seed) immediately before the call and passes generator=torch.Generator("cpu").manual_seed(seed). Nothing else in the pipeline consumes global random numbers between those points. When latents are passed, seed only seeds the step noise; without a seed that noise is random.

Memory. In float32 the transformer needs about 68 GB, the Llama encoder about 32 GB, the T5 encoder about 19 GB, the two CLIP encoders about 3.5 GB and the VAE about 0.3 GB. The backend loads the text encoders one at a time, encodes the prompt, frees each of them, then loads the transformer, so the peak is the transformer alone; the next call loads the encoders again from disk. bfloat16 halves every figure.

Seed compatibility. On CPU, the same seed and settings give the same image as diffusers with the generator and global seed described above (validation). MPS and CUDA runs are smoke tests only; there is no claim of exact equality on these devices.

Schedulers. Fast and Dev use flow-match LCM (Dev uses its published shift of 6.0); Full uses its published UniPC flow-prediction scheduler. Separate prompts for the individual encoders, precomputed prompt embeddings and fp16/bf16 parity are not supported.

License. The HiDream repository declares the MIT license. The text encoders carry their own terms: Llama 3.1 Community License and Apache-2.0 for T5; the VAE comes from FLUX.1 [schnell]. imagehub does not distribute weights.