Memory and loading
Start with default loading. Add memory controls only where needed. Fully Sharded Data Parallel (FSDP) divides model parameters among existing ranks. Transformer tensor parallelism (TP) divides transformer computation.
Memory controls
| Control | Effect and restrictions |
|---|---|
--fully_shard_degree N |
FSDP parameter sharding across existing model ranks. Pair with a matching layout, such as --ulysses_degree N; it does not launch more processes. Transformer TP and FSDP cannot both exceed 1. |
--fully_shard_components ... |
Select pipeline components for FSDP. With PipeFusion, select replicated components such as a supported text encoder, rather than the stage-local transformer. |
--text_encoder_tp_degree N |
Shards a supported text encoder across existing ranks. MiniMax-H3 and MiniMax-H3-Image require this degree to be 1 or match --ulysses_degree. MiniMax-H3 multi-GPU runs use this instead of text-encoder CPU offload. MiniMax-H3-Image encodes on the first rank by default; use this only if the encoder does not fit. |
--memory_efficient_sharding |
Requires FSDP degree greater than 1 and a supported load route. Loads checkpoint blocks through rank 0 before filling shards. This avoids loading the full model on every device first. |
--memory_efficient_replicated_load |
Reduces load-time memory for supported multi-rank replicated layouts; weights remain replicated after loading. Supported FLUX and Qwen Image PipeFusion runners instead load only each stage's transformer blocks. Other pipeline components remain eager. A single-rank run uses standard loading. |
--enable_model_cpu_offload |
Moves pipeline components between host and device. Supported with FSDP; MiniMax-H3 and MiniMax-H3-Image use a specialized text-encoder-only offload on one GPU. |
--enable_group_cpu_offload / --enable_sequential_cpu_offload |
Finer-grained host offload; both are rejected with FSDP. AITER FP4 also rejects group offload. MiniMax-H3 and MiniMax-H3-Image reject both. |
--enable_tiling / --enable_slicing |
Model-specific VAE memory controls. See parallel VAE for their relationship to distributed decoding. |
Do not combine memory-efficient replicated loading with transformer tensor parallelism or ordinary FSDP. Supported PipeFusion layouts can use the two memory-efficient flags for different components. Each stage loads its own transformer blocks; FSDP loads selected replicated components.
Meta loading constructs modules without allocating their weight tensors. SD3.5 does not support this loading path for PipeFusion stages.
Supported routes vary by model, including between related checkpoint variants.
The model's capabilities, load_support, and settings declarations in
runner models
define supported loading paths. The runner rejects unsupported requests before
allocation. FSDP support alone does not establish support for memory-efficient loading.
xdit --model Qwen-Image --prompt "An isometric botanical library" \
--ulysses_degree 4 --memory_efficient_replicated_load
For parameter sharding, start with --fully_shard_degree 4 and a matching
four-rank layout, then add --memory_efficient_sharding if the model supports
that loading route. Quantization is an additional memory
tradeoff; model capabilities distinguish these features.