Runner CLI reference

This page is generated from the runner's argument parser. See the runner guide for launch options, examples, model defaults, and compatibility restrictions. Dedicated model examples use a different parser.

Parser defaults are not necessarily effective model defaults. An omitted value may be resolved by the selected model or configuration validation; None does not by itself mean that a feature is disabled. A registered option is not a promise that every model or device supports it.

Runner long options accept underscores or dashes (for example, --ulysses_degree and --ulysses-degree). The spellings below are those registered by argparse, including negative spellings for Boolean options.

In the syntax column, TEXT, INT, and FLOAT name value types; VALUE uses the format described by the option's help. CHOICE refers to the accepted values listed in the description. Brackets mean optional values: [TEXT ...] accepts zero or more, and TEXT [TEXT ...] accepts one or more. Flags without a value placeholder take no value. Boolean positive spellings enable a setting and their --no-... spellings disable it.

Model selection and adapters

Syntax Parser default Description
--model TEXT None Required. Registered model alias or Hugging Face repository ID. Local checkpoint loading is model-specific.
--lora_path VALUE None LoRA repository or local path to fuse at startup (supported runners only).
--lora_weight_name VALUE None LoRA weight filename within --lora_path when it names a repository or directory.
--lora_scale FLOAT 1.0 Finite scale applied when fusing a startup LoRA adapter.
--vdn_transformer_dir VALUE None Path to the materialized VDN-H3 transformer directory (must contain a transformer/ subfolder). Overrides the VDN_H3_TRANSFORMER_DIR environment variable. Required when --model is OpenVDN/vdn-minimax-h3 (or VDN-H3).
--vdn_window_backend {sdpa,aiter,aiter_ck,aiter_fp8} None VDN-H3 only: override the window-softmax varlen kernel. By default it is derived from --attention_backend (SDPA->sdpa, AITER->aiter_ck, AITER_FP8->aiter_fp8); 'aiter' selects AITER's Triton varlen bf16 kernel.
--task TEXT None Task to perform. Only applicable if the model supports multiple tasks.
--distilled_transformer_path VALUE None Path to the high-noise distilled transformer safetensors file.
--distilled_transformer_2_path VALUE None Path to the low-noise distilled transformer_2 safetensors file.

Generation inputs

Syntax Parser default Description
--height INT None Output height in pixels.
--width INT None Output width in pixels.
--num_frames INT None Number of output video frames.
--prompt [TEXT ...] None Prompt for the model.
--negative_prompt [TEXT ...] None Negative prompt for the model.
--num_inference_steps INT None Number of inference steps.
--max_sequence_length INT None Maximum prompt token length for supported text encoders.
--seed INT 42 Random seed for operations.
--guidance_scale FLOAT None Model guidance scale; interpretation and default depend on the selected model.
--guidance_scale_2 FLOAT None Second guidance scale for models with a separate denoising stage.
--flow_shift FLOAT None Flow shift for the scheduler.
--input_images TEXT [TEXT ...] [] Path(s)/URL(s) to input image(s).
--resize_input_images False Whether to resize and crop the input image(s) to the specified width and height.
--batch_size INT None Split prompts into batches of this size. Required with --dataset_path; if omitted, run prompts together.
--dataset_path TEXT None CSV file with a prompt column. Replaces --prompt and requires --batch_size.
--audio_prompt VALUE None Prompt for the audio stream of a joint video-audio model (Prism). Defaults to --prompt.

Parallelism and PipeFusion

Syntax Parser default Description
--use_cfg_parallel False Split conditional and unconditional CFG branches across two ranks on supported models.
--data_parallel_degree INT 1 Data parallel degree.
--ulysses_degree INT 1 Ulysses sequence parallel degree. Used in attention layer.
--ring_degree INT 1 Ring sequence parallel degree. Used in attention layer.
--pipefusion_parallel_degree INT 1 Pipefusion parallel degree. Indicates the number of pipeline stages.
--num_pipeline_patch INT None Number of patches used by the PipeFusion schedule. Defaults to the pipeline degree.
--attn_layer_num_for_pp [INT ...] None Number of transformer layers assigned to each PipeFusion stage.
--warmup_steps INT 1 Number of synchronous denoising steps before the asynchronous PipeFusion schedule.
--tensor_parallel_degree INT 1 Tensor parallel degree.
--text_encoder_tp_degree INT 1 Tensor parallel degree for supported text encoders, reusing the existing model ranks.
--fully_shard_degree INT 1 FSDP sharding group size for supported models. Does not add launcher ranks; 1 disables sharding.
--fully_shard_components TEXT [TEXT ...] None Only FSDP-wrap these pipeline components (for example 'text_encoder'). By default every component named by the model's FSDP strategy is sharded.
--no_reshard_after_forward True
Setting: reshard_after_forward
Keep parameters gathered after each block's forward instead of resharding. Trades memory for latency by eliminating repeated all-gathers. Only valid with --fully_shard_degree > 1. Sets reshard_after_forward to False.

Loading and CPU offload

Syntax Parser default Description
--memory_efficient_sharding False Reduce peak VRAM during load: shard transformer blocks one at a time on GPU during init, so the full unsharded component never materializes on device. Slightly slower to load - use if the model OOMs on GPU during init. Requires --fully_shard_degree > 1.
--memory_efficient_replicated_load False Reduce replicated-weight host memory on supported multi-rank layouts by loading on rank 0 and broadcasting to peers built on meta. With supported PipeFusion runners, stream only stage-local transformer blocks instead. Conflicts with transformer tensor parallelism and untargeted FSDP; a single-rank run remains eager.
--enable_sequential_cpu_offload False Move weights from CPU as individual submodules execute; saves device memory at a latency cost.
--enable_model_cpu_offload False Move whole pipeline components between CPU and device as needed on supported models.
--enable_group_cpu_offload False Stream groups of weights from CPU: block-level for compiled components, leaf-level otherwise.
--group_offload_low_cpu_mem False With --enable_group_cpu_offload, pin tensors on demand instead of pre-pinning whole components. Reduces pinned host memory at a possible latency cost.

VAE memory and layout

Syntax Parser default Description
--use_parallel_vae False Distribute supported VAE work across existing ranks. DistVAE uses row sharding by default; tiling can select tile dispatch.
--enable_tiling False Enable VAE tiling to save GPU memory.
--enable_slicing False Decode one batch item at a time to reduce GPU memory use. This has no effect when the batch size is 1.
--vae_tile_size_height INT None Exact output-pixel height of each VAE tile. Requires --enable_tiling. Must be used with --vae_tile_size_width.
--vae_tile_size_width INT None Exact output-pixel width of each VAE tile. Requires --enable_tiling. Must be used with --vae_tile_size_height.
--vae_tile_overlap_height INT None Exact VAE tile overlap along the height axis, in output pixels. Requires --enable_tiling. Must be used with --vae_tile_overlap_width. Height and width may differ; use 0 for an inactive strip axis.
--vae_tile_overlap_width INT None Exact VAE tile overlap along the width axis, in output pixels. Requires --enable_tiling. Must be used with --vae_tile_overlap_height. Height and width may differ; use 0 for an inactive strip axis.
--use_vae_channels_last_format False Use channels last memory format for the VAE.
--vae_dtype {bf16,fp32} None Load the VAE in this dtype instead of the model's default. Faster in bf16, at some image fidelity. Only for models that list it.

Compilation and GEMM quantization

Syntax Parser default Description
--use_torch_compile False Compile supported model components with torch.compile, including supported parallel configurations.
--gemm_quantization VALUE None Transformer GEMM precision: none, fp8, fp4, fp6, a6w4, int8, or a supported low=fp4,high=<format> pair.
--gemm_config VALUE None Optional YAML file with advanced GEMM target or schedule settings.
--use_int8_gemms False [Deprecated] Use --gemm_quantization int8 for supported transformer targets.
--use_fp8_gemms False [Deprecated] Use --gemm_quantization fp8 for supported transformer targets.
--use_fp8_text_encoder False Quantize supported text-encoder linear layers to FP8. Requires a GEMM profile containing FP8; reduces weight memory and may change output quality.
--use_fp4_gemms False [Deprecated] Use --gemm_quantization low=fp4,high=fp8 for supported transformer targets.
--fp8_precision_override_prefix_patterns VALUE None [Deprecated] Comma-delimited high-precision target prefixes for a tiered GEMM profile. Prefer --gemm_config.
--fp8_precision_override_suffix_patterns VALUE None [Deprecated] Comma-delimited high-precision target suffixes for a tiered GEMM profile. Prefer --gemm_config.
--use_hybrid_gemm_schedule False Use the profile's high format at denoising endpoints and FP4 in the middle.
--num_hybrid_gemm_high_precision_steps INT None Number of high-precision GEMM steps at both the start and end of denoising.

Attention backends and communication

Syntax Parser default Description
--attention_backend TEXT None Attention backend name. If omitted, use the model's preferred backend or automatic selection.
--cross_attention_backend TEXT None Attention backend to use for cross-attention. If not specified, falls back to --attention_backend.
--mha_v4_kv_splits TEXT '1' Dense AITER MHA v4 KV split count: 1 disables splitting; 'auto' delegates to AITER. A larger count requires a compatible AITER kernel and shape. Tune with measured latency; splitting adds host overhead and does not apply to every attention format.
--attention_a2a CHOICE 'none' AITER Attention A2A input-hop profile. Explicit profiles require and validate a matching --attention_backend; auto infers the profile from the active backend, including hybrid schedules. When Tier-2 async lowering is unavailable, xDiT uses the Tier-1 ordered path; if neither path is supported, initialization fails and this option must be disabled.
Choices: none, auto, e4m3-e4m3, int8-e4m3, mxfp8-e4m3, e4m3-mxfp6, mxfp6-e4m3, mxfp6-mxfp6, mxfp4-mxfp4, mxfp6-mxfp4.
--attention_a2a_hadamard {auto,preprocess,transport,epilogue,none} 'auto' Q/K Hadamard placement for Attention A2A. 'auto' uses the recipe default (none for INT8, preprocess for the MX/FP8 profiles).
--ulysses_a2a {default,sdma} 'default' Ulysses all-to-all transport: default (the process group's collective, NCCL or RCCL) or sdma (the GPUs' SDMA copy engines; ROCm, one node).
--ulysses_sdma_head_groups INT None With --ulysses_a2a sdma: split each rank's heads into this many groups (default 2); a group attends while later groups' data is still moving, and its output goes back while later groups attend.
--ulysses_sdma_group_streams False With --ulysses_a2a sdma: attend each head group on a stream of its own.
--ulysses_sdma_overlap_qkv False With --ulysses_a2a sdma: overlap the Q/K/V projection with the exchange, in models that support it. Where the QK norm works per head (MiniMax-H3, FLUX.3), Q/K/V are projected one head group at a time, from a second copy of the Q/K/V weights; where it spans all heads (Wan), Q/K are sent before V is projected, in one head group. Under CPU offload only the latter applies.
--use_fp8_comms False Quantize Ulysses all-to-all communication to FP8.
--fp8_comms_scale FLOAT None Override the model-specific FP8 communication scale.
--fp8_comms_safety_factor FLOAT 0.85 Safety factor for the calibrated FP8 comms scale: scale = amax / (FP8_MAX * safety_factor). LOWER it (e.g. 0.4) to enlarge the scale and leave more headroom before fp8 saturation. Default 0.85.
--use_hybrid_attn_schedule False Switch attention backends during denoising on supported models. Performance and quality depend on the schedule.
--hybrid_attn_low_precision_backend VALUE None Low-precision attention backend for hybrid schedule. If set, high-precision backend must also be set.
--hybrid_attn_high_precision_backend VALUE None High-precision attention backend for hybrid schedule. If set, low-precision backend must also be set.
--num_hybrid_attn_high_precision_steps INT None High-precision denoising steps at both the start and end of the hybrid attention schedule.
--hybrid_attn_schedule TEXT None Comma-separated per-step attention backend names. Alternative to the low/high-precision backend pair.

Sparse attention

Syntax Parser default Description
--use_ssta_sparse_text_to_image False Use sparse attention for text-to-image attention path in SSTA.
--spargeattn_simthreshold FLOAT 0.3 Similarity threshold for sparge attention.
--spargeattn_cdfthreshold FLOAT 0.92 CDF threshold for sparge attention.
--spargeattn_reorder_sequence
--no-spargeattn_reorder_sequence
True Reorder image tokens via the gilbert space-filling curve before Sparge attention. Use --no-spargeattn_reorder_sequence to disable.
--use_spargeattn_static_block_mask
--no-use_spargeattn_static_block_mask
True OR a static gilbert block-neighbor mask into the dynamic Sparge block mask. Only meaningful when --spargeattn_reorder_sequence is set. Use --no-use_spargeattn_static_block_mask to disable.
--use_sparseattn_head_balance False Balance per-rank attention work across Ulysses ranks by permuting heads (block-sparse load balancing). Only has an effect with ulysses_degree>1, equal query and KV head counts, and a block-sparse attention backend that publishes a per-head cost (the Sparge backends or the AITER Sol backends).
--use_spargeattn_head_balance False Deprecated, use --use_sparseattn_head_balance.
--sparseattn_block_size QxKV None Block-sparse tile as QxKV, e.g. 64x64: the query rows and keys one mask entry covers. Omit to use each recipe's default tile. Honoured by the AITER MHA v4 Sparge and AITER Sol backends; the run is refused at startup if this GPU has no kernel at that tile for the selected recipe.
--solattn_beta FLOAT 0.5 Routing threshold for the AITER Sol backends. A KV block is computed exactly when its pooled proxy score exceeds mean + beta*std over the blocks of that query tile, so larger beta keeps fewer blocks.
--solattn_beta_schedule SCHEDULE None Vary --solattn_beta across denoising steps, since steps are not equally approximable. Either 'first:last' for a linear ramp over the run, e.g. '1.0:-0.25' to start aggressive and end exact, or one beta per step as '0.5,0.5,0.25,...' -- per DENOISING STEP, so a 40-step run takes 40 whether or not guidance is on: a guided step's conditional and unconditional forward share the step's beta. Overrides --solattn_beta when set. A spec starting with a negative beta needs the --solattn_beta_schedule=-0.5:0.0 form, or argparse reads the leading minus as another option.
--solattn_tilt_rank RANK 0 Correct each pooled block of the AITER Sol backends with its leading RANK terms of Cov(v, k) q, on top of a second-order term on its logit, so a skipped block stands in for its keys more closely. 0 (default) keeps the block means. Not served by aiter_mxfp6_sol.
--sol_attn_tau FLOAT 0.2 NVIDIA Sol-Attn (SOL_ATTN backend) routing temperature. Higher values keep fewer KV blocks exact.
--sol_attn_thresh_type {diag,exact} 'exact' NVIDIA Sol-Attn routing threshold. 'diag' uses the diagonal covariance; 'exact' uses the full covariance.
--sol_attn_kv_splits {auto,1,2,4} 'auto' KV splits for the SM90 Sol-Attn kernel. 'auto' uses 4 on SM90 CuTe when the sequence is at least 65536 tokens.
--sol_attn_sink_tokens INT 0 NVIDIA Sol-Attn: number of tokens whose KV blocks stay exact for every query. 0 disables the sink.
--sol_attn_sink_start INT None NVIDIA Sol-Attn: first token of the exact KV sink. Omit to place the sink on the token suffix.
--vsa_block_size INT 128 Query/KV block size for the AITER CK-Tile VSA backend.
--vsa_top_k INT 1 Minimum KV blocks selected per query block by AITER VSA.
--vsa_top_k_ratio FLOAT 0.0 Minimum selected KV-block fraction for AITER VSA. For Jenga drop rate r, set this to 1-r.
--vsa_drop_rates FLOAT [FLOAT ...] None Enable Jenga's per-step sparse schedule. One value applies to all steps; two values switch after the midpoint. Drop rates <=0.25 use dense AITER attention.
--vsa_prob_threshold FLOAT 0.9 Jenga cumulative probability threshold for AITER VSA.
--bsa_sparsity FLOAT 0.75 TRITON_BSA: fraction of key blocks each query block skips (top-k).
--bsa_cdf_threshold FLOAT 0.2 TRITON_BSA: also keep the key blocks holding this much attention weight (top-p); 0 disables.
--vsa_reorder_sequence
--no-vsa_reorder_sequence
True Reorder Wan video tokens with the sliced Gilbert curve before VSA.
--use_vsa_static_block_mask
--no-use_vsa_static_block_mask
True Union Gilbert physical-neighbor blocks into the dynamic VSA mask.
--use_vsa_first_frame_mask
--no-use_vsa_first_frame_mask
True Preserve first-frame block relations in the VSA mask.
--vsa_collect_density False Record the selected VSA block density for profiling.

Step caching

Syntax Parser default Description
--use_teacache False [Deprecated] Use --cache_method teacache instead.
--use_fbcache False [Deprecated] Use --cache_method fbcache instead.
--cache_method {teacache,fbcache,dbcache,pipefusion} None Step cache for supported models. pipefusion requires pipeline parallelism; other methods reject it.
--cache_config TEXT None JSON string of advanced config overrides merged on top of preset defaults. E.g. '{"residual_diff_threshold": 0.5, "max_warmup_steps": 6}'. Cache-DiT engines use DBCacheConfig fields; in-tree TeaCache/FBCache accept only residual_diff_threshold.

Benchmarking, profiling, and output

Syntax Parser default Description
--num_iterations INT 1 Number of iterations to run the model.
--determinism_check INT 0 Set to a positive failure threshold to compare every timed iteration with the first using exact equality, not a tolerance. The failure handler is called for each rank enabled by --determinism_check_report_ranks on each divergence until the threshold is reached. Disabled at 0 or below. Enabling retains and compares full output payloads, which adds memory, synchronization, and transfer overhead.
--determinism_check_report_ranks TEXT 'last' Ranks on which determinism failures save serialized outputs. Accepts a comma-separated list of integers, an empty value, or one of: all, first, last, none. Empty and none select no ranks. Defaults to last. Many failures may produce many large output files; restrict output to selected ranks based on model specifics. The none value is useful with a positive --determinism_check when only log messages, but no files, are needed.
--profile False Whether to run Pytorch profiler. See --profile_wait, --profile_warmup and --profile_active for profiler specific warmup.
--profile_wait INT 2 wait argument for torch.profiler.schedule. Only used with --profile.
--profile_warmup INT 2 warmup argument for torch.profiler.schedule. Only used with --profile.
--profile_active INT 1 active argument for torch.profiler.schedule. Only used with --profile.
--profile_capture_phase False Profile the CUDA graph recording phase and save the capture trace.
--profile_with_stack False Record call stack info in the profiler. Only used with --profile.
--warmup_calls INT 0 Full untimed pipeline calls before measured iterations; separate from PipeFusion --warmup_steps.
--output_directory TEXT '.' Directory where to save outputs, profiles and timings.