Communication options
Ulysses exchanges attention tokens and heads using all-to-all collectives. The default path uses the selected device runtime and attention backend, including supported CUDA, ROCm, and Ascend environments. Start with that path and measure your workload before changing the transport.
| Control | What changes | Current requirements |
|---|---|---|
| Default | Regular Ulysses collectives | A supported model, layout, and device backend |
--ulysses_a2a sdma |
Uses SDMA copy engines for eligible input and output exchanges | ROCm, one node, and supported dense attention calls |
--use_fp8_comms |
Quantizes eligible Q/K/V and attention-output exchanges | Model support and a pre-quantization attention backend; current kernels use ROCm/AITER |
--attention_a2a |
Replaces the input exchange with AITER's intranode transport | Supported Wan models, ROCm/AITER, and MORI SHMEM |
These controls affect communication tensors. GEMM quantization controls linear-layer compute precision. Attention backend selection controls the attention kernel. Check the model tables before combining features.
SDMA Ulysses transport
On ROCm, --ulysses_a2a sdma uses GPU copy engines for both Ulysses exchanges.
It can overlap communication with attention and Q/K/V projections. Support was
validated upstream on gfx942 and gfx950; measure latency on your workload.
xdit --model flux-3-image-klein-base --prompt "A lighthouse on a rocky coast" \
--use_cfg_parallel --ulysses_degree 4 --attention_backend aiter_bf16 \
--ulysses_a2a sdma --ulysses_sdma_overlap_qkv --use_torch_compile
Use one node with peer-accessible GPUs. The transport builds with hipcc on first
use, or loads a prebuilt library from XDIT_SDMA_LIB. See the
build script.
--attention_a2a and SDMA are mutually exclusive.
| Option | Effect |
|---|---|
--ulysses_sdma_head_groups |
Splits each rank's heads into groups; effective default 2. Exchanges overlap other groups' attention. |
--ulysses_sdma_group_streams |
Runs each group's attention on a separate stream. |
--ulysses_sdma_overlap_qkv |
Overlaps projection with exchange. MiniMax-H3 and FLUX.3 project per group and retain a second Q/K/V weight copy. Wan sends Q/K before projecting V, without that copy. With CPU offload, only Wan's approach applies. |
These options require --ulysses_a2a sdma. Unsupported calls fall back to regular
collectives with a warning: Ring, FP8 communication, PipeFusion KV caching, head
balancing, sparse kernels, attention masks, and certain padded joint-text layouts.
While SDMA is selected, the runner disables transformer CUDA graphs because its
host-issued copies cannot replay; other components can retain their graphs.
XDIT_SDMA_WAIT_S sets the wait timeout in seconds (default 30).
XDIT_SDMA_ALLOW_UNTESTED=1 permits other GPU architectures.
FP8 communication
Q, K, and V are the attention queries, keys, and values. For a supported ROCm/AITER environment, this example uses four Ulysses ranks:
xdit --model FLUX.1-dev --prompt "A lighthouse in a winter storm" \
--ulysses_degree 4 --attention_backend aiter_fp8 --use_fp8_comms
Requirements:
- Ulysses degree must exceed 1, and the model must declare FP8 communication support.
- At least one selected or scheduled backend must accept pre-quantized inputs.
Current choices are
aiter_fp8andaiter_flydsl_fp8; the latter requiresgfx1201. Their Ring path is unsupported, so use--ring_degree 1. An FP8 attention backend that lacks this capability cannot enable FP8 comms. - CPU offload and PipeFusion are rejected. Joint/replicated text can impose further model restrictions: FLUX.1-dev rejects FP8 comms when text cannot be split evenly.
- The required AITER APIs and supported tensor layouts must be available. Check the installed kernel requirements before enabling the flag.
The runner uses the generation inputs for an additional calibration run before inference and compilation. It discards that output and keeps per-layer Q/K/V/output scales. FP8 can reduce exchanged tensor sizes, but adds quantization work and can change output. It does not guarantee lower latency. Compare the same prompts and seeds with regular communication.
| Scale option | Use |
|---|---|
--fp8_comms_scale |
Positive fixed scale; skips automatic per-layer calibration. Use only with a scale validated for your workload. |
--fp8_comms_safety_factor |
Calibration headroom in (0, 1], default 0.85. A smaller value increases the scale and leaves more room before saturation. |
Attention A2A
Supported Wan2.2 T2V and Wan2.1/2.2 I2V adapters can use AITER's intranode
Attention A2A transport for the input Ulysses exchange. The output exchange uses
the regular collective. --attention_a2a and --use_fp8_comms are mutually
exclusive. Choose one transport optimization.
xdit --model Wan2.2-T2V --prompt "A motorcycle on a coastal highway" \
--ulysses_degree 8 --attention_backend aiter_fp8 \
--attention_a2a auto --use_torch_compile
This recipe requires gfx950. The transport requires compatible AITER and MORI
SHMEM, batch size 1, head dimension 128, and intranode Ulysses degree 2, 4, or 8.
Ring and sparse-attention head balancing are unsupported. Auto and MX profiles require
gfx950; gfx942 can use supported explicit non-MX profiles such as
--attention_a2a e4m3-e4m3 with aiter_fp8.
Auto selects a recipe from the dense attention backend:
| Backend | Profile |
|---|---|
aiter_fp8 |
e4m3-e4m3 |
aiter_i8fp8 |
int8-e4m3 |
aiter_mxfp8 |
mxfp8-e4m3 |
aiter_f8f6 |
e4m3-mxfp6 |
aiter_mxfp6 |
mxfp6-e4m3 |
aiter_f6f6 |
mxfp6-mxfp6 |
aiter_f6f4 |
mxfp6-mxfp4 |
aiter_mxfp4, aiter_f4f4 |
mxfp4-mxfp4 |
An explicit profile requires the matching attention backend from this table.
For hybrid attention, set --attention_a2a auto. Every scheduled backend must
have an A2A recipe. A BF16 endpoint such as aiter has no recipe, so the ordinary
BF16/FP8 hybrid example cannot also enable A2A. These transport requirements are
stricter than selecting the attention kernel alone.
--attention_a2a_hadamard auto uses the recipe's validated rotation placement.
Overrides are preprocess, transport, epilogue, and none.
The int8-e4m3 profile rejects transport rotation; use auto/none for its
native recipe or preprocess for explicit rotation. The runner rejects unsupported
Torch/AITER interfaces. Set --attention_a2a none to use the default transport.
See the CLI reference for syntax, FP8 communication implementation, and A2A validation for current contracts.