Outputs, timing, and profiling
Timing and warmup
--num_iterations repeats generation. --warmup_calls performs complete untimed
pipeline calls before those iterations. This differs from --warmup_steps,
which counts synchronous denoising steps in the PipeFusion schedule, and
--profile_warmup, which configures the profiler's sampling schedule.
If no explicit or CUDA graph capture warmup runs, the runner discards the first
timing when it has multiple measurements. timings.json contains
a list of timings in seconds; batched inference records each batch separately.
--use_torch_compile can also perform compilation warmup during initialization.
--profile runs PyTorch profiling with --profile_wait, --profile_warmup, and
--profile_active, measured in full pipeline calls (one complete batch traversal
when batching). The profiler runs for the sum of these values instead of --num_iterations.
It does not apply --warmup_calls. It saves a compressed Chrome trace for each rank.
--profile_capture_phase is a separate option for supported model compilation
paths; it records CUDA graph capture under the conditions described by those paths.
--determinism_check compares timed outputs against the first using exact
equality. Use --determinism_check_report_ranks to select which ranks save
failure outputs. This retains complete outputs and adds memory and synchronization costs. The
total elapsed time includes these costs.
xdit --model FLUX.1-dev --prompt "A botanical garden" \
--ulysses_degree 4 --warmup_calls 1 --num_iterations 5 \
--output_directory ./benchmark_results
For profiling, replace the iteration controls with --profile. Use distinct
output directories when comparing configurations.
Compilation
Add --use_torch_compile to a supported model command. Initial compilation adds
startup cost and can use extra memory. Reuse the same input shapes to reduce
the compilation cost per run.
Dataset inference
The CSV must contain a prompt column, and --batch_size is required:
xdit --model FLUX.1-dev --dataset_path ./prompts.csv --batch_size 4 \
--ulysses_degree 4 --output_directory ./dataset_outputs
Output files
The runner saves outputs under --output_directory (default: the current directory).
The default saver writes PNG images or MP4 videos, plus timings.json for a normal
run. Profiling writes profile_trace_rank_{rank}.json.gz; capture-phase traces go
under capture_traces/. Models with custom video/audio savers may write additional
files or use different output names.
Output names normally include the model's output prefix, parallel degrees,
compilation setting, dimensions, optional task, and output index. Reusing the same
output directory and configuration can overwrite files, including timings.json.
Programmatic usage
Run the following as a Python script under torchrun --nproc_per_node=4.
The runner constructor initializes the distributed configuration. initialize()
loads the model. Use the same devices and model access required by the CLI.
from xfuser.runner import xFuserModelRunner
config = {
"model": "FLUX.1-dev",
"prompt": "A beautiful garden with flowers",
"height": 1024,
"width": 1024,
"ulysses_degree": 4,
"num_inference_steps": 50,
"seed": 42,
"output_directory": "./outputs",
"input_images": [],
}
runner = xFuserModelRunner(config)
input_args = runner.preprocess_args(config)
runner.initialize(input_args)
output, timings = runner.run(input_args)
runner.save(output=output, timings=timings)
runner.cleanup()
To profile, replace the run and save calls with:
output, timings, profile = runner.profile(input_args)
runner.save(profile=profile)
The profiling path returns an empty timing list. Save output separately if
needed, and use run() for benchmark timings.