DeepSpeed — Distributed Training & Hybrid Engine

I contribute to DeepSpeed, focusing on the correctness and reliability of distributed training and Hybrid Engine rollouts.

PR #8264 — Fix ZeRO-3 synchronization during OPSD rollout · Merged August 27, 2026. Fixed a distributed deadlock caused by different ranks stopping generation at different times. Kept decoding and parameter-gather collectives aligned across ranks, added a regression test, and validated the fix on NVIDIA H200 and AMD MI250 GPUs.
PR #8265 — Fallback for unsupported Hybrid Engine policies · Merged August 29, 2026. Prevented partial inference-policy injection for unsupported model architectures, allowing models such as Qwen2.5 to retain native generation. Added CPU unit coverage and validated distributed OPSD training on AMD MI250 GPUs.