Hi, I'm Chenxi Ma, a third-year undergraduate student currently interning at UW Systems Lab under the guidance of Baris Kasikci and Arvind Krishnamurthy.
Special thanks to Kan Zhu for his support and mentorship! He’s graduating soon—an exceptional researcher well worth hiring.
My technical interests include inference runtimes, GPU scheduling, mixture-of-experts models, and communication and computation in distributed inference. I'm also interested in co-design across model architectures, agent harnesses, hardware, and infrastructure.
I'm a contributor to vLLM, working on MoE expert parallelism, dual-batch overlap (DBO), and NIXL-based dispatch in Model Runner V2 (MRV2). With the MRV2 architecture update, DBO will become the default implementation.
I also contribute to SGLang, focusing on the correctness of expert-routing records in two-batch overlap (TBO) and data-parallel inference.
This page brings together my projects and research interests. My personal blog has just launched, where I'll share technical articles and experiment notes.
I'm looking for internship and full-time opportunities in foundation-model (LLM base-model) teams.
Exploring efficient model execution and reliable reasoning and decision-making.
An inference runtime, from request to next token.
A readable re-implementation of DeepInfer's core design, running the real DeepSeek-V4.1-Flash on 4 × B200. Every optimization is recorded in a GitHub issue with its baseline, profile, correctness tests and measured result. Prefill throughput is up about 4× so far, and still 8.4× behind the official engine.
MoE expert parallelism for vLLM.
Fused Triton kernels, 2× NCCL all-to-all, and CUDA Graph decode on B200.
I contribute to vLLM and SGLang, with a focus on MoE expert parallelism, batch overlap, expert-routing records, NIXL dispatch, and KV caches. Selected merged and ongoing contributions are listed below. Thanks to everyone who helped along the way.
Report the first recorded microbatch failure so that follow-on errors do not hide the original problem.
Preserve expert routing at each microbatch's token offset, fixing overwritten routing results.
Size NIXL dispatch from synchronized DP / SP execution sizes to reduce unnecessary work for small batches.
Distribute eligible MLA decode tokens across captured microbatch regions under uneven DP loads, then restore logical order before sampling.
Build on #63556 to return expert-routing records in logical token order after decode staging, with snapshots that remain valid when capture buffers are reused.
Add regression coverage for capped asynchronous KV-load admission and align block sizes in prefill/decode tests. Merged into the collaborator's fix branch.
Map each TBO child's expert-routing records to its rows in the parent batch, preventing shared-buffer overwrites and aligning row offsets for non-EP data parallelism.
Read DP-local expert-routing records consistently across all-to-all backends under DP attention, fixing incorrect expert IDs returned by NIXL on nonzero DP ranks.
