Profile photo
Reese

Hi, I'm Reese.I build and optimize LLM serving systems.

I'm currently on Baidu's AI Computing team, working on inference scheduling, KV cache management, data transfer, and accelerator backends. Previously, I worked on cloud workload analysis, CPU architecture, and distributed storage at Huawei.

Inference schedulingKV cacheData movement
View projectsRead notes

Current focus

LLM serving systems

I currently focus on scheduling, KV cache lifecycle, data movement, and accelerator backend performance.

  • I look for throughput, latency, and memory bottlenecks in request scheduling and execution.
  • I trace KV cache state through prefill, decode, cache loading, and data transfer.
  • I validate Mooncake Transfer Engine integration with inference backends.
ProjectsCode walkthroughs

Recent projects

Recent engineering projects and ongoing code walkthroughs.

01

SGLang Rust Runtime

Runtime prototype

A Rust prototype that models request handling, scheduling, prefix caching, protocol boundaries, and PD transfer. It is intended for architecture and integration experiments, not as a production replacement for SGLang.

View project scope
02

NexusKV

KV cache platform

Separates KV cache reuse planning, state indexing, data movement, and engine integration into independent components. The baseline implementation and tests are in place; remote transport is still in development.

Read the design notes
03

Mooncake and HiCache internals

Code walkthrough

Follows real read and write paths across SGLang, HiCache, and Mooncake Store to explain cache lookup, host-memory loading, metadata updates, and cross-node transfer.

Read the walkthrough

Recent posts

Notes on LLM serving, distributed systems, engineering practice, and developer tools.

All articles

前缀缓存命中 50%，预填充为什么没有快 2 倍？

从因果注意力的三角计算量出发，逐层推导前缀缓存命中后的剩余注意力计算量、完整预填充的理想加速上限，以及它与真实 TTFT 之间的差距。

Kimi Linear 的 KDA 缓存：SGLang、vLLM 与 Mooncake Store 全链路

从 Kimi Linear 的 KDA 状态出发，追踪 SGLang、vLLM 与 Mooncake Store 的混合缓存流程，并解释细粒度检查点、写时复制、部分分组命中与远端复用。

SGLang v0.5.14 接入 Mooncake Store：缓存页标识、零拷贝与共享 Transfer Engine

沿着一次 KV Cache 的写入与读回，说明 SGLang 如何根据缓存页标识生成 Mooncake Store 对象，以及主机端 KV 缓冲区为何是两套系统的数据交接点。