From distributed storage and CPU performance analysis to LLM serving.

I currently optimize LLM inference engines, focusing on scheduling, KV cache management, cross-node data movement, and accelerator backends.

Earlier in my career, I worked on high availability and cluster management for distributed storage, then on cloud workload analysis and CPU architecture.

This site collects my engineering projects, notes from reading source code, and lessons from debugging and validation work.

Experience

My work has taken me from distributed storage and computer architecture to LLM inference.

2025.06.30 — Present
Baidu
AI Computing
Software Engineer, LLM Inference
Optimize LLM inference engine performance.
2023.07 — 2024.07
Huawei Cloud
Shuhai Lab
Research Engineer, Computer Architecture
Built performance tooling for cloud workloads and analyzed instruction streams and microarchitectural behavior.
2022.05 — 2023.07
Huawei Data Storage
Infrastructure Development Department
Software Engineer, Distributed Storage
Developed high-availability and cluster-management components for distributed storage products.

Areas of focus

Scheduling, cache management, data movement, and accelerator backends.

Inference engines

Scheduling and execution

Analyze performance across request scheduling, operators, kernels, and backend execution.

Throughput and latency · Operator and kernel paths · Prefill and decode behavior

KV cache

Cache management

Study prefix reuse, KV page lifecycle, memory residency, and loading across storage tiers.

Prefix cache behavior · KV page lifecycle · Cache hits and residency

Mooncake Transfer Engine

Data movement

Work on transfer planning, memory registration, descriptors, and readiness checks for KV data.

KV transfer planning · Descriptors and checksums · Readiness checks

Accelerator backends

Backend performance

Adapt and optimize inference paths for different accelerators while keeping device-specific code isolated.

Accelerator execution paths · Device-specific code · Backend interfaces