Open source
Projects
Stuff I've built in agentic AI and LLM engineering. Reinforcement learning for reasoning agents, plus the runtime, caching and eval tooling you need to actually ship them.
RolloutCore
· RL Rollout RuntimeVersioned RL rollout runtime for vLLM with NCCL hot weight updates, version-pure rollouts, and cache-coherent transitions.
HiQCache
· Quantized SGLang HiCacheINT8 hierarchical KV caching for SGLang HiCache, delivering 43.75% lower host KV memory and 1.78× more L2 cache capacity for Qwen3-8B.
MiniServe
· LLM Inference Runtimea from-scratch LLM serving runtime with continuous batching, chunked prefill, block-based KV management, preemption, and prefix caching.
AgentFlow-Pro
· Agentic RL ResearchProcess-supervised RL that made an 8B model reason better, and the gain carried over to a domain it never trained on.
SmartMemo
· Semantic LLM CacheA semantic cache for LLM agents where a trained classifier decides if a cached answer is safe to reuse, instead of raw cosine similarity.
Orchflow
· Agent Orchestration FrameworkA dependency-free Python framework for readable multi-agent pipelines: sequential, parallel, conditional, and resumable flows.
agenteval
· LLM Evaluation ToolingBehavioral eval for agents: replaces brittle exact-match asserts with repeated-run pass-rate scoring for CI gates.