About Me

I am a Ph.D. student at UC Berkeley, advised by Prof. Ion Stoica. I am broadly interested in LLM inference and AI infra.

woosuk.kwon[at]berkeley.edu

Education

  • Ph.D. in Computer Science

    UC Berkeley, 2021 - Present

  • B.S. in Computer Science and Mathematics

    Seoul National University, 2015 - 2021

Current Projects

vLLM
A high-throughput and memory-efficient inference and serving engine for LLMs.
GitHub Blog
vLLM

Publications

Gemma 2: Improving Open Language Models at a Practical Size
Gemma Team
arXiv 2024
PDF
Optimizing Speculative Decoding for Serving Large Language Models Using Goodput
Xiaoxuan Liu, Cade Daniel, Langxiang Hu, Woosuk Kwon, Zhuohan Li, Xiangxi Mo, Alvin Cheung, Zhijie Deng, Ion Stoica, Hao Zhang
arXiv 2024
PDF Code
Efficient Memory Management for Large Language Model Serving with PagedAttention
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, Ion Stoica
SOSP 2023
PDF Code
SkyPilot: An Intercloud Broker for Sky Computing
Zongheng Yang, Zhanghao Wu, Michael Luo, Wei-Lin Chiang, Romil Bhardwaj, Woosuk Kwon, Siyuan Zhuang, Frank Sifei Luan, Gautam Mittal, Scott Shenker, Ion Stoica
NSDI 2023
PDF Code
A Fast Post-Training Pruning Framework for Transformers
Woosuk Kwon, Sehoon Kim, Michael W. Mahoney, Joseph Hassoun, Kurt Keutzer, Amir Gholami
NeurIPS 2022
PDF Code
Learned Token Pruning for Transformers
Sehoon Kim, Sheng Shen, David Thorsley, Amir Gholami, Woosuk Kwon, Joseph Hassoun, Kurt Keutzer
KDD 2022
PDF Code
Nimble: Lightweight and Parallel GPU Task Scheduling for Deep Learning
Woosuk Kwon, Gyeong-In Yu, Eunji Jeong, Byung-Gon Chun
NeurIPS 2020 (Spotlight)
PDF Code
Graphene: Strong yet Lightweight Row Hammer Protection
Yeonhong Park, Woosuk Kwon, Eojin Lee, Tae Jun Ham, Jung Ho Ahn, Jae W. Lee
MICRO 2020 (IEEE Micro Top Picks Honorable Mention)
PDF