DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning
Authors: Daya Guo, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiaokang Zhang, Xingkai Yu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bowen Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Guan-Ting Chen, Guowei Li, Hanwei Xu, Honghui Ding, Huazuo Gao, Hui Li, Jianzhong Guo, Jiashi Li, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, Jiaqi Ni, Jian Liang, Jing Chen, Kai Cheng Hu, Kaichao You, Kexin Huang, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang, Liyue Zhang, Lei Xu, L. Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Mingxu Zhou, Meng Li, Miaojun Wang, Mingming Li, Panpan Huang, Qinyu Chen, Qiushi Du, Ruisong Zhang, Runji Wang, R. J. Chen, Rong Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shunfeng Zhou, Shuting Pan, Sansan Li, Shuang Zhou, Tianyu Sun, Wangding Zeng, Wenfeng Liang, Wenjun Gao, Yu Wu, Wentao Zhang, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Nan Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, X.-J Su, Xuheng Lin, Xiaojin Shen, Xiaosha Chen, Xiaowen Sun, Xinnan Song, X.G. Wang, Y. K. Li, Yufei Wang, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo, Yangjie Ou, Yuduan Wang, Yue Gong, Yujia He, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Yuxin Zhu, Yaohui Li, Ying Tang, Yukun Zha, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, Zhen Zhang
Journal: Nature
Publication year: 2025
DOI: 10.1038/s41586-025-09422-z
Abstract
Abstract General reasoning represents a long-standing and formidable challenge in artificial intelligence (AI). Recent breakthroughs, exemplified by large language models (LLMs) 1,2 and chain-of-thought (CoT) prompting 3 , have achieved considerable success on foundational reasoning tasks. However, this success is heavily contingent on extensive human-annotated demonstrations and the capabilities of models are still insufficient for more complex problems. Here we show that the reasoning abilities of LLMs can be incentivized through pure reinforcement learning (RL), obviating the need for human-labelled reasoning trajectories. The proposed RL framework facilitates the emergent development of advanced reasoning patterns, such as self-reflection, verification and dynamic strategy adaptation. Consequently, the trained model achieves superior performance on verifiable tasks such as mathematics, coding competitions and STEM fields, surpassing its counterparts trained through conventional supervised learning on human demonstrations. Moreover, the emergent reasoning patterns exhibited by these large-scale models can be systematically used to guide and enhance the reasoning capabilities of smaller models.