Instruction Tuning with GPT-4

Prior work has shown that finetuning large language models (LLMs) using machine-generated instruction-following data enables such models to achieve remarkable zero-shot capabilities on new tasks, and no human-written instructions are needed. In this paper, we present the first attempt to use GPT-4 t...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Hauptverfasser:	Peng, Baolin, Li, Chunyuan, He, Pengcheng, Galley, Michel, Gao, Jianfeng
Format:	Artikel
Sprache:	eng
Schlagworte:	Computer Science - Artificial Intelligence Computer Science - Computation and Language
Online-Zugang:	Volltext bestellen
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

container_end_page
container_issue
container_start_page
container_title
container_volume
creator	Peng, Baolin Li, Chunyuan He, Pengcheng Galley, Michel Gao, Jianfeng
description	Prior work has shown that finetuning large language models (LLMs) using machine-generated instruction-following data enables such models to achieve remarkable zero-shot capabilities on new tasks, and no human-written instructions are needed. In this paper, we present the first attempt to use GPT-4 to generate instruction-following data for LLM finetuning. Our early experiments on instruction-tuned LLaMA models show that the 52K English and Chinese instruction-following data generated by GPT-4 leads to superior zero-shot performance on new tasks to the instruction-following data generated by previous state-of-the-art models. We also collect feedback and comparison data from GPT-4 to enable a comprehensive evaluation and reward model training. We make our data generated using GPT-4 as well as our codebase publicly available.
doi_str_mv	10.48550/arxiv.2304.03277
format	Article
fullrecord	<record><control><sourceid>arxiv_GOX</sourceid><recordid>TN_cdi_arxiv_primary_2304_03277</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2304_03277</sourcerecordid><originalsourceid>FETCH-LOGICAL-a677-3458e6d3cbc17703f1ea32dcc4c1effcf726323c90739343f4385bd25fb2d9d33</originalsourceid><addsrcrecordid>eNotzrsOgjAYQOEuDgZ9AAcjLwCW_i2F0RBFEhId2EnpRZtoNVy8vL0Rnc528iG0iHBIE8bwWrQv-wgJYBpiIJxP0bJwXd8Osrc351eDs-7kP21_9vNjFdAZmhhx6fT8Xw9Vu22V7YPykBfZpgxEzHkAlCU6ViAbGXGOwURaAFFSUhlpY6ThJAYCMsUcUqBgKCSsUYSZhqhUAXho9duOvvre2qto3_XXWY9O-ACTXzZP</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>Instruction Tuning with GPT-4</title><source>arXiv.org</source><creator>Peng, Baolin ; Li, Chunyuan ; He, Pengcheng ; Galley, Michel ; Gao, Jianfeng</creator><creatorcontrib>Peng, Baolin ; Li, Chunyuan ; He, Pengcheng ; Galley, Michel ; Gao, Jianfeng</creatorcontrib><description>Prior work has shown that finetuning large language models (LLMs) using machine-generated instruction-following data enables such models to achieve remarkable zero-shot capabilities on new tasks, and no human-written instructions are needed. In this paper, we present the first attempt to use GPT-4 to generate instruction-following data for LLM finetuning. Our early experiments on instruction-tuned LLaMA models show that the 52K English and Chinese instruction-following data generated by GPT-4 leads to superior zero-shot performance on new tasks to the instruction-following data generated by previous state-of-the-art models. We also collect feedback and comparison data from GPT-4 to enable a comprehensive evaluation and reward model training. We make our data generated using GPT-4 as well as our codebase publicly available.</description><identifier>DOI: 10.48550/arxiv.2304.03277</identifier><language>eng</language><subject>Computer Science - Artificial Intelligence ; Computer Science - Computation and Language</subject><creationdate>2023-04</creationdate><rights>http://creativecommons.org/licenses/by-nc-sa/4.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,780,885</link.rule.ids><linktorsrc>$$Uhttps://arxiv.org/abs/2304.03277$$EView_record_in_Cornell_University$$FView_record_in_$$GCornell_University$$Hfree_for_read</linktorsrc><backlink>$$Uhttps://doi.org/10.48550/arXiv.2304.03277$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Peng, Baolin</creatorcontrib><creatorcontrib>Li, Chunyuan</creatorcontrib><creatorcontrib>He, Pengcheng</creatorcontrib><creatorcontrib>Galley, Michel</creatorcontrib><creatorcontrib>Gao, Jianfeng</creatorcontrib><title>Instruction Tuning with GPT-4</title><description>Prior work has shown that finetuning large language models (LLMs) using machine-generated instruction-following data enables such models to achieve remarkable zero-shot capabilities on new tasks, and no human-written instructions are needed. In this paper, we present the first attempt to use GPT-4 to generate instruction-following data for LLM finetuning. Our early experiments on instruction-tuned LLaMA models show that the 52K English and Chinese instruction-following data generated by GPT-4 leads to superior zero-shot performance on new tasks to the instruction-following data generated by previous state-of-the-art models. We also collect feedback and comparison data from GPT-4 to enable a comprehensive evaluation and reward model training. We make our data generated using GPT-4 as well as our codebase publicly available.</description><subject>Computer Science - Artificial Intelligence</subject><subject>Computer Science - Computation and Language</subject><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><sourceid>GOX</sourceid><recordid>eNotzrsOgjAYQOEuDgZ9AAcjLwCW_i2F0RBFEhId2EnpRZtoNVy8vL0Rnc528iG0iHBIE8bwWrQv-wgJYBpiIJxP0bJwXd8Osrc351eDs-7kP21_9vNjFdAZmhhx6fT8Xw9Vu22V7YPykBfZpgxEzHkAlCU6ViAbGXGOwURaAFFSUhlpY6ThJAYCMsUcUqBgKCSsUYSZhqhUAXho9duOvvre2qto3_XXWY9O-ACTXzZP</recordid><startdate>20230406</startdate><enddate>20230406</enddate><creator>Peng, Baolin</creator><creator>Li, Chunyuan</creator><creator>He, Pengcheng</creator><creator>Galley, Michel</creator><creator>Gao, Jianfeng</creator><scope>AKY</scope><scope>GOX</scope></search><sort><creationdate>20230406</creationdate><title>Instruction Tuning with GPT-4</title><author>Peng, Baolin ; Li, Chunyuan ; He, Pengcheng ; Galley, Michel ; Gao, Jianfeng</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a677-3458e6d3cbc17703f1ea32dcc4c1effcf726323c90739343f4385bd25fb2d9d33</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Computer Science - Artificial Intelligence</topic><topic>Computer Science - Computation and Language</topic><toplevel>online_resources</toplevel><creatorcontrib>Peng, Baolin</creatorcontrib><creatorcontrib>Li, Chunyuan</creatorcontrib><creatorcontrib>He, Pengcheng</creatorcontrib><creatorcontrib>Galley, Michel</creatorcontrib><creatorcontrib>Gao, Jianfeng</creatorcontrib><collection>arXiv Computer Science</collection><collection>arXiv.org</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Peng, Baolin</au><au>Li, Chunyuan</au><au>He, Pengcheng</au><au>Galley, Michel</au><au>Gao, Jianfeng</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Instruction Tuning with GPT-4</atitle><date>2023-04-06</date><risdate>2023</risdate><abstract>Prior work has shown that finetuning large language models (LLMs) using machine-generated instruction-following data enables such models to achieve remarkable zero-shot capabilities on new tasks, and no human-written instructions are needed. In this paper, we present the first attempt to use GPT-4 to generate instruction-following data for LLM finetuning. Our early experiments on instruction-tuned LLaMA models show that the 52K English and Chinese instruction-following data generated by GPT-4 leads to superior zero-shot performance on new tasks to the instruction-following data generated by previous state-of-the-art models. We also collect feedback and comparison data from GPT-4 to enable a comprehensive evaluation and reward model training. We make our data generated using GPT-4 as well as our codebase publicly available.</abstract><doi>10.48550/arxiv.2304.03277</doi><oa>free_for_read</oa></addata></record>
fulltext	fulltext_linktorsrc
identifier	DOI: 10.48550/arxiv.2304.03277
ispartof
issn
language	eng
recordid	cdi_arxiv_primary_2304_03277
source	arXiv.org
subjects	Computer Science - Artificial Intelligence Computer Science - Computation and Language
title	Instruction Tuning with GPT-4
url	https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-07T08%3A56%3A48IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-arxiv_GOX&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Instruction%20Tuning%20with%20GPT-4&rft.au=Peng,%20Baolin&rft.date=2023-04-06&rft_id=info:doi/10.48550/arxiv.2304.03277&rft_dat=%3Carxiv_GOX%3E2304_03277%3C/arxiv_GOX%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true