Exploring Prompting Approaches in Legal Textual Entailment

We report explorations into prompt engineering with large pre-trained language models that were not fine-tuned to solve the legal entailment task (Task 4) of the 2023 COLIEE competition. Our most successful strategy used simple text similarity measures to retrieve articles and queries from the train...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:The review of socionetwork strategies 2024, Vol.18 (1), p.75-100
Hauptverfasser: Bilgin, Onur, Fields, Logan, Laverghetta, Antonio, Marji, Zaid, Nighojkar, Animesh, Steinle, Stephen, Licato, John
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page 100
container_issue 1
container_start_page 75
container_title The review of socionetwork strategies
container_volume 18
creator Bilgin, Onur
Fields, Logan
Laverghetta, Antonio
Marji, Zaid
Nighojkar, Animesh
Steinle, Stephen
Licato, John
description We report explorations into prompt engineering with large pre-trained language models that were not fine-tuned to solve the legal entailment task (Task 4) of the 2023 COLIEE competition. Our most successful strategy used simple text similarity measures to retrieve articles and queries from the training set. We report on our efforts to optimize performance with both OpenAI’s GPT-4 and FLaN-T5. We also used an ensemble approach to find the best combination of models and prompts. Finally, we analyze our results and suggest ideas for future improvements.
doi_str_mv 10.1007/s12626-023-00154-y
format Article
fullrecord <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_3041311328</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>3041311328</sourcerecordid><originalsourceid>FETCH-LOGICAL-c257t-593c868662a50cd608d9ba6c43f4f9bd03ac66902be2760206769fea5012eae3</originalsourceid><addsrcrecordid>eNp9kE1rwzAMhs3YYKXrH9gpsLM32U5ke7dSug8obIfejes6XUqaZHYKzb-fuwx6my4S-Hkl8xByz-CRAcinyDhypMAFBWBFTocrMmEKJRVc4DWZ8CI9CSbFLZnFuIdUgkuFbEKel6eubkPV7LLP0B66_jzNuy601n35mFVNtvI7W2drf-qPqS-b3lb1wTf9HbkpbR397K9PyfpluV680dXH6_tivqKOF7KnhRZOoULktgC3RVBbvbHoclHmpd5sQViHqIFvPJcIHFCiLn2CGffWiyl5GNemP30ffezNvj2GJl00AnImGBNcJYqPlAttjMGXpgvVwYbBMDBnS2a0ZJIl82vJDCmUjSHv2qaKl4jSjMkCNSZEjEjszpJ8uFz_Z_EPlGhzxA</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>3041311328</pqid></control><display><type>article</type><title>Exploring Prompting Approaches in Legal Textual Entailment</title><source>SpringerLink Journals - AutoHoldings</source><creator>Bilgin, Onur ; Fields, Logan ; Laverghetta, Antonio ; Marji, Zaid ; Nighojkar, Animesh ; Steinle, Stephen ; Licato, John</creator><creatorcontrib>Bilgin, Onur ; Fields, Logan ; Laverghetta, Antonio ; Marji, Zaid ; Nighojkar, Animesh ; Steinle, Stephen ; Licato, John</creatorcontrib><description>We report explorations into prompt engineering with large pre-trained language models that were not fine-tuned to solve the legal entailment task (Task 4) of the 2023 COLIEE competition. Our most successful strategy used simple text similarity measures to retrieve articles and queries from the training set. We report on our efforts to optimize performance with both OpenAI’s GPT-4 and FLaN-T5. We also used an ensemble approach to find the best combination of models and prompts. Finally, we analyze our results and suggest ideas for future improvements.</description><identifier>ISSN: 2523-3173</identifier><identifier>EISSN: 1867-3236</identifier><identifier>DOI: 10.1007/s12626-023-00154-y</identifier><language>eng</language><publisher>Singapore: Springer Nature Singapore</publisher><subject>Business and Management ; Cognition &amp; reasoning ; Competition policy ; Information Systems Applications (incl.Internet) ; IT in Business ; Language ; Prompt engineering ; Simulation and Modeling ; Text analysis</subject><ispartof>The review of socionetwork strategies, 2024, Vol.18 (1), p.75-100</ispartof><rights>The Author(s), under exclusive licence to Springer Nature Japan KK, part of Springer Nature 2024. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><cites>FETCH-LOGICAL-c257t-593c868662a50cd608d9ba6c43f4f9bd03ac66902be2760206769fea5012eae3</cites><orcidid>0009-0002-1690-4779</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://link.springer.com/content/pdf/10.1007/s12626-023-00154-y$$EPDF$$P50$$Gspringer$$H</linktopdf><linktohtml>$$Uhttps://link.springer.com/10.1007/s12626-023-00154-y$$EHTML$$P50$$Gspringer$$H</linktohtml><link.rule.ids>314,780,784,27924,27925,41488,42557,51319</link.rule.ids></links><search><creatorcontrib>Bilgin, Onur</creatorcontrib><creatorcontrib>Fields, Logan</creatorcontrib><creatorcontrib>Laverghetta, Antonio</creatorcontrib><creatorcontrib>Marji, Zaid</creatorcontrib><creatorcontrib>Nighojkar, Animesh</creatorcontrib><creatorcontrib>Steinle, Stephen</creatorcontrib><creatorcontrib>Licato, John</creatorcontrib><title>Exploring Prompting Approaches in Legal Textual Entailment</title><title>The review of socionetwork strategies</title><addtitle>Rev Socionetwork Strat</addtitle><description>We report explorations into prompt engineering with large pre-trained language models that were not fine-tuned to solve the legal entailment task (Task 4) of the 2023 COLIEE competition. Our most successful strategy used simple text similarity measures to retrieve articles and queries from the training set. We report on our efforts to optimize performance with both OpenAI’s GPT-4 and FLaN-T5. We also used an ensemble approach to find the best combination of models and prompts. Finally, we analyze our results and suggest ideas for future improvements.</description><subject>Business and Management</subject><subject>Cognition &amp; reasoning</subject><subject>Competition policy</subject><subject>Information Systems Applications (incl.Internet)</subject><subject>IT in Business</subject><subject>Language</subject><subject>Prompt engineering</subject><subject>Simulation and Modeling</subject><subject>Text analysis</subject><issn>2523-3173</issn><issn>1867-3236</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><recordid>eNp9kE1rwzAMhs3YYKXrH9gpsLM32U5ke7dSug8obIfejes6XUqaZHYKzb-fuwx6my4S-Hkl8xByz-CRAcinyDhypMAFBWBFTocrMmEKJRVc4DWZ8CI9CSbFLZnFuIdUgkuFbEKel6eubkPV7LLP0B66_jzNuy601n35mFVNtvI7W2drf-qPqS-b3lb1wTf9HbkpbR397K9PyfpluV680dXH6_tivqKOF7KnhRZOoULktgC3RVBbvbHoclHmpd5sQViHqIFvPJcIHFCiLn2CGffWiyl5GNemP30ffezNvj2GJl00AnImGBNcJYqPlAttjMGXpgvVwYbBMDBnS2a0ZJIl82vJDCmUjSHv2qaKl4jSjMkCNSZEjEjszpJ8uFz_Z_EPlGhzxA</recordid><startdate>2024</startdate><enddate>2024</enddate><creator>Bilgin, Onur</creator><creator>Fields, Logan</creator><creator>Laverghetta, Antonio</creator><creator>Marji, Zaid</creator><creator>Nighojkar, Animesh</creator><creator>Steinle, Stephen</creator><creator>Licato, John</creator><general>Springer Nature Singapore</general><general>Springer Nature B.V</general><scope>OQ6</scope><scope>AAYXX</scope><scope>CITATION</scope><orcidid>https://orcid.org/0009-0002-1690-4779</orcidid></search><sort><creationdate>2024</creationdate><title>Exploring Prompting Approaches in Legal Textual Entailment</title><author>Bilgin, Onur ; Fields, Logan ; Laverghetta, Antonio ; Marji, Zaid ; Nighojkar, Animesh ; Steinle, Stephen ; Licato, John</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c257t-593c868662a50cd608d9ba6c43f4f9bd03ac66902be2760206769fea5012eae3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Business and Management</topic><topic>Cognition &amp; reasoning</topic><topic>Competition policy</topic><topic>Information Systems Applications (incl.Internet)</topic><topic>IT in Business</topic><topic>Language</topic><topic>Prompt engineering</topic><topic>Simulation and Modeling</topic><topic>Text analysis</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Bilgin, Onur</creatorcontrib><creatorcontrib>Fields, Logan</creatorcontrib><creatorcontrib>Laverghetta, Antonio</creatorcontrib><creatorcontrib>Marji, Zaid</creatorcontrib><creatorcontrib>Nighojkar, Animesh</creatorcontrib><creatorcontrib>Steinle, Stephen</creatorcontrib><creatorcontrib>Licato, John</creatorcontrib><collection>ECONIS</collection><collection>CrossRef</collection><jtitle>The review of socionetwork strategies</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Bilgin, Onur</au><au>Fields, Logan</au><au>Laverghetta, Antonio</au><au>Marji, Zaid</au><au>Nighojkar, Animesh</au><au>Steinle, Stephen</au><au>Licato, John</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Exploring Prompting Approaches in Legal Textual Entailment</atitle><jtitle>The review of socionetwork strategies</jtitle><stitle>Rev Socionetwork Strat</stitle><date>2024</date><risdate>2024</risdate><volume>18</volume><issue>1</issue><spage>75</spage><epage>100</epage><pages>75-100</pages><issn>2523-3173</issn><eissn>1867-3236</eissn><abstract>We report explorations into prompt engineering with large pre-trained language models that were not fine-tuned to solve the legal entailment task (Task 4) of the 2023 COLIEE competition. Our most successful strategy used simple text similarity measures to retrieve articles and queries from the training set. We report on our efforts to optimize performance with both OpenAI’s GPT-4 and FLaN-T5. We also used an ensemble approach to find the best combination of models and prompts. Finally, we analyze our results and suggest ideas for future improvements.</abstract><cop>Singapore</cop><pub>Springer Nature Singapore</pub><doi>10.1007/s12626-023-00154-y</doi><tpages>26</tpages><orcidid>https://orcid.org/0009-0002-1690-4779</orcidid></addata></record>
fulltext fulltext
identifier ISSN: 2523-3173
ispartof The review of socionetwork strategies, 2024, Vol.18 (1), p.75-100
issn 2523-3173
1867-3236
language eng
recordid cdi_proquest_journals_3041311328
source SpringerLink Journals - AutoHoldings
subjects Business and Management
Cognition & reasoning
Competition policy
Information Systems Applications (incl.Internet)
IT in Business
Language
Prompt engineering
Simulation and Modeling
Text analysis
title Exploring Prompting Approaches in Legal Textual Entailment
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-06T05%3A20%3A46IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Exploring%20Prompting%20Approaches%20in%20Legal%20Textual%20Entailment&rft.jtitle=The%20review%20of%20socionetwork%20strategies&rft.au=Bilgin,%20Onur&rft.date=2024&rft.volume=18&rft.issue=1&rft.spage=75&rft.epage=100&rft.pages=75-100&rft.issn=2523-3173&rft.eissn=1867-3236&rft_id=info:doi/10.1007/s12626-023-00154-y&rft_dat=%3Cproquest_cross%3E3041311328%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=3041311328&rft_id=info:pmid/&rfr_iscdi=true