A Case Study on the Diminishing Popularity of Encoder-Only Architectures in Machine Learning Models

A Case Study on the Diminishing Popularity of Encoder-Only Architectures in Machine Learning Models

This paper examines the shift from encoder-only to decoder and encoder-decoder models in machine learning, highlighting the decline in popularity of encoder-only architectures. It explores the reasons behind this trend, such as the advancements in decoder models that offer superior generative capabi...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:International journal of innovative technology and exploring engineering 2024-03, Vol.13 (4), p.22-27
Hauptverfasser: Sridhar, Praveen Kumar, Srinivasan, Nitin, Kumar, Adithyan Arun, Rajendran, Gowthamaraj, Perumalsamy, Kishore Kumar
Format: Artikel
Sprache:eng
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page 27
container_issue 4
container_start_page 22
container_title International journal of innovative technology and exploring engineering
container_volume 13
creator Sridhar, Praveen Kumar
Srinivasan, Nitin
Kumar, Adithyan Arun
Rajendran, Gowthamaraj
Perumalsamy, Kishore Kumar
description This paper examines the shift from encoder-only to decoder and encoder-decoder models in machine learning, highlighting the decline in popularity of encoder-only architectures. It explores the reasons behind this trend, such as the advancements in decoder models that offer superior generative capabilities, flexibility across various domains, and enhancements in unsupervised learning techniques. The study also discusses the role of prompting techniques in simplifying model architectures and enhancing model versatility. By analyzing the evolution, applications, and shifting preferences within the research community and industry, this paper aims to provide insights into the changing landscape of machine learning model architectures.
doi_str_mv 10.35940/ijitee.D9827.13040324
format Article
fullrecord <record><control><sourceid>crossref</sourceid><recordid>TN_cdi_crossref_primary_10_35940_ijitee_D9827_13040324</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>10_35940_ijitee_D9827_13040324</sourcerecordid><originalsourceid>FETCH-LOGICAL-c1004-53c288f32d5f8b7f9644b42ea5448ce110aafbd0c0d6f44eb67b7b57d717c5b03</originalsourceid><addsrcrecordid>eNpNkN1KAzEQhYMoWGpfQfICWyd_m-xlaasWWiqo10uSTWzKNluS7UXf3rVV8GqGOWfOgQ-hRwJTJioOT2Efeuemi0pROSUMODDKb9CIUqkKBlLc_tvv0STnPQAQxokqqxGyMzzX2eH3_tSccRdxv3N4EQ4hhrwL8Qu_dcdTq1PoB9XjZbRd41Kxje0Zz5LdDeW2PyWXcYh4o4dDdHjtdIo_z5vB3OYHdOd1m93kd47R5_PyY_5arLcvq_lsXVgCwAvBLFXKM9oIr4z0Vcm54dRpwbmyjhDQ2psGLDSl59yZUhpphGwkkVYYYGNUXnNt6nJOztfHFA46nWsC9YVWfaVVX2jVf7TYN9wwYAo</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>A Case Study on the Diminishing Popularity of Encoder-Only Architectures in Machine Learning Models</title><source>EZB-FREE-00999 freely available EZB journals</source><creator>Sridhar, Praveen Kumar ; Srinivasan, Nitin ; Kumar, Adithyan Arun ; Rajendran, Gowthamaraj ; Perumalsamy, Kishore Kumar</creator><creatorcontrib>Sridhar, Praveen Kumar ; Srinivasan, Nitin ; Kumar, Adithyan Arun ; Rajendran, Gowthamaraj ; Perumalsamy, Kishore Kumar ; Department of Information Security, Carnegie Mellon University, San Jose, United States ; Department of Computer Science, Carnegie Mellon University, San Jose, United States ; Department of Computer Science, University of Massachusetts Amherst, Sunnyvale, United States ; Department of Data Science, Northeastern University, San Jose, United States</creatorcontrib><description>This paper examines the shift from encoder-only to decoder and encoder-decoder models in machine learning, highlighting the decline in popularity of encoder-only architectures. It explores the reasons behind this trend, such as the advancements in decoder models that offer superior generative capabilities, flexibility across various domains, and enhancements in unsupervised learning techniques. The study also discusses the role of prompting techniques in simplifying model architectures and enhancing model versatility. By analyzing the evolution, applications, and shifting preferences within the research community and industry, this paper aims to provide insights into the changing landscape of machine learning model architectures.</description><identifier>ISSN: 2278-3075</identifier><identifier>EISSN: 2278-3075</identifier><identifier>DOI: 10.35940/ijitee.D9827.13040324</identifier><language>eng</language><ispartof>International journal of innovative technology and exploring engineering, 2024-03, Vol.13 (4), p.22-27</ispartof><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c1004-53c288f32d5f8b7f9644b42ea5448ce110aafbd0c0d6f44eb67b7b57d717c5b03</citedby><cites>FETCH-LOGICAL-c1004-53c288f32d5f8b7f9644b42ea5448ce110aafbd0c0d6f44eb67b7b57d717c5b03</cites><orcidid>0000-0001-9790-2657 ; 0009-0001-6472-4441 ; 0009-0001-6486-1614 ; 0000-0001-9422-3907 ; 0009-0009-5661-7644</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>314,780,784,27924,27925</link.rule.ids></links><search><creatorcontrib>Sridhar, Praveen Kumar</creatorcontrib><creatorcontrib>Srinivasan, Nitin</creatorcontrib><creatorcontrib>Kumar, Adithyan Arun</creatorcontrib><creatorcontrib>Rajendran, Gowthamaraj</creatorcontrib><creatorcontrib>Perumalsamy, Kishore Kumar</creatorcontrib><creatorcontrib>Department of Information Security, Carnegie Mellon University, San Jose, United States</creatorcontrib><creatorcontrib>Department of Computer Science, Carnegie Mellon University, San Jose, United States</creatorcontrib><creatorcontrib>Department of Computer Science, University of Massachusetts Amherst, Sunnyvale, United States</creatorcontrib><creatorcontrib>Department of Data Science, Northeastern University, San Jose, United States</creatorcontrib><title>A Case Study on the Diminishing Popularity of Encoder-Only Architectures in Machine Learning Models</title><title>International journal of innovative technology and exploring engineering</title><description>This paper examines the shift from encoder-only to decoder and encoder-decoder models in machine learning, highlighting the decline in popularity of encoder-only architectures. It explores the reasons behind this trend, such as the advancements in decoder models that offer superior generative capabilities, flexibility across various domains, and enhancements in unsupervised learning techniques. The study also discusses the role of prompting techniques in simplifying model architectures and enhancing model versatility. By analyzing the evolution, applications, and shifting preferences within the research community and industry, this paper aims to provide insights into the changing landscape of machine learning model architectures.</description><issn>2278-3075</issn><issn>2278-3075</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><recordid>eNpNkN1KAzEQhYMoWGpfQfICWyd_m-xlaasWWiqo10uSTWzKNluS7UXf3rVV8GqGOWfOgQ-hRwJTJioOT2Efeuemi0pROSUMODDKb9CIUqkKBlLc_tvv0STnPQAQxokqqxGyMzzX2eH3_tSccRdxv3N4EQ4hhrwL8Qu_dcdTq1PoB9XjZbRd41Kxje0Zz5LdDeW2PyWXcYh4o4dDdHjtdIo_z5vB3OYHdOd1m93kd47R5_PyY_5arLcvq_lsXVgCwAvBLFXKM9oIr4z0Vcm54dRpwbmyjhDQ2psGLDSl59yZUhpphGwkkVYYYGNUXnNt6nJOztfHFA46nWsC9YVWfaVVX2jVf7TYN9wwYAo</recordid><startdate>20240330</startdate><enddate>20240330</enddate><creator>Sridhar, Praveen Kumar</creator><creator>Srinivasan, Nitin</creator><creator>Kumar, Adithyan Arun</creator><creator>Rajendran, Gowthamaraj</creator><creator>Perumalsamy, Kishore Kumar</creator><scope>AAYXX</scope><scope>CITATION</scope><orcidid>https://orcid.org/0000-0001-9790-2657</orcidid><orcidid>https://orcid.org/0009-0001-6472-4441</orcidid><orcidid>https://orcid.org/0009-0001-6486-1614</orcidid><orcidid>https://orcid.org/0000-0001-9422-3907</orcidid><orcidid>https://orcid.org/0009-0009-5661-7644</orcidid></search><sort><creationdate>20240330</creationdate><title>A Case Study on the Diminishing Popularity of Encoder-Only Architectures in Machine Learning Models</title><author>Sridhar, Praveen Kumar ; Srinivasan, Nitin ; Kumar, Adithyan Arun ; Rajendran, Gowthamaraj ; Perumalsamy, Kishore Kumar</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c1004-53c288f32d5f8b7f9644b42ea5448ce110aafbd0c0d6f44eb67b7b57d717c5b03</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><toplevel>online_resources</toplevel><creatorcontrib>Sridhar, Praveen Kumar</creatorcontrib><creatorcontrib>Srinivasan, Nitin</creatorcontrib><creatorcontrib>Kumar, Adithyan Arun</creatorcontrib><creatorcontrib>Rajendran, Gowthamaraj</creatorcontrib><creatorcontrib>Perumalsamy, Kishore Kumar</creatorcontrib><creatorcontrib>Department of Information Security, Carnegie Mellon University, San Jose, United States</creatorcontrib><creatorcontrib>Department of Computer Science, Carnegie Mellon University, San Jose, United States</creatorcontrib><creatorcontrib>Department of Computer Science, University of Massachusetts Amherst, Sunnyvale, United States</creatorcontrib><creatorcontrib>Department of Data Science, Northeastern University, San Jose, United States</creatorcontrib><collection>CrossRef</collection><jtitle>International journal of innovative technology and exploring engineering</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Sridhar, Praveen Kumar</au><au>Srinivasan, Nitin</au><au>Kumar, Adithyan Arun</au><au>Rajendran, Gowthamaraj</au><au>Perumalsamy, Kishore Kumar</au><aucorp>Department of Information Security, Carnegie Mellon University, San Jose, United States</aucorp><aucorp>Department of Computer Science, Carnegie Mellon University, San Jose, United States</aucorp><aucorp>Department of Computer Science, University of Massachusetts Amherst, Sunnyvale, United States</aucorp><aucorp>Department of Data Science, Northeastern University, San Jose, United States</aucorp><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>A Case Study on the Diminishing Popularity of Encoder-Only Architectures in Machine Learning Models</atitle><jtitle>International journal of innovative technology and exploring engineering</jtitle><date>2024-03-30</date><risdate>2024</risdate><volume>13</volume><issue>4</issue><spage>22</spage><epage>27</epage><pages>22-27</pages><issn>2278-3075</issn><eissn>2278-3075</eissn><abstract>This paper examines the shift from encoder-only to decoder and encoder-decoder models in machine learning, highlighting the decline in popularity of encoder-only architectures. It explores the reasons behind this trend, such as the advancements in decoder models that offer superior generative capabilities, flexibility across various domains, and enhancements in unsupervised learning techniques. The study also discusses the role of prompting techniques in simplifying model architectures and enhancing model versatility. By analyzing the evolution, applications, and shifting preferences within the research community and industry, this paper aims to provide insights into the changing landscape of machine learning model architectures.</abstract><doi>10.35940/ijitee.D9827.13040324</doi><tpages>6</tpages><orcidid>https://orcid.org/0000-0001-9790-2657</orcidid><orcidid>https://orcid.org/0009-0001-6472-4441</orcidid><orcidid>https://orcid.org/0009-0001-6486-1614</orcidid><orcidid>https://orcid.org/0000-0001-9422-3907</orcidid><orcidid>https://orcid.org/0009-0009-5661-7644</orcidid></addata></record>
fulltext fulltext
identifier ISSN: 2278-3075
ispartof International journal of innovative technology and exploring engineering, 2024-03, Vol.13 (4), p.22-27
issn 2278-3075
2278-3075
language eng
recordid cdi_crossref_primary_10_35940_ijitee_D9827_13040324
source EZB-FREE-00999 freely available EZB journals
title A Case Study on the Diminishing Popularity of Encoder-Only Architectures in Machine Learning Models
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-02T07%3A22%3A00IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-crossref&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=A%20Case%20Study%20on%20the%20Diminishing%20Popularity%20of%20Encoder-Only%20Architectures%20in%20Machine%20Learning%20Models&rft.jtitle=International%20journal%20of%20innovative%20technology%20and%20exploring%20engineering&rft.au=Sridhar,%20Praveen%20Kumar&rft.aucorp=Department%20of%20Information%20Security,%20Carnegie%20Mellon%20University,%20San%20Jose,%20United%20States&rft.date=2024-03-30&rft.volume=13&rft.issue=4&rft.spage=22&rft.epage=27&rft.pages=22-27&rft.issn=2278-3075&rft.eissn=2278-3075&rft_id=info:doi/10.35940/ijitee.D9827.13040324&rft_dat=%3Ccrossref%3E10_35940_ijitee_D9827_13040324%3C/crossref%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true