Desiderata for next generation of ML model serving

Desiderata for next generation of ML model serving

Inference is a significant part of ML software infrastructure. Despite the variety of inference frameworks available, the field as a whole can be considered in its early days. This position paper puts forth a range of important qualities that next generation of inference platforms should be aiming f...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: Akoush, Sherif, Paleyes, Andrei, Van Looveren, Arnaud, Cox, Clive
Format: Artikel
Sprache:eng
Schlagworte:
Computer Science - Learning
Computer Science - Software Engineering
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page
container_issue
container_start_page
container_title
container_volume
creator Akoush, Sherif
Paleyes, Andrei
Van Looveren, Arnaud
Cox, Clive
description Inference is a significant part of ML software infrastructure. Despite the variety of inference frameworks available, the field as a whole can be considered in its early days. This position paper puts forth a range of important qualities that next generation of inference platforms should be aiming for. We present our rationale for the importance of each quality, and discuss ways to achieve it in practice. We propose to focus on data-centricity as the overarching design pattern which enables smarter ML system deployment and operation at scale.
doi_str_mv 10.48550/arxiv.2210.14665
format Article
fullrecord <record><control><sourceid>arxiv_GOX</sourceid><recordid>TN_cdi_arxiv_primary_2210_14665</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2210_14665</sourcerecordid><originalsourceid>FETCH-LOGICAL-a675-60e6ea783c2a2b6c0f80df3cdaf675a0c6585015904b8d15ce4032f1499504ef3</originalsourceid><addsrcrecordid>eNotjs1qAjEURrPpolgfoKvmBUZvfm7MLIu2Kox04364JjcS0JmSEbFvX_9WH5wPDkeIdwUT6xFhSuWSzxOtr0BZ5_BV6AUPOXKhE8nUF9nx5ST33N1I7jvZJ7lp5LGPfJADl3Pu9m_iJdFh4PFzR2L7_bWdr6rmZ7mefzYVuRlWDtgxzbwJmvTOBUgeYjIhUrreBMGhR1BYg935qDCwBaOTsnWNYDmZkfh4aO_R7W_JRyp_7S2-vcebf5HKPf8</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>Desiderata for next generation of ML model serving</title><source>arXiv.org</source><creator>Akoush, Sherif ; Paleyes, Andrei ; Van Looveren, Arnaud ; Cox, Clive</creator><creatorcontrib>Akoush, Sherif ; Paleyes, Andrei ; Van Looveren, Arnaud ; Cox, Clive</creatorcontrib><description>Inference is a significant part of ML software infrastructure. Despite the variety of inference frameworks available, the field as a whole can be considered in its early days. This position paper puts forth a range of important qualities that next generation of inference platforms should be aiming for. We present our rationale for the importance of each quality, and discuss ways to achieve it in practice. We propose to focus on data-centricity as the overarching design pattern which enables smarter ML system deployment and operation at scale.</description><identifier>DOI: 10.48550/arxiv.2210.14665</identifier><language>eng</language><subject>Computer Science - Learning ; Computer Science - Software Engineering</subject><creationdate>2022-10</creationdate><rights>http://arxiv.org/licenses/nonexclusive-distrib/1.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,777,882</link.rule.ids><linktorsrc>$$Uhttps://arxiv.org/abs/2210.14665$$EView_record_in_Cornell_University$$FView_record_in_$$GCornell_University$$Hfree_for_read</linktorsrc><backlink>$$Uhttps://doi.org/10.48550/arXiv.2210.14665$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Akoush, Sherif</creatorcontrib><creatorcontrib>Paleyes, Andrei</creatorcontrib><creatorcontrib>Van Looveren, Arnaud</creatorcontrib><creatorcontrib>Cox, Clive</creatorcontrib><title>Desiderata for next generation of ML model serving</title><description>Inference is a significant part of ML software infrastructure. Despite the variety of inference frameworks available, the field as a whole can be considered in its early days. This position paper puts forth a range of important qualities that next generation of inference platforms should be aiming for. We present our rationale for the importance of each quality, and discuss ways to achieve it in practice. We propose to focus on data-centricity as the overarching design pattern which enables smarter ML system deployment and operation at scale.</description><subject>Computer Science - Learning</subject><subject>Computer Science - Software Engineering</subject><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2022</creationdate><recordtype>article</recordtype><sourceid>GOX</sourceid><recordid>eNotjs1qAjEURrPpolgfoKvmBUZvfm7MLIu2Kox04364JjcS0JmSEbFvX_9WH5wPDkeIdwUT6xFhSuWSzxOtr0BZ5_BV6AUPOXKhE8nUF9nx5ST33N1I7jvZJ7lp5LGPfJADl3Pu9m_iJdFh4PFzR2L7_bWdr6rmZ7mefzYVuRlWDtgxzbwJmvTOBUgeYjIhUrreBMGhR1BYg935qDCwBaOTsnWNYDmZkfh4aO_R7W_JRyp_7S2-vcebf5HKPf8</recordid><startdate>20221026</startdate><enddate>20221026</enddate><creator>Akoush, Sherif</creator><creator>Paleyes, Andrei</creator><creator>Van Looveren, Arnaud</creator><creator>Cox, Clive</creator><scope>AKY</scope><scope>GOX</scope></search><sort><creationdate>20221026</creationdate><title>Desiderata for next generation of ML model serving</title><author>Akoush, Sherif ; Paleyes, Andrei ; Van Looveren, Arnaud ; Cox, Clive</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a675-60e6ea783c2a2b6c0f80df3cdaf675a0c6585015904b8d15ce4032f1499504ef3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2022</creationdate><topic>Computer Science - Learning</topic><topic>Computer Science - Software Engineering</topic><toplevel>online_resources</toplevel><creatorcontrib>Akoush, Sherif</creatorcontrib><creatorcontrib>Paleyes, Andrei</creatorcontrib><creatorcontrib>Van Looveren, Arnaud</creatorcontrib><creatorcontrib>Cox, Clive</creatorcontrib><collection>arXiv Computer Science</collection><collection>arXiv.org</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Akoush, Sherif</au><au>Paleyes, Andrei</au><au>Van Looveren, Arnaud</au><au>Cox, Clive</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Desiderata for next generation of ML model serving</atitle><date>2022-10-26</date><risdate>2022</risdate><abstract>Inference is a significant part of ML software infrastructure. Despite the variety of inference frameworks available, the field as a whole can be considered in its early days. This position paper puts forth a range of important qualities that next generation of inference platforms should be aiming for. We present our rationale for the importance of each quality, and discuss ways to achieve it in practice. We propose to focus on data-centricity as the overarching design pattern which enables smarter ML system deployment and operation at scale.</abstract><doi>10.48550/arxiv.2210.14665</doi><oa>free_for_read</oa></addata></record>
fulltext fulltext_linktorsrc
identifier DOI: 10.48550/arxiv.2210.14665
ispartof
issn
language eng
recordid cdi_arxiv_primary_2210_14665
source arXiv.org
subjects Computer Science - Learning
Computer Science - Software Engineering
title Desiderata for next generation of ML model serving
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-18T00%3A31%3A57IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-arxiv_GOX&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Desiderata%20for%20next%20generation%20of%20ML%20model%20serving&rft.au=Akoush,%20Sherif&rft.date=2022-10-26&rft_id=info:doi/10.48550/arxiv.2210.14665&rft_dat=%3Carxiv_GOX%3E2210_14665%3C/arxiv_GOX%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true