PUMGPT: A Large Vision-Language Model for Product Understanding

E-commerce platforms benefit from accurate product understanding to enhance user experience and operational efficiency. Traditional methods often focus on isolated tasks such as attribute extraction or categorization, posing adaptability issues to evolving tasks and leading to usability challenges w...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: Xue, Wei, Guo, Zongyi, Cui, Baoliang, Xing, Zheng, Zeng, Xiaoyi, Wang, Xiufei, Wu, Shuhui, Lu, Weiming
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page
container_issue
container_start_page
container_title
container_volume
creator Xue, Wei
Guo, Zongyi
Cui, Baoliang
Xing, Zheng
Zeng, Xiaoyi
Wang, Xiufei
Wu, Shuhui
Lu, Weiming
description E-commerce platforms benefit from accurate product understanding to enhance user experience and operational efficiency. Traditional methods often focus on isolated tasks such as attribute extraction or categorization, posing adaptability issues to evolving tasks and leading to usability challenges with noisy data from the internet. Current Large Vision Language Models (LVLMs) lack domain-specific fine-tuning, thus falling short in precision and instruction following. To address these issues, we introduce PumGPT, the first e-commerce specialized LVLM designed for multi-modal product understanding tasks. We collected and curated a dataset of over one million products from AliExpress, filtering out non-inferable attributes using a universal hallucination detection framework, resulting in 663k high-quality data samples. PumGPT focuses on five essential tasks aimed at enhancing workflows for e-commerce platforms and retailers. We also introduce PumBench, a benchmark to evaluate product understanding across LVLMs. Our experiments show that PumGPT outperforms five other open-source LVLMs and GPT-4V in product understanding tasks. We also conduct extensive analytical experiments to delve deeply into the superiority of PumGPT, demonstrating the necessity for a specialized model in the e-commerce domain.
doi_str_mv 10.48550/arxiv.2308.09568
format Article
fullrecord <record><control><sourceid>arxiv_GOX</sourceid><recordid>TN_cdi_arxiv_primary_2308_09568</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2308_09568</sourcerecordid><originalsourceid>FETCH-LOGICAL-a678-ec98f9271d58cba86a978544f92f9c861550e4e27aad64f2800b7e04155a06903</originalsourceid><addsrcrecordid>eNotj71OwzAUhb0woMIDMOEXSLhJ_HPdBVUVFKRUZEi7RrexHVkqDnJaBG9PKExH5xuOzsfYXQG5QCnhgdJX-MzLCjAHIxVes8dmt9007ZKveE1pcHwfpjDGrKY4nGnu29G6I_dj4k0a7bk_8V20Lk0nijbE4YZdeTpO7vY_F6x9fmrXL1n9tnldr-qMlMbM9Qa9KXVhJfYHQkVGoxRiZt70qIr5nBOu1ERWCV8iwEE7EDMnUAaqBbv_m70YdB8pvFP67n5NuotJ9QO41EGS</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>PUMGPT: A Large Vision-Language Model for Product Understanding</title><source>arXiv.org</source><creator>Xue, Wei ; Guo, Zongyi ; Cui, Baoliang ; Xing, Zheng ; Zeng, Xiaoyi ; Wang, Xiufei ; Wu, Shuhui ; Lu, Weiming</creator><creatorcontrib>Xue, Wei ; Guo, Zongyi ; Cui, Baoliang ; Xing, Zheng ; Zeng, Xiaoyi ; Wang, Xiufei ; Wu, Shuhui ; Lu, Weiming</creatorcontrib><description>E-commerce platforms benefit from accurate product understanding to enhance user experience and operational efficiency. Traditional methods often focus on isolated tasks such as attribute extraction or categorization, posing adaptability issues to evolving tasks and leading to usability challenges with noisy data from the internet. Current Large Vision Language Models (LVLMs) lack domain-specific fine-tuning, thus falling short in precision and instruction following. To address these issues, we introduce PumGPT, the first e-commerce specialized LVLM designed for multi-modal product understanding tasks. We collected and curated a dataset of over one million products from AliExpress, filtering out non-inferable attributes using a universal hallucination detection framework, resulting in 663k high-quality data samples. PumGPT focuses on five essential tasks aimed at enhancing workflows for e-commerce platforms and retailers. We also introduce PumBench, a benchmark to evaluate product understanding across LVLMs. Our experiments show that PumGPT outperforms five other open-source LVLMs and GPT-4V in product understanding tasks. We also conduct extensive analytical experiments to delve deeply into the superiority of PumGPT, demonstrating the necessity for a specialized model in the e-commerce domain.</description><identifier>DOI: 10.48550/arxiv.2308.09568</identifier><language>eng</language><subject>Computer Science - Computation and Language ; Computer Science - Computer Vision and Pattern Recognition</subject><creationdate>2023-08</creationdate><rights>http://arxiv.org/licenses/nonexclusive-distrib/1.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,780,885</link.rule.ids><linktorsrc>$$Uhttps://arxiv.org/abs/2308.09568$$EView_record_in_Cornell_University$$FView_record_in_$$GCornell_University$$Hfree_for_read</linktorsrc><backlink>$$Uhttps://doi.org/10.48550/arXiv.2308.09568$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Xue, Wei</creatorcontrib><creatorcontrib>Guo, Zongyi</creatorcontrib><creatorcontrib>Cui, Baoliang</creatorcontrib><creatorcontrib>Xing, Zheng</creatorcontrib><creatorcontrib>Zeng, Xiaoyi</creatorcontrib><creatorcontrib>Wang, Xiufei</creatorcontrib><creatorcontrib>Wu, Shuhui</creatorcontrib><creatorcontrib>Lu, Weiming</creatorcontrib><title>PUMGPT: A Large Vision-Language Model for Product Understanding</title><description>E-commerce platforms benefit from accurate product understanding to enhance user experience and operational efficiency. Traditional methods often focus on isolated tasks such as attribute extraction or categorization, posing adaptability issues to evolving tasks and leading to usability challenges with noisy data from the internet. Current Large Vision Language Models (LVLMs) lack domain-specific fine-tuning, thus falling short in precision and instruction following. To address these issues, we introduce PumGPT, the first e-commerce specialized LVLM designed for multi-modal product understanding tasks. We collected and curated a dataset of over one million products from AliExpress, filtering out non-inferable attributes using a universal hallucination detection framework, resulting in 663k high-quality data samples. PumGPT focuses on five essential tasks aimed at enhancing workflows for e-commerce platforms and retailers. We also introduce PumBench, a benchmark to evaluate product understanding across LVLMs. Our experiments show that PumGPT outperforms five other open-source LVLMs and GPT-4V in product understanding tasks. We also conduct extensive analytical experiments to delve deeply into the superiority of PumGPT, demonstrating the necessity for a specialized model in the e-commerce domain.</description><subject>Computer Science - Computation and Language</subject><subject>Computer Science - Computer Vision and Pattern Recognition</subject><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><sourceid>GOX</sourceid><recordid>eNotj71OwzAUhb0woMIDMOEXSLhJ_HPdBVUVFKRUZEi7RrexHVkqDnJaBG9PKExH5xuOzsfYXQG5QCnhgdJX-MzLCjAHIxVes8dmt9007ZKveE1pcHwfpjDGrKY4nGnu29G6I_dj4k0a7bk_8V20Lk0nijbE4YZdeTpO7vY_F6x9fmrXL1n9tnldr-qMlMbM9Qa9KXVhJfYHQkVGoxRiZt70qIr5nBOu1ERWCV8iwEE7EDMnUAaqBbv_m70YdB8pvFP67n5NuotJ9QO41EGS</recordid><startdate>20230818</startdate><enddate>20230818</enddate><creator>Xue, Wei</creator><creator>Guo, Zongyi</creator><creator>Cui, Baoliang</creator><creator>Xing, Zheng</creator><creator>Zeng, Xiaoyi</creator><creator>Wang, Xiufei</creator><creator>Wu, Shuhui</creator><creator>Lu, Weiming</creator><scope>AKY</scope><scope>GOX</scope></search><sort><creationdate>20230818</creationdate><title>PUMGPT: A Large Vision-Language Model for Product Understanding</title><author>Xue, Wei ; Guo, Zongyi ; Cui, Baoliang ; Xing, Zheng ; Zeng, Xiaoyi ; Wang, Xiufei ; Wu, Shuhui ; Lu, Weiming</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a678-ec98f9271d58cba86a978544f92f9c861550e4e27aad64f2800b7e04155a06903</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Computer Science - Computation and Language</topic><topic>Computer Science - Computer Vision and Pattern Recognition</topic><toplevel>online_resources</toplevel><creatorcontrib>Xue, Wei</creatorcontrib><creatorcontrib>Guo, Zongyi</creatorcontrib><creatorcontrib>Cui, Baoliang</creatorcontrib><creatorcontrib>Xing, Zheng</creatorcontrib><creatorcontrib>Zeng, Xiaoyi</creatorcontrib><creatorcontrib>Wang, Xiufei</creatorcontrib><creatorcontrib>Wu, Shuhui</creatorcontrib><creatorcontrib>Lu, Weiming</creatorcontrib><collection>arXiv Computer Science</collection><collection>arXiv.org</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Xue, Wei</au><au>Guo, Zongyi</au><au>Cui, Baoliang</au><au>Xing, Zheng</au><au>Zeng, Xiaoyi</au><au>Wang, Xiufei</au><au>Wu, Shuhui</au><au>Lu, Weiming</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>PUMGPT: A Large Vision-Language Model for Product Understanding</atitle><date>2023-08-18</date><risdate>2023</risdate><abstract>E-commerce platforms benefit from accurate product understanding to enhance user experience and operational efficiency. Traditional methods often focus on isolated tasks such as attribute extraction or categorization, posing adaptability issues to evolving tasks and leading to usability challenges with noisy data from the internet. Current Large Vision Language Models (LVLMs) lack domain-specific fine-tuning, thus falling short in precision and instruction following. To address these issues, we introduce PumGPT, the first e-commerce specialized LVLM designed for multi-modal product understanding tasks. We collected and curated a dataset of over one million products from AliExpress, filtering out non-inferable attributes using a universal hallucination detection framework, resulting in 663k high-quality data samples. PumGPT focuses on five essential tasks aimed at enhancing workflows for e-commerce platforms and retailers. We also introduce PumBench, a benchmark to evaluate product understanding across LVLMs. Our experiments show that PumGPT outperforms five other open-source LVLMs and GPT-4V in product understanding tasks. We also conduct extensive analytical experiments to delve deeply into the superiority of PumGPT, demonstrating the necessity for a specialized model in the e-commerce domain.</abstract><doi>10.48550/arxiv.2308.09568</doi><oa>free_for_read</oa></addata></record>
fulltext fulltext_linktorsrc
identifier DOI: 10.48550/arxiv.2308.09568
ispartof
issn
language eng
recordid cdi_arxiv_primary_2308_09568
source arXiv.org
subjects Computer Science - Computation and Language
Computer Science - Computer Vision and Pattern Recognition
title PUMGPT: A Large Vision-Language Model for Product Understanding
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-25T03%3A45%3A42IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-arxiv_GOX&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=PUMGPT:%20A%20Large%20Vision-Language%20Model%20for%20Product%20Understanding&rft.au=Xue,%20Wei&rft.date=2023-08-18&rft_id=info:doi/10.48550/arxiv.2308.09568&rft_dat=%3Carxiv_GOX%3E2308_09568%3C/arxiv_GOX%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true