TWIN V2: Scaling Ultra-Long User Behavior Sequence Modeling for Enhanced CTR Prediction at Kuaishou
The significance of modeling long-term user interests for CTR prediction tasks in large-scale recommendation systems is progressively gaining attention among researchers and practitioners. Existing work, such as SIM and TWIN, typically employs a two-stage approach to model long-term user behavior se...
Gespeichert in:
Veröffentlicht in: | arXiv.org 2024-08 |
---|---|
Hauptverfasser: | , , , , , , , , , , , , , , |
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
container_end_page | |
---|---|
container_issue | |
container_start_page | |
container_title | arXiv.org |
container_volume | |
creator | Si, Zihua Guan, Lin Sun, ZhongXiang Zang, Xiaoxue Lu, Jing Hui, Yiqun Cao, Xingchao Yang, Zeyu Zheng, Yichen Leng, Dewei Zheng, Kai Zhang, Chenbin Niu, Yanan Yang, Song Gai, Kun |
description | The significance of modeling long-term user interests for CTR prediction tasks in large-scale recommendation systems is progressively gaining attention among researchers and practitioners. Existing work, such as SIM and TWIN, typically employs a two-stage approach to model long-term user behavior sequences for efficiency concerns. The first stage rapidly retrieves a subset of sequences related to the target item from a long sequence using a search-based mechanism namely the General Search Unit (GSU), while the second stage calculates the interest scores using the Exact Search Unit (ESU) on the retrieved results. Given the extensive length of user behavior sequences spanning the entire life cycle, potentially reaching up to 10^6 in scale, there is currently no effective solution for fully modeling such expansive user interests. To overcome this issue, we introduced TWIN-V2, an enhancement of TWIN, where a divide-and-conquer approach is applied to compress life-cycle behaviors and uncover more accurate and diverse user interests. Specifically, a hierarchical clustering method groups items with similar characteristics in life-cycle behaviors into a single cluster during the offline phase. By limiting the size of clusters, we can compress behavior sequences well beyond the magnitude of 10^5 to a length manageable for online inference in GSU retrieval. Cluster-aware target attention extracts comprehensive and multi-faceted long-term interests of users, thereby making the final recommendation results more accurate and diverse. Extensive offline experiments on a multi-billion-scale industrial dataset and online A/B tests have demonstrated the effectiveness of TWIN-V2. Under an efficient deployment framework, TWIN-V2 has been successfully deployed to the primary traffic that serves hundreds of millions of daily active users at Kuaishou. |
doi_str_mv | 10.48550/arxiv.2407.16357 |
format | Article |
fullrecord | <record><control><sourceid>proquest_arxiv</sourceid><recordid>TN_cdi_arxiv_primary_2407_16357</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>3084093356</sourcerecordid><originalsourceid>FETCH-LOGICAL-a526-9809dc46de1e44edc313e98dcccf88d77ff8deb5b320e7c2adc3bdd3b4d24ca73</originalsourceid><addsrcrecordid>eNotkMtOwzAQRS0kJKrSD2CFJdYpjh-xww6qAhXhIRpgGTn2hLoKcXGSCv6etGU1o6ujq5mD0FlMplwJQS51-HHbKeVETuOECXmERpSxOFKc0hM0ads1IYQmkgrBRsjkH4sn_E6v8NLo2jWf-K3ugo4yv1tbCPgGVnrrfMBL-O6hMYAfvYU9Wg3pvFnpIbR4lr_ilwDWmc75BusOP_TatSvfn6LjStctTP7nGOW383x2H2XPd4vZdRZpQZMoVSS1hicWYuAcrGExg1RZY0yllJWyqpSFUpSMEpCG6oEorWUlt5QbLdkYnR9q9waKTXBfOvwWOxPF3sRAXByITfDDL21XrH0fmuGmghHFScqYSNgfVkFhYw</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>3084093356</pqid></control><display><type>article</type><title>TWIN V2: Scaling Ultra-Long User Behavior Sequence Modeling for Enhanced CTR Prediction at Kuaishou</title><source>arXiv.org</source><source>Free E- Journals</source><creator>Si, Zihua ; Guan, Lin ; Sun, ZhongXiang ; Zang, Xiaoxue ; Lu, Jing ; Hui, Yiqun ; Cao, Xingchao ; Yang, Zeyu ; Zheng, Yichen ; Leng, Dewei ; Zheng, Kai ; Zhang, Chenbin ; Niu, Yanan ; Yang, Song ; Gai, Kun</creator><creatorcontrib>Si, Zihua ; Guan, Lin ; Sun, ZhongXiang ; Zang, Xiaoxue ; Lu, Jing ; Hui, Yiqun ; Cao, Xingchao ; Yang, Zeyu ; Zheng, Yichen ; Leng, Dewei ; Zheng, Kai ; Zhang, Chenbin ; Niu, Yanan ; Yang, Song ; Gai, Kun</creatorcontrib><description>The significance of modeling long-term user interests for CTR prediction tasks in large-scale recommendation systems is progressively gaining attention among researchers and practitioners. Existing work, such as SIM and TWIN, typically employs a two-stage approach to model long-term user behavior sequences for efficiency concerns. The first stage rapidly retrieves a subset of sequences related to the target item from a long sequence using a search-based mechanism namely the General Search Unit (GSU), while the second stage calculates the interest scores using the Exact Search Unit (ESU) on the retrieved results. Given the extensive length of user behavior sequences spanning the entire life cycle, potentially reaching up to 10^6 in scale, there is currently no effective solution for fully modeling such expansive user interests. To overcome this issue, we introduced TWIN-V2, an enhancement of TWIN, where a divide-and-conquer approach is applied to compress life-cycle behaviors and uncover more accurate and diverse user interests. Specifically, a hierarchical clustering method groups items with similar characteristics in life-cycle behaviors into a single cluster during the offline phase. By limiting the size of clusters, we can compress behavior sequences well beyond the magnitude of 10^5 to a length manageable for online inference in GSU retrieval. Cluster-aware target attention extracts comprehensive and multi-faceted long-term interests of users, thereby making the final recommendation results more accurate and diverse. Extensive offline experiments on a multi-billion-scale industrial dataset and online A/B tests have demonstrated the effectiveness of TWIN-V2. Under an efficient deployment framework, TWIN-V2 has been successfully deployed to the primary traffic that serves hundreds of millions of daily active users at Kuaishou.</description><identifier>EISSN: 2331-8422</identifier><identifier>DOI: 10.48550/arxiv.2407.16357</identifier><language>eng</language><publisher>Ithaca: Cornell University Library, arXiv.org</publisher><subject>Attention ; Cluster analysis ; Clustering ; Clusters ; Computer Science - Artificial Intelligence ; Computer Science - Information Retrieval ; Effectiveness ; Modelling ; Recommender systems ; Searching ; User behavior</subject><ispartof>arXiv.org, 2024-08</ispartof><rights>2024. This work is published under http://creativecommons.org/licenses/by-nc-sa/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><rights>http://creativecommons.org/licenses/by-nc-sa/4.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,780,784,885,27925</link.rule.ids><backlink>$$Uhttps://doi.org/10.48550/arXiv.2407.16357$$DView paper in arXiv$$Hfree_for_read</backlink><backlink>$$Uhttps://doi.org/10.1145/3627673.3680030$$DView published paper (Access to full text may be restricted)$$Hfree_for_read</backlink></links><search><creatorcontrib>Si, Zihua</creatorcontrib><creatorcontrib>Guan, Lin</creatorcontrib><creatorcontrib>Sun, ZhongXiang</creatorcontrib><creatorcontrib>Zang, Xiaoxue</creatorcontrib><creatorcontrib>Lu, Jing</creatorcontrib><creatorcontrib>Hui, Yiqun</creatorcontrib><creatorcontrib>Cao, Xingchao</creatorcontrib><creatorcontrib>Yang, Zeyu</creatorcontrib><creatorcontrib>Zheng, Yichen</creatorcontrib><creatorcontrib>Leng, Dewei</creatorcontrib><creatorcontrib>Zheng, Kai</creatorcontrib><creatorcontrib>Zhang, Chenbin</creatorcontrib><creatorcontrib>Niu, Yanan</creatorcontrib><creatorcontrib>Yang, Song</creatorcontrib><creatorcontrib>Gai, Kun</creatorcontrib><title>TWIN V2: Scaling Ultra-Long User Behavior Sequence Modeling for Enhanced CTR Prediction at Kuaishou</title><title>arXiv.org</title><description>The significance of modeling long-term user interests for CTR prediction tasks in large-scale recommendation systems is progressively gaining attention among researchers and practitioners. Existing work, such as SIM and TWIN, typically employs a two-stage approach to model long-term user behavior sequences for efficiency concerns. The first stage rapidly retrieves a subset of sequences related to the target item from a long sequence using a search-based mechanism namely the General Search Unit (GSU), while the second stage calculates the interest scores using the Exact Search Unit (ESU) on the retrieved results. Given the extensive length of user behavior sequences spanning the entire life cycle, potentially reaching up to 10^6 in scale, there is currently no effective solution for fully modeling such expansive user interests. To overcome this issue, we introduced TWIN-V2, an enhancement of TWIN, where a divide-and-conquer approach is applied to compress life-cycle behaviors and uncover more accurate and diverse user interests. Specifically, a hierarchical clustering method groups items with similar characteristics in life-cycle behaviors into a single cluster during the offline phase. By limiting the size of clusters, we can compress behavior sequences well beyond the magnitude of 10^5 to a length manageable for online inference in GSU retrieval. Cluster-aware target attention extracts comprehensive and multi-faceted long-term interests of users, thereby making the final recommendation results more accurate and diverse. Extensive offline experiments on a multi-billion-scale industrial dataset and online A/B tests have demonstrated the effectiveness of TWIN-V2. Under an efficient deployment framework, TWIN-V2 has been successfully deployed to the primary traffic that serves hundreds of millions of daily active users at Kuaishou.</description><subject>Attention</subject><subject>Cluster analysis</subject><subject>Clustering</subject><subject>Clusters</subject><subject>Computer Science - Artificial Intelligence</subject><subject>Computer Science - Information Retrieval</subject><subject>Effectiveness</subject><subject>Modelling</subject><subject>Recommender systems</subject><subject>Searching</subject><subject>User behavior</subject><issn>2331-8422</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><sourceid>ABUWG</sourceid><sourceid>AFKRA</sourceid><sourceid>AZQEC</sourceid><sourceid>BENPR</sourceid><sourceid>CCPQU</sourceid><sourceid>DWQXO</sourceid><sourceid>GOX</sourceid><recordid>eNotkMtOwzAQRS0kJKrSD2CFJdYpjh-xww6qAhXhIRpgGTn2hLoKcXGSCv6etGU1o6ujq5mD0FlMplwJQS51-HHbKeVETuOECXmERpSxOFKc0hM0ads1IYQmkgrBRsjkH4sn_E6v8NLo2jWf-K3ugo4yv1tbCPgGVnrrfMBL-O6hMYAfvYU9Wg3pvFnpIbR4lr_ilwDWmc75BusOP_TatSvfn6LjStctTP7nGOW383x2H2XPd4vZdRZpQZMoVSS1hicWYuAcrGExg1RZY0yllJWyqpSFUpSMEpCG6oEorWUlt5QbLdkYnR9q9waKTXBfOvwWOxPF3sRAXByITfDDL21XrH0fmuGmghHFScqYSNgfVkFhYw</recordid><startdate>20240816</startdate><enddate>20240816</enddate><creator>Si, Zihua</creator><creator>Guan, Lin</creator><creator>Sun, ZhongXiang</creator><creator>Zang, Xiaoxue</creator><creator>Lu, Jing</creator><creator>Hui, Yiqun</creator><creator>Cao, Xingchao</creator><creator>Yang, Zeyu</creator><creator>Zheng, Yichen</creator><creator>Leng, Dewei</creator><creator>Zheng, Kai</creator><creator>Zhang, Chenbin</creator><creator>Niu, Yanan</creator><creator>Yang, Song</creator><creator>Gai, Kun</creator><general>Cornell University Library, arXiv.org</general><scope>8FE</scope><scope>8FG</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>HCIFZ</scope><scope>L6V</scope><scope>M7S</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>PTHSS</scope><scope>AKY</scope><scope>GOX</scope></search><sort><creationdate>20240816</creationdate><title>TWIN V2: Scaling Ultra-Long User Behavior Sequence Modeling for Enhanced CTR Prediction at Kuaishou</title><author>Si, Zihua ; Guan, Lin ; Sun, ZhongXiang ; Zang, Xiaoxue ; Lu, Jing ; Hui, Yiqun ; Cao, Xingchao ; Yang, Zeyu ; Zheng, Yichen ; Leng, Dewei ; Zheng, Kai ; Zhang, Chenbin ; Niu, Yanan ; Yang, Song ; Gai, Kun</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a526-9809dc46de1e44edc313e98dcccf88d77ff8deb5b320e7c2adc3bdd3b4d24ca73</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Attention</topic><topic>Cluster analysis</topic><topic>Clustering</topic><topic>Clusters</topic><topic>Computer Science - Artificial Intelligence</topic><topic>Computer Science - Information Retrieval</topic><topic>Effectiveness</topic><topic>Modelling</topic><topic>Recommender systems</topic><topic>Searching</topic><topic>User behavior</topic><toplevel>online_resources</toplevel><creatorcontrib>Si, Zihua</creatorcontrib><creatorcontrib>Guan, Lin</creatorcontrib><creatorcontrib>Sun, ZhongXiang</creatorcontrib><creatorcontrib>Zang, Xiaoxue</creatorcontrib><creatorcontrib>Lu, Jing</creatorcontrib><creatorcontrib>Hui, Yiqun</creatorcontrib><creatorcontrib>Cao, Xingchao</creatorcontrib><creatorcontrib>Yang, Zeyu</creatorcontrib><creatorcontrib>Zheng, Yichen</creatorcontrib><creatorcontrib>Leng, Dewei</creatorcontrib><creatorcontrib>Zheng, Kai</creatorcontrib><creatorcontrib>Zhang, Chenbin</creatorcontrib><creatorcontrib>Niu, Yanan</creatorcontrib><creatorcontrib>Yang, Song</creatorcontrib><creatorcontrib>Gai, Kun</creatorcontrib><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>Materials Science & Engineering Collection</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Engineering Collection</collection><collection>Engineering Database</collection><collection>Publicly Available Content Database</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>Engineering Collection</collection><collection>arXiv Computer Science</collection><collection>arXiv.org</collection><jtitle>arXiv.org</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Si, Zihua</au><au>Guan, Lin</au><au>Sun, ZhongXiang</au><au>Zang, Xiaoxue</au><au>Lu, Jing</au><au>Hui, Yiqun</au><au>Cao, Xingchao</au><au>Yang, Zeyu</au><au>Zheng, Yichen</au><au>Leng, Dewei</au><au>Zheng, Kai</au><au>Zhang, Chenbin</au><au>Niu, Yanan</au><au>Yang, Song</au><au>Gai, Kun</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>TWIN V2: Scaling Ultra-Long User Behavior Sequence Modeling for Enhanced CTR Prediction at Kuaishou</atitle><jtitle>arXiv.org</jtitle><date>2024-08-16</date><risdate>2024</risdate><eissn>2331-8422</eissn><abstract>The significance of modeling long-term user interests for CTR prediction tasks in large-scale recommendation systems is progressively gaining attention among researchers and practitioners. Existing work, such as SIM and TWIN, typically employs a two-stage approach to model long-term user behavior sequences for efficiency concerns. The first stage rapidly retrieves a subset of sequences related to the target item from a long sequence using a search-based mechanism namely the General Search Unit (GSU), while the second stage calculates the interest scores using the Exact Search Unit (ESU) on the retrieved results. Given the extensive length of user behavior sequences spanning the entire life cycle, potentially reaching up to 10^6 in scale, there is currently no effective solution for fully modeling such expansive user interests. To overcome this issue, we introduced TWIN-V2, an enhancement of TWIN, where a divide-and-conquer approach is applied to compress life-cycle behaviors and uncover more accurate and diverse user interests. Specifically, a hierarchical clustering method groups items with similar characteristics in life-cycle behaviors into a single cluster during the offline phase. By limiting the size of clusters, we can compress behavior sequences well beyond the magnitude of 10^5 to a length manageable for online inference in GSU retrieval. Cluster-aware target attention extracts comprehensive and multi-faceted long-term interests of users, thereby making the final recommendation results more accurate and diverse. Extensive offline experiments on a multi-billion-scale industrial dataset and online A/B tests have demonstrated the effectiveness of TWIN-V2. Under an efficient deployment framework, TWIN-V2 has been successfully deployed to the primary traffic that serves hundreds of millions of daily active users at Kuaishou.</abstract><cop>Ithaca</cop><pub>Cornell University Library, arXiv.org</pub><doi>10.48550/arxiv.2407.16357</doi><oa>free_for_read</oa></addata></record> |
fulltext | fulltext |
identifier | EISSN: 2331-8422 |
ispartof | arXiv.org, 2024-08 |
issn | 2331-8422 |
language | eng |
recordid | cdi_arxiv_primary_2407_16357 |
source | arXiv.org; Free E- Journals |
subjects | Attention Cluster analysis Clustering Clusters Computer Science - Artificial Intelligence Computer Science - Information Retrieval Effectiveness Modelling Recommender systems Searching User behavior |
title | TWIN V2: Scaling Ultra-Long User Behavior Sequence Modeling for Enhanced CTR Prediction at Kuaishou |
url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-24T14%3A22%3A20IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_arxiv&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=TWIN%20V2:%20Scaling%20Ultra-Long%20User%20Behavior%20Sequence%20Modeling%20for%20Enhanced%20CTR%20Prediction%20at%20Kuaishou&rft.jtitle=arXiv.org&rft.au=Si,%20Zihua&rft.date=2024-08-16&rft.eissn=2331-8422&rft_id=info:doi/10.48550/arxiv.2407.16357&rft_dat=%3Cproquest_arxiv%3E3084093356%3C/proquest_arxiv%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=3084093356&rft_id=info:pmid/&rfr_iscdi=true |