UNSUPERVISED PRE-TRAINING OF NEURAL NETWORKS USING GENERATIVE MODELS
In various examples, systems and methods are disclosed relating to generating a response from image and/or video input for image/video-based artificial intelligence (AI) systems and applications. Systems and methods are disclosed for a first model (e.g., a teacher model) distilling its knowledge to...
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Patent |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext bestellen |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | |
|---|---|
| container_issue | |
| container_start_page | |
| container_title | |
| container_volume | |
| creator | Li, Daiqing Kim, Seung Wook Kreis, Karsten Julian Kar, Amlan Fidler, Sanja Ling, Huan Torralba Barriuso, Antonio |
| description | In various examples, systems and methods are disclosed relating to generating a response from image and/or video input for image/video-based artificial intelligence (AI) systems and applications. Systems and methods are disclosed for a first model (e.g., a teacher model) distilling its knowledge to a second model (a student model). The second model receives a downstream image in a downstream task and generates at least one feature. The first model generates first features corresponding to an image which can be a real image or a synthetic image. The second model generates second features using the image as an input to the second model. Loss with respect to first features is determined. The second model is updated using the loss. |
| format | Patent |
| fullrecord | <record><control><sourceid>epo_EVB</sourceid><recordid>TN_cdi_epo_espacenet_US2024256831A1</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>US2024256831A1</sourcerecordid><originalsourceid>FETCH-epo_espacenet_US2024256831A13</originalsourceid><addsrcrecordid>eNrjZHAJ9QsODXANCvMMdnVRCAhy1Q0JcvT08_RzV_B3U_BzDQ1y9AFSIeH-Qd7BCqHBIAl3Vz_XIMcQzzBXBV9_F1efYB4G1rTEnOJUXijNzaDs5hri7KGbWpAfn1pckJicmpdaEh8abGRgZGJkamZhbOhoaEycKgBH-S0r</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>patent</recordtype></control><display><type>patent</type><title>UNSUPERVISED PRE-TRAINING OF NEURAL NETWORKS USING GENERATIVE MODELS</title><source>esp@cenet</source><creator>Li, Daiqing ; Kim, Seung Wook ; Kreis, Karsten Julian ; Kar, Amlan ; Fidler, Sanja ; Ling, Huan ; Torralba Barriuso, Antonio</creator><creatorcontrib>Li, Daiqing ; Kim, Seung Wook ; Kreis, Karsten Julian ; Kar, Amlan ; Fidler, Sanja ; Ling, Huan ; Torralba Barriuso, Antonio</creatorcontrib><description>In various examples, systems and methods are disclosed relating to generating a response from image and/or video input for image/video-based artificial intelligence (AI) systems and applications. Systems and methods are disclosed for a first model (e.g., a teacher model) distilling its knowledge to a second model (a student model). The second model receives a downstream image in a downstream task and generates at least one feature. The first model generates first features corresponding to an image which can be a real image or a synthetic image. The second model generates second features using the image as an input to the second model. Loss with respect to first features is determined. The second model is updated using the loss.</description><language>eng</language><subject>CALCULATING ; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS ; COMPUTING ; COUNTING ; IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ; PHYSICS</subject><creationdate>2024</creationdate><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://worldwide.espacenet.com/publicationDetails/biblio?FT=D&date=20240801&DB=EPODOC&CC=US&NR=2024256831A1$$EHTML$$P50$$Gepo$$Hfree_for_read</linktohtml><link.rule.ids>230,308,776,881,25542,76289</link.rule.ids><linktorsrc>$$Uhttps://worldwide.espacenet.com/publicationDetails/biblio?FT=D&date=20240801&DB=EPODOC&CC=US&NR=2024256831A1$$EView_record_in_European_Patent_Office$$FView_record_in_$$GEuropean_Patent_Office$$Hfree_for_read</linktorsrc></links><search><creatorcontrib>Li, Daiqing</creatorcontrib><creatorcontrib>Kim, Seung Wook</creatorcontrib><creatorcontrib>Kreis, Karsten Julian</creatorcontrib><creatorcontrib>Kar, Amlan</creatorcontrib><creatorcontrib>Fidler, Sanja</creatorcontrib><creatorcontrib>Ling, Huan</creatorcontrib><creatorcontrib>Torralba Barriuso, Antonio</creatorcontrib><title>UNSUPERVISED PRE-TRAINING OF NEURAL NETWORKS USING GENERATIVE MODELS</title><description>In various examples, systems and methods are disclosed relating to generating a response from image and/or video input for image/video-based artificial intelligence (AI) systems and applications. Systems and methods are disclosed for a first model (e.g., a teacher model) distilling its knowledge to a second model (a student model). The second model receives a downstream image in a downstream task and generates at least one feature. The first model generates first features corresponding to an image which can be a real image or a synthetic image. The second model generates second features using the image as an input to the second model. Loss with respect to first features is determined. The second model is updated using the loss.</description><subject>CALCULATING</subject><subject>COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS</subject><subject>COMPUTING</subject><subject>COUNTING</subject><subject>IMAGE DATA PROCESSING OR GENERATION, IN GENERAL</subject><subject>PHYSICS</subject><fulltext>true</fulltext><rsrctype>patent</rsrctype><creationdate>2024</creationdate><recordtype>patent</recordtype><sourceid>EVB</sourceid><recordid>eNrjZHAJ9QsODXANCvMMdnVRCAhy1Q0JcvT08_RzV_B3U_BzDQ1y9AFSIeH-Qd7BCqHBIAl3Vz_XIMcQzzBXBV9_F1efYB4G1rTEnOJUXijNzaDs5hri7KGbWpAfn1pckJicmpdaEh8abGRgZGJkamZhbOhoaEycKgBH-S0r</recordid><startdate>20240801</startdate><enddate>20240801</enddate><creator>Li, Daiqing</creator><creator>Kim, Seung Wook</creator><creator>Kreis, Karsten Julian</creator><creator>Kar, Amlan</creator><creator>Fidler, Sanja</creator><creator>Ling, Huan</creator><creator>Torralba Barriuso, Antonio</creator><scope>EVB</scope></search><sort><creationdate>20240801</creationdate><title>UNSUPERVISED PRE-TRAINING OF NEURAL NETWORKS USING GENERATIVE MODELS</title><author>Li, Daiqing ; Kim, Seung Wook ; Kreis, Karsten Julian ; Kar, Amlan ; Fidler, Sanja ; Ling, Huan ; Torralba Barriuso, Antonio</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-epo_espacenet_US2024256831A13</frbrgroupid><rsrctype>patents</rsrctype><prefilter>patents</prefilter><language>eng</language><creationdate>2024</creationdate><topic>CALCULATING</topic><topic>COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS</topic><topic>COMPUTING</topic><topic>COUNTING</topic><topic>IMAGE DATA PROCESSING OR GENERATION, IN GENERAL</topic><topic>PHYSICS</topic><toplevel>online_resources</toplevel><creatorcontrib>Li, Daiqing</creatorcontrib><creatorcontrib>Kim, Seung Wook</creatorcontrib><creatorcontrib>Kreis, Karsten Julian</creatorcontrib><creatorcontrib>Kar, Amlan</creatorcontrib><creatorcontrib>Fidler, Sanja</creatorcontrib><creatorcontrib>Ling, Huan</creatorcontrib><creatorcontrib>Torralba Barriuso, Antonio</creatorcontrib><collection>esp@cenet</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Li, Daiqing</au><au>Kim, Seung Wook</au><au>Kreis, Karsten Julian</au><au>Kar, Amlan</au><au>Fidler, Sanja</au><au>Ling, Huan</au><au>Torralba Barriuso, Antonio</au><format>patent</format><genre>patent</genre><ristype>GEN</ristype><title>UNSUPERVISED PRE-TRAINING OF NEURAL NETWORKS USING GENERATIVE MODELS</title><date>2024-08-01</date><risdate>2024</risdate><abstract>In various examples, systems and methods are disclosed relating to generating a response from image and/or video input for image/video-based artificial intelligence (AI) systems and applications. Systems and methods are disclosed for a first model (e.g., a teacher model) distilling its knowledge to a second model (a student model). The second model receives a downstream image in a downstream task and generates at least one feature. The first model generates first features corresponding to an image which can be a real image or a synthetic image. The second model generates second features using the image as an input to the second model. Loss with respect to first features is determined. The second model is updated using the loss.</abstract><oa>free_for_read</oa></addata></record> |
| fulltext | fulltext_linktorsrc |
| identifier | |
| ispartof | |
| issn | |
| language | eng |
| recordid | cdi_epo_espacenet_US2024256831A1 |
| source | esp@cenet |
| subjects | CALCULATING COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS COMPUTING COUNTING IMAGE DATA PROCESSING OR GENERATION, IN GENERAL PHYSICS |
| title | UNSUPERVISED PRE-TRAINING OF NEURAL NETWORKS USING GENERATIVE MODELS |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-02-12T20%3A39%3A55IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-epo_EVB&rft_val_fmt=info:ofi/fmt:kev:mtx:patent&rft.genre=patent&rft.au=Li,%20Daiqing&rft.date=2024-08-01&rft_id=info:doi/&rft_dat=%3Cepo_EVB%3EUS2024256831A1%3C/epo_EVB%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true |