Stability of Gated Recurrent Unit Neural Networks: Convex Combination Formulation Approach
In this paper, a particular discrete-time nonlinear and time-invariant system represented as a vector difference equation is analyzed for its stability properties. The motivation for analyzing this particular system is that it models gated recurrent unit neural networks commonly used and well known...
Gespeichert in:
| Veröffentlicht in: | Journal of optimization theory and applications 2021, Vol.188 (1), p.291-306 |
|---|---|
| Hauptverfasser: | , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 306 |
|---|---|
| container_issue | 1 |
| container_start_page | 291 |
| container_title | Journal of optimization theory and applications |
| container_volume | 188 |
| creator | Stipanović, Dušan M. Kapetina, Mirna N. Rapaić, Milan R. Murmann, Boris |
| description | In this paper, a particular discrete-time nonlinear and time-invariant system represented as a vector difference equation is analyzed for its stability properties. The motivation for analyzing this particular system is that it models gated recurrent unit neural networks commonly used and well known in machine learning applications. From the technical perspective, the analyses exploit the systems similarities to a convex combination of discrete-time systems, where one of the systems is trivial, and thus, the overall properties are mostly dependent on the other one. Stability results are formulated for the nonlinear system and its linearization with respect to the systems, in general, multiple equilibria. To motivate and illustrate the potential of these results in applications, some particular results are derived for the gated recurrent unit neural network models and a connection between local stability analysis and learning is provided. |
| doi_str_mv | 10.1007/s10957-020-01776-w |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_2478285479</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2478285479</sourcerecordid><originalsourceid>FETCH-LOGICAL-c319t-ba5979ce556a58785f4a9855515d508ecd9cb90aad44d1452334c09dfe5674fb3</originalsourceid><addsrcrecordid>eNp9kLtOwzAUhi0EEqXwAkyRmA3HtzpmqyooSBVIQBcWy0kcSEnjYjuUvj2GILEx_Wf4L0cfQqcEzgmAvAgElJAYKGAgUk7wdg-NiJAM01zm-2gEQClmlKlDdBTCCgBULvkIPT9GUzRtE3eZq7O5ibbKHmzZe2-7mC27JmZ3tvemTRK3zr-Fy2zmug_7mWRdNJ2Jjeuya-fXfTvc083GO1O-HqOD2rTBnvzqGC2vr55mN3hxP7-dTRe4ZERFXBihpCqtEBMj0rOi5kblQggiKgG5LStVFgqMqTivCBeUMV6CqmorJpLXBRujs6E3zb73NkS9cr3v0qSmXOY0F1yq5KKDq_QuBG9rvfHN2vidJqC_GeqBoU4M9Q9DvU0hNoRCMncv1v9V_5P6AtFgdXs</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2478285479</pqid></control><display><type>article</type><title>Stability of Gated Recurrent Unit Neural Networks: Convex Combination Formulation Approach</title><source>Springer Nature - Complete Springer Journals</source><creator>Stipanović, Dušan M. ; Kapetina, Mirna N. ; Rapaić, Milan R. ; Murmann, Boris</creator><creatorcontrib>Stipanović, Dušan M. ; Kapetina, Mirna N. ; Rapaić, Milan R. ; Murmann, Boris</creatorcontrib><description>In this paper, a particular discrete-time nonlinear and time-invariant system represented as a vector difference equation is analyzed for its stability properties. The motivation for analyzing this particular system is that it models gated recurrent unit neural networks commonly used and well known in machine learning applications. From the technical perspective, the analyses exploit the systems similarities to a convex combination of discrete-time systems, where one of the systems is trivial, and thus, the overall properties are mostly dependent on the other one. Stability results are formulated for the nonlinear system and its linearization with respect to the systems, in general, multiple equilibria. To motivate and illustrate the potential of these results in applications, some particular results are derived for the gated recurrent unit neural network models and a connection between local stability analysis and learning is provided.</description><identifier>ISSN: 0022-3239</identifier><identifier>EISSN: 1573-2878</identifier><identifier>DOI: 10.1007/s10957-020-01776-w</identifier><language>eng</language><publisher>New York: Springer US</publisher><subject>Applications of Mathematics ; Calculus of Variations and Optimal Control; Optimization ; Difference equations ; Discrete time systems ; Engineering ; Machine learning ; Mathematics ; Mathematics and Statistics ; Neural networks ; Nonlinear systems ; Operations Research/Decision Theory ; Optimization ; Stability analysis ; Technical Note ; Theory of Computation ; Time invariant systems</subject><ispartof>Journal of optimization theory and applications, 2021, Vol.188 (1), p.291-306</ispartof><rights>Springer Science+Business Media, LLC, part of Springer Nature 2020</rights><rights>Springer Science+Business Media, LLC, part of Springer Nature 2020.</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c319t-ba5979ce556a58785f4a9855515d508ecd9cb90aad44d1452334c09dfe5674fb3</citedby><cites>FETCH-LOGICAL-c319t-ba5979ce556a58785f4a9855515d508ecd9cb90aad44d1452334c09dfe5674fb3</cites><orcidid>0000-0002-2826-5241</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://link.springer.com/content/pdf/10.1007/s10957-020-01776-w$$EPDF$$P50$$Gspringer$$H</linktopdf><linktohtml>$$Uhttps://link.springer.com/10.1007/s10957-020-01776-w$$EHTML$$P50$$Gspringer$$H</linktohtml><link.rule.ids>314,776,780,27903,27904,41467,42536,51297</link.rule.ids></links><search><creatorcontrib>Stipanović, Dušan M.</creatorcontrib><creatorcontrib>Kapetina, Mirna N.</creatorcontrib><creatorcontrib>Rapaić, Milan R.</creatorcontrib><creatorcontrib>Murmann, Boris</creatorcontrib><title>Stability of Gated Recurrent Unit Neural Networks: Convex Combination Formulation Approach</title><title>Journal of optimization theory and applications</title><addtitle>J Optim Theory Appl</addtitle><description>In this paper, a particular discrete-time nonlinear and time-invariant system represented as a vector difference equation is analyzed for its stability properties. The motivation for analyzing this particular system is that it models gated recurrent unit neural networks commonly used and well known in machine learning applications. From the technical perspective, the analyses exploit the systems similarities to a convex combination of discrete-time systems, where one of the systems is trivial, and thus, the overall properties are mostly dependent on the other one. Stability results are formulated for the nonlinear system and its linearization with respect to the systems, in general, multiple equilibria. To motivate and illustrate the potential of these results in applications, some particular results are derived for the gated recurrent unit neural network models and a connection between local stability analysis and learning is provided.</description><subject>Applications of Mathematics</subject><subject>Calculus of Variations and Optimal Control; Optimization</subject><subject>Difference equations</subject><subject>Discrete time systems</subject><subject>Engineering</subject><subject>Machine learning</subject><subject>Mathematics</subject><subject>Mathematics and Statistics</subject><subject>Neural networks</subject><subject>Nonlinear systems</subject><subject>Operations Research/Decision Theory</subject><subject>Optimization</subject><subject>Stability analysis</subject><subject>Technical Note</subject><subject>Theory of Computation</subject><subject>Time invariant systems</subject><issn>0022-3239</issn><issn>1573-2878</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2021</creationdate><recordtype>article</recordtype><sourceid>8G5</sourceid><sourceid>BENPR</sourceid><sourceid>GUQSH</sourceid><sourceid>M2O</sourceid><recordid>eNp9kLtOwzAUhi0EEqXwAkyRmA3HtzpmqyooSBVIQBcWy0kcSEnjYjuUvj2GILEx_Wf4L0cfQqcEzgmAvAgElJAYKGAgUk7wdg-NiJAM01zm-2gEQClmlKlDdBTCCgBULvkIPT9GUzRtE3eZq7O5ibbKHmzZe2-7mC27JmZ3tvemTRK3zr-Fy2zmug_7mWRdNJ2Jjeuya-fXfTvc083GO1O-HqOD2rTBnvzqGC2vr55mN3hxP7-dTRe4ZERFXBihpCqtEBMj0rOi5kblQggiKgG5LStVFgqMqTivCBeUMV6CqmorJpLXBRujs6E3zb73NkS9cr3v0qSmXOY0F1yq5KKDq_QuBG9rvfHN2vidJqC_GeqBoU4M9Q9DvU0hNoRCMncv1v9V_5P6AtFgdXs</recordid><startdate>2021</startdate><enddate>2021</enddate><creator>Stipanović, Dušan M.</creator><creator>Kapetina, Mirna N.</creator><creator>Rapaić, Milan R.</creator><creator>Murmann, Boris</creator><general>Springer US</general><general>Springer Nature B.V</general><scope>AAYXX</scope><scope>CITATION</scope><scope>3V.</scope><scope>7SC</scope><scope>7TB</scope><scope>7WY</scope><scope>7WZ</scope><scope>7XB</scope><scope>87Z</scope><scope>88I</scope><scope>8AO</scope><scope>8FD</scope><scope>8FE</scope><scope>8FG</scope><scope>8FK</scope><scope>8FL</scope><scope>8G5</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>ARAPS</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BEZIV</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>FR3</scope><scope>FRNLG</scope><scope>F~G</scope><scope>GNUQQ</scope><scope>GUQSH</scope><scope>HCIFZ</scope><scope>JQ2</scope><scope>K60</scope><scope>K6~</scope><scope>K7-</scope><scope>KR7</scope><scope>L.-</scope><scope>L6V</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><scope>M0C</scope><scope>M2O</scope><scope>M2P</scope><scope>M7S</scope><scope>MBDVC</scope><scope>P5Z</scope><scope>P62</scope><scope>PQBIZ</scope><scope>PQBZA</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>PTHSS</scope><scope>Q9U</scope><orcidid>https://orcid.org/0000-0002-2826-5241</orcidid></search><sort><creationdate>2021</creationdate><title>Stability of Gated Recurrent Unit Neural Networks: Convex Combination Formulation Approach</title><author>Stipanović, Dušan M. ; Kapetina, Mirna N. ; Rapaić, Milan R. ; Murmann, Boris</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c319t-ba5979ce556a58785f4a9855515d508ecd9cb90aad44d1452334c09dfe5674fb3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2021</creationdate><topic>Applications of Mathematics</topic><topic>Calculus of Variations and Optimal Control; Optimization</topic><topic>Difference equations</topic><topic>Discrete time systems</topic><topic>Engineering</topic><topic>Machine learning</topic><topic>Mathematics</topic><topic>Mathematics and Statistics</topic><topic>Neural networks</topic><topic>Nonlinear systems</topic><topic>Operations Research/Decision Theory</topic><topic>Optimization</topic><topic>Stability analysis</topic><topic>Technical Note</topic><topic>Theory of Computation</topic><topic>Time invariant systems</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Stipanović, Dušan M.</creatorcontrib><creatorcontrib>Kapetina, Mirna N.</creatorcontrib><creatorcontrib>Rapaić, Milan R.</creatorcontrib><creatorcontrib>Murmann, Boris</creatorcontrib><collection>CrossRef</collection><collection>ProQuest Central (Corporate)</collection><collection>Computer and Information Systems Abstracts</collection><collection>Mechanical & Transportation Engineering Abstracts</collection><collection>ABI/INFORM Collection</collection><collection>ABI/INFORM Global (PDF only)</collection><collection>ProQuest Central (purchase pre-March 2016)</collection><collection>ABI/INFORM Global (Alumni Edition)</collection><collection>Science Database (Alumni Edition)</collection><collection>ProQuest Pharma Collection</collection><collection>Technology Research Database</collection><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>ProQuest Central (Alumni) (purchase pre-March 2016)</collection><collection>ABI/INFORM Collection (Alumni Edition)</collection><collection>Research Library (Alumni Edition)</collection><collection>Materials Science & Engineering Collection</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>Advanced Technologies & Aerospace Collection</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Business Premium Collection</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>Engineering Research Database</collection><collection>Business Premium Collection (Alumni)</collection><collection>ABI/INFORM Global (Corporate)</collection><collection>ProQuest Central Student</collection><collection>Research Library Prep</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Computer Science Collection</collection><collection>ProQuest Business Collection (Alumni Edition)</collection><collection>ProQuest Business Collection</collection><collection>Computer Science Database</collection><collection>Civil Engineering Abstracts</collection><collection>ABI/INFORM Professional Advanced</collection><collection>ProQuest Engineering Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><collection>ABI/INFORM Global</collection><collection>Research Library</collection><collection>Science Database</collection><collection>Engineering Database</collection><collection>Research Library (Corporate)</collection><collection>Advanced Technologies & Aerospace Database</collection><collection>ProQuest Advanced Technologies & Aerospace Collection</collection><collection>ProQuest One Business</collection><collection>ProQuest One Business (Alumni)</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>Engineering Collection</collection><collection>ProQuest Central Basic</collection><jtitle>Journal of optimization theory and applications</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Stipanović, Dušan M.</au><au>Kapetina, Mirna N.</au><au>Rapaić, Milan R.</au><au>Murmann, Boris</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Stability of Gated Recurrent Unit Neural Networks: Convex Combination Formulation Approach</atitle><jtitle>Journal of optimization theory and applications</jtitle><stitle>J Optim Theory Appl</stitle><date>2021</date><risdate>2021</risdate><volume>188</volume><issue>1</issue><spage>291</spage><epage>306</epage><pages>291-306</pages><issn>0022-3239</issn><eissn>1573-2878</eissn><abstract>In this paper, a particular discrete-time nonlinear and time-invariant system represented as a vector difference equation is analyzed for its stability properties. The motivation for analyzing this particular system is that it models gated recurrent unit neural networks commonly used and well known in machine learning applications. From the technical perspective, the analyses exploit the systems similarities to a convex combination of discrete-time systems, where one of the systems is trivial, and thus, the overall properties are mostly dependent on the other one. Stability results are formulated for the nonlinear system and its linearization with respect to the systems, in general, multiple equilibria. To motivate and illustrate the potential of these results in applications, some particular results are derived for the gated recurrent unit neural network models and a connection between local stability analysis and learning is provided.</abstract><cop>New York</cop><pub>Springer US</pub><doi>10.1007/s10957-020-01776-w</doi><tpages>16</tpages><orcidid>https://orcid.org/0000-0002-2826-5241</orcidid></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 0022-3239 |
| ispartof | Journal of optimization theory and applications, 2021, Vol.188 (1), p.291-306 |
| issn | 0022-3239 1573-2878 |
| language | eng |
| recordid | cdi_proquest_journals_2478285479 |
| source | Springer Nature - Complete Springer Journals |
| subjects | Applications of Mathematics Calculus of Variations and Optimal Control Optimization Difference equations Discrete time systems Engineering Machine learning Mathematics Mathematics and Statistics Neural networks Nonlinear systems Operations Research/Decision Theory Optimization Stability analysis Technical Note Theory of Computation Time invariant systems |
| title | Stability of Gated Recurrent Unit Neural Networks: Convex Combination Formulation Approach |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-27T03%3A39%3A16IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Stability%20of%20Gated%20Recurrent%20Unit%20Neural%20Networks:%20Convex%20Combination%20Formulation%20Approach&rft.jtitle=Journal%20of%20optimization%20theory%20and%20applications&rft.au=Stipanovi%C4%87,%20Du%C5%A1an%20M.&rft.date=2021&rft.volume=188&rft.issue=1&rft.spage=291&rft.epage=306&rft.pages=291-306&rft.issn=0022-3239&rft.eissn=1573-2878&rft_id=info:doi/10.1007/s10957-020-01776-w&rft_dat=%3Cproquest_cross%3E2478285479%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2478285479&rft_id=info:pmid/&rfr_iscdi=true |