Dynamics of multilayer networks in the vicinity of temporary minima
A dynamical system model is derived for a single-output, two-layer neural network, which learns according to the back-propagation algorithm. Particular emphasis is placed on the analysis of the occurrence of temporary minima. The Jacobian matrix of the system is derived, whose eigenvalues characteri...
Gespeichert in:
| Veröffentlicht in: | Neural networks 1999, Vol.12 (1), p.43-58 |
|---|---|
| Hauptverfasser: | , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 58 |
|---|---|
| container_issue | 1 |
| container_start_page | 43 |
| container_title | Neural networks |
| container_volume | 12 |
| creator | Ampazis, Nikolaos Perantonis, S.J. Taylor, J.G. |
| description | A dynamical system model is derived for a single-output, two-layer neural network, which learns according to the back-propagation algorithm. Particular emphasis is placed on the analysis of the occurrence of temporary minima. The Jacobian matrix of the system is derived, whose eigenvalues characterize the evolution of learning. Temporary minima correspond to critical points of the phase plane trajectories, and the bifurcation of the Jacobian matrix eigenvalues signifies their abandonment. Following this analysis, we show that the employment of constrained optimization methods can decrease the time spent in the vicinity of this type of minima. A number of numerical results illustrates the analytical conclusions. |
| doi_str_mv | 10.1016/S0893-6080(98)00103-8 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_miscellaneous_26924689</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><els_id>S0893608098001038</els_id><sourcerecordid>26924689</sourcerecordid><originalsourceid>FETCH-LOGICAL-c452t-77b9b3a2382a16b235164b59508d8b15f25ecb68a9262e00a4bf6f6716e6ca873</originalsourceid><addsrcrecordid>eNqFkMtKAzEUhoMoWquPoMxCpC5Gc5ncViL1CoILdR0y6RmMzqUmU2Xe3tQW3enqwOH7z-VD6IDgU4KJOHvESrNcYIUnWp1gTDDL1QYaESV1TqWim2j0g-yg3RhfMcZCFWwb7RAqBJVEjND0cmht413MuiprFnXvaztAyFroP7vwFjPfZv0LZB_e-db3wxLroZl3wYYha1KvsXtoq7J1hP11HaPn66un6W1-_3BzN724z13BaZ9LWeqSWcoUtUSUlHEiipJrjtVMlYRXlIMrhbKaCgoY26KsRCXSmSCcVZKN0fFq7jx07wuIvWl8dFDXtoVuEQ0VmhYi_TxGkz9BorhmWmvOEspXqAtdjAEqMw_ppTAYgs1StPkWbZYWjVbmW7RRKXe4XrEoG5j9ptZmE3C0Bmx0tq6CbZ2Pv5wSUkqcsPMVBknch4dgovPQOpj5AK43s87_c8kXnHuY9A</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>1859399953</pqid></control><display><type>article</type><title>Dynamics of multilayer networks in the vicinity of temporary minima</title><source>Elsevier ScienceDirect Journals Complete</source><creator>Ampazis, Nikolaos ; Perantonis, S.J. ; Taylor, J.G.</creator><creatorcontrib>Ampazis, Nikolaos ; Perantonis, S.J. ; Taylor, J.G.</creatorcontrib><description>A dynamical system model is derived for a single-output, two-layer neural network, which learns according to the back-propagation algorithm. Particular emphasis is placed on the analysis of the occurrence of temporary minima. The Jacobian matrix of the system is derived, whose eigenvalues characterize the evolution of learning. Temporary minima correspond to critical points of the phase plane trajectories, and the bifurcation of the Jacobian matrix eigenvalues signifies their abandonment. Following this analysis, we show that the employment of constrained optimization methods can decrease the time spent in the vicinity of this type of minima. A number of numerical results illustrates the analytical conclusions.</description><identifier>ISSN: 0893-6080</identifier><identifier>EISSN: 1879-2782</identifier><identifier>DOI: 10.1016/S0893-6080(98)00103-8</identifier><identifier>PMID: 12662716</identifier><language>eng</language><publisher>Oxford: Elsevier Ltd</publisher><subject>Applied sciences ; Artificial intelligence ; Back-propagation ; Computer science; control theory; systems ; Connectionism. Neural networks ; Constrained optimization ; Dynamical systems ; Eigenvalues ; Exact sciences and technology ; Feed-forward neural networks ; Jacobian matrix ; Learning and adaptive systems ; Supervised learning ; Temporary minima</subject><ispartof>Neural networks, 1999, Vol.12 (1), p.43-58</ispartof><rights>1998 Elsevier Science Ltd</rights><rights>1999 INIST-CNRS</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c452t-77b9b3a2382a16b235164b59508d8b15f25ecb68a9262e00a4bf6f6716e6ca873</citedby><cites>FETCH-LOGICAL-c452t-77b9b3a2382a16b235164b59508d8b15f25ecb68a9262e00a4bf6f6716e6ca873</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://dx.doi.org/10.1016/S0893-6080(98)00103-8$$EHTML$$P50$$Gelsevier$$H</linktohtml><link.rule.ids>314,780,784,3550,4024,27923,27924,27925,45995</link.rule.ids><backlink>$$Uhttp://pascal-francis.inist.fr/vibad/index.php?action=getRecordDetail&idt=1867770$$DView record in Pascal Francis$$Hfree_for_read</backlink><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/12662716$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Ampazis, Nikolaos</creatorcontrib><creatorcontrib>Perantonis, S.J.</creatorcontrib><creatorcontrib>Taylor, J.G.</creatorcontrib><title>Dynamics of multilayer networks in the vicinity of temporary minima</title><title>Neural networks</title><addtitle>Neural Netw</addtitle><description>A dynamical system model is derived for a single-output, two-layer neural network, which learns according to the back-propagation algorithm. Particular emphasis is placed on the analysis of the occurrence of temporary minima. The Jacobian matrix of the system is derived, whose eigenvalues characterize the evolution of learning. Temporary minima correspond to critical points of the phase plane trajectories, and the bifurcation of the Jacobian matrix eigenvalues signifies their abandonment. Following this analysis, we show that the employment of constrained optimization methods can decrease the time spent in the vicinity of this type of minima. A number of numerical results illustrates the analytical conclusions.</description><subject>Applied sciences</subject><subject>Artificial intelligence</subject><subject>Back-propagation</subject><subject>Computer science; control theory; systems</subject><subject>Connectionism. Neural networks</subject><subject>Constrained optimization</subject><subject>Dynamical systems</subject><subject>Eigenvalues</subject><subject>Exact sciences and technology</subject><subject>Feed-forward neural networks</subject><subject>Jacobian matrix</subject><subject>Learning and adaptive systems</subject><subject>Supervised learning</subject><subject>Temporary minima</subject><issn>0893-6080</issn><issn>1879-2782</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>1999</creationdate><recordtype>article</recordtype><recordid>eNqFkMtKAzEUhoMoWquPoMxCpC5Gc5ncViL1CoILdR0y6RmMzqUmU2Xe3tQW3enqwOH7z-VD6IDgU4KJOHvESrNcYIUnWp1gTDDL1QYaESV1TqWim2j0g-yg3RhfMcZCFWwb7RAqBJVEjND0cmht413MuiprFnXvaztAyFroP7vwFjPfZv0LZB_e-db3wxLroZl3wYYha1KvsXtoq7J1hP11HaPn66un6W1-_3BzN724z13BaZ9LWeqSWcoUtUSUlHEiipJrjtVMlYRXlIMrhbKaCgoY26KsRCXSmSCcVZKN0fFq7jx07wuIvWl8dFDXtoVuEQ0VmhYi_TxGkz9BorhmWmvOEspXqAtdjAEqMw_ppTAYgs1StPkWbZYWjVbmW7RRKXe4XrEoG5j9ptZmE3C0Bmx0tq6CbZ2Pv5wSUkqcsPMVBknch4dgovPQOpj5AK43s87_c8kXnHuY9A</recordid><startdate>1999</startdate><enddate>1999</enddate><creator>Ampazis, Nikolaos</creator><creator>Perantonis, S.J.</creator><creator>Taylor, J.G.</creator><general>Elsevier Ltd</general><general>Elsevier Science</general><scope>IQODW</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7X8</scope><scope>7SC</scope><scope>8FD</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope></search><sort><creationdate>1999</creationdate><title>Dynamics of multilayer networks in the vicinity of temporary minima</title><author>Ampazis, Nikolaos ; Perantonis, S.J. ; Taylor, J.G.</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c452t-77b9b3a2382a16b235164b59508d8b15f25ecb68a9262e00a4bf6f6716e6ca873</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>1999</creationdate><topic>Applied sciences</topic><topic>Artificial intelligence</topic><topic>Back-propagation</topic><topic>Computer science; control theory; systems</topic><topic>Connectionism. Neural networks</topic><topic>Constrained optimization</topic><topic>Dynamical systems</topic><topic>Eigenvalues</topic><topic>Exact sciences and technology</topic><topic>Feed-forward neural networks</topic><topic>Jacobian matrix</topic><topic>Learning and adaptive systems</topic><topic>Supervised learning</topic><topic>Temporary minima</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Ampazis, Nikolaos</creatorcontrib><creatorcontrib>Perantonis, S.J.</creatorcontrib><creatorcontrib>Taylor, J.G.</creatorcontrib><collection>Pascal-Francis</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>MEDLINE - Academic</collection><collection>Computer and Information Systems Abstracts</collection><collection>Technology Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><jtitle>Neural networks</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Ampazis, Nikolaos</au><au>Perantonis, S.J.</au><au>Taylor, J.G.</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Dynamics of multilayer networks in the vicinity of temporary minima</atitle><jtitle>Neural networks</jtitle><addtitle>Neural Netw</addtitle><date>1999</date><risdate>1999</risdate><volume>12</volume><issue>1</issue><spage>43</spage><epage>58</epage><pages>43-58</pages><issn>0893-6080</issn><eissn>1879-2782</eissn><abstract>A dynamical system model is derived for a single-output, two-layer neural network, which learns according to the back-propagation algorithm. Particular emphasis is placed on the analysis of the occurrence of temporary minima. The Jacobian matrix of the system is derived, whose eigenvalues characterize the evolution of learning. Temporary minima correspond to critical points of the phase plane trajectories, and the bifurcation of the Jacobian matrix eigenvalues signifies their abandonment. Following this analysis, we show that the employment of constrained optimization methods can decrease the time spent in the vicinity of this type of minima. A number of numerical results illustrates the analytical conclusions.</abstract><cop>Oxford</cop><pub>Elsevier Ltd</pub><pmid>12662716</pmid><doi>10.1016/S0893-6080(98)00103-8</doi><tpages>16</tpages></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 0893-6080 |
| ispartof | Neural networks, 1999, Vol.12 (1), p.43-58 |
| issn | 0893-6080 1879-2782 |
| language | eng |
| recordid | cdi_proquest_miscellaneous_26924689 |
| source | Elsevier ScienceDirect Journals Complete |
| subjects | Applied sciences Artificial intelligence Back-propagation Computer science control theory systems Connectionism. Neural networks Constrained optimization Dynamical systems Eigenvalues Exact sciences and technology Feed-forward neural networks Jacobian matrix Learning and adaptive systems Supervised learning Temporary minima |
| title | Dynamics of multilayer networks in the vicinity of temporary minima |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-04T06%3A57%3A54IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Dynamics%20of%20multilayer%20networks%20in%20the%20vicinity%20of%20temporary%20minima&rft.jtitle=Neural%20networks&rft.au=Ampazis,%20Nikolaos&rft.date=1999&rft.volume=12&rft.issue=1&rft.spage=43&rft.epage=58&rft.pages=43-58&rft.issn=0893-6080&rft.eissn=1879-2782&rft_id=info:doi/10.1016/S0893-6080(98)00103-8&rft_dat=%3Cproquest_cross%3E26924689%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=1859399953&rft_id=info:pmid/12662716&rft_els_id=S0893608098001038&rfr_iscdi=true |