Comparison between layer-to-layer network training and conventional network training using Deep Convolutional Neural Networks

Title: Comparison between layer-to-layer network training and conventional network training using Deep Convolutional Neural Networks Abstract: Convolutional neural networks (CNNs) are widely used in various applications due to their effectiveness in extracting features from data. However, the perfor...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: Bhyravabhottla, Kiran Kumar Ashish, Lee, WonSook
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page
container_issue
container_start_page
container_title
container_volume
creator Bhyravabhottla, Kiran Kumar Ashish
Lee, WonSook
description Title: Comparison between layer-to-layer network training and conventional network training using Deep Convolutional Neural Networks Abstract: Convolutional neural networks (CNNs) are widely used in various applications due to their effectiveness in extracting features from data. However, the performance of a CNN heavily depends on its architecture and training process. In this study, we propose a layer-to-layer training method and compare its performance with the conventional training method. In the layer-to-layer training approach, we treat a portion of the early layers as a student network and the later layers as a teacher network. During each training step, we incrementally train the student network to learn from the output of the teacher network, and vice versa. We evaluate this approach on VGG16, ResNext, and DenseNet networks without pre-trained ImageNet weights and a regular CNN model. Our experiments show that the layer-to-layer training method outperforms the conventional training method for both models. Specifically, we achieve higher accuracy on the test set for the VGG16, ResNext, and DeseNet networks and the CNN model using layer-to-layer training compared to the conventional training method. Overall, our study highlights the importance of layer-wise training in CNNs and suggests that layer-to-layer training can be a promising approach for improving the accuracy of CNNs.
doi_str_mv 10.48550/arxiv.2303.15245
format Article
fullrecord <record><control><sourceid>arxiv_GOX</sourceid><recordid>TN_cdi_arxiv_primary_2303_15245</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2303_15245</sourcerecordid><originalsourceid>FETCH-LOGICAL-a675-ae7e844a4ec32e07c7500066f2f14a1d6fbaeb2277c0bd700dd5de34f39874c83</originalsourceid><addsrcrecordid>eNplkM1OwzAQhH3hgAoPwAm_gIPjnzgcUfiVKrj0Hm3sNbJI7chJCj3w7rSht152VqtvRtoh5Kbkhaq15neQf8KuEJLLotRC6Uvy26TtADmMKdIOp2_ESHvYY2ZTYstC4-Gc8hedMoQY4ieF6KhNcYdxCilCf07M43E-Ig60OYCpn0_kO855kcUwXpELD_2I1yddkc3z06Z5ZeuPl7fmYc2gMpoBGqyVAoVWCuTGGs05ryovfKmgdJXvADshjLG8c4Zz57RDqby8r42ytVyR2__Y5f92yGELed8ee2iXHuQflZNcTg</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>Comparison between layer-to-layer network training and conventional network training using Deep Convolutional Neural Networks</title><source>arXiv.org</source><creator>Bhyravabhottla, Kiran Kumar Ashish ; Lee, WonSook</creator><creatorcontrib>Bhyravabhottla, Kiran Kumar Ashish ; Lee, WonSook</creatorcontrib><description>Title: Comparison between layer-to-layer network training and conventional network training using Deep Convolutional Neural Networks Abstract: Convolutional neural networks (CNNs) are widely used in various applications due to their effectiveness in extracting features from data. However, the performance of a CNN heavily depends on its architecture and training process. In this study, we propose a layer-to-layer training method and compare its performance with the conventional training method. In the layer-to-layer training approach, we treat a portion of the early layers as a student network and the later layers as a teacher network. During each training step, we incrementally train the student network to learn from the output of the teacher network, and vice versa. We evaluate this approach on VGG16, ResNext, and DenseNet networks without pre-trained ImageNet weights and a regular CNN model. Our experiments show that the layer-to-layer training method outperforms the conventional training method for both models. Specifically, we achieve higher accuracy on the test set for the VGG16, ResNext, and DeseNet networks and the CNN model using layer-to-layer training compared to the conventional training method. Overall, our study highlights the importance of layer-wise training in CNNs and suggests that layer-to-layer training can be a promising approach for improving the accuracy of CNNs.</description><identifier>DOI: 10.48550/arxiv.2303.15245</identifier><language>eng</language><subject>Computer Science - Computer Vision and Pattern Recognition</subject><creationdate>2023-03</creationdate><rights>http://creativecommons.org/licenses/by/4.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,780,885</link.rule.ids><linktorsrc>$$Uhttps://arxiv.org/abs/2303.15245$$EView_record_in_Cornell_University$$FView_record_in_$$GCornell_University$$Hfree_for_read</linktorsrc><backlink>$$Uhttps://doi.org/10.48550/arXiv.2303.15245$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Bhyravabhottla, Kiran Kumar Ashish</creatorcontrib><creatorcontrib>Lee, WonSook</creatorcontrib><title>Comparison between layer-to-layer network training and conventional network training using Deep Convolutional Neural Networks</title><description>Title: Comparison between layer-to-layer network training and conventional network training using Deep Convolutional Neural Networks Abstract: Convolutional neural networks (CNNs) are widely used in various applications due to their effectiveness in extracting features from data. However, the performance of a CNN heavily depends on its architecture and training process. In this study, we propose a layer-to-layer training method and compare its performance with the conventional training method. In the layer-to-layer training approach, we treat a portion of the early layers as a student network and the later layers as a teacher network. During each training step, we incrementally train the student network to learn from the output of the teacher network, and vice versa. We evaluate this approach on VGG16, ResNext, and DenseNet networks without pre-trained ImageNet weights and a regular CNN model. Our experiments show that the layer-to-layer training method outperforms the conventional training method for both models. Specifically, we achieve higher accuracy on the test set for the VGG16, ResNext, and DeseNet networks and the CNN model using layer-to-layer training compared to the conventional training method. Overall, our study highlights the importance of layer-wise training in CNNs and suggests that layer-to-layer training can be a promising approach for improving the accuracy of CNNs.</description><subject>Computer Science - Computer Vision and Pattern Recognition</subject><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><sourceid>GOX</sourceid><recordid>eNplkM1OwzAQhH3hgAoPwAm_gIPjnzgcUfiVKrj0Hm3sNbJI7chJCj3w7rSht152VqtvRtoh5Kbkhaq15neQf8KuEJLLotRC6Uvy26TtADmMKdIOp2_ESHvYY2ZTYstC4-Gc8hedMoQY4ieF6KhNcYdxCilCf07M43E-Ig60OYCpn0_kO855kcUwXpELD_2I1yddkc3z06Z5ZeuPl7fmYc2gMpoBGqyVAoVWCuTGGs05ryovfKmgdJXvADshjLG8c4Zz57RDqby8r42ytVyR2__Y5f92yGELed8ee2iXHuQflZNcTg</recordid><startdate>20230327</startdate><enddate>20230327</enddate><creator>Bhyravabhottla, Kiran Kumar Ashish</creator><creator>Lee, WonSook</creator><scope>AKY</scope><scope>GOX</scope></search><sort><creationdate>20230327</creationdate><title>Comparison between layer-to-layer network training and conventional network training using Deep Convolutional Neural Networks</title><author>Bhyravabhottla, Kiran Kumar Ashish ; Lee, WonSook</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a675-ae7e844a4ec32e07c7500066f2f14a1d6fbaeb2277c0bd700dd5de34f39874c83</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Computer Science - Computer Vision and Pattern Recognition</topic><toplevel>online_resources</toplevel><creatorcontrib>Bhyravabhottla, Kiran Kumar Ashish</creatorcontrib><creatorcontrib>Lee, WonSook</creatorcontrib><collection>arXiv Computer Science</collection><collection>arXiv.org</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Bhyravabhottla, Kiran Kumar Ashish</au><au>Lee, WonSook</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Comparison between layer-to-layer network training and conventional network training using Deep Convolutional Neural Networks</atitle><date>2023-03-27</date><risdate>2023</risdate><abstract>Title: Comparison between layer-to-layer network training and conventional network training using Deep Convolutional Neural Networks Abstract: Convolutional neural networks (CNNs) are widely used in various applications due to their effectiveness in extracting features from data. However, the performance of a CNN heavily depends on its architecture and training process. In this study, we propose a layer-to-layer training method and compare its performance with the conventional training method. In the layer-to-layer training approach, we treat a portion of the early layers as a student network and the later layers as a teacher network. During each training step, we incrementally train the student network to learn from the output of the teacher network, and vice versa. We evaluate this approach on VGG16, ResNext, and DenseNet networks without pre-trained ImageNet weights and a regular CNN model. Our experiments show that the layer-to-layer training method outperforms the conventional training method for both models. Specifically, we achieve higher accuracy on the test set for the VGG16, ResNext, and DeseNet networks and the CNN model using layer-to-layer training compared to the conventional training method. Overall, our study highlights the importance of layer-wise training in CNNs and suggests that layer-to-layer training can be a promising approach for improving the accuracy of CNNs.</abstract><doi>10.48550/arxiv.2303.15245</doi><oa>free_for_read</oa></addata></record>
fulltext fulltext_linktorsrc
identifier DOI: 10.48550/arxiv.2303.15245
ispartof
issn
language eng
recordid cdi_arxiv_primary_2303_15245
source arXiv.org
subjects Computer Science - Computer Vision and Pattern Recognition
title Comparison between layer-to-layer network training and conventional network training using Deep Convolutional Neural Networks
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-25T19%3A20%3A46IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-arxiv_GOX&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Comparison%20between%20layer-to-layer%20network%20training%20and%20conventional%20network%20training%20using%20Deep%20Convolutional%20Neural%20Networks&rft.au=Bhyravabhottla,%20Kiran%20Kumar%20Ashish&rft.date=2023-03-27&rft_id=info:doi/10.48550/arxiv.2303.15245&rft_dat=%3Carxiv_GOX%3E2303_15245%3C/arxiv_GOX%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true