A comparison of machine learning algorithms for the surveillance of autism spectrum disorder

The Centers for Disease Control and Prevention (CDC) coordinates a labor-intensive process to measure the prevalence of autism spectrum disorder (ASD) among children in the United States. Random forests methods have shown promise in speeding up this process, but they lag behind human classification...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:PloS one 2019-09, Vol.14 (9), p.e0222907-e0222907
Hauptverfasser: Lee, Scott H, Maenner, Matthew J, Heilig, Charles M
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page e0222907
container_issue 9
container_start_page e0222907
container_title PloS one
container_volume 14
creator Lee, Scott H
Maenner, Matthew J
Heilig, Charles M
description The Centers for Disease Control and Prevention (CDC) coordinates a labor-intensive process to measure the prevalence of autism spectrum disorder (ASD) among children in the United States. Random forests methods have shown promise in speeding up this process, but they lag behind human classification accuracy by about 5%. We explore whether more recently available document classification algorithms can close this gap. Using data gathered from a single surveillance site, we applied 8 supervised learning algorithms to predict whether children meet the case definition for ASD based solely on the words in their evaluations. We compared the algorithms' performance across 10 random train-test splits of the data, using classification accuracy, F1 score, and number of positive calls to evaluate their potential use for surveillance. Across the 10 train-test cycles, the random forest and support vector machine with Naive Bayes features (NB-SVM) each achieved slightly more than 87% mean accuracy. The NB-SVM produced significantly more false negatives than false positives (P = 0.027), but the random forest did not, making its prevalence estimates very close to the true prevalence in the data. The best-performing neural network performed similarly to the random forest on both measures. The random forest performed as well as more recently available models like the NB-SVM and the neural network, and it also produced good prevalence estimates. NB-SVM may not be a good candidate for use in a fully-automated surveillance workflow due to increased false negatives. More sophisticated algorithms, like hierarchical convolutional neural networks, may not be feasible to train due to characteristics of the data. Current algorithms might perform better if the data are abstracted and processed differently and if they take into account information about the children in addition to their evaluations. Deep learning models performed similarly to traditional machine learning methods at predicting the clinician-assigned case status for CDC's autism surveillance system. While deep learning methods had limited benefit in this task, they may have applications in other surveillance systems.
doi_str_mv 10.1371/journal.pone.0222907
format Article
fullrecord <record><control><sourceid>gale_plos_</sourceid><recordid>TN_cdi_plos_journals_2297122100</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><galeid>A600743023</galeid><doaj_id>oai_doaj_org_article_2dfb7a20c7384b80a556c9088c0de079</doaj_id><sourcerecordid>A600743023</sourcerecordid><originalsourceid>FETCH-LOGICAL-c692t-eed33ee4dddf53990e7ff83f3e3899c6e9a0f3c8334b551248a78f74f7f205dc3</originalsourceid><addsrcrecordid>eNqNk12L1DAUhoso7jr6D0QLgujFjGnSNu2NMCx-DCws-HUlhExy0mZIk5q0i_57053uMpW9kF60JM_7JuftOUnyPEObjNDs3cGN3nKz6Z2FDcIY14g-SM6zmuB1iRF5ePJ9ljwJ4YBQQaqyfJyckawoCKX5efJzmwrX9dzr4GzqVNpx0WoLqQHurbZNyk3jvB7aLqTK-XRoIQ2jvwZtDLcCJg0fBx26NPQgBj92qYxmXoJ_mjxS3AR4Nr9XyfePH75dfF5fXn3aXWwv16Ks8bAGkIQA5FJKVZC6RkCVqogiQKq6FiXUHCkiKkLyfVFkOK84rRTNFVUYFVKQVfLy6NsbF9gcTGAxEpphnCEUid2RkI4fWO91x_0f5rhmNwvON4z7QQsDDEu1pxwjQUmV7yvEi6IUNaoqgSQgWkev9_Np474DKcAOnpuF6XLH6pY17pqVtIz6yeDNbODdrxHCwDodBEx5ghtv7l1lOc1iwavk1T_o_dXNVMNjAdoqF88VkynblgjRnCA8eW3uoeIjodMidpHScX0heLsQRGaA30PDxxDY7uuX_2evfizZ1ydsC9wMbXAmNpGzYQnmR1B4F4IHdRdyhtg0BLdpsGkI2DwEUfbi9AfdiW67nvwFesECCg</addsrcrecordid><sourcetype>Open Website</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2297122100</pqid></control><display><type>article</type><title>A comparison of machine learning algorithms for the surveillance of autism spectrum disorder</title><source>MEDLINE</source><source>DOAJ Directory of Open Access Journals</source><source>Elektronische Zeitschriftenbibliothek - Frei zugängliche E-Journals</source><source>Public Library of Science (PLoS) Journals Open Access</source><source>PubMed Central</source><source>Free Full-Text Journals in Chemistry</source><creator>Lee, Scott H ; Maenner, Matthew J ; Heilig, Charles M</creator><contributor>Yan, Jingwen</contributor><creatorcontrib>Lee, Scott H ; Maenner, Matthew J ; Heilig, Charles M ; Yan, Jingwen</creatorcontrib><description>The Centers for Disease Control and Prevention (CDC) coordinates a labor-intensive process to measure the prevalence of autism spectrum disorder (ASD) among children in the United States. Random forests methods have shown promise in speeding up this process, but they lag behind human classification accuracy by about 5%. We explore whether more recently available document classification algorithms can close this gap. Using data gathered from a single surveillance site, we applied 8 supervised learning algorithms to predict whether children meet the case definition for ASD based solely on the words in their evaluations. We compared the algorithms' performance across 10 random train-test splits of the data, using classification accuracy, F1 score, and number of positive calls to evaluate their potential use for surveillance. Across the 10 train-test cycles, the random forest and support vector machine with Naive Bayes features (NB-SVM) each achieved slightly more than 87% mean accuracy. The NB-SVM produced significantly more false negatives than false positives (P = 0.027), but the random forest did not, making its prevalence estimates very close to the true prevalence in the data. The best-performing neural network performed similarly to the random forest on both measures. The random forest performed as well as more recently available models like the NB-SVM and the neural network, and it also produced good prevalence estimates. NB-SVM may not be a good candidate for use in a fully-automated surveillance workflow due to increased false negatives. More sophisticated algorithms, like hierarchical convolutional neural networks, may not be feasible to train due to characteristics of the data. Current algorithms might perform better if the data are abstracted and processed differently and if they take into account information about the children in addition to their evaluations. Deep learning models performed similarly to traditional machine learning methods at predicting the clinician-assigned case status for CDC's autism surveillance system. While deep learning methods had limited benefit in this task, they may have applications in other surveillance systems.</description><identifier>ISSN: 1932-6203</identifier><identifier>EISSN: 1932-6203</identifier><identifier>DOI: 10.1371/journal.pone.0222907</identifier><identifier>PMID: 31553774</identifier><language>eng</language><publisher>United States: Public Library of Science</publisher><subject>Accuracy ; Algorithms ; Artificial intelligence ; Artificial neural networks ; Autism ; Autism Spectrum Disorder - diagnosis ; Autism Spectrum Disorder - epidemiology ; Bayesian analysis ; Biology and Life Sciences ; Care and treatment ; Centers for Disease Control and Prevention, U.S ; Child ; Children ; Children &amp; youth ; Classification ; Computer and Information Sciences ; Data mining ; Deep Learning ; Developmental disabilities ; Disease control ; Disease prevention ; Epidemiological Monitoring ; Feasibility Studies ; Forests ; Georgia ; Humans ; Information management ; International conferences ; Learning algorithms ; Linguistics ; Machine learning ; Medicine and Health Sciences ; Neural networks ; People and Places ; Pervasive developmental disorders ; Physical Sciences ; Prevalence ; Public Health Surveillance - methods ; Research and Analysis Methods ; Risk factors ; Sentiment analysis ; Social Sciences ; Support Vector Machine ; Support vector machines ; Surveillance ; Surveillance systems ; United States - epidemiology ; Workflow ; Workflow software</subject><ispartof>PloS one, 2019-09, Vol.14 (9), p.e0222907-e0222907</ispartof><rights>COPYRIGHT 2019 Public Library of Science</rights><rights>This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication: https://creativecommons.org/publicdomain/zero/1.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c692t-eed33ee4dddf53990e7ff83f3e3899c6e9a0f3c8334b551248a78f74f7f205dc3</citedby><cites>FETCH-LOGICAL-c692t-eed33ee4dddf53990e7ff83f3e3899c6e9a0f3c8334b551248a78f74f7f205dc3</cites><orcidid>0000-0003-4584-6758</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC6760799/pdf/$$EPDF$$P50$$Gpubmedcentral$$Hfree_for_read</linktopdf><linktohtml>$$Uhttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC6760799/$$EHTML$$P50$$Gpubmedcentral$$Hfree_for_read</linktohtml><link.rule.ids>230,314,727,780,784,864,885,2102,2928,23866,27924,27925,53791,53793</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/31553774$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><contributor>Yan, Jingwen</contributor><creatorcontrib>Lee, Scott H</creatorcontrib><creatorcontrib>Maenner, Matthew J</creatorcontrib><creatorcontrib>Heilig, Charles M</creatorcontrib><title>A comparison of machine learning algorithms for the surveillance of autism spectrum disorder</title><title>PloS one</title><addtitle>PLoS One</addtitle><description>The Centers for Disease Control and Prevention (CDC) coordinates a labor-intensive process to measure the prevalence of autism spectrum disorder (ASD) among children in the United States. Random forests methods have shown promise in speeding up this process, but they lag behind human classification accuracy by about 5%. We explore whether more recently available document classification algorithms can close this gap. Using data gathered from a single surveillance site, we applied 8 supervised learning algorithms to predict whether children meet the case definition for ASD based solely on the words in their evaluations. We compared the algorithms' performance across 10 random train-test splits of the data, using classification accuracy, F1 score, and number of positive calls to evaluate their potential use for surveillance. Across the 10 train-test cycles, the random forest and support vector machine with Naive Bayes features (NB-SVM) each achieved slightly more than 87% mean accuracy. The NB-SVM produced significantly more false negatives than false positives (P = 0.027), but the random forest did not, making its prevalence estimates very close to the true prevalence in the data. The best-performing neural network performed similarly to the random forest on both measures. The random forest performed as well as more recently available models like the NB-SVM and the neural network, and it also produced good prevalence estimates. NB-SVM may not be a good candidate for use in a fully-automated surveillance workflow due to increased false negatives. More sophisticated algorithms, like hierarchical convolutional neural networks, may not be feasible to train due to characteristics of the data. Current algorithms might perform better if the data are abstracted and processed differently and if they take into account information about the children in addition to their evaluations. Deep learning models performed similarly to traditional machine learning methods at predicting the clinician-assigned case status for CDC's autism surveillance system. While deep learning methods had limited benefit in this task, they may have applications in other surveillance systems.</description><subject>Accuracy</subject><subject>Algorithms</subject><subject>Artificial intelligence</subject><subject>Artificial neural networks</subject><subject>Autism</subject><subject>Autism Spectrum Disorder - diagnosis</subject><subject>Autism Spectrum Disorder - epidemiology</subject><subject>Bayesian analysis</subject><subject>Biology and Life Sciences</subject><subject>Care and treatment</subject><subject>Centers for Disease Control and Prevention, U.S</subject><subject>Child</subject><subject>Children</subject><subject>Children &amp; youth</subject><subject>Classification</subject><subject>Computer and Information Sciences</subject><subject>Data mining</subject><subject>Deep Learning</subject><subject>Developmental disabilities</subject><subject>Disease control</subject><subject>Disease prevention</subject><subject>Epidemiological Monitoring</subject><subject>Feasibility Studies</subject><subject>Forests</subject><subject>Georgia</subject><subject>Humans</subject><subject>Information management</subject><subject>International conferences</subject><subject>Learning algorithms</subject><subject>Linguistics</subject><subject>Machine learning</subject><subject>Medicine and Health Sciences</subject><subject>Neural networks</subject><subject>People and Places</subject><subject>Pervasive developmental disorders</subject><subject>Physical Sciences</subject><subject>Prevalence</subject><subject>Public Health Surveillance - methods</subject><subject>Research and Analysis Methods</subject><subject>Risk factors</subject><subject>Sentiment analysis</subject><subject>Social Sciences</subject><subject>Support Vector Machine</subject><subject>Support vector machines</subject><subject>Surveillance</subject><subject>Surveillance systems</subject><subject>United States - epidemiology</subject><subject>Workflow</subject><subject>Workflow software</subject><issn>1932-6203</issn><issn>1932-6203</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2019</creationdate><recordtype>article</recordtype><sourceid>EIF</sourceid><sourceid>ABUWG</sourceid><sourceid>AFKRA</sourceid><sourceid>AZQEC</sourceid><sourceid>BENPR</sourceid><sourceid>CCPQU</sourceid><sourceid>DWQXO</sourceid><sourceid>GNUQQ</sourceid><sourceid>DOA</sourceid><recordid>eNqNk12L1DAUhoso7jr6D0QLgujFjGnSNu2NMCx-DCws-HUlhExy0mZIk5q0i_57053uMpW9kF60JM_7JuftOUnyPEObjNDs3cGN3nKz6Z2FDcIY14g-SM6zmuB1iRF5ePJ9ljwJ4YBQQaqyfJyckawoCKX5efJzmwrX9dzr4GzqVNpx0WoLqQHurbZNyk3jvB7aLqTK-XRoIQ2jvwZtDLcCJg0fBx26NPQgBj92qYxmXoJ_mjxS3AR4Nr9XyfePH75dfF5fXn3aXWwv16Ks8bAGkIQA5FJKVZC6RkCVqogiQKq6FiXUHCkiKkLyfVFkOK84rRTNFVUYFVKQVfLy6NsbF9gcTGAxEpphnCEUid2RkI4fWO91x_0f5rhmNwvON4z7QQsDDEu1pxwjQUmV7yvEi6IUNaoqgSQgWkev9_Np474DKcAOnpuF6XLH6pY17pqVtIz6yeDNbODdrxHCwDodBEx5ghtv7l1lOc1iwavk1T_o_dXNVMNjAdoqF88VkynblgjRnCA8eW3uoeIjodMidpHScX0heLsQRGaA30PDxxDY7uuX_2evfizZ1ydsC9wMbXAmNpGzYQnmR1B4F4IHdRdyhtg0BLdpsGkI2DwEUfbi9AfdiW67nvwFesECCg</recordid><startdate>20190925</startdate><enddate>20190925</enddate><creator>Lee, Scott H</creator><creator>Maenner, Matthew J</creator><creator>Heilig, Charles M</creator><general>Public Library of Science</general><general>Public Library of Science (PLoS)</general><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>IOV</scope><scope>ISR</scope><scope>3V.</scope><scope>7QG</scope><scope>7QL</scope><scope>7QO</scope><scope>7RV</scope><scope>7SN</scope><scope>7SS</scope><scope>7T5</scope><scope>7TG</scope><scope>7TM</scope><scope>7U9</scope><scope>7X2</scope><scope>7X7</scope><scope>7XB</scope><scope>88E</scope><scope>8AO</scope><scope>8C1</scope><scope>8FD</scope><scope>8FE</scope><scope>8FG</scope><scope>8FH</scope><scope>8FI</scope><scope>8FJ</scope><scope>8FK</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>ARAPS</scope><scope>ATCPS</scope><scope>AZQEC</scope><scope>BBNVY</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>BHPHI</scope><scope>C1K</scope><scope>CCPQU</scope><scope>D1I</scope><scope>DWQXO</scope><scope>FR3</scope><scope>FYUFA</scope><scope>GHDGH</scope><scope>GNUQQ</scope><scope>H94</scope><scope>HCIFZ</scope><scope>K9.</scope><scope>KB.</scope><scope>KB0</scope><scope>KL.</scope><scope>L6V</scope><scope>LK8</scope><scope>M0K</scope><scope>M0S</scope><scope>M1P</scope><scope>M7N</scope><scope>M7P</scope><scope>M7S</scope><scope>NAPCQ</scope><scope>P5Z</scope><scope>P62</scope><scope>P64</scope><scope>PATMY</scope><scope>PDBOC</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>PTHSS</scope><scope>PYCSY</scope><scope>RC3</scope><scope>7X8</scope><scope>5PM</scope><scope>DOA</scope><orcidid>https://orcid.org/0000-0003-4584-6758</orcidid></search><sort><creationdate>20190925</creationdate><title>A comparison of machine learning algorithms for the surveillance of autism spectrum disorder</title><author>Lee, Scott H ; Maenner, Matthew J ; Heilig, Charles M</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c692t-eed33ee4dddf53990e7ff83f3e3899c6e9a0f3c8334b551248a78f74f7f205dc3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2019</creationdate><topic>Accuracy</topic><topic>Algorithms</topic><topic>Artificial intelligence</topic><topic>Artificial neural networks</topic><topic>Autism</topic><topic>Autism Spectrum Disorder - diagnosis</topic><topic>Autism Spectrum Disorder - epidemiology</topic><topic>Bayesian analysis</topic><topic>Biology and Life Sciences</topic><topic>Care and treatment</topic><topic>Centers for Disease Control and Prevention, U.S</topic><topic>Child</topic><topic>Children</topic><topic>Children &amp; youth</topic><topic>Classification</topic><topic>Computer and Information Sciences</topic><topic>Data mining</topic><topic>Deep Learning</topic><topic>Developmental disabilities</topic><topic>Disease control</topic><topic>Disease prevention</topic><topic>Epidemiological Monitoring</topic><topic>Feasibility Studies</topic><topic>Forests</topic><topic>Georgia</topic><topic>Humans</topic><topic>Information management</topic><topic>International conferences</topic><topic>Learning algorithms</topic><topic>Linguistics</topic><topic>Machine learning</topic><topic>Medicine and Health Sciences</topic><topic>Neural networks</topic><topic>People and Places</topic><topic>Pervasive developmental disorders</topic><topic>Physical Sciences</topic><topic>Prevalence</topic><topic>Public Health Surveillance - methods</topic><topic>Research and Analysis Methods</topic><topic>Risk factors</topic><topic>Sentiment analysis</topic><topic>Social Sciences</topic><topic>Support Vector Machine</topic><topic>Support vector machines</topic><topic>Surveillance</topic><topic>Surveillance systems</topic><topic>United States - epidemiology</topic><topic>Workflow</topic><topic>Workflow software</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Lee, Scott H</creatorcontrib><creatorcontrib>Maenner, Matthew J</creatorcontrib><creatorcontrib>Heilig, Charles M</creatorcontrib><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>Gale In Context: Opposing Viewpoints</collection><collection>Gale In Context: Science</collection><collection>ProQuest Central (Corporate)</collection><collection>Animal Behavior Abstracts</collection><collection>Bacteriology Abstracts (Microbiology B)</collection><collection>Biotechnology Research Abstracts</collection><collection>Nursing &amp; Allied Health Database</collection><collection>Ecology Abstracts</collection><collection>Entomology Abstracts (Full archive)</collection><collection>Immunology Abstracts</collection><collection>Meteorological &amp; Geoastrophysical Abstracts</collection><collection>Nucleic Acids Abstracts</collection><collection>Virology and AIDS Abstracts</collection><collection>Agricultural Science Collection</collection><collection>Health &amp; Medical Collection</collection><collection>ProQuest Central (purchase pre-March 2016)</collection><collection>Medical Database (Alumni Edition)</collection><collection>ProQuest Pharma Collection</collection><collection>Public Health Database</collection><collection>Technology Research Database</collection><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>ProQuest Natural Science Collection</collection><collection>Hospital Premium Collection</collection><collection>Hospital Premium Collection (Alumni Edition)</collection><collection>ProQuest Central (Alumni) (purchase pre-March 2016)</collection><collection>Materials Science &amp; Engineering Collection</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>Advanced Technologies &amp; Aerospace Collection</collection><collection>Agricultural &amp; Environmental Science Collection</collection><collection>ProQuest Central Essentials</collection><collection>Biological Science Collection</collection><collection>ProQuest Central</collection><collection>Technology Collection</collection><collection>Natural Science Collection</collection><collection>Environmental Sciences and Pollution Management</collection><collection>ProQuest One Community College</collection><collection>ProQuest Materials Science Collection</collection><collection>ProQuest Central Korea</collection><collection>Engineering Research Database</collection><collection>Health Research Premium Collection</collection><collection>Health Research Premium Collection (Alumni)</collection><collection>ProQuest Central Student</collection><collection>AIDS and Cancer Research Abstracts</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Health &amp; Medical Complete (Alumni)</collection><collection>Materials Science Database</collection><collection>Nursing &amp; Allied Health Database (Alumni Edition)</collection><collection>Meteorological &amp; Geoastrophysical Abstracts - Academic</collection><collection>ProQuest Engineering Collection</collection><collection>ProQuest Biological Science Collection</collection><collection>Agricultural Science Database</collection><collection>Health &amp; Medical Collection (Alumni Edition)</collection><collection>Medical Database</collection><collection>Algology Mycology and Protozoology Abstracts (Microbiology C)</collection><collection>Biological Science Database</collection><collection>Engineering Database</collection><collection>Nursing &amp; Allied Health Premium</collection><collection>Advanced Technologies &amp; Aerospace Database</collection><collection>ProQuest Advanced Technologies &amp; Aerospace Collection</collection><collection>Biotechnology and BioEngineering Abstracts</collection><collection>Environmental Science Database</collection><collection>Materials Science Collection</collection><collection>Access via ProQuest (Open Access)</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>Engineering Collection</collection><collection>Environmental Science Collection</collection><collection>Genetics Abstracts</collection><collection>MEDLINE - Academic</collection><collection>PubMed Central (Full Participant titles)</collection><collection>DOAJ Directory of Open Access Journals</collection><jtitle>PloS one</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Lee, Scott H</au><au>Maenner, Matthew J</au><au>Heilig, Charles M</au><au>Yan, Jingwen</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>A comparison of machine learning algorithms for the surveillance of autism spectrum disorder</atitle><jtitle>PloS one</jtitle><addtitle>PLoS One</addtitle><date>2019-09-25</date><risdate>2019</risdate><volume>14</volume><issue>9</issue><spage>e0222907</spage><epage>e0222907</epage><pages>e0222907-e0222907</pages><issn>1932-6203</issn><eissn>1932-6203</eissn><abstract>The Centers for Disease Control and Prevention (CDC) coordinates a labor-intensive process to measure the prevalence of autism spectrum disorder (ASD) among children in the United States. Random forests methods have shown promise in speeding up this process, but they lag behind human classification accuracy by about 5%. We explore whether more recently available document classification algorithms can close this gap. Using data gathered from a single surveillance site, we applied 8 supervised learning algorithms to predict whether children meet the case definition for ASD based solely on the words in their evaluations. We compared the algorithms' performance across 10 random train-test splits of the data, using classification accuracy, F1 score, and number of positive calls to evaluate their potential use for surveillance. Across the 10 train-test cycles, the random forest and support vector machine with Naive Bayes features (NB-SVM) each achieved slightly more than 87% mean accuracy. The NB-SVM produced significantly more false negatives than false positives (P = 0.027), but the random forest did not, making its prevalence estimates very close to the true prevalence in the data. The best-performing neural network performed similarly to the random forest on both measures. The random forest performed as well as more recently available models like the NB-SVM and the neural network, and it also produced good prevalence estimates. NB-SVM may not be a good candidate for use in a fully-automated surveillance workflow due to increased false negatives. More sophisticated algorithms, like hierarchical convolutional neural networks, may not be feasible to train due to characteristics of the data. Current algorithms might perform better if the data are abstracted and processed differently and if they take into account information about the children in addition to their evaluations. Deep learning models performed similarly to traditional machine learning methods at predicting the clinician-assigned case status for CDC's autism surveillance system. While deep learning methods had limited benefit in this task, they may have applications in other surveillance systems.</abstract><cop>United States</cop><pub>Public Library of Science</pub><pmid>31553774</pmid><doi>10.1371/journal.pone.0222907</doi><tpages>e0222907</tpages><orcidid>https://orcid.org/0000-0003-4584-6758</orcidid><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier ISSN: 1932-6203
ispartof PloS one, 2019-09, Vol.14 (9), p.e0222907-e0222907
issn 1932-6203
1932-6203
language eng
recordid cdi_plos_journals_2297122100
source MEDLINE; DOAJ Directory of Open Access Journals; Elektronische Zeitschriftenbibliothek - Frei zugängliche E-Journals; Public Library of Science (PLoS) Journals Open Access; PubMed Central; Free Full-Text Journals in Chemistry
subjects Accuracy
Algorithms
Artificial intelligence
Artificial neural networks
Autism
Autism Spectrum Disorder - diagnosis
Autism Spectrum Disorder - epidemiology
Bayesian analysis
Biology and Life Sciences
Care and treatment
Centers for Disease Control and Prevention, U.S
Child
Children
Children & youth
Classification
Computer and Information Sciences
Data mining
Deep Learning
Developmental disabilities
Disease control
Disease prevention
Epidemiological Monitoring
Feasibility Studies
Forests
Georgia
Humans
Information management
International conferences
Learning algorithms
Linguistics
Machine learning
Medicine and Health Sciences
Neural networks
People and Places
Pervasive developmental disorders
Physical Sciences
Prevalence
Public Health Surveillance - methods
Research and Analysis Methods
Risk factors
Sentiment analysis
Social Sciences
Support Vector Machine
Support vector machines
Surveillance
Surveillance systems
United States - epidemiology
Workflow
Workflow software
title A comparison of machine learning algorithms for the surveillance of autism spectrum disorder
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-23T01%3A57%3A20IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-gale_plos_&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=A%20comparison%20of%20machine%20learning%20algorithms%20for%20the%20surveillance%20of%20autism%20spectrum%20disorder&rft.jtitle=PloS%20one&rft.au=Lee,%20Scott%20H&rft.date=2019-09-25&rft.volume=14&rft.issue=9&rft.spage=e0222907&rft.epage=e0222907&rft.pages=e0222907-e0222907&rft.issn=1932-6203&rft.eissn=1932-6203&rft_id=info:doi/10.1371/journal.pone.0222907&rft_dat=%3Cgale_plos_%3EA600743023%3C/gale_plos_%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2297122100&rft_id=info:pmid/31553774&rft_galeid=A600743023&rft_doaj_id=oai_doaj_org_article_2dfb7a20c7384b80a556c9088c0de079&rfr_iscdi=true