File Download

There are no files associated with this item.

  Links for fulltext
     (May Require Subscription)
Supplementary

Conference Paper: Testing and validating machine learning classifiers by metamorphic testing

TitleTesting and validating machine learning classifiers by metamorphic testing
Authors
KeywordsValidation
Test oracle
Oracle problem
Metamorphic testing
Machine learning
Verification
Issue Date2011
Citation
Journal of Systems and Software, 2011, v. 84, n. 4, p. 544-558 How to Cite?
AbstractMachine learning algorithms have provided core functionality to many application domains - such as bioinformatics, computational linguistics, etc. However, it is difficult to detect faults in such applications because often there is no "test oracle" to verify the correctness of the computed outputs. To help address the software quality, in this paper we present a technique for testing the implementations of machine learning classification algorithms which support such applications. Our approach is based on the technique "metamorphic testing", which has been shown to be effective to alleviate the oracle problem. Also presented include a case study on a real-world machine learning application framework, and a discussion of how programmers implementing machine learning algorithms can avoid the common pitfalls discovered in our study. We also conduct mutation analysis and cross-validation, which reveal that our method has high effectiveness in killing mutants, and that observing expected cross-validation result alone is not sufficiently effective to detect faults in a supervised classification program. The effectiveness of metamorphic testing is further confirmed by the detection of real faults in a popular open-source classification program. © 2010 Elsevier Inc. All rights reserved.
Persistent Identifierhttp://hdl.handle.net/10722/262637
ISSN
2019 Impact Factor: 2.45
2015 SCImago Journal Rankings: 0.897
ISI Accession Number ID

 

DC FieldValueLanguage
dc.contributor.authorXie, Xiaoyuan-
dc.contributor.authorHo, Joshua W.K.-
dc.contributor.authorMurphy, Christian-
dc.contributor.authorKaiser, Gail-
dc.contributor.authorXu, Baowen-
dc.contributor.authorChen, Tsong Yueh-
dc.date.accessioned2018-10-08T02:46:36Z-
dc.date.available2018-10-08T02:46:36Z-
dc.date.issued2011-
dc.identifier.citationJournal of Systems and Software, 2011, v. 84, n. 4, p. 544-558-
dc.identifier.issn0164-1212-
dc.identifier.urihttp://hdl.handle.net/10722/262637-
dc.description.abstractMachine learning algorithms have provided core functionality to many application domains - such as bioinformatics, computational linguistics, etc. However, it is difficult to detect faults in such applications because often there is no "test oracle" to verify the correctness of the computed outputs. To help address the software quality, in this paper we present a technique for testing the implementations of machine learning classification algorithms which support such applications. Our approach is based on the technique "metamorphic testing", which has been shown to be effective to alleviate the oracle problem. Also presented include a case study on a real-world machine learning application framework, and a discussion of how programmers implementing machine learning algorithms can avoid the common pitfalls discovered in our study. We also conduct mutation analysis and cross-validation, which reveal that our method has high effectiveness in killing mutants, and that observing expected cross-validation result alone is not sufficiently effective to detect faults in a supervised classification program. The effectiveness of metamorphic testing is further confirmed by the detection of real faults in a popular open-source classification program. © 2010 Elsevier Inc. All rights reserved.-
dc.languageeng-
dc.relation.ispartofJournal of Systems and Software-
dc.subjectValidation-
dc.subjectTest oracle-
dc.subjectOracle problem-
dc.subjectMetamorphic testing-
dc.subjectMachine learning-
dc.subjectVerification-
dc.titleTesting and validating machine learning classifiers by metamorphic testing-
dc.typeConference_Paper-
dc.description.naturelink_to_subscribed_fulltext-
dc.identifier.doi10.1016/j.jss.2010.11.920-
dc.identifier.scopuseid_2-s2.0-79751532765-
dc.identifier.volume84-
dc.identifier.issue4-
dc.identifier.spage544-
dc.identifier.epage558-
dc.identifier.isiWOS:000288142500003-
dc.identifier.issnl0164-1212-

Export via OAI-PMH Interface in XML Formats


OR


Export to Other Non-XML Formats