PDF(984 KB)
PDF(984 KB)
PDF(984 KB)
基于Wasserstein GAN的新一代人工智能小样本数据增强方法——以生物领域癌症分期数据为例
Wasserstein GAN-Based Small-Sample Augmentation for New-Generation Artificial Intelligence: A Case Study of Cancer-Staging Data in Biology
It is essential to utilize deep-learning algorithms based on big data for the implementation of the new generation of artificial intelligence. Effective utilization of deep learning relies considerably on the number of labeled samples, which restricts the application of deep learning in an environment with a small sample size. In this paper, we propose an approach based on a generative adversarial network (GAN) combined with a deep neural network (DNN). First, the original samples were divided into a training set and a test set. The GAN was trained with the training set to generate synthetic sample data, which enlarged the training set. Next, the DNN classifier was trained with the synthetic samples. Finally, the classifier was tested with the test set, and the effectiveness of the approach for multi-classification with a small sample size was validated by the indicators. As an empirical case, the approach was then applied to identify the stages of cancers with a small labeled sample size. The experimental results verified that the proposed approach achieved a greater accuracy than traditional methods. This research was an attempt to transform the classical statistical machine-learning classification method based on original samples into a deep-learning classification method based on data augmentation. The use of this approach will contribute to an expansion of application scenarios for the new generation of artificial intelligence based on deep learning, and to an increase in application effectiveness. This research is also expected to contribute to the comprehensive promotion of new-generation artificial intelligence.
Artificial intelligence / Generative adversarial network / Deep neural network / Small sample size / Cancer
| [1] |
Crevier D.. AI: the tumultuous history of the search for artificial intelligence.
|
| [2] |
Pan Y.. Heading toward Artificial Intelligence 2.0. Engineering. 2016; 2(4): 409-413.
|
| [3] |
State Council of the People’s Republic of China. Development Plan for a Next-Generation Artificial Intelligence [Internet]. Beijing: www.gov.cn. [cited 2018 Mar 5]. Available from: http://english.gov.cn/policies/latest_releases/2017/07/20/content_281475742458322.htm.
|
| [4] |
State Council Information Office of the People’s Republic of China. The policy interpretation of Development Planning for a Next-Generation Artificial Intelligence [Internet]. Beijing: www.scio.gov.cn. [cited 2018 Mar 5]. Available from: http://www.scio.gov.cn/34473/34515/Document/1559231/1559231.htm. Chinese.
|
| [5] |
Hinton G.E., Salakhutdinov R.R.. Reducing the dimensionality of data with neural networks. Science. 2006; 313(5786): 504-507.
|
| [6] |
Zhuang Y., Chen C., Pan Y.. Challenges and opportunities: from big data to knowledge in AI 2.0. Front Inf Technol Electronic Eng. 2017; 18(1): 3-14.
|
| [7] |
Al-Qizwini M., Barjasteh I., Al-Qassab H., Radha H.. Deep learning algorithm for autonomous driving using GoogLeNet. In: Proceedings of the 2017 IEEE Intelligent Vehicles Symposium; 2017 Jun 11–14; Los Angeles, CA, USA. New York: IEEE; 2017. p. 89-96.
|
| [8] |
Wang L, Sng D. Deep learning algorithms with applications to video analytics for a smart city: a survey. 2016: arXiv:1511.06434.
|
| [9] |
Mohamed A., Dahl G., Hinton G.. Acoustic modeling using deep belief networks. IEEE Trans Audio Speech Lang Process. 2012; 20(1): 14-22.
|
| [10] |
Jones N. Artificial-intelligence institute launches free science search engine [Internet]. Heidelberg: Springer Nature. c2018 [cited 2018 Mar 5]. Available from: https://www.nature.com/news/artificial-intelligence-institute-launches-free-science-search-engine-1.18703.
|
| [11] |
Goodfellow I., Bengio Y., Courville A.. Deep learning.
|
| [12] |
Zhuang F., Luo P., Qing H., Shi Z.. Survey on transfer learning research. J Software. 2015; 26: 26-39. Chinese
|
| [13] |
Chen J., Yang J., Zhou H., Xiang H., Zhu Z., Li Y.,
|
| [14] |
Urban F., Zhou Y., Nordensvard J., Narain A.. Firm-level technology transfer and technology cooperation for wind energy between Europe, China and India: from north–south to south–north cooperation?. Energy Sustainable Dev. 2015; 28: 29-40.
|
| [15] |
Zhou Y., Zhang H., Ding M., Su J.. How public demonstration project affects the emergence of a new industry: an empirical study on electric vehicle demonstration project in China. In: Proceedings of the 2013 Suzhou Silicon Valley–Beijing International Innovation Conference; 2013 Jul 8–9. New York: IEEE; 2013. p. 234-239.
|
| [16] |
Zhou Y., Minshall T.. Building global products and competing in innovation: the role of Chinese university spin–outs and required innovation capabilities. Int J Technol Manage. 2014; 64(2): 180-209.
|
| [17] |
Xu G., Wu Y., Minshall T., Zhou Y.. Exploring innovation ecosystems across science, technology, and business: a case of 3D printing in China. Technol Forecast Social Change. 2017; 136: 180-221.
|
| [18] |
Li X., Zhou Y., Xue L., Huang L.. Roadmapping for industrial emergence and innovation gaps to catch-up: a patent analysis of OLED industry in China. Int J Technol Manage. 2016; 7(1–3): 105-143.
|
| [19] |
Li X., Zhou Y., Xue L., Huang L.. Integrating bibliometrics and roadmapping methods: a case of dye-sensitized solar cell technology-based industry in China. Technol Forecast Social Change. 2015; 97: 205-222.
|
| [20] |
Zhou Y., Pan M., Urban F.. Comparing the international knowledge flow of China’s wind and solar photovoltaic (PV) industries: patent analysis and implications for sustainable development. Sustainability. 2018; 10(6): 1883.
|
| [21] |
Theodoridis S., Koutroumbas K.. Pattern recognition. 3rd ed.
|
| [22] |
Nordensvard J., Zhou Y., Zhang X.. Innovation core, innovation semi-periphery and technology transfer: the case of wind energy patents. Energy Policy. 2018; 120: 213-227.
|
| [23] |
Pan M, Zhou Y, Zhou DK. Comparing the innovation strategies of Chinese and European wind turbine firms through a patent lens. Environ Innovation Societal Transitions. Epub 2017 Dec 27.
|
| [24] |
Zhou Y., Pan M., Zhou D.K., Xue L.. Stakeholder risk and trust perceptions in the diffusion of green manufacturing technologies: evidence from China. J Environ Dev. 2017; 27(1): 46-73.
|
| [25] |
Zhou Y., Li X., Lema R., Urban F.. Comparing the knowledge bases of wind turbine firms in Asia and Europe: patent trajectories, networks, and globalisation. Sci Public Policy. 2016; 43(2): 476-491.
|
| [26] |
Chen L., Xu J., Zhou Y.. Regulating the environmental behavior of manufacturing SMEs: interfirm alliance as a facilitator. J Cleaner Prod. 2017; 165: 393-404.
|
| [27] |
DeRouin E., Brown J.. Neural network training on unequally represented classes.
|
| [28] |
Chawla N.V., Bowyer K.W., Hall L.O., Kegelmeyer W.P.. SMOTE: synthetic minority over-sampling technique. J Artif Intell Res. 2002; 16(1): 321-357.
|
| [29] |
Han H., Wang W.Y., Mao B.H.. Borderline-SMOTE: a new over-sampling method in imbalanced data sets learning. In:
|
| [30] |
He H., Bai Y., Garcia E.A., Shutao L.. ADASYN: adaptive synthetic sampling approach for imbalanced learning. In: Proceedings of the 2008 IEEE International Joint Conference on Neural Networks; 2008 Jun 1–8; Hong Kong, China. New York: IEEE; 2008. p. 1322-1328.
|
| [31] |
Barua S., Islam M.M., Yao X., Kazuyuki M.. MWMOTE–majority weighted minority oversampling technique for imbalanced data set learning. IEEE Trans Knowl Data Eng. 2014; 26(2): 405-425.
|
| [32] |
Xie Z., Jiang L., Ye T., Li X.. A synthetic minority oversampling method based on local densities in low-dimensional space for imbalanced learning. In:
|
| [33] |
Douzas G., Bacao F.. Self-Organizing Map Oversampling (SOMO) for imbalanced data set learning. Expert Syst Appl. 2017; 82: 40-52.
|
| [34] |
Bishop C.M.. Training with noise is equivalent to Tiknonov regularization. Neural Comput. 1995; 7(1): 108-116.
|
| [35] |
Zhou Z., Jiang Y.. Nec4.5: neural ensemble based C4.5. IEEE Trans Knowl Data Eng. 2004; 16(6): 770-773.
|
| [36] |
Li D.C., Lin Y.S.. Using virtual sample generation to build up management knowledge in the early manufacturing stages. Eur J Operat Res. 2006; 175(1): 413-434.
|
| [37] |
Li D., Fang Y.. A non-linearly virtual sample generation technique using group discovery and parametric equations of hypersphere. Exp Syst Appl. 2009; 36(1): 844-851.
|
| [38] |
Wang K., Gou C., Duan Y., Lin Y., Zheng X., Wang F.. Generative adversarial networks: introduction and outlook. IEEE/CAA J Autom Sin. 2017; 4(4): 588-598.
|
| [39] |
Goodfellow I., Pouget-Abadie J., Mirza M., Xu B., Warde-Farley D., Ozair S.,
|
| [40] |
Creswell A., White T., Dumoulin V., Arulkumaran K., Sengupta B., Bharath A.A.. Generative adversarial networks: an overview. IEEE Signal Process Mag. 2018; 35(1): 53-65.
|
| [41] |
Radford A, Metz L, Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks. 2015:arXiv:1512.03131.
|
| [42] |
Santana E, Hotz G. Learning a driving simulator. 2016:arXiv:1608.01230.
|
| [43] |
Gou C., Wu Y., Wang K., Wang F., Ji Q.. Learning-by-synthesis for accurate eye detection. In: Proceedings of the 23rd International Conference on Pattern Recognition; 2016 Dec 4–8; Cancun, Mexico. New York: IEEE; 2017. p. 3362-3367.
|
| [44] |
Li J, Monroe W, Shi T, Jean S, Ritter A, Jurafsky D. Adversarial learning for neural dialogue generation. 2017:arXiv:1701.06547.
|
| [45] |
Pascual S, Bonafonte A, Serrà J. SEGAN: speech enhancement generative adversarial network. 2017:arXiv:1703.09452.
|
| [46] |
Fiore U., Santis A.D., Perla F., Zanetti P., Palmieri F.. Using generative adversarial networks for improving classification effectiveness in credit card fraud detection. Inf Sci. 2019; 479: 448-455.
|
| [47] |
Douzas G., Bacao F.. Effective data generation for imbalanced learning using conditional generative adversarial networks. Expert Syst Appl. 2017; 91: 464-471.
|
| [48] |
Bloice MD, Stocker C, Holzinger A. Augmentor: an image augmentation library for machine learning. 2017:arXiv:1708.04680
|
| [49] |
Arjovsky M, Chintala S, Bottou L. Wasserstein GAN. 2017:arXiv:1701.07875.
|
| [50] |
Ratliff L.J., Burden S.A., Sastry S.S.. Characterization and computation of local Nash equilibria in continuous games. In: Proceedings of the 51st Annual Allerton Conference on Communication, Control, and Computing; 2013 Oct 2–4; Monticello, IL, USA. New York: IEEE; 2006. p. 917-924.
|
| [51] |
Danihelka I, Lakshminarayanan B, Uria B, Wierstra D, Dayan P. Comparison of maximum likelihood and GAN-based training of real NVPs. 2017:arXiv:1705.05263.
|
| [52] |
Yang Q., Yan P., Zhang Y., Yu H., Shi Y., Mou X.,
|
| [53] |
Mcdaniel P., Papernot N., Celik Z.B.. Machine learning in adversarial settings. IEEE Secur Privacy. 2016; 14(3): 68-72.
|
| [54] |
Sousa L.R., Miranda T., Sousa R.L., Tinoco J.. The use of data mining techniques in rockburst risk assessment. Engineering. 2017; 3(4): 552-558.
|
| [55] |
Sun Y., Kamel M.S., Wong A.K.C., Wang Y.. Cost-sensitive boosting for classification of imbalanced data. Pattern Recognit. 2007; 40(12): 3358-3378.
|
| [56] |
Farazi A.P., DePinho R.A.. Hepatocellular carcinoma pathogenesis: from genes to environment. Nat Rev Cancer. 2006; 6(9): 674-687.
|
| [57] |
Arzumanyan A., Reis H.M.G.P.V., Feitelson M.A.. Pathogenic mechanisms in HBV- and HCV-associated hepatocellular carcinoma. Nat Rev Cancer. 2013; 13(2): 123-135.
|
| [58] |
Mechref Y., Hu Y., Garcia A., Zhou S., Desantos-Garcia J.L., Hussein A.. Defining putative glycan cancer biomarkers by MS. Bioanalysis. 2012; 4(20): 2457-2469.
|
| [59] |
Tang Z., Varghese R.S., Bekesova S., Loffredo C.A., Hamid M.A., Kyselova Z.,
|
| [60] |
Kronewitter S.R., De Leoz M.L.A., Strum J.S., An H.J., Dimapasoc L.M., Guerrero A.,
|
| [61] |
Pierce M., Buckhaults P., Chen L., Fregien N.. Regulation of N-acetylglucosaminyltransferase V and Asn-linked oligosaccharide β(1,6) branching by a growth factor signaling pathway and effects on cell adhesion and metastatic potential. Glycoconjugate J. 1997; 14(5): 623-630.
|
| [62] |
Lau K.S., Dennis J.W.. N-Glycans in cancer progression. Glycobiology. 2008; 18(10): 750-760.
|
| [63] |
Saldova R., Royle L., Radcliffe C.M., Abd Hamid U.M., Evans R., Arnold J.N.,
|
| [64] |
Noda K., Miyoshi E., Gu J., Gao C.X., Nakahara S., Kitada T.,
|
| [65] |
Basu P.S., Majhi R., Batabyal S.K.. Lectin and serum-PSA interaction as a screening test for prostate cancer. Clin Biochem. 2003; 36(5): 373-376.
|
| [66] |
Arnold J.N., Saldova R., Hamid U.M.A., Rudd P.M.. Evaluation of the serum N-linked glycome for the diagnosis of cancer and chronic inflammation. Proteomics. 2008; 8(16): 3284-3293.
|
| [67] |
Adamczyk B., Tharmalingam T., Rudd P.M.. Glycans as cancer biomarkers. Biochim Biophys Acta Gen Subj. 2012; 1820(9): 1347-1353.
|
| [68] |
Deguchi K., Keira T., Yamada K., Ito H., Takegawa Y., Nakagawa H.,
|
| [69] |
Siemerink E., Mulder N.H., Brouwers A.H., Hospers G.A.. Early prediction of response to sorafenib treatment in patients with hepatocellular carcinoma (HCC) with 18F-fluorodeoxyglucose-positron emission tomography (18F-FDG-PET). J Clin Oncol. 2008; 26(21): 1-15.
|
| [70] |
Holzinger A., Malle B., Kieseberg P., Roth P.M., Müller H., Reihs R.. Machine learning and knowledge extraction in digital pathology needs an integrative approach. lecture notes in computer scienceIn:
|
| [71] |
Breiman L.. Random forests. Mach Learn. 2001; 45(1): 5-32.
|
| [72] |
Mitchell T.. Machine learning. 1st ed. Zeng H, Zhang Y, translator. Beijing: China Machine Press; 2003. Chinese
|
| [73] |
Liu P., Zhou Y., Zhou D.K., Xue L.. Energy Performance Contract models for the diffusion of green-manufacturing technologies in China: a stakeholder analysis from SMEs’ perspective. Energy Policy. 2017; 106: 59-67.
|
| [74] |
Kong D., Feng Q., Zhou Y., Xue L.. Local implementation for green-manufacturing technology diffusion policy in China: from the user firms’ perspectives. J Cleaner Prod. 2016; 129: 113-124.
|
| [75] |
Zhou Y., Xu G., Minshall T., Liu P.. How do public demonstration projects promote green-manufacturing technologies? A case study from China. Sustainable Dev. 2015; 23(4): 217-231.
|
| [76] |
Kong D., Zhou Y., Liu Y., Xue L.. Using the data mining method to assess the innovation gap: a case of industrial robotics in a catching-up country. Technol Forecasting Social Change. 2017; 119: 80-97.
|
| [77] |
Li M., Zhou Y.. Visualizing the knowledge profile on self-powered technology. Nano Energy. 2018; 51: 250-259.
|
| [78] |
Wang B., Liu Y., Zhou Y., Wen Z.. Emerging nanogenerator technology in China: a review and forecast using integrating bibliometrics, patent analysis and technology roadmapping methods. Nano Energy. 2018; 46: 322-330.
|
This work was supported by the National Natural Science Foundation of China (91646102, L1724034, L16240452, L1524015, and 20905027), the MOE (Ministry of Education in China) Project of Humanities and Social Sciences (16JDGC011), the Chinese Academy of Engineering’s China Knowledge Center for Engineering Sciences and Technology Project (CKCEST-2018-1-13), the UK–China Industry Academia Partnership Programme (UK-CIAPP\260), Volvo-Supported Green Economy and Sustainable Development at Tsinghua University (20153000181), and the Tsinghua Initiative Research Project (2016THZW).
Yufei Liu, Yuan Zhou, Xin Liu, Fang Dong, Chang Wang, and Zihong Wang declare that they have no conflict of interest or financial conflicts to disclose.
/
| 〈 |
|
〉 |
