Nonvolatile computing-in-memory (nvCIM) exhibits high potential for neuromorphic computing involving massive parallel computations and for achieving high energy efficiency. nvCIM is especially suitable for deep neural networks, which are required to perform large amounts of matrix-vector multiplications. However, a comprehensive quantization algorithm has yet to be developed, which overcomes the hardware limitations of resistive random access memory (ReRAM)-based nvCIM, such as the number of I/O, word lines (WLs), and ADC outputs. In this article, we propose a quantization training method for compressing deep models. The method comprises three steps: input and weight quantization, ReRAM convolution (ReConv), and ADC quantization. ADC quantization optimizes the error sampling problem by using the Gumbel-softmax trick. Under a 4-bit ADC of nvCIM, the accuracy only decreases by 0.05% and 1.31% for the MNIST and CIFAR-10, respectively, compared with the corresponding accuracies obtained under an ideal ADC. The experimental results indicate that the proposed method is effective for compensating the hardware limitations of nvCIM macros. INDEX TERMS Compression, computing-in-memory (CIM), deep learning, quantization, resistive random access memory (ReRAM).
Introduction Electronic nose (E-nose) has many applications in gas detection and classification such as identifying toxic gases from the environment or detecting breath biomarkers for various cancer diseases. The E-noses are usually designed using an array of gas sensors and a machine learning classifier, which is comprised of various models to distinguish the gas sensor data. However, the response of the gas sensors often suffers from unpredictable and uncertain drift issues due to sensor aging, process variation, and environmental interference. From the perspective of pattern recognition, drift causes test data distribution to differ from prior data distribution and reduce the classification accuracy. To solve the above-mentioned problems, drift correction methods such as component correction based on principle component analysis (CC-PCA) and orthogonal signal correction (OSC) can be employed. The correction methods try to remove some components from the data, while the drift is a dynamic and nonlinear one that cannot be easily separated from the data. The other method can be adaption in which new labeled data collected at different time or from different devices are required to update the classifier. The adaption method has better classification capability than the correction method; however, collecting new labeled data or transferring data is a laborious job. Hence, this paper proposes a transfer learning method to adapt different distributions of data without any additional data. Since no transfer data is required, the learning method can also be named as zero-shot learning. Method Fig. 1 shows the proposed method that considers both sensor response correction and adaption. This method combines auto-encoder as well as a neural network (NN) classifier with certain restrictions. The auto-encoder comprises an encoder and a decoder made by a 64-30-20 fully-connected neural network and a 20-30-64 fully-connected neural network, respectively. The purpose of the auto-encoder is to find latent vectors that can highly represent the input data. To find a highly representative latent vector, the encoder encodes the input sample to a 20-dimension latent vector, and then this latent vector is used to reconstruct a 64-dimension vector through the decoder. By forcing the reconstructed vector close to the input data, the latent vector can extract useful information from the input data and represent the corresponding input data. However, a highly representative latent space made up of latent vectors is not equivalent to a space that is easy to be clustered into different classes. For seeking a better classifiable latent space, the performance of the classifier is taken into consideration. In this work, a 20-10-6 fully-connected neural network is adopted as a classifier. In addition, statistical measurements are introduced to determine the transfer sample. The proposed method can be divided into two main stages: (1) Training stage (2) Calibration stage. During the training stage, training data are used to train the auto-encoder and the classifier along with the objective function including reconstruction error, classification error, and statistic error. The reconstruction error is the mean square error between the reconstructed data and the input. The classification error is determined by the classifier, which would be the binary cross-entropy error between NN output and the true label. Further, the statistic error measures the sparsity of the data distribution. For the calibration stage, some unknown samples are treated as test data. First, the test data will be given as the input of the model, consequently, the output of the model is obtained in the form of latent vector and classification result. Then, the statistic error can be calculated by the latent vector and the training data distribution in latent space. According to the customized qualification, the sample is determined to be reliable or not. If it is reliable, then it is put into the transfer dataset with its classification result as the label. Finally, the transfer dataset is used to calibrate the pre-train model through utilizing the transfer dataset as the training data in the first stage. Results and Conclusion The proposed method is evaluated on an open dataset discuss in [1], which collects 13,910 samples using a 16 metal-oxide (MOX) gas sensor array over a period of 36 months. While the dataset is processed using the proposed method, attention should be paid in determining what kind of sample is reliable. If less reliable samples are used, the model might collapse. On the other hand, if the transfer dataset contains only highly reliable samples, the classifier may be stuck with a similar distribution of training data, which means it is unable to adopt the new data distribution and leads to a big hurdle. Hence, the constraints of selecting transfer data are varied with calibration epochs. This work is the first zero-shot transfer learning method applied to E-nose data. The experimental results show that the performance is improved after data transfer. Besides, the other calibration methods on this dataset usually select the transfer data which can describe the overall data distribution in the test batch. Yet, the proposed method can be more practical and general, because a similar operation can be also applied to calibrate the instrumental variation. Furthermore, the proposed method lacks human labeling after the pre-trained model. That is, it is possible to obtain a better result working together with the traditional method and collecting newly labeled transfer data once in a while. Reference [1] A. Vergara, S. Vembu, T. Ayhan, M.A. Ryan, M.L. Homer, R. Huerta, Chemical gas sensor drift compensation using classifier ensembles, Sens. Actuators B: Chem. 166 (2012) 320-329 Figure 1
Introduction Human exhaled breath contains more than 1,000 of the vast variety of volatile organic compounds (VOCs), providing valuable information about the metabolic process in human beings. The information of breath could include current state of disease, leading to great potential of noninvasive diagnosis in medical industry 1-6. The concentration of the exhaled breath VOCs varies in sub-ppm or even lower in ppb level in healthy people7. However, when disease occurs in the human body, the metabolic process becomes disbalance. Hence, the concentration profile of exhaled breath VOCs drastically increases. Recently, deaths caused by lung cancer have reached 1.6 million each year8. Early screening and diagnosing of lung cancer are big challenges in the healthcare industry. There are many technologies to detect and diagnose lung cancer, such as low-dose chest computed tomography (CT), which enhances the likelihood of early-stage tumor diagnosis9, resulting in an increase in the survival rate10. However, the aforementioned technique still suffers from a large rate of false positive due to cross reactive responses10. In addition, due to the existence of a low dose of ionizing radiation in the CT, employing this technique increases the risk of cancer. Therefore, a noninvasive technology for breath analysis is desired to diagnose lung cancer. In this paper, we report a fast screening method of the lung cancer biomarker in exhaled breath by using gas chromatography-mass spectroscopy coupled with thermal desorption (TD-GC-MS), which is one of the noninvasive technologies that is able to perform this task. In the experiments, Tenax TA material is used as absorbent to absorb the exhaled breath VOCs, thermal desorption system is used to desorb the VOCs, which are separated by the gas chromatography column and further detected by the mass spectrometer. Method A 1 Liter Tedlar bag was used to sample the exhaled breath for the lung cancer patients at the NTU hospital Hsinchu. Further, the breath sample was transferred from Tedlar bag to Tenax TA tube with a flow rate of 40 cc/min for 25 minutes. An Agilent type7890A GC system with 5975C inert MSD with a triple-axis detector along with Perkin Elmer thermal desorption system (Turbo matrix 100) was used. Breath VOCs were separated by Elite-5 MS column (30 m × 0.25 mm, film thickness 0.25μm, Perkin Elmer) while working in a constant pressure mode (10 psi). The mass spectrometer was set to scan mode. The program of column temperature was maintained at 80 for 4 min, and then increased at a rate of 15 per min, to 230 and held at 230 for 4 min11. After breath sampling, Tenax TA absorbent was used to absorb the volatile organic compound, then the Tenax TA was connected to the Perkin Elmer thermal desorption system. Results and Conclusions Gas chromatography mass Spectro meter coupled with the thermal desorption system was used in the detection of lower concentration of the volatile organic compound in the exhaled breath for lung cancer patients. The thermal desorption process was used to desorb the volatile organic compound from the Tenax TA absorbent. The desorbed samples from Tenax TA absorbent were transferred through the column. The VOCs were moved into the column by the inert gas mobile phase; then, they were separated by the stationary phase fixed into the column. The separation efficiency depended upon the gas chromatography column. The detection principle of the GC-MS was based on the mass to charge ratio (M/Z) of the ionized atom for the detection of biomarkers in the lung cancer patients. After separation in gas chromatography, VOCs were detected by the mass spectrometer. The peak area of various kind of VOCs for lung cancer have been obtained. We have found some biomarkers from lung cancer’s exhaled breath such as acetone, toluene, ethyl benzene, decane, etc. In future, we will sample the breath for the control group, after analysis by GC-MS compare the result with lung cancer patient to identify the unique biomarkers. The Figure 1 shows the profile of the peak area with respect to the various kind of VOCs for the lung cancer patient. Therefore, the TD-GC-MS is an effective technique for noninvasive diagnosis by employing exhaled breath VOCs for the health care industry. These exhaled breath biomarkers can be used to screen the early lung cancerous disease to save millions of lives worldwide. Figure 1
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.