A UNIFIED PARADIGM OF CLASSIFYING GI TRACT DISEASES IN ENDOSCOPY IMAGES USING MULTIPLE FEATURES FUSION

Details

Hits: 1019

Volume 6 (1), June 2023, Pages 49-76

Muhammad Afraz, Abdul Muiz Fayyaz, Abdul Haseeb

COMSATS University Islamabad Wah Campus, Pakistan, This email address is being protected from spambots. You need JavaScript enabled to view it., This email address is being protected from spambots. You need JavaScript enabled to view it., This email address is being protected from spambots. You need JavaScript enabled to view it.

Abstract

The automatic identification of Gastrointestinal (GI) tract diseases in endoscopy images has been associated with the domain of medical imaging and computer vision. Its classification has various challenges, including color, low contrast, lesion shape, and complex background. A Deep features-based method for the classification of gastrointestinal disease is implemented in this article. The method suggested involves four significant steps: preprocessing, extraction of handcrafted, and deep Convolutional neural network features (Deep CNN), selection of solid features, fusion, and classification. 3D-Median filtering in the preprocessing stage increases the lesion contrast. The second stage extracts the features centered on the shape. The extracted features are of three types: HOG features, ResNet50, and Xception. Principal Component Analysis (PCA) is chosen to select extracted features, combined by concatenating them in a single array. A support vector system eventually categorizes fused features into multiple classes. The Kvasir dataset is used for the proposed model. The SVM has outstanding efficiency reached 96.6 percent, showing the proposed system's robustness.

Keywords:

GI Tract Diseases, WCE, Feature Extraction, Deep Features, Feature Selection, Classification.

DOI: https://doi.org/10.32010/26166127.2023.6.1.49.76

Reference 

Akbari, M., et al. (2018, July). Polyp segmentation in colonoscopy images using fully convolutional network. In 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (pp. 69-72). IEEE.

Ali, H., Yasmin, M., Sharif, M., & Rehmani, M. H. (2018). Computer assisted gastric abnormalities detection using hybrid texture descriptors for chromoendoscopy images. Computer methods and programs in biomedicine, 157, 39-47.

Amin, J., et al. (2020). Brain tumor detection: a long short-term memory (LSTM)-based learning model. Neural Computing and Applications, 32, 15965-15973.

Amin, J., et al. (2020). Convolutional neural network with batch normalization for glioma and stroke lesion detection using MRI. Cognitive Systems Research, 59, 304-311.

Amin, J., Sharif, M., Raza, M., & Yasmin, M. (2018). Detection of brain tumor based on features fusion and machine learning. Journal of Ambient Intelligence and Humanized Computing, 1-17.

Amin, J., Sharif, M., Raza, M., Saba, T., & Anjum, M. A. (2019). Brain tumor detection using statistical and machine learning method. Computer methods and programs in biomedicine, 177, 69-79.

Amin, J., Sharif, M., Rehman, A., Raza, M., & Mufti, M. R. (2018). Diabetic retinopathy detection and classification using hybrid feature set. Microscopy research and technique, 81(9), 990-996.

Amin, J., Sharif, M., Yasmin, M., & Fernandes, S. L. (2018). Big data analysis for brain tumor detection: Deep convolutional neural networks. Future Generation Computer Systems, 87, 290-297.

Amin, J., Sharif, M., Yasmin, M., Saba, T., Anjum, M. A., & Fernandes, S. L. (2019). A new approach for brain tumor segmentation and classification based on score level fusion using transfer learning. Journal of medical systems, 43, 1-16.

Asperti, A., & Mastronardo, C. (2017). The effectiveness of data augmentation for detection of gastrointestinal diseases from endoscopical images. arXiv preprint arXiv:1712.03689.

Berens, J., Mackiewicz, M., & Bell, D. (2005, April). Stomach, intestine, and colon tissue discriminators for wireless capsule endoscopy images. In Medical Imaging 2005: Image Processing (Vol. 5747, pp. 283-290). SPIE.

Berens, J., Mackiewicz, M., & Bell, D. (2005, April). Stomach, intestine, and colon tissue discriminators for wireless capsule endoscopy images. In Medical Imaging 2005: Image Processing (Vol. 5747, pp. 283-290). SPIE.

Brandao, P., et al. (2018). Towards a computed-aided diagnosis system in colonoscopy: automatic polyp segmentation using convolution neural networks. Journal of Medical Robotics Research, 3(02), 1840002.

Charfi, S., & Ansari, M. E. (2018). Computer-aided diagnosis system for colon abnormalities detection in wireless capsule endoscopy images. Multimedia Tools and Applications, 77, 4047-4064.

Chen, T., et al. (2020). Computer-aided diagnosis of gallbladder polyps based on high resolution ultrasonography. Computer methods and programs in biomedicine, 185, 105118.

de Lange, T., Larsen, S., & Aabakken, L. (2005). Image documentation of endoscopic findings in ulcerative colitis: photographs or video clips?. Gastrointestinal endoscopy, 61(6), 715-720.

de Souza-Filho, E. M., & de Amorim Fernandes, F. (2021). Deep Learning and Artificial Intelligence in Nuclear Cardiology. Nuclear Cardiology: Basic and Advanced Concepts in Clinical Practice, 741-762.

El-Matary, W. (2008). Wireless capsule endoscopy: indications, limitations, and future challenges. Journal of pediatric gastroenterology and nutrition, 46(1), 4-12.

Eskandari, H., Talebpour, A., Alizadeh, M., & Soltanian-Zadeh, H. (2012, December). Polyp detection in Wireless Capsule Endoscopy images by using region-based active contour model. In 2012 19th Iranian Conference of Biomedical Engineering (ICBME) (pp. 305-308). IEEE.

Fayyaz, A. M., et al. (2023). Analysis of Diabetic Retinopathy (DR) Based on the Deep Learning. Information, 14(1), 30.

Feng, R., et al. (2020, April). SSN: A stair-shape network for real-time polyp segmentation in colonoscopy images. In 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI) (pp. 225-229). IEEE.

Hajabdollahi, M., et al. (2018). Segmentation of bleeding regions in wireless capsule endoscopy images an approach for inside capsule video summarization. arXiv preprint arXiv:1802.07788.

Hasan, S. M., et al. (2022). A Machine Learning Framework for Early-Stage Detection of Autism Spectrum Disorders. IEEE Access.

Høiland, T. N. (2017). Automatic Analysis of Endoscopic Videos (Master›s thesis).

Hwang, S., Oh, J., Cox, J., Tang, S. J., & Tibbals, H. F. (2006, March). Blood detection in wireless capsule endoscopy using expectation maximization clustering. In Medical Imaging 2006: Image Processing (Vol. 6144, pp. 577-587). SPIE.

Iddan, G., Meron, G., Glukhovsky, A., & Swain, P. (2000). Wireless capsule endoscopy. Nature, 405(6785), 417-417.

Jha, D., et al. (2019, December). Resunet++: An advanced architecture for medical image segmentation. In 2019 IEEE International Symposium on Multimedia (ISM) (pp. 225-2255). IEEE.

Kaiser, L., Gomez, A. N., & Chollet, F. (2017). Depthwise separable convolutions for neural machine translation. arXiv preprint arXiv:1706.03059.

Khan, M. A., et al. (2020). An automated system for cucumber leaf diseased spot detection and classification using improved saliency method and deep features selection. Multimedia Tools and Applications, 79, 18627-18656.

Khan, M. A., et al. (2020). Gastrointestinal diseases segmentation and classification based on duo-deep architectures. Pattern Recognition Letters, 131, 193-204.

Khan, M. A., et al. (2020). Lungs cancer classification from CT images: An integrated design of contrast based classical features fusion and selection. Pattern Recognition Letters, 129, 77-85.

Khan, M. A., et al. (2020). StomachNet: Optimal deep learning features fusion for stomach abnormalities classification. IEEE Access, 8, 197969-197981.

Khan, M. A., et al. (2022). Skin lesion segmentation and classification: A unified framework of deep neural network features fusion and selection. Expert Systems, 39(7), e12497.

Khan, M. A., Javed, M. Y., Sharif, M., Saba, T., & Rehman, A. (2019, April). Multi-model deep neural network based features extraction and optimal selection approach for skin lesion classification. In 2019 international conference on computer and information sciences (ICCIS) (pp. 1-7). IEEE.

Khan, M. A., Sharif, M., Akram, T., Yasmin, M., & Nayak, R. S. (2019). Stomach deformities recognition using rank-based deep features selection. Journal of medical systems, 43, 1-15.

Li, J., & Wang, J. Z. (2003). Automatic linguistic indexing of pictures by a statistical modeling approach. IEEE Transactions on pattern analysis and machine intelligence, 25(9), 1075-1088.

Li, P., et al. (2015). Using circular RNA as a novel type of biomarker in the screening of gastric cancer. Clinica chimica acta, 444, 132-136.

Liaqat, A., et al. (2018). Automated ulcer and bleeding classification from WCE images using multiple features fusion and selection. Journal of Mechanics in Medicine and Biology, 18(04), 1850038.

Majid, A., et al. (2020). Classification of stomach infections: A paradigm of convolutional neural network along with classical features fusion and selection. Microscopy research and technique, 83(5), 562-576.

Mughal, B., Muhammad, N., & Sharif, M. (2019). Adaptive hysteresis thresholding segmentation technique for localizing the breast masses in the curve stitching domain. International journal of medical informatics, 126, 26-34.

Mughal, B., Sharif, M., & Muhammad, N. (2017). Bi-model processing for early detection of breast tumor in CAD system. The European Physical Journal Plus, 132, 1-14.

Nguyen, N. Q., Vo, D. M., & Lee, S. W. (2020). Contour-aware polyp segmentation in colonoscopy images using detailed upsampling encoder-decoder networks. IEEE Access, 8, 99495-99508.

Nida, N., et al. (2016). A framework for automatic colorization of medical imaging. IIOAB J, 7, 202-209.

Nisa, M., et al. (2020). Hybrid malware classification method using segmentation-based fractal texture analysis and deep convolution neural network features. Applied Sciences, 10(14), 4966.

Pogorelov, K.,et al. (2017, June). Kvasir: A multi-class image dataset for computer aided gastrointestinal disease detection. In Proceedings of the 8th ACM on Multimedia Systems Conference (pp. 164-169).

Poorneshwaran, J. M., Kumar, S. S., Ram, K., Joseph, J., & Sivaprakasam, M. (2019, July). Polyp segmentation using generative adversarial network. In 2019 41St annual international conference of the IEEE engineering in medicine and biology society (EMBC) (pp. 7201-7204). IEEE.

Poudel, S., Kim, Y. J., Vo, D. M., & Lee, S. W. (2020). Colorectal disease classification using efficiently scaled dilation in convolutional neural network. IEEE Access, 8, 99227-99238.

Pozdeev, A. A., Obukhova, N. A., & Motyko, A. A. (2019, January). Automatic analysis of endoscopic images for polyps detection and segmentation. In 2019 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (EIConRus) (pp. 1216-1220). IEEE.

Ramzan, M., Raza, M., Sharif, M. I., & Kadry, S. (2022). Gastrointestinal Tract Polyp Anomaly Segmentation on Colonoscopy Images Using Graft-U-Net. Journal of Personalized Medicine, 12(9), 1459.

Rashid, M., et al. (2019). Object detection and classification: a joint selection and fusion strategy of deep convolutional neural network and SIFT point features. Multimedia Tools and Applications, 78, 15751-15777.

Redwan, S. M., Uddin, M. P., Ulhaq, A., & Sharif, M. I. (2022). Power Spectral Density-Based Resting-State EEG Classification of First-Episode Psychosis. arXiv preprint arXiv:2301.01588.

Rezende, E., et al. (2017, December). Malicious software classification using transfer learning of resnet-50 deep neural network. In 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA) (pp. 1011-1014). IEEE.

Saba, T., Mohamed, A. S., El-Affendi, M., Amin, J., & Sharif, M. (2020). Brain tumor detection using fusion of hand crafted and deep learning features. Cognitive Systems Research, 59, 221-230.

Saba, T., Sameh, A., Khan, F., Shad, S. A., & Sharif, M. (2019). Lung nodule detection based on ensemble of hand crafted and deep features. Journal of medical systems, 43, 1-12.

Sánchez-González, A., García-Zapirain, B., Sierra-Sosa, D., & Elmaghraby, A. (2018). Automatized colon polyp segmentation via contour region analysis. Computers in biology and medicine, 100, 152-164.

Sánchez-González, A., García-Zapirain, B., Sierra-Sosa, D., & Elmaghraby, A. (2018). Automatized colon polyp segmentation via contour region analysis. Computers in biology and medicine, 100, 152-164.

Shahzad, A., Raza, M., Shah, J. H., Sharif, M., & Nayak, R. S. (2022). Categorizing white blood cells by utilizing deep features of proposed 4B-AdditionNet-based CNN network with ant colony optimization. Complex & Intelligent Systems, 1-17.

Sharif, M. I., Khan, M. A., Alhussein, M., Aurangzeb, K., & Raza, M. (2021). A decision support system for multimodal brain tumor classification using deep learning. Complex & Intelligent Systems, 1-14.

Sharif, M., Amin, J., Raza, M., Yasmin, M., & Satapathy, S. C. (2020). An integrated design of particle swarm optimization (PSO) with fusion of features for detection of brain tumor. Pattern Recognition Letters, 129, 150-157.

Sharif, M., et al. (2020). Brain tumor detection based on extreme learning. Neural Computing and Applications, 32, 15975-15987.

Sharif, M., et al. (2021). Deep CNN and geometric features-based gastrointestinal tract diseases detection and classification from wireless capsule endoscopy images. Journal of Experimental & Theoretical Artificial Intelligence, 33(4), 577-599.

Sharif, M., et al. (2021). Deep CNN and geometric features-based gastrointestinal tract diseases detection and classification from wireless capsule endoscopy images. Journal of Experimental & Theoretical Artificial Intelligence, 33(4), 577-599.

Shin, Y., Qadir, H. A., Aabakken, L., Bergsland, J., & Balasingham, I. (2018). Automatic colon polyp detection using region based deep CNN and post learning approaches. IEEE Access, 6, 40950-40962.

Souaidi, M., et al. (2019). Multi-scale completed local binary patterns for ulcer detection in wireless capsule endoscopy images. Multimedia Tools and Applications, 78, 13091-13108.

Tajbakhsh, N., Gurudu, S. R., & Liang, J. (2013). A classification-enhanced vote accumulation scheme for detecting colonic polyps. In Abdominal Imaging. Computation and Clinical Applications: 5th International Workshop, Held in Conjunction with MICCAI 2013, Nagoya, Japan, September 22, 2013. Proceedings 5 (pp. 53-62). Springer Berlin Heidelberg.

Tjalma, J. J., et al. (2016). Molecular fluorescence endoscopy targeting vascular endothelial growth factor a for improved colorectal polyp detection. Journal of Nuclear Medicine, 57(3), 480-485.

Umer, M. J., & Sharif, M. I. (2022). A Comprehensive Survey on Quantum Machine Learning and Possible Applications. International Journal of E-Health and Medical Communications (IJEHMC), 13(5), 1-17.

Wang, Y., et al. (2015). Polyp-alert: Near real-time feedback during colonoscopy. Computer methods and programs in biomedicine, 120(3), 164-179.

Wickstrøm, K., Kampffmeyer, M., & Jenssen, R. (2018, September). Uncertainty modeling and interpretability in convolutional neural networks for polyp segmentation. In 2018 ieee 28th international workshop on machine learning for signal processing (mlsp) (pp. 1-6). IEEE.

Wu, X., et al. (2016). Automatic hookworm detection in wireless capsule endoscopy images. IEEE transactions on medical imaging, 35(7), 1741-1752.

Yu, L., Yuen, P. C., & Lai, J. (2012, November). Ulcer detection in wireless capsule endoscopy images. In Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012) (pp. 45-48). IEEE.

Yuan, Y., & Meng, M. Q. H. (2017). Deep learning for polyp recognition in wireless capsule endoscopy images. Medical physics, 44(4), 1379-1389.

Yuan, Y., Li, B., & Meng, M. Q. H. (2015). Improved bag of feature for automatic polyp detection in wireless capsule endoscopy images. IEEE Transactions on automation science and engineering, 13(2), 529-535.

Yuan, Y., Wang, J., Li, B., & Meng, M. Q. H. (2015). Saliency based ulcer detection for wireless capsule endoscopy diagnosis. IEEE transactions on medical imaging, 34(10), 2046-2057.

Zafar, M., et al. (2023). Skin Lesion Analysis and Cancer Detection Based on Machine/Deep Learning Techniques: A Comprehensive Survey. Life, 13(1), 146.

Zou, Y., et al. (2015, July). Classifying digestive organs in wireless capsule endoscopy images based on deep convolutional neural network. In 2015 IEEE International Conference on Digital Signal Processing (DSP) (pp. 1274-1278). IEEE.