Classification of Different Tympanic Membrane Conditions Using Fused Deep Hypercolumn Features and Bidirectional LSTM

Objectives: Middle ear inflammatory diseases are global health problem that can have serious consequences such as hearing loss and speech disorders. The high cost of medical devices such as oto-endoscope and oto-microscope used by the specialists for the diagnosis of the disease prevents its widespread use. In addition, the decisions of otolaryngologists may differ due to the subjective visual examinations. For this reason, computer-aided middle ear disease diagnosis systems are needed to eliminate subjective diagnosis and high cost problems. To this aim, a hybrid deep learning approach was proposed for automatic recognition of different tympanic membrane conditions such as earwax plug, myringosclerosis, chronic otitis media and normal from the otoscopy images.Materials and methods: In this study we used public Ear Imagery dataset containing 880 otoscopy images. The proposed approach detects keypoints from the otoscopy images and following the obtained keypoint positions, extracts hypercolumn deep features from 5 different layers of the VGG 16 model. Classification of tympanic membrane conditions were realized by feeding the deep hypercolumn features to Bi-LSTM network in the form of non-time related data.Results: The performance of the proposed model was evaluated in three different color spaces as Red-Green-Blue (RGB), Hue-Saturation-Value (HSV) and Haematoxylin-Eosin-Diaminobenzidine (HED). The proposed model achieved acceptable results in all color spaces, moreover it showed a very successful performance in classifying tympanic membrane conditions especially in RGB space. Experimental studies showed that the proposed model achieved Acc of 99.06%, Sen of 98.13% and Spe of 99.38%.Conclusion: As a result, a robust model with high sensitivity was obtained for classification of tympanic membrane conditions and it was shown that Bi-LSTM network, which is generally used with time-related data, could also be used successfully with non-time related data for diagnosis of tympanic membrane conditions.