Development of an image pre‐processor for operational hyperspectral laryngeal cancer detection

Hyperspectral imaging (HSI) is a technology with high potential in the field of non‐invasive detection of cancer. However, in complex imaging situations like HSI of the larynx with a rigid endoscope, various image interferences can disable a proper classification of cancerous tissue. We identified three main problems: i) misregistration of single images in a HS cube due to patient heartbeat ii) image noise and iii) specular reflections (SR). Consequently, an image pre‐processor is developed in the current paper to overcome these image interferences. It encompasses i) image registration ii) noise removal by minimum noise fraction (MNF) transformation and iii) a novel SR detection method. The results reveal that the pre‐processor improves classification performance, while the newly developed SR detection method outperforms global thresholding technique hitherto used by 46%. The novel pre‐processor will be used for future studies towards the development of an operational scheme for HS‐based larynx cancer detection.

RGB image of the larynx derived from the hyperspectral cube and corresponding specular reflections ( a ) manually segmented and ( b ) detected by a novel specular reflection detection method.     

Label‐free spectroscopic tissue characterization using fluorescence excitation‐scanning spectral imaging

Spectral imaging approaches provide new possibilities for measuring and discriminating fluorescent molecules in living cells and tissues. These approaches often employ tunable filters and robust image processing algorithms to identify many fluorescent labels in a single image set. Here, we present results from a novel spectral imaging technology that scans the fluorescence excitation spectrum, demonstrating that excitation‐scanning hyperspectral image data can discriminate among tissue types and estimate the molecular composition of tissues. This approach allows fast, accurate quantification of many fluorescent species from multivariate image data without the need of exogenous labels or dyes. We evaluated the ability of the excitation‐scanning approach to identify endogenous fluorescence signatures in multiple unlabeled tissue types. Signatures were screened using multi‐pass principal component analysis. Endmember extraction techniques revealed conserved autofluorescent signatures across multiple tissue types. We further examined the ability to detect known molecular signatures by constructing spectral libraries of common endogenous fluorophores and applying multiple spectral analysis techniques on test images from lung, liver and kidney. Spectral deconvolution revealed structure‐specific morphologic contrast generated from pure molecule signatures. These results demonstrate that excitation‐scanning spectral imaging, coupled with spectral imaging processing techniques, provides an approach for discriminating among tissue types and assessing the molecular composition of tissues. Additionally, excitation scanning offers the ability to rapidly screen molecular markers across a range of tissues without using fluorescent labels. This approach lays the groundwork for translation of excitation‐scanning technologies to clinical imaging platforms.     

Hyperspectral characterization of re‐epithelialization in an in vitro wound model

In vitro wound models are useful for research on wound re‐epithelialization. Hyperspectral imaging represents a non‐destructive alternative to histology analysis for detection of re‐epithelialization. This study aims to characterize the main optical behavior of a wound model in order to enable development of detection algorithms. K‐Means clustering and agglomerative analysis were used to group spatial regions based on the spectral behavior, and an inverse photon transport model was used to explain differences in optical properties. Six samples of the wound model were prepared from human tissue and followed over 22 days. Re‐epithelialization occurred at a mean rate of 0.24 mm²/day after day 8 to 10. Suppression of wound spectral features was the main feature characterizing re‐epithelialized and intact tissue. Modeling the photon transport through a diffuse layer placed on top of wound tissue properties reproduced the spectral behavior. The missing top layer represented by wounds is thus optically detectable using hyperspectral imaging.     

Optical hyperspectral imaging in microscopy and spectroscopy – a review of data acquisition

Rather than simply acting as a photographic camera capturing two‐dimensional (x, y) intensity images or a spectrometer acquiring spectra (λ), a hyperspectral imager measures entire three‐dimensional (x, y, λ) datacubes for multivariate analysis, providing structural, molecular, and functional information about biological cells or tissue with unprecedented detail. Such data also gives clinical insights for disease diagnosis and treatment. We summarize the principles underpinning this technology, highlight its practical implementation, and discuss its recent applications at microscopic to macroscopic scales.

Datacube acquisition strategies in hyperspectral imaging x, y, spatial coordinates; λ, wavelength.     

Hyperspectral imaging and characterization of allergic contact dermatitis in the short‐wave infrared

Short‐wave infrared hyperspectral imaging is applied to diagnose and monitor a case of allergic contact dermatitis (ACD) due to poison ivy exposure in one subject. This approach directly demonstrates increased tissue fluid content in ACD lesional skin with a spectral signature that matches the spectral signature of intradermally injected normal saline. The best contrast between the affected and unaffected skin is achieved through a selection of specific wavelengths at 1070, 1340 and 1605 nm and combining them in a pseudo‐red‐green‐blue color space. An image derived from these wavelengths normalized to unaffected skin defines a “tissue fluid index” that may aid in the quantitative diagnosis and monitoring of ACD. Further clinical testing of this promising approach towards disease detection and monitoring with tissue fluid content quantification is warranted.     

Integrated photoacoustic and hyperspectral dual‐modality microscopy for co‐imaging of melanoma and cutaneous squamous cell carcinoma in vivo

Skin carcinoma such as melanoma (MM) and cutaneous squamous cell carcinoma (cSCC) are considered as the highest mortality and the most aggressive skin cancers in dermatology. In view that early diagnosis and treatment can greatly improve the survival rate and life quality of the patients, developing noninvasive and effective evaluation methods is of great significance for the detection and identification of early stage cutaneous cancers. In this article, we propose a hybrid photoacoustic and hyperspectral dual‐modality microscopy to evaluate and differentiate skin carcinoma by structural and multiphysiological parameters. The proposed system's imaging abilities are verified by mimic phantoms and normal mice experiments. Furthermore, in vivo characterization and evaluation results of MM and cSCC mice are obtained successfully, which prove this novel method could be used as a reliable and useful method for skin cancer detection in early stages.     

On‐the‐fly detection of changes on and below the surface in epithelium mucosal tissue architecture from scattered light

In this paper we present a technique to raise a flag on the fly when a transition occurs between different mucosal architectures on or below the surface. The segmentation is based on a novel difference metric for detecting an abrupt change in the parameters extracted from a Stochastic Decomposition Method (SDM) that models the scattered light reflected from the mucosal tissue structure over an area (2‐D scan) illuminated by an optical sensor (fiber) emitting light at either one wavelength or with white light. This work has the potential to enhance the endoscopist's ability to locate and identify abnormal mucosal architectures in particular when the disease is developing below the surface and hence becoming hidden during colonoscopy or endoscopic examination. It also has also potential in helping deciding as to when and where to take biopsies; steps that should lead to improvement in the diagnostic yield. (© 2011 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)     

Characterization of blood coagulation dynamics and oxygenation in ex‐vivo retinal vessels by fluorescence hyperspectral imaging

Blood coagulation mechanisms forming a blood clot and preventing hemorrhage have been extensively studied in the last decades. Knowing the mechanisms behind becomes very important particularly in the case of blood vessel diseases. Real‐time and accurate diagnostics accompanied by the therapy are particularly needed, for example, in diseases related to retinal vasculature. In our study, we employ for the first time fluorescence hyperspectral imaging (fHSI) combined with the spectral analysis algorithm concept to assess physical as well as functional information of blood coagulation in real‐time. By laser‐induced local disruption of retinal vessels to mimic blood leaking and subsequent coagulation and a proper fitting algorithm, we were able to reveal and quantify the extent of local blood coagulation through direct identification of the change of oxyhemoglobin concentration within few minutes. We confirmed and illuminated the spatio‐temporal evolution of the essential role of erythrocytes in the coagulation cascade as the suppliers of oxygenated hemoglobin. By additional optical tweezers force manipulation, we showed immediate aggregation of erythrocytes at the coagulation site. The presented fluorescence‐based imaging concept could become a valuable tool in various blood coagulation diagnostics as well as theranostic systems if coupled with the laser therapy.     

Extended multiplicative signal correction for FTIR spectral quality test and pre‐processing of infrared imaging data

Spectral quality control is an important step in the analysis of infrared spectral data, however, often neglected in scientific literature. A frequently used quality test that was originally developed for infrared spectra of bacteria is provided by OPUS software from Bruker Optik GmbH. In this study, the OPUS quality test is applied to a large number of spectra of bacteria, yeasts and moulds and hyperspectral images of microorganisms. It is shown that the use of strict thresholds for parameters of the OPUS quality test leads to discarding too many spectra. A strategy for optimizing parameters thresholds of the OPUS quality test is provided and a novel approach for spectral quality testing based on extended multiplicative signal correction (EMSC) is suggested. For all the data sets considered in our study, the EMSC quality test is shown to be the best among different alternatives of OPUS quality test provided.     

The ImageJ ecosystem: Open‐source software for image visualization,processing, and analysis

For decades, biologists have relied on software to visualize and interpret imaging data. As techniques for acquiring images increase in complexity, resulting in larger multidimensional datasets, imaging software must adapt. ImageJ is an open‐source image analysis software platform that has aided researchers with a variety of image analysis applications, driven mainly by engaged and collaborative user and developer communities. The close collaboration between programmers and users has resulted in adaptations to accommodate new challenges in image analysis that address the needs of ImageJ's diverse user base. ImageJ consists of many components, some relevant primarily for developers and a vast collection of user‐centric plugins. It is available in many forms, including the widely used Fiji distribution. We refer to this entire ImageJ codebase and community as the ImageJ ecosystem. Here we review the core features of this ecosystem and highlight how ImageJ has responded to imaging technology advancements with new plugins and tools in recent years. These plugins and tools have been developed to address user needs in several areas such as visualization, segmentation, and tracking of biological entities in large, complex datasets. Moreover, new capabilities for deep learning are being added to ImageJ, reflecting a shift in the bioimage analysis community towards exploiting artificial intelligence. These new tools have been facilitated by profound architectural changes to the ImageJ core brought about by the ImageJ2 project. Therefore, we also discuss the contributions of ImageJ2 to enhancing multidimensional image processing and interoperability in the ImageJ ecosystem.     

Intraoperative multispectral and hyperspectral label‐free imaging: A systematic review of in vivo clinical studies

Multispectral and hyperspectral imaging (HSI) are emerging optical imaging techniques with the potential to transform the way surgery is performed but it is not clear whether current systems are capable of delivering real‐time tissue characterization and surgical guidance. We conducted a systematic review of surgical in vivo label‐free multispectral and HSI systems that have been assessed intraoperatively in adult patients, published over a 10‐year period to May 2018. We analysed 14 studies including 8 different HSI systems. Current in‐vivo HSI systems generate an intraoperative tissue oxygenation map or enable tumour detection. Intraoperative tissue oxygenation measurements may help to predict those patients at risk of postoperative complications and in‐vivo intraoperative tissue characterization may be performed with high specificity and sensitivity. All systems utilized a line‐scanning or wavelength‐scanning method but the spectral range and number of spectral bands employed varied significantly between studies and according to the system's clinical aim. The time to acquire a hyperspectral cube dataset ranged between 5 and 30 seconds. No safety concerns were reported in any studies. A small number of studies have demonstrated the capabilities of intraoperative in‐vivo label‐free HSI but further work is needed to fully integrate it into the current surgical workflow.      

Full‐field swept‐source optical coherence tomography and neural tissue classification for deep brain imaging

Optical coherence tomography can differentiate brain regions with intrinsic contrast and at a micron scale resolution. Such a device can be particularly useful as a real‐time neurosurgical guidance tool. We present, to our knowledge, the first full‐field swept‐source optical coherence tomography system operating near a wavelength of 1310 nm. The proof‐of‐concept system was integrated with an endoscopic probe tip, which is compatible with deep brain stimulation keyhole neurosurgery. Neuroimaging experiments were performed on ex vivo brain tissues and in vivo in rat brains. Using classification algorithms involving texture features and optical attenuation, images were successfully classified into three brain tissue types.     

Median‐modified Wiener filter provides efficient denoising,preserving spot edge and morphology in 2‐DE image processing

Denoising is a fundamental early stage in 2‐DE image analysis strongly influencing spot detection or pixel‐based methods. A novel nonlinear adaptive spatial filter (median‐modified Wiener filter, MMWF), is here compared with five well‐established denoising techniques (Median, Wiener, Gaussian, and Polynomial‐Savitzky–Golay filters; wavelet denoising) to suggest, by means of fuzzy sets evaluation, the best denoising approach to use in practice. Although median filter and wavelet achieved the best performance in spike and Gaussian denoising respectively, they are unsuitable for contemporary removal of different types of noise, because their best setting is noise‐dependent. Vice versa, MMWF that arrived second in each single denoising category, was evaluated as the best filter for global denoising, being its best setting invariant of the type of noise. In addition, median filter eroded the edge of isolated spots and filled the space between close‐set spots, whereas MMWF because of a novel filter effect (drop‐off‐effect) does not suffer from erosion problem, preserves the morphology of close‐set spots, and avoids spot and spike fuzzyfication, an aberration encountered for Wiener filter. In our tests, MMWF was assessed as the best choice when the goal is to minimize spot edge aberrations while removing spike and Gaussian noise.     

Bsoft: image and molecular processing in electron microscopy

Software for the processing of electron micrographs in structural biology suffers from incompatibility between different packages, poor definition and choice of conventions, and a lack of coherence in software development. The solution lies in adopting a common philosophy of interaction and conventions between the packages. To understand the choices required to have such common interfaces, I am developing a package called "Bsoft." Its foundations lie in the variety of different image file formats used in electron microscopy-a continually frustrating experience to the user and programmer alike. In Bsoft, this problem is greatly diminished by support for many different formats (including MRC, SPIDER, IMAGIC, SUPRIM, and PIF) and by separating algorithmic issues from image format-specific issues. In addition, I implemented a generalized functionality for reading the tag-base STAR (self-defining text archiving and retrieval) parameter file format as a mechanism to exchanging parameters between different packages. Bsoft is written in highly portable code (tested on several Unix systems and under VMS) and offers a continually growing range of image processing functionality, such as Fourier transformation, cross-correlation, and interpolation. Finally, prerequisites for software collaboration are explored, which include agreements on information exchange and conventions, and tests to evaluate compatibility between packages.     

An easy,inexpensive, and sensitive method for the quantification of chitin in insect peritrophic membrane by image processing

ABSTRACT

Chitin, poly (β-(1→4)-N-acetyl-d-glucosamine), is an important biopolymer for insects that is utilized as a major component of peritrophic membrane. The chitin content in peritrophic membrane is of expedient interest from a pest control perspective, although it is hard to quantify chitin. In this study, we establish a facile method for the quantification of chitin in peritrophic membrane by image processing. In this method, chitin was indirectly quantified using chitosan–I₃^? complex, which exhibited a specific red-purple color. A calibration curve using a chitosan solution showed good linearity in a concentration range of 0.05–0.5 μg/μL. We quantified the amount of chitin in peritrophic membrane of Spodoptera litura (Lepidoptera: Noctuidae) larvae using this method. Throughout the study, only common inexpensive regents and easily attainable apparatuses were employed. This method can be easily applied to the sensitive quantification of the amounts of chitin and chitosan in materials by wide range of researchers.

Abbreviations: LOD: limit of detection; LOQ: limit of quantification; ROI: region of interest; RSD: relative standard deviation.     

Enhancing vascular visualization in laser speckle contrast imaging of blood flow using multi‐focus image fusion

Laser speckle contrast imaging (LSCI) is a full‐field optical imaging method for monitoring blood flow and vascular morphology with high spatiotemporal resolution. However, due to the limited depth of field of optical system, it is difficult to capture a clear blood flow image with all blood vessels focused, especially for the non‐planar biological tissues. In this study, a multi‐focus image fusion method based on contourlet transform is introduced to reduce the misfocus effects in LSCI. The experimental results suggest that this method can provide an all‐in‐focus blood flow image, which is convenient to observe the blood vessels.      

An Arabidopsis divergent pumilio protein,APUM24, is essential for embryogenesis and required for faithful pre‐rRNA processing

Pumilio RNA‐binding proteins are largely involved in mRNA degradation and translation repression. However, a few evolutionarily divergent Pumilios are also responsible for proper pre‐rRNA processing in human and yeast. Here, we describe an essential Arabidopsis nucleolar Pumilio, APUM24, that is expressed in tissues undergoing rapid proliferation and cell division. A T‐DNA insertion for APUM24 did not affect the male and female gametogenesis, but instead resulted in a negative female gametophytic effect on zygotic cell division immediately after fertilization. Additionally, the mutant embryos displayed defects in cell patterning from pro‐embryo through globular stages. The mutant embryos were marked by altered auxin maxima, which were substantiated by the mislocalization of PIN1 and PIN7 transporters in the defective embryos. Homozygous apum24 callus accumulates rRNA processing intermediates, including uridylated and adenylated 5.8S and 25S rRNA precursors. An RNA–protein interaction assay showed that the histidine‐tagged recombinant APUM24 binds RNAin vitro with no apparent specificity. Overall, our results demonstrated that APUM24 is required for rRNA processing and early embryogenesis in Arabidopsis.