Improvement of design of a surgical interface using an eye tracking device

Background

Surgical interfaces are used for helping surgeons in interpretation and quantification of the patient information, and for the presentation of an integrated workflow where all available data are combined to enable optimal treatments. Human factors research provides a systematic approach to design user interfaces with safety, accuracy, satisfaction and comfort. One of the human factors research called user-centered design approach is used to develop a surgical interface for kidney tumor cryoablation. An eye tracking device is used to obtain the best configuration of the developed surgical interface.

Methods

Surgical interface for kidney tumor cryoablation has been developed considering the four phases of user-centered design approach, which are analysis, design, implementation and deployment. Possible configurations of the surgical interface, which comprise various combinations of menu-based command controls, visual display of multi-modal medical images, 2D and 3D models of the surgical environment, graphical or tabulated information, visual alerts, etc., has been developed. Experiments of a simulated cryoablation of a tumor task have been performed with surgeons to evaluate the proposed surgical interface. Fixation durations and number of fixations at informative regions of the surgical interface have been analyzed, and these data are used to modify the surgical interface.

Results

Eye movement data has shown that participants concentrated their attention on informative regions more when the number of displayed Computer Tomography (CT) images has been reduced. Additionally, the time required to complete the kidney tumor cryoablation task by the participants had been decreased with the reduced number of CT images. Furthermore, the fixation durations obtained after the revision of the surgical interface are very close to what is observed in visual search and natural scene perception studies suggesting more efficient and comfortable interaction with the surgical interface. The National Aeronautics and Space Administration Task Load Index (NASA-TLX) and Short Post-Assessment Situational Awareness (SPASA) questionnaire results have shown that overall mental workload of surgeons related with surgical interface has been low as it has been aimed, and overall situational awareness scores of surgeons have been considerably high.

Conclusions

This preliminary study highlights the improvement of a developed surgical interface using eye tracking technology to obtain the best SI configuration. The results presented here reveal that visual surgical interface design prepared according to eye movement characteristics may lead to improved usability.

     

A non-invasive acoustic and vibration analysis technique for evaluation of hip joint conditions

The performance evaluation of THA outcome is difficult and surgeons often use invasive methods to investigate effectiveness. A non-invasive acoustic and vibration analysis technique has recently been developed for more-in-depth evaluation of in vivo hip conditions.Gait kinematics, corresponding vibration and sound measurement of five THA subjects were analyzed post-operatively using video-fluoroscopy, sound and accelerometer measurements while walking on a treadmill. The sound sensor and a pair of tri-axial accelerometers, externally attached to the pelvic and femoral bone prominences, detected frequencies that are propagated through the femoral head and acetabular cup interactions. A data acquisition system was used to amplify the signal and filter out noise generated by undesired frequencies. In vivo kinematics and femoral head sliding quantified using video fluoroscopy were correlated to the sound and acceleration measurements.Distinct variations between the different subjects were identified. A correlation of sound and acceleration impulses with separation has been achieved. Although, in vivo sounds are quite variable in nature and all correlated well with the visual images.This is the first study to document and correlate visual and audible effects of THA under in-vivo conditions. This study has shown that the development of the acoustic and vibration technique provides a practical method and generates new possibilities for a better understanding of THA performance.     

Biomechanical properties of abdominal organs in vivo and postmortem under compression loads

Accurate knowledge of biomechanical characteristics of tissues is essential for developing realistic computer-based surgical simulators incorporating haptic feedback, as well as for the design of surgical robots and tools. As simulation technologies continue to be capable of modeling more complex behavior, an in vivo tissue property database is needed. Most past and current biomechanical research is focused on soft and hard anatomical structures that are subject to physiological loading, testing the organs in situ. Internal organs are different in that respect since they are not subject to extensive loads as part of their regular physiological function. However, during surgery, a different set of loading conditions are imposed on these organs as a result of the interaction with the surgical tools. Following previous research studying the kinematics and dynamics of tool/tissue interaction in real surgical procedures, the focus of the current study was to obtain the structural biomechanical properties (engineering stress-strain and stress relaxation) of seven abdominal organs, including bladder, gallbladder, large and small intestines, liver, spleen, and stomach, using a porcine animal model. The organs were tested in vivo, in situ, and ex corpus (the latter two conditions being postmortem) under cyclical and step strain compressions using a motorized endoscopic grasper and a universal-testing machine. The tissues were tested with the same loading conditions commonly applied by surgeons during minimally invasive surgical procedures. Phenomenological models were developed for the various organs, testing conditions, and experimental devices. A property database-unique to the literature-has been created that contains the average elastic and relaxation model parameters measured for these tissues in vivo and postmortem. The results quantitatively indicate the significant differences between tissue properties measured in vivo and postmortem. A quantitative understanding of how the unconditioned tissue properties and model parameters are influenced by time postmortem and loading condition has been obtained. The results provide the material property foundations for developing science-based haptic surgical simulators, as well as surgical tools for manual and robotic systems.     

Image filtering for two-photon deep imaging of lymphonodes

Non-linear excitation microscopy is considered an ideal spectroscopic method for imaging thick tissues in vivo due to the reduced scattering of infrared radiation. Although imaging has been reported on brain neocortex at 600-800 mum of depth, much less uniform tissues, such as lymphonodes, are characterized by highly anisotropic light scattering that limits the penetration length. We show that the most severe limitation for deep imaging of lymphonodes appears to be the tissue scattering and the diffuse fluorescence emission of labeled cell (lymphocytes) in layers above the focusing plane. We report a study of the penetration depth of the infrared radiation in a model system and in ex vivo lymphonodes and discuss the possibility to apply Fourier filtering to the images in order to improve the observation depth.     

Automated breast cancer diagnosis based on fine needle aspiration

OBJECTIVE: To design and analyze an automated diagnostic system for breast carcinoma based on fine needle aspiration (FNA). STUDY DESIGN: FNA is a noninvasive alternative to surgical biopsy for the diagnosis of breast carcinoma. Widespread clinical use of FNA is limited by the relatively poor interobserver reproducibility of the visual interpretation of FNA images. To overcome the reproducibility problem, past research has focused on the development of automated diagnosis systems that yield accurate, reproducible results. While automated diagnosis is, by definition, reproducible, it has yet to achieve diagnostic accuracy comparable to that of surgical biopsy. In this article we describe a sophisticated new diagnostic system in which the mean sensitivity (of FNA diagnosis) approaches that of surgical biopsy. The diagnostic system that we devised analyzes the digital FNA data extracted from FNA images. To achieve high sensitivity, the system needs to solve large, equality-constrained, integer nonlinear optimization problems repeatedly. Powerful techniques from the theory of Lie groups and a novel optimization technique are built into the system to solve the underlying optimization problems effectively. The system is trained using digital data from FNA samples with confirmed diagnosis. To analyze the diagnostic accuracy of the system > 8,000 computational experiments were performed using digital FNA data from the Wisconsin Breast Cancer Database. RESULTS: The system has a mean sensitivity of 99.62% and mean specificity of 93.31%. Statistical analysis shows that at the 95% confidence level, the system can be trusted to correctly diagnose new malignant FNA samples with an accuracy of 99.44-99.8% and new benign FNA samples with an accuracy of 92.43-93.93%. CONCLUSION: The diagnostic system is robust and has higher sensitivity than do all the other systems reported in the literature. The specificity of the system needs to be improved.     

In vivo magnetic resonance microscopy of differentiation in Xenopus laevis embryos from the first cleavage onwards

Differentiation inside a developing embryo can be observed by a variety of optical methods but hardly so in opaque organisms. Embryos of the frog Xenopus laevis--a popular model system--belong to the latter category and, for this reason, are predominantly being investigated by means of physical sectioning. Magnetic resonance imaging (MRI) is a noninvasive method independent of the optical opaqueness of the object. Starting out from clinical diagnostics, the technique has now developed into a branch of microscopy--MR microscopy--that provides spatial resolutions of tens of microns for small biological objects. Nondestructive three-dimensional images of various embryos have been obtained using this technique. They were, however, usually acquired by long scans of fixed embryos. Previously reported in vivo studies did not cover the very early embryonic stages, mainly for sensitivity reasons. Here, by applying high field MR microscopy to the X. laevis system, we achieved the temporal and spatial resolution required for observing subcellular dynamics during early cell divisions in vivo. We present image series of dividing cells and nuclei and of the whole embryonic development from the zygote onto the hatching of the tadpole. Additionally, biomechanical analyses from successive MR images are introduced. These results demonstrate that MR microscopy can provide unique contributions to investigations of differentiating cells and tissues in vivo.     

Évaluation par AFSIM d’une méthode régularisée de correction de l’atténuation en imagerie de contraste ultrasonore

The robustness of a regularized method to correct for microbubble attenuation of in vivo contrast ultrasound images (CUI) is evaluated. The regularized approach is based on the proportionality between attenuation and retrodiffusion coefficients as well as on a boundary condition reflecting the microbubble cumulated attenuation in a reference region. The method has been tested on temporal sequences of CUI, which are related to the renal perfusion on mice. The evaluation of the method robustness is performed using the estimation of Factorial Analysis of Medical Image Sequences (FAMIS) images, which enable a synthesized interpretation of the temporal sequence. The importance of an adequate choice of the reference zone has been highlighted. The regularized correction permits an almost complete elimination of attenuation artifacts and a more reliable representation of microbubble concentration.     

Separated flow demonstrated by digitized cineangiography compared with LDV

In order to demonstrate separated flow in vivo, a method for the computerized analysis of cineangiographies has been developed, tested in vitro, and compared with LDV. A pulsatile flow was created in a glass model bifurcation, and velocity profiles were obtained with LDV at several phase angles. The flow was cinefilmed during contrast injection and the images were digitized. The computer then transformed the image sequence into parametric images representing arrival times of the contrast. The separation regions demonstrated with LDV were identified as areas with delayed contrast arrival. A preliminary analysis of a cineangiography in vivo is also included.     

Non-invasive evaluation of an injectable bone substitute

Despite the increasing number of techniques for the preservation of bone ridges after dental avulsion, no precise evaluation of alveolar filling has been performed to date. The criteria of available measurement techniques (probes, retroalveolar or panoramic radiography, and lateral teleradiography) are not sufficiently reliable and precise. This study investigated the reliability of evaluation based on CT images in comparison with retroalveolar radiography (the most precise radiographic technique, providing standardised images), direct measurements, and images obtained in scanning electron microscopy. After a preliminary investigation ex vivo, a study was performed in vivo on three beagles. Mandibular premolars were extracted, and the corresponding alveoli were filled with an injectable bone substitute composed of a calcium phosphate mineral load associated with hydroxypropyl methylcellulose. Measurements performed on CT images relative to visual and automatic detection of density changes and studies of density curves provided better precision than those obtained by retroalveolar radiography.     

In Vivo Analysis of Heterogeneous Brain Deformation Computations for Model-Updated Image Guidance

Neurosurgical image-guidance has historically relied on the registration of the patient and preoperative imaging series with surgical instruments in the operating room (OR) coordinate space. Recent studies measuring intraoperative tissue motion have suggested that deformation-induced misregistration from surgical loading is a serious concern with such systems. In an effort to improve registration fidelity during surgery, we are pursuing an approach which uses a predictive computational model in conjunction with data available in the OR to update the high resolution preoperative image series. In previous work, we have developed an in vivo experimental system in the porcine brain which has been used to investigate a homogeneous finite element rendering of consolidation theory as a tissue deformation model. In this paper, our computational approach has been extended to include heterogeneous tissue property distributions determined from an image-to-grid segmentation scheme. Results produced under two different loading conditions show that heterogeneity in the stiffness properties and interstitial pressure gradients varied over a range of physiologically reasonable values account for 1-3% and 5-8% of the predicted tissue motion, respectively, while homogeneous linear elasticity is responsible for 60-70% of the surgically-induced motion that has been recoverable with our model-based approach.     

Image-guided surgery: From X-rays to Virtual Reality

Since the discovery of X-rays, medical imaging has played a major role in the guidance of surgical procedures. While medical imaging began with simple X-ray plates to indicate the presence of foreign objects within the human body, the advent of the computer has been a major factor in the recent development of this field. Imaging techniques have grown greatly in their sophistication and can now provide the surgeon with high quality three-dimensional images depicting not only the normal anatomy and pathology, but also vascularity and function. One key factor in the advances in Image-Guided Surgery (IGS) is the ability not only to register images derived from the various imaging modalities amongst themselves, but also to register them to the patient. The other crucial aspect of IGS is the ability to track instruments in real time during the procedure, and to portray them as part of a realistic model of the operative volume. Stereoscopic and virtual-reality techniques can usefully enhance the visualization process. IGS nevertheless relies heavily on the assumption that the images acquired prior to surgery, and upon which the surgical guidance is based, accurately represent the morphology of the tissue during the surgical procedure. In many instances this assumption is invalid, and intra-operative real-time imaging, using interventional MRI, Ultrasound, and electrophysiological recordings are often employed to overcome this limitation. Although now in extensive clinical use, IGS is often currently perceived as an intrusion into the operating room. It must evolve towards becoming a routine surgical tool, but this will only happen if natural and intuitive human interfaces are developed for these systems.     

Full-field OCT

Optical coherence tomography (OCT) is an emerging technique for imaging of biological media with micrometer-scale resolution, whose most significant impact concerns ophthalmology. Since its introduction in the early 1990's, OCT has known a lot of improvements and sophistications. Full-field OCT is our original approach of OCT, based on white-light interference microscopy. Tomographic images are obtained by combination of interferometric images recorded in parallel by a detector array such as a CCD camera. Whereas conventional OCT produces B-mode (axially-oriented) images like ultrasound imaging, full-field OCT acquires tomographic images in the en face (transverse) orientation. Full-field OCT is an alternative method to conventional OCT to provide ultrahigh resolution images (approximately 1 microm), using a simple halogen lamp instead of a complex laser-based source. Various studies have been carried, demonstrating the performances of this technology for three-dimensional imaging of ex vivo specimens. Full-field OCT can be used for non-invasive histological studies without sample preparation. In vivo imaging is still difficult because of the object motions. A lot of efforts are currently devoted to overcome this limitation. Ultra-fast full-field OCT was recently demonstrated with unprecedented image acquisition speed, but the detection sensitivity has still to be improved. Other research directions include the increase of the imaging penetration depth in highly scattering biological tissues such as skin, and the exploitation of new contrasts such as optical birefringence to provide additional information on the tissue morphology and composition.     

In vivo modeling of interstitial pressure in the brain under surgical load using finite elements

Current brain deformation models have predominantly reflected solid constitutive relationships generated from empirical ex vivo data and have largely overlooked interstitial hydrodynamic effects. In the context of a technique to update images intraoperatively for image-guided neuronavigation, we have developed and quantified the deformation characteristics of a three-dimensional porous media finite element model of brain deformation in vivo. Results have demonstrated at least 75-85 percent predictive capability, but have also indicated that interstitial hydrodynamics are important. In this paper we investigate interstitial pressure transient behavior in brain tissue when subjected to an acute surgical load consistent with neurosurgical events. Data are presented from three in vivo porcine experiments where subsurface tissue deformation and interhemispheric pressure gradients were measured under conditions of an applied mechanical deformation and then compared to calculations with our three-dimensional brain model. Results demonstrate that porous-media consolidation captures the hydraulic behavior of brain tissue subjected to comparable surgical loads and that the experimental protocol causes minimal trauma to porcine brain tissue. Working values for hydraulic conductivity of white and gray matter are also reported and an assessment of transient pressure gradient effects with respect to deformation is provided.     

Telecentric design for digital-scanning-based HiLo optical sectioning endomicroscopy with an electrically tunable lens

Confocal endoscopy has been widely used to obtain fine optically sectioned images. However, confocal endomicroscopic images are formed by point-by-point scanning in both lateral and axial directions, which results in long image acquisition time. Here, an endomicroscope with telecentric configuration is presented to achieve nonmechanical and rapid axial scanning for volumetric fluorescence imaging. In our system, optical sectioning in wide-field fashion is obtained through HiLo imaging with a digital micromirror device. Axial scanning, without mechanical moving parts, is conducted by digital focus adjustment using an electrically tunable lens, offering constant magnification and contrast. We demonstrate imaging performance of our system with optically sectioned images using fluorescently labeled beads, as well as ex vivo mice cardiac tissue samples. Our system provides multiple advantages, in terms of improved scanning range, and reduced image acquisition time, which shows great potentials for three-dimensional biopsies of volumetric biological samples.     

N.m.r. imaging of intact biological systems

Internal images of structured objects may be obtained with n.m.r. by labelling component parts with different magnetic field strengths and therefore recognizably different n.m.r. frequencies. A linear field gradient generates a one-dimensional projection of nuclear density and a variety of techniques are employed to manipulate this one-dimensional probe to yield internal images in two and three dimensions. In the past few years, n.m.r. imaging, sometimes also called zeugmatography or spin mapping, has been applied progressively to provide proton images of small phantoms, fruit, vegetables and small animals, and finally to in vivo imaging of the human body; it promises to provide a valuable means of interior investigation of intact biological systems generally. For medical imaging the method is non-invasive, does not use ionizing radiations, appears to be without hazard and penetrates bony cavities without attenuation. Furthermore, other n.m.r. parameters, for example, relaxation times and fluid flow, may also be mapped; there is evidence that the relaxation times from tumours are significantly longer than those from corresponding normal tissue. Effort to date has mostly been concentrated on proton n.m.r., but some work has been done with other nuclei. Three examples are shown of n.m.r. images of intact biological systems: a fruit, an animal and a human system. The discussion includes the quantitative nature of the images, tissue discrimination, the relation between the resolution in the image and image acquisition time, attenuation and phase shift of the r.f. field in the biological tissue, and magnets suitable for n.m.r. imaging. In principle, all conventional n.m.r. techniques can be combined with n.m.r. methods in order to investigate heterogeneous systems. Overhauser imaging is briefly discussed.     

Image-guided surgery: from X-rays to virtual reality

Since the discovery of X-rays, medical imaging has played a major role in the guidance of surgical procedures. While medical imaging began with simple X-ray plates to indicate the presence of foreign objects within the human body, the advent of the computer has been a major factor in the recent development of this field. Imaging techniques have grown greatly in their sophistication and can now provide the surgeon with high quality three-dimensional images depicting not only the normal anatomy and pathology, but also vascularity and function. One key factor in the advances in Image-Guided Surgery (IGS) is the ability not only to register images derived from the various imaging modalities amongst themselves, but also to register them to the patient. The other crucial aspect of IGS is the ability to track instruments in real time during the procedure, and to portray them as part of a realistic model of the operative volume. Stereoscopic and virtual-reality techniques can usefully enhance the visualization process. IGS nevertheless relies heavily on the assumption that the images acquired prior to surgery, and upon which the surgical guidance is based, accurately represent the morphology of the tissue during the surgical procedure. In many instances this assumption is invalid, and intra-operative real-time imaging, using interventional MRI, Ultrasound, and electrophysiological recordings are often employed to overcome this limitation. Although now in extensive clinical use, IGS is often currently perceived as an intrusion into the operating room. It must evolve towards becoming a routine surgical tool, but this will only happen if natural and intuitive human interfaces are developed for these systems.     

Use of Indocyanine Green for Detecting the Sentinel Lymph Node in Breast Cancer Patients: From Preclinical Evaluation to Clinical Validation

Assessment of the sentinel lymph node (SLN) in patients with early stage breast cancer is vital in selecting the appropriate surgical approach. However, the existing methods, including methylene blue and nuclides, possess low efficiency and effectiveness in mapping SLNs, and to a certain extent exert side effects during application. Indocyanine green (ICG), as a fluorescent dye, has been proved reliable usage in SLN detection by several other groups. In this paper, we introduce a novel surgical navigation system to detect SLN with ICG. This system contains two charge-coupled devices (CCD) to simultaneously capture real-time color and fluorescent video images through two different bands. During surgery, surgeons only need to follow the fluorescence display. In addition, the system saves data automatically during surgery enabling surgeons to find the registration point easily according to image recognition algorithms. To test our system, 5 mice and 10 rabbits were used for the preclinical setting and 22 breast cancer patients were utilized for the clinical evaluation in our experiments. The detection rate was 100% and an average of 2.7 SLNs was found in 22 patients. Our results show that the usage of our surgical navigation system with ICG to detect SLNs in breast cancer patients is technically feasible.     

FMT-XCT: in vivo animal studies with hybrid fluorescence molecular tomography-X-ray computed tomography

The development of hybrid optical tomography methods to improve imaging performance has been suggested over a decade ago and has been experimentally demonstrated in animals and humans. Here we examined in vivo performance of a camera-based hybrid fluorescence molecular tomography (FMT) system for 360° imaging combined with X-ray computed tomography (XCT). Offering an accurately co-registered, information-rich hybrid data set, FMT-XCT has new imaging possibilities compared to stand-alone FMT and XCT. We applied FMT-XCT to a subcutaneous 4T1 tumor mouse model, an Aga2 osteogenesis imperfecta model and a Kras lung cancer mouse model, using XCT information during FMT inversion. We validated in vivo imaging results against post-mortem planar fluorescence images of cryoslices and histology data. Besides offering concurrent anatomical and functional information, FMT-XCT resulted in the most accurate FMT performance to date. These findings indicate that addition of FMT optics into the XCT gantry may be a potent upgrade for small-animal XCT systems.