MITK-ToF

One of the main challenges in computer-assisted interventions is the intra-operative imaging of tissue shape, motion and morphology. This information is a prerequisite for the registration of multi-modal patient specific data that can be used for surgical planning, diagnosis and intra-operative guidance. In the context of minimally invasive surgery, an increasingly attractive approach involves 3D reconstruction of the soft-tissue surfaces using Time-of-Flight (ToF) cameras which provide range images and corresponding intensity images with high update rates. Using a ToF camera as intraoperative imaging modality requires a seamless integration in the clinical workflow. In recent years, the Medical Imaging Interaction Toolkit (MITK) became one of the most popular open-source software frameworks for developing medical image processing and image-guided therapy applications. Similar to MITK-IGT, MITK-ToF is an extension of MITK for image-guided surgery applications. It integrates ToF camera devices into the toolkit and allows the development of software for fast and flexible acquisition and processing of ToF data.

1. Structure

MITK-ToF was designed to be flexible, extendable, robust, performant, as well as portable and was developed according to the software process of MITK. It provides three basic modules (cf. Fig. 1): ToFHardware, ToFProcessing and ToFUI.

ToFHardware

This module provides means for acquisition of images from a ToF device in real-time as well as for recording and playing ToF data. A Controller class (cf. Fig. 2) serves as the direct interface to the hardware. The Device class performs a threaded update on the controller and ensures a loss-less data acquisition with an internal buffer structure. Access to the images is realized in the Grabber class which is implemented as the source of an MITK filter pipeline.

ToFProcessing

This module provides filter classes for processing the acquired ToF data. It seamlessly integrates into the pipeline architecture of MITK. Existing image filters of MITK and ITK as well as functions of OpenCV can be applied to the acquired ToF data in a straightforward manner. Currently, filters for surface generation, visualization, and basic preprocessing are implemented in the toolkit.

[[File:MITK-ToF$MITK-TOF-Overview.jpg|center|500px|Fig. 2 Overview of MITK-ToF with its layer structure. ]

ToFUI

Several basic GUI elements have been designed facilitate application development based on MITK-TOF. Implemented as widgets (with the QT toolkit for cross-platform application and GUI development) they can be integrated into Qt-based MITK applications. The basic widgets comprise a ConnectionWidget for configuring the ToF device with the standard parameters and establishing a connection to the device, a RecorderWidget for playing and recording ToF data, a VisualizationWidget for changing visualization parameters such as color look-up tables for the gray-scale ToF images and a FilterConfigurationWidget for controlling the current configuration of the so-called ToFCompositeFilter (see below). A sample application based on these widgets is shown in Fig. 3.

2. Current Features

The basic version currently supports the cameras CamCube 3.0, CamCube 2.0, CamBoard and O3 (PMDTechnologies, Siegen, Germany). New range cameras can be integrated in a straightforward manner by implementing corresponding controller and device classes.

The currently implemented ToF specific filter classes are:

• ToFCompositeFilter: Combines multiple common preprocessing steps to achieve a faster processing result. In the current version, a threshold filter for basic image segmentation as well as a bilateral filter, a spatial median filter and a temporal median filter for image smoothing can be applied.
• ToFImageRecorderFilter: Allows for recording images which have optionally already been processed by a filter pipeline.
• ToFDistanceImageToSurfaceFilter: Converts the index based distance image to a coordinate-based 3D surface representation. Texture information can be optionally added to the surface, e.g. from the corresponding intensity image.

New processing filters can be integrated by deriving them from the base class of MITK for image filters.

3. Publications and Contact

The following publications explain the MITK-ToF structure in more detail:

[1] K. Yung et al.: MITK-ToF: Time-of-Flight Kamera-Integration in das Medical Imaging Interaction Toolkit, Bildverarbeitung für die Medizin 2011, pages 204-208 (German)

[2] A. Seitel et al.: MITK-ToF - range data within MITK, Int. J. CARS, published online, May 2011

[3] A. Seitel et al.: MITK-ToF - range data within MITK, Int. J. CARS, Suppl. 1, page 126, Berlin (Germany), June 2011

For support and bug-reports please use the MITK Mailinglist

4. Getting Started

• Information about how to get started using MITK-ToF for ToF based applications can be found on a seperate page (MITK-ToF - Getting started)
• Using Kinect with MITK-ToF