Multi Label Segmentation

Using multiple segmentations with the Segmentation bundle

Following a proposal of Sebastian Ordas to support multi-label segmentations, this page will be used to sketch future changes to the Segmentation bundle and the related framework.

Shortcomings of the current implementation

• 8-bit images are used for segmentations by default, wasting 7 of 8 bits, occupying far more memory than technically necessary

Wishlist

• what is not possible or cumbersome and should be changed?
  • multiple labels on the same segmentation image: have 8-bit images with 0 value as background and the remaining 255 values as labeled objects.
• what enhancements would help in which situations?
  • have label "lock"/"editable" boolean property
  • "stick" label boundaries to neighboring labels: e.g. if the label is smoothed or dilated, so will be its neighbors. If the label is eroded, the "sticked" neighbors will grow into the space left by the eroded label.
  • "background" label should be created by default, and should also be possible to lock it
  • "label splitter" tool: given a label, construct a new label from each connected subpart (3D method)
  • "label assigner": given a label and a clicked position on it, build a new label from the connected subpart around that position (3D method). The same mechanism can be used to delete the connected region or add it to another existing label.
  • have a workable "paintbrush". The current implementation is just not usable.
  • have a 2D/3D region growing tool based on ITK classes
• useful resources:
  • vtkEdge paintbrush classes e.g. http://www.vtkedge.org/doc/nightly/html/classvtkKWEPaintbrushMergeSketches.html#_details
  • itkSnap http://www.itksnap.org/pmwiki/pmwiki.php
  • to support multi-label images, ITK's label object classes (http://www.insight-journal.org/browse/publication/176) could be useful
  • Segmentation module in Amira (as it used to be in v3.1) http://www.amira.com/images/stories/pdf/31/Amira31-manual.pdf (see Sec. 2.5)
• how does I/O work?
  • same as currently: .nrrd images
• do we need a new data type for MITK (derived from BaseData?)
  • no. i think we can use 8-bit images and assign a color to each label as usual
• how could these be mapped types for rendering?
  • we can use the same mappers we have so far

Implications on the implementation

• what parts of MITK would need to be changed?
  • mitk::Tool, mitk::ToolManager, mitk::ContourUtils, ...
  • in QmitkNodeDescriptorManager add a QmitkNodeDescriptor "ImageLabel" for images with pixel type "unsigned char"
• what needs to be changed in the Segmentation view?
  • new controls:
    • label manager widget ( e.g. have a QmitkDataStorageTable): should easily allow to (multiple) select labels for visualization or editing/locking toggling, change their representation (filled or border line), enable volume render feedback, etc.
• Use the old datastorage combobox (IMHO: the current working/reference node selection approach via the workbench selection service is not user friendly since work with the segmentation bundle requires a lot of data (multiple) selection and deselection.)

Proposed working plan

• Stage 1: add lock/edit feature to the current approach
  • Add "binaryimage.locked" bool propery to data node (e.g. in mitk::ImageMapperGL2D::SetDefaultProperties() ). Should be set to TRUE by default.
  • Have in Tool Manager the following methods:
    • GetEditableData();
    • GetLockedData();
• For every mitk::Tool, upon finish editing (e.g. mouse release), correct the feedback image for every intersection with other editable/locked segmentations
  • The Segmentation Interpolation should also respect other locked and editable labels
• Stage 2: do not use binary images any more
• Stage 3: