Difference between revisions of "Multi Label Segmentation"
Jump to navigation
Jump to search
(username removed) |
(username removed) |
||
Line 26: | Line 26: | ||
*** "stick" label boundaries to neighboring labels: e.g. if the label is smoothed or dilated, so will be its neighbors. If the label is eroded, the "sticked" neighbors will grow into the space left by the eroded label. | *** "stick" label boundaries to neighboring labels: e.g. if the label is smoothed or dilated, so will be its neighbors. If the label is eroded, the "sticked" neighbors will grow into the space left by the eroded label. | ||
*** "background" label should be created by default, and should also be possible to lock it. | *** "background" label should be created by default, and should also be possible to lock it. | ||
− | *** "label splitter" tool: given a label(segmentation), construct a new label from each connected subpart (3D method). | + | *** "label splitter" tool: given a label (segmentation), construct a new label from each connected subpart (3D method). |
*** The post http://bugs.mitk.org/show_bug.cgi?id=6591 by Andreas Fetzer looks very interesting. TODO: ask which use cases he had in mind to implement this tool (marching squares?) | *** The post http://bugs.mitk.org/show_bug.cgi?id=6591 by Andreas Fetzer looks very interesting. TODO: ask which use cases he had in mind to implement this tool (marching squares?) | ||
*** "label assigner": given a label and a clicked position on it, build a new label from the connected subpart around that position (3D method). The same mechanism can be used to delete the connected region or add it to another existing label. | *** "label assigner": given a label and a clicked position on it, build a new label from the connected subpart around that position (3D method). The same mechanism can be used to delete the connected region or add it to another existing label. | ||
− | *** "label assigner" would differ from "label splitter" so that in the former, the connected region around the clicked position is either assigned to a new label, to an existing | + | *** "label assigner" would differ from "label splitter" so that in the former, the connected region around the clicked position is either assigned to a new label, to an existing label (indicated by the user), or deleted. In the "label splitter", all connected regions of a given label will be assigned a new (and different) label automatically. |
− | *** have a workable "paintbrush". The current implementation is just not usable. Based on a post found in: | + | *** have a workable "paintbrush". The current implementation is just not usable. Based on a post found in http://bugs.mitk.org/show_bug.cgi?id=6589 I could improve its use quite a lot (TODO: see how to send patch). Does it make sense to have a 3D paint brush? |
* have a 2D/3D region growing tool based on ITK classes [seba: add comment regarding MITK's 3D grow tool] | * have a 2D/3D region growing tool based on ITK classes [seba: add comment regarding MITK's 3D grow tool] | ||
** useful resources: | ** useful resources: |
Revision as of 21:34, 17 February 2011
Using multiple segmentations with the Segmentation bundle
Following a proposal of Sebastian Ordas to support multi-label segmentations, this page will be used to sketch future changes to the Segmentation bundle and the related framework.
Shortcomings of the current implementation
- 8-bit images are used for segmentations by default, wasting 7 of 8 bits, occupying far more memory than technically necessary
Wishlist
- what is not possible or cumbersome and should be changed?
- multiple labels on the same segmentation image: have 8-bit images with 0 value as background and the remaining 255 values as labeled objects.
- this would imply non-overlapping structures. This might be a problem, unless users use another segmentation in those cases (they would have done so anyway in the current approach)
- we should find a way to segment overlapping structures and assign two memberships to a voxel.
- multiple labels on the same segmentation image: have 8-bit images with 0 value as background and the remaining 255 values as labeled objects.
- if we'd use only a single bit we would still reduce memory consumption to 1/8. This could be taken as an intermediate step in the re-engineering process
- Should try itk::SingleBitBinaryImage: see Insight Journal paper Alternative Memory Models for ITK Images - http://www.insight-journal.org/browse/publication/646.
- Will need an I/O mechanism see: http://old.nabble.com/Re%3A-SingleBitBinaryImage-td30804691.html
- what enhancements would help in which situations?
- Primary importance enhancements: memory reduction will greatly improve usability when working with several segmentations and big datasets
- Secondary importance enhancements
- have label "lock"/"editable" boolean property
- "stick" label boundaries to neighboring labels: e.g. if the label is smoothed or dilated, so will be its neighbors. If the label is eroded, the "sticked" neighbors will grow into the space left by the eroded label.
- "background" label should be created by default, and should also be possible to lock it.
- "label splitter" tool: given a label (segmentation), construct a new label from each connected subpart (3D method).
- The post http://bugs.mitk.org/show_bug.cgi?id=6591 by Andreas Fetzer looks very interesting. TODO: ask which use cases he had in mind to implement this tool (marching squares?)
- "label assigner": given a label and a clicked position on it, build a new label from the connected subpart around that position (3D method). The same mechanism can be used to delete the connected region or add it to another existing label.
- "label assigner" would differ from "label splitter" so that in the former, the connected region around the clicked position is either assigned to a new label, to an existing label (indicated by the user), or deleted. In the "label splitter", all connected regions of a given label will be assigned a new (and different) label automatically.
- have a workable "paintbrush". The current implementation is just not usable. Based on a post found in http://bugs.mitk.org/show_bug.cgi?id=6589 I could improve its use quite a lot (TODO: see how to send patch). Does it make sense to have a 3D paint brush?
- have a 2D/3D region growing tool based on ITK classes [seba: add comment regarding MITK's 3D grow tool]
- useful resources:
- vtkEdge paintbrush classes e.g. http://www.vtkedge.org/doc/nightly/html/classvtkKWEPaintbrushMergeSketches.html#_details
- itkSnap http://www.itksnap.org/pmwiki/pmwiki.php to support multi-label images
- ITK's label object classes (http://www.insight-journal.org/browse/publication/176) could be useful
- Segmentation module in Amira (as it used to be in v3.1) http://www.amira.com/images/stories/pdf/31/Amira31-manual.pdf (see Sec. 2.5)
- useful resources:
- how does I/O work?
- same as currently: .nrrd images
- do we need a new data type for MITK (derived from BaseData?)
- no. i think we can use 8-bit images and assign a color to each label via a lookup table
- in the case we switch to 1-bit images, we will need to add it to MITK (derived from BaseData)
- how could these be mapped types for rendering?
- if we continue with 8-bit images, we can use the same mappers we have so far
- if we switch to 1-bit images, maybe not [seba TODO, check this]
- how does I/O work?
- same as currently: .nrrd images
- do we need a new data type for MITK (derived from BaseData?)
- no. i think we can use 8-bit images and assign a color to each label via a lookup table
- in the case we switch to 1-bit images, we will need to add it to MITK (derived from BaseData)
- how could these be mapped types for rendering?
- we can use the same mappers we have so far
Implications on the implementation
- what parts of MITK would need to be changed?
- mitk::Tool, mitk::ToolManager, mitk::ContourUtils, ...
- in QmitkNodeDescriptorManager add a QmitkNodeDescriptor "Labeled Image" for images with pixel type "unsigned char" (see if we want to keep purely binary images)
- searching the code for "binary" occurrences:
+ In mitk::ImageMapperGL2D::Paint, volume calculation should slightly change: itkimage->GetCountOfMaxValuedVoxelsNoRecompute() will need to have a "label" parameter to count the voxels with that given value. + In mitk::ImageMapperGL2D::SetDefaultProperties(), the heuristic for a labeled image would be: "uchar" == image->GetPixelType().GetItkTypeAsString(), PixelType::GetBpe() == 8 or alike + change names of "outline binary" to e.g. "outline label"
- what needs to be changed in the Segmentation view?
- new controls:
+ label manager widget ( e.g. have a QmitkDataStorageTable): should easily allow to (multiple) select labels for visualization or editing/locking toggling, change their representation (filled or border line), enable volume render feedback, etc.
- Use the old datastorage combobox (IMHO: the current working/reference node selection approach via the workbench selection service is not user friendly since work with the segmentation bundle requires a lot of data (multiple) selection and deselection.)
Agreed working plan
- Agree on how to handle overlapping structures. If we work on this part of the application, we should get it right and produce a UI concept that will work both with overlapping and mutually-exclusive structures.
- Implement a proof of concept for using the ITK label object classes
- bundle should be able to create a new object and prepare it for rendering (LUT or similar)
- the contour tool should work with this object, remaining tools don't count yet
- interpolation can be ignored at this step
- Implement a proof of concept for using the ITK label object classes
- make interpolation work again
- enhance the new application
- implement other tools, mostly ignoring old code and replacing it with ITK/VTK implementations
- implement locking, sticking and other useful features
- enhance the new application