New image display classes for VTK

Finally, I've finished a big and rather important project that I started in December (though I was interrupted a few times): a complete re-write of the image display classes for VTK. In brief, these classes replace vtkImageActor and provide a much higher level of functionality. Whereas image display used to be a clunky add-on feature in VTK, it is now a central feature just like geometry and volume display.

The main additions are three new classes:

  • vtkImageSlice – the new actor class for images
  • vtkImageResliceMapper – mapper for drawing images for MPR viewers
  • vtkImageProperty – for controlling lookup tables, window/level, opacity, etc

These are meant to mirror the vtkActor, vtkMapper, and vtkProperty classes used for geometry. The vtkImageSlice and vtkImageProperty classes are very simple, they are just information containers: the vtkImageSlice controls the position and orientation of the data with respect to the world coordinate system, and vtkImageProperty controls the color table, interpolation, and other display-related properties. The vtkImageResliceMapper (and its associated classes vtkImageMapper3D, vtkImageSliceMapper, and vtkOpenGLImageSliceMapper) are where all the work gets done.

My primary goal was to simplify user interaction with the image display, and I decided that the easiest way to do this, since most VTK interaction occurs via the scene "camera", was to make the camera control both the position and orientation of the slice that is displayed, via the camera focal point and view point. This turned out to be a very intuitive way of doing things, and also allowed the VTK interactor classes to be used for image viewing with very little modification.

A brief run-down of the features:

  1. oblique views (obviously!)
  2. nearest-neighbor, linear, and cubic interpolation
  3. easy modification of image transformation via prop's UserTransform
  4. fully multi-threaded reslicing and color mapping operations
  5. streaming, i.e. only pulls the data needed for rendering
  6. works well with very large images (as long as they fit in memory)
  7. LOD-style interaction (switches to faster rendering for interaction)
  8. blend images by adding multiple vtkImageSlice objects to a vtkImageStack
  9. checkerboard images for easy comparison after registration
  10. thick-slab views (see vtkImageResliceMapper for details)
  11. works with vtkCellPicker for picking image voxels as points or cells

For large image viewing, on a 4GB computer (64-bit OS X) I’ve displayed images up to 3GB in size with full interactivity, but YMMV. As long as the mapper’s ResampleToScreenPixels option is On, the mapper will never try to load the full image onto the GPU (which is what causes the vtkImageActor to be slow for large images). The size of the images that you can display is limited only by main memory, I have displayed stacks of RGB images that are 10000×10000 pixels in size.