3D Visualization: What Does it Mean?

Michael P. Zeleznik, Ph.D.
RAHD Oncology Products, St. Louis, MO
University of Utah, Salt Lake City, UT

zeleznik@cs.utah.edu 801-485-1106

Originally presented at the XII International Conference on the Use of Computers in Radiation Therapy (ICCR), May 27-30, 1997, Salt Lake City, UT. Slides analyzing dose volume histogram and voxel vs. point data issues were added for the presentation ``3D Image Quality Assessment,'' RAHD 3rd Annual Users Meeting, Park City, UT, January 19-21, 1998.

Refer to the paper for more information.

IBM Visualization Data Explorer (DX) was used to generate the original 1280 x 1024 true color images. These have been reduced in quality for this WEB presentation.

Click on image to enlarge 2x, then move through slides via Next and Previous buttons.
Select "View Full Image" for full 1280x1024 image (can be very slow).

Back to top


3D visualization techniques are commonly employed in radiation treatment planning (RTP).

Full Size Image (1280x1024, can be slow)


These are powerful tools, and create very pretty pictures. But a false sense of security exists because the pictures look so much like the reality we THINK they accurately represent. Visualizations can actually be very misleading, taking extreme liberties with the data, as the visualization community is well aware. They can supress information about the original data quality, add or lose information, and different visualizations of the same data can disagree.

Full Size Image (1280x1024, can be slow)


Whether any of this matters depends on its impact on the overall RTP process. To understand this we must first understand what these issues are, then find a way to quantify and evaluate them. This is critical when trading accuracy for performace as is often the case.

Full Size Image (1280x1024, can be slow)


The goal of the paper was to initiate discussion on this problem and possible solutions. A number of common RTP visualization issues are analyzed in an attempt to characterize the problem, and one method of modeling and evaluating these issues is then presented as a strawman proposal, but only the surface is scratched. This presentation will focus on some examples of these issues.

Full Size Image (1280x1024, can be slow)


The examples will involve dose visualizations from the simple 3 beam plan shown here. The tumor is the cyan structure in the center. The beams are rectangular with no blocks.

Full Size Image (1280x1024, can be slow)


This is a 30% dose isosurface, clearly showing the shape of the beams, generated with a marching cubes algorithm. One problem with marching cubes is that the calculational time is proportional to the number of volume elements (over 300K here). This can be slow - too slow for interactive response, such as turning a dial interactor with the surface growing/shrinking in real-time. Thus, strong incentive exists to reduce the dose volume complexity by spatially filtering it down into a smaller number of elements.

Full Size Image (1280x1024, can be slow)


Another problem with marching cubes is that the surface triangle size is determined by the dose voxel size. This surface is far more complex than necessary, which slows down graphics performance (this has over 7600 vertices). Certainly a surface simplification algorithm can now be employed, but this further adds to the calculation time, and does so for EACH surface. If instead, the dose volume is reduced up front, we get multiple wins. Not only does the calculation speed up, but the dose voxels become larger, resulting in larger surface triangles and reduced surface complexity! Further, this occurs for ALL surfaces created from that volume. Thus, there can be tremendous incentive to work with a reduced dose volume.

Full Size Image (1280x1024, can be slow)


Here the dose volume has been reduced by factor of 100. The 3 planes shown simply demonstrate the original and resulting resolutions.

Full Size Image (1280x1024, can be slow)


This shows< the surface from the reduced volume along with the original. The surface from the reduced volume is 100X faster to calculate, with a surface that is 1/25 as complex, resulting in a 25X speedup in graphics interactivity. But how accurate is it?

Full Size Image (1280x1024, can be slow)


Clearly, this reduced volume surface (blue solid) is very different from the original (orange mesh). What does it really represent?

Full Size Image (1280x1024, can be slow)


This can be evaluated by dropping the reduced volume surface back into the original dose volume, and mapping the dose back onto it. The dose actually represented by this "30% surface" is seen to vary from 10% to 50%, with VERY LITTLE of it being 30% (green)! Note, the full color range now represents 10%-50% dose, shown in the colorbar.

Full Size Image (1280x1024, can be slow)


A better approach would be to apply a surface simplification algorithm to the original dose surface, as shown here. This also results in 1/25 the complexity for a 25X graphics speedup, but we are back to the original slow calculation time, PLUS the additional time to simplify the surface (which, for accurate algorithms, can be substantial).

Full Size Image (1280x1024, can be slow)


Clearly, this surface (blue solid) is a much better approximation to the original (orange mesh). How accurate is it?

Full Size Image (1280x1024, can be slow)


Again we map the original dose onto this surface. The values now range only from 15% to 35% with MOST of it actually being 30% (green). Note, the full color range now represents 15%-35% dose, shown in the colorbar.

Full Size Image (1280x1024, can be slow)


Clearly, working with the original dose volume, then simplifying, creates a much more accurate the result. But, the penalty is calculation time. It will not be interactive.

Full Size Image (1280x1024, can be slow)


In this demonstration, we had the luxury of evaluating the different results. But if a particular RTP system simply provided one of these three surfaces, how would you know what it really represented? Each claims to be THE "30% dose surface." One way of evaluating is to compare it with other dose visualizations. Two possibilities are dose-structure mappings, and dose volume histograms. We'll look at the former.

Full Size Image (1280x1024, can be slow)


Here, the dose has been mapped onto all structure surfaces. Note the beam footprints exiting the lungs, and the expected high dose at the tumor surface. This is a much simpler calculation than dose surfaces, since a calculation only occurs at each structure vertex, not at each dose point. There are only 13K vertices versus 300K dose points. The point is, one is more likely to use the original dose volume for this type of operation. If dose surfaces and dose mappings are calculated from different dose volumes, we might be able to see discrepencies by comparing the two.

Full Size Image (1280x1024, can be slow)


The 97% dose band has been mapped onto the tumor surface. Red indicates 97% or greater dose; blue indicates below 97%. If this dividing line truely represents the 97% dose transition, we should expect the 97% dose surface to intersect the tumor volume at this same boundary.

Full Size Image (1280x1024, can be slow)


As expected, the original 97% dose surface intersection does in fact align very well (also at the other end which is not shown).

Full Size Image (1280x1024, can be slow)


This, however, is the 97% dose surface from the reduced dose volume, which shows very poor alignment of the intersection at both ends. Clearly, this type of disagreement says something is wrong, but how would you know which one to believe? We might be tempted to believe the dose mapping, since I said it is more likely to use the original dose volume. But let's take a look at what really goes on with dose mapping...

Full Size Image (1280x1024, can be slow)


This shows one plane of the dose volume along with the lung surface triangles, to demonstrate the relative granularity. That fine grain, high resolution dose matrix must be mapped onto those largely spaced vertices. Clearly a substantial amount of interpolation will occur, with a corresponding loss of dose information.

Full Size Image (1280x1024, can be slow)


But that doesn't stop us from creating an arbitrary dose mapping. This 30% dose band was created by first interpolating the dose onto the coarse vertices, and then INTERPOLATING AGAIN between vertices to find the 30% location! How much error is introduced? Well, this should be a rectangular footprint, since the beam was rectangular...

Full Size Image (1280x1024, can be slow)


The RIGHT way is to map the dose onto a high resolution version of the surface, where the vertex spacing is comparable to the dose matrix grid. The result here is the rectangular footprint we expect. However, this surface is 20X more complex, making it slower to calculate, and resulting in 20X slower graphics interactivity. We can't just simplify this surface as before, or we'd be back at the previous situation.

Full Size Image (1280x1024, can be slow)


However, if the surface is simplified based on both a geometric error AND a DATA ERROR, our goal is achieved. Here, the triangles are large except where either the surface or its data varies rapidly. Thus, the rectangular footprint is maintained while still reducing surface complexity by 6X, resulting in a 6X graphics speedup. However, calculation time is further increased (significantly) for this simplification step. This would likely preclude interactive dose mapping.

Full Size Image (1280x1024, can be slow)


So, again, we can have slow and accurate or fast and inaccurate. And again, here we could evaluate these situations. But if a particular RTP system simply provided one of these mappings, how would you know which it was? What could be said about its accuracy?

Full Size Image (1280x1024, can be slow)


Having just compared dose surfaces with dose mappings, we now compare dose surfaces with dose volume histograms (DVH). For example, if the DVH says that 100% of a structure receives a dose of X or greater, the X dose surface would be expected to fully enclose that structure.

Full Size Image (1280x1024, can be slow)


The DVH says that 100% of the tumor receives 95% or greater dose. Thus, a 95% dose surface should fully enclose the tumor.

Full Size Image (1280x1024, can be slow)


But the tumor is not fully covered by the 95% dose surface.

Full Size Image (1280x1024, can be slow)


The tumor is only covered by a 91% dose surface. Why the discrepancy? To understand this, a much simpler 2D example will be used.

Full Size Image (1280x1024, can be slow)


This is a single plane 2D study, a 5x5 dose matrix with 100% dose in center 3x3 pixels and 0% dose elsewhere. The structure contour is the yellow line, exactly enclosing the 100% dose pixels.

Full Size Image (1280x1024, can be slow)


As expected, the DVH says that 100% of the structure receives 100% dose (bar graph shown filled in orange).

Full Size Image (1280x1024, can be slow)


Now consider dose isolines in this plane. Using pixel data as we have, where would the 0% line be? Since everything inside this line must receive 0% or greater, it must lie at the outside boundary. Similarly, the 100% line would lie at the 100% pixel boundaries. What about the 50% line? Since the pixels have only 2 values, there can be only two lines, 0% and 100%. The 50% line can't lie at the 0% boundary, since everything inside is not 50% or greater, so it must lie at the 100% boundary. In the end, EVERY line except the 0% line would lie at the 100% boundary. Clearly this is not what we expect to see. We are used to thinking in terms of a dose continuum which is created by interpolation from a set of points, not pixels.

Full Size Image (1280x1024, can be slow)


This shows dose POINTS instead of PIXELS, where the dose is assumed to exist at the point in the center of each pixel, rather than filling the pixel.

Full Size Image (1280x1024, can be slow)


Arbitrary dose lines can now be generated by interpolating between these points. The 100% line goes through 100% points, the 0% line through 0% points, and the others are interpolated in between. Our concept of dose lines (or surfaces in 3D) is based on points, not pixels.

Full Size Image (1280x1024, can be slow)


This also allows the familiar representation of a dose continuum. Dose is seen as smoothly varying between the points (except for the rainbow colormap banding artifact). In this display, the area outside the perimeter dose points is assumed to be invalid since there are no points beyond those with which to interpolate.

Full Size Image (1280x1024, can be slow)


The structure contour is shown here in yellow. The DVH, based on pixels, said it was covered by 100% dose. But the 100% dose line does not cover it. It is only completely covered by a 25% dose line, a very large discrepancy. What if the DVH were based on dose points? Unless we interpolate in creating the DVH, the result would not change. All points inside the contour are still 100%. However, since we interpolated the dose lines, the same could be done for the DVH.

Full Size Image (1280x1024, can be slow)


Here, the original dose matrix has been refined by a factor of 3 in each dimension (subsampled by interpolation). The grid shown here no longer defines the pixel boundaries, but is instead the grid associated with the dose points. The original dose points lie at the intersections of the grid lines. The structure contour is shown in white. It now contains some non-100% dose points which will be reflected in the DVH.

Full Size Image (1280x1024, can be slow)


This is the DVH based on the 3x interpolated data. 100% of the structure is now covered by only 45% dose, and only 60% of the structure receives 100% dose.

Full Size Image (1280x1024, can be slow)


The dose matrix can be further refined (interpolated), here by 10X. Clearly the structure now includes more dose points with a wider range of non-100% values.

Full Size Image (1280x1024, can be slow)


This is the DVH based on the 10x interpolated data. 100% of the structure is now covered by only 30% dose, and only 49% of the structure receives 100% dose.

Full Size Image (1280x1024, can be slow)


This is the DVH based on 100x interpolated data, now approaching the integral of differential dose X differential volume for a dose continuum. 100% of the structure is now covered by only 28% dose, and only 45% of the structure receives 100% dose.

Full Size Image (1280x1024, can be slow)


The 100x interpolated DVH agrees well with the dose lines. The 25% dose line just covers the structure. The DVH said 28%. The 100% dose line covers 45% of the structure (16 blocks out of 36 blocks). The DVH said 45%.

Full Size Image (1280x1024, can be slow)


This simply shows the differences among the DVHs based on 3x, 10x, and 100x interpolation. The pixel based DVH would be a flat line out to right edge.

Full Size Image (1280x1024, can be slow)


This example has been a single plane study. Slice thickness did not matter.

Full Size Image (1280x1024, can be slow)


Now consider extending the 5x5 dose plane to a 5 plane study. The 3 center planes are the same as before. The two end planes have 0% dose. Thus, we have a simple 3x3x3 volume of 100% dose, with 0% dose outside. The structure is now defined by the 3 contours (in yellow) that each exactly bound the 3x3 100% dose pixels.

Full Size Image (1280x1024, can be slow)


In any multiple plane study, each plane represents some fraction of the volume. Here, since the planes are regularly spaced, each represents 1/5 of the volume and the actual thickness would not really matter.

Full Size Image (1280x1024, can be slow)


But the planes could be irregularly spaced, as shown here. The middle plane might represent more of the structure volume than the left plane, which might represent more than the right plane.

Full Size Image (1280x1024, can be slow)


A common approach is to view each plane as a slab extending halfway to the adjacent plane, as shown here for our center plane. However, the structure "contour" now extends as a "tube" through this slab.

Full Size Image (1280x1024, can be slow)


Thus, the entire structure extends through the 3 slabs as shown here. The contours are extruded through the volume.

Full Size Image (1280x1024, can be slow)


This extrusion is essentially how the 3D structure surface is created. It is shown here with just the center plane slabbed, clearly extending into the volume between planes. This is very different from the 2D situation. In 2D, the contours completely defined the structure, and did so exactly. In 3D, the structure is assumed to extend beyond those contours in the Z dimension. In the same manner, the dose pixels now become dose voxels.

Full Size Image (1280x1024, can be slow)


This is the DVH based on voxel data (bar graph shown filled in orange). 100% of the structure receives 100% dose, as expected since the structure boundary clearly coincides with the 100% dose voxel boundary.

Full Size Image (1280x1024, can be slow)


But where would the 100% dose surface be? As we might expect, the voxel based DVH disagrees with the 100% dose surface which goes through the 100% dose points. Here, the structure is shown in translucent yellow, with the 100% dose surface well inside.

Full Size Image (1280x1024, can be slow)


As in 2D, the DVH can be calculated using dose points instead of voxels. The 3D dose points are shown here. If we don't interpolate, the DVH will be the same since the structure encloses only 100% dose points. But, as in 2D, this matrix can be refined by interpolation.

Full Size Image (1280x1024, can be slow)


Here, the 3D dose points have been refined by 3x in each dimension. Note the new planes interpolated in Z, especially those with values between the 0% end planes and the adjacent planes containing 100%. The extruded structure now includes non-100% dose points from these new planes as well as those from interpolation within the original planes (as in the 2D case), which the DVH will now reflect.

Full Size Image (1280x1024, can be slow)


This is the DVH based on the 3x interpolation, showing the effect of including the interpolated dose points.

Full Size Image (1280x1024, can be slow)


These are the DVHs based on 3x, 10x, and 100x interpolation of the dose points, with the latter approaching the dose-volume integral. At 100x interpolation, the DVH says 100% of the structure is covered by 18% dose, and only 30% of structure receives 100% dose.

Full Size Image (1280x1024, can be slow)


The 18% dose surface is shown as a wire mesh, with the structure as a solid inside. Clearly, it just covers it, with only the very corners of structure penetrating the surface, agreeing with the DVH.

Full Size Image (1280x1024, can be slow)


The 100% dose surface is shown as a red surface, with the structure as a wire mesh inside. By rotating this into 2 orthogonal views, the areas of the structure and dose surfaces were measured. The 100% dose surface did in fact fill 30% of the structure volume, as the DVH indicated.

Full Size Image (1280x1024, can be slow)


We have seen that DVHs and dose surfaces can agree if both are based on dose points with substantial interpolation. What about voxel-based calculations? Although voxel-based dose lines (or surfaces) were degenerate in our simple example, they are quite doable with real dose distributions which contain a wide range of values.

Full Size Image (1280x1024, can be slow)


Returning to the original 3 beam plan, this is the 30% voxel-based dose surface. These surfaces are generally more complex than point surfaces, and require flat (not smooth) shading, both resulting in reduced graphics interactivity. Further, they can not be simplified without losing the clean voxel edges. More importantly, what does this really represent? A dose voxel is included if its value is 30% or greater. For example, if two adjacent voxels are 29% and 60%, the 60% one would be included. Thus, although this surface has been colored to reflect 30% dose, it actually represents dose values of 30% OR GREATER. This is very different from a point-based dose surface.

Full Size Image (1280x1024, can be slow)


To see what this 30% dose voxel surface actually represents, each voxel has been colored with its dose value. This surface contains values ranging from 30% to 90%! Very little is 30%, which would be the light blue color. Voxel surfaces would have to be color mapped in this manner in order to not be misleading.

Full Size Image (1280x1024, can be slow)


This 65% dose voxel surface (blown up) actually ranges from 65% to 95%, with large discontinuities. However, to what spatial error do these dose errors correspond? The maximum spatial error at any point should not be greater than the size of a voxel. In XY this is usually small, but can be substantial in Z (if planes are far apart). This is where one often sees such discrepancies, usually at the end caps of structures.

Full Size Image (1280x1024, can be slow)


This is the 65% dose point surface together with the voxel surface (as a wire frame), turned on its side for visibility. As expected, the spatial error is less than the voxel dimensions. The point surface will lie between the centers of the outermost voxels and the voxels just outside of those.

Full Size Image (1280x1024, can be slow)


This is just a blow-up of the upper left corner of the previous image, with faceted shading to show the detail. The spatial discrepancy is always less then a voxel dimension.

Full Size Image (1280x1024, can be slow)


This view shows the spatial error in Z, normally much more pronounced. On the left, the surfaces align nearly exactly, but on the right they disagree by 1/2 the Z thickness of the voxel. Remember, the whole reason we got into this DVH discussion was the discrepancy between the 95% surface and the DVH; the tumor stuck out in the end cap (Z).

Full Size Image (1280x1024, can be slow)


This was the situation we started with.

Full Size Image (1280x1024, can be slow)


As you might expect, a 95% dose voxel surface DOSE cover the structure. Had we used a voxel surface originally, there would have been no disagreement. Thus, voxel-based DVHs and dose surfaces can agree as well.

Full Size Image (1280x1024, can be slow)


We have seen that DVHs and dose surfaces can agree, as long as both are based on voxels or points. But there is more to the story. Point based dose surfaces are created with essentially infinite resolution. That is, dose points are interpolated to the floating point accuracy of the machine, and those coordinates become the dose surface coordinates. With DVHs, dose points are interpolated to a finite grid. Although a 100x interpolation approached the dose-volume integral, it requires 100x in each dimension, or 100 cubed for 3D. This increases calculation time by a factor of 1 million! Even a 4x interpolation results in a 64x slowdown, turning 1 second into 1 minute, or .1 seconds into 6 seconds. As such, one is likely to use a much lower refinement for the DVH, and thus still see disagreement with the (relatively) infinite interpolation of the dose surfaces.

Full Size Image (1280x1024, can be slow)


The issue here is not whether voxel or point calculations are better. It is simply that they can, and often will disagree.

Full Size Image (1280x1024, can be slow)


This concludes the examples, which demonstrated three RTP visualization issues: dose surfaces, dose mappings, and DVHs. The paper discusses others, such as volume data regularization, color mapping, handling of missing data, and smooth vs. flat shading.

Full Size Image (1280x1024, can be slow)


These visualizations were seen to disagree due to raw data reduction (as with dose surfaces), use of simplified geometries (as with dose mapping), and voxel vs. point data (as with dose surfaces vs. DVHs). In practice, in the presence of such disagreement, how can we know which (if any) is the more accurate representation of the true data? Unlike this presentation, we generally will have no idea of what is going on behind the scene.

Full Size Image (1280x1024, can be slow)


The point of this talk was not the details of these specific examples, but rather to demonstrate that 1) there are many ways to create a given visualization, 2) it can be very difficult to know what it really means, and as such, 3) how can we know its impact on the RTP process? To understand this, we must first define these issues, and then find some way of quantifying and evaluating them.

Full Size Image (1280x1024, can be slow)


To this end, in the paper I present one approach to modeling and evaluating these issues, as a strawman proposal. The visualization process is modeled in an object-oriented manner. The key element of this model is the Quality Metric (QM) associated with each data type. The QM acptures the intrinsic quality of the data, including things like error information (e.g., the dose range represented by a dose surface, or the positional error resulting from surface simplification), non-information that is added (e.g., smooth shading a surface with very coarse triangulation), information that is lost (e.g., reducing the dose volume, or losses due to the interaction of the colormap with human visual perception), or the intrinsic value of a visualization (e.g., displaying dose as raw numbers is certainly accurate, but worthless as a visualization tool). QMs are modified by each transformation, and thus are propagated from the raw data to the final observer, providing a quantitative handle on the meaning of the resulting visualization. Only the surface has been scratched, but hopefully this provides a framework for future discussion.

Full Size Image (1280x1024, can be slow)


So, in answer to the original question -- "What Does it Mean?" Rather than wondering, perhaps we should be trying to DEFINE what it means, by clearly specifying what we expect these different visualizations to convey, and to what level of accuracy, with RTP systems then conforming to those specifications.

Full Size Image (1280x1024, can be slow)


Back to top
END