Skip to main content

Custom shaders

In the previous guide, we learned how to use the <set-view-volumetric> component to render a CT (computed tomography) in 3D directly in the browser. This component comes with 4 rendering modes (or shaders):

<set-view-volumetric shader="max-intensity"></set-view-volumetric>
  • "basic"
  • "lighting"
  • "shadows"
  • "max-intensity"

Even though this 4 shaders should cover most of the use cases, it's possible to go low level and write a custom one for special use cases or just experimenting new ways of rendering your data.

Sethealth API makes writing this volumetric shaders extremelly easy, by abstracting away all the setup required to get a production-ready ray caster.

Anatomy of a volumetric ray caster#

A computed tomography can be seen as a 3D image, a volume or a tensor in ℝ3. The CT is converted and loaded by Sethealth into the GPU, then a volume ray casting algorithm renders a 2D projection into the screen.

Notice that this algoritm needs to "simulate" a light ray going from the camera all the way to the end of the volume, and that needs to be computed for every pixel of the screen.

Fortunately, today's GPUs can run this algorithm efficiently in realtime! These programs are called shaders, concretely we are going to write a fragment shader using the GLSL language.

A fragment shader is a small program that runs for each pixel, taking variables (uniforms), textures and the screen position as input and returning a color (RGBA) as output.

Alright! let's look at the max-intensity shader that comes with Sethealth:

void main() {
// Internally this function takes all the relevant input and
// prepares a high level "Ray" struct with all the relevant information
// for our ray casting, such as: direction, number of steps, delta vector...
Ray ray = computeRay();
// Early return if we are outside the volume
if (ray.outside) {
// If we are outside the volume, just set color to black (0,0,0,0) and return early.
gl_FragColor = vec4(0.0, 0.0, 0.0, 0.0);
return;
}
// Initialize a cursor variable at the start
vec3 cursor = ray.start;
float maxValue = 0.0;
// Walk N steps, advancing our cursor
// This loop is literaly our ray advancing through the 3D space.
for (int i = 0; i < ray.steps; i++) {
// Advance the cursor using delta on each iteration
cursor += ray.delta;
// Read the current value from the medical image
float value = readVolume(cursor);
// Update maxValue, taking the maximun value
maxValue = max(value, maxValue);
}
// Return the final pixel color based on the maxValue recorded.
// In this case, we are setting the R(red), G(green) and B(blue) components to the same value (maxValue),
// so the final color will be greyscaled.
// This last 1.0, correspond to the Alpha(transparency) component.
gl_FragColor = vec4(maxValue, maxValue, maxValue, 1.0);
}

Built-in functions#

Sethealth provides built-in functions to easily create your own volumetric ray caster on top of the medical data. You can focus in writing the logic of your ray caster instead of getting lost in the details.

Ray computeRay()#

Return a Ray struct, containing all the ray properties required to perform a ray casting.

struct Ray {
vec3 start; // start point of the ray caster
vec3 end; // end point of the ray caster
vec3 delta; // delta vector of each iteration
vec3 direction; // normalized direction of the eye ray
int steps; // number of steps of our ray caster
bool outside; // "true" when the ray is outside the volume
};

float readVolume(vec3 cursor)#

Returns the normalized pixel value (density) of our volume at the specified 3D point (cursor).

vec3 readNormal(vec3 cursor)#

Returns the normalized normal vector at the surface of the specified point (cursor). Since it's a volumetric render, there is not a surface strictly speaking, so the normal is the gradient of the volume.

vec4 readColormap(float density)#

It uses the provided "colormap", resolving a volume value to a vector color (RGBA).

vec4 readColor(vec3 cursor)#

This function uses readVolume() and readColormap() under the hood to return the resolve the color of a specific 3D point (cursor) using the provided colormap.

float depthAt(vec3 cursor)#

Returns the non-linear depth at the specified 3D point. This value can be passed directly to gl_FragDepth. This is used to give our shader depth so it can be integrated with normal rasterized geometry.

Shader playground#

Open Shader Playground