Aivia Software

3D Object Analysis - Cellpose

The 3D Object Analysis - Cellpose recipe (for Aivia 11 and later) detects objects (such as nuclei or cells) in 3D volumetric images using Cellpose, a generalist deep learning algorithm for cell segmentation, developed by Carsten Stringer et. al. [1]. The recipe uses Cellpose to generate confidence map of the input image for 3D object segmentation that improves overall 3D segmentation accuracy. Unlike the standard Cellpose algorithm, the 3D Object Analysis - Cellpose recipe do not use the flow maps output for segmentation. For the image enhancement recipe that utilizes flow map for segmenting and labeling detected objects, please refer to the documentation on Cellpose Object Detection / 3D Cellpose Object Detection.

 

Technical Background

Cellpose is a generalist algorithm for segmentation of cells with various morphological types acquired via different imaging modalities. Cellpose was originally published by Stringer et al in 2020 (publication link). The raw confidence map range is in logit, meaning its range is (-∞, +∞) and we convert the confidence map to (0, 1) using the sigmoid function. This conversion, however, will result in most signals concentrating at 0 or 1, with very little information in between, making it difficult for Aivia’s 3D Object Analysis recipe to work with such a narrow range. Instead of converting it to probability values using the sigmoid function, we found it is useful to convert it to input image bit depth using the following steps.

  1. Normalize the confidence map using the 60th percentile as 0.0 and 99.98th percentile as 1.0.

  2. Clip the voxels that are below 0.0 or exceed 1.0.

  3. Rescale the intensity to input image bit depth.

Parameters and Presets

Parameters

Recipe parameters for 3D Object Analysis - Cellpose and their descriptions are summarized in the table below. By default, the Morphological Smoothing and Skip Remove Background options are enabled. We recommend keeping Skip Remove Background as the standard processing option as this option has been demonstrated to provide better segmentation outcomes in our test cases.

Preset Group

Parameter Name

Min Value

Max Value

Description

Preset Group

Parameter Name

Min Value

Max Value

Description

Detection

Model Type

1

5

Specifies the model type that can be applied

The available model types are the following:
1: ‘cyto’; 2: ‘nuclei’, 3: ‘cyto2’

 

cyto2 yields good results in most tested cases including nuclei and cytoplasm and is the recommended first step.

Typical Object Diameter

0

1,000

Specifies the typical diameter of objects (e.g. cells / nuclei) on the image. This parameter is used for rescaling the image input to the Cellpose detection algorithm

Image Smoothing Filter Size

(excluding Skip Smooth Image only)

1

100

Specifies the diameter of the filter that is used to smooth the input channel before further processing

The available smoothing types are the following:

  • Skip Smoothing - recommended setting for Cellpose as the first step before trying additional smoothing below

  • Morphological Smoothing applies a morphological filter consisted of an average filter and morphological opening/closing operations based on the value of the Image Smoothing Filter Size. This is the default option.

  • Average Filter Smoothing applies an average filter that outputs the average intensity value for each voxel across a circular region, with diameter specified by the value of the Image Smoothing Filter Size.

  • Gaussian Filter Smoothing applies a Gaussian filter with a filter width specified by the value of the Image Smoothing Filter Size.

  • Median Filter Smoothing applies a median filter that outputs the median intensity value for each voxel across a circular region, with diameter specified by the value of the Image Smoothing Filter Size.

Average Object Radius

(Remove Background only)

0

1,000 (px or µm)

Specifies the radius of a typical object in the image for object enhancement and background removal; a lower value will preserve smaller objects

Min Edge Intensity

0

255 (8-bit)

65,535 (16-bit)

Specifies the minimum object intensity that is typically found at the edge of the object for detection; when Remove Background is enabled, this parameter value is used to specify the minimum object intensity on the enhanced image; a lower value will detect bigger and more objects. In our tested cases, Skip Remove Background is the default recommended first step to try when working with Cellpose.

Fill Holes Size

0

1,000,000 (px2 or µm2)

Specifies the maximum size of gaps in detected objects that are filled; a lower value leads to the preservation of more holes in the detected objects

Partition



Object Radius

0

50,000 (px or µm)

Specifies the range of objects to be included in the analysis results based on the radii of the detected objects

Mesh Smoothing Factor

0

10

Adjusts the amount of smoothing applied to the surface reconstructions of the detected objects; a lower value will generate surfaces with greater similarity to the input image

Min Edge to Center Distance

(Apply Partition only)

0

1,000 (px or µm)

Specifies the minimum distance from the center of an object to the edge that is touching its closest neighboring object; a lower value will apply object partitioning more aggressively, resulting in smaller, more uniform objects

 

Detection

As mentioned above, with Cellpose, it is recommended to first try turning off image smoothing and retaining Skip Remove Background before trying these preset parameters below.

Parameter Name

Low

Medium

High

Parameter Name

Low

Medium

High

Model Type

3 (cyto2)

1 (cyto)

3 (cyto2)

Typical Object Diameter

5

25

80

Image Smoothing Filter Size

(Morphological Smoothing)

3

9

13

Min Edge Intensity

51 (8-bit)

31 (8-bit)

10 (8-bit)

13,107 (16-bit)

7,864 (16-bit)

2,621 (16-bit)

Fill Holes Size

0

0

0

 

Partition

By default, the Apply Partition option is enabled.

Parameter Name

Small

Medium

Large

Parameter Name

Small

Medium

Large

Object Diameter

2 - 5

20 - 50

100 - 300

Mesh Smoothing Factor

0

0

0

Min Edge-to-Center Distance

2

20

100

 

Measurements

The 3D Object Analysis - Cellpose recipe generates morphological and intensity measurements for each detected 3D object as well as a count of the total number of 3D objects on the image. You can add additional measurements to the analysis results by using the Measurement Tool in Aivia and explore measurement definitions on the Measurement Definitions by Object Type page. The measurements generated by the meshes version of the recipe are given in the table below.

Morphological

Intensity

Summary

Morphological

Intensity

Summary

  • Surface Area

  • Volume

  • Mean Intensity

  • Max Intensity

  • Min Intensity

  • Total Intensity

  • Std. Dev. Intensity

  • Object Count

 

Tutorial

Before beginning the tutorial, please download the 3D Object Analysis Cellpose demo data. For information on how to select presets or modify parameter values, please refer to the tutorial on how to use the Recipe Console.

  1. Unzip the demo file and load the demo image, 3DCellposeAnalysisDemo.aivia.tif, into Aivia

  2. In the Recipe Console, click on the Recipe selection dropdown menu and select the 3D Object Analysis - Cellpose recipe

  3. In the Detection preset group, select Small. In the Partition preset group, select Medium.

  4. Click on the Show Advanced Interface icon (see right) to expand the Recipe Console

  5. Click on the first Switch Recipe Operations icon (see right, blue square) to show a list of available recipe operations; select the Skip Smooth Image.

  6. In the expanded Detection preset group, change the following parameters:

    • Min Edge Intensity: 7864

  7. In the expanded Partition preset group, change the following parameters:

    • Object Diameter (µm): 2 - 5

    • Mesh Smoothing Factor: 1

    • Min Edge to Center Distance (µm): 5

  8. Click the Start button or press the F4 key on your keyboard to begin applying the recipe.


You will see one (1) object group with two (2) object sets as outputs: Meshes and Cross Sections. The Cross Sections output allows you to view the cross sections of the detected objects in Main View (2D); the Meshes output allows you to view the surface reconstructions of the detected objects in 3D View.

 

You can color by name and adjust transparency, and (optionally) turn off specular lighting to get the results below.

 

Show Advanced Interface Icon

 

 

 

Switch Recipe Operations to Skip Smooth Image

 



Results

 

 

Image credits

Andreas Moor, ETH Zurich, intestine organoid with smFISH labeling with DNA labeling with DAPI (blue), plasma membrane labeling (green), and smFISH probe (magenta)

 

Citations

  1. Stringer C, Wang T, Michaelos M, Pachitariu M (2021). Cellpose: a generalist algorithm for cellular segmentation. Nature Methods. Dec. 14 2020; 18: 100-106.

Related articles