Aivia Software
3D Object Analysis - Cellpose
The 3D Object Analysis - Cellpose recipe (for Aivia 11 and later) detects objects (such as nuclei or cells) in 3D volumetric images using Cellpose, a generalist deep learning algorithm for cell segmentation, developed by Carsten Stringer et. al. [1]. The recipe uses Cellpose to generate confidence map of the input image for 3D object segmentation that improves overall 3D segmentation accuracy. Unlike the standard Cellpose algorithm, the 3D Object Analysis - Cellpose recipe do not use the flow maps output for segmentation. For the image enhancement recipe that utilizes flow map for segmenting and labeling detected objects, please refer to the documentation on Cellpose Object Detection / 3D Cellpose Object Detection.
On this page:
Technical Background
Cellpose is a generalist algorithm for segmentation of cells with various morphological types acquired via different imaging modalities. Cellpose was originally published by Stringer et al in 2020 (publication link). The raw confidence map range is in logit, meaning its range is (-∞, +∞) and we convert the confidence map to (0, 1) using the sigmoid function. This conversion, however, will result in most signals concentrating at 0 or 1, with very little information in between, making it difficult for Aivia’s 3D Object Analysis recipe to work with such a narrow range. Instead of converting it to probability values using the sigmoid function, we found it is useful to convert it to input image bit depth using the following steps.
Normalize the confidence map using the 60th percentile as 0.0 and 99.98th percentile as 1.0.
Clip the voxels that are below 0.0 or exceed 1.0.
Rescale the intensity to input image bit depth.
Parameters and Presets
Parameters
Recipe parameters for 3D Object Analysis - Cellpose and their descriptions are summarized in the table below. By default, the Morphological Smoothing and Skip Remove Background options are enabled. We recommend keeping Skip Remove Background as the standard processing option as this option has been demonstrated to provide better segmentation outcomes in our test cases.
Preset Group | Parameter Name | Min Value | Max Value | Description |
|---|---|---|---|---|
Detection | Model Type | 1 | 5 | Specifies the model type that can be applied The available model types are the following:
cyto2 yields good results in most tested cases including nuclei and cytoplasm and is the recommended first step. |
Typical Object Diameter | 0 | 1,000 | Specifies the typical diameter of objects (e.g. cells / nuclei) on the image. This parameter is used for rescaling the image input to the Cellpose detection algorithm | |
Image Smoothing Filter Size (excluding Skip Smooth Image only) | 1 | 100 | Specifies the diameter of the filter that is used to smooth the input channel before further processing The available smoothing types are the following:
| |
Average Object Radius (Remove Background only) | 0 | 1,000 (px or µm) | Specifies the radius of a typical object in the image for object enhancement and background removal; a lower value will preserve smaller objects | |
Min Edge Intensity | 0 | 255 (8-bit) 65,535 (16-bit) | Specifies the minimum object intensity that is typically found at the edge of the object for detection; when Remove Background is enabled, this parameter value is used to specify the minimum object intensity on the enhanced image; a lower value will detect bigger and more objects. In our tested cases, Skip Remove Background is the default recommended first step to try when working with Cellpose. | |
Fill Holes Size | 0 | 1,000,000 (px2 or µm2) | Specifies the maximum size of gaps in detected objects that are filled; a lower value leads to the preservation of more holes in the detected objects | |
Partition | Object Radius | 0 | 50,000 (px or µm) | Specifies the range of objects to be included in the analysis results based on the radii of the detected objects |
Mesh Smoothing Factor | 0 | 10 | Adjusts the amount of smoothing applied to the surface reconstructions of the detected objects; a lower value will generate surfaces with greater similarity to the input image | |
Min Edge to Center Distance (Apply Partition only) | 0 | 1,000 (px or µm) | Specifies the minimum distance from the center of an object to the edge that is touching its closest neighboring object; a lower value will apply object partitioning more aggressively, resulting in smaller, more uniform objects |
Detection
As mentioned above, with Cellpose, it is recommended to first try turning off image smoothing and retaining Skip Remove Background before trying these preset parameters below.
Parameter Name | Low | Medium | High |
|---|---|---|---|
Model Type | 3 (cyto2) | 1 (cyto) | 3 (cyto2) |
Typical Object Diameter | 5 | 25 | 80 |
Image Smoothing Filter Size (Morphological Smoothing) | 3 | 9 | 13 |
Min Edge Intensity | 51 (8-bit) | 31 (8-bit) | 10 (8-bit) |
13,107 (16-bit) | 7,864 (16-bit) | 2,621 (16-bit) | |
Fill Holes Size | 0 | 0 | 0 |
Partition
By default, the Apply Partition option is enabled.
Parameter Name | Small | Medium | Large |
|---|---|---|---|
Object Diameter | 2 - 5 | 20 - 50 | 100 - 300 |
Mesh Smoothing Factor | 0 | 0 | 0 |
Min Edge-to-Center Distance | 2 | 20 | 100 |
Measurements
The 3D Object Analysis - Cellpose recipe generates morphological and intensity measurements for each detected 3D object as well as a count of the total number of 3D objects on the image. You can add additional measurements to the analysis results by using the Measurement Tool in Aivia and explore measurement definitions on the Measurement Definitions by Object Type page. The measurements generated by the meshes version of the recipe are given in the table below.
Morphological | Intensity | Summary |
|---|---|---|
|
|
|
Tutorial
Before beginning the tutorial, please download the 3D Object Analysis Cellpose demo data. For information on how to select presets or modify parameter values, please refer to the tutorial on how to use the Recipe Console.
Unzip the demo file and load the demo image, 3DCellposeAnalysisDemo.aivia.tif, into Aivia
In the Recipe Console, click on the Recipe selection dropdown menu and select the 3D Object Analysis - Cellpose recipe
In the Detection preset group, select Small. In the Partition preset group, select Medium.
Click on the Show Advanced Interface icon (see right) to expand the Recipe Console
Click on the first Switch Recipe Operations icon (see right, blue square) to show a list of available recipe operations; select the Skip Smooth Image.
In the expanded Detection preset group, change the following parameters:
Min Edge Intensity: 7864
In the expanded Partition preset group, change the following parameters:
Object Diameter (µm): 2 - 5
Mesh Smoothing Factor: 1
Min Edge to Center Distance (µm): 5
Click the Start button or press the F4 key on your keyboard to begin applying the recipe.
You will see one (1) object group with two (2) object sets as outputs: Meshes and Cross Sections. The Cross Sections output allows you to view the cross sections of the detected objects in Main View (2D); the Meshes output allows you to view the surface reconstructions of the detected objects in 3D View.
You can color by name and adjust transparency, and (optionally) turn off specular lighting to get the results below.
Results
Image credits
Andreas Moor, ETH Zurich, intestine organoid with smFISH labeling with DNA labeling with DAPI (blue), plasma membrane labeling (green), and smFISH probe (magenta)
Citations
Stringer C, Wang T, Michaelos M, Pachitariu M (2021). Cellpose: a generalist algorithm for cellular segmentation. Nature Methods. Dec. 14 2020; 18: 100-106.