The Pixel Classifier is an image enhancement tool that enhances specific features on an image based on user-input teaching patterns. A number of image transformations are applied to the user-defined teaching regions (and their respective adjacent pixels) internally to identify unique features (such as edges or speckles) for classification.
Interface
The Pixel Classifier is a tab in the Analysis Panel, which is pinned and found on the right side of the Aivia window by default. When you launch the Pixel Classifier, the tool selects the most recently used classifier or starts a new classifier with two (2) predefined classes to help you get started right away.
The main Pixel Classifier GUI is shown below. There are five (5) sections in the Pixel Classifier interface:
The Classifier Toolbar along the top of the GUI allows you to perform file actions related to the Pixel Classifier.
The Input & Training section allows you to specify the image channel(s) to use for training and applying the classifier.
The Drawing section lets you select a tool for defining training regions in the Image Panel.
The Classifier section lets you select the pixel class to teach as well as specify output options.
The Apply Controls along the bottom of the GUI allow you to preview, teach, or apply a classifier and specify regions of interest (ROIs) for previewing and applying.
Pixel Classifier GUI
| Panel |
|---|
On this page: |
| Table of Contents |
|---|
Classifier Toolbar
The Classifier Toolbar is located at the top of the Pixel Classifier tab. The toolbar lets you perform file actions related to the Pixel Classifier, such as creating a new classifier; saving a classifier, training set, or annotations; and loading classifier files.
Classifier Toolbar
The function for each item in the toolbar is summarized in the table below.
Function | Icon | Description |
|---|---|---|
Select Classifier | Shows the name and type (2D or 3D) of the currently selected classifier; click on the item to expand a dropdown menu from which you may select a loaded/created pixel classifier for training and/or applying | |
Create New |  | Allows you to create a new, blank pixel classifier for teaching; click on the downward-pointing triangle to expand a menu where you may specify whether to create a 2D or 3D classifier |
Delete |  | Removes the current pixel classifier from Aivia; this does not delete the pixel classifier from its file if the classifier was loaded from a file |
Remove All Classifiers |  | Removes all currently loaded classifiers |
Load |  | Allows you to load a pre-trained pixel classifier (.pxclassifier), training set (.pxtraining), or set of annotations (.pxannotations) from a file |
Save |  | Allows you to save the current pixel classifier, training set, or annotations to a file |
Input & Training
The Input & Training section lets you select the image channel(s) to use for training when creating a new pixel classifier or to specify the image channel(s) to use as input for applying a previously trained and finalized (saved as a .pxclassifier file) pixel classifier. During training, the teaching images are automatically set as the input images.
You may specify the number of channels to use as inputs for the classifier as well as the names of the channel inputs in the Input Definitions column. The table below describes the options available in the Input Definitions column.
Function | Icon | Description |
|---|---|---|
Edit Input Name |  | Enables editing of the classifier input name; you can also double-click on an input name to enable editing of the name |
Add Input Channel |  | Adds a channel input |
Remove Input | | Removes the corresponding classifier input |
In the Training Channels column, you may use the dropdown menu(s) to select which channel(s) on the current image to use as which input(s) for training the pixel classifier.
The Manage Training Set button opens a window with options for saving the annotations for the individual images in the training set as well as removing images from the training set. See the "Train a pixel classifier over multiple sessions" section of this page for more details about saving training data.
| Tip |
|---|
For teaching on colocalized regions (regions with overlapping signals on multiple channels), make sure to include all the colocalized channels as training channels. |
Only the Input section is displayed when you are applying a previously trained and finalized pixel classifier to an image. You may use the dropdown menu(s) to select which image channel(s) on the current image to provide as which input(s) to the classifier.
Input & Training section during training
Input section for applying a previously trained classifier
Drawing
When teaching a pixel classifier, use the variety of drawing tools to draw some examples of your classes. All drawing, except with the Magic Wand, must be done in Main View (2D).
Drawing tools
The function and description for each of the drawing tools are summarized in the table below.
Function | Icon | Description |
|---|---|---|
Paint |  | Paints a circle under the cursor for the currently selected class; the brush size is adjustable |
Erase |  | Erases a circle under the cursor for every class; the eraser size is adjustable |
Flood |  | Fills in enclosed regions with the selected class; the enclosure can be made up of any class(es) |
Region Drawing |  | Draws a freehand drawing that automatically closes |
Auto Draw |  | Predicts the contour on the next z-plane based on previous drawing; see the "Auto Draw mode" section for more details |
Magic Wand |  | Uses flood-filling to paint a region that has similar intensity and is connected to the selected location; in 3D, the Magic Wand paints regions for two (2) classes, one (1) for the selected class and one (1) for the background class or other class (see the "Magic Wand" section) |
Size Selector |  | Adjusts the size of the brush or eraser |
Jump to Previous Teaching Frame |  | Jumps to the previous frame with a teaching region drawn on it, moving back through Z and then through time |
Jump to Next Teaching Frame |  | Jumps to the next frame with a teaching region drawn on it, moving forward through Z and then through time |
Automatically Move to Next Frame (only available for 3D images) |  | Sets the mode for moving between z-frames after a drawing is completed; the modes are as follows:
|
Auto Draw mode
Auto Draw options
When Auto Draw mode is selected, a set of options are shown underneath the row of drawing tools (see above). First, select the channel to use for prediction in the dropdown menu on the left, which shows "GFP" in the example above. Click and drag the mouse to draw the outline of a region on the image; when the mouse button is released, the ends of the drawn line are automatically connected to create a closed loop. Use the Interpolate Down button and Interpolate Up button (or hotkeys S and D) to interpolate the next drawing down or up in Z respectively. The Step to Bottom button and Step to Top button (or hotkeys A and F) can be used to visit the bottom and top drawings respectively. Click on the Clear icon to remove all drawing from the current z-slice or on the Reset icon to clear all drawing. Click on Finish Drawing to assign the drawn regions (enclosed by the blue outlines) on all slices to the selected class.
Magic Wand
Magic Wand options
Options for the Magic Wand tool are provided below the row of drawing tools when the Magic Wand is selected (see above). The Magic Wand may be used for annotation in 2D or 3D viewing modes. In 2D or 3D, the first step is to select the image channel to use from the dropdown menu, which shows "GFP" in the image above.
In Main View (2D), expand the Thresholding Mode dropdown menu to choose one of the following modes:
Bidirectional mode accepts connected pixels within the sensitivity limit above and below the intensity at the selected location.
Peak Finding mode accepts connected pixels within the sensitivity limit above the intensity at the selected location.
Valley Finding mode accepts connected pixels within the sensitivity limit below the intensity at the selected location.
Select a location on the image for the Magic Wand to search from. A preview of the region to be painted is shown in blue (see right). Adjust the Sensitivity slider (or use hotkeys W and R) to change the tolerance for intensity differences. The narrowest range of accepted intensity levels is used when the slider is all the way to the left. Adjust the Search Range slider (or use Ctrl + Shift + Mouse Scroll over the Image Panel) to increase or decrease the area over which the Magic Wand will search. No pixels outside of the search region are included in the painted region. The search range is indicated by the green box around the cursor when the Magic Wand is used (see right). Check the Fill Multiple Z Frames box to extend the user-defined search range limits in the z-direction; when this box is not checked, the Magic Wand only searches on the current z-frame. Finally, click on Paint (or press E) to paint the region and assign it to the selected class.
| Info |
|---|
When the Fill Multiple Z Frames option is used, the painted regions may not exactly match the blue preview. |
With the Magic Wand in 3D View, click on the image to show a surface preview. You may adjust the search range using either the Search Range slider or the Ctrl + Shift + Mouse Scroll hotkey combination. To adjust the range of intensities included in the region, adjust the LUT of the channel being used for the Magic Wand and click again on the image. Regions do not have to be connected to be painted and assigned to the selected class when the Magic Wand is used in 3D. The Magic Wand paints to two (2) classes: one (1) is the selected class, and the other is what is presumed to be the background class. This means that the total search volume is painted. When you click on Paint, a yellow frame around the painted region appears in 3D View. Return to Main View (2D) to closely examine the resulting annotations.
Selected location (small green square), search limits (large green square), and region preview (blue) for the Magic Wand
Hotkeys for drawing
Drawing mode | Hotkey | Description |
|---|---|---|
Paint |
| Switches to erasing while held down |
| Adjusts the brush size when done over the Image Panel | |
Erase |
| Switches to painting while held down |
| Adjusts the eraser size when done over the Image Panel | |
Auto Draw |
| Goes to the bottom drawing |
| Predicts the next drawing down | |
| Predicts the next drawing up | |
| Goes to the top drawing | |
| Clears all drawing | |
Magic Wand |
| Decreases the sensitivity |
| Increases the sensitivity | |
| Paints the selected region | |
| Adjusts the search range when done over the Image Panel |
Classifier section
The Classifier section is the primary space for interacting with individual pixel classes in the Pixel Classifier. To begin training a pixel class, select the class's row prior to drawing the regions in the Image Viewer. The Classifier section also lets you specify the output type and options for each pixel class and add additional training features.
To edit the name of the classifier, double-click on the classifier name above the "[s]elect pixel class" text and then edit the text.
The function and description for each item in the Classifier section are summarized in the table below.
Classifier section
Function | Icon | Description |
|---|---|---|
Expand/Collapse Output Parameters |   | Expands or collapses the output parameter section for each class |
Advanced Options |  | Lets you select different features and kernel sizes to use for the pixel classifier |
Hide/Show Class Annotations |   | Hides/Shows the training regions on the image |
Select Output Type |  | Lets you select from the following four (4) output types for each pixel class:
|
Class Color |  | Specifies the color of the drawn training region and, if Confidence is selected as the output, the channel color for the preview and output of the given class |
Select Output |  | Opens the options for enabling/disabling output(s); you can disable the output for a given class by unchecking each available output; if Masked Channel is selected as the output, you may have the option to select multiple channels to mask out. |
Remove Class | | Removes the class from the pixel classifier |
Add Class | | Adds a new class to the pixel classifier |
Output parameters
Depending on the output type chosen, there are additional parameters that allow you to fine-tune the final output. To expand the parameters, click on the caret to the left of the Hide/Show Class Annotations button. The parameters are summarized in the table below by output type.
Output type | Parameter name | Description |
|---|---|---|
Masked Channel | Detection | Specifies the confidence threshold for the masked region; moving the slider to the right lowers the minimum confidence threshold and results in more pixels being included in the output |
Remove Small Objects | Removes small objects that are likely the result of noise | |
Segmentation | Detection | Specifies the confidence threshold for the segmented objects; moving the slider to the right lowers the minimum confidence threshold and results in detection of more low-confidence objects as well as larger and more connected objects |
Remove Small Objects | Removes small objects that are likely the result of noise | |
Smart Segmentation | Detection | Specifies the confidence threshold for the segmented objects; moving the slider to the right lowers the minimum confidence threshold and results in detection of more low-confidence objects as well as larger and more connected objects |
Partition | Specifies the aggressiveness of the partitioning algorithm applied to the segmented regions; moving the slider to the right increases the aggressiveness of partitioning and results in more partitioned objects | |
Object Size | Specifies the object size range based on the user-drawn regions; use the lower threshold (left slider control) to modify the minimum size and the upper threshold (right slider control) to modify the maximum size |
Advanced Options
The Advanced Options menu lets the user specify the image features to use for creating the pixel classifier. Click on a feature name in the table to add the feature to the list; features with the GPU-Computed symbol (see below) can be processed on a GPU.
GPU-Computed symbol
Additionally, Aivia provides three (3) preset groups of features:
Clear deselects all features; you will not be able to teach the Pixel Classifier until at least one (1) feature is selected.
Standard includes the default features that are used in the Pixel Classifier.
Line includes specialized features designed to highlight fine, line-like features.
The Kernel Size option lets you choose Small (micro), Medium, and/or Large (macro) kernels for applying the image feature transformations. Smaller kernels are more attuned to highly localized features, while larger kernels are more accurate for detecting major image features.
Advanced Options menu
Apply Controls
The Apply Controls are located at the bottom of the Pixel Classifier tab and include options for previewing, teaching, and applying as well as specifying ROIs. Prior to applying a pixel classifier, the user must click the Teach button to train the pixel classifier.
Preview
Preview icon highlighted in the Apply Controls section
Click on the Preview icon (see above) to initiate preview generation; while previewing is toggled on, there is live previewing that updates as changes to the drawings/annotations are made. All output types may be previewed in Main View (2D), and Segmentation and Smart Segmentation outputs can be previewed in 3D View.
Click on the downward-pointing triangle next to the Preview icon to reveal the Speed/Accuracy slider (see below), which lets you control the quality of the preview. Moving the slider toward Speed leads to faster preview generation (at the expense of accuracy), while moving the slider toward Accuracy leads to the generation of previews that more closely resemble the output (but take longer to generate). If any ROIs are selected, previews are only generated in those ROIs.
Speed/Accuracy slider for previews
| Tip |
|---|
Turn previewing on while you are drawing to gauge when your annotations may be sufficient. |
Specify regions of interest
Click on the ROI Processing Options icon to open the Regions Of Interest Table (see right). When the ROI Processing Options icon appears as nested squares, a ROI is selected. Check the box in an ROI's row of the Regions Of Interest Table to select it for previewing and/or applying. Use the topmost checkbox, which is in the header row, to (de)select all ROIs. See the Regions of Interest page for more information about using ROIs.
Teach
The Teach button creates a pixel classifier using the drawn training regions for all training images and the feature calculation from those regions. An asterisk to the left of the Teach button indicates that changes have been made to the pixel classifier since it was last taught. At least two (2) classes must have training regions and at least one (1) feature must be selected before a pixel classifier can be taught.
Apply
The Apply button applies the taught pixel classifier to the selected ROIs or, if no ROIs are selected, the whole image. You can choose to apply to the whole image sequence (over all of T and Z), just the current t-frame, or just the current t- and z-frame (except if you are creating 3D objects).
The downward-pointing triangle to the right of the Apply button provides additional options for generating pixel classifier output. Select one (1) of the output options from the dropdown menu to apply the pixel classifier. There are four (4) apply options:
Apply applies the pixel classifier to the entire image sequence (over all of T and Z).
Apply to the current time frame applies the pixel classifier to the current t-frame.
Apply to the current time and depth frame applies the pixel classifier to the currently displayed t- and z-frame only.
Batch apply the classifier opens a window where you can specify a collection of images to apply the pixel classifier to as well as an output folder for the results.
ROI Processing Options icon (bottom right) and Regions Of Interest (ROI) Table
General usage of the Pixel Classifier
Create a new pixel classifier
To create a new pixel classifier, click on the Create New icon in the Classifier Toolbar. A blank pixel classifier will then be created with two (2) pre-configured classes: "Want" and "Background." The general workflow for training a new pixel classifier is as follows:
1. Draw teaching regions using any tools in the Drawing section to paint desired features of interest. Select a different class from the class table to draw teaching regions for that class.
| Tip |
|---|
You need at least two (2) pixel classes, each with at least one (1) teaching region, to preview, train, and apply a pixel classifier. |
2. Preview the results. Use ROIs for faster previews.
3. Add more teaching regions to the image, following the same instructions as are in Step 1; keep previewing on to determine the quality of the teaching as you add more teaching regions.
| Tip |
|---|
You can load or switch to other images to add them to your training set. |
4. Teach the pixel classifier by pressing the Teach button.
5. Select the output types and destinations using the corresponding menus in each class's row of the class table.
6. Apply the pixel classifier to the image.
Train a pixel classifier over multiple sessions
There are two (2) types of files that you can save and load back into Aivia for training at a later date: annotation files (.pxannotations) and training set files (.pxtraining).
Annotation files
Annotation files contain the drawn teaching regions for an image. To save annotations for the displayed image and current classifier, click on the downward-pointing triangle next to the Save icon in the Classifier Toolbar and then select Save Annotations for the current image (see right); this opens a window where you may specify the location and name for the .pxannotations file. Another option is to click on the Manage Training Set button and then on the Save icon for the image in the training set whose annotations you would like to save; note that this method allows you to save annotations for an image in the training set that is not currently displayed in the Image Viewer.
It is assumed that when annotations are loaded, the image that they are from and the classifier that was being trained (with the same number of classes, class names, etc.) are already open and displayed. To load annotations, click on the Load icon in the Classifier Toolbar, navigate to the .pxannotations file, and then click on Open. The previously drawn teaching regions are then displayed and may be edited and used for previewing, training, and applying pixel classifiers.
Training set files
Training set files include the following:
Paths to all images in the training set
Drawn teaching regions (annotations) for all images in the training set
Classifier settings
The taught classifier, if the training set was saved with a taught pixel classifier
To save a training set file, click on the downward-pointing triangle next to the Save icon in the Classifier Toolbar and select Save Training Set; this opens a window where you may specify the location and name for the .pxtraining file. Another option is to not only save the .pxtraining file but also copy the images in the training set to another location. To do this, click on the downward-pointing triangle next to the Save icon in the Classifier Toolbar, select the Copy all the training fields and training data to one spot option, navigate to the folder where you would like to copy the training images and .pxtraining file to, and then click on Select Folder.
To load a training set, click on the Load icon in the Classifier Toolbar, navigate to the .pxtraining file, and then click on Open or drag and drop the .pxtraining file onto the Pixel Classifier interface; this will load all of the images in the training set into the workspace as well as load the previously drawn training regions and classifier settings for the in-progress pixel classifier. You may then continue drawing teaching regions, adding images to the training set, and otherwise building upon and/or using your pixel classifier.
| Info |
|---|
Changing the file location of images in a training set can prevent loading of the training set. You may wish to use the Copy all the training fields and training data to one spot option to avoid such an occurrence. |
Pixel Classifier saving options
Save a pixel classifier
When you have a taught classifier that needs no further tuning, you may save the classifier as a .pxclassifier file; to do so, click on the downward-pointing triangle next to the Save icon in the Classifier Toolbar, select Save Classifier, choose the name and location for the file, and then click on Save. You may then load the classifier by clicking on the Load icon in the Classifier Toolbar, navigating to the .pxclassifier file, and then clicking on Open or by dragging and dropping the .pxclassifier onto the Pixel Classifier interface.
| Tip |
|---|
If you would like to save a taught classifier but there is a possibility you may want to change its settings or teaching regions later, save the training set instead of (or in addition to) the classifier. The training set file includes the taught classifier, if one is available, so that the classifier can be immediately applied in the future but can also be retaught. |
Technical details
The Pixel Classifier consists of two (2) fundamental steps:
Feature Calculation: Image features (such as edges) are calculated for every pixel of the input image to generate a feature vector.
Classification Application: The feature vector is used to train a classifier to classify unknown feature vectors into one (1) of the classes specified by the user.
Each image feature is derived from applying one (1) of several processing filters on the image over several scales, or kernel sizes, to identify both regional (macro) and local (micro) features inherent in the image. A random forest algorithm is used for creating the pixel classifier from the calculated features.
The classification result is output as new image channels that map the probability that a given pixel belongs to a specific class from 0 (high likelihood of not belonging) to 255 (high likelihood of belonging). The probability map channel can then be used for additional analysis.
Related articles
| Filter by label (Content by label) | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
| Page Properties | ||
|---|---|---|
| ||
|