Aivia Software
Workflow Processor (Enhanced Batch, Version 15+)
AboutAivia’s Workflow Processor is a powerful way to chain core parts of Aivia together and run the sequence of steps on single or batches of files. The standard way of using the Workflow Processor is to first complete the analysis you want without the Workflow Processor. Then build a workflow using the knowledge and settings you have. Finally batch run the analysis on many files and enjoy the results.
Open Workflow ProcessorTo access the Workflow Processor, at the top left of Aivia choose the Analysis menu, and then the Workflow Processor option. When automatically making a workflow, the Workflow Processor will also open automatically to the Runner section. Main Topics |
---|
On this page:
|
---|
Automatically Create Workflow
The Batch Workflow Creator tool allows automatic generation of workflows. This tool will allow you to see the overview of the workflow in a more simple form before creation. Fear not, you can also edit the created workflow to improve or advance it.
To create a workflow automatically, please refer to the Batch Workflow Creator tool page for more detailed information.
Legacy Automatic Workflow Creation
For the older legacy workflow creation please refer to the legacy Workflow Processor page
Navigation
The Workflow Processor has the main navigation tabs in top right.
The areas from left to right are: Editor, Runner, Queue, History.
Normal use progresses from left to right.
Editor
The Editor is where most of the settings are picked. This is where parts of Aivia are able to be chained together. Once you are done, save the .workflow file with the Save icon on the right side and you can use those settings over and over.
Legacy Workflow Editing can still be performed on any old workflow file or through the advanced menu in the Create New Workflow button.
Workflow Creation
There are two primary paths to creating workflows.
Automatic creation through the Batch Workflow Creator tool (Recommended)
Advanced Manual Creation from the “Create New Workflow” button
Advanced Manual Creation
Clicking the “Create New Workflow” button will create a blank slate to generate a workflow from. A single processing step for the purposes of loading an image will by default be added. Processing is done from top to bottom in sequence, to add a new processing step to the workflow, click the “Add Processing Step” button. The trash can button allows removal of processing steps. The arrow buttons allow reordering the steps.
Processing Steps
There are many different types of processing steps that a workflow can perform. The general premise of a workflow is to Load an image, process the image, and then save the results. Mixing and matching different processing steps allows a powerful combination of capabilities.
Loading
Image Load
Loads a single file defined in the “Runner” tab
Loads a sequence of files into a single file with automatic identification of the format
Processing
Preset Measurements are considered Recipes for the purposes of workflows
Segment By Example Apply
Additional
Save Data
This step is generally not needed except for in advanced use cases, such as saving intermediate data. The Runner step will handle saving data based on the selection in its UI. Adding a save data step will save independently of the Runner’s settings
Expanding the Options section of a processing step will allow modification of the parameters of that step.
Custom Measurements
This is an optional, and advanced step. Most users will be happy using the measurements that Recipes include. In case you have specific measurements you want to include, you can save your measurements in the Analysis menu, Measurement Preset Tool. Once you have a Preset in the Measurement Preset Tool (either create or load one), you can select that preset in the Workflow in the Custom Measurements, Recipe area. To add custom measurements use the “Recipe Apply” processing type and the Preset Measurement recipes loaded in the Measurement Preset Tool will be available for selection.
Also see the Input Selections below.
Warning: Custom Measurements can be confusing and only recommended for highly advanced users.
Input Selections
Each processing step can have one or more Input Selections, depending on what you have selected. It is here that you describe to the workflow how you want your images to be processed. There are cases where we are unable to know what is desired. For example, the images you are going to process already have results, or will have more/less channels than the one you did your analysis on. Since Aivia has not opened or seen these images, it cannot guess what is desired.
Knowing the format of the input files as well as the required inputs to the processing steps is important in selecting the right values. When automatically created leaving the input selections at their default will yield the results you are looking for. Other times you may have to adjust some Channel or Object Set Inputs. The input sector defines which channel is used by the step. If your image was loaded with 2 channels, and the previous step creates a 3rd that you want to use in this step, choose 3.
For the purposes of selecting Object Sets/Groups the index is based on any hierarchy being flattened. For example, “New Classification” is the 3rd index in the picture provided.
Warnings:
Some recipes have a large number of input selectors, please try on a example image to ensure you have the settings right.
It is recommended to not adjust any Object Set inputs unless you are an expert.
Beginners are recommended to use the Automatically Create Workflow method.
Runner
The runner is where you run workflows. You can load previously created workflows, or select one you have just created. There is an Edit icon on each available workflow which will take you to the Editor.
File Input
Select your files, folders full of files (includes all children in subfolders too), or Aivia Experiment file (see the Experiment Editor [todo: add link]). Any file that Aivia can normally open is processable here. Library files will load all the files and you can filter by removing fields you don’t want. Aivia Experiment files will give you an option of filtering by Conditions that were previously setup in the experiment.
Note: If the workflow had its Image Format set as Single from Many, all selected images will be stacked together into a single image, and you will get one set of results.
Saving
Here you can choose what and where you want your things.
Warning: The Save Folder should be a place with lots of space, and ideally with not too long of a path name. Windows has a 256 character limit, and Workflows will output some folders and files with longer names (assuming you have long-named workflows and files). If the save folder + folder structure that workflows make + file names are beyond that limit, the job will fail and you will need to make some changes before running again. There is a rumor of a setting in Windows that allows longer paths, but we have not seen that work. Each time you start a workflow it makes a subfolder in the Save Folder with the date and subfolders in that folder for each image/job. Experiments add some additional folders to keep everything organized.
Additional Save Options: Make sure you choose Save Field if you want to save an Aivia.tif with all the objects/measurements on it.
Save Field - Save the processed image, detected objects and measurements to a single Aivia.tif.
Export Measurements to .xlsx - Export in Excel format
Export Measurements to .csv - Export in comma-separated format
Export Channels - Each channel in the processed image will be saved to a separate tif file
Export Objects as Masks - Each object set in the processed image will be saved to a labeled tif
Queue
The Queue displays jobs that have yet to start, as well as the currently processing job, along with its status. You can hit Pause to hold the queue. The next job won’t start, but the current job will complete.
You can resort the queue by pressing the Up / Down arrows next to each job, or remove a job by pressing the delete button. There is a Clear Queue button at the bottom that will remove all queued jobs.
History
The History panel shows all the jobs that have completed (or failed). Restarting Aivia will clear the History, or you can push Clear All at the bottom of the history list.
Clicking on an item in the history list will show some details and even a preview if possible. Previews list the centers of all detected objects (if any) with pink dots.
Each Item has some details, and a few buttons:
Save: Saves off a .json file of details about the objects detected.
Open: Opens a Windows File Explorer to that file’s location on disk.
Delete: Removes that history item from the list.
The bottom of the History list has a Clear All button that you can use to remove all history items.
Lastly, in case of any error, the history item will have a red border, and clicking on it will tell some details about what happened. Please check Aivia’s log (Aivia > File > Options > Logging) for more details.
Additional Information
Here is some extra info that may help. If in doubt, give it a try, and as always please give us feedback to let us know what you want and what is working well for you or isn’t working all that great.
Components Possible to Connect into a Workflow
Image Importer (limitations apply [1])
Calibration
Image Enhancement
Pixel Classifier
Recipes
DL Model Recipes (limitations apply [2])
Python Recipes (limitations apply [2])
Custom Measurements
Object Classifier
Spatial Relations
Segment By Example
[1] The workflow image importer does not have manual configuration available. Rather it only automatically processes the input files based on filename. To achieve more flexibility and control, please use Aivia’s File > Import > Image Sequence. You can also check how the Workflow Image Importer will behave by using the Import Image Sequence tool and watching how it automatically sorts/orders. By renaming the image files it may be able to automatically sort your input images as you wish. Additionally, when using the workflow importer to stack many images into a single one, that workflow will only process a single many-to-one sequence at a time. You could setup and run the workflow many times, but it wouldn’t be ideal. Please let us know if this is a scenario you face often.
[2] In some cases you may need to load the custom recipe file into Aivia’s Recipe Console, and then select it again in the Workflow Edit panel before running the workflow.
Formats that Workflows can Process
Any image format that Aivia can open normally
Leica .lif files and similar Library files
Aivia Experiment files (eg, wellplates)
Tips
Here we will share some ideas, tips, tricks, and steps to help you get the most out of Aivia’s Workflows
Editing the Recipe settings used in an existing Workflow
If you have an existing workflow and want to change the Recipe settings that are used, follow these steps:
Ensure the Recipe and Recipe’s settings are in the recipe console as you want them used in the workflow. Normally you would have just run the Recipe and like the results, so you can then update the workflow immediately.
Open and Edit the Workflow you want to make changes to. Go to the Analysis section.
Click the Load Settings from Recipe Console button
Save workflow
Done!
Sharing Workflows with Friends
Workflows currently need a few extra files to be passed, along with the workflow file itself. If your workflow uses any of the following, make sure you share them also with your friends:
Pixel Classifier
Object Classifier
Spatial Relations
Segment by Example
Custom Measurement Recipe
Custom Recipe
Custom DL model
These additional files should be loaded into Aivia before Running the workflow. Or they can be located at the same location on disk though that is often more difficult to perform. For example, a common network drive path would work, but the user’s desktop probably would not. In case a required file is missed, Aivia will inform you and you can go ask your friend.