Python Segmentation API

[13]:

# must run in an environment where polarityJaM is installed
from polarityjam import Plotter
from polarityjam import RuntimeParameter, PlotParameter, ImageParameter
from polarityjam import PolarityJamLogger
from polarityjam import load_segmenter
from polarityjam import SegmentationMode
from polarityjam import SegmentationParameter
from polarityjam.utils.io import read_image

from pathlib import Path

[14]:

# Setup a logger to only report WARNINGS, Put "INFO" or "DEBUG" to get more information
plog = PolarityJamLogger("WARNING")

Setup your data

We expect you to be already familiar with basic usage of polarityjam. Hence we will shorten necessary data preparation steps. Please look at the Python API section of the documentation for more information.

[15]:

### ADAPT ME ###
path_root = Path("")
input_file = path_root.joinpath("data/golgi_nuclei/set_2/060721_EGM2_18dyn_02.tif")
output_path = path_root.joinpath("polarityjam_out/")
output_file_prefix = "060721_EGM2_18dyn_02"
### ADAPT ME ###

img = read_image(input_file)

# Parameters
params_image = ImageParameter()
params_image.channel_organelle = 0  # golgi channel
params_image.channel_nucleus = 2
params_image.channel_junction = 3
params_image.channel_expression_marker = 3
params_image.pixel_to_micron_ratio = 2.4089

# Parameters
params_runtime = RuntimeParameter()
params_plot = PlotParameter()
params_plot.dpi = 96

# plotter object
plotter = Plotter(params_plot)

Several segmentation algorithms in parallel

Polarity-JaM currently supports three segmentation algorithms out of the box.

Here, we show how easy it is to switch from one to another and how to compare performance.

Step 1 - Prepare your image

[16]:

# Cellpose - switch off mask loading (if exists) and saving
cp_seg_p = SegmentationParameter(params_runtime.segmentation_algorithm)
cp_seg_p.use_given_mask = False
cp_seg_p.store_segmentation = False
cp_seg, _ = load_segmenter(params_runtime, cp_seg_p)
cp_seg_prep, cp_seg_prep_p = cp_seg.prepare(img, params_image)
plotter.plot_channels(cp_seg_prep, cp_seg_prep_p, output_path=output_path, filename=input_file);

../_images/notebooks_polarityjam-notebook_seg_7_0.png

[17]:

# Deepcell - default parameters
params_runtime.segmentation_algorithm = "DeepCellSegmenter"
dc_seg, _ = load_segmenter(params_runtime)
dc_seg_prep, dc_seg_prep_p = dc_seg.prepare(img, params_image)
plotter.plot_channels(dc_seg_prep, dc_seg_prep_p, output_path, input_file);

../_images/notebooks_polarityjam-notebook_seg_8_0.png

[18]:

# microSam - default parameters
params_runtime.segmentation_algorithm = "MicrosamSegmenter"
ms_seg, _ = load_segmenter(params_runtime)
ms_seg_prep, ms_seg_prep_p = ms_seg.prepare(img, params_image)
plotter.plot_channels(ms_seg_prep, ms_seg_prep_p, output_path, input_file);

../_images/notebooks_polarityjam-notebook_seg_9_0.png

Step 2 - Segment your image

[19]:

# Cellpose
cp_mask = cp_seg.segment(cp_seg_prep)
plotter.plot_mask(cp_mask, cp_seg_prep, cp_seg_prep_p, output_path, output_file_prefix);

../_images/notebooks_polarityjam-notebook_seg_11_0.png

[20]:

# Deepcell
dc_mask = dc_seg.segment(dc_seg_prep)
plotter.plot_mask(dc_mask, dc_seg_prep, dc_seg_prep_p, output_path, output_file_prefix);

17:10:11 INFO

Solution credits:

Noah F. Greenwald and Geneva Miller and Erick Moen and Alex Kong and Adam Kagel and Thomas Dougherty and Christine Camacho Fullaway and Brianna J. McIntosh and Ke Xuan Leow and Morgan Sarah Schwartz and Cole Pavelchek and Sunny Cui and Isabella Camplisson and Omer Bar-Tal and Jaiveer Singh and Mara Fong and Gautam Chaudhry and Zion Abraham and Jackson Moseley and Shiri Warshawsky and Erin Soon and Shirley Greenbaum and Tyler Risom and Travis Hollmann and Sean C. Bendall and Leeat Keren and William Graf and Michael Angelo and David Van Valen; Whole-cell segmentation of tissue images with human-level performance using large-scale data annotation and deep learning (DOI: https://doi.org/10.1038/s41587-021-01094-0)


17:10:11 INFO    ~ Starting DeepCell-predict
17:10:19 INFO    ~ 2024-10-02 17:10:19.255773: E tensorflow/stream_executor/cuda/cuda_driver.cc:271] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
17:10:19 INFO    ~ 2024-10-02 17:10:19.255844: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (Veiovis): /proc/driver/nvidia/version does not exist
17:10:19 INFO    ~ 2024-10-02 17:10:19.256772: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
17:10:19 INFO    ~ To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
17:10:24 INFO    ~ WARNING:tensorflow:No training configuration found in save file, so the model was *not* compiled. Compile it manually.
17:10:24 INFO    ~ Image resolution the network was trained on: 0.5 microns per pixel
17:10:33 INFO    ~ Recent segmentation saved to: /tmp/tmpth7hcian/segmentation_segmentation
17:10:33 INFO    ~ Finished DeepCell-predict

../_images/notebooks_polarityjam-notebook_seg_12_1.png

[21]:

# microSam
ms_mask = ms_seg.segment(ms_seg_prep)
plotter.plot_mask(ms_mask, ms_seg_prep, ms_seg_prep_p, output_path, output_file_prefix);

17:10:37 INFO    ~ Starting microSAM-predict
17:10:37 INFO    ~ Set MPLBACKEND from module://matplotlib_inline.backend_inline to qtagg
17:10:43 INFO    ~ /home/jpa/PycharmProjects/polarityjam/src/polarityjam/segmentation/lnk/env/1/lib/python3.10/site-packages/segment_anything/build_sam.py:105: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
17:10:43 INFO    ~   state_dict = torch.load(f)
17:11:10 INFO    ~
17:11:18 INFO    ~ Predict masks for point grid prompts:   0%|          | 0/16 [00:00<?, ?it/s]
17:11:25 INFO    ~ Predict masks for point grid prompts:   6%|▋         | 1/16 [00:08<02:03,  8.22s/it]
17:11:32 INFO    ~ Predict masks for point grid prompts:  12%|█▎        | 2/16 [00:15<01:46,  7.57s/it]
17:11:40 INFO    ~ Predict masks for point grid prompts:  19%|█▉        | 3/16 [00:22<01:37,  7.51s/it]
17:11:47 INFO    ~ Predict masks for point grid prompts:  25%|██▌       | 4/16 [00:30<01:29,  7.47s/it]
17:11:55 INFO    ~ Predict masks for point grid prompts:  31%|███▏      | 5/16 [00:37<01:22,  7.54s/it]
17:12:02 INFO    ~ Predict masks for point grid prompts:  38%|███▊      | 6/16 [00:45<01:14,  7.49s/it]
17:12:09 INFO    ~ Predict masks for point grid prompts:  44%|████▍     | 7/16 [00:52<01:06,  7.41s/it]
17:12:16 INFO    ~ Predict masks for point grid prompts:  50%|█████     | 8/16 [00:59<00:57,  7.16s/it]
17:12:24 INFO    ~ Predict masks for point grid prompts:  56%|█████▋    | 9/16 [01:06<00:50,  7.17s/it]
17:12:30 INFO    ~ Predict masks for point grid prompts:  62%|██████▎   | 10/16 [01:14<00:44,  7.39s/it]
17:12:37 INFO    ~ Predict masks for point grid prompts:  69%|██████▉   | 11/16 [01:20<00:35,  7.18s/it]
17:12:43 INFO    ~ Predict masks for point grid prompts:  75%|███████▌  | 12/16 [01:27<00:28,  7.01s/it]
17:12:50 INFO    ~ Predict masks for point grid prompts:  81%|████████▏ | 13/16 [01:33<00:20,  6.81s/it]
17:12:57 INFO    ~ Predict masks for point grid prompts:  88%|████████▊ | 14/16 [01:40<00:13,  6.72s/it]
17:13:05 INFO    ~ Predict masks for point grid prompts:  94%|█████████▍| 15/16 [01:47<00:06,  6.95s/it]
17:13:05 INFO    ~ Predict masks for point grid prompts: 100%|██████████| 16/16 [01:55<00:00,  7.02s/it]
17:13:05 INFO    ~ Predict masks for point grid prompts: 100%|██████████| 16/16 [01:55<00:00,  7.19s/it]
17:13:12 INFO    ~ Finished microSAM-predict

../_images/notebooks_polarityjam-notebook_seg_13_1.png

Segmentation Mode “Nuclei”

Each segmentation algorithm supported in Polarity-JaM has several modi for different segmentation tasks. For example, Cellpose can segment nuclei and cytoplasm (based on the junction channel).

Here, we show how to switch to the “nucleus” mode for all three segmentation algorithms and compare the results.

[22]:

# Cellpose - switch to "nucleus" mode
params_runtime.segmentation_algorithm = "CellposeSegmenter"
cp_seg, _ = load_segmenter(params_runtime, cp_seg_p)

cp_mask_nuc = cp_seg.segment(cp_seg_prep, mode=SegmentationMode.NUCLEUS)
plotter.plot_mask(cp_mask_nuc, cp_seg_prep, cp_seg_prep_p, output_path, output_file_prefix);

../_images/notebooks_polarityjam-notebook_seg_15_0.png

[23]:

# Deepcell - switch to "nucleus" mode
params_runtime.segmentation_algorithm = "DeepCellSegmenter"
dc_seg, _ = load_segmenter(params_runtime)

dc_mask_nuc = dc_seg.segment(dc_seg_prep, mode=SegmentationMode.NUCLEUS)
plotter.plot_mask(dc_mask_nuc, dc_seg_prep, dc_seg_prep_p, output_path, output_file_prefix);

17:13:22 INFO

Solution credits:

Noah F. Greenwald and Geneva Miller and Erick Moen and Alex Kong and Adam Kagel and Thomas Dougherty and Christine Camacho Fullaway and Brianna J. McIntosh and Ke Xuan Leow and Morgan Sarah Schwartz and Cole Pavelchek and Sunny Cui and Isabella Camplisson and Omer Bar-Tal and Jaiveer Singh and Mara Fong and Gautam Chaudhry and Zion Abraham and Jackson Moseley and Shiri Warshawsky and Erin Soon and Shirley Greenbaum and Tyler Risom and Travis Hollmann and Sean C. Bendall and Leeat Keren and William Graf and Michael Angelo and David Van Valen; Whole-cell segmentation of tissue images with human-level performance using large-scale data annotation and deep learning (DOI: https://doi.org/10.1038/s41587-021-01094-0)


17:13:22 INFO    ~ Starting DeepCell-predict
17:13:30 INFO    ~ 2024-10-02 17:13:30.084588: E tensorflow/stream_executor/cuda/cuda_driver.cc:271] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
17:13:30 INFO    ~ 2024-10-02 17:13:30.084618: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (Veiovis): /proc/driver/nvidia/version does not exist
17:13:30 INFO    ~ 2024-10-02 17:13:30.085022: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
17:13:30 INFO    ~ To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
17:13:35 INFO    ~ WARNING:tensorflow:No training configuration found in save file, so the model was *not* compiled. Compile it manually.
17:13:35 INFO    ~ Image resolution the network was trained on: 0.5 microns per pixel
17:14:16 INFO    ~ Recent segmentation saved to: /tmp/tmpi7k83gg_/segmentation_segmentation
17:14:16 INFO    ~ Finished DeepCell-predict

../_images/notebooks_polarityjam-notebook_seg_16_1.png

[24]:

# microSam - switch to "nucleus" mode
params_runtime.segmentation_algorithm = "MicrosamSegmenter"
ms_seg, _ = load_segmenter(params_runtime)

ms_mask_nuc = ms_seg.segment(ms_seg_prep, mode=SegmentationMode.NUCLEUS)
plotter.plot_mask(ms_mask_nuc, ms_seg_prep, ms_seg_prep_p, output_path, output_file_prefix);

17:14:20 INFO    ~ Starting microSAM-predict
17:14:20 INFO    ~ Set MPLBACKEND from module://matplotlib_inline.backend_inline to qtagg
17:14:26 INFO    ~ /home/jpa/PycharmProjects/polarityjam/src/polarityjam/segmentation/lnk/env/1/lib/python3.10/site-packages/segment_anything/build_sam.py:105: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
17:14:26 INFO    ~   state_dict = torch.load(f)
17:14:53 INFO    ~
17:15:00 INFO    ~ Predict masks for point grid prompts:   0%|          | 0/16 [00:00<?, ?it/s]
17:15:07 INFO    ~ Predict masks for point grid prompts:   6%|▋         | 1/16 [00:07<01:49,  7.27s/it]
17:15:14 INFO    ~ Predict masks for point grid prompts:  12%|█▎        | 2/16 [00:14<01:39,  7.10s/it]
17:15:21 INFO    ~ Predict masks for point grid prompts:  19%|█▉        | 3/16 [00:21<01:32,  7.12s/it]
17:15:28 INFO    ~ Predict masks for point grid prompts:  25%|██▌       | 4/16 [00:28<01:24,  7.03s/it]
17:15:35 INFO    ~ Predict masks for point grid prompts:  31%|███▏      | 5/16 [00:35<01:17,  7.01s/it]
17:15:42 INFO    ~ Predict masks for point grid prompts:  38%|███▊      | 6/16 [00:42<01:10,  7.05s/it]
17:15:49 INFO    ~ Predict masks for point grid prompts:  44%|████▍     | 7/16 [00:49<01:02,  6.96s/it]
17:15:56 INFO    ~ Predict masks for point grid prompts:  50%|█████     | 8/16 [00:56<00:56,  7.00s/it]
17:16:03 INFO    ~ Predict masks for point grid prompts:  56%|█████▋    | 9/16 [01:03<00:49,  7.10s/it]
17:16:10 INFO    ~ Predict masks for point grid prompts:  62%|██████▎   | 10/16 [01:10<00:42,  7.07s/it]
17:16:18 INFO    ~ Predict masks for point grid prompts:  69%|██████▉   | 11/16 [01:17<00:35,  7.06s/it]
17:16:25 INFO    ~ Predict masks for point grid prompts:  75%|███████▌  | 12/16 [01:25<00:29,  7.31s/it]
17:16:32 INFO    ~ Predict masks for point grid prompts:  81%|████████▏ | 13/16 [01:32<00:21,  7.14s/it]
17:16:40 INFO    ~ Predict masks for point grid prompts:  88%|████████▊ | 14/16 [01:39<00:14,  7.19s/it]
17:16:46 INFO    ~ Predict masks for point grid prompts:  94%|█████████▍| 15/16 [01:47<00:07,  7.34s/it]
17:16:46 INFO    ~ Predict masks for point grid prompts: 100%|██████████| 16/16 [01:53<00:00,  6.99s/it]
17:16:46 INFO    ~ Predict masks for point grid prompts: 100%|██████████| 16/16 [01:53<00:00,  7.09s/it]
17:16:54 INFO    ~ Finished microSAM-predict

../_images/notebooks_polarityjam-notebook_seg_17_1.png

Segmentation Mode “Organelle”

Currently, only microSam supports the “organelle” segmentation mode. As the models focus is not the segmentation of nuclei, results should be taken with a grain of salt. Nevertheless, here we show how to use it.

[25]:

# microSam - switch to "organelle" mode
params_runtime.segmentation_algorithm = "MicrosamSegmenter"

# alter default parameters to enhance segmentation output
ms_seg_p = SegmentationParameter(params_runtime.segmentation_algorithm)
ms_seg_p.pred_iou_thresh = 0.85

# load segmenter with altered parameters
ms_seg, _ = load_segmenter(params_runtime, ms_seg_p)

ms_mask_nuc = ms_seg.segment(ms_seg_prep, mode=SegmentationMode.ORGANELLE)
plotter.plot_mask(ms_mask_nuc, ms_seg_prep, ms_seg_prep_p, output_path, output_file_prefix);

17:16:56 INFO    ~ Starting microSAM-predict
17:16:56 INFO    ~ Set MPLBACKEND from module://matplotlib_inline.backend_inline to qtagg
17:17:02 INFO    ~ /home/jpa/PycharmProjects/polarityjam/src/polarityjam/segmentation/lnk/env/1/lib/python3.10/site-packages/segment_anything/build_sam.py:105: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
17:17:02 INFO    ~   state_dict = torch.load(f)
17:17:25 INFO    ~
17:17:31 INFO    ~ Predict masks for point grid prompts:   0%|          | 0/16 [00:00<?, ?it/s]
17:17:37 INFO    ~ Predict masks for point grid prompts:   6%|▋         | 1/16 [00:06<01:36,  6.41s/it]
17:17:42 INFO    ~ Predict masks for point grid prompts:  12%|█▎        | 2/16 [00:12<01:27,  6.27s/it]
17:17:49 INFO    ~ Predict masks for point grid prompts:  19%|█▉        | 3/16 [00:17<01:14,  5.71s/it]
17:17:54 INFO    ~ Predict masks for point grid prompts:  25%|██▌       | 4/16 [00:24<01:11,  5.99s/it]
17:18:00 INFO    ~ Predict masks for point grid prompts:  31%|███▏      | 5/16 [00:29<01:03,  5.79s/it]
17:18:06 INFO    ~ Predict masks for point grid prompts:  38%|███▊      | 6/16 [00:35<00:58,  5.85s/it]
17:18:12 INFO    ~ Predict masks for point grid prompts:  44%|████▍     | 7/16 [00:41<00:53,  5.89s/it]
17:18:18 INFO    ~ Predict masks for point grid prompts:  50%|█████     | 8/16 [00:47<00:47,  5.91s/it]
17:18:24 INFO    ~ Predict masks for point grid prompts:  56%|█████▋    | 9/16 [00:53<00:41,  5.89s/it]
17:18:29 INFO    ~ Predict masks for point grid prompts:  62%|██████▎   | 10/16 [00:59<00:35,  5.86s/it]
17:18:36 INFO    ~ Predict masks for point grid prompts:  69%|██████▉   | 11/16 [01:04<00:29,  5.87s/it]
17:18:42 INFO    ~ Predict masks for point grid prompts:  75%|███████▌  | 12/16 [01:11<00:24,  6.11s/it]
17:18:49 INFO    ~ Predict masks for point grid prompts:  81%|████████▏ | 13/16 [01:17<00:18,  6.13s/it]
17:18:55 INFO    ~ Predict masks for point grid prompts:  88%|████████▊ | 14/16 [01:24<00:12,  6.24s/it]
17:19:02 INFO    ~ Predict masks for point grid prompts:  94%|█████████▍| 15/16 [01:30<00:06,  6.37s/it]
17:19:02 INFO    ~ Predict masks for point grid prompts: 100%|██████████| 16/16 [01:37<00:00,  6.35s/it]
17:19:02 INFO    ~ Predict masks for point grid prompts: 100%|██████████| 16/16 [01:37<00:00,  6.08s/it]
17:19:05 INFO    ~ Finished microSAM-predict

../_images/notebooks_polarityjam-notebook_seg_19_1.png