DeepFeatureExtractor

tiatoolbox . models . engine . deep_feature_extractor . DeepFeatureExtractor

class DeepFeatureExtractor(model, batch_size=8, num_workers=0, weights=None, *, device='cpu', verbose=True)

Generic deep feature extractor for digital pathology images.

This class extends PatchPredictor to extract deep features from whole slide images (WSIs) or image patches using a deep learning model. It is designed for use cases where the goal is to obtain intermediate feature representations (e.g., embeddings) rather than final classification or segmentation outputs.

The extracted features are returned or saved in Zarr format for downstream analysis, such as clustering, visualization, or training other machine learning models.

Parameters:

• model (str | ModelABC) – A PyTorch model instance or the name of a pretrained model from TIAToolbox.

• batch_size (int) – Number of image patches processed per forward pass. Default is 8.

• num_workers (int) – Number of workers for data loading. Default is 0.

• weights (str | Path | None) – Path to model weights. If None, default weights are used.

• device (str) – Device to run the model on (e.g., “cpu”, “cuda”). Default is “cpu”.

• verbose (bool) – Whether to enable verbose logging. Default is True.

process_prediction_per_batch

Flag to control whether predictions are processed per batch. Default is False.

Type:

bool

Initialize DeepFeatureExtractor.

Parameters:

• model (str | ModelABC) – A PyTorch model instance or the name of a pretrained model from TIAToolbox. If a string is provided, the corresponding pretrained weights will be downloaded unless overridden via weights.

• batch_size (int) – Number of image patches processed per forward pass. Default is 8.

• num_workers (int) – Number of workers for data loading. Default is 0.

• weights (str | Path | None) – Path to model weights. If None, default weights are used.

• device (str) – Device to run the model on (e.g., “cpu”, “cuda”). Default is “cpu”.

• verbose (bool) – Whether to enable verbose logging. Default is True.

Methods

infer_wsi	Perform model inference on a whole slide image (WSI).
post_process_patches	Post-process raw patch predictions from model inference.
run	Run the DeepFeatureExtractor engine on input images.
save_predictions	Save patch-level feature predictions to disk or return them in memory.

infer_wsi(dataloader, save_path, **kwargs)

Perform model inference on a whole slide image (WSI).

This method processes a WSI using the provided DataLoader and extracts deep features from each patch using the model. It supports memory-aware caching by spilling intermediate results to disk when memory usage exceeds a specified threshold. The final output includes feature maps and their corresponding spatial coordinates.

Parameters:

• dataloader (DataLoader) – PyTorch DataLoader configured for WSI processing.

• save_path (Path) – Path to save intermediate Zarr output. Used for caching.

• **kwargs (PredictorRunParams) –

  Additional runtime parameters to configure prediction.

  Optional Keys:

  auto_get_mask (bool):

  Automatically generate segmentation masks using wsireader.tissue_mask() during processing.

  batch_size (int):

  Number of image patches per forward pass.

  class_dict (dict):

  Mapping of classification outputs to class names.

  device (str):

  Device to run the model on (e.g., “cpu”, “cuda”).

  labels (list):

  Optional labels for input images. Only a single label per image is supported.

  memory_threshold (int):

  Memory usage threshold (percentage) to trigger caching behavior.

  num_workers (int):

  Number of workers for DataLoader and post-processing.

  output_file (str):

  Filename for saving output (e.g., “.zarr” or “.db”).

  return_labels (bool):

  Whether to return labels with predictions.

  return_probabilities (bool):

  Whether to return per-class probabilities in the output. If False, only predicted labels are returned.

  scale_factor (tuple[float, float]):

  Scale factor for annotations (model_mpp / slide_mpp). Used to convert coordinates to baseline resolution.

  stride_shape (tuple[int, int]):

  Stride used during WSI processing. Defaults to patch_input_shape if not provided.

  verbose (bool):

  Whether to enable verbose logging.

• self (DeepFeatureExtractor)

Returns:

Dictionary containing: - “probabilities”: Extracted feature maps from the model. - “coordinates”: Patch coordinates corresponding to the features.

Return type:

dict[str, dask.array.Array]

post_process_patches(raw_predictions, **kwargs)

Post-process raw patch predictions from model inference.

This method overrides the base implementation to return raw feature maps without applying any additional processing. It is intended for use cases where intermediate features are required as output.

Parameters:

• raw_predictions (dask.array.Array) – Raw model predictions as a Dask array.

• **kwargs (PredictorRunParams) –

  Additional runtime parameters to configure prediction.

  Optional Keys:

  auto_get_mask (bool):

  Automatically generate segmentation masks using wsireader.tissue_mask() during processing.

  batch_size (int):

  Number of image patches per forward pass.

  class_dict (dict):

  Mapping of classification outputs to class names.

  device (str):

  Device to run the model on (e.g., “cpu”, “cuda”).

  labels (list):

  Optional labels for input images. Only a single label per image is supported.

  memory_threshold (int):

  Memory usage threshold (percentage) to trigger caching behavior.

  num_workers (int):

  Number of workers for DataLoader and post-processing.

  output_file (str):

  Filename for saving output (e.g., “.zarr” or “.db”).

  return_labels (bool):

  Whether to return labels with predictions.

  return_probabilities (bool):

  Whether to return per-class probabilities in the output. If False, only predicted labels are returned.

  scale_factor (tuple[float, float]):

  Scale factor for annotations (model_mpp / slide_mpp). Used to convert coordinates to baseline resolution.

  stride_shape (tuple[int, int]):

  Stride used during WSI processing. Defaults to patch_input_shape if not provided.

  verbose (bool):

  Whether to enable verbose logging.

• self (DeepFeatureExtractor)

Returns:

Unmodified raw predictions.

Return type:

dask.array.Array

run(images, *, masks=None, input_resolutions=None, patch_input_shape=None, ioconfig=None, patch_mode=True, save_dir=None, overwrite=False, output_type='dict', **kwargs)

Run the DeepFeatureExtractor engine on input images.

This method orchestrates the full inference pipeline, including preprocessing, model inference, and saving of extracted deep features. It supports both patch-level and whole slide image (WSI) modes. The output is returned or saved in Zarr format.

Note

The return_probabilities flag is always set to True for this engine, as it is designed to extract intermediate feature maps.

Parameters:

• images (list[PathLike | WSIReader] | np.ndarray) – Input images or patches. Can be a list of file paths, WSIReader objects, or a NumPy array of image patches.

• masks (list[PathLike] | np.ndarray | None) – Optional masks for WSI processing. Only used when patch_mode is False.

• input_resolutions (list[dict[Units, Resolution]] | None) – Resolution settings for input heads. Supported units are level, power, and mpp. Keys should be “units” and “resolution”, e.g., [{“units”: “mpp”, “resolution”: 0.25}]. See WSIReader for details.

• patch_input_shape (IntPair | None) – Shape of input patches (height, width), requested at read resolution. Must be positive.

• ioconfig (IOPatchPredictorConfig | None) – IO configuration for patch extraction and resolution.

• patch_mode (bool) – Whether to treat input as patches (True) or WSIs (False). Default is True.

• save_dir (PathLike | None) – Directory to save output files. Required for WSI mode.

• overwrite (bool) – Whether to overwrite existing output files. Default is False.

• output_type (str) – Desired output format. Must be “zarr” or “dict”.

• **kwargs (PredictorRunParams) –

  Additional runtime parameters to configure prediction.

  Optional Keys:

  auto_get_mask (bool):

  Automatically generate segmentation masks using wsireader.tissue_mask() during processing.

  batch_size (int):

  Number of image patches per forward pass.

  class_dict (dict):

  Mapping of classification outputs to class names.

  device (str):

  Device to run the model on (e.g., “cpu”, “cuda”).

  labels (list):

  Optional labels for input images. Only a single label per image is supported.

  memory_threshold (int):

  Memory usage threshold (percentage) to trigger caching behavior.

  num_workers (int):

  Number of workers for DataLoader and post-processing.

  output_file (str):

  Filename for saving output (e.g., “.zarr” or “.db”).

  return_labels (bool):

  Whether to return labels with predictions.

  return_probabilities (bool):

  Whether to return per-class probabilities in the output. If False, only predicted labels are returned.

  scale_factor (tuple[float, float]):

  Scale factor for annotations (model_mpp / slide_mpp). Used to convert coordinates to baseline resolution.

  stride_shape (tuple[int, int]):

  Stride used during WSI processing. Defaults to patch_input_shape if not provided.

  verbose (bool):

  Whether to enable verbose logging.

• self (DeepFeatureExtractor)

Returns:

• If patch_mode is True: returns predictions or path to saved output.

• If patch_mode is False: returns a dictionary mapping each WSI to its output path.

Return type:

AnnotationStore | Path | str | dict | list[Path]

Raises:

ValueError – If output_type is not “zarr” or “dict”.

save_predictions(processed_predictions, output_type, save_path=None, **kwargs)

Save patch-level feature predictions to disk or return them in memory.

This method saves the extracted deep features in the specified output format. Only the “zarr” format is supported for this engine. The method disables saving the “predictions” key, as it is not relevant for feature extraction.

Parameters:

• processed_predictions (dict) – Dictionary containing processed model outputs.

• output_type (str) – Desired output format. Must be “zarr”.

• save_path (Path | None) – Path to save the output file. Required for “zarr” format.

• **kwargs (PredictorRunParams) –

  Additional runtime parameters to configure prediction.

  Optional Keys:

  auto_get_mask (bool):

  Automatically generate segmentation masks using wsireader.tissue_mask() during processing.

  batch_size (int):

  Number of image patches per forward pass.

  class_dict (dict):

  Mapping of classification outputs to class names.

  device (str):

  Device to run the model on (e.g., “cpu”, “cuda”).

  labels (list):

  Optional labels for input images. Only a single label per image is supported.

  memory_threshold (int):

  Memory usage threshold (percentage) to trigger caching behavior.

  num_workers (int):

  Number of workers for DataLoader and post-processing.

  output_file (str):

  Filename for saving output (e.g., “.zarr” or “.db”).

  return_labels (bool):

  Whether to return labels with predictions.

  return_probabilities (bool):

  Whether to return per-class probabilities in the output. If False, only predicted labels are returned.

  scale_factor (tuple[float, float]):

  Scale factor for annotations (model_mpp / slide_mpp). Used to convert coordinates to baseline resolution.

  stride_shape (tuple[int, int]):

  Stride used during WSI processing. Defaults to patch_input_shape if not provided.

  verbose (bool):

  Whether to enable verbose logging.

• self (DeepFeatureExtractor)

Returns:

• If output_type is “zarr”: returns the path to the saved Zarr file.

• If output_type is “dict”: returns predictions as a dictionary.

Return type:

dict | Path

Raises:

ValueError – If an unsupported output format is provided.