Image Explainable AI

Installation

To install Image Explainable AI package:

$ pip install eazyml-image-xai

Available APIs

This package focuses on segmentation prediction, explainability, active learning and online learning for image dataset.

Active learning focuses on reducing the amount of labeled data required to train the model while maximizing performance, making it particularly useful when labeling data is expensive or time-consuming. By prioritizing uncertain or diverse examples, active learning accelerates model improvement and enhances efficiency.

Online learning is a machine learning approach where models are trained incrementally as data becomes available, rather than using a fixed, pre-existing dataset. This method is well-suited for dynamic environments, enabling real-time updates and adaptability to new patterns or changes in data streams.

ez_init(access_key: str | None = None, usage_share_consent: bool | None = None, usage_delete: bool = False)

Initialize EazyML package by passing access_key

Args:

access_key (str): The access key to be set as an environment variable for EazyML.
usage_share_consent (bool): User’s agreement to allow their usage information to be shared. If consent is given, only OS information, Python version, and EazyML packages API call counts are collected.

Returns:

A dictionary containing the results of the initialization process with the following fields:

success (bool): Indicates whether the operation was successful.
message (str): A message describing the success or failure of the operation.

Example:

from eazyml import ez_init

# Initialize the EazyML library with the access key.
# This sets the `EAZYML_ACCESS_KEY` environment variable
access_key = "your_access_key_here"  # Replace with your actual access key
_ = ez_init(access_key)

Notes:

Make sure to call this function before using other functionalities of the EazyML library that require a valid access key.
The access key will be stored in the environment, and other functions in EazyML will automatically use it when required.

ez_xai_image_explain(filename, model_path, predicted_filename, options=None)

This API provides confidence scores and image explanations for model predictions. It can process a single image or multiple images, returning explanations for all predictions.

Parameters :

filename (str | list[str]): Absolute path(s) of the image(s).
model_path (str): Absolute path to the model used for predictions.
predicted_filename (str | list[str]): Absolute path(s) of the predicted output(s).
options (dict): Configuration for explaining predictions. Example:

options = {
    "training_data_path": "...",
    "score_strategy": "weighted-moments",
    "xai_strategy": "gradcam",
    "xai_image_path": "...",
    "gradcam_layer": "layer_name",
    "model_num": "1",
    "required_functions": {...}
}

Key Fields:
- training_data_path: Path to training data (absolute path; .csv format; columns: inputs, labels).
- score_strategy (default: “weighted-moments”): Strategy for explainability score calculation.
- xai_strategy (default: “gradcam”): Strategy for image explanation (e.g., “gradcam”, “image-lime”).
- xai_image_path: Filename(s) for saving the explainability images.
- gradcam_layer: Layer name for gradcam-based explanation.
- model_num: Model ID if score_strategy is “trainable-*”.

Returns :

Dictionary with Fields :

success (bool): Indicates if the explanation generation was successful.
message (str): Describes the success or failure of the operation.
explanations (list, optional): The generated explanations (if successful) contains the explanation string or a local importance dataframe.
confidence (float): Explainability score for the prediction.

{
    "success": <True|False>,
    "explanations": {
        "explanation": "path/to/explainability/image",
        "confidence": 95.4
    }
}

Example:

ez_xai_image_explain(
    filename='path/to/image/image',
    model_path='path/to/model.h5',
    predicted_filename='path/to/prediction.csv/npy',
    options = {"score_strategy": "weighted-moments"}
)

ez_image_active_learning(filenames, model_path, predicted_filenames, options={})

This API sorts test images based on explainability scores for the model’s predictions. If a “query count” is specified in the options, it returns the indices and corresponding scores for that number of inputs.

Parameters :

filenames (list[str]):
List of absolute paths to the images for labeling (querying).
model_path (str)
Path to the model. Must be an absolute path. Supported formats: .h5, .pth.
predicted_filenames (list[str]):
List of absolute paths to the model’s predictions. Must correspond to filenames. Supported formats: .csv, .npy.
options (dict):
Configuration for obtaining query information.

options = {
    "query_count": 10,
    "training_data_path": "path/to/training/data.csv",
    "score_strategy": "weighted-moments",
    "al_strategy": "pool-based",
    "xai_strategy": "gradcam",
    "gradcam_layer": "layer_name",
    "model_num": "1"
}

Key Fields:

query_count: Number of test images to select for labeling.
training_data_path: Path to training data (absolute path; .csv format; columns: inputs, labels).
score_strategy (default: “weighted-moments”): Strategy for explainability scores.
al_strategy: Active Learning strategy (e.g., “pool-based”).
xai_strategy (default: “gradcam”): Strategy for image explanation. Required for trainable-{1|2} in score_strategy.
gradcam_layer (default: layers[-2].name): Layer name for gradcam if score_strategy is trainable-{1|2}.
model_num (default: “1”): Model ID if score_strategy is trainable-*.

Returns :

Dictionary with Fields:

success (bool): Indicates success (True) or failure (False) of the API call.

query_info (dict): Dictionary containing:

query_indices (list): List of indices of test images selected for labeling.

query_scores (list): Corresponding explainability scores for the selected inputs.
{
    "success": <True|False>,
    "query_info": {
        "query_indices": [...],
        "query_scores": [...]
    }
}

Example:

ez_image_active_learning(
    filenames='classification',
    model_path='target',
    predicted_filenames='train.csv',
    options={"data_source": "parquet", "record_number": [1, 2, 3]}
)

ez_image_online_learning(new_training_data_path, model_path, options=None)

This API updates a given model using new training data and saves the updated model. The update process adapts based on the Online Learning strategy or optimizes performance on provided validation data.

Parameters :

new_training_data_path (str):
Path to the new data for training. Must be an absolute path. Supported format: .csv. Expected columns: “inputs”, “labels”.
model_path (str) :
Path to the model. Must be an absolute path. Supported formats: .h5, .pth.
options (dict) :
Configuration for training the model. Example:

options = {
    "training_parameters": {...},
    "required_functions": {
        "input_preprocess_fn": "pre-processing function",
        "label_preprocess_fn: "label pre-processing function",
        "output_process_fn": "Function to process outputs before applying the loss function.",
        "loss_fn": "tensorflow loss function or custom loss function",
        "metric_fns": {
            "metric_iou" : "first metric function",
            "metric_fscore" : "second metric function"
        }
    },
    "training_data_path": "...",
    "validation_data_path": "...",
    "new_model_path": "...",
    "ol_strategy": "fine-tuning",
    "tr_strategy": "normal",
    "compiled": True,
    "log_file": "..."
}

Key Fields:

“training_parameters”: Dictionary with training settings, e.g., “learning_rate”, “loss”, “metric”, “epochs”, etc.
“training_data_path”: Path to previously used training data (required for “joint-training” strategy).
“validation_data_path”: Path to validation data for performance optimization.
“new_model_path”: Path to save the updated model.
“ol_strategy”: Online learning strategy (“joint-training”, “fine-tuning”, or “knowledge-distillation”; default: “fine-tuning”).
“tr_strategy”: Training strategy (“normal”, “transfer-learning”, “dropouts”; default: “whole-model”).

Required Functions:

“loss_fn”: Loss function to train the model.
“metric_fns”: Dictionary of metric functions for evaluation.
“input_preprocess_fn”: Function to preprocess inputs.
“label_preprocess_fn”: Function to preprocess labels.
“output_process_fn”: Function to process outputs before applying the loss function.

Returns :

Dictionary with Fields :
- success (bool): Indicates success (True) or failure (False) of the API call.
- training_info (dict): Contains the updated model path and training history, including losses and metrics for TensorFlow or PyTorch models.

{
    "success": <True|False>,
    "training_info": {
        "model": "updated_model_path",
        "history": {
            # Training history details
        }
    }
}

Example:

ez_image_online_learning(
    training_data_path='classification',
    model_path='target',
    options={
        "training_parameters": {
            "batchsize": 4,
            "epochs": 2,
            "learning_rate": 1e-4
            },
        "ol_strategy": "fine-tuning",
        "tr_strategy": "normal",
        "validation_data_path": "validation/data/path.csv",
        "new_model_path": "new/model/path.h5",
        "required_functions": {
            "input_preprocess_fn": "pre-processing function",
            "label_preprocess_fn: "label pre-processing function",
            "output_process_fn": "Function to process outputs before applying the loss function.",
            "loss_fn": "tensorflow loss function or custom loss function",
            "metric_fns": {
                "metric_iou" : "first metric function",
                "metric_fscore" : "second metric function"
                }
            },
        "log_file": "path/to/log_file.csv"
)

ez_image_model_evaluate(validation_data_path, model_path, options=None)

This API validates a model using provided data and returns the model evaluation.

Parameters :

validation_data_path (str):
Path to the new data for validation. Must be an absolute path. Supported format: .csv. Expected columns: “inputs”, “labels”.
model_path (str):
Path to the model. Supported formats: .h5, .pth. Must be an absolute path.
options (dict):
Configuration for model training. Example:

options = {
    "required_functions": {
                        "loss_fn": '...',
                        "metric_fns": '...',
                        "input_preprocess_fn": '',
                        "label_preprocess_fn": '',
                        "output_process_fn": ''
                        },
    "batch_size": 32,
    "log_file": "path/to/log/file"
}

Required Functions:
- “loss_fn”: Loss function to train the model.
- “metric_fns”: Dictionary of metric functions for evaluation.
- “input_preprocess_fn”: Function to preprocess inputs.
- “label_preprocess_fn”: Function to preprocess labels.
- “output_process_fn”: Function to process outputs before applying the loss function.

Returns :

Dictionary with Fields:
- success (bool): True if the call is successful, else False.
- eval_info (str): Contains model evaluation details.

Example:

ez_image_model_evaluate(
    validation_data_path='path/to/image/image',
    model_path='path/to/model.h5',
    options = {
            "batch_size": 4,
            "required_functions": {
                "input_preprocess_fn": "pre-processing function",
                "label_preprocess_fn: "label pre-processing function",
                "output_process_fn": "Function to process outputs before applying the loss function.",
                "loss_fn": "tensorflow loss function or custom loss function",
                "metric_fns": {
                    "metric_iou" : "first metric function",
                    "metric_fscore" : "second metric function"
                    }
                }
            }
)