EazyML Modeling

Installation

To install EazyML Modeling package:

pip

$ pip install eazyml-automl

Available APIs

This API allows users to build machine learning models.

ez_init(access_key: str | None = None, usage_share_consent: bool | None = None, usage_delete: bool = False)

Initialize EazyML package by passing access_key

Args:

• access_key (str): The access key to be set as an environment variable for EazyML.

• usage_share_consent (bool): User’s agreement to allow their usage information to be shared. If consent is given, only OS information, Python version, and EazyML packages API call counts are collected.

Returns:

A dictionary containing the results of the initialization process with the following fields:

• success (bool): Indicates whether the operation was successful.

• message (str): A message describing the success or failure of the operation.

Example:

from eazyml import ez_init

# Initialize the EazyML library with the access key.
# This sets the `EAZYML_ACCESS_KEY` environment variable
access_key = "your_access_key_here"  # Replace with your actual access key
_ = ez_init(access_key)

Notes:

• Make sure to call this function before using other functionalities of the EazyML library that require a valid access key.

• The access key will be stored in the environment, and other functions in EazyML will automatically use it when required.

ez_build_model(train_data, outcome, options={})

Initialize and build a predictive model based on the provided dataset and options.

Args:

• train_data (DataFrame or str): A pandas DataFrame containing the dataset for model initialization. Alternatively, you can provide the file path of the dataset (as a string).

• outcome (str): The target variable for the model.

• options (dict, optional): A dictionary of options to configure the model initialization process. If not provided, the function will use default settings. Supported keys include:

  • model_type (str, optional): Specifies the type of model to build. The supported value is “predictive”.

  • spark_session (SparkSession or None, optional): If a Spark session is provided, distributed computation will be used. If None, standard computation is used.

Returns:

A dictionary containing the results of the model building process with the following fields:

• success (bool): Indicates whether the model was successfully trained.

• message (str): A message describing the success or failure of the operation.

On Success:

• model_performance (DataFrame): A DataFrame providing the performance metrics of the trained model(s).

• global_importance (DataFrame): A DataFrame containing the feature importance scores.

• features_selected (DataFrame): A DataFrame containing the features selected.

• model_info (Bytes): Encrypted model information that will be used by ez_predict for making predictions on test data.

Note:

• Please save the response obtained after building the model and provide the model_info to the ez_predict function for making predictions on test data.

• If you are using a spark_session, save the necessary Spark models separately from the model_info and pass them as spark_model in the options dictionary when calling ez_predict, along with the session and model_info.

• Since Spark models cannot be directly saved in the model_info output, you must manually save the individual models from response[“model_info”][“spark_module”][“Models”][index][“model”] for each index. Use the Pipeline module to save and load the models as needed.

Example:

import pandas as pd
import joblib
from eazyml import ez_build_model

# Load the training data (make sure the file path is correct).
train_file_path = "path_to_your_training_data.csv"  # Replace with the correct file path
train_data = pd.read_csv(train_file_path)

# Define the outcome (target variable) for the model
outcome = "target"  # Replace with your actual target variable name

# Set the options for building the model
build_options = {"model_type": "predictive"}

# Call the eazyml function to build the model
build_response = ez_build_model(train_data, outcome, options=build_options)

# build_response is a dictionary object with following keys.
# print(build_response.keys())
# dict_keys(['success', 'message', 'model_performance', 'global_importance', 'features_selected', 'model_info'])

# Save the response for later use (e.g., for predictions with ez_predict)
build_model_response_path = 'model_response.joblib'
joblib.dump(build_response, build_model_response_path)

ez_predict(test_data, model_info, options={})

Perform predictions on the provided test data using the model parameters generated by ez_build_model.

Parameters:

• test_data (DataFrame or str): The dataset to be evaluated. It must have the same features as the dataset used for training.

• model_info (Bytes): Contains the encrypted or unencrypted details about the trained model and its environment.

• options (dict): A dictionary of configuration options for model initialization and prediction. Supported keys include:

  • model (str, optional): Specifies the model to be used for prediction. If not provided, the default model from model_info is used.

  • confidence_score (bool, optional): Default is False. If True, the function provides a confidence score for classification models.

  • spark_session (SparkSession or None, optional): If provided, a Spark session will be used for distributed computation. If None, standard computation is used.

  • spark_model (model or pipeline, optional): If the model is saved and spark_session is provided, the trained Spark model or pipeline should be loaded and passed here.

Returns:

• dict: A dictionary containing the result of the evaluation. The dictionary contains the following keys:

  • “success” (bool): Indicates whether the operation was successful.

  • “message” (str): A message containing either an error or informational details.

  If successful, the dictionary also contains:

  • “pred_df” (DataFrame): A DataFrame containing the predictions for the test dataset.

Example:

import pandas as pd
import joblib
from eazyml import ez_predict

# Load test data.
test_file_path = "path_to_your_test_data.csv"
test_data = pd.read_csv(test_file_path)

# Load output from ez_build_model. This should be the file where model information is stored.
build_model_response_path = 'model_response.joblib'
build_model_response = joblib.load(build_model_response_path)
model_info = build_model_response["model_info"]

# Choose the model for prediction from the key "model_performance" in the build_model_response object above. The default model is the top-performing model if no value is provided.
pred_options = {"model": "Random Forest with Information Gain"}

# Call the eazyml function to predict
pred_response = ez_predict(test_data, model_info, options=pred_options)

# prediction response is a dictionary object with following keys.
# print(pred_response.keys())
# dict_keys(['success', 'message', 'pred_df'])

ez_select_features(train_data, outcome, options={})

Perform Feature seelction on the provided training data.

Args:

• train_data (DataFrame or str): A pandas DataFrame containing the dataset for feature selection. Alternatively, you can provide the file path of the dataset (as a string).

• outcome (str): The target variable for the model.

• options (dict, optional): A dictionary of options to configure the feature selection process. If not provided, the function will use default settings. Supported keys include:

  • dtypes (dict, optional): Specifies the type of data columns provided. Can only be either numeric or categorical.

Returns:

A dictionary containing the results of the model building process with the following fields:

• success (bool): Indicates whether the feature selection was successfully determined.

• message (str): A message describing the success or failure of the operation.

On Success:

• scores (dict): A dictionary providing the selected features metrics for the data provided.

Example:

import pandas as pd
import joblib
from eazyml import ez_select_features

# Load test data.
train_file_path = "path_to_your_test_data.csv"
train_data = pd.read_csv(train_file_path)

# Define the outcome (target variable) for the model
outcome = "target"  # Replace with your actual target variable name

# options = {"dtypes": {"column1":'numeric', "column2":'categorical'}}

# Call the eazyml function for feature selection
feat_response = ez_select_features(train_data, outcome, options=options)

# select features response is a dictionary object with following keys.
# print(feat_response.keys())
# dict_keys(['success', 'message', 'scores'])

ez_types(train_data, options={})

EazyML Type Inference API.

Returns the type inferred by EazyML on the dataset provided.

Accepts dataset_id.

Args:

• train_data (DataFrame or str): A pandas DataFrame containing the dataset for determining datatypes. Alternatively, you can provide the file path of the dataset (as a string).

• options (dict, optional): A dictionary of options to configure the ez_types process. If not provided, the function will use default settings. Supported keys include:

  • dtypes (dict, optional): Specifies the type of data columns provided. Can only be either numeric or categorical.

Returns:

A dictionary containing the results of the model building process with the following fields:

• success (bool): Indicates whether the types of data was successfully determined.

• message (str): A message describing the success or failure of the operation.

On Success:

• ez_dtypes (dict): A dictionary providing the ez_dtypes for the data provided.

Example:

import pandas as pd
import joblib
from eazyml import ez_types

# Load test data.
train_file_path = "path_to_your_test_data.csv"
train_data = pd.read_csv(train_file_path)

# options = {"dtypes": {"column1":'numeric', "column2":'categorical'}}

# Call the eazyml function for feature selection
dtypes_response = ez_types(train_data, options=options)

# dtypes response is a dictionary object with following keys.
# print(dtypes_response.keys())
# dict_keys(['success', 'message', 'ez_dtypes'])

ez_store_spark_response(build_response, output_dir)

Function to store ez_build spark response

Args:

• build_response (dict): Response object from ez_build for spark.

• output_dir (str): Path to the directory to store the output of the response.

Returns:

A dictionary containing the results of the model building process with the following fields:

• success (bool): Indicates whether the object was successfully saved.

• message (str): A message describing the success or failure of the operation.

On Success:

• output_filepath (str): The path where the response is stored.

Example:

import pandas as pd
from pyspark.sql import SparkSession
from eazyml import ez_store_spark_response

#create new spark session
spark_sess = SparkSession.builder.appName("EazyMLSparkModeling").getOrCreate()

# Load the training data (make sure the file path is correct).
train_file_path = "path_to_your_training_data.csv"  # Replace with the correct file path
train_data = pd.read_csv(train_file_path)

# Define the outcome (target variable) for the model
outcome = "target"  # Replace with your actual target variable name

#build model options for spark.
build_options = {"model_type": "predictive", "spark_session":spark_sess}

# Call the eazyml function to build the model
build_response = ez_build_model(train_data, outcome, options=build_options)


#store the build_response in a Directory.
output_dir = "output_directory_path"
response = ez.ez_store_spark_response(build_response, output_dir)

# response is a dictionary object with following keys.
# print(response.keys())
# dict_keys(['success', 'message', 'output_filepath'])

ez_error_metrics(y_true, y_pred, classification_labels=None, regression=False, n_features=None)

Compute classification or regression metrics.

Args::

• y_true (Series): Ground truth labels or values

• y_pred (Series): Predicted labels or values

• regression (Bool): if True compute regression metrics

• classification_labels (list): List of class labels in order (only used for classification)

• n_features (int): Number of features used in model (for adjusted R², optional)

Returns:

A dictionary containing the results of the model error metrics with the following fields:

• success (bool): Indicates whether the object was successfully saved.

• message (str): A message describing the success or failure of the operation.

On Success:

• error_metrics (str): Error Metrics for the data provided

Example:

import pandas as pd
from eazyml import ez_error_metrics

# prediction response is a dictionary object with following keys from ez_predict.
# print(pred_response.keys())
# dict_keys(['success', 'message', 'pred_df'])

p_df = pred_response["pred_df"]

y_true = p_df[outcome] 
y_pred = p_df[f"Predicted {outcome}"], 
classification_labels=list(p_df[outcome].value_counts().index)

# Call the eazyml function for error metrics
metrics_response = error_metrics(y_true, y_pred, classification_labels)

# dtypes response is a dictionary object with following keys.
# print(metrics_response.keys())
# dict_keys(['success', 'message', 'error_metrics'])

ez_display_json(resp)

Function to display formatted json

ez_display_df(resp)

Function to display formatted dataframe

ez_display_md(resp)

Function to display formatted markdown