modeva.models.MoGAMINetClassifier

class modeva.models.MoGAMINetClassifier(name: str = None, feature_names=None, feature_types=None, interact_num=10, subnet_size_main_effect=(20,), subnet_size_interaction=(20, 20), activation_func='ReLU', max_epochs=(1000, 1000, 1000), learning_rates=(0.001, 0.001, 0.0001), early_stop_thres=('auto', 'auto', 'auto'), batch_size=1000, batch_size_inference=10000, max_iter_per_epoch=100, val_ratio=0.2, warm_start=True, gam_sample_size=5000, mlp_sample_size=1000, heredity=True, reg_clarity=0.1, loss_threshold=0.01, reg_mono=0.1, mono_increasing_list=(), mono_decreasing_list=(), mono_sample_size=1000, include_interaction_list=(), boundary_clip=True, normalize=True, verbose=False, n_jobs=10, device=None, random_state=0)

Generalized additive model with pairwise interaction classifier.

Parameters:

• name (str, default=None) – The name of the model.

• feature_names (list or None, default=None) – The list of feature names. If None, will use “X0”, “X1”, “X2”, etc.

• feature_types (list or None, default=None) – The list of feature types. Available types include “numerical” and “categorical”. If None, will use numerical for all features.

• interact_num (int, default=10) – The max number of interactions to be included in the second stage training.

• subnet_size_main_effect (tuple of int, default=(20, )) – The hidden layer architecture of each subnetwork in the main effect block.

• subnet_size_interaction (tuple of int, default=(20, 20)) – The hidden layer architecture of each subnetwork in the interaction block.

• activation_func ({"ReLU", "Sigmoid", "Tanh"}, default="ReLU") – The name of the activation function.

• max_epochs (tuple of THREE int, default=(1000, 1000, 1000)) – The max number of epochs in the first (main effect training), second (interaction training), and third (fine-tuning) stages, respectively.

• learning_rates (tuple of THREE float, default=(1e-3, 1e-3, 1e-4)) – The initial learning rates of Adam optimizer in the first (main effect training), second (interaction training), and third (fine-tuning) stages, respectively.

• early_stop_thres (tuple of THREE int or "auto", default=["auto", "auto", "auto"]) – The early stopping threshold in the first (main effect training), second (interaction training), and third (fine-tuning) stages, respectively. In auto mode, the value is set to max(5, min(5000 * n_features / (max_iter_per_epoch * batch_size), 100)).

• batch_size (int, default=1000) – The batch size. Note that it should not be larger than the training size * (1 - validation ratio).

• batch_size_inference (int, default=10000) – The batch size used in the inference stage. It is imposed to avoid out-of-memory issue when dealing very large dataset.

• max_iter_per_epoch (int, default=100) – The max number of iterations per epoch. In the init stage of model fit, its value will be clipped by min(max_iter_per_epoch, int(sample_size / batch_size)). For each epoch, the data would be reshuffled and only the first “max_iter_per_epoch” batches would be used for training. It is imposed to make the training scalable for very large dataset.

• val_ratio (float, default=0.2) – The validation ratio, should be greater than 0 and smaller than 1.

• warm_start (bool, default=True) – Initialize the network by fitting a rough LGBM model. The initialization is performed by, 1) fit a LGBM model as teacher model, 2) generate random samples from the teacher model, 3) fit each subnetwork using the generated samples. And it is used for both main effect and interaction subnetwork initialization.

• gam_sample_size (int, default=5000) – The sub-sample size for GAM fitting as warm_start=True.

• mlp_sample_size (int, default=1000) – The generated sample size for individual subnetwork fitting as warm_start=True.

• heredity (bool, default=True) – Whether to perform interaction screening subject to heredity constraint.

• loss_threshold (float, default=0.01) – The loss tolerance threshold for selecting fewer main effects or interactions, according to the validation performance. For instance, assume the best validation performance is achieved when using 10 main effects; if only use the top 5 main effects also gives similar validation performance, we could prune the last 5 by setting this parameter to be positive.

• reg_clarity (float, default=0.1) – The regularization strength of marginal clarity constraint.

• reg_mono (float, default=0.1) – The regularization strength of monotonicity constraint.

• mono_sample_size (int, default=1000) – As monotonicity constraint is used, we would generate some data points uniformly within the feature space per epoch, to impose the monotonicity regularization in addition to original training samples.

• mono_increasing_list (tuple of str, default=()) – The feature name tuple subject to monotonic increasing constraint.

• mono_decreasing_list (tuple of str, default=()) – The feature name tuple subject to monotonic decreasing constraint.

• include_interaction_list (tuple of (str, str), default=()) – The tuple of interaction to be included for fitting, each interaction is expressed by (feature_name1, feature_name2).

• boundary_clip (bool, default=True) – In the inference stage, whether to clip the feature values by their min and max values in the training data.

• normalize (bool, default=True) – Whether to normalize the data before inputting to the network.

• verbose (bool, default=False) – Whether to output the training logs.

• n_jobs (int, default=10) – The number of cpu cores for parallel computing. -1 means all the available cpus will be used.

• device (string, default=None) – The hardware device name used for training.

• random_state (int, default=0) – The random seed.

net_

The fitted GAMI-Net module.

Type:

torch network object

interaction_list_

The list of feature index pairs (tuple) for each fitted interaction.

Type:

list of tuples

active_main_effect_index_

The selected main effect index.

Type:

list of int

active_interaction_index_

The selected interaction index.

Type:

list of int

main_effect_val_loss_

The validation loss as the most important main effects are sequentially added.

Type:

list of float

interaction_val_loss_

The validation loss as the most important interactions are sequentially added.

Type:

list of float

time_cost_

The time cost of each stage.

Type:

list of tuple

n_interactions_

The actual number of interactions used in the fitting stage. It is greater or equal to the number of active interactions.

Type:

int

calibrate_interval(X, y, alpha=0.1)

Fit a conformal prediction model to the given data.

This method computes the model’s prediction interval calibrated to the given data.

It computes the calibration quantile based on predicted probabilities for the positive class.

Parameters:

• X (X : np.ndarray of shape (n_samples, n_features)) – Feature matrix for prediction.

• y (array-like of shape (n_samples, )) – Target values.

• alpha (float, default=0.1) – Expected miscoverage for the conformal prediction.

Raises:

ValueError – If the model is neither a regressor nor a classifier.:

calibrate_proba(X, y, sample_weight=None, method='sigmoid')

Fit the calibration method on the model’s predictions.

Parameters:

• X (np.ndarray of shape (n_samples, n_features)) – Feature matrix for prediction.

• y (np.ndarray of shape (n_samples, )) – Ground truth labels.

• sample_weight (array-like, shape (n_samples,), default=None) – Sample weights.

• method ({'sigmoid', 'isotonic'}, default='sigmoid') –

  The calibration method.

  • ’sigmoid’: Platt’s method, i.e., fit a logistic regression on predicted probabilities and y

  • ’isotonic’: Fit an isotonic regression on predicted probabilities and y.

Returns:

self

Return type:

Calibrated estimator

decision_function(X, calibration: bool = True)

Computes the decision function for the given input data.

Parameters:

• X (np.ndarray of shape (n_samples, n_features)) – Feature matrix for prediction.

• calibration (bool, default=True) – If True, will use calibrated probability if calibration is done. Otherwise, will use raw probability.

Returns:

logit_prediction – Array of (calibrated) logit predictions.

Return type:

array, shape (n_samples,) or (n_samples, n_classes)

fit(X, y, sample_weight=None)

Fit GAMINetClassifier model.

Parameters:

• X (np.ndarray of shape (n_samples, n_features)) – Data features.

• y (np.ndarray of shape (n_samples, )) – Target response.

• sample_weight (np.ndarray of shape (n_samples, )) – Sample weight.

Returns:

self – Fitted Estimator.

Return type:

object

get_params(deep=True)

Get parameters for this estimator.

Parameters:

deep (bool, default=True) – If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

params – Parameter names mapped to their values.

Return type:

dict

load(file_name: str)

Load the model into memory from file system.

Parameters:

file_name (str) – The path and name of the file.

Return type:

estimator object

predict(X, calibration: bool = True)

Model predictions, calling the child class’s ‘_predict’ method.

Parameters:

• X (np.ndarray of shape (n_samples, n_features)) – Feature matrix for prediction.

• calibration (bool, default=True) – If True, will use calibrated probability if calibration is done. Otherwise, will use raw probability.

Returns:

np.ndarray

Return type:

The (calibrated) final prediction

predict_interval(X)

Predict the prediction set for the given data based on the conformal prediction model.

This method computes the model prediction interval (regression) or prediction sets (classification) using conformal prediction.

Parameters:

X (np.ndarray of shape (n_samples, n_features)) – Feature matrix for prediction.

Returns:

np.ndarray – in the format [n_samples, 2] for regressors or a flattened array for classifiers.

Return type:

The lower and upper bounds of the prediction intervals for each sample

Raises:

ValueError – If fit_conformal has not been called to fit the conformal prediction model: before calling this method.

predict_proba(X, calibration: bool = True)

Predict (calibrated) probabilities for X.

Parameters:

• X (np.ndarray of shape (n_samples, n_features)) – Feature matrix for prediction.

• calibration (bool, default=True) – If True, will return calibrated probability if calibration is done. Otherwise, will return raw probability.

Returns:

np.ndarray

Return type:

The (calibrated) predicted probabilities

save(file_name: str)

Save the model into file system.

Parameters:

file_name (str) – The path and name of the file.

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:

**params (dict) – Estimator parameters.

Returns:

self – Estimator instance.

Return type:

estimator instance