modeva.TestSuite.compare_accuracy_table#
- TestSuite.compare_accuracy_table(train_dataset: str = 'train', test_dataset: str = 'test', metric: str | Tuple = None)#
Compare predictive performance metrics across multiple models.
This function evaluates and compares the predictive performance of multiple models using specified metrics on both training and test datasets.
- Parameters:
train_dataset (str, default="train") – Specifies the training dataset to evaluate. Options: “main”, “train”, or “test”
test_dataset (str, default="test") – Specifies the test dataset to evaluate. Options: “main”, “train”, or “test”
metric (str or tuple of str, default=None) –
Performance metric(s) to calculate. If None:
For regression: Uses MSE, MAE, and R2
For classification: Uses ACC, AUC, F1, LogLoss, and Brier
- Returns:
A container object with the following components:
key: “compare_accuracy_table”
data: Name of the dataset used
model: List of model names being compared
inputs: Input parameters used for the comparison
value: Dictionary of (“<model_name>”, item), each item is also a nested dictionary with (“<metric_name>”, subitem), where each subitem is also a dictionary with:
”<train_dataset>”: The metric value of training dataset.
”<test_dataset>”: The metric value of testing dataset.
”GAP”: The performance gap is calculated as (test_score - train_score).
table: Pandas DataFrame containing detailed performance metrics
value: Dictionary of (“<model_name>”, item), each item is also a nested dictionary with (“<metric_name>”, subitem), where each subitem is also a dictionary with:
”<train_dataset>”: The metric value of training dataset.
”<test_dataset>”: The metric value of testing dataset.
”GAP”: The performance gap is calculated as (test_score - train_score).
options: Dictionary of visualizations configuration. Run results.plot() to show all plots; Run results.plot(name=xxx) to display one preferred plot; and the following names are available:
”<metric_name>”: a bar plot where x-axis is the model names, and y-axis is performance metric
- Return type:
Examples
