modeva.TestSuite.diagnose_mitigate_unfair_binning

TestSuite.diagnose_mitigate_unfair_binning(group_config, favorable_label: int = 1, dataset: str = 'test', metric: str = None, performance_metric: str = None, binning_features: str | Tuple = None, binning_method: str = 'uniform', bins: int | Dict = 10)

Mitigate model unfairness through feature value binning.

This method attempts to reduce model unfairness by binning feature values, which can help smooth out predictions across different groups.

1. For numerical features:

   {
       "feature": str,           # Feature name
       "protected": {            # Protected group bounds
           "lower": float,       # Lower bound
           "lower_inclusive": bool,
           "upper": float,       # Optional upper bound
           "upper_inclusive": bool
       },
       "reference": {            # Reference group bounds
           "lower": float,       # Optional lower bound
           "lower_inclusive": bool,
           "upper": float,       # Upper bound
           "upper_inclusive": bool
       }
   }
   

2. For categorical features:

   {
       "feature": str,                  # Feature name
       "protected": str or int,         # Protected group category
       "reference": str or int          # Reference group category
   }
   

3. For probabilistic group membership:

   {
       "by_weights": True,
       "protected": str,         # Column name with protected group probabilities
       "reference": str          # Column name with reference group probabilities
   }
   

favorable_label : {0, 1}, default=1

• For classification: The preferred class label.

• For regression: 1 means larger predictions are preferred, 0 means smaller predictions are preferred.

dataset{“main”, “train”, “test”}, default=”test”

Which dataset partition to analyze

metricstr, default=None

Fairness metric to calculate. Higher values indicate less unfairness. If None, defaults are used based on task type.

For regression (default=”SMD”):

• SMD: Standardized Mean Difference (%) between protected and reference groups

For classification (default=”AIR”):

• AIR: Adverse Impact Ratio of predicted probabilities

• PR: Precision Ratio

• RR: Recall Ratio

performance_metricstr, default=None

Model performance metric to use.

• For classification (default=”AUC”): “ACC”, “AUC”, “F1”, “LogLoss”, and “Brier”

• For regression (default=”MSE”): “MSE”, “MAE”, and “R2”

binning_featuresstr or tuple, default=None

Features to apply binning to. If None, bins all features.

binning_method{“uniform”, “quantile”, “auto-xgb1”, “precompute”}, default=”uniform”

Method for binning numerical features:

• “uniform”: Equal-width bins

• “quantile”: Equal-frequency bins

• “auto-xgb1”: XGBoost prebinning

• “precompute”: Use pre-specified bin edges from bins parameter

binsint or dict, default=10

For int: Number of bins for numerical features; For dict: Pre-computed bin edges for each feature when binning_method=”precompute”. Example: {“feature1”: [0.1, 0.5, 0.9], “feature2”: [0.3, 0.7]}

Returns:

Contains:

• key: “diagnose_mitigate_unfair_binning”

• data: Name of the dataset used

• model: Name of the model used

• inputs: Input parameters used for the test

• value: Nested dictionary containing the (“<feature_name>”, item) pairs for each feature, and the item is also a dictionary with:

  • ”Performance”: Predictive performance scores after binning each feature

  • ”Fairness”: Fairness scores after binning each feature

• table: dictionary of dataframe with perforamnce and fairness metric scores after binning.

  • ”Fairness”: Fairness scores table

  • ”Performance”: Predictive performance table

• options: Dictionary of visualizations configuration. Run results.plot() to show all plots; Run results.plot(name=xxx) to display one preferred plot; and the following names are available:

  • ”<group_name>”: Line and bar plots visualizing the performance and fairness scores against each binning features.

Return type:

ValidationResult

Examples

Model Fairness Analysis (Classification)

Model Fairness Analysis (Classification)