`transparentai.fairness`¶

Fairness module¶

transparentai.fairness.fairness.create_privilieged_df(df, privileged_group)[source]¶

Returns a formated dataframe with protected attribute columns and whether the row is privileged (1) or not (0).

example of a privileged_group dictionnary :

>>> privileged_group = {
     'gender':['Male'],                # privileged group is man for gender attribute
     'age': lambda x: x > 30 & x < 55  # privileged group aged between 30 and 55 years old
 }

Parameters:	df (pd.DataFrame) – Dataframe to extract privilieged group from. privileged_group (dict) – Dictionnary with protected attribute as key (e.g. age or gender) and a list of favorable value (like [‘Male’]) or a function returning a boolean corresponding to a privileged group
Returns:	DataFrame with protected attribute columns and whether the row is privileged (1) or not (0)
Return type:	pd.DataFrame
Raises:	TypeError: – df is not a pandas.DataFrame TypeError: – privileged_group is not a dictionnary ValueError: – privileged_group has not valid keys (in df columns)

transparentai.fairness.fairness.compute_fairness_metrics(y_true, y_pred, df, privileged_group, metrics=None, pos_label=1, regr_split=None)[source]¶

Computes the fairness metrics for one attribute

metrics can have str or function. If it’s a string then it has to be a key from FAIRNESS_METRICS global variable dict. By default it uses the 5 fairness function :

statistical_parity_difference
disparate_impact
equal_opportunity_difference
average_odds_difference
theil_index

You can also use it for a regression problem. You can set a value in the regr_split argument so it converts it to a binary classification problem. To use the mean use ‘mean’. If the favorable label is more than the split value set pos_label argument to 1 else to 0.

Example

>>> from transparentai.datasets import load_boston
>>> from sklearn.linear_model import LinearRegression

>>> data = load_boston()
>>> X, y = data.drop(columns='MEDV'), data['MEDV']
>>> regr = LinearRegression().fit(X, y)

>>> privileged_group = {
    'AGE': lambda x: (x > 30) & (x < 55)
}
>>> y_true, y_pred = y, regr.predict(X)
>>> compute_fairness_metrics(y_true, y_pred, data,
                             privileged_group, regr_split='mean')
{'AGE': {'statistical_parity_difference': -0.2041836536594836,
  'disparate_impact': 0.674582301980198,
  'equal_opportunity_difference': 0.018181818181818188,
  'average_odds_difference': -0.0884835589941973,
  'theil_index': 0.06976073748626294}}

Returns a dictionnary with protected attributes name’s as key containing a dictionnary with metric’s name as key and metric function’s result as value

Parameters:	y_true (array like) – True labels y_pred (array like) – Predicted labels df (pd.DataFrame) – Dataframe to extract privilieged group from. privileged_group (dict) – Dictionnary with protected attribute as key (e.g. age or gender) and a list of favorable value (like [‘Male’]) or a function returning a boolean corresponding to a privileged group metrics (list (default None)) – List of metrics to compute, if None then it uses the 5 default Fairness function pos_label (number) – The label of the positive class. regr_split ('mean' or number (default None)) – If it’s a regression problem then you can convert result to a binary classification using ‘mean’ or a choosen number. both y_true and y_pred become 0 and 1 : 0 if it’s equal or less than the split value (the average if ‘mean’) and 1 if more. If the favorable label is more than the split value set pos_label=1 else pos_label=0
Returns:	Dictionnary with protected attributes name’s as key containing a dictionnary with metric’s name as key and metric function’s result as value
Return type:	dict
Raises:	ValueError: – y_true and y_pred must have the same length ValueError: – y_true and df must have the same length TypeError: – metrics must be a list

transparentai.fairness.fairness.model_bias(y_true, y_pred, df, privileged_group, pos_label=1, regr_split=None, returns_text=False)[source]¶

Computes the fairness metrics for protected attributes refered in the privileged_group argument.

It uses the 4 fairness function :

statistical_parity_difference
disparate_impact
equal_opportunity_difference
average_odds_difference

You can also use it for a regression problem. You can set a value in the regr_split argument so it converts it to a binary classification problem. To use the mean use ‘mean’. If the favorable label is more than the split value set pos_label argument to 1 else to 0.

This function is using the fairness.compute_metrics function. So if returns_text is False then it’s the same output.

Example

>>> from transparentai.datasets import load_boston
>>> from sklearn.linear_model import LinearRegression

>>> data = load_boston()
>>> X, y = data.drop(columns='MEDV'), data['MEDV']
>>> regr = LinearRegression().fit(X, y)

>>> privileged_group = {
    'AGE': lambda x: (x > 30) & (x < 55)
}
>>> y_true, y_pred = y, regr.predict(X)
>>> model_bias(y_true, y_pred, data,
               privileged_group, regr_split='mean')
{'AGE': {'statistical_parity_difference': -0.2041836536594836,
  'disparate_impact': 0.674582301980198,
  'equal_opportunity_difference': 0.018181818181818188,
  'average_odds_difference': -0.0884835589941973,
  'theil_index': 0.06976073748626294}}

>>> bias_txt = model_bias(y_true, y_pred, data,
                          privileged_group, regr_split='mean',
                          returns_text=True)
>>> print(bias_txt['AGE'])
The privileged group is predicted with the positive output 20.42% more often than the unprivileged group. This is considered to be not fair.
The privileged group is predicted with the positive output 1.48 times more often than the unprivileged group. This is considered to be not fair.
For a person in the unprivileged group, the model predict a correct positive output 1.82% more often than a person in the privileged group. This is considered to be fair.
For a person in the privileged group, the model predict a correct positive output or a correct negative output 8.85% more often than a person in the unprivileged group. This is considered to be fair.
The model has 2 fair metrics over 4 (50%).

Parameters:	y_true (array like) – True labels y_pred (array like) – Predicted labels df (pd.DataFrame) – Dataframe to extract privilieged group from. privileged_group (dict) – Dictionnary with protected attribute as key (e.g. age or gender) and a list of favorable value (like [‘Male’]) or a function returning a boolean corresponding to a privileged group pos_label (number) – The label of the positive class. regr_split ('mean' or number (default None)) – If it’s a regression problem then you can convert result to a binary classification using ‘mean’ or a choosen number. both y_true and y_pred become 0 and 1 : 0 if it’s equal or less than the split value (the average if ‘mean’) and 1 if more. If the favorable label is more than the split value set pos_label=1 else pos_label=0 returns_text (bool (default False)) – Whether it return computed metrics score or a text explaination for the computed bias.
Returns:	Dictionnary with metric’s name as key and metric function’s result as value if returns_text is False else it returns a text explaining the model fairness over the 4 metrics.
Return type:	dict

transparentai.fairness.fairness.find_correlated_feature(df, privileged_group, corr_threshold=0.4)[source]¶

Finds correlated feature with protected attribute set in the privileged_group argument.

This function is a helper to find out if protected attribute can be found in other features.

Returns a dictionnary with protected attributes name’s as key containing a dictionnary with metric’s name as key and metric function’s result as value.

Example

>>> from transparentai.datasets import load_adult
>>> from transparentai import fairness

>>> data = load_adult()

>>> privileged_group = {
        'gender':['Male'],
        'marital-status': lambda x: 'Married' in x,
        'race':['White']
    }

>>> fairness.find_correlated_feature(data, privileged_group,
                                     corr_threshold=0.4)
{'gender': {'marital-status': 0.4593,
  'occupation': 0.4239,
  'relationship': 0.6465},
 'marital-status': {'relationship': 0.4881,
  'gender': 0.4593,
  'income': 0.4482},
 'race': {'native-country': 0.4006}}

Parameters:	df (pd.DataFrame) – Dataframe to extract privilieged group from. privileged_group (dict) – Dictionnary with protected attribute as key (e.g. age or gender) and a list of favorable value (like [‘Male’]) or a function returning a boolean corresponding to a privileged group corr_threshold (float (default 0.4)) – Threshold for which features are considered to be correlated
Returns:	Dictionnary with protected attributes name’s as key containing a dictionnary with correlated features as key and correlation coeff as value
Return type:	dict

Fairness metrics¶

Statistical parity difference¶

transparentai.fairness.metrics.statistical_parity_difference(y, prot_attr, pos_label=1)[source]¶

Computes the statistical parity difference for a protected attribute and a specified label

Computed as the difference of the rate of favorable outcomes received by the unprivileged group to the privileged group.

The ideal value of this metric is 0 A value < 0 implies higher benefit for the privileged group and a value > 0 implies a higher benefit for the unprivileged group.

Fairness for this metric is between -0.1 and 0.1

\[Pr(\hat{Y} = v | D = \text{unprivileged}) - Pr(\hat{Y} = v | D = \text{privileged})\]

code source inspired from aif360 statistical_parity_difference

Parameters:	y (array like) – list of predicted labels prot_attr (array like) – Array of 0 and 1 same length as y which indicates if the row is member of a privileged group or not pos_label (int (default 1)) – number of the positive label
Returns:	Statistical parity difference bias metric
Return type:	float
Raises:	ValueError: – y and prot_attr must have the same length

Disparate Impact¶

transparentai.fairness.metrics.disparate_impact(y, prot_attr, pos_label=1)[source]¶

Computes the Disparate impact for a protected attribute and a specified label

Computed as the ratio of rate of favorable outcome for the unprivileged group to that of the privileged group.

The ideal value of this metric is 1.0 A value < 1 implies higher benefit for the privileged group and a value > 1 implies a higher benefit for the unprivileged group.

Fairness for this metric is between 0.8 and 1.2

\[\frac{Pr(\hat{Y} = v | D = \text{unprivileged})} {Pr(\hat{Y} = v | D = \text{privileged})}\]

code source inspired from aif360 disparate_impact

Parameters:	y (array like) – list of predicted labels prot_attr (array like) – Array of 0 and 1 same length as y which indicates if the row is member of a privileged group or not pos_label (int (default 1)) – number of the positive label
Returns:	Disparate impact bias metric
Return type:	float
Raises:	ValueError: – y and prot_attr must have the same length

Equal opportunity difference¶

transparentai.fairness.metrics.equal_opportunity_difference(y_true, y_pred, prot_attr, pos_label=1)[source]¶

Computes the equal opportunity difference for a protected attribute and a specified label

This metric is computed as the difference of true positive rates between the unprivileged and the privileged groups. The true positive rate is the ratio of true positives to the total number of actual positives for a given group.

The ideal value is 0. A value of < 0 implies higher benefit for the privileged group and a value > 0 implies higher benefit for the unprivileged group.

Fairness for this metric is between -0.1 and 0.1

\(TPR_{D = \text{unprivileged}} - TPR_{D = \text{privileged}}\)

code source from aif360 equal_opportunity_difference

Parameters:	y_true (array like) – True labels y_pred (array like) – Predicted labels prot_attr (array like) – Array of 0 and 1 same length as y which indicates if the row is member of a privileged group or not pos_label (int (default 1)) – number of the positive label
Returns:	Equal opportunity difference bias metric
Return type:	float
Raises:	ValueError: – y_true and y_pred must have the same length ValueError: – y_true and prot_attr must have the same length

Average odds difference¶

transparentai.fairness.metrics.average_odds_difference(y_true, y_pred, prot_attr, pos_label=1)[source]¶

Computes the average odds difference for a protected attribute and a specified label

Computed as average difference of false positive rate (false positives / negatives) and true positive rate (true positives / positives) between unprivileged and privileged groups.

The ideal value of this metric is 0. A value of < 0 implies higher benefit for the privileged group and a value > 0 implies higher benefit for the unprivileged group.

Fairness for this metric is between -0.1 and 0.1

\[\frac{1}{2}\left[|FPR_{D = \text{unprivileged}} - FPR_{D = \text{privileged}}| + |TPR_{D = \text{unprivileged}} - TPR_{D = \text{privileged}}|\right]\]

A value of 0 indicates equality of odds.

code source from aif360 average_odds_difference

Parameters:	y_true (array like) – True labels y_pred (array like) – Predicted labels prot_attr (array like) – Array of 0 and 1 same length as y which indicates if the row is member of a privileged group or not pos_label (int (default 1)) – number of the positive label
Returns:	Average of absolute difference bias metric
Return type:	float
Raises:	ValueError: – y_true and y_pred must have the same length ValueError: – y_true and prot_attr must have the same length

Theil Index¶

transparentai.fairness.metrics.theil_index(y_true, y_pred, prot_attr, pos_label=1)[source]¶

Computes the theil index for a protected attribute and a specified label

Computed as the generalized entropy of benefit for all individuals in the dataset, with alpha = 1. It measures the inequality in benefit allocation for individuals.

A value of 0 implies perfect fairness.

Fairness is indicated by lower scores, higher scores are problematic

With \(b_i = \hat{y}_i - y_i + 1\):

\[\frac{1}{n}\sum_{i=1}^n\frac{b_{i}}{\mu}\ln\frac{b_{i}}{\mu}\]

code source from aif360 theil_index

Parameters:	y_true (array like) – True labels y_pred (array like) – Predicted labels prot_attr (array like) – Array of 0 and 1 same length as y which indicates if the row is member of a privileged group or not pos_label (int (default 1)) – number of the positive label
Returns:	Theil index bias metric
Return type:	float
Raises:	ValueError: – y_true and y_pred must have the same length ValueError: – y_true and prot_attr must have the same length

transparentai.fairness¶

Fairness module¶

Fairness metrics¶

Statistical parity difference¶

Disparate Impact¶

Equal opportunity difference¶

Average odds difference¶

Theil Index¶

`transparentai.fairness`¶