transparentai.monitoring

Monitoring submodule

transparentai.monitoring.monitoring.compute_metrics_groupby(y_true, y_pred, groupby, metrics, classification)

Computes metrics groupby an array.

Parameters:

• y_true (array like) – True labels
• y_pred (array like (1D or 2D)) – if 1D array Predicted labels, if 2D array probabilities (returns of a predict_proba function)
• groupby (array like) – Array of values to groupby the computed metrics by
• metrics (list) – List of metrics to compute
• classification (bool) – Whether the ML task is a classification or not

Returns:

DataFrame with groubpy values as indexes and computed metrics as columns

Return type:

pd.DataFrame

transparentai.monitoring.monitoring.monitor_model(y_true, y_pred, timestamp=None, interval='month', metrics=None, classification=False)

Monitor model over a timestamp array which represent the date or timestamp of the prediction.

If timestamp is None or interval then it just compute the metrics on all the predictions.

If interval is not None it can be one of the following : ‘year’, ‘month’, ‘day’ or ‘hour’.

• ‘year’ : format ‘%Y’
• ‘month’ : format ‘%Y-%m’
• ‘day’ : format ‘%Y-%m-%d’
• ‘hour’ : format ‘%Y-%m-%d-%r’

If it’s for a classification and you’re using y_pred as probabilities don’t forget to pass the classification=True argument !

You can use your choosing metrics. for that refer to the evaluation metrics documentation.

Parameters:

• y_true (array like) – True labels
• y_pred (array like (1D or 2D)) – if 1D array Predicted labels, if 2D array probabilities (returns of a predict_proba function)
• timestamp (array like or None (default None)) – Array of datetime when the prediction occured
• interval (str or None (default 'month')) – interval to format the timestamp with
• metrics (list (default None)) – List of metrics to compute
• classification (bool (default True)) – Whether the ML task is a classification or not

Returns:

DataFrame with datetime interval as indexes and computed metrics as columns

Return type:

pd.DataFrame

Raises:

• ValueError: – interval must be ‘year’, ‘month’, ‘day’ or ‘hour’
• TypeError: – y_true must be an array like
• TypeError: – timestamp must be an array like