Skip to contents

Classification Metric

Source: R/classification_metric.R
classification_metric.Rd

Class for computing metrics based on two BinaryLabelDatasets. The first dataset is the original one and the second is the output of the classification transformer (or similar)

Usage

classification_metric(dataset, classified_dataset, unprivileged_groups, privileged_groups)

Arguments

dataset

(BinaryLabelDataset) Dataset containing ground-truth labels

classified_dataset

(BinaryLabelDataset) Dataset containing predictions

unprivileged_groups

Unprivileged groups. List containing unprivileged protected attribute name and value of the unprivileged protected attribute.

privileged_groups

Privileged groups. List containing privileged protected attribute name and value of the privileged protected attribute.

See also

Explore available classification metrics explanations here

Available metrics:

  • accuracy

  • average_abs_odds_difference

  • average_odds_difference

  • between_all_groups_coefficient_of_variation

  • between_all_groups_generalized_entropy_index

  • between_all_groups_theil_index

  • between_group_coefficient_of_variation

  • between_group_generalized_entropy_index

  • between_group_theil_index

  • binary_confusion_matrix

  • coefficient_of_variation

  • disparate_impact

  • equal_opportunity_difference

  • error_rate

  • error_rate_difference

  • error_rate_ratio

  • false_discovery_rate

  • false_discovery_rate_difference

  • false_discovery_rate_ratio

  • false_negative_rate

  • false_negative_rate_difference

  • false_negative_rate_ratio

  • false_omission_rate

  • false_omission_rate_difference

  • false_omission_rate_ratio

  • false_positive_rate

  • false_positive_rate_difference

  • false_positive_rate_ratio

  • generalized_binary_confusion_matrix

  • generalized_entropy_index

  • generalized_false_negative_rate

  • generalized_false_positive_rate

  • generalized_true_negative_rate

  • generalized_true_positive_rate

  • negative_predictive_value

  • num_false_negatives

  • num_false_positives

  • num_generalized_false_negatives

  • num_generalized_false_positives

  • num_generalized_true_negatives

  • num_generalized_true_positives

  • num_pred_negatives

  • num_pred_positives

  • num_true_negatives

  • num_true_positives

  • performance_measures

  • positive_predictive_value

  • power

  • precision

  • recall

  • selection_rate

  • sensitivity

  • specificity

  • statistical_parity_difference

  • theil_index

  • true_negative_rate

  • true_positive_rate

  • true_positive_rate_difference

Examples

if (FALSE) {
load_aif360_lib()
# Input dataset
data <- data.frame("feat" = c(0,0,1,1,1,1,0,1,1,0), "label" = c(1,0,0,1,0,0,1,0,1,1))
# Create aif compatible input dataset
act <- aif360::binary_label_dataset(data_path = data,  favor_label=0, unfavor_label=1,
                            unprivileged_protected_attribute=0,
                            privileged_protected_attribute=1,
                            target_column="label", protected_attribute="feat")
# Classified dataset
pred_data <- data.frame("feat" = c(0,0,1,1,1,1,0,1,1,0), "label" = c(1,0,1,1,1,0,1,0,0,1))
# Create aif compatible classified dataset
pred <- aif360::binary_label_dataset(data_path = pred_data,  favor_label=0, unfavor_label=1,
                             unprivileged_protected_attribute=0,
                             privileged_protected_attribute=1,
                             target_column="label", protected_attribute="feat")
# Create an instance of classification metric
cm <- classification_metric(act, pred, list('feat', 1), list('feat', 0))
# Access metric functions
cm$accuracy()
}