reject is a Python library for classification with rejection. Neural networks are often confidently wrong when confronted with out-of-distribution data. When the prediction's uncertainty is too high, the model abstains from predicting and the observation is passed on to a human expert who takes the final decision. It is useful for applications where making an error can be more costly than asking a human expert for help.

Installation

$ pip install reject

Documentation

The documentation is deployed to reject.readthedocs.io.

Usage

from reject.reject import ClassificationRejector

y_pred  # Array of predictions. Shape (n_observations, n_classes) or (n_observations, n_samples, n_classes).
y_true  # Array of true labels. Shape (n_observations,).

# initialize the rejector
rej = ClassificationRejector(y_true_all, y_pred_all)

# single rejection point
rej.reject(threshold=0.5, unc_type="TU", relative=True, show=True)

             Non-rejected    Rejected
---------  --------------  ----------
Correct               891          20
Incorrect             109         980

  Non-rejected accuracy    Classification quality    Rejection quality
-----------------------  ------------------------  -------------------
                 0.8910                    0.9355              40.9908

# rejection curve
fig = rej.plot_reject(unc_type="TU", metric="NRA")
print(fig)

User guide notebooks are provided in the reject.readthedocs.io documentation.