Robustness Implies Fairness in Causal Algorithmic Recourse

This project implements the paper “Robustness implies Fairness in Casual Algorithmic Recourse” using the R language.

This study explores the concept of individual fairness and adversarial robustness in causal algorithmic recourse and addresses the challenge of achieving both. To resolve the challenges, we propose a new framework for defining adversarially robust recourse.

The new setting views the protected feature as a pseudometric and demonstrates that individual fairness is a special case of adversarial robustness. Finally, we introduce the fair robust recourse problem to achieve both desirable properties and show how it can be satisfied both theoretically and empirically.

If you find it useful, please consider citing:

@misc{https://doi.org/10.48550/arxiv.2302.03465,
  doi = {10.48550/ARXIV.2302.03465},
  url = {https://arxiv.org/abs/2302.03465},
  author = {Ehyaei, Ahmad-Reza and Karimi, Amir-Hossein and 
  Schölkopf, Bernhard and Maghsudi, Setareh},
  title = {Robustness Implies Fairness in Casual Algorithmic Recourse},
  publisher = {arXiv},
  year = {2023},
}

Experiments

In our experiments, we validate our claims through experiments and assess the effects of different recourse definitions on individual fairness. At first, we perform numerical simulations on various models and classifiers. Next, we apply our findings to both a real-world and semi-synthetic dataset as a case study. The codes and instructions for reproducing our experiments are available at Github.

Numerical Simulation

Since the recourse actions require knowledge of the underlying SCM, we begin by defining two linear and non-linear ANM models for the SCM. In our experiments, we utilize two non-protected continuous features $X_i$ and a binary protected attribute $A$ with a value of either 0 or 1. The structural equations of linear SCM (LIN) is given by:

$$\begin{cases} A := U_A, & U_A \sim \mathcal{R}(0.5) \\ X_1 := 2A + U_1, & U_1\sim \mathcal{N}(0,1) \\ X_2 := A-X_1 + U_2, & U_2 \sim \mathcal{N}(0,1) \end{cases}$$

For the non-linear SCM (ANM) the following structural equations:

$$ \begin{cases} A := U_A, & U_A \sim \mathcal{R}(0.5) \\ X_1 := 2A^2 + U_1, & U_1\sim \mathcal{N}(0,1) \\ X_2 := AX_1 + U_2, & U_2 \sim \mathcal{N}(0,1) \end{cases}$$

where $\mathcal{R}(p)$ is Rademacher random variables with probability $p$ and $\mathcal{N}(\mu,\sigma^2)$ is normal r.v. with mean $\mu$ and variance $\sigma^2$. To see, modified or define new SCM, see the script utils/scm_models.R.

To add the ground truth label, we consider both linear and non-linear functions in the form of $Y = sign(f(v,w) - b)$, where $w \in \mathbb{R}^{n+1}$ is coefficient of $V_i$ and $b \in \mathbb{R}$. We also examine an unaware baseline where $h$ does not depend on protected variable $A$. The label functions formula is given by:

$$h(A, X_1,X_2,X_3) = \begin{cases} \textrm{sign}(A + X_1 + X_2 < 0) & \text{Linear and Aware} \\ \textrm{sign}(X_1 + X_2 < 0) & \text{Linear and Unaware} \\ \textrm{sign}((A + X_1 + X_2)^2 <2) & \text{Non-Linear and Aware}\\ \textrm{sign}((X_1 + X_2)^2 <2) & \text{Non-Linear and Unaware} \end{cases}$$

The file named utils/labels.R holds the label functions. For each model, we generate 10,000 samples through utilizing the structural equations of the SCMs. The following presents two examples of data generation.

For each dataset, we split the samples into 80% for training and 20% for testing. Then, we train a logistic regression (LR), support vector machine (SVM), and gradient boosting machine (GBM) using all features or just the non-protected features $\mathbf{X}$ as an unaware baseline. We use the \citet{h2o.ai} package to train models and the \textit{h2o.grid} for tuning hyperparameters with the below searching parameters:

• GLM: use alpha = seq(0, 1, 0.1) with lambda_search = TRUE.
• SVM: set gamma = 0.01 , rank_ratio = 0.1, and use a Gaussian kernel.
• GBM: search for the optimal model among the following parameters: learn_rate = c(0.01, 0.1), max_depth = c(3, 5, 9), and sample_rate = c(0.8, 1.0).

To find the classifier’s scripts see utils/models.R. The decision boundary for the two models is displayed in the figures below.

We consider discrete and trivial pseudometric ($d(a,a')=0$ for all $a,a'$) for protected feature $A$. For continuous variables and product metric space, we use the $L_2$ norm. The cost is defined as the $L_2$ norm, with $cost(v,a) = |v - \mathbf{CF}(v,a) |_2$. Finally, we evaluate the methods presented in paper for having individual fairness by testing different perturbation radii $\Delta \in {1, 0.5, 0.1}$.

Since the main objective of this work is not to provide an algorithmic solution for causal recourse, we use a brute-force search to find the optimal action. The function for calculating recourse and robust recourse can be found in the utils/models.R file. In the below the adversarial recourse in some simulations, with $\Delta=1$, for unaware labels and classifiers, including instances and their twins is show.

Case Studies

We use the Adult Income Demographic dataset (ACSIncome), an updated version of the UCI Adult dataset , which contains over 195,000 records from California state in 2018. The data was obtained by using the Folktables Python package . To fetch data, we used the utils/fetch_adult_data.py script.

The data processing and modeling procedures adopted in this study are consistent with those reported in Nabi and Shpitser work.

Furthermore, we consider a semi-synthetic SCM proposed by Karimi et al that is based on a loan approval scenario. The data aims to reflect the intuitive relationships between variables in a practical loan approval process.

This semi-synthetic data consists of gender, age, education, loan amount, duration, income, and saving variables with the following structural equations and exogenous distributions:

$$\begin{cases} G := U_G & U_G \sim \text{Bernoulli}(0.5)\\ A := -35+U_A & U_A \sim \text{Gamma}(10, 3.5) \\ E := -0.5 + \bigg(1 + e^{-\big(-1 + 0.5 G + (1 + e^{- 0.1 A})^{-1} + U_E \big)}\bigg)^{-1} & U_E \sim \mathcal{N}(0,0.25) \\ L := 1 + 0.01 (A - 5) (5 - A) + G + U_L & U_L \sim \mathcal{N}(0,4) \\ D := -1 + 0.1A + 2G + L + U_D & U_D \sim \mathcal{N}(0, 9)\\ I := -4 + 0.1(A + 35) + 2G + G E + U_I & U_I \sim \mathcal{N}(0, 4)\\ S := -4 + 1.5 \mathbb{I}_{{I > 0}} I + U_S & U_S\sim\mathcal{N}(0, 25) \end{cases}$$

The labels $Y$ were generated using the following formula:

$$Y\sim \text{Bernoulli}\left(\left(1+e^{-0.3(-L-D+I+S+IS)}\right)^{-1}\right).$$

Main References

[1] Dominguez-Olmedo, Ricardo, Amir H. Karimi, and Bernhard Schölkopf. “On the adversarial robustness of causal algorithmic recourse.” International Conference on Machine Learning. PMLR, 2022.

[2] von Kügelgen, Julius, et al. “On the fairness of causal algorithmic recourse.” Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 36. No. 9. 2022.

[3] Karimi, Amir-Hossein, Bernhard Schölkopf, and Isabel Valera. “Algorithmic recourse: from counterfactual explanations to interventions.” Proceedings of the 2021 ACM conference on fairness, accountability, and transparency. 2021.

[4] Glymour, Madelyn, Judea Pearl, and Nicholas P. Jewell. Causal inference in statistics: A primer. John Wiley & Sons, 2016.

[5] Peters, Jonas, Dominik Janzing, and Bernhard Schölkopf. Elements of causal inference: foundations and learning algorithms. The MIT Press, 2017.

License

MIT