From 6de49982b65e9ef5cde4a7b83f7a180fab187aad Mon Sep 17 00:00:00 2001 From: "Documenter.jl" Date: Wed, 24 Apr 2024 11:18:25 +0000 Subject: [PATCH] build based on 7cf8774 --- dev/.documenter-siteinfo.json | 2 +- dev/CHANGELOG/index.html | 2 +- dev/assets/resources/index.html | 2 +- dev/contribute/index.html | 2 +- dev/contribute/performance/index.html | 2 +- dev/explanation/architecture/index.html | 2 +- dev/explanation/categorical/index.html | 2 +- .../generators/clap_roar/index.html | 2 +- dev/explanation/generators/clue/index.html | 2 +- dev/explanation/generators/dice/index.html | 2 +- .../generators/feature_tweak/index.html | 2 +- dev/explanation/generators/generic/index.html | 2 +- .../generators/gravitational/index.html | 2 +- dev/explanation/generators/greedy/index.html | 2 +- .../generators/growing_spheres/index.html | 2 +- .../generators/overview/index.html | 2 +- dev/explanation/generators/probe/index.html | 2 +- dev/explanation/generators/revise/index.html | 2 +- dev/explanation/index.html | 2 +- dev/explanation/optimisers/jsma/index.html | 2 +- .../optimisers/overview/index.html | 2 +- dev/extensions/index.html | 2 +- dev/extensions/laplace_redux/index.html | 2 +- dev/extensions/neurotree/index.html | 2 +- .../custom_generators/index.html | 2 +- dev/how_to_guides/custom_models/index.html | 2 +- dev/how_to_guides/index.html | 2 +- dev/index.html | 2 +- dev/objects.inv | Bin 6964 -> 6959 bytes dev/reference/index.html | 64 +++++++++--------- dev/release-notes/index.html | 2 +- dev/search_index.js | 2 +- dev/tutorials/benchmarking/index.html | 2 +- dev/tutorials/convergence/index.html | 2 +- dev/tutorials/data_catalogue/index.html | 2 +- dev/tutorials/data_preprocessing/index.html | 2 +- dev/tutorials/evaluation/index.html | 2 +- dev/tutorials/generators/index.html | 2 +- dev/tutorials/index.html | 2 +- dev/tutorials/model_catalogue/index.html | 2 +- dev/tutorials/models/index.html | 2 +- dev/tutorials/parallelization/index.html | 2 +- dev/tutorials/simple_example/index.html | 2 +- dev/tutorials/whistle_stop/index.html | 2 +- 44 files changed, 74 insertions(+), 74 deletions(-) diff --git a/dev/.documenter-siteinfo.json b/dev/.documenter-siteinfo.json index 724c197d3..26ba891ff 100644 --- a/dev/.documenter-siteinfo.json +++ b/dev/.documenter-siteinfo.json @@ -1 +1 @@ -{"documenter":{"julia_version":"1.10.2","generation_timestamp":"2024-04-22T13:12:46","documenter_version":"1.4.0"}} \ No newline at end of file +{"documenter":{"julia_version":"1.10.2","generation_timestamp":"2024-04-24T11:18:02","documenter_version":"1.4.0"}} \ No newline at end of file diff --git a/dev/CHANGELOG/index.html b/dev/CHANGELOG/index.html index 7c2c15875..5b3197adc 100644 --- a/dev/CHANGELOG/index.html +++ b/dev/CHANGELOG/index.html @@ -1,2 +1,2 @@ -Changelog · CounterfactualExplanations.jl
GitHub

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Note: We try to adhere to these practices as of version [v1.1.1].

Version [v1.1.4] - 2024-04-19

Changed

  • Refactors the encodings and decodings such that it is now more streamlined. Instead of conditional statements, encodings are now dispatched on the type of a new unifying data.input_encoder field. [#432]
  • Refactors the check for redundancy. This is now based on the convergence type and done right before the counterfactual search begins, if not redundant. [#432]

Version [v1.1.3] - 2024-04-17

Added

  • Adds a section on Convergence to the documentation, Changelog.jl functionality and a few doc tests. [#429]

Changed

  • Changes style of taking gradients for the counterfactual search from implicit to explicit. [#430]
  • Removed all implicit imports. [#430]

Removed

  • Removed CUDA.jl dependency, because redundant. [#430]
  • Removed Parameters.jl dependency, because redundant. [#430]

Version [v1.1.2] - 2024-04-16

Changed

  • Replaces the GIF in the README and introduction of docs for a static image.

Version [v1.1.1] - 2024-04-15

Added

  • Added tests for LaplaceRedux extension. Bumped upper compat bound for LaplaceRedux.jl. [#428]

<!– Links generated by Changelog.jl –>

[#428]: https://github.com/juliatrustworthyai/CounterfactualExplanations.jl/issues/428 [#429]: https://github.com/juliatrustworthyai/CounterfactualExplanations.jl/issues/429

+Changelog · CounterfactualExplanations.jl
GitHub

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Note: We try to adhere to these practices as of version [v1.1.1].

Version [v1.1.4] - 2024-04-24

Changed

  • Refactors the encodings and decodings such that it is now more streamlined. Instead of conditional statements, encodings are now dispatched on the type of a new unifying data.input_encoder field. [#432]
  • Refactors the check for redundancy. This is now based on the convergence type and done right before the counterfactual search begins, if not redundant. [#432]

Version [v1.1.3] - 2024-04-17

Added

  • Adds a section on Convergence to the documentation, Changelog.jl functionality and a few doc tests. [#429]

Changed

  • Changes style of taking gradients for the counterfactual search from implicit to explicit. [#430]
  • Removed all implicit imports. [#430]

Removed

  • Removed CUDA.jl dependency, because redundant. [#430]
  • Removed Parameters.jl dependency, because redundant. [#430]

Version [v1.1.2] - 2024-04-16

Changed

  • Replaces the GIF in the README and introduction of docs for a static image.

Version [v1.1.1] - 2024-04-15

Added

  • Added tests for LaplaceRedux extension. Bumped upper compat bound for LaplaceRedux.jl. [#428]

<!– Links generated by Changelog.jl –>

[#428]: https://github.com/juliatrustworthyai/CounterfactualExplanations.jl/issues/428 [#429]: https://github.com/juliatrustworthyai/CounterfactualExplanations.jl/issues/429

diff --git a/dev/assets/resources/index.html b/dev/assets/resources/index.html index 065a9a978..a10c8dd95 100644 --- a/dev/assets/resources/index.html +++ b/dev/assets/resources/index.html @@ -1,2 +1,2 @@ -📚 Additional Resources · CounterfactualExplanations.jl
GitHub
+📚 Additional Resources · CounterfactualExplanations.jl
GitHub
diff --git a/dev/contribute/index.html b/dev/contribute/index.html index 482a785f3..74c7be8e1 100644 --- a/dev/contribute/index.html +++ b/dev/contribute/index.html @@ -1,2 +1,2 @@ -🛠 Contribute · CounterfactualExplanations.jl
GitHub

🛠 Contribute

Contributions of any kind are very much welcome! Take a look at the issue to see what things we are currently working on. If you have an idea for a new feature or want to report a bug, please open a new issue.

Development

If your looking to contribute code, it may be helpful to check out the Explanation section of the docs.

Testing

Please always make sure to add tests for any new features or changes.

Documentation

If you add new features or change existing ones, please make sure to update the documentation accordingly. The documentation is written in Documenter.jl and is located in the docs/src folder.

Log Changes

As of version 1.1.1, we have tried to be more stringent about logging changes. Please make sure to add a note to the CHANGELOG.md file for any changes you make. It is sufficient to add a note under the Unreleased section.

General Pointers

There are also some general pointers for people looking to contribute to any of our Taija packages here.

Please follow the SciML ColPrac guide.

+🛠 Contribute · CounterfactualExplanations.jl
GitHub

🛠 Contribute

Contributions of any kind are very much welcome! Take a look at the issue to see what things we are currently working on. If you have an idea for a new feature or want to report a bug, please open a new issue.

Development

If your looking to contribute code, it may be helpful to check out the Explanation section of the docs.

Testing

Please always make sure to add tests for any new features or changes.

Documentation

If you add new features or change existing ones, please make sure to update the documentation accordingly. The documentation is written in Documenter.jl and is located in the docs/src folder.

Log Changes

As of version 1.1.1, we have tried to be more stringent about logging changes. Please make sure to add a note to the CHANGELOG.md file for any changes you make. It is sufficient to add a note under the Unreleased section.

General Pointers

There are also some general pointers for people looking to contribute to any of our Taija packages here.

Please follow the SciML ColPrac guide.

diff --git a/dev/contribute/performance/index.html b/dev/contribute/performance/index.html index 00f2df807..4a34c17ed 100644 --- a/dev/contribute/performance/index.html +++ b/dev/contribute/performance/index.html @@ -12,4 +12,4 @@ # Search: generator = GenericGenerator() ce = generate_counterfactual(x, target, counterfactual_data, M, generator)
data_large = TaijaData.load_linearly_separable(100000)
-counterfactual_data_large = DataPreprocessing.CounterfactualData(data_large...)
@time generate_counterfactual(x, target, counterfactual_data, M, generator)
@time generate_counterfactual(x, target, counterfactual_data_large, M, generator)
+counterfactual_data_large = DataPreprocessing.CounterfactualData(data_large...)
@time generate_counterfactual(x, target, counterfactual_data, M, generator)
@time generate_counterfactual(x, target, counterfactual_data_large, M, generator)
diff --git a/dev/explanation/architecture/index.html b/dev/explanation/architecture/index.html index 3a1ed5e6b..ede687a88 100644 --- a/dev/explanation/architecture/index.html +++ b/dev/explanation/architecture/index.html @@ -1,2 +1,2 @@ -Package Architecture · CounterfactualExplanations.jl
GitHub

Package Architecture

The diagram below provides an overview of the package architecture. It is built around two core modules that are designed to be as extensible as possible through dispatch: 1) Models is concerned with making any arbitrary model compatible with the package; 2) Generators is used to implement arbitrary counterfactual search algorithms.[1]

The core function of the package, generate_counterfactual, uses an instance of type AbstractFittedModel produced by the Models module and an instance of type AbstractGenerator produced by the Generators module.

Metapackages from the Taija ecosystem provide additional functionality such as datasets, language interoperability, parallelization, and plotting. The CounterfactualExplanations package is designed to be used in conjunction with these metapackages, but can also be used as a standalone package.

[1] We have made an effort to keep the code base a flexible and extensible as possible, but cannot guarantee at this point that any counterfactual generator can be implemented without further adaptation.

+Package Architecture · CounterfactualExplanations.jl
GitHub

Package Architecture

The diagram below provides an overview of the package architecture. It is built around two core modules that are designed to be as extensible as possible through dispatch: 1) Models is concerned with making any arbitrary model compatible with the package; 2) Generators is used to implement arbitrary counterfactual search algorithms.[1]

The core function of the package, generate_counterfactual, uses an instance of type AbstractFittedModel produced by the Models module and an instance of type AbstractGenerator produced by the Generators module.

Metapackages from the Taija ecosystem provide additional functionality such as datasets, language interoperability, parallelization, and plotting. The CounterfactualExplanations package is designed to be used in conjunction with these metapackages, but can also be used as a standalone package.

[1] We have made an effort to keep the code base a flexible and extensible as possible, but cannot guarantee at this point that any counterfactual generator can be implemented without further adaptation.

diff --git a/dev/explanation/categorical/index.html b/dev/explanation/categorical/index.html index f9c657b9c..88542a57a 100644 --- a/dev/explanation/categorical/index.html +++ b/dev/explanation/categorical/index.html @@ -117,4 +117,4 @@ 0.0 0.0 1.0 - 1.85 + 1.85 diff --git a/dev/explanation/generators/clap_roar/index.html b/dev/explanation/generators/clap_roar/index.html index dc4a2eb4b..207c8b7fe 100644 --- a/dev/explanation/generators/clap_roar/index.html +++ b/dev/explanation/generators/clap_roar/index.html @@ -3,4 +3,4 @@ \text{extcost}(f(\mathbf{s}^\prime)) = l(M(f(\mathbf{s}^\prime)),y^\prime) \end{aligned}\]

for each counterfactual $k$ where $l$ denotes the loss function used to train $M$. This approach is based on the intuition that (endogenous) model shifts will be triggered by counterfactuals that increase classifier loss (Altmeyer et al. 2023).

Usage

The approach can be used in our package as follows:

generator = ClaPROARGenerator()
 ce = generate_counterfactual(x, target, counterfactual_data, M, generator)
-plot(ce)

Comparison to GenericGenerator

The figure below compares the outcome for the GenericGenerator and the ClaPROARGenerator.

References

Altmeyer, Patrick, Giovan Angela, Aleksander Buszydlik, Karol Dobiczek, Arie van Deursen, and Cynthia Liem. 2023. “Endogenous Macrodynamics in Algorithmic Recourse.” In First IEEE Conference on Secure and Trustworthy Machine Learning. https://doi.org/10.1109/satml54575.2023.00036.

Upadhyay, Sohini, Shalmali Joshi, and Himabindu Lakkaraju. 2021. “Towards Robust and Reliable Algorithmic Recourse.” https://arxiv.org/abs/2102.13620.

+plot(ce)

Comparison to GenericGenerator

The figure below compares the outcome for the GenericGenerator and the ClaPROARGenerator.

References

Altmeyer, Patrick, Giovan Angela, Aleksander Buszydlik, Karol Dobiczek, Arie van Deursen, and Cynthia Liem. 2023. “Endogenous Macrodynamics in Algorithmic Recourse.” In First IEEE Conference on Secure and Trustworthy Machine Learning. https://doi.org/10.1109/satml54575.2023.00036.

Upadhyay, Sohini, Shalmali Joshi, and Himabindu Lakkaraju. 2021. “Towards Robust and Reliable Algorithmic Recourse.” https://arxiv.org/abs/2102.13620.

diff --git a/dev/explanation/generators/clue/index.html b/dev/explanation/generators/clue/index.html index 849bc80a4..abfadeac7 100644 --- a/dev/explanation/generators/clue/index.html +++ b/dev/explanation/generators/clue/index.html @@ -7,4 +7,4 @@ ce = generate_counterfactual( x, target, counterfactual_data, M, generator; convergence=conv) -plot(ce)

Extra: The CLUE generator can also be used upon already having achieved a counterfactual with a different generator. In this case, you can use CLUE and make the counterfactual more robust.

Note: The above documentation is based on the information provided in the CLUE paper. Please refer to the original paper for more detailed explanations and implementation specifics.

References

Antorán, Javier, Umang Bhatt, Tameem Adel, Adrian Weller, and José Miguel Hernández-Lobato. 2020. “Getting a Clue: A Method for Explaining Uncertainty Estimates.” https://arxiv.org/abs/2006.06848.

+plot(ce)

Extra: The CLUE generator can also be used upon already having achieved a counterfactual with a different generator. In this case, you can use CLUE and make the counterfactual more robust.

Note: The above documentation is based on the information provided in the CLUE paper. Please refer to the original paper for more detailed explanations and implementation specifics.

References

Antorán, Javier, Umang Bhatt, Tameem Adel, Adrian Weller, and José Miguel Hernández-Lobato. 2020. “Getting a Clue: A Method for Explaining Uncertainty Estimates.” https://arxiv.org/abs/2006.06848.

diff --git a/dev/explanation/generators/dice/index.html b/dev/explanation/generators/dice/index.html index 4e8d42d84..3514e4bd1 100644 --- a/dev/explanation/generators/dice/index.html +++ b/dev/explanation/generators/dice/index.html @@ -40,4 +40,4 @@ num_counterfactuals=n_cf, convergence=conv ) ) -end

The figure below shows the resulting counterfactual paths. As expected, the resulting counterfactuals are more dispersed across the feature domain for higher choices of $\lambda_2$

References

Mothilal, Ramaravind K, Amit Sharma, and Chenhao Tan. 2020. “Explaining Machine Learning Classifiers Through Diverse Counterfactual Explanations.” In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 607–17. https://doi.org/10.1145/3351095.3372850.

[1] With thanks to the respondents on Discourse

+end

The figure below shows the resulting counterfactual paths. As expected, the resulting counterfactuals are more dispersed across the feature domain for higher choices of $\lambda_2$

References

Mothilal, Ramaravind K, Amit Sharma, and Chenhao Tan. 2020. “Explaining Machine Learning Classifiers Through Diverse Counterfactual Explanations.” In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 607–17. https://doi.org/10.1145/3351095.3372850.

[1] With thanks to the respondents on Discourse

diff --git a/dev/explanation/generators/feature_tweak/index.html b/dev/explanation/generators/feature_tweak/index.html index c420875ab..9c139ee75 100644 --- a/dev/explanation/generators/feature_tweak/index.html +++ b/dev/explanation/generators/feature_tweak/index.html @@ -31,4 +31,4 @@ colorbar=false, ) -display(plot(p1, p2; size=(800, 400)))

References

Tolomei, Gabriele, Fabrizio Silvestri, Andrew Haines, and Mounia Lalmas. 2017. “Interpretable Predictions of Tree-Based Ensembles via Actionable Feature Tweaking.” In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 465–74. https://doi.org/10.1145/3097983.3098039.

+display(plot(p1, p2; size=(800, 400)))

References

Tolomei, Gabriele, Fabrizio Silvestri, Andrew Haines, and Mounia Lalmas. 2017. “Interpretable Predictions of Tree-Based Ensembles via Actionable Feature Tweaking.” In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 465–74. https://doi.org/10.1145/3097983.3098039.

diff --git a/dev/explanation/generators/generic/index.html b/dev/explanation/generators/generic/index.html index 9e813959b..7c40ddaff 100644 --- a/dev/explanation/generators/generic/index.html +++ b/dev/explanation/generators/generic/index.html @@ -1,4 +1,4 @@ Generic · CounterfactualExplanations.jl
GitHub

GenericGenerator

We use the term generic to relate to the basic counterfactual generator proposed by Wachter, Mittelstadt, and Russell (2017) with $L1$-norm regularization. There is also a variant of this generator that uses the distance metric proposed in Wachter, Mittelstadt, and Russell (2017), which we call WachterGenerator.

Description

As the term indicates, this approach is simple: it forms the baseline approach for gradient-based counterfactual generators. Wachter, Mittelstadt, and Russell (2017) were among the first to realise that

[…] explanations can, in principle, be offered without opening the “black box.”

— Wachter, Mittelstadt, and Russell (2017)

Gradient descent is performed directly in the feature space. Concerning the cost heuristic, the authors choose to penalize the distance of counterfactuals from the factual value. This is based on the intuitive notion that larger feature perturbations require greater effort.

Usage

The approach can be used in our package as follows:

generator = GenericGenerator()
 ce = generate_counterfactual(x, target, counterfactual_data, M, generator)
-plot(ce)

References

Wachter, Sandra, Brent Mittelstadt, and Chris Russell. 2017. “Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR.” Harv. JL & Tech. 31: 841. https://doi.org/10.2139/ssrn.3063289.

+plot(ce)

References

Wachter, Sandra, Brent Mittelstadt, and Chris Russell. 2017. “Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR.” Harv. JL & Tech. 31: 841. https://doi.org/10.2139/ssrn.3063289.

diff --git a/dev/explanation/generators/gravitational/index.html b/dev/explanation/generators/gravitational/index.html index 5230fc3fc..48a43d267 100644 --- a/dev/explanation/generators/gravitational/index.html +++ b/dev/explanation/generators/gravitational/index.html @@ -5,4 +5,4 @@ \text{extcost}(f(\mathbf{s}^\prime)) = \text{dist}(f(\mathbf{s}^\prime),\bar{x}^*) \end{aligned}\]

where $\bar{x}$ is some sensible point in the target domain, for example, the subsample average $\bar{x}^*=\text{mean}(x)$, $x \in \mathcal{D}_1$.

There is a tradeoff then, between the distance of counterfactuals from their factual value and the chosen point in the target domain. The chart below illustrates how the counterfactual outcome changes as the penalty $\lambda_2$ on the distance to the point in the target domain is increased from left to right (holding the other penalty term constant).

Usage

The approach can be used in our package as follows:

generator = GravitationalGenerator()
 ce = generate_counterfactual(x, target, counterfactual_data, M, generator)
-display(plot(ce))

Comparison to GenericGenerator

The figure below compares the outcome for the GenericGenerator and the GravitationalGenerator.

References

Altmeyer, Patrick, Giovan Angela, Aleksander Buszydlik, Karol Dobiczek, Arie van Deursen, and Cynthia Liem. 2023. “Endogenous Macrodynamics in Algorithmic Recourse.” In First IEEE Conference on Secure and Trustworthy Machine Learning. https://doi.org/10.1109/satml54575.2023.00036.

+display(plot(ce))

Comparison to GenericGenerator

The figure below compares the outcome for the GenericGenerator and the GravitationalGenerator.

References

Altmeyer, Patrick, Giovan Angela, Aleksander Buszydlik, Karol Dobiczek, Arie van Deursen, and Cynthia Liem. 2023. “Endogenous Macrodynamics in Algorithmic Recourse.” In First IEEE Conference on Secure and Trustworthy Machine Learning. https://doi.org/10.1109/satml54575.2023.00036.

diff --git a/dev/explanation/generators/greedy/index.html b/dev/explanation/generators/greedy/index.html index 74a5904b8..a53de8a3e 100644 --- a/dev/explanation/generators/greedy/index.html +++ b/dev/explanation/generators/greedy/index.html @@ -2,4 +2,4 @@ Greedy · CounterfactualExplanations.jl
GitHub

GreedyGenerator

We use the term greedy to describe the counterfactual generator introduced by Schut et al. (2021).

Description

The Greedy generator works under the premise of generating realistic counterfactuals by minimizing predictive uncertainty. Schut et al. (2021) show that for models that incorporates predictive uncertainty in their predictions, maximizing the predictive probability corresponds to minimizing the predictive uncertainty: by construction, the generated counterfactual will therefore be realistic (low epistemic uncertainty) and unambiguous (low aleatoric uncertainty).

For the counterfactual search Schut et al. (2021) propose using a Jacobian-based Saliency Map Attack(JSMA). It is greedy in the sense that it is an “iterative algorithm that updates the most salient feature, i.e. the feature that has the largest influence on the classification, by $\delta$ at each step” (Schut et al. 2021).

Usage

The approach can be used in our package as follows:

M = fit_model(counterfactual_data, :DeepEnsemble)
 generator = GreedyGenerator()
 ce = generate_counterfactual(x, target, counterfactual_data, M, generator)
-plot(ce)

References

Schut, Lisa, Oscar Key, Rory Mc Grath, Luca Costabello, Bogdan Sacaleanu, Yarin Gal, et al. 2021. “Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertainties.” In International Conference on Artificial Intelligence and Statistics, 1756–64. PMLR.

+plot(ce)

References

Schut, Lisa, Oscar Key, Rory Mc Grath, Luca Costabello, Bogdan Sacaleanu, Yarin Gal, et al. 2021. “Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertainties.” In International Conference on Artificial Intelligence and Statistics, 1756–64. PMLR.

diff --git a/dev/explanation/generators/growing_spheres/index.html b/dev/explanation/generators/growing_spheres/index.html index 0d23df4a6..a236cf37f 100644 --- a/dev/explanation/generators/growing_spheres/index.html +++ b/dev/explanation/generators/growing_spheres/index.html @@ -3,4 +3,4 @@ M = fit_model(counterfactual_data, :DeepEnsemble) ce = generate_counterfactual( x, target, counterfactual_data, M, generator) -plot(ce)

References

Laugel, Thibault, Marie-Jeanne Lesot, Christophe Marsala, Xavier Renard, and Marcin Detyniecki. 2017. “Inverse Classification for Comparison-Based Interpretability in Machine Learning.” arXiv. https://doi.org/10.48550/arXiv.1712.08443.

+plot(ce)

References

Laugel, Thibault, Marie-Jeanne Lesot, Christophe Marsala, Xavier Renard, and Marcin Detyniecki. 2017. “Inverse Classification for Comparison-Based Interpretability in Machine Learning.” arXiv. https://doi.org/10.48550/arXiv.1712.08443.

diff --git a/dev/explanation/generators/overview/index.html b/dev/explanation/generators/overview/index.html index 31c2c9a23..e4823e24e 100644 --- a/dev/explanation/generators/overview/index.html +++ b/dev/explanation/generators/overview/index.html @@ -12,4 +12,4 @@ :generic => GenericGenerator :greedy => GreedyGenerator

The following sections provide brief descriptions of all of them.

Gradient-based Counterfactual Generators

At the time of writing, all generators are gradient-based: that is, counterfactuals are searched through gradient descent. In Altmeyer et al. (2023) we lay out a general methodological framework that can be applied to all of these generators:

\[\begin{aligned} \mathbf{s}^\prime &= \arg \min_{\mathbf{s}^\prime \in \mathcal{S}} \left\{ {\text{yloss}(M(f(\mathbf{s}^\prime)),y^*)}+ \lambda {\text{cost}(f(\mathbf{s}^\prime)) } \right\} -\end{aligned} \]

“Here $\mathbf{s}^\prime=\left\{s_k^\prime\right\}_K$ is a $K$-dimensional array of counterfactual states and $f: \mathcal{S} \mapsto \mathcal{X}$ maps from the counterfactual state space to the feature space.” (Altmeyer et al. 2023)

For most generators, the state space is the feature space ($f$ is the identity function) and the number of counterfactuals $K$ is one. Latent Space generators instead search counterfactuals in some latent space $\mathcal{S}$. In this case, $f$ corresponds to the decoder part of the generative model, that is the function that maps back from the latent space to inputs.

References

Altmeyer, Patrick, Giovan Angela, Aleksander Buszydlik, Karol Dobiczek, Arie van Deursen, and Cynthia Liem. 2023. “Endogenous Macrodynamics in Algorithmic Recourse.” In First IEEE Conference on Secure and Trustworthy Machine Learning. https://doi.org/10.1109/satml54575.2023.00036.

+\end{aligned} \]

“Here $\mathbf{s}^\prime=\left\{s_k^\prime\right\}_K$ is a $K$-dimensional array of counterfactual states and $f: \mathcal{S} \mapsto \mathcal{X}$ maps from the counterfactual state space to the feature space.” (Altmeyer et al. 2023)

For most generators, the state space is the feature space ($f$ is the identity function) and the number of counterfactuals $K$ is one. Latent Space generators instead search counterfactuals in some latent space $\mathcal{S}$. In this case, $f$ corresponds to the decoder part of the generative model, that is the function that maps back from the latent space to inputs.

References

Altmeyer, Patrick, Giovan Angela, Aleksander Buszydlik, Karol Dobiczek, Arie van Deursen, and Cynthia Liem. 2023. “Endogenous Macrodynamics in Algorithmic Recourse.” In First IEEE Conference on Secure and Trustworthy Machine Learning. https://doi.org/10.1109/satml54575.2023.00036.

diff --git a/dev/explanation/generators/probe/index.html b/dev/explanation/generators/probe/index.html index b5be55ff7..cffcfa4e8 100644 --- a/dev/explanation/generators/probe/index.html +++ b/dev/explanation/generators/probe/index.html @@ -10,4 +10,4 @@ generator = CounterfactualExplanations.Generators.ProbeGenerator(opt=opt) conv = CounterfactualExplanations.Convergence.InvalidationRateConvergence(;invalidation_rate=0.5) ce = generate_counterfactual(x, target, counterfactual_data, M, generator, convergence=conv) -plot(ce)

Choosing different invalidation rates makes the counterfactual more or less robust. The following plot shows the counterfactuals generated for different invalidation rates.

References

Pawelczyk, Martin, Teresa Datta, Johannes van-den-Heuvel, Gjergji Kasneci, and Himabindu Lakkaraju. 2022. “Probabilistically Robust Recourse: Navigating the Trade-Offs Between Costs and Robustness in Algorithmic Recourse.” arXiv Preprint arXiv:2203.06768.

+plot(ce)

Choosing different invalidation rates makes the counterfactual more or less robust. The following plot shows the counterfactuals generated for different invalidation rates.

References

Pawelczyk, Martin, Teresa Datta, Johannes van-den-Heuvel, Gjergji Kasneci, and Himabindu Lakkaraju. 2022. “Probabilistically Robust Recourse: Navigating the Trade-Offs Between Costs and Robustness in Algorithmic Recourse.” arXiv Preprint arXiv:2203.06768.

diff --git a/dev/explanation/generators/revise/index.html b/dev/explanation/generators/revise/index.html index d40576757..4b95f907e 100644 --- a/dev/explanation/generators/revise/index.html +++ b/dev/explanation/generators/revise/index.html @@ -52,4 +52,4 @@ # Define generator: generator = REVISEGenerator(opt=Flux.Adam(0.1)) # Generate recourse: -ce = generate_counterfactual(x, target, counterfactual_data, M, generator; convergence=conv)

The chart below shows the results:

References

Joshi, Shalmali, Oluwasanmi Koyejo, Warut Vijitbenjaronk, Been Kim, and Joydeep Ghosh. 2019. “Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems.” https://arxiv.org/abs/1907.09615.

[1] In general, we believe that there may be a trade-off between creating counterfactuals that respect the DGP vs. counterfactuals reflect the behaviour of the black-model in question - both accurately and complete.

[2] We believe that there is another potentially crucial disadvantage of relying on a separate generative model: it reallocates the task of learning realistic explanations for the data from the black-box model to the generative model.

+ce = generate_counterfactual(x, target, counterfactual_data, M, generator; convergence=conv)

The chart below shows the results:

References

Joshi, Shalmali, Oluwasanmi Koyejo, Warut Vijitbenjaronk, Been Kim, and Joydeep Ghosh. 2019. “Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems.” https://arxiv.org/abs/1907.09615.

[1] In general, we believe that there may be a trade-off between creating counterfactuals that respect the DGP vs. counterfactuals reflect the behaviour of the black-model in question - both accurately and complete.

[2] We believe that there is another potentially crucial disadvantage of relying on a separate generative model: it reallocates the task of learning realistic explanations for the data from the black-box model to the generative model.

diff --git a/dev/explanation/index.html b/dev/explanation/index.html index 5a346c487..c2d6bff69 100644 --- a/dev/explanation/index.html +++ b/dev/explanation/index.html @@ -1,2 +1,2 @@ -Overview · CounterfactualExplanations.jl
GitHub

Explanation

In this section you will find detailed explanations about the methodology and code.

Explanation clarifies, deepens and broadens the reader’s understanding of a subject.

— Diátaxis

In other words, you come here because you are interested in understanding how all of this actually works 🤓.

+Overview · CounterfactualExplanations.jl
GitHub

Explanation

In this section you will find detailed explanations about the methodology and code.

Explanation clarifies, deepens and broadens the reader’s understanding of a subject.

— Diátaxis

In other words, you come here because you are interested in understanding how all of this actually works 🤓.

diff --git a/dev/explanation/optimisers/jsma/index.html b/dev/explanation/optimisers/jsma/index.html index c6b644897..a2ee248cb 100644 --- a/dev/explanation/optimisers/jsma/index.html +++ b/dev/explanation/optimisers/jsma/index.html @@ -2,4 +2,4 @@ JSMA · CounterfactualExplanations.jl
GitHub

Jacobian-based Saliency Map Attack

To search counterfactuals, Schut et al. (2021) propose to use a Jacobian-Based Saliency Map Attack (JSMA) inspired by the literature on adversarial attacks. It works by moving in the direction of the most salient feature at a fixed step size in each iteration. Schut et al. (2021) use this optimisation rule in the context of Bayesian classifiers and demonstrate good results in terms of plausibility — how realistic counterfactuals are — and redundancy — how sparse the proposed feature changes are.

JSMADescent

To implement this approach in a reusable manner, we have added JSMA as a Flux optimiser. In particular, we have added a class JSMADescent<:Flux.Optimise.AbstractOptimiser, for which we have overloaded the Flux.Optimise.apply! method. This makes it possible to reuse JSMADescent as an optimiser in composable generators.

The optimiser can be used with with any generator as follows:

using CounterfactualExplanations.Generators: JSMADescent
 generator = GenericGenerator() |>
     gen -> @with_optimiser(gen,JSMADescent(;η=0.1))
-ce = generate_counterfactual(x, target, counterfactual_data, M, generator)

The figure below compares the resulting counterfactual search outcome to the corresponding outcome with generic Descent.

plot(p1,p2,size=(1000,400))

Schut, Lisa, Oscar Key, Rory Mc Grath, Luca Costabello, Bogdan Sacaleanu, Yarin Gal, et al. 2021. “Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertainties.” In International Conference on Artificial Intelligence and Statistics, 1756–64. PMLR.

+ce = generate_counterfactual(x, target, counterfactual_data, M, generator)

The figure below compares the resulting counterfactual search outcome to the corresponding outcome with generic Descent.

plot(p1,p2,size=(1000,400))

Schut, Lisa, Oscar Key, Rory Mc Grath, Luca Costabello, Bogdan Sacaleanu, Yarin Gal, et al. 2021. “Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertainties.” In International Conference on Artificial Intelligence and Statistics, 1756–64. PMLR.

diff --git a/dev/explanation/optimisers/overview/index.html b/dev/explanation/optimisers/overview/index.html index 593a5b455..016f2926f 100644 --- a/dev/explanation/optimisers/overview/index.html +++ b/dev/explanation/optimisers/overview/index.html @@ -1,2 +1,2 @@ -Overview · CounterfactualExplanations.jl
GitHub

Optimisation Rules

Counterfactual search is an optimization problem. Consequently, the choice of the optimisation rule affects the generated counterfactuals. In the short term, we aim to enable users to choose any of the available Flux optimisers. This has not been sufficiently tested yet, and you may run into issues.

Custom Optimisation Rules

Flux optimisers are specifically designed for deep learning, and in particular, for learning model parameters. In counterfactual search, the features are the free parameters that we are optimising over. To this end, some custom optimisation rules are necessary to incorporate ideas presented in the literature. In the following, we introduce those rules.

+Overview · CounterfactualExplanations.jl
GitHub

Optimisation Rules

Counterfactual search is an optimization problem. Consequently, the choice of the optimisation rule affects the generated counterfactuals. In the short term, we aim to enable users to choose any of the available Flux optimisers. This has not been sufficiently tested yet, and you may run into issues.

Custom Optimisation Rules

Flux optimisers are specifically designed for deep learning, and in particular, for learning model parameters. In counterfactual search, the features are the free parameters that we are optimising over. To this end, some custom optimisation rules are necessary to incorporate ideas presented in the literature. In the following, we introduce those rules.

diff --git a/dev/extensions/index.html b/dev/extensions/index.html index 8e4ba83bc..6740c963e 100644 --- a/dev/extensions/index.html +++ b/dev/extensions/index.html @@ -1,2 +1,2 @@ -Overview · CounterfactualExplanations.jl
GitHub

⛓️ Extensions

In this section, you will find information about package extensions of the CounterfactualExplanations package. Extensions are a relatively new feature of Julia that allows users to conditionally load code based on the presence of other packages. This is useful for creating packages that extend the functionality of other packages, without requiring the user to install the package being extended.

+Overview · CounterfactualExplanations.jl
GitHub

⛓️ Extensions

In this section, you will find information about package extensions of the CounterfactualExplanations package. Extensions are a relatively new feature of Julia that allows users to conditionally load code based on the presence of other packages. This is useful for creating packages that extend the functionality of other packages, without requiring the user to install the package being extended.

diff --git a/dev/extensions/laplace_redux/index.html b/dev/extensions/laplace_redux/index.html index f0fc4ca9c..c0dc36cfa 100644 --- a/dev/extensions/laplace_redux/index.html +++ b/dev/extensions/laplace_redux/index.html @@ -14,4 +14,4 @@ decision_threshold=0.9, max_iter=100 ) ce = generate_counterfactual(x, target, data, M, generator; convergence=conv) -plot(ce, alpha=0.1)

References

Daxberger, Erik, Agustinus Kristiadi, Alexander Immer, Runa Eschenhagen, Matthias Bauer, and Philipp Hennig. 2021. “Laplace Redux-Effortless Bayesian Deep Learning.” Advances in Neural Information Processing Systems 34.

Immer, Alexander, Maciej Korzepa, and Matthias Bauer. 2020. “Improving Predictions of Bayesian Neural Networks via Local Linearization.” https://arxiv.org/abs/2008.08400.

Schut, Lisa, Oscar Key, Rory Mc Grath, Luca Costabello, Bogdan Sacaleanu, Yarin Gal, et al. 2021. “Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertainties.” In International Conference on Artificial Intelligence and Statistics, 1756–64. PMLR.

+plot(ce, alpha=0.1)

References

Daxberger, Erik, Agustinus Kristiadi, Alexander Immer, Runa Eschenhagen, Matthias Bauer, and Philipp Hennig. 2021. “Laplace Redux-Effortless Bayesian Deep Learning.” Advances in Neural Information Processing Systems 34.

Immer, Alexander, Maciej Korzepa, and Matthias Bauer. 2020. “Improving Predictions of Bayesian Neural Networks via Local Linearization.” https://arxiv.org/abs/2008.08400.

Schut, Lisa, Oscar Key, Rory Mc Grath, Luca Costabello, Bogdan Sacaleanu, Yarin Gal, et al. 2021. “Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertainties.” In International Conference on Artificial Intelligence and Statistics, 1756–64. PMLR.

diff --git a/dev/extensions/neurotree/index.html b/dev/extensions/neurotree/index.html index 25deef647..6dd6d2cec 100644 --- a/dev/extensions/neurotree/index.html +++ b/dev/extensions/neurotree/index.html @@ -17,4 +17,4 @@ decision_threshold=0.9, max_iter=100 ) ce = generate_counterfactual(x, target, data, M, generator; convergence=conv) -plot(ce, alpha=0.1)

References

Grinsztajn, Léo, Edouard Oyallon, and Gaël Varoquaux. 2022. “Why Do Tree-Based Models Still Outperform Deep Learning on Tabular Data?” https://arxiv.org/abs/2207.08815.

+plot(ce, alpha=0.1)

References

Grinsztajn, Léo, Edouard Oyallon, and Gaël Varoquaux. 2022. “Why Do Tree-Based Models Still Outperform Deep Learning on Tabular Data?” https://arxiv.org/abs/2207.08815.

diff --git a/dev/how_to_guides/custom_generators/index.html b/dev/how_to_guides/custom_generators/index.html index 4cd21b37e..4caa3205f 100644 --- a/dev/how_to_guides/custom_generators/index.html +++ b/dev/how_to_guides/custom_generators/index.html @@ -55,4 +55,4 @@ x, target, counterfactual_data, M, generator; num_counterfactuals=5) -plot(ce)

+plot(ce)

diff --git a/dev/how_to_guides/custom_models/index.html b/dev/how_to_guides/custom_models/index.html index 47d0929b3..464c1d6bd 100644 --- a/dev/how_to_guides/custom_models/index.html +++ b/dev/how_to_guides/custom_models/index.html @@ -51,4 +51,4 @@ # Counterfactual search: generator = GenericGenerator() ce = generate_counterfactual(x, target, counterfactual_data, M, generator) -plot(ce)

References

Innes, Mike. 2018. “Flux: Elegant Machine Learning with Julia.” Journal of Open Source Software 3 (25): 602. https://doi.org/10.21105/joss.00602.

+plot(ce)

References

Innes, Mike. 2018. “Flux: Elegant Machine Learning with Julia.” Journal of Open Source Software 3 (25): 602. https://doi.org/10.21105/joss.00602.

diff --git a/dev/how_to_guides/index.html b/dev/how_to_guides/index.html index 57b147c18..3d3b1b6b6 100644 --- a/dev/how_to_guides/index.html +++ b/dev/how_to_guides/index.html @@ -1,2 +1,2 @@ -Overview · CounterfactualExplanations.jl
GitHub

How-To Guides

In this section, you will find a series of how-to-guides that showcase specific use cases of counterfactual explanations (CE).

How-to guides are directions that take the reader through the steps required to solve a real-world problem. How-to guides are goal-oriented.

— Diátaxis

In other words, you come here because you may have some particular problem in mind, would like to see how it can be solved using CE and then most likely head off again 🫡.

+Overview · CounterfactualExplanations.jl
GitHub

How-To Guides

In this section, you will find a series of how-to-guides that showcase specific use cases of counterfactual explanations (CE).

How-to guides are directions that take the reader through the steps required to solve a real-world problem. How-to guides are goal-oriented.

— Diátaxis

In other words, you come here because you may have some particular problem in mind, would like to see how it can be solved using CE and then most likely head off again 🫡.

diff --git a/dev/index.html b/dev/index.html index e0d77c9d4..0529e2132 100644 --- a/dev/index.html +++ b/dev/index.html @@ -43,4 +43,4 @@ author = {Patrick Altmeyer and Arie van Deursen and Cynthia C. s. Liem}, title = {Explaining Black-Box Models through Counterfactuals}, journal = {Proceedings of the JuliaCon Conferences} -}

📚 References

Altmeyer, Patrick, Giovan Angela, Aleksander Buszydlik, Karol Dobiczek, Arie van Deursen, and Cynthia CS Liem. 2023. “Endogenous Macrodynamics in Algorithmic Recourse.” In 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), 418–31. IEEE.

Antorán, Javier, Umang Bhatt, Tameem Adel, Adrian Weller, and José Miguel Hernández-Lobato. 2020. “Getting a Clue: A Method for Explaining Uncertainty Estimates.” https://arxiv.org/abs/2006.06848.

Joshi, Shalmali, Oluwasanmi Koyejo, Warut Vijitbenjaronk, Been Kim, and Joydeep Ghosh. 2019. “Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems.” https://arxiv.org/abs/1907.09615.

Kaggle. 2011. “Give Me Some Credit, Improve on the State of the Art in Credit Scoring by Predicting the Probability That Somebody Will Experience Financial Distress in the Next Two Years.” https://www.kaggle.com/c/GiveMeSomeCredit; Kaggle. https://www.kaggle.com/c/GiveMeSomeCredit.

Laugel, Thibault, Marie-Jeanne Lesot, Christophe Marsala, Xavier Renard, and Marcin Detyniecki. 2017. “Inverse Classification for Comparison-Based Interpretability in Machine Learning.” https://arxiv.org/abs/1712.08443.

Mothilal, Ramaravind K, Amit Sharma, and Chenhao Tan. 2020. “Explaining Machine Learning Classifiers Through Diverse Counterfactual Explanations.” In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 607–17. https://doi.org/10.1145/3351095.3372850.

Pawelczyk, Martin, Teresa Datta, Johannes van-den-Heuvel, Gjergji Kasneci, and Himabindu Lakkaraju. 2023. “Probabilistically Robust Recourse: Navigating the Trade-Offs Between Costs and Robustness in Algorithmic Recourse.” https://arxiv.org/abs/2203.06768.

Schut, Lisa, Oscar Key, Rory Mc Grath, Luca Costabello, Bogdan Sacaleanu, Yarin Gal, et al. 2021. “Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertainties.” In International Conference on Artificial Intelligence and Statistics, 1756–64. PMLR.

Tolomei, Gabriele, Fabrizio Silvestri, Andrew Haines, and Mounia Lalmas. 2017. “Interpretable Predictions of Tree-Based Ensembles via Actionable Feature Tweaking.” In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 465–74. https://doi.org/10.1145/3097983.3098039.

Wachter, Sandra, Brent Mittelstadt, and Chris Russell. 2017. “Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR.” Harv. JL & Tech. 31: 841. https://doi.org/10.2139/ssrn.3063289.

+}

📚 References

Altmeyer, Patrick, Giovan Angela, Aleksander Buszydlik, Karol Dobiczek, Arie van Deursen, and Cynthia CS Liem. 2023. “Endogenous Macrodynamics in Algorithmic Recourse.” In 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), 418–31. IEEE.

Antorán, Javier, Umang Bhatt, Tameem Adel, Adrian Weller, and José Miguel Hernández-Lobato. 2020. “Getting a Clue: A Method for Explaining Uncertainty Estimates.” https://arxiv.org/abs/2006.06848.

Joshi, Shalmali, Oluwasanmi Koyejo, Warut Vijitbenjaronk, Been Kim, and Joydeep Ghosh. 2019. “Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems.” https://arxiv.org/abs/1907.09615.

Kaggle. 2011. “Give Me Some Credit, Improve on the State of the Art in Credit Scoring by Predicting the Probability That Somebody Will Experience Financial Distress in the Next Two Years.” https://www.kaggle.com/c/GiveMeSomeCredit; Kaggle. https://www.kaggle.com/c/GiveMeSomeCredit.

Laugel, Thibault, Marie-Jeanne Lesot, Christophe Marsala, Xavier Renard, and Marcin Detyniecki. 2017. “Inverse Classification for Comparison-Based Interpretability in Machine Learning.” https://arxiv.org/abs/1712.08443.

Mothilal, Ramaravind K, Amit Sharma, and Chenhao Tan. 2020. “Explaining Machine Learning Classifiers Through Diverse Counterfactual Explanations.” In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 607–17. https://doi.org/10.1145/3351095.3372850.

Pawelczyk, Martin, Teresa Datta, Johannes van-den-Heuvel, Gjergji Kasneci, and Himabindu Lakkaraju. 2023. “Probabilistically Robust Recourse: Navigating the Trade-Offs Between Costs and Robustness in Algorithmic Recourse.” https://arxiv.org/abs/2203.06768.

Schut, Lisa, Oscar Key, Rory Mc Grath, Luca Costabello, Bogdan Sacaleanu, Yarin Gal, et al. 2021. “Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertainties.” In International Conference on Artificial Intelligence and Statistics, 1756–64. PMLR.

Tolomei, Gabriele, Fabrizio Silvestri, Andrew Haines, and Mounia Lalmas. 2017. “Interpretable Predictions of Tree-Based Ensembles via Actionable Feature Tweaking.” In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 465–74. https://doi.org/10.1145/3097983.3098039.

Wachter, Sandra, Brent Mittelstadt, and Chris Russell. 2017. “Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR.” Harv. JL & Tech. 31: 841. https://doi.org/10.2139/ssrn.3063289.

diff --git a/dev/objects.inv b/dev/objects.inv index b09cfb6a1cd69d34f5a777510f6b0e9bdaebb8d3..8d9f1e0d5866b24fe3ac87ab827acc68ef3feb11 100644 GIT binary patch delta 6874 zcmV<08YShlHm^32k$*i+Z{#+X_xcqaWDRV9maDt$q?5)R#%;Slf_ArII|(pBAkZb+ zrEZl}kW}vW1i1_b*tzV^E|SAy=8!oou-MD)b$?I(z~(3Hnn;Vkegyg-{ zJPVhri1_3KrvK{uscoj{oc%_}{I>+`{UXeAXqn~7N)JkIF-2d~U&K47%`cL5DuD`c zj{Q|iR%tRPSr*3EdVtM#Q}hDIK}35T!OJ?#Is7L|uGfaKs%@qW1Uf8!OTd;%K%z_! zgujyrIbjd#y??uCsWqB%Fj{J^Nla1}_ZoSPudBfRp{1g91-%eJr#5k5V8b8uz`yEe{gtr-A& z_2sb%P!P^-0MTY3Ey6=1peusua{2jtg5Oy{RN7C`Gyd~}{xYskif_W4J;ssE8p-Wp zrR1Mx9Djraw^qPuLkgAtYX+CRhrRA+s|86(W(8O2K@qSA=lr(`5h#dDn}uZUV5gnE zd1_qBl-z`w4KUttiq4L|dU>vbOICTfgqIdORnye4Q>B(~vn93wNSjU37w0Dj3LsEA zuNyi^Z6QVg{7}oa2-vjXm@uP z;UGZ#6{=q8u{4!>OS1-a{ut&lCvirVmQE>@@8lIMAI4;zCOPaFL#Oin6uqW@UcjFU zMv#yZo)bSM!PK3R4<UclDl zKhbOYb5hto-gA|oB76y!mC0oK=pVF3(=%9J3M*d_?=VU-lI7lU9Fyg(3e!@M0Tuxf z0YoGqA`Wr#JOQFSpu&q_?{XiO z@;D(mrUtWjm89NbvW|0-USW8LBTqKoOjE4bet@KahX5W@;i<;pOX_$m3A_*Lcufln z;D7%2fBV1x{R@&h*O5qz`;ELUcHc(d{^KIgSJ{*4^g7HJ>&xAFvYdXgjzWBq!hiOE zpQQQX_Tc5T*6ri^6dQKoM?0T9t##e`cd#>i2LG>c!;dC*cvuHA`yACC zO(YZ63u6z)zs$Vk$^&yC2w`4~>m?h)FrN|}6vl|bLCF;=;Y_&RD`(YeTL_&A@V&{4 z24liz4~F2KQrD5teV2KsctvVt zHiQzuEczm;FJgT`DCE>3Vq|+Kl|Io-XWhSJ0)X{q1Q%Z85xELu3i2p~>CVF3PP{OS z6hIc;QQ3WRe02pom@fG#0jb&(An#2VTJ)ZLMd89sesl9^7ykFHH+wkS_kSMldyhU* zAs@atc>VnN)tl$jkBUCP_q-RdyWCH_M+Hpeq3SJTj69*-;2p9B*G{zsXXFXf;2cOX z!0r}F*2%=fc>pt#TF8gpA9X@9`lnH?IS6-`lz z7V(C)Jq22Bxx#6fnTJ_ZXWr!C28YlNkQx}m9AAR|Wp9Z?7`3n+Qo{F)Jv473;;T=* z=ODOYX1w!c3DW`gLYUX(4{u5(kgjn73DK7@afrS|{N?0~fS~$1Nu$7H!A=8-wZJG) zSasxq)>BPf&UUP&GJghMpY_ncw0XselnmO~+TJX9sp#0)(K@5=Ek&MJW)}{pD(lJ8 z9PF*B@JO<0j<2U2@W)0`p}H?xanWuSHpzpBtpvjRf<*Weh=l(NWV6q8vWdS`B@@-i z7m4?r>Do3m%oQ)RMZE2#P~c)e+X zB&E;5Elk_}Jxl+ADt-nWap3e>1$-jNqn5y2Aw2Wm!$ufod{nSe4K|{%H8x4j;yWkQ zb2eeDWzw2c&wsnu#L(WR#L!+Ph8{icC5HC4DTelBF$9DvVrWlx3?A2JZnmkJn@KbG zX|I`^ZFA;k%FKPL&fH9%xlbn-VQ}lcS?8;DZYPas2gEx}PE(KvDGp;WrrRy^hU~3+ z7G`zKv7eJSSASgcoi9jq)jNc!^p=GzwLZK< zB^PXEfYox-Wo)z4j8pUFa&(g!axdcyU`85U!Xk`+{Vyc9m^{T?;k~M1XR$S? zQoW~%G+<3A(T1c@dSwGd`#4%(ZT1yX|wui
N)+Qi=|AGDMHlUIVf`2xm z{;(haRRxUPOiIIe&LY<$qy_>5Y;+jz!3t&@*8@rRo4ABr#M27MsDM1#D~-&aHs)Sw zTu#oEY9Z5H)uUO)pi_&8*i%Hrm?EgzDi%3tR$RRldnhYyocalJYz#bgMLe(O6E>vg)GF0GwuD)DL8s%Rg*+*<2V> zNsj@Qb_U_mR)*5zmQv&PR5l29@oE*_`a!Y;mFt72moep=XWqpc@O-DjqTL^$gG<_I zo*$$szWpA(jh-w?zDNS3EPt3oDs~<}ZPEcEeP{Mip=IasQL7QWTQT~sK57DE$!}K# zHLV6S&^MQF3GgFq^W6Ygf(AeQD2%Vf3>Fj`Aq<(pcAPL@ome~Ps<}uh;VZMA=X5jq zKxM+*F3Uty4lx-pMjR4bH?0rpu_p;4-$%6}pJE^VS(m8h7M zA!F|X3d8QZWK5WDVVxvAE*57+i+g~#t;jEXmG**zJ!dAQZG25qMoQ?6Wz-q#(?Jwk zzZb89c}?=)bkAbBBWNkqG8-~$4Asu;kWSd{AZ$D~5HtNhLCNj%Pcc?eBcP*U*k-BhIP?Nh;$G z-_gSACB7;##GIVqJPkiQKwrkt_&cy}Ki#k9;#rj79K2Cz5?{l=x68{U`u;)9?o*X4 z_4P;~q<|VyEC)JdDqDvEv&jd-CyfmTp0VMm^T>w8AMs&{zkk`#U@rO3F6}K1lD{wv zv~zKI{ll1zB(Yd6f$2sOYJu7)!m8GHw~W^blD-17F-cqW;Sie5_<&lDu|qTV(F?y5 zilN9}W-B}=g`A0n#>4Y886+$AADY2a$q|qDGTd-S3uqf2ktdY;{KF@n_nj zBNn~}T)(G!s7ClJ48XbGgo|iIdxNDVJ=6V6d-{&LPP82#w3+@+@L+vzAtvF2dq2gPCJZN;~n zIGx0|C4Y9NqM6P_S}| z?+^+tJqVUyyTNG)TlyR%TxBL5g-f0YW@YZwaIS@~w&O4KCUA&3+|0UMzXv=?|EP;`UzvLA-U8b}gt1kW;=@e)wX!Q*=0qXKs59YZFR(Z6vuWRZ7=2OXs)VwKD3#~O)!z6J(%x%Ro7BX6L z(s+d(^rrka#%LUQnOW<@bV3%u-#&N_X|mcD)0q&vVLC%8IOR!d7Aj)7PEXU1%?*cJ`L@?g8b}= zzOEs{<7EgxNU#b0R);=Ht;wk;bD3_LjAbpp-$EL!t4g<)Q5CI^TqVW=vgH&)6X9l3 z2ugs6h?t`oA=f-(qfRPNNvBv5c7|hcM4lxe7xIl25gCw-atX?smO-Ymlz$Z{Zyp&0 zNS3bUmVt--6AYPr8O-n~V_zgT@yDj|!Y(M_C!cJNB)YMf+K>+84@s)27mB#a+UZ~p zo9~b&cKsk+Zg`4oc!tet7z3~2DEtFGdkea7BY}}R5;p-6X|LyrPZOJ0DP9))kt;S9 z`+b0(CP}3J*Xj6LDXGeq%70ie`-9S9{%iQOWI0*UdvF(coo&c+(xPxtjCoCRpMLx0 zCJbogVIE;%^{bGi11dB}=(R2~dKPE)7a_e9g{*jm+2N%l1h32LlKYSZ>>#m6eQK2~ z{OaI%Y=zalRJT687$}2f4(bN{9TmQsNxO>@*xw~Zu*db8mKtE~aeo%skB*Y-Fb8z- z!%?Qf0=g5Q-O?R$kT2i1E9HRXA#Rv#K~t$MN0g~luqi?1+Ig(~ZpLm?dRn;QPNEEV z9#n8}*mn&o$B)8yBnlTvQb=T82%(;HXe{KQw5t{$jIgL-sGY6QFP^QL(-N#4tInj# zUN?SG17MHykcy%naevyX>yW+Mc)^HyGlR34HRG$=R%fE|UZ3OW7)hgk;a78fqi);R zPf8DRH)_x<4@S?GSDSG2bQoL&$t$rST|YGV$ZP}X5wlwgr%CJ&`JWLI(SSwe+Qy$f z)Ozg_%i;z&Xtu3NT{8u4Yj$Ph=>&gx3C-lou!l%<_O%?JMSmwYk}#@+ca<@6SQ{c2 z!F8A7kMXa&ShP{T06xmWL+qiR-VS=`MVJegJ`gftf~{#RtH2{{P5B z4tgAYZ?@1}Y!YF5iSefHHEBY()ZX=w_%_1s?&7q(P`J8Hb8Dr0dc=;a_gZfJaWk0K zD}DwLtQq{61Al)!5o9*VEFplo&{woK$6nN0Q#TK|fO+5SN8$f$AQ)gz7cIzcP^Nk_ zZUP4zwDR4V_e_$nlAO8aksp)rdU2Vo)8;{Nm*lr2bz-VY`;~sCJ`OPb)WHVUq6N{Y zl~Jx|8xh~o!0B-&iWLC}?B&rq#|lEi0hE80$~B(06P5!_RL zj*aEFQpDpLlqTK4O(8n<(YQlaY$<4LfRp?CaCNT(pIyhdn#1|^#6IO>DSkR|kddj= zMLky3oqyUIGv}S>&LaA$`&*wLn8GQ`4bmc#O)895lrAs%k;Aw2+fYC!NyW#Ze3-5I zyTP&CrsbWH2#jRE&Hm+kdT?ad4rhzX=oRCwCXc&H-E~d6T$%%|*h)`(fs>A+k&3aD zo1l&Ib$ZEL+>>Bj8*y8U%86&2yNF+dz(uI&h=0sO8qlk!%$!>N1|-9xDcVDA54V1d z?%{@VE$xNoj&3@!7Ta9&!+-NL9A6h_C+dgz?nJH^_CZOUe6G)U+|TkzdQ~64jVz58 zrRz+M@Uuol4`m3Q(L*`ZI5v4K-%!N4B-~!YwSm}ep0%N^`7GZb!$YQLJBEj_kJ)v* zgMV2^mZ^<9FD?7Mlma_vz{i`vBD0O^_Q|)T+ufC#R+XLHCEIE-r|<%Y)NiU$0d8t- z>kC+QWG;+5Rx7=gi|S=e?ry5z)YLngWy*#^R)ujuJ~&9c>p}L-v-il{)$?=rBWtJ2 z%DK(hX|8~Iox~0+eY8%)qpiCsE=?6F{eM*=SQa1lZOuclxa!Ru>+c0IAt`HgTaR4m zI<((~P!ZCuQgx6!k5gznbe5uwhm)=WaXk#ODkxaLq%qHRQaR{i=L}` z7EtO<6*pp~4R()$5*$Svxi#xB!l!3%4$d5p+5pyOz%^e^x{`0OP!)6dDvSx<$W`+) z?S60Q*rJ1#><%sM1!Syr`$)ma=B>Iw{5n3vEyBQ0jyWO?^eD5CT^FoNDq#%JreHYEajg z8D6d;ea(dX@>3Pv=*rV3hJRK0iz!DRj4^*1z1^@nZpAuGHIM!_&|Q5NXh?p2kSXan z?>^Xo>|GH|_?35osoW9zznX#oxUF6vLdzv#dy{?+3uG4aqWRnB1^v`ha zI;^XO+A~`ekSh#EnPuX)1*TlkQx*sp7v+ylV$y&+OqAY*8Eeofuz$zhk;^k1Bh!q! z#Qzf4ugTnatwD*>NwI+r!&RCrMS0A&2iHLDviP$z*M4tO?Bj>r0PLuwy{Ba%LT%M~ z(%+xY6VMQjujK7-ddP7#BXwnHr^ly4x?Pc89}2NI`{i=pja5?`kaSCDiy7=;ms(BX zxFFXYNAoqk*v1EAqkoaO-jsyUV%t<#W((F3FdVRJ0q8jXRKe zY-F?dESqq+R({Xd-Ca6v+RMZ%4J@6en`- z{>FzUA=|z2L1W)-*__mtn}6z9`OEmgnr-rV5qCc^K9;(4@O#W=tp5cyYMPcAd{ z&gpOOi2W9bypv+M*Pi zGBVoMT;l|12!0t_RKFKm^|E>>P{KDTUZWyP6FsH+{nR_+e*>+f533j@^tnjw5-0P; z5@qj<6DWU{n0n8}@&=kKl%Y7rarB3lW-5iI-l_aRpf^tw*cUKDbHh9c1*YCX{bPdb z1~&Jd__E%4;!~8(7g2%d#X7^LnqnE!iTOKpjlF&0sYFtk=ajQFQPvbSv{NWa|BU7q zGTV>JQL~$OYRgk49v!Gd(g=vV<>Din8Gc$kLwQ)?9VnFQGmq#W-VKlcPM2dm5 zhz^W|t}r6g6%_9=dS^jVDnIpJu%8#?mvMG7bQ2ZyF^X;GNGuOCCH^$?z&O0MB2Eet zs^niYx%55k^&np@aE5a$x>60ofIc{9zfFulL0sBQBrOL!?d;8S<5Xt&CdzGyv4T_Y z?C5{1<8zr@x+7N>uzENJ$oZfD>YbIpC!iXOEV^75Qu+ERsqnYI6Q#)?Oi-TV zg6xbO0N-5R@#;HL`-i`J`(YSynRZsbf`-Nz<=9WsLM9@;fUU=VdaudPNr`>@m-2sl ziqRz?D-&e;=pVG8=><$Lp;aufe-NiR&I|u2N$~PkrfDh2fD5PS0Y#5f^ccm-%M=>r zLo56k_AZmsO51_JFF*au|9G^h2Sq)h=!9bje_ur%+IC8~0|;x8#|bVF0nGkYn)wIm zIw^2=h2R~EeG$BwhFHRWz@((-0X=_DrYD2JV`6wLF}x2|zNU!<@<0Fkzy06;{>2k2 zt4xH+{YKmtyKm!f|8Y?itNht?dL0#u_2uq7T~5DP#}T^7VEeyMvtn_(e>_#HeXI_# zVHbY1^T~6i>dq&JJG0Ly*PX}vJF^$?|C%@aXmW>#Dv{Y!PkA&EAgmuHKJPl1jXB_5^YrB_}>XTtPeHL6zIg6mA6?@vBY-}^7(^#^MB`0o*H zfG~t4?d`%l%Wd!b&q6LJ4S=bv>c3N9%%6 zD2PEso9&;}`b0CFs(nWV0Ih#b8(dnAhxjT=2+6|;hC7c6JMlshDUd9ydsX$x(bW~~ zU^3;W7^G@Xf~+=~XkL5r6`>0w`OVFvUHIR({_Npw&wse*Kl((beDL}H>z7Bb-n^WC zRMvrY&uYPP7whr&UP;sQz4|R(7)44}gMUCLTszbjE2BuM24_Ud5kG$-GU1uMMEi(V zfKA$3<(Hp+{BMtu?h4}vd&-xE9yOna~f1h zoq0t~e)4sOR;#h(OClQj7wcsB)?$^Vodv@V!vs3yYMsnA^IGE`vg~cW##nE8J+TOuNWk7R#~rZ0lWg(%{>u2$>X zE1wM#EY1wG6>Q%Cf5N?;&-|AlMqy<9^K=Qr0opq%RB6D95_x~1D_m0Y$V(VFkG%BQ z%gGrAvGH}9#i37KkA@~HZc1ojU6HvxPc>dL+rCz+5hU)chc+a|>y=2FK?+-=zj7^= z6{(cJNd1>oez%EsCd{F%$I4DhrNCFE-PS6V0IxJTO##q5H&YA(4n zD-d2ui2ky@MB;xHiBh(MqDL@&Y32z?JE_12jx&Iy@_b7!{)AkhF9-*z#a|(($ z$HW>laa5u+&7o)26_EB=YtOtTJGrPW3KUIcVj@rhQ(}M0jqQ9|pzA3k{Be!qo~Y_n zw9Fd_@NuWX(`+6*Y;7RCFPjK|LL=e7oZ0ND*w7MgvX>&*e4>opY*QmQ6GraSUL!Z# z=E%*Yk^590xtTa}pH42K@Ya8`E>`QpP8yL49;+}p%|ISxC`tgJwWGa;?5%5-P_G8n z(l#Cjs-=J3nk(yLPcVO;TAM$6lKJygF@N?X^XF*|MowNIpI?~a2YX!s4*GNQ=8DPM z^97EtdOO0T+M-w|TH8O6BAQmfMTxLh9=cBWBS(Ej9~}ii$4L$}V~zHrezL#*mnW8( zJV$vn_g`>$iQ_YYqAb|OYe`wgKSZl&WPn(5U(v{yocpeM^BqO4qFl9*;REVnJcti4T?_(qC?vQtT+mwHI zdy>3+D$Bb)8RUPNGfz>SnJKUAAOG2-MLb?)A}7KOjY0bpbJrTJQ0I5{tUF@9!} z@$;!-{LH?UjY=s^sV`0v6P}n)`1H7+Zxqeb%LpZWTb~2a2N`+mpP-e$Ulbr}dZ8|v zoL7?bfSkw3O&ll-jg~lGF4qNxy=8yCPCs*}s$b{(90AAMVnHM-D*UuXPpdkouIZt| zBQ+;X)K?AwP}wAM1AD96+~De^^Z;EO>hI>WluauM&R|9*m9$bV~mhI}e~=O>KT^}&u%4&amH z047S(Ap(L}g^KJP0ZHiXyEA{+e6@a4CDt)HBIW+z8xX5`vCHv6lunzary;(Xn)80@ z6h4yk)2yz8BMr9jOjC3A0$9@;Prc@AP11x3g?8mtYLh1f!n z7bjPzZ!+VAlbalqH4h5iDvVzi79zi~TG^x8Hkj z<7Z1;EYi@ECd^?gcAh+MwgW`^&g`MomYpYutsr=}Ug*1ee-rejxLskdi5kpA-(0@M zupVKX?}o4>h}$TLqvU^zk6_tCBZVO&*v=EitCMTzST!dp!)#{O^MY)q09u(+$HG>Z zN_#7uEfqQ78`RFA+hB!2BI^1)a#F2WHagg6PPs;@lPZU-cVQDLS)zPUhV;FQD0I8; zlu=>2g?SSAxR{(3NhgA}ZPom$R)rTF>^XBo+WOaQ%18;F)9il-y7WmWO0C~(tpdEp z#c#StvDy(N`D&RD88ymkXZFZuq!f0xqXP1tkMR)^dU%+=;v~v>d7UPQ(G{&+M8qvX zlfs;choArSgQ5hhmk+9VYrSBdZc?EcJ4dsf&-QknK5l5ost{YMWs~HEJA6lT)QbaI zV(>9JK}8mQc;J71nLy$10NsANSC7SuI7J0mqfjKdhJSCDmudX{0|ocVN|xAqIOI}5 zb}3dJ9Ws=y-GI?#9btn;I|I*XcjR%TU1E<|xA@<*YcQ7VXP5RCI>}xbI@&ooELUaB zMiQ2o68d$c2(?6Q6k(lRyj%I}c$2;Y*qCN5`fx~_&HR6$wTxqjX6&OEe#aF<8TZXs zXpT!clW>iP=2<#OR_r}AlP8iR8E<8{Ssg8;ZPti5pu{E*DhG5ulptChjHiw|A4#^e zEryhWd>kNxAzI9hJ}Okv0p$w^_^H|07OTWTlZW6e@nl)0>jy!s3N5CWxuWeQh2##F zR*VEq-cWxFgJGxSfpoJ~H6(4es9>!fB{^OKHe+~kjJE1rO=B=+ed5owM@LM&LR`P6 zdZy#|Jx)_gTAo z;)0+5{KtiAZt#SQ=Ojcc*dAN4I6m9?Y_@2c9NK>-xw1GU^r#EAyF#XB+BTA{-7w-9 z^|#O1s`!3WCQ`!++=Ovd!C$tr%Y}Y;SLRaF^zAg11YdKw3c~U*^0xfjO`cBt+rn}t z=?gfB(_95GLXzO-g@)P+sh3qdTuBP!tFfkV73vc%NduD9X^)a(C|NN%dPoZ`EeIN5 zyFq_h1Y7zXBwP)X4x=Rt1k(co1UM`1tL@key&E_L4mYDN7ck!8ak5$$7a2-&k{|8H z{m4SlvIlZF4z--+*C|a|+VBxi|eXb$d!$6^FW}vOY%6H%t_czFoM*j13 z2@4qqOO}Dt1*Jt_b@|swqd?n&R@*@1pss&@^;kStQ4*?94ZGduiT5nV7ES1uyli_XN@&7w1;LT+XO8ko6Rj9O^~j0#K^{R+%f zr}ly^yJ7Z2OL~@8OFFfI^to6@wKt%90`P5jRSyD|{F#{3xQXR#WHV|R3*NU~;S{k0 z|JL|5wdr)fK1*zXTSJ@2JllHa^lYPf(?Pzz%&}S4fd*y~4;lql+qYhvihoTgg^V`+bh@3#;J>&ntqBC5Rfp{u}Hh_@Uxoh^e1iPE*&GVqXn zf*}{L(HZV#?90T)|JVdC^nwC*^2z2%q8r52#&p>F5Tu%X6^omwosNIzu0mHzoW!gbJOl31@?Do>FjZRrbQvFJ3cG6gK@ZCu2IXvpe)4Q#PK&d0tU41id)?SY4T3$+ zLrRKz#A&OpL-ua{1tZ4IOwMN1jIWffPDSIrKF86~lScl+ujcrMitW{t(u2$mg_`BT z=(+T26K0-{gA0Erc`X)%>xTv(nQag~q`0Ltn)v<@{}~|>4P4|^+xWAGT30TyEN_5= zV%y5pHMhWR&aMcaPSA&AC?;MbK17<+FB=8aJF$^~kxJfG#K>W8h?oS|or*u&Mrg9@ z&buFbddA7Y8DaES>`qwbn!G2bzot!hVe_*C2}#bvtAng_vMHou*z6R0ZeSMr(qBt+y>2OF4+7Dl6HM!KGDgnvUr zr^lHnngqbPS4Zm{D+qxHw5;MXSQm&^N|ey+XXu+CYl20mOTmJ%%cGMd%QQq;=q?M@ zGS<-%7Tk6mqbo`|?zH&=G;+o*>tLRySx6HQO96jLE@i*QnbPNmb5Hs?)|cPfBIeg1 zZPInzRdfkC{r{c}==lnggxa+Mf0VCtZt1D#l7~f;KAF*(ED+ zPn>aWjoX@6jz8PnMf@5DE?h;2cpedlo*FW9iuw&qhIv!8huR)){TSWD4aHj86U`mn zbYd;GndXQ87H25AF3(P=hxqP9u9x_rYMg&~uFrVf&+qrf^vqpFi zMGBqXL($bZHh3)FP{dYAw7r09gR$EfCtCN^t#=_OGlKcjXN(b z>m3V@oikX+o4+Ent<~*=Z`p2lS87^ib~2M}tI3?)3LKKZDSHJN)Y{e;uvBI)z#V_h zmEOul^&%&CH`Q-y=pBtRS%zFzMM;Q1I0(G!N%oDi_lVr7@wxjEwbMoA+(zs)SHQeZ z6Ni~TTBG67*3A@`P(=uTRScHphkaY)5Y(@FGe`P+K}<}l9NpF<7rF}VcOldbX;-N_ z%ALn4w3WI>R{2KSparKn#4tAXs`Y=HW{pb$HU+?~KWV1DFJSm`KSS>?s3<1q#5qre1&Do({+YhX)OskvO3}MvSyTJ{AUlG=lQ7WMu@3tqC}hg#=$1f zr}8axH|I~w{o1)!sVgC5VIn#YV|Qe2q7tMPHsMei*&UTWY2u<^RXr0Z^-zDsjaX@; z-6Np{#qmaN%{qwD>Din8GsnF)khSTsV#^6v@(mWMd<aYF?x~%o7g^(#4*< z0`L>i#2|96VO1fZF}Dn=xlHA?;~P(BDj5xy=AF1@ED~usnYuMd*Lg(MF_aTn2cWXZ z*8qbA6qO6dbjx^d@3X(11L}YJbHBF1VeOYY?>r`~ZI!E`ad2Nq?+@SHhZ{utf32tg zk?fOKUyS~6%R9>2p{{aJ*Oxh3u3~-8g!}YU znce8j(fCd^nsC(#~vM=pzOj0`jC0{=^-UX!`;T!V_G zlVk%OhpQ}I^7NQ(cdo(OW%6fdto`1E*vEIdLD*4BdrxG+U2T8$dD7pX&r{G4j<4kH z4?RS`nz412Yp2JjLb^?nULOjvhy7wY@5Ztz4NSVFv&9JZz@=70IL^s6NAY}3F185( zY%~Jbo01S(uA51KuU6Q}h-|3!D%{j{`HZxiOY+4d745*WaR)NXFrIBspcOpivA(gtNoC5jZ@$IMf=0If IKg~uR8i1%(ssI20 diff --git a/dev/reference/index.html b/dev/reference/index.html index 9c5861ac2..7377f941e 100644 --- a/dev/reference/index.html +++ b/dev/reference/index.html @@ -1,5 +1,5 @@ -🧐 Reference · CounterfactualExplanations.jl
GitHub

Reference

In this reference, you will find a detailed overview of the package API.

Reference guides are technical descriptions of the machinery and how to operate it. Reference material is information-oriented.

— Diátaxis

In other words, you come here because you want to take a very close look at the code 🧐.

Content

Exported functions

CounterfactualExplanations.CounterfactualExplanation — Method
function CounterfactualExplanation(;
+🧐 Reference · CounterfactualExplanations.jl
GitHub
Reference
In this reference, you will find a detailed overview of the package API.
Reference guides are technical descriptions of the machinery and how to operate it. Reference material is information-oriented.
— Diátaxis
In other words, you come here because you want to take a very close look at the code 🧐.
Content
Exported functions
CounterfactualExplanations.CounterfactualExplanation — Method
function CounterfactualExplanation(;
 	x::AbstractArray,
 	target::RawTargetType,
 	data::CounterfactualData,
@@ -8,14 +8,14 @@
 	num_counterfactuals::Int = 1,
 	initialization::Symbol = :add_perturbation,
     convergence::Union{AbstractConvergence,Symbol}=:decision_threshold,
-)
Outer method to construct a CounterfactualExplanation structure.
source
CounterfactualExplanations.generate_counterfactual — Method
generate_counterfactual(
     x::Base.Iterators.Zip,
     target::RawTargetType,
     data::CounterfactualData,
     M::Models.AbstractFittedModel,
     generator::AbstractGenerator;
     kwargs...,
-)
Overloads the generate_counterfactual method to accept a zip of factuals x and return a vector of counterfactuals.
source
CounterfactualExplanations.generate_counterfactual — Method
generate_counterfactual(
     x::Matrix,
     target::RawTargetType,
     data::CounterfactualData,
@@ -64,7 +64,7 @@
  CounterfactualExplanation
 Convergence: ✅ after 6 steps.
  CounterfactualExplanation
-Convergence: ✅ after 7 steps.
source
CounterfactualExplanations.generate_counterfactual — Method
generate_counterfactual(
     x::Matrix,
     target::RawTargetType,
     data::DataPreprocessing.CounterfactualData,
@@ -75,33 +75,33 @@
         decision_threshold=(1 / length(data.y_levels)), max_iter=1000
     ),
     kwrgs...,
-)
Overloads the generate_counterfactual for the GrowingSpheresGenerator generator.
source
CounterfactualExplanations.generate_counterfactual — Method
generate_counterfactual(
     x::Vector{<:Matrix},
     target::RawTargetType,
     data::CounterfactualData,
     M::Models.AbstractFittedModel,
     generator::AbstractGenerator;
     kwargs...,
-)
Overloads the generate_counterfactual method to accept a vector of factuals x and return a vector of counterfactuals.
source
CounterfactualExplanations.target_probs — Function
target_probs(
     ce::CounterfactualExplanation,
     x::Union{AbstractArray,Nothing}=nothing,
-)
Returns the predicted probability of the target class for x. If x is nothing, the predicted probability corresponding to the counterfactual value is returned.
source
CounterfactualExplanations.update! — Method
update!(ce::CounterfactualExplanation)
An important subroutine that updates the counterfactual explanation. It takes a snapshot of the current counterfactual search state and passes it to the generator. Based on the current state the generator generates perturbations. Various constraints are then applied to the proposed vector of feature perturbations. Finally, the counterfactual search state is updated.
source
CounterfactualExplanations.Convergence.DecisionThresholdConvergence — Type
DecisionThresholdConvergence
Convergence criterion based on the target class probability threshold. The search stops when the target class probability exceeds the predefined threshold.
Fields
  • decision_threshold::AbstractFloat: The predefined threshold for the target class probability.
  • max_iter::Int: The maximum number of iterations.
  • min_success_rate::AbstractFloat: The minimum success rate for the target class probability.
source
CounterfactualExplanations.Convergence.GeneratorConditionsConvergence — Type
GeneratorConditionsConvergence
Convergence criterion for counterfactual explanations based on the generator conditions. The search stops when the gradients of the search objective are below a certain threshold and the generator conditions are satisfied.
Fields
  • decision_threshold::AbstractFloat: The threshold for the decision probability.
  • gradient_tol::AbstractFloat: The tolerance for the gradients of the search objective.
  • max_iter::Int: The maximum number of iterations.
  • min_success_rate::AbstractFloat: The minimum success rate for the generator conditions (across counterfactuals).
source
CounterfactualExplanations.Convergence.converged — Function
converged(
-    convergence::DecisionThresholdConvergence,
-    ce::AbstractCounterfactualExplanation,
-    x::Union{AbstractArray,Nothing}=nothing,
-)
Checks if the counterfactual search has converged when the convergence criterion is the decision threshold.
source
CounterfactualExplanations.update! — Method
update!(ce::CounterfactualExplanation)
An important subroutine that updates the counterfactual explanation. It takes a snapshot of the current counterfactual search state and passes it to the generator. Based on the current state the generator generates perturbations. Various constraints are then applied to the proposed vector of feature perturbations. Finally, the counterfactual search state is updated.
source
CounterfactualExplanations.Convergence.DecisionThresholdConvergence — Type
DecisionThresholdConvergence
Convergence criterion based on the target class probability threshold. The search stops when the target class probability exceeds the predefined threshold.
Fields
  • decision_threshold::AbstractFloat: The predefined threshold for the target class probability.
  • max_iter::Int: The maximum number of iterations.
  • min_success_rate::AbstractFloat: The minimum success rate for the target class probability.
source
CounterfactualExplanations.Convergence.GeneratorConditionsConvergence — Type
GeneratorConditionsConvergence
Convergence criterion for counterfactual explanations based on the generator conditions. The search stops when the gradients of the search objective are below a certain threshold and the generator conditions are satisfied.
Fields
  • decision_threshold::AbstractFloat: The threshold for the decision probability.
  • gradient_tol::AbstractFloat: The tolerance for the gradients of the search objective.
  • max_iter::Int: The maximum number of iterations.
  • min_success_rate::AbstractFloat: The minimum success rate for the generator conditions (across counterfactuals).
source
CounterfactualExplanations.Convergence.converged — Function
converged(
     convergence::MaxIterConvergence,
     ce::AbstractCounterfactualExplanation,
     x::Union{AbstractArray,Nothing}=nothing,
-)
Checks if the counterfactual search has converged when the convergence criterion is maximum iterations. This means the counterfactual search will not terminate until the maximum number of iterations has been reached independently of the other convergence criteria.
source
CounterfactualExplanations.Convergence.converged — Function
converged(
+)
Checks if the counterfactual search has converged when the convergence criterion is maximum iterations. This means the counterfactual search will not terminate until the maximum number of iterations has been reached independently of the other convergence criteria.
source
CounterfactualExplanations.Convergence.converged — Function
converged(
     convergence::GeneratorConditionsConvergence,
     ce::AbstractCounterfactualExplanation,
     x::Union{AbstractArray,Nothing}=nothing,
-)
Checks if the counterfactual search has converged when the convergence criterion is generator_conditions.
source
CounterfactualExplanations.Convergence.converged — Function
converged(
     convergence::InvalidationRateConvergence,
     ce::AbstractCounterfactualExplanation,
     x::Union{AbstractArray,Nothing}=nothing,
-)
Checks if the counterfactual search has converged when the convergence criterion is invalidation rate.
source
CounterfactualExplanations.Convergence.hinge_loss — Method
hinge_loss(convergence::InvalidationRateConvergence, ce::AbstractCounterfactualExplanation)
Calculates the hinge loss of a counterfactual explanation.
Arguments
  • convergence::InvalidationRateConvergence: The convergence criterion to use.
  • ce::AbstractCounterfactualExplanation: The counterfactual explanation to calculate the hinge loss for.
Returns
The hinge loss of the counterfactual explanation.
source
CounterfactualExplanations.Convergence.invalidation_rate — Method
invalidation_rate(ce::AbstractCounterfactualExplanation)
Calculates the invalidation rate of a counterfactual explanation.
Arguments
  • ce::AbstractCounterfactualExplanation: The counterfactual explanation to calculate the invalidation rate for.
  • kwargs: Additional keyword arguments to pass to the function.
Returns
The invalidation rate of the counterfactual explanation.
source
CounterfactualExplanations.Convergence.converged — Function
converged(
+    convergence::DecisionThresholdConvergence,
+    ce::AbstractCounterfactualExplanation,
+    x::Union{AbstractArray,Nothing}=nothing,
+)
Checks if the counterfactual search has converged when the convergence criterion is the decision threshold.
source
CounterfactualExplanations.Convergence.hinge_loss — Method
hinge_loss(convergence::InvalidationRateConvergence, ce::AbstractCounterfactualExplanation)
Calculates the hinge loss of a counterfactual explanation.
Arguments
  • convergence::InvalidationRateConvergence: The convergence criterion to use.
  • ce::AbstractCounterfactualExplanation: The counterfactual explanation to calculate the hinge loss for.
Returns
The hinge loss of the counterfactual explanation.
source
CounterfactualExplanations.Convergence.invalidation_rate — Method
invalidation_rate(ce::AbstractCounterfactualExplanation)
Calculates the invalidation rate of a counterfactual explanation.
Arguments
  • ce::AbstractCounterfactualExplanation: The counterfactual explanation to calculate the invalidation rate for.
  • kwargs: Additional keyword arguments to pass to the function.
Returns
The invalidation rate of the counterfactual explanation.
source
CounterfactualExplanations.Evaluation.benchmark — Method
benchmark(
     data::CounterfactualData;
     models::Dict{<:Any,<:Any}=standard_models_catalogue,
     generators::Union{Nothing,Dict{<:Any,<:AbstractGenerator}}=nothing,
@@ -113,7 +113,7 @@
     store_ce::Bool=false,
     parallelizer::Union{Nothing,AbstractParallelizer}=nothing,
     kwrgs...,
-)
Runs the benchmarking exercise as follows:
  1. Randomly choose a factual and target label unless specified. 
  2. If no pretrained models are provided, it is assumed that a dictionary of callable model objects is provided (by default using the standard_models_catalogue). 
  3. Each of these models is then trained on the data. 
  4. For each model separately choose n_individuals randomly from the non-target (factual) class. For each generator create a benchmark as in benchmark(xs::Union{AbstractArray,Base.Iterators.Zip}).
  5. Finally, concatenate the results.
If vertical_splits is specified to an integer, the computations are split vertically into vertical_splits chunks. In this case, the results are stored in a temporary directory and concatenated afterwards. 
source
CounterfactualExplanations.Evaluation.benchmark — Method
benchmark(
+)
Runs the benchmarking exercise as follows:
  1. Randomly choose a factual and target label unless specified. 
  2. If no pretrained models are provided, it is assumed that a dictionary of callable model objects is provided (by default using the standard_models_catalogue). 
  3. Each of these models is then trained on the data. 
  4. For each model separately choose n_individuals randomly from the non-target (factual) class. For each generator create a benchmark as in benchmark(xs::Union{AbstractArray,Base.Iterators.Zip}).
  5. Finally, concatenate the results.
If vertical_splits is specified to an integer, the computations are split vertically into vertical_splits chunks. In this case, the results are stored in a temporary directory and concatenated afterwards. 
source
CounterfactualExplanations.Evaluation.benchmark — Method
benchmark(
     x::Union{AbstractArray,Base.Iterators.Zip},
     target::RawTargetType,
     data::CounterfactualData;
@@ -126,19 +126,19 @@
     store_ce::Bool=false,
     parallelizer::Union{Nothing,AbstractParallelizer}=nothing,
     kwrgs...,
-)
First generates counterfactual explanations for factual x, the target and data using each of the provided models and generators. Then generates a Benchmark for the vector of counterfactual explanations as in benchmark(counterfactual_explanations::Vector{CounterfactualExplanation}).
source
CounterfactualExplanations.Evaluation.benchmark — Method
benchmark(
     counterfactual_explanations::Vector{CounterfactualExplanation};
     meta_data::Union{Nothing,<:Vector{<:Dict}}=nothing,
     measure::Union{Function,Vector{Function}}=default_measures,
     store_ce::Bool=false,
-)
Generates a Benchmark for a vector of counterfactual explanations. Optionally meta_data describing each individual counterfactual explanation can be supplied. This should be a vector of dictionaries of the same length as the vector of counterfactuals. If no meta_data is supplied, it will be automatically inferred. All measure functions are applied to each counterfactual explanation. If store_ce=true, the counterfactual explanations are stored in the benchmark.
source
CounterfactualExplanations.Evaluation.evaluate — Function
evaluate(
+)
Generates a Benchmark for a vector of counterfactual explanations. Optionally meta_data describing each individual counterfactual explanation can be supplied. This should be a vector of dictionaries of the same length as the vector of counterfactuals. If no meta_data is supplied, it will be automatically inferred. All measure functions are applied to each counterfactual explanation. If store_ce=true, the counterfactual explanations are stored in the benchmark.
source
CounterfactualExplanations.Evaluation.evaluate — Function
evaluate(
     ce::CounterfactualExplanation;
     measure::Union{Function,Vector{Function}}=default_measures,
     agg::Function=mean,
     report_each::Bool=false,
     output_format::Symbol=:Vector,
     pivot_longer::Bool=true
-)
Just computes evaluation measures for the counterfactual explanation. By default, no meta data is reported. For report_meta=true, meta data is automatically inferred, unless this overwritten by meta_data. The optional meta_data argument should be a vector of dictionaries of the same length as the vector of counterfactual explanations. 
source
CounterfactualExplanations.Evaluation.validity — Method
validity(ce::CounterfactualExplanation; γ=0.5)
Checks of the counterfactual search has been successful with respect to the probability threshold γ. In case multiple counterfactuals were generated, the function returns the proportion of successful counterfactuals.
source
CounterfactualExplanations.DataPreprocessing.CounterfactualData — Method
CounterfactualData(
+)
Just computes evaluation measures for the counterfactual explanation. By default, no meta data is reported. For report_meta=true, meta data is automatically inferred, unless this overwritten by meta_data. The optional meta_data argument should be a vector of dictionaries of the same length as the vector of counterfactual explanations. 
source
CounterfactualExplanations.Evaluation.validity — Method
validity(ce::CounterfactualExplanation; γ=0.5)
Checks of the counterfactual search has been successful with respect to the probability threshold γ. In case multiple counterfactuals were generated, the function returns the proportion of successful counterfactuals.
source
CounterfactualExplanations.DataPreprocessing.CounterfactualData — Method
CounterfactualData(
     X::AbstractMatrix,
     y::RawOutputArrayType;
     mutability::Union{Vector{Symbol},Nothing}=nothing,
@@ -149,54 +149,54 @@
 )
This outer constructor method prepares features X and labels y to be used with the package. Mutability and domain constraints can be added for the features. The function also accepts arguments that specify which features are categorical and which are continues. These arguments are currently not used. 
Examples
using CounterfactualExplanations.Data
 x, y = toy_data_linear()
 X = hcat(x...)
-counterfactual_data = CounterfactualData(X,y')
source
CounterfactualExplanations.Models.DecisionTreeModel — Method
DecisionTreeModel(data::CounterfactualData; kwargs...)
Constructs a new TreeModel object wrapped around a decision tree from the data in a CounterfactualData object. Not called by the user directly.
Arguments
  • data::CounterfactualData: The CounterfactualData object containing the data to be used for training the model.
Returns
  • model::TreeModel: A TreeModel object.
source
CounterfactualExplanations.Models.Linear — Method
Linear(data::CounterfactualData; kwargs...)
Constructs a model with one linear layer. If the output is binary, this corresponds to logistic regression, since model outputs are passed through the sigmoid function. If the output is multi-class, this corresponds to multinomial logistic regression, since model outputs are passed through the softmax function.
source
CounterfactualExplanations.Models.RandomForestModel — Method
RandomForestModel(data::CounterfactualData; kwargs...)
Constructs a new TreeModel object wrapped around a random forest from the data in a CounterfactualData object. Not called by the user directly.
Arguments
  • data::CounterfactualData: The CounterfactualData object containing the data to be used for training the model.
Returns
  • model::TreeModel: A TreeModel object.
source
CounterfactualExplanations.Models.DecisionTreeModel — Method
DecisionTreeModel(data::CounterfactualData; kwargs...)
Constructs a new TreeModel object wrapped around a decision tree from the data in a CounterfactualData object. Not called by the user directly.
Arguments
  • data::CounterfactualData: The CounterfactualData object containing the data to be used for training the model.
Returns
  • model::TreeModel: A TreeModel object.
source
CounterfactualExplanations.Models.Linear — Method
Linear(data::CounterfactualData; kwargs...)
Constructs a model with one linear layer. If the output is binary, this corresponds to logistic regression, since model outputs are passed through the sigmoid function. If the output is multi-class, this corresponds to multinomial logistic regression, since model outputs are passed through the softmax function.
source
CounterfactualExplanations.Models.RandomForestModel — Method
RandomForestModel(data::CounterfactualData; kwargs...)
Constructs a new TreeModel object wrapped around a random forest from the data in a CounterfactualData object. Not called by the user directly.
Arguments
  • data::CounterfactualData: The CounterfactualData object containing the data to be used for training the model.
Returns
  • model::TreeModel: A TreeModel object.
source
CounterfactualExplanations.Models.fit_model — Function
fit_model(
     counterfactual_data::CounterfactualData, model::Symbol=:MLP;
     kwrgs...
-)
Fits one of the available default models to the counterfactual_data. The model argument can be used to specify the desired model. The available values correspond to the keys of the all_models_catalogue dictionary.
source
CounterfactualExplanations.Models.logits — Method
logits(M::AbstractFittedModel, X::AbstractArray)
Generic method that is compulsory for all models. It returns the raw model predictions. In classification this is sometimes referred to as logits: the non-normalized predictions that are fed into a link function to produce predicted probabilities. In regression (not currently implemented) raw outputs typically correspond to final outputs. In other words, there is typically no normalization involved.
source
CounterfactualExplanations.Models.logits — Method
logits(M::TreeModel, X::AbstractArray)
Calculates the logit scores output by the model M for the input data X.
Arguments
  • M::TreeModel: The model selected by the user.
  • X::AbstractArray: The feature vector for which the logit scores are calculated.
Returns
  • logits::Matrix: A matrix of logits for each output class for each data point in X.
Example
logits = Models.logits(M, x) # calculates the logit scores for each output class for the data point x
source
CounterfactualExplanations.Models.model_evaluation — Method
model_evaluation(M::AbstractFittedModel, test_data::CounterfactualData)
Helper function to compute F-Score for AbstractFittedModel on a (test) data set. By default, it computes the accuracy. Any other measure, e.g. from the StatisticalMeasures package, can be passed as an argument. Currently, only measures applicable to classification tasks are supported.
source
CounterfactualExplanations.Models.predict_label — Method
predict_label(M::TreeModel, X::AbstractArray)
Returns the predicted label for X.
Arguments
  • M::TreeModel: The model selected by the user.
  • X::AbstractArray: The input array for which the label is predicted.
Returns
  • labels::AbstractArray: The predicted label for each data point in X.
Example
label = Models.predict_label(M, x) # returns the predicted label for each data point in x
source
CounterfactualExplanations.Models.predict_proba — Method
predict_proba(M::AbstractFittedModel, counterfactual_data::CounterfactualData, X::Union{Nothing,AbstractArray})
Returns the predicted output probabilities for a given model M, data set counterfactual_data and input data X.
source
CounterfactualExplanations.Models.probs — Method
probs(M::AbstractFittedModel, X::AbstractArray)
Generic method that is compulsory for all models. It returns the normalized model predictions, so the predicted probabilities in the case of classification. In regression (not currently implemented) this method is redundant. 
source
CounterfactualExplanations.Models.probs — Method
probs(M::TreeModel, X::AbstractArray{<:Number, 2})
Calculates the probability scores for each output class for the two-dimensional input data matrix X.
Arguments
  • M::TreeModel: The TreeModel.
  • X::AbstractArray: The feature vector for which the predictions are made.
Returns
  • p::Matrix: A matrix of probability scores for each output class for each data point in X.
Example
probabilities = Models.probs(M, X) # calculates the probability scores for each output class for each data point in X.
source
CounterfactualExplanations.Models.probs — Method
probs(M::TreeModel, X::AbstractArray{<:Number, 1})
Works the same way as the probs(M::TreeModel, X::AbstractArray{<:Number, 2}) method above, but handles 1-dimensional rather than 2-dimensional input data.
source
CounterfactualExplanations.Generators.FeatureTweakGenerator — Method
FeatureTweakGenerator(; penalty::Union{Nothing,Function,Vector{Function}}=Objectives.distance_l2, ϵ::AbstractFloat=0.1)
Constructs a new Feature Tweak Generator object.
Uses the L2-norm as the penalty to measure the distance between the counterfactual and the factual. According to the paper by Tolomei et al., another recommended choice for the penalty in addition to the L2-norm is the L0-norm. The L0-norm simply minimizes the number of features that are changed through the tweak.
Arguments
  • penalty::Union{Nothing,Function,Vector{Function}}: The penalty function to use for the generator. Defaults to distance_l2.
  • ϵ::AbstractFloat: The tolerance value for the feature tweaks. Described at length in Tolomei et al. (https://arxiv.org/pdf/1706.06691.pdf). Defaults to 0.1.
Returns
  • generator::FeatureTweakGenerator: A non-gradient-based generator that can be used to generate counterfactuals using the feature tweak method.
source
CounterfactualExplanations.Models.logits — Method
logits(M::AbstractFittedModel, X::AbstractArray)
Generic method that is compulsory for all models. It returns the raw model predictions. In classification this is sometimes referred to as logits: the non-normalized predictions that are fed into a link function to produce predicted probabilities. In regression (not currently implemented) raw outputs typically correspond to final outputs. In other words, there is typically no normalization involved.
source
CounterfactualExplanations.Models.logits — Method
logits(M::TreeModel, X::AbstractArray)
Calculates the logit scores output by the model M for the input data X.
Arguments
  • M::TreeModel: The model selected by the user.
  • X::AbstractArray: The feature vector for which the logit scores are calculated.
Returns
  • logits::Matrix: A matrix of logits for each output class for each data point in X.
Example
logits = Models.logits(M, x) # calculates the logit scores for each output class for the data point x
source
CounterfactualExplanations.Models.model_evaluation — Method
model_evaluation(M::AbstractFittedModel, test_data::CounterfactualData)
Helper function to compute F-Score for AbstractFittedModel on a (test) data set. By default, it computes the accuracy. Any other measure, e.g. from the StatisticalMeasures package, can be passed as an argument. Currently, only measures applicable to classification tasks are supported.
source
CounterfactualExplanations.Models.predict_label — Method
predict_label(M::TreeModel, X::AbstractArray)
Returns the predicted label for X.
Arguments
  • M::TreeModel: The model selected by the user.
  • X::AbstractArray: The input array for which the label is predicted.
Returns
  • labels::AbstractArray: The predicted label for each data point in X.
Example
label = Models.predict_label(M, x) # returns the predicted label for each data point in x
source
CounterfactualExplanations.Models.predict_proba — Method
predict_proba(M::AbstractFittedModel, counterfactual_data::CounterfactualData, X::Union{Nothing,AbstractArray})
Returns the predicted output probabilities for a given model M, data set counterfactual_data and input data X.
source
CounterfactualExplanations.Models.probs — Method
probs(M::AbstractFittedModel, X::AbstractArray)
Generic method that is compulsory for all models. It returns the normalized model predictions, so the predicted probabilities in the case of classification. In regression (not currently implemented) this method is redundant. 
source
CounterfactualExplanations.Models.probs — Method
probs(M::TreeModel, X::AbstractArray{<:Number, 2})
Calculates the probability scores for each output class for the two-dimensional input data matrix X.
Arguments
  • M::TreeModel: The TreeModel.
  • X::AbstractArray: The feature vector for which the predictions are made.
Returns
  • p::Matrix: A matrix of probability scores for each output class for each data point in X.
Example
probabilities = Models.probs(M, X) # calculates the probability scores for each output class for each data point in X.
source
CounterfactualExplanations.Models.probs — Method
probs(M::TreeModel, X::AbstractArray{<:Number, 1})
Works the same way as the probs(M::TreeModel, X::AbstractArray{<:Number, 2}) method above, but handles 1-dimensional rather than 2-dimensional input data.
source
CounterfactualExplanations.Generators.FeatureTweakGenerator — Method
FeatureTweakGenerator(; penalty::Union{Nothing,Function,Vector{Function}}=Objectives.distance_l2, ϵ::AbstractFloat=0.1)
Constructs a new Feature Tweak Generator object.
Uses the L2-norm as the penalty to measure the distance between the counterfactual and the factual. According to the paper by Tolomei et al., another recommended choice for the penalty in addition to the L2-norm is the L0-norm. The L0-norm simply minimizes the number of features that are changed through the tweak.
Arguments
  • penalty::Union{Nothing,Function,Vector{Function}}: The penalty function to use for the generator. Defaults to distance_l2.
  • ϵ::AbstractFloat: The tolerance value for the feature tweaks. Described at length in Tolomei et al. (https://arxiv.org/pdf/1706.06691.pdf). Defaults to 0.1.
Returns
  • generator::FeatureTweakGenerator: A non-gradient-based generator that can be used to generate counterfactuals using the feature tweak method.
source
CounterfactualExplanations.Generators.GradientBasedGenerator — Method
GradientBasedGenerator(;
 	loss::Union{Nothing,Function}=nothing,
 	penalty::Penalty=nothing,
 	λ::Union{Nothing,AbstractFloat,Vector{AbstractFloat}}=nothing,
 	latent_space::Bool::false,
 	opt::Flux.Optimise.AbstractOptimiser=Flux.Descent(),
     generative_model_params::NamedTuple=(;),
-)
Default outer constructor for GradientBasedGenerator.
Arguments
  • loss::Union{Nothing,Function}=nothing: The loss function used by the model.
  • penalty::Penalty=nothing: A penalty function for the generator to penalize counterfactuals too far from the original point.
  • λ::Union{Nothing,AbstractFloat,Vector{AbstractFloat}}=nothing: The weight of the penalty function.
  • latent_space::Bool=false: Whether to use the latent space of a generative model to generate counterfactuals.
  • opt::Flux.Optimise.AbstractOptimiser=Flux.Descent(): The optimizer to use for the generator.
  • generative_model_params::NamedTuple: The parameters of the generative model associated with the generator.
Returns
  • generator::GradientBasedGenerator: A gradient-based counterfactual generator.
source
CounterfactualExplanations.Generators.conditions_satisfied — Method
conditions_satisfied(generator::AbstractGradientBasedGenerator, ce::AbstractCounterfactualExplanation)
The default method to check if the all conditions for convergence of the counterfactual search have been satisified for gradient-based generators. By default, gradient-based search is considered to have converged as soon as the proposed feature changes for all features are smaller than one percent of its standard deviation.
source
CounterfactualExplanations.Generators.feature_tweaking! — Method
feature_tweaking!(ce::AbstractCounterfactualExplanation)
Returns a counterfactual instance of ce.x based on the ensemble of classifiers provided.
Arguments
  • ce::AbstractCounterfactualExplanation: The counterfactual explanation object.
Returns
  • ce::AbstractCounterfactualExplanation: The counterfactual explanation object.
Example
ce = feature_tweaking!(ce) # returns a counterfactual inside the ce.s′ field based on the ensemble of classifiers provided
source
CounterfactualExplanations.Generators.hinge_loss — Method
hinge_loss(convergence::AbstractConvergence, ce::AbstractCounterfactualExplanation)
The default hinge loss for any convergence criterion. Can be overridden inside the Convergence module as part of the definition of specific convergence criteria.
source
CounterfactualExplanations.Objectives.ddp_diversity — Method
ddp_diversity(
+)
Default outer constructor for GradientBasedGenerator.
Arguments
  • loss::Union{Nothing,Function}=nothing: The loss function used by the model.
  • penalty::Penalty=nothing: A penalty function for the generator to penalize counterfactuals too far from the original point.
  • λ::Union{Nothing,AbstractFloat,Vector{AbstractFloat}}=nothing: The weight of the penalty function.
  • latent_space::Bool=false: Whether to use the latent space of a generative model to generate counterfactuals.
  • opt::Flux.Optimise.AbstractOptimiser=Flux.Descent(): The optimizer to use for the generator.
  • generative_model_params::NamedTuple: The parameters of the generative model associated with the generator.
Returns
  • generator::GradientBasedGenerator: A gradient-based counterfactual generator.
source
CounterfactualExplanations.Generators.conditions_satisfied — Method
conditions_satisfied(generator::AbstractGradientBasedGenerator, ce::AbstractCounterfactualExplanation)
The default method to check if the all conditions for convergence of the counterfactual search have been satisified for gradient-based generators. By default, gradient-based search is considered to have converged as soon as the proposed feature changes for all features are smaller than one percent of its standard deviation.
source
CounterfactualExplanations.Generators.feature_tweaking! — Method
feature_tweaking!(ce::AbstractCounterfactualExplanation)
Returns a counterfactual instance of ce.x based on the ensemble of classifiers provided.
Arguments
  • ce::AbstractCounterfactualExplanation: The counterfactual explanation object.
Returns
  • ce::AbstractCounterfactualExplanation: The counterfactual explanation object.
Example
ce = feature_tweaking!(ce) # returns a counterfactual inside the ce.s′ field based on the ensemble of classifiers provided
source
CounterfactualExplanations.Generators.hinge_loss — Method
hinge_loss(convergence::AbstractConvergence, ce::AbstractCounterfactualExplanation)
The default hinge loss for any convergence criterion. Can be overridden inside the Convergence module as part of the definition of specific convergence criteria.
source
Flux.Losses.logitbinarycrossentropy — Method
Flux.Losses.logitbinarycrossentropy(ce::AbstractCounterfactualExplanation)
Simply extends the logitbinarycrossentropy method to work with objects of type AbstractCounterfactualExplanation.
source
Flux.Losses.logitcrossentropy — Method
Flux.Losses.logitcrossentropy(ce::AbstractCounterfactualExplanation)
Simply extends the logitcrossentropy method to work with objects of type AbstractCounterfactualExplanation.
source
Flux.Losses.mse — Method
Flux.Losses.mse(ce::AbstractCounterfactualExplanation)
Simply extends the mse method to work with objects of type AbstractCounterfactualExplanation.
source
Internal functions
Flux.Losses.logitbinarycrossentropy — Method
Flux.Losses.logitbinarycrossentropy(ce::AbstractCounterfactualExplanation)
Simply extends the logitbinarycrossentropy method to work with objects of type AbstractCounterfactualExplanation.
source
Flux.Losses.logitcrossentropy — Method
Flux.Losses.logitcrossentropy(ce::AbstractCounterfactualExplanation)
Simply extends the logitcrossentropy method to work with objects of type AbstractCounterfactualExplanation.
source
Flux.Losses.mse — Method
Flux.Losses.mse(ce::AbstractCounterfactualExplanation)
Simply extends the mse method to work with objects of type AbstractCounterfactualExplanation.
source
Internal functions
CounterfactualExplanations.decode_array — Method
decode_array(dt::GenerativeModels.AbstractGenerativeModel, x::AbstractArray)
Helper function to decode an array x using a data transform dt::GenerativeModels.AbstractGenerativeModel.
source
CounterfactualExplanations.decode_array — Method
decode_array(dt::MultivariateStats.AbstractDimensionalityReduction, x::AbstractArray)
Helper function to decode an array x using a data transform dt::MultivariateStats.AbstractDimensionalityReduction.
source
CounterfactualExplanations.decode_state — Function
function decode_state(     ce::CounterfactualExplanation,     x::Union{AbstractArray,Nothing}=nothing, )
Applies all the applicable decoding functions:
  1. If applicable, map the state variable back from the latent space to the feature space.
  2. If and where applicable, inverse-transform features.
  3. Reconstruct all categorical encodings.
Finally, the decoded counterfactual is returned.
source
CounterfactualExplanations.encode_array — Method
encode_array(dt::GenerativeModels.AbstractGenerativeModel, x::AbstractArray)
Helper function to encode an array x using a data transform dt::GenerativeModels.AbstractGenerativeModel.
source
CounterfactualExplanations.encode_array — Method
encode_array(dt::MultivariateStats.AbstractDimensionalityReduction, x::AbstractArray)
Helper function to encode an array x using a data transform dt::MultivariateStats.AbstractDimensionalityReduction.
source
CounterfactualExplanations.encode_state — Function
function encode_state(     ce::CounterfactualExplanation,      x::Union{AbstractArray,Nothing} = nothing, )
Applies all required encodings to x:
  1. If applicable, it maps x to the latent space learned by the generative model.
  2. If and where applicable, it rescales features. 
Finally, it returns the encoded state variable.
source
CounterfactualExplanations.guess_likelihood — Method
guess_likelihood(y::RawOutputArrayType)
Guess the likelihood based on the scientific type of the output array. Returns a symbol indicating the guessed likelihood and the scientific type of the output array.
source
CounterfactualExplanations.initialize_state — Method
initialize_state(ce::CounterfactualExplanation)
Initializes the starting point for the factual(s):
  1. If ce.initialization is set to :identity or counterfactuals are searched in a latent space, then nothing is done.
  2. If ce.initialization is set to :add_perturbation, then a random perturbation is added to the factual following following Slack (2021): https://arxiv.org/abs/2106.02666. The authors show that this improves adversarial robustness.
source
Base.vcat — Method
Base.vcat(bmk1::Benchmark, bmk2::Benchmark)
Vertically concatenates two Benchmark objects.
source
CounterfactualExplanations.Evaluation.compute_measure — Method
compute_measure(ce::CounterfactualExplanation, measure::Function, agg::Function)
Computes a single measure for a counterfactual explanation. The measure is applied to the counterfactual explanation ce and aggregated using the aggregation function agg.
source
CounterfactualExplanations.decode_array — Method
decode_array(dt::GenerativeModels.AbstractGenerativeModel, x::AbstractArray)
Helper function to decode an array x using a data transform dt::GenerativeModels.AbstractGenerativeModel.
source
CounterfactualExplanations.decode_array — Method
decode_array(dt::MultivariateStats.AbstractDimensionalityReduction, x::AbstractArray)
Helper function to decode an array x using a data transform dt::MultivariateStats.AbstractDimensionalityReduction.
source
CounterfactualExplanations.decode_state — Function
function decode_state(     ce::CounterfactualExplanation,     x::Union{AbstractArray,Nothing}=nothing, )
Applies all the applicable decoding functions:
  1. If applicable, map the state variable back from the latent space to the feature space.
  2. If and where applicable, inverse-transform features.
  3. Reconstruct all categorical encodings.
Finally, the decoded counterfactual is returned.
source
CounterfactualExplanations.encode_array — Method
encode_array(dt::GenerativeModels.AbstractGenerativeModel, x::AbstractArray)
Helper function to encode an array x using a data transform dt::GenerativeModels.AbstractGenerativeModel.
source
CounterfactualExplanations.encode_array — Method
encode_array(dt::MultivariateStats.AbstractDimensionalityReduction, x::AbstractArray)
Helper function to encode an array x using a data transform dt::MultivariateStats.AbstractDimensionalityReduction.
source
CounterfactualExplanations.encode_state — Function
function encode_state(     ce::CounterfactualExplanation,      x::Union{AbstractArray,Nothing} = nothing, )
Applies all required encodings to x:
  1. If applicable, it maps x to the latent space learned by the generative model.
  2. If and where applicable, it rescales features. 
Finally, it returns the encoded state variable.
source
CounterfactualExplanations.guess_likelihood — Method
guess_likelihood(y::RawOutputArrayType)
Guess the likelihood based on the scientific type of the output array. Returns a symbol indicating the guessed likelihood and the scientific type of the output array.
source
CounterfactualExplanations.initialize_state — Method
initialize_state(ce::CounterfactualExplanation)
Initializes the starting point for the factual(s):
  1. If ce.initialization is set to :identity or counterfactuals are searched in a latent space, then nothing is done.
  2. If ce.initialization is set to :add_perturbation, then a random perturbation is added to the factual following following Slack (2021): https://arxiv.org/abs/2106.02666. The authors show that this improves adversarial robustness.
source
Base.vcat — Method
Base.vcat(bmk1::Benchmark, bmk2::Benchmark)
Vertically concatenates two Benchmark objects.
source
CounterfactualExplanations.Evaluation.compute_measure — Method
compute_measure(ce::CounterfactualExplanation, measure::Function, agg::Function)
Computes a single measure for a counterfactual explanation. The measure is applied to the counterfactual explanation ce and aggregated using the aggregation function agg.
source
CounterfactualExplanations.Evaluation.to_dataframe — Method
evaluate_dataframe(
     ce::CounterfactualExplanation,
     measure::Vector{Function},
     agg::Function,
     report_each::Bool,
     pivot_longer::Bool,
     store_ce::Bool,
-)
Evaluates a counterfactual explanation and returns a dataframe of evaluation measures.
source
CounterfactualExplanations.DataPreprocessing.InputTransformer — Type
InputTransformer
Abstract type for data transformers. This can be any of the following:
  • StatsBase.AbstractDataTransform: A data transformation object from the StatsBase package.
  • MultivariateStats.AbstractDimensionalityReduction: A dimensionality reduction object from the MultivariateStats package.
  • GenerativeModels.AbstractGenerativeModel: A generative model object from the GenerativeModels module.
source
CounterfactualExplanations.DataPreprocessing.convert_to_1d — Method
convert_to_1d(y::Matrix, y_levels::AbstractArray)
Helper function to convert a one-hot encoded matrix to a vector of labels. This is necessary because MLJ models require the labels to be represented as a vector, but the synthetic datasets in this package hold the labels in one-hot encoded form.
Arguments
  • y::Matrix: The one-hot encoded matrix.
  • y_levels::AbstractArray: The levels of the categorical variable.
Returns
  • labels: A vector of labels.
source
CounterfactualExplanations.DataPreprocessing.InputTransformer — Type
InputTransformer
Abstract type for data transformers. This can be any of the following:
  • StatsBase.AbstractDataTransform: A data transformation object from the StatsBase package.
  • MultivariateStats.AbstractDimensionalityReduction: A dimensionality reduction object from the MultivariateStats package.
  • GenerativeModels.AbstractGenerativeModel: A generative model object from the GenerativeModels module.
source
CounterfactualExplanations.DataPreprocessing.convert_to_1d — Method
convert_to_1d(y::Matrix, y_levels::AbstractArray)
Helper function to convert a one-hot encoded matrix to a vector of labels. This is necessary because MLJ models require the labels to be represented as a vector, but the synthetic datasets in this package hold the labels in one-hot encoded form.
Arguments
  • y::Matrix: The one-hot encoded matrix.
  • y_levels::AbstractArray: The levels of the categorical variable.
Returns
  • labels: A vector of labels.
source
CounterfactualExplanations.DataPreprocessing.preprocess_data_for_mlj — Method
preprocess_data_for_mlj(data::CounterfactualData)
Helper function to preprocess data::CounterfactualData for MLJ models.
Arguments
  • data::CounterfactualData: The data to be preprocessed.
Returns
  • (df_x, y): A tuple containing the preprocessed data, with df_x being a DataFrame object and y being a categorical vector.
Example
X, y = preprocessdatafor_mlj(data)
source
CounterfactualExplanations.DataPreprocessing.train_test_split — Method
train_test_split(data::CounterfactualData;test_size=0.2,keep_class_ratio=false)
Splits data into train and test split.
Arguments
  • data::CounterfactualData: The data to be preprocessed.
  • test_size=0.2: Proportion of the data to be used for testing. 
  • keep_class_ratio=false: Decides whether to sample equally from each class, or keep their relative size.
Returns
  • (train_data::CounterfactualData, test_data::CounterfactualData): A tuple containing the train and test splits.
Example
train, test = traintestsplit(data, testsize=0.1, keepclass_ratio=true)
source
CounterfactualExplanations.Models.TreeModel — Type
TreeModel <: AbstractNonDifferentiableJuliaModel
Constructor for tree-based models from the MLJ library. 
Arguments
  • model::Any: The model selected by the user. Must be a model from the MLJ library.
  • likelihood::Symbol: The likelihood of the model. Must be one of [:classification_binary, :classification_multi].
Returns
  • TreeModel: A tree-based model from the MLJ library wrapped inside the TreeModel class.
source
CounterfactualExplanations.Models.get_individual_classifiers — Method
get_individual_classifiers(M::TreeModel)
Returns the individual classifiers in the forest. If the input is a decision tree, the method returns the decision tree itself inside an array.
Arguments
  • M::TreeModel: The model selected by the user.
Returns
  • classifiers::AbstractArray: An array of individual classifiers in the forest.
Example
classifiers = Models.getindividualclassifiers(M) # returns the individual classifiers in the forest
source
CounterfactualExplanations.Models.train — Method
train(M::TreeModel, data::CounterfactualData; kwargs...)
Fits the model M to the data in the CounterfactualData object. This method is not called by the user directly.
Arguments
  • M::TreeModel: The wrapper for a TreeModel.
  • data::CounterfactualData: The CounterfactualData object containing the data to be used for training the model.
Returns
  • M::TreeModel: The fitted TreeModel.
source
Base.rand — Function
Random.rand(encoder::Encoder, x, device=cpu)
Draws random samples from the latent distribution.
source
CounterfactualExplanations.Generators.calculate_delta — Method
calculate_delta(ce::AbstractCounterfactualExplanation, penalty::Vector{Function})
Calculates the penalty for the proposed feature tweak.
Arguments
  • ce::AbstractCounterfactualExplanation: The counterfactual explanation object.
Returns
  • delta::Float64: The calculated penalty for the proposed feature tweak.
source
CounterfactualExplanations.Generators.esatisfactory_instance — Method
esatisfactory_instance(generator::FeatureTweakGenerator, x::AbstractArray, paths::Dict{String, Dict{String, Any}})
Returns an epsilon-satisfactory counterfactual for x based on the paths provided.
Arguments
  • generator::FeatureTweakGenerator: The feature tweak generator.
  • x::AbstractArray: The factual instance.
  • paths::Dict{String, Dict{String, Any}}: A list of paths to the leaves of the tree to be used for tweaking the feature.
Returns
  • esatisfactory::AbstractArray: The epsilon-satisfactory instance.
Example
esatisfactory = esatisfactory_instance(generator, x, paths) # returns an epsilon-satisfactory counterfactual for x based on the paths provided
source
CounterfactualExplanations.Generators.feature_selection! — Method
feature_selection!(ce::AbstractCounterfactualExplanation)
Perform feature selection to find the dimension with the closest (but not equal) values between the ce.x (factual) and ce.s′ (counterfactual) arrays.
Arguments
  • ce::AbstractCounterfactualExplanation: An instance of the AbstractCounterfactualExplanation type representing the counterfactual explanation.
Returns
  • nothing
The function iteratively modifies the ce.s′ counterfactual array by updating its elements to match the corresponding elements in the ce.x factual array, one dimension at a time, until the predicted label of the modified ce.s′ matches the predicted label of the ce.x array.
source
CounterfactualExplanations.Generators.find_closest_dimension — Method
find_closest_dimension(factual, counterfactual)
Find the dimension with the closest (but not equal) values between the factual and counterfactual arrays.
Arguments
  • factual: The factual array.
  • counterfactual: The counterfactual array.
Returns
  • closest_dimension: The index of the dimension with the closest values.
The function iterates over the indices of the factual array and calculates the absolute difference between the corresponding elements in the factual and counterfactual arrays. It returns the index of the dimension with the smallest difference, excluding dimensions where the values in factual and counterfactual are equal.
source
CounterfactualExplanations.Generators.find_counterfactual — Method
find_counterfactual(model, factual_class, counterfactual_data, counterfactual_candidates)
Find the first counterfactual index by predicting labels.
Arguments
  • model: The fitted model used for prediction.
  • target_class: Expected target class.
  • counterfactual_data: Data required for counterfactual generation.
  • counterfactual_candidates: The array of counterfactual candidates.
Returns
  • counterfactual: The index of the first counterfactual found.
source
CounterfactualExplanations.Generators.growing_spheres_generation! — Method
growing_spheres_generation(ce::AbstractCounterfactualExplanation)
Generate counterfactual candidates using the growing spheres generation algorithm.
Arguments
  • ce::AbstractCounterfactualExplanation: An instance of the AbstractCounterfactualExplanation type representing the counterfactual explanation.
Returns
  • nothing
This function applies the growing spheres generation algorithm to generate counterfactual candidates. It starts by generating random points uniformly on a sphere, gradually reducing the search space until no counterfactuals are found. Then it expands the search space until at least one counterfactual is found or the maximum number of iterations is reached.
The algorithm iteratively generates counterfactual candidates and predicts their labels using the model stored in ce.M. It checks if any of the predicted labels are different from the factual class. The process of reducing the search space involves halving the search radius, while the process of expanding the search space involves increasing the search radius.
source
CounterfactualExplanations.Generators.h — Method
h(generator::AbstractGenerator, penalty::Function, ce::AbstractCounterfactualExplanation)
Overloads the h function for the case where a single penalty function is provided.
source
CounterfactualExplanations.Generators.h — Method
h(generator::AbstractGenerator, penalty::Tuple, ce::AbstractCounterfactualExplanation)
Overloads the h function for the case where a single penalty function is provided with additional keyword arguments.
source
CounterfactualExplanations.Generators.h — Method
h(generator::AbstractGenerator, penalty::Tuple, ce::AbstractCounterfactualExplanation)
Overloads the h function for the case where a single penalty function is provided with additional keyword arguments.
source
CounterfactualExplanations.Generators.hyper_sphere_coordinates — Method
hyper_sphere_coordinates(n_search_samples::Int, instance::Vector{Float64}, low::Int, high::Int; p_norm::Int=2)
Generates candidate counterfactuals using the growing spheres method based on hyper-sphere coordinates.
The implementation follows the Random Point Picking over a sphere algorithm described in the paper: "Learning Counterfactual Explanations for Tabular Data" by Pawelczyk, Broelemann & Kascneci (2020), presented at The Web Conference 2020 (WWW). It ensures that points are sampled uniformly at random using insights from: http://mathworld.wolfram.com/HyperspherePointPicking.html
The growing spheres method is originally proposed in the paper: "Comparison-based Inverse Classification for Interpretability in Machine Learning" by Thibaut Laugel et al (2018), presented at the International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems (2018).
Arguments
  • n_search_samples::Int: The number of search samples (int > 0).
  • instance::AbstractArray: The input point array.
  • low::AbstractFloat: The lower bound (float >= 0, l < h).
  • high::AbstractFloat: The upper bound (float >= 0, h > l).
  • p_norm::Integer: The norm parameter (int >= 1).
Returns
  • candidate_counterfactuals::Array: An array of candidate counterfactuals.
source
CounterfactualExplanations.Generators.search_path — Function
search_path(tree::Union{DecisionTree.Leaf, DecisionTree.Node}, target::RawTargetType, path::AbstractArray)
Return a path index list with the inequality symbols, thresholds and feature indices.
Arguments
  • tree::Union{DecisionTree.Leaf, DecisionTree.Node}: The root node of a decision tree.
  • target::RawTargetType: The target class.
  • path::AbstractArray: A list containing the paths found thus far.
Returns
  • paths::AbstractArray: A list of paths to the leaves of the tree to be used for tweaking the feature.
Example
paths = search_path(tree, target) # returns a list of paths to the leaves of the tree to be used for tweaking the feature
source
CounterfactualExplanations.Generators.∂h — Method
∂h(generator::AbstractGradientBasedGenerator, ce::AbstractCounterfactualExplanation)
The default method to compute the gradient of the complexity penalty at the current counterfactual state for gradient-based generators. It assumes that Zygote.jl has gradient access. 
If the penalty is not provided, it returns 0.0. By default, Zygote never works out the gradient for constants and instead returns 'nothing', so we need to add a manual step to override this behaviour. See here: https://discourse.julialang.org/t/zygote-gradient/26715.
source
CounterfactualExplanations.Generators.∂ℓ — Method
∂ℓ(generator::AbstractGradientBasedGenerator, M::Union{Models.LogisticModel, Models.BayesianLogisticModel}, ce::AbstractCounterfactualExplanation)
The default method to compute the gradient of the loss function at the current counterfactual state for gradient-based generators. It assumes that Zygote.jl has gradient access.
source
CounterfactualExplanations.Generators.∇ — Method
∇(generator::AbstractGradientBasedGenerator, M::Models.AbstractDifferentiableModel, ce::AbstractCounterfactualExplanation)
The default method to compute the gradient of the counterfactual search objective for gradient-based generators. It simply computes the weighted sum over partial derivates. It assumes that Zygote.jl has gradient access. If the counterfactual is being generated using Probe, the hinge loss is added to the gradient.
source
CounterfactualExplanations.DataPreprocessing.preprocess_data_for_mlj — Method
preprocess_data_for_mlj(data::CounterfactualData)
Helper function to preprocess data::CounterfactualData for MLJ models.
Arguments
  • data::CounterfactualData: The data to be preprocessed.
Returns
  • (df_x, y): A tuple containing the preprocessed data, with df_x being a DataFrame object and y being a categorical vector.
Example
X, y = preprocessdatafor_mlj(data)
source
CounterfactualExplanations.DataPreprocessing.train_test_split — Method
train_test_split(data::CounterfactualData;test_size=0.2,keep_class_ratio=false)
Splits data into train and test split.
Arguments
  • data::CounterfactualData: The data to be preprocessed.
  • test_size=0.2: Proportion of the data to be used for testing. 
  • keep_class_ratio=false: Decides whether to sample equally from each class, or keep their relative size.
Returns
  • (train_data::CounterfactualData, test_data::CounterfactualData): A tuple containing the train and test splits.
Example
train, test = traintestsplit(data, testsize=0.1, keepclass_ratio=true)
source
CounterfactualExplanations.Models.TreeModel — Type
TreeModel <: AbstractNonDifferentiableJuliaModel
Constructor for tree-based models from the MLJ library. 
Arguments
  • model::Any: The model selected by the user. Must be a model from the MLJ library.
  • likelihood::Symbol: The likelihood of the model. Must be one of [:classification_binary, :classification_multi].
Returns
  • TreeModel: A tree-based model from the MLJ library wrapped inside the TreeModel class.
source
CounterfactualExplanations.Models.get_individual_classifiers — Method
get_individual_classifiers(M::TreeModel)
Returns the individual classifiers in the forest. If the input is a decision tree, the method returns the decision tree itself inside an array.
Arguments
  • M::TreeModel: The model selected by the user.
Returns
  • classifiers::AbstractArray: An array of individual classifiers in the forest.
Example
classifiers = Models.getindividualclassifiers(M) # returns the individual classifiers in the forest
source
CounterfactualExplanations.Models.train — Method
train(M::TreeModel, data::CounterfactualData; kwargs...)
Fits the model M to the data in the CounterfactualData object. This method is not called by the user directly.
Arguments
  • M::TreeModel: The wrapper for a TreeModel.
  • data::CounterfactualData: The CounterfactualData object containing the data to be used for training the model.
Returns
  • M::TreeModel: The fitted TreeModel.
source
Base.rand — Function
Random.rand(encoder::Encoder, x, device=cpu)
Draws random samples from the latent distribution.
source
CounterfactualExplanations.Generators.calculate_delta — Method
calculate_delta(ce::AbstractCounterfactualExplanation, penalty::Vector{Function})
Calculates the penalty for the proposed feature tweak.
Arguments
  • ce::AbstractCounterfactualExplanation: The counterfactual explanation object.
Returns
  • delta::Float64: The calculated penalty for the proposed feature tweak.
source
CounterfactualExplanations.Generators.esatisfactory_instance — Method
esatisfactory_instance(generator::FeatureTweakGenerator, x::AbstractArray, paths::Dict{String, Dict{String, Any}})
Returns an epsilon-satisfactory counterfactual for x based on the paths provided.
Arguments
  • generator::FeatureTweakGenerator: The feature tweak generator.
  • x::AbstractArray: The factual instance.
  • paths::Dict{String, Dict{String, Any}}: A list of paths to the leaves of the tree to be used for tweaking the feature.
Returns
  • esatisfactory::AbstractArray: The epsilon-satisfactory instance.
Example
esatisfactory = esatisfactory_instance(generator, x, paths) # returns an epsilon-satisfactory counterfactual for x based on the paths provided
source
CounterfactualExplanations.Generators.feature_selection! — Method
feature_selection!(ce::AbstractCounterfactualExplanation)
Perform feature selection to find the dimension with the closest (but not equal) values between the ce.x (factual) and ce.s′ (counterfactual) arrays.
Arguments
  • ce::AbstractCounterfactualExplanation: An instance of the AbstractCounterfactualExplanation type representing the counterfactual explanation.
Returns
  • nothing
The function iteratively modifies the ce.s′ counterfactual array by updating its elements to match the corresponding elements in the ce.x factual array, one dimension at a time, until the predicted label of the modified ce.s′ matches the predicted label of the ce.x array.
source
CounterfactualExplanations.Generators.find_closest_dimension — Method
find_closest_dimension(factual, counterfactual)
Find the dimension with the closest (but not equal) values between the factual and counterfactual arrays.
Arguments
  • factual: The factual array.
  • counterfactual: The counterfactual array.
Returns
  • closest_dimension: The index of the dimension with the closest values.
The function iterates over the indices of the factual array and calculates the absolute difference between the corresponding elements in the factual and counterfactual arrays. It returns the index of the dimension with the smallest difference, excluding dimensions where the values in factual and counterfactual are equal.
source
CounterfactualExplanations.Generators.find_counterfactual — Method
find_counterfactual(model, factual_class, counterfactual_data, counterfactual_candidates)
Find the first counterfactual index by predicting labels.
Arguments
  • model: The fitted model used for prediction.
  • target_class: Expected target class.
  • counterfactual_data: Data required for counterfactual generation.
  • counterfactual_candidates: The array of counterfactual candidates.
Returns
  • counterfactual: The index of the first counterfactual found.
source
CounterfactualExplanations.Generators.growing_spheres_generation! — Method
growing_spheres_generation(ce::AbstractCounterfactualExplanation)
Generate counterfactual candidates using the growing spheres generation algorithm.
Arguments
  • ce::AbstractCounterfactualExplanation: An instance of the AbstractCounterfactualExplanation type representing the counterfactual explanation.
Returns
  • nothing
This function applies the growing spheres generation algorithm to generate counterfactual candidates. It starts by generating random points uniformly on a sphere, gradually reducing the search space until no counterfactuals are found. Then it expands the search space until at least one counterfactual is found or the maximum number of iterations is reached.
The algorithm iteratively generates counterfactual candidates and predicts their labels using the model stored in ce.M. It checks if any of the predicted labels are different from the factual class. The process of reducing the search space involves halving the search radius, while the process of expanding the search space involves increasing the search radius.
source
CounterfactualExplanations.Generators.h — Method
h(generator::AbstractGenerator, penalty::Function, ce::AbstractCounterfactualExplanation)
Overloads the h function for the case where a single penalty function is provided.
source
CounterfactualExplanations.Generators.h — Method
h(generator::AbstractGenerator, penalty::Tuple, ce::AbstractCounterfactualExplanation)
Overloads the h function for the case where a single penalty function is provided with additional keyword arguments.
source
CounterfactualExplanations.Generators.h — Method
h(generator::AbstractGenerator, penalty::Tuple, ce::AbstractCounterfactualExplanation)
Overloads the h function for the case where a single penalty function is provided with additional keyword arguments.
source
CounterfactualExplanations.Generators.hyper_sphere_coordinates — Method
hyper_sphere_coordinates(n_search_samples::Int, instance::Vector{Float64}, low::Int, high::Int; p_norm::Int=2)
Generates candidate counterfactuals using the growing spheres method based on hyper-sphere coordinates.
The implementation follows the Random Point Picking over a sphere algorithm described in the paper: "Learning Counterfactual Explanations for Tabular Data" by Pawelczyk, Broelemann & Kascneci (2020), presented at The Web Conference 2020 (WWW). It ensures that points are sampled uniformly at random using insights from: http://mathworld.wolfram.com/HyperspherePointPicking.html
The growing spheres method is originally proposed in the paper: "Comparison-based Inverse Classification for Interpretability in Machine Learning" by Thibaut Laugel et al (2018), presented at the International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems (2018).
Arguments
  • n_search_samples::Int: The number of search samples (int > 0).
  • instance::AbstractArray: The input point array.
  • low::AbstractFloat: The lower bound (float >= 0, l < h).
  • high::AbstractFloat: The upper bound (float >= 0, h > l).
  • p_norm::Integer: The norm parameter (int >= 1).
Returns
  • candidate_counterfactuals::Array: An array of candidate counterfactuals.
source
CounterfactualExplanations.Generators.search_path — Function
search_path(tree::Union{DecisionTree.Leaf, DecisionTree.Node}, target::RawTargetType, path::AbstractArray)
Return a path index list with the inequality symbols, thresholds and feature indices.
Arguments
  • tree::Union{DecisionTree.Leaf, DecisionTree.Node}: The root node of a decision tree.
  • target::RawTargetType: The target class.
  • path::AbstractArray: A list containing the paths found thus far.
Returns
  • paths::AbstractArray: A list of paths to the leaves of the tree to be used for tweaking the feature.
Example
paths = search_path(tree, target) # returns a list of paths to the leaves of the tree to be used for tweaking the feature
source
CounterfactualExplanations.Generators.∂h — Method
∂h(generator::AbstractGradientBasedGenerator, ce::AbstractCounterfactualExplanation)
The default method to compute the gradient of the complexity penalty at the current counterfactual state for gradient-based generators. It assumes that Zygote.jl has gradient access. 
If the penalty is not provided, it returns 0.0. By default, Zygote never works out the gradient for constants and instead returns 'nothing', so we need to add a manual step to override this behaviour. See here: https://discourse.julialang.org/t/zygote-gradient/26715.
source
CounterfactualExplanations.Generators.∂ℓ — Method
∂ℓ(generator::AbstractGradientBasedGenerator, M::Union{Models.LogisticModel, Models.BayesianLogisticModel}, ce::AbstractCounterfactualExplanation)
The default method to compute the gradient of the loss function at the current counterfactual state for gradient-based generators. It assumes that Zygote.jl has gradient access.
source
CounterfactualExplanations.Generators.∇ — Method
∇(generator::AbstractGradientBasedGenerator, M::Models.AbstractDifferentiableModel, ce::AbstractCounterfactualExplanation)
The default method to compute the gradient of the counterfactual search objective for gradient-based generators. It simply computes the weighted sum over partial derivates. It assumes that Zygote.jl has gradient access. If the counterfactual is being generated using Probe, the hinge loss is added to the gradient.
source

+)

Additional penalty for ClaPROARGenerator.

source
diff --git a/dev/release-notes/index.html b/dev/release-notes/index.html index c0994738d..9fc2bf9f1 100644 --- a/dev/release-notes/index.html +++ b/dev/release-notes/index.html @@ -1,2 +1,2 @@ -Release Notes · CounterfactualExplanations.jl
GitHub

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Note: We try to adhere to these practices as of version v1.1.1.

Version v1.1.4 - 2024-04-19

Changed

  • Refactors the encodings and decodings such that it is now more streamlined. Instead of conditional statements, encodings are now dispatched on the type of a new unifying data.input_encoder field. #432
  • Refactors the check for redundancy. This is now based on the convergence type and done right before the counterfactual search begins, if not redundant. #432

Version v1.1.3 - 2024-04-17

Added

  • Adds a section on Convergence to the documentation, Changelog.jl functionality and a few doc tests. #429

Changed

  • Changes style of taking gradients for the counterfactual search from implicit to explicit. #430
  • Removed all implicit imports. #430

Removed

  • Removed CUDA.jl dependency, because redundant. #430
  • Removed Parameters.jl dependency, because redundant. #430

Version v1.1.2 - 2024-04-16

Changed

  • Replaces the GIF in the README and introduction of docs for a static image.

Version v1.1.1 - 2024-04-15

Added

  • Added tests for LaplaceRedux extension. Bumped upper compat bound for LaplaceRedux.jl. #428
+Release Notes · CounterfactualExplanations.jl
GitHub

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Note: We try to adhere to these practices as of version v1.1.1.

Version v1.1.4 - 2024-04-24

Changed

  • Refactors the encodings and decodings such that it is now more streamlined. Instead of conditional statements, encodings are now dispatched on the type of a new unifying data.input_encoder field. #432
  • Refactors the check for redundancy. This is now based on the convergence type and done right before the counterfactual search begins, if not redundant. #432

Version v1.1.3 - 2024-04-17

Added

  • Adds a section on Convergence to the documentation, Changelog.jl functionality and a few doc tests. #429

Changed

  • Changes style of taking gradients for the counterfactual search from implicit to explicit. #430
  • Removed all implicit imports. #430

Removed

  • Removed CUDA.jl dependency, because redundant. #430
  • Removed Parameters.jl dependency, because redundant. #430

Version v1.1.2 - 2024-04-16

Changed

  • Replaces the GIF in the README and introduction of docs for a static image.

Version v1.1.1 - 2024-04-15

Added

  • Added tests for LaplaceRedux extension. Bumped upper compat bound for LaplaceRedux.jl. #428
diff --git a/dev/search_index.js b/dev/search_index.js index e043d8fca..1f73ea168 100644 --- a/dev/search_index.js +++ b/dev/search_index.js @@ -1,3 +1,3 @@ var documenterSearchIndex = {"docs": -[{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"tutorials/models/#Handling-Models","page":"Handling Models","title":"Handling Models","text":"","category":"section"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"The typical use-case for Counterfactual Explanations and Algorithmic Recourse is as follows: users have trained some supervised model that is not inherently interpretable and are looking for a way to explain it. In this tutorial, we will see how pre-trained models can be used with this package.","category":"page"},{"location":"tutorials/models/#Models-trained-in-Flux.jl","page":"Handling Models","title":"Models trained in Flux.jl","text":"","category":"section"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"We will train a simple binary classifier in Flux.jl on the popular Moons dataset:","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"n = 500\ndata = TaijaData.load_moons(n)\ncounterfactual_data = DataPreprocessing.CounterfactualData(data...)\nX = counterfactual_data.X\ny = counterfactual_data.y\nplt = plot()\nscatter!(counterfactual_data)","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"(Image: )","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"The following code chunk sets up a Deep Neural Network for the task at hand:","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"data = Flux.DataLoader((X,y),batchsize=1)\ninput_dim = size(X,1)\nn_hidden = 32\nactivation = relu\noutput_dim = 2\nnn = Chain(\n Dense(input_dim, n_hidden, activation),\n Dropout(0.1),\n Dense(n_hidden, output_dim)\n)\nloss(yhat, y) = Flux.Losses.logitcrossentropy(nn(yhat), y)","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"Next, we fit the network to the data:","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"using Flux.Optimise: update!, Adam\nopt = Adam()\nepochs = 100\navg_loss(data) = mean(map(d -> loss(d[1],d[2]), data))\nshow_every = epochs/5\n# Training:\nfor epoch = 1:epochs\n for d in data\n gs = gradient(Flux.params(nn)) do\n l = loss(d...)\n end\n update!(opt, Flux.params(nn), gs)\n end\n if epoch % show_every == 0\n println(\"Epoch \" * string(epoch))\n @show avg_loss(data)\n end\nend","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"Epoch 20\navg_loss(data) = 0.1407434f0\nEpoch 40\navg_loss(data) = 0.11345118f0\nEpoch 60\navg_loss(data) = 0.046319224f0\nEpoch 80\navg_loss(data) = 0.011847609f0\nEpoch 100\navg_loss(data) = 0.007242911f0","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"To prepare the fitted model for use with our package, we need to wrap it inside a container. For plain-vanilla models trained in Flux.jl, the corresponding constructor is called FluxModel. There is also a separate constructor called FluxEnsemble, which applies to Deep Ensembles. Deep Ensembles are a popular approach to approximate Bayesian Deep Learning and have been shown to generate good predictive uncertainty estimates (Lakshminarayanan, Pritzel, and Blundell 2016).","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"The appropriate API call to wrap our simple network in a container follows below:","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"M = FluxModel(nn)","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"FluxModel(Chain(Dense(2 => 32, relu), Dropout(0.1, active=false), Dense(32 => 2)), :classification_binary)","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"The likelihood function of the output variable is automatically inferred from the data. The generic plot() method can be called on the model and data to visualise the results:","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"plot(M, counterfactual_data)","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"(Image: )","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"Our model M is now ready for use with the package.","category":"page"},{"location":"tutorials/models/#References","page":"Handling Models","title":"References","text":"","category":"section"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"Lakshminarayanan, Balaji, Alexander Pritzel, and Charles Blundell. 2016. “Simple and Scalable Predictive Uncertainty Estimation Using Deep Ensembles.” https://arxiv.org/abs/1612.01474.","category":"page"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"explanation/generators/dice/#DiCEGenerator","page":"DiCE","title":"DiCEGenerator","text":"","category":"section"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"The DiCEGenerator can be used to generate multiple diverse counterfactuals for a single factual.","category":"page"},{"location":"explanation/generators/dice/#Description","page":"DiCE","title":"Description","text":"","category":"section"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"Counterfactual Explanations are not unique and there are therefore many different ways through which valid counterfactuals can be generated. In the context of Algorithmic Recourse this can be leveraged to offer individuals not one, but possibly many different ways to change a negative outcome into a positive one. One might argue that it makes sense for those different options to be as diverse as possible. This idea is at the core of DiCE, a counterfactual generator introduce by Mothilal, Sharma, and Tan (2020) that generate a diverse set of counterfactual explanations.","category":"page"},{"location":"explanation/generators/dice/#Defining-Diversity","page":"DiCE","title":"Defining Diversity","text":"","category":"section"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"To ensure that the generated counterfactuals are diverse, Mothilal, Sharma, and Tan (2020) add a diversity constraint to the counterfactual search objective. In particular, diversity is explicitly proxied via Determinantal Point Processes (DDP).","category":"page"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"We can implement DDP in Julia as follows:[1]","category":"page"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"using LinearAlgebra\nfunction ddp_diversity(X::AbstractArray{<:Real, 3})\n xs = eachslice(X, dims = ndims(X))\n K = [1/(1 + norm(x .- y)) for x in xs, y in xs]\n return det(K)\nend","category":"page"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"Below we generate some random points in mathbbR^2 and apply gradient ascent on this function evaluated at the whole array of points. As we can see in the animation below, the points are sent away from each other. In other words, diversity across the array of points increases as we ascend the ddp_diversity function.","category":"page"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"lims = 5\nN = 5\nX = rand(2,1,N)\nT = 50\nη = 0.1\nanim = @animate for t in 1:T\n X .+= gradient(ddp_diversity, X)[1]\n Z = reshape(X,2,N)\n scatter(\n Z[1,:],Z[2,:],ms=25, \n xlims=(-lims,lims),ylims=(-lims,lims),\n label=\"\",colour=1:N,\n size=(500,500),\n title=\"Diverse Counterfactuals\"\n )\nend\ngif(anim, joinpath(www_path, \"dice_intro.gif\"))","category":"page"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"(Image: )","category":"page"},{"location":"explanation/generators/dice/#Usage","page":"DiCE","title":"Usage","text":"","category":"section"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"The approach can be used in our package as follows:","category":"page"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"generator = DiCEGenerator()\nconv = CounterfactualExplanations.Convergence.GeneratorConditionsConvergence()\nce = generate_counterfactual(\n x, target, counterfactual_data, M, generator; \n num_counterfactuals=5, convergence=conv\n)\nplot(ce)","category":"page"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"(Image: )","category":"page"},{"location":"explanation/generators/dice/#Effect-of-Penalty","page":"DiCE","title":"Effect of Penalty","text":"","category":"section"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"Λ₂ = [0.1, 1.0, 5.0]\nces = []\nn_cf = 5\nusing Flux\nfor λ₂ ∈ Λ₂ \n λ = [0.00, λ₂]\n generator = DiCEGenerator(λ=λ)\n ces = vcat(\n ces...,\n generate_counterfactual(\n x, target, counterfactual_data, M, generator; \n num_counterfactuals=n_cf, convergence=conv\n )\n )\nend","category":"page"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"The figure below shows the resulting counterfactual paths. As expected, the resulting counterfactuals are more dispersed across the feature domain for higher choices of lambda_2","category":"page"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"(Image: )","category":"page"},{"location":"explanation/generators/dice/#References","page":"DiCE","title":"References","text":"","category":"section"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"Mothilal, Ramaravind K, Amit Sharma, and Chenhao Tan. 2020. “Explaining Machine Learning Classifiers Through Diverse Counterfactual Explanations.” In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 607–17. https://doi.org/10.1145/3351095.3372850.","category":"page"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"[1] With thanks to the respondents on Discourse","category":"page"},{"location":"how_to_guides/","page":"Overview","title":"Overview","text":"CurrentModule = CounterfactualExplanations","category":"page"},{"location":"how_to_guides/#How-To-Guides","page":"Overview","title":"How-To Guides","text":"","category":"section"},{"location":"how_to_guides/","page":"Overview","title":"Overview","text":"In this section, you will find a series of how-to-guides that showcase specific use cases of counterfactual explanations (CE).","category":"page"},{"location":"how_to_guides/","page":"Overview","title":"Overview","text":"How-to guides are directions that take the reader through the steps required to solve a real-world problem. How-to guides are goal-oriented.— Diátaxis","category":"page"},{"location":"how_to_guides/","page":"Overview","title":"Overview","text":"In other words, you come here because you may have some particular problem in mind, would like to see how it can be solved using CE and then most likely head off again 🫡.","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"tutorials/generators/#Handling-Generators","page":"Handling Generators","title":"Handling Generators","text":"","category":"section"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"Generating Counterfactual Explanations can be seen as a generative modelling task because it involves generating samples in the input space: x sim mathcalX. In this tutorial, we will introduce how Counterfactual GradientBasedGenerators are used. They are discussed in more detail in the explanatory section of the documentation.","category":"page"},{"location":"tutorials/generators/#Composable-Generators","page":"Handling Generators","title":"Composable Generators","text":"","category":"section"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"warning: Breaking Changes Expected\nWork on this feature is still in its very early stages and breaking changes should be expected. ","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"One of the key objectives for this package is Composability. It turns out that many of the various counterfactual generators that have been proposed in the literature, essentially do the same thing: they optimize an objective function. Formally we have,","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"\nbeginaligned\nmathbfs^prime = arg min_mathbfs^prime in mathcalS left textyloss(M(f(mathbfs^prime))y^*)+ lambda textcost(f(mathbfs^prime)) right \nendaligned \n qquad(1)","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"where textyloss denotes the main loss function and textcost is a penalty term (Altmeyer et al. 2023).","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"Without going into further detail here, the important thing to mention is that Equation 1 very closely describes how counterfactual search is actually implemented in the package. In other words, all off-the-shelf generators currently implemented work with that same objective. They just vary in the way that penalties are defined, for example. This gives rise to an interesting idea:","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"Why not compose generators that combine ideas from different off-the-shelf generators?","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"The GradientBasedGenerator class provides a straightforward way to do this, without requiring users to build custom GradientBasedGenerators from scratch. It can be instantiated as follows:","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"generator = GradientBasedGenerator()","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"By default, this creates a generator that simply performs gradient descent without any penalties. To modify the behaviour of the generator, you can define the counterfactual search objective function using the @objective macro:","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"@objective(generator, logitbinarycrossentropy + 0.1distance_l2 + 1.0ddp_diversity)","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"Here we have essentially created a version of the DiCEGenerator:","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"ce = generate_counterfactual(x, target, counterfactual_data, M, generator; num_counterfactuals=5)\nplot(ce)","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"(Image: )","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"Multiple macros can be chained using Chains.jl making it easy to create entirely new flavours of counterfactual generators. The following generator, for example, combines ideas from DiCE (Mothilal, Sharma, and Tan 2020) and REVISE (Joshi et al. 2019):","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"@chain generator begin\n @objective logitcrossentropy + 1.0ddp_diversity # DiCE (Mothilal et al. 2020)\n @with_optimiser Flux.Adam(0.1) \n @search_latent_space # REVISE (Joshi et al. 2019)\nend","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"Let’s take this generator to our MNIST dataset and generate a counterfactual explanation for turning a 0 into a 8.","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"(Image: )","category":"page"},{"location":"tutorials/generators/#Off-the-Shelf-Generators","page":"Handling Generators","title":"Off-the-Shelf Generators","text":"","category":"section"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"Off-the-shelf generators are just default recipes for counterfactual generators. Currently, the following off-the-shelf counterfactual generators are implemented in the package:","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"generator_catalogue","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"Dict{Symbol, Any} with 11 entries:\n :gravitational => GravitationalGenerator\n :growing_spheres => GrowingSpheresGenerator\n :revise => REVISEGenerator\n :clue => CLUEGenerator\n :probe => ProbeGenerator\n :dice => DiCEGenerator\n :feature_tweak => FeatureTweakGenerator\n :claproar => ClaPROARGenerator\n :wachter => WachterGenerator\n :generic => GenericGenerator\n :greedy => GreedyGenerator","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"To specify the type of generator you want to use, you can simply instantiate it:","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"# Search:\ngenerator = GenericGenerator()\nce = generate_counterfactual(x, target, counterfactual_data, M, generator)\nplot(ce)","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"(Image: )","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"We generally make an effort to follow the literature as closely as possible when implementing off-the-shelf generators.","category":"page"},{"location":"tutorials/generators/#References","page":"Handling Generators","title":"References","text":"","category":"section"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"Altmeyer, Patrick, Giovan Angela, Aleksander Buszydlik, Karol Dobiczek, Arie van Deursen, and Cynthia Liem. 2023. “Endogenous Macrodynamics in Algorithmic Recourse.” In First IEEE Conference on Secure and Trustworthy Machine Learning. https://doi.org/10.1109/satml54575.2023.00036.","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"Joshi, Shalmali, Oluwasanmi Koyejo, Warut Vijitbenjaronk, Been Kim, and Joydeep Ghosh. 2019. “Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems.” https://arxiv.org/abs/1907.09615.","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"Mothilal, Ramaravind K, Amit Sharma, and Chenhao Tan. 2020. “Explaining Machine Learning Classifiers Through Diverse Counterfactual Explanations.” In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 607–17. https://doi.org/10.1145/3351095.3372850.","category":"page"},{"location":"tutorials/simple_example/","page":"Simple Example","title":"Simple Example","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"tutorials/simple_example/#Simple-Example","page":"Simple Example","title":"Simple Example","text":"","category":"section"},{"location":"tutorials/simple_example/","page":"Simple Example","title":"Simple Example","text":"In this tutorial, we will go through a simple example involving synthetic data and a generic counterfactual generator.","category":"page"},{"location":"tutorials/simple_example/#Data-and-Classifier","page":"Simple Example","title":"Data and Classifier","text":"","category":"section"},{"location":"tutorials/simple_example/","page":"Simple Example","title":"Simple Example","text":"Below we generate some linearly separable data and fit a simple MLP classifier with batch normalization to it. For more information on generating data and models, refer to the Handling Data and Handling Models tutorials respectively.","category":"page"},{"location":"tutorials/simple_example/","page":"Simple Example","title":"Simple Example","text":"# Counteractual data and model:\nflux_training_params.batchsize = 10\ndata = TaijaData.load_linearly_separable()\ncounterfactual_data = DataPreprocessing.CounterfactualData(data...)\ncounterfactual_data.standardize = true\nM = fit_model(counterfactual_data, :MLP, batch_norm=true)","category":"page"},{"location":"tutorials/simple_example/#Counterfactual-Search","page":"Simple Example","title":"Counterfactual Search","text":"","category":"section"},{"location":"tutorials/simple_example/","page":"Simple Example","title":"Simple Example","text":"Next, determine a target and factual class for our counterfactual search and select a random factual instance to explain.","category":"page"},{"location":"tutorials/simple_example/","page":"Simple Example","title":"Simple Example","text":"target = 2\nfactual = 1\nchosen = rand(findall(predict_label(M, counterfactual_data) .== factual))\nx = select_factual(counterfactual_data, chosen)","category":"page"},{"location":"tutorials/simple_example/","page":"Simple Example","title":"Simple Example","text":"Finally, we generate and visualize the generated counterfactual:","category":"page"},{"location":"tutorials/simple_example/","page":"Simple Example","title":"Simple Example","text":"# Search:\ngenerator = WachterGenerator()\nce = generate_counterfactual(x, target, counterfactual_data, M, generator)\nplot(ce)","category":"page"},{"location":"tutorials/simple_example/","page":"Simple Example","title":"Simple Example","text":"(Image: )","category":"page"},{"location":"explanation/optimisers/jsma/","page":"JSMA","title":"JSMA","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"explanation/optimisers/jsma/#Jacobian-based-Saliency-Map-Attack","page":"JSMA","title":"Jacobian-based Saliency Map Attack","text":"","category":"section"},{"location":"explanation/optimisers/jsma/","page":"JSMA","title":"JSMA","text":"To search counterfactuals, Schut et al. (2021) propose to use a Jacobian-Based Saliency Map Attack (JSMA) inspired by the literature on adversarial attacks. It works by moving in the direction of the most salient feature at a fixed step size in each iteration. Schut et al. (2021) use this optimisation rule in the context of Bayesian classifiers and demonstrate good results in terms of plausibility — how realistic counterfactuals are — and redundancy — how sparse the proposed feature changes are.","category":"page"},{"location":"explanation/optimisers/jsma/#JSMADescent","page":"JSMA","title":"JSMADescent","text":"","category":"section"},{"location":"explanation/optimisers/jsma/","page":"JSMA","title":"JSMA","text":"To implement this approach in a reusable manner, we have added JSMA as a Flux optimiser. In particular, we have added a class JSMADescent<:Flux.Optimise.AbstractOptimiser, for which we have overloaded the Flux.Optimise.apply! method. This makes it possible to reuse JSMADescent as an optimiser in composable generators.","category":"page"},{"location":"explanation/optimisers/jsma/","page":"JSMA","title":"JSMA","text":"The optimiser can be used with with any generator as follows:","category":"page"},{"location":"explanation/optimisers/jsma/","page":"JSMA","title":"JSMA","text":"using CounterfactualExplanations.Generators: JSMADescent\ngenerator = GenericGenerator() |>\n gen -> @with_optimiser(gen,JSMADescent(;η=0.1))\nce = generate_counterfactual(x, target, counterfactual_data, M, generator)","category":"page"},{"location":"explanation/optimisers/jsma/","page":"JSMA","title":"JSMA","text":"The figure below compares the resulting counterfactual search outcome to the corresponding outcome with generic Descent.","category":"page"},{"location":"explanation/optimisers/jsma/","page":"JSMA","title":"JSMA","text":"plot(p1,p2,size=(1000,400))","category":"page"},{"location":"explanation/optimisers/jsma/","page":"JSMA","title":"JSMA","text":"(Image: )","category":"page"},{"location":"explanation/optimisers/jsma/","page":"JSMA","title":"JSMA","text":"Schut, Lisa, Oscar Key, Rory Mc Grath, Luca Costabello, Bogdan Sacaleanu, Yarin Gal, et al. 2021. “Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertainties.” In International Conference on Artificial Intelligence and Statistics, 1756–64. PMLR.","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"tutorials/data_catalogue/#Data-Catalogue","page":"Data Catalogue","title":"Data Catalogue","text":"","category":"section"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"To allow researchers and practitioners to test and compare counterfactual generators, the TAIJA environment includes the package TaijaData.jl which comes with pre-processed synthetic and real-world benchmark datasets from different domains. This page explains how to use TaijaData.jl in tandem with CounterfactualExplanations.jl.","category":"page"},{"location":"tutorials/data_catalogue/#Synthetic-Data","page":"Data Catalogue","title":"Synthetic Data","text":"","category":"section"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"The following dictionary can be used to inspect the available methods to generate synthetic datasets where the key indicates the name of the data and the value is the corresponding method:","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"TaijaData.data_catalogue[:synthetic]","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"Dict{Symbol, Function} with 6 entries:\n :overlapping => load_overlapping\n :linearly_separable => load_linearly_separable\n :blobs => load_blobs\n :moons => load_moons\n :circles => load_circles\n :multi_class => load_multi_class","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"The chart below shows the generated data using default parameters:","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"plts = []\n_height = 200\n_n = length(keys(data_catalogue[:synthetic]))\nfor (key, fun) in data_catalogue[:synthetic]\n data = fun()\n counterfactual_data = DataPreprocessing.CounterfactualData(data...)\n plt = plot()\n scatter!(counterfactual_data, title=key)\n plts = [plts..., plt]\nend\nplot(plts..., size=(_n * _height, _height), layout=(1, _n))","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"(Image: )","category":"page"},{"location":"tutorials/data_catalogue/#Real-World-Data","page":"Data Catalogue","title":"Real-World Data","text":"","category":"section"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"As for real-world data, the same dictionary can be used to inspect the available data from different domains.","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"TaijaData.data_catalogue[:tabular]","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"Dict{Symbol, Function} with 5 entries:\n :german_credit => load_german_credit\n :california_housing => load_california_housing\n :credit_default => load_credit_default\n :adult => load_uci_adult\n :gmsc => load_gmsc","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"TaijaData.data_catalogue[:vision]","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"Dict{Symbol, Function} with 3 entries:\n :fashion_mnist => load_fashion_mnist\n :mnist => load_mnist\n :cifar_10 => load_cifar_10","category":"page"},{"location":"tutorials/data_catalogue/#Loading-Data","page":"Data Catalogue","title":"Loading Data","text":"","category":"section"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"To load or generate any of the datasets listed above, you can just use the corresponding method, for example:","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"data = TaijaData.load_linearly_separable()\ncounterfactual_data = DataPreprocessing.CounterfactualData(data...)","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"Optionally, you can specify how many samples you want to generate like so:","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"n = 100\ndata = TaijaData.load_overlapping(n)\ncounterfactual_data = DataPreprocessing.CounterfactualData(data...)","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"This also applies to real-world datasets, which by default are loaded in their entirety. If n is supplied, the dataset will be randomly undersampled:","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"data = TaijaData.load_mnist(n)\ncounterfactual_data = DataPreprocessing.CounterfactualData(data...)","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"The undersampled dataset is automatically balanced:","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"sum(counterfactual_data.y; dims=2)","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"10×1 Matrix{Int64}:\n 10\n 10\n 10\n 10\n 10\n 10\n 10\n 10\n 10\n 10","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"We can also use a helper function to split the data into train and test sets:","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"train_data, test_data = \n CounterfactualExplanations.DataPreprocessing.train_test_split(counterfactual_data)","category":"page"},{"location":"how_to_guides/custom_generators/","page":"... add custom generators","title":"... add custom generators","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"how_to_guides/custom_generators/#How-to-add-Custom-Generators","page":"... add custom generators","title":"How to add Custom Generators","text":"","category":"section"},{"location":"how_to_guides/custom_generators/","page":"... add custom generators","title":"... add custom generators","text":"As we will see in this short tutorial, building custom counterfactual generators is straightforward. We hope that this will facilitate contributions through the community.","category":"page"},{"location":"how_to_guides/custom_generators/#Generic-generator-with-dropout","page":"... add custom generators","title":"Generic generator with dropout","text":"","category":"section"},{"location":"how_to_guides/custom_generators/","page":"... add custom generators","title":"... add custom generators","text":"To illustrate how custom generators can be implemented we will consider a simple example of a generator that extends the functionality of our GenericGenerator. We have noted elsewhere that the effectiveness of counterfactual explanations depends to some degree on the quality of the fitted model. Another, perhaps trivial, thing to note is that counterfactual explanations are not unique: there are potentially many valid counterfactual paths. One interesting (or silly) idea following these two observations might be to introduce some form of regularization in the counterfactual search. For example, we could use dropout to randomly switch features on and off in each iteration. Without dwelling further on the usefulness of this idea, let us see how it can be implemented.","category":"page"},{"location":"how_to_guides/custom_generators/","page":"... add custom generators","title":"... add custom generators","text":"The first code chunk below implements two important steps: 1) create an abstract subtype of the AbstractGradientBasedGenerator and 2) create a constructor similar to the GenericConstructor, but with one additional field for the probability of dropout.","category":"page"},{"location":"how_to_guides/custom_generators/","page":"... add custom generators","title":"... add custom generators","text":"# Abstract suptype:\nabstract type AbstractDropoutGenerator <: AbstractGradientBasedGenerator end\n\n# Constructor:\nstruct DropoutGenerator <: AbstractDropoutGenerator\n loss::Function # loss function\n penalty::Function\n λ::AbstractFloat # strength of penalty\n latent_space::Bool\n opt::Any # optimizer\n generative_model_params::NamedTuple\n p_dropout::AbstractFloat # dropout rate\nend\n\n# Instantiate:\ngenerator = DropoutGenerator(\n Flux.logitbinarycrossentropy,\n CounterfactualExplanations.Objectives.distance_l1,\n 0.1,\n false,\n Flux.Optimise.Descent(0.1),\n (;),\n 0.5,\n)","category":"page"},{"location":"how_to_guides/custom_generators/","page":"... add custom generators","title":"... add custom generators","text":"Next, we define how feature perturbations are generated for our dropout generator: in particular, we extend the relevant function through a method that implemented the dropout logic.","category":"page"},{"location":"how_to_guides/custom_generators/","page":"... add custom generators","title":"... add custom generators","text":"using CounterfactualExplanations.Generators\nusing StatsBase\nfunction Generators.generate_perturbations(\n generator::AbstractDropoutGenerator, \n ce::CounterfactualExplanation\n)\n s′ = deepcopy(ce.s′)\n new_s′ = Generators.propose_state(generator, ce)\n Δs′ = new_s′ - s′ # gradient step\n\n # Dropout:\n set_to_zero = sample(\n 1:length(Δs′),\n Int(round(generator.p_dropout*length(Δs′))),\n replace=false\n )\n Δs′[set_to_zero] .= 0\n return Δs′\nend","category":"page"},{"location":"how_to_guides/custom_generators/","page":"... add custom generators","title":"... add custom generators","text":"Finally, we proceed to generate counterfactuals in the same way we always do:","category":"page"},{"location":"how_to_guides/custom_generators/","page":"... add custom generators","title":"... add custom generators","text":"# Data and Classifier:\nM = fit_model(counterfactual_data, :DeepEnsemble)\n\n# Factual and Target:\nyhat = predict_label(M, counterfactual_data)\ntarget = 2 # target label\ncandidates = findall(vec(yhat) .!= target)\nchosen = rand(candidates)\nx = select_factual(counterfactual_data, chosen)\n\n# Counterfactual search:\nce = generate_counterfactual(\n x, target, counterfactual_data, M, generator;\n num_counterfactuals=5)\n\nplot(ce)","category":"page"},{"location":"how_to_guides/custom_generators/","page":"... add custom generators","title":"... add custom generators","text":"(Image: )","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"tutorials/parallelization/#Parallelization","page":"Parallelization","title":"Parallelization","text":"","category":"section"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"Version 0.1.15 adds support for parallelization through multi-processing. Currently, the only available backend for parallelization is MPI.jl.","category":"page"},{"location":"tutorials/parallelization/#Available-functions","page":"Parallelization","title":"Available functions","text":"","category":"section"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"Parallelization is only available for certain functions. To check if a function is parallelizable, you can use parallelizable function:","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"using CounterfactualExplanations.Evaluation: evaluate, benchmark\nprintln(parallelizable(generate_counterfactual))\nprintln(parallelizable(evaluate))\nprintln(parallelizable(predict_label))","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"true\ntrue\nfalse","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"In the following, we will generate multiple counterfactuals and evaluate them in parallel:","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"chosen = rand(findall(predict_label(M, counterfactual_data) .== factual), 1000)\nxs = select_factual(counterfactual_data, chosen)","category":"page"},{"location":"tutorials/parallelization/#Multi-threading","page":"Parallelization","title":"Multi-threading","text":"","category":"section"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"We first instantiate an ThreadParallelizer object:","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"parallelizer = ThreadsParallelizer()","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"ThreadsParallelizer()","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"To generate counterfactuals in parallel, we use the parallelize function:","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"ces = @with_parallelizer parallelizer begin\n generate_counterfactual(\n xs,\n target,\n counterfactual_data,\n M,\n generator\n )\nend","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"Generating counterfactuals ... 0%| | ETA: 0:01:29 (89.14 ms/it)Generating counterfactuals ... 100%|███████| Time: 0:00:01 ( 1.59 ms/it)\n\n1000-element Vector{AbstractCounterfactualExplanation}:\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 6 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 6 steps.\n ⋮\n CounterfactualExplanation\nConvergence: ✅ after 9 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 6 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"To evaluate counterfactuals in parallel, we again use the parallelize function:","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"@with_parallelizer parallelizer evaluate(ces)","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"Evaluating counterfactuals ... 0%| | ETA: 0:07:03 ( 0.42 s/it)Evaluating counterfactuals ... 100%|███████| Time: 0:00:00 ( 0.86 ms/it)\n\n1000-element Vector{Any}:\n Vector[[1.0], Float32[3.2939816], [0.0]]\n Vector[[1.0], Float32[3.019046], [0.0]]\n Vector[[1.0], Float32[3.701171], [0.0]]\n Vector[[1.0], Float32[2.5611918], [0.0]]\n Vector[[1.0], Float32[2.9027307], [0.0]]\n Vector[[1.0], Float32[3.7893882], [0.0]]\n Vector[[1.0], Float32[3.5026522], [0.0]]\n Vector[[1.0], Float32[3.6317568], [0.0]]\n Vector[[1.0], Float32[3.084984], [0.0]]\n Vector[[1.0], Float32[3.2268934], [0.0]]\n Vector[[1.0], Float32[2.834947], [0.0]]\n Vector[[1.0], Float32[3.656587], [0.0]]\n Vector[[1.0], Float32[2.5985842], [0.0]]\n ⋮\n Vector[[1.0], Float32[4.067538], [0.0]]\n Vector[[1.0], Float32[3.02231], [0.0]]\n Vector[[1.0], Float32[2.748292], [0.0]]\n Vector[[1.0], Float32[2.9483426], [0.0]]\n Vector[[1.0], Float32[3.066149], [0.0]]\n Vector[[1.0], Float32[3.6018147], [0.0]]\n Vector[[1.0], Float32[3.0138078], [0.0]]\n Vector[[1.0], Float32[3.5724509], [0.0]]\n Vector[[1.0], Float32[3.117551], [0.0]]\n Vector[[1.0], Float32[2.9670508], [0.0]]\n Vector[[1.0], Float32[3.4107168], [0.0]]\n Vector[[1.0], Float32[3.0252533], [0.0]]","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"Benchmarks can also be run with parallelization by specifying parallelizer argument:","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"# Models:\nbmk = benchmark(counterfactual_data; parallelizer = parallelizer)","category":"page"},{"location":"tutorials/parallelization/#MPI","page":"Parallelization","title":"MPI","text":"","category":"section"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"note: Note\nTo use MPI, you need to have MPI installed on your machine. Running the following code straight from a running Julia session will work if you have MPI installed on your machine, but it will be run on a single process. To execute the code on multiple processes, you need to run it from the command line with mpirun or mpiexec. For example, to run a script on 4 processes, you can run the following command from the command line:\n\nmpiexecjl --project -n 4 julia -e 'include(\"docs/src/srcipts/mpi.jl\")'For more information, see MPI.jl. ","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"We first instantiate an MPIParallelizer object:","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"import MPI\nMPI.Init()\nparallelizer = MPIParallelizer(MPI.COMM_WORLD; threaded=true)","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"Precompiling MPIExt\n ✓ TaijaParallel → MPIExt\n 1 dependency successfully precompiled in 3 seconds. 255 already precompiled.\n[ Info: Precompiling MPIExt [48137b38-b316-530b-be8a-261f41e68c23]\n┌ Warning: Module TaijaParallel with build ID ffffffff-ffff-ffff-0001-2d458926c256 is missing from the cache.\n│ This may mean TaijaParallel [bf1c2c22-5e42-4e78-8b6b-92e6c673eeb0] does not support precompilation but is imported by a module that does.\n└ @ Base loading.jl:1948\n[ Info: Skipping precompilation since __precompile__(false). Importing MPIExt [48137b38-b316-530b-be8a-261f41e68c23].\n[ Info: Using `MPI.jl` for multi-processing.\n\nRunning on 1 processes.\n\nMPIExt.MPIParallelizer(MPI.Comm(1140850688), 0, 1, nothing, true)","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"To generate counterfactuals in parallel, we use the parallelize function:","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"ces = @with_parallelizer parallelizer begin\n generate_counterfactual(\n xs,\n target,\n counterfactual_data,\n M,\n generator\n )\nend","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"Generating counterfactuals ... 9%|▋ | ETA: 0:00:01 ( 1.15 ms/it)Generating counterfactuals ... 19%|█▍ | ETA: 0:00:01 ( 1.07 ms/it)Generating counterfactuals ... 29%|██ | ETA: 0:00:01 ( 1.10 ms/it)Generating counterfactuals ... 39%|██▊ | ETA: 0:00:01 ( 1.08 ms/it)Generating counterfactuals ... 49%|███▍ | ETA: 0:00:01 ( 1.08 ms/it)Generating counterfactuals ... 59%|████▏ | ETA: 0:00:00 ( 1.08 ms/it)Generating counterfactuals ... 69%|████▊ | ETA: 0:00:00 ( 1.08 ms/it)Generating counterfactuals ... 79%|█████▌ | ETA: 0:00:00 ( 1.07 ms/it)Generating counterfactuals ... 89%|██████▎| ETA: 0:00:00 ( 1.07 ms/it)Generating counterfactuals ... 99%|██████▉| ETA: 0:00:00 ( 1.06 ms/it)Generating counterfactuals ... 100%|███████| Time: 0:00:01 ( 1.06 ms/it)\n\n1000-element Vector{AbstractCounterfactualExplanation}:\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 6 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 6 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 6 steps.\n ⋮\n CounterfactualExplanation\nConvergence: ✅ after 9 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 6 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"To evaluate counterfactuals in parallel, we again use the parallelize function:","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"@with_parallelizer parallelizer evaluate(ces)","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"1000-element Vector{Any}:\n Vector[[1.0], Float32[3.0941274], [0.0]]\n Vector[[1.0], Float32[3.0894346], [0.0]]\n Vector[[1.0], Float32[3.5737448], [0.0]]\n Vector[[1.0], Float32[2.6201036], [0.0]]\n Vector[[1.0], Float32[2.8519764], [0.0]]\n Vector[[1.0], Float32[3.7762523], [0.0]]\n Vector[[1.0], Float32[3.4162796], [0.0]]\n Vector[[1.0], Float32[3.6095932], [0.0]]\n Vector[[1.0], Float32[3.1347957], [0.0]]\n Vector[[1.0], Float32[3.0313473], [0.0]]\n Vector[[1.0], Float32[2.7612567], [0.0]]\n Vector[[1.0], Float32[3.6191392], [0.0]]\n Vector[[1.0], Float32[2.610616], [0.0]]\n ⋮\n Vector[[1.0], Float32[4.0844703], [0.0]]\n Vector[[1.0], Float32[3.0119], [0.0]]\n Vector[[1.0], Float32[2.4461186], [0.0]]\n Vector[[1.0], Float32[3.071967], [0.0]]\n Vector[[1.0], Float32[3.132917], [0.0]]\n Vector[[1.0], Float32[3.5403214], [0.0]]\n Vector[[1.0], Float32[3.0588162], [0.0]]\n Vector[[1.0], Float32[3.5600657], [0.0]]\n Vector[[1.0], Float32[3.2205954], [0.0]]\n Vector[[1.0], Float32[2.896302], [0.0]]\n Vector[[1.0], Float32[3.2603998], [0.0]]\n Vector[[1.0], Float32[3.1369917], [0.0]]","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"tip: Tip\nNote that parallelizable processes can be supplied as input to the macro either as a block or directly as an expression.","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"Benchmarks can also be run with parallelization by specifying parallelizer argument:","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"# Models:\nbmk = benchmark(counterfactual_data; parallelizer = parallelizer)","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"The following code snippet shows a complete example script that uses MPI for running a benchmark in parallel:","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"using CounterfactualExplanations\nusing CounterfactualExplanations.Evaluation: benchmark\nusing CounterfactualExplanations.Models\nimport MPI\n\nMPI.Init()\n\ndata = TaijaData.load_linearly_separable()\ncounterfactual_data = DataPreprocessing.CounterfactualData(data...)\nM = fit_model(counterfactual_data, :Linear)\nfactual = 1\ntarget = 2\nchosen = rand(findall(predict_label(M, counterfactual_data) .== factual), 100)\nxs = select_factual(counterfactual_data, chosen)\ngenerator = GenericGenerator()\n\nparallelizer = MPIParallelizer(MPI.COMM_WORLD)\n\nbmk = benchmark(counterfactual_data; parallelizer=parallelizer)\n\nMPI.Finalize()","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"The file can be executed from the command line as follows:","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"mpiexecjl --project -n 4 julia -e 'include(\"docs/src/srcipts/mpi.jl\")'","category":"page"},{"location":"release-notes/","page":"Release Notes","title":"Release Notes","text":"EditURL = \"https://github.com/juliatrustworthyai/CounterfactualExplanations.jl/blob/master/CHANGELOG.md\"","category":"page"},{"location":"release-notes/#Changelog","page":"Release Notes","title":"Changelog","text":"","category":"section"},{"location":"release-notes/","page":"Release Notes","title":"Release Notes","text":"All notable changes to this project will be documented in this file.","category":"page"},{"location":"release-notes/","page":"Release Notes","title":"Release Notes","text":"The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.","category":"page"},{"location":"release-notes/","page":"Release Notes","title":"Release Notes","text":"Note: We try to adhere to these practices as of version v1.1.1.","category":"page"},{"location":"release-notes/#Version-[v1.1.4](https://github.com/juliatrustworthyai/CounterfactualExplanations.jl/releases/tag/v1.1.4)-2024-04-19","page":"Release Notes","title":"Version v1.1.4 - 2024-04-19","text":"","category":"section"},{"location":"release-notes/#Changed","page":"Release Notes","title":"Changed","text":"","category":"section"},{"location":"release-notes/","page":"Release Notes","title":"Release Notes","text":"Refactors the encodings and decodings such that it is now more streamlined. Instead of conditional statements, encodings are now dispatched on the type of a new unifying data.input_encoder field. #432\nRefactors the check for redundancy. This is now based on the convergence type and done right before the counterfactual search begins, if not redundant. #432","category":"page"},{"location":"release-notes/#Version-[v1.1.3](https://github.com/juliatrustworthyai/CounterfactualExplanations.jl/releases/tag/v1.1.3)-2024-04-17","page":"Release Notes","title":"Version v1.1.3 - 2024-04-17","text":"","category":"section"},{"location":"release-notes/#Added","page":"Release Notes","title":"Added","text":"","category":"section"},{"location":"release-notes/","page":"Release Notes","title":"Release Notes","text":"Adds a section on Convergence to the documentation, Changelog.jl functionality and a few doc tests. #429","category":"page"},{"location":"release-notes/#Changed-2","page":"Release Notes","title":"Changed","text":"","category":"section"},{"location":"release-notes/","page":"Release Notes","title":"Release Notes","text":"Changes style of taking gradients for the counterfactual search from implicit to explicit. #430\nRemoved all implicit imports. #430","category":"page"},{"location":"release-notes/#Removed","page":"Release Notes","title":"Removed","text":"","category":"section"},{"location":"release-notes/","page":"Release Notes","title":"Release Notes","text":"Removed CUDA.jl dependency, because redundant. #430\nRemoved Parameters.jl dependency, because redundant. #430","category":"page"},{"location":"release-notes/#Version-[v1.1.2](https://github.com/juliatrustworthyai/CounterfactualExplanations.jl/releases/tag/v1.1.2)-2024-04-16","page":"Release Notes","title":"Version v1.1.2 - 2024-04-16","text":"","category":"section"},{"location":"release-notes/#Changed-3","page":"Release Notes","title":"Changed","text":"","category":"section"},{"location":"release-notes/","page":"Release Notes","title":"Release Notes","text":"Replaces the GIF in the README and introduction of docs for a static image. ","category":"page"},{"location":"release-notes/#Version-[v1.1.1](https://github.com/juliatrustworthyai/CounterfactualExplanations.jl/releases/tag/v1.1.1)-2024-04-15","page":"Release Notes","title":"Version v1.1.1 - 2024-04-15","text":"","category":"section"},{"location":"release-notes/#Added-2","page":"Release Notes","title":"Added","text":"","category":"section"},{"location":"release-notes/","page":"Release Notes","title":"Release Notes","text":"Added tests for LaplaceRedux extension. Bumped upper compat bound for LaplaceRedux.jl. #428","category":"page"},{"location":"tutorials/","page":"Overview","title":"Overview","text":"CurrentModule = CounterfactualExplanation","category":"page"},{"location":"tutorials/#Tutorials","page":"Overview","title":"Tutorials","text":"","category":"section"},{"location":"tutorials/","page":"Overview","title":"Overview","text":"In this section, you will find a series of tutorials that should help you gain a basic understanding of Conformal Prediction and how to apply it in Julia using this package.","category":"page"},{"location":"tutorials/","page":"Overview","title":"Overview","text":"Tutorials are lessons that take the reader by the hand through a series of steps to complete a project of some kind. Tutorials are learning-oriented.— Diátaxis","category":"page"},{"location":"tutorials/","page":"Overview","title":"Overview","text":"In other words, you come here because you are new to this topic and are looking for a first peek at the methodology and code 🫣.","category":"page"},{"location":"explanation/generators/growing_spheres/","page":"GrowingSpheres","title":"GrowingSpheres","text":"CurrentModule = CounterfactualExplanations","category":"page"},{"location":"explanation/generators/growing_spheres/#GrowingSpheres","page":"GrowingSpheres","title":"GrowingSpheres","text":"","category":"section"},{"location":"explanation/generators/growing_spheres/","page":"GrowingSpheres","title":"GrowingSpheres","text":"Growing Spheres refers to the generator introduced by Laugel et al. (2017). Our implementation takes inspiration from the CARLA library.","category":"page"},{"location":"explanation/generators/growing_spheres/#Principle-of-the-Proposed-Approach","page":"GrowingSpheres","title":"Principle of the Proposed Approach","text":"","category":"section"},{"location":"explanation/generators/growing_spheres/","page":"GrowingSpheres","title":"GrowingSpheres","text":"In order to interpret a prediction through comparison, the Growing Spheres algorithm focuses on finding an observation belonging to the other class and answers the question: “Considering an observation and a classifier, what is the minimal change we need to apply in order to change the prediction of this observation?”. This problem is similar to inverse classification but applied to interpretability.","category":"page"},{"location":"explanation/generators/growing_spheres/","page":"GrowingSpheres","title":"GrowingSpheres","text":"Explaining how to change a prediction can help the user understand what the model considers as locally important. The Growing Spheres approach provides insights into the classifier’s behavior without claiming any causal knowledge. It differs from other interpretability approaches and is not concerned with the global behavior of the model. Instead, it aims to provide local insights into the classifier’s decision-making process.","category":"page"},{"location":"explanation/generators/growing_spheres/","page":"GrowingSpheres","title":"GrowingSpheres","text":"The algorithm finds the closest “ennemy” observation, which is an observation classified into the other class than the input observation. The final explanation is the difference vector between the input observation and the ennemy.","category":"page"},{"location":"explanation/generators/growing_spheres/#Finding-the-Closest-Ennemy","page":"GrowingSpheres","title":"Finding the Closest Ennemy","text":"","category":"section"},{"location":"explanation/generators/growing_spheres/","page":"GrowingSpheres","title":"GrowingSpheres","text":"The algorithm solves the following minimization problem to find the closest ennemy:","category":"page"},{"location":"explanation/generators/growing_spheres/","page":"GrowingSpheres","title":"GrowingSpheres","text":"e^* = arg min_e in X c(x e) f(e) neq f(x) ","category":"page"},{"location":"explanation/generators/growing_spheres/","page":"GrowingSpheres","title":"GrowingSpheres","text":"The cost function c(x, e) is defined as:","category":"page"},{"location":"explanation/generators/growing_spheres/","page":"GrowingSpheres","title":"GrowingSpheres","text":"c(x e) = x - e_2 + gamma x - e_0","category":"page"},{"location":"explanation/generators/growing_spheres/","page":"GrowingSpheres","title":"GrowingSpheres","text":"where ||.||_2 is the Euclidean norm and ||.||_0 is the sparsity measure. The weight gamma balances the importance of sparsity in the cost function.","category":"page"},{"location":"explanation/generators/growing_spheres/","page":"GrowingSpheres","title":"GrowingSpheres","text":"To approximate the solution, the Growing Spheres algorithm uses a two-step heuristic approach. The first step is the Generation phase, where observations are generated in spherical layers around the input observation. The second step is the Feature Selection phase, where the generated observation with the smallest change in each feature is selected.","category":"page"},{"location":"explanation/generators/growing_spheres/#Example","page":"GrowingSpheres","title":"Example","text":"","category":"section"},{"location":"explanation/generators/growing_spheres/","page":"GrowingSpheres","title":"GrowingSpheres","text":"generator = GrowingSpheresGenerator()\nM = fit_model(counterfactual_data, :DeepEnsemble)\nce = generate_counterfactual(\n x, target, counterfactual_data, M, generator)\nplot(ce)","category":"page"},{"location":"explanation/generators/growing_spheres/","page":"GrowingSpheres","title":"GrowingSpheres","text":"(Image: )","category":"page"},{"location":"explanation/generators/growing_spheres/#References","page":"GrowingSpheres","title":"References","text":"","category":"section"},{"location":"explanation/generators/growing_spheres/","page":"GrowingSpheres","title":"GrowingSpheres","text":"Laugel, Thibault, Marie-Jeanne Lesot, Christophe Marsala, Xavier Renard, and Marcin Detyniecki. 2017. “Inverse Classification for Comparison-Based Interpretability in Machine Learning.” arXiv. https://doi.org/10.48550/arXiv.1712.08443.","category":"page"},{"location":"contribute/#Contribute","page":"🛠 Contribute","title":"🛠 Contribute","text":"","category":"section"},{"location":"contribute/","page":"🛠 Contribute","title":"🛠 Contribute","text":"Contributions of any kind are very much welcome! Take a look at the issue to see what things we are currently working on. If you have an idea for a new feature or want to report a bug, please open a new issue.","category":"page"},{"location":"contribute/#Development","page":"🛠 Contribute","title":"Development","text":"","category":"section"},{"location":"contribute/","page":"🛠 Contribute","title":"🛠 Contribute","text":"If your looking to contribute code, it may be helpful to check out the Explanation section of the docs.","category":"page"},{"location":"contribute/#Testing","page":"🛠 Contribute","title":"Testing","text":"","category":"section"},{"location":"contribute/","page":"🛠 Contribute","title":"🛠 Contribute","text":"Please always make sure to add tests for any new features or changes.","category":"page"},{"location":"contribute/#Documentation","page":"🛠 Contribute","title":"Documentation","text":"","category":"section"},{"location":"contribute/","page":"🛠 Contribute","title":"🛠 Contribute","text":"If you add new features or change existing ones, please make sure to update the documentation accordingly. The documentation is written in Documenter.jl and is located in the docs/src folder.","category":"page"},{"location":"contribute/#Log-Changes","page":"🛠 Contribute","title":"Log Changes","text":"","category":"section"},{"location":"contribute/","page":"🛠 Contribute","title":"🛠 Contribute","text":"As of version 1.1.1, we have tried to be more stringent about logging changes. Please make sure to add a note to the CHANGELOG.md file for any changes you make. It is sufficient to add a note under the Unreleased section.","category":"page"},{"location":"contribute/#General-Pointers","page":"🛠 Contribute","title":"General Pointers","text":"","category":"section"},{"location":"contribute/","page":"🛠 Contribute","title":"🛠 Contribute","text":"There are also some general pointers for people looking to contribute to any of our Taija packages here.","category":"page"},{"location":"contribute/","page":"🛠 Contribute","title":"🛠 Contribute","text":"Please follow the SciML ColPrac guide.","category":"page"},{"location":"explanation/generators/clap_roar/","page":"ClaPROAR","title":"ClaPROAR","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"explanation/generators/clap_roar/#ClaPROARGenerator","page":"ClaPROAR","title":"ClaPROARGenerator","text":"","category":"section"},{"location":"explanation/generators/clap_roar/","page":"ClaPROAR","title":"ClaPROAR","text":"The ClaPROARGenerator was introduced in Altmeyer et al. (2023).","category":"page"},{"location":"explanation/generators/clap_roar/#Description","page":"ClaPROAR","title":"Description","text":"","category":"section"},{"location":"explanation/generators/clap_roar/","page":"ClaPROAR","title":"ClaPROAR","text":"The acronym Clap stands for classifier-preserving. The approach is loosely inspired by ROAR (Upadhyay, Joshi, and Lakkaraju 2021). Altmeyer et al. (2023) propose to explicitly penalize the loss incurred by the classifer when evaluated on the counterfactual x^prime at given parameter values. Formally, we have","category":"page"},{"location":"explanation/generators/clap_roar/","page":"ClaPROAR","title":"ClaPROAR","text":"beginaligned\ntextextcost(f(mathbfs^prime)) = l(M(f(mathbfs^prime))y^prime)\nendaligned","category":"page"},{"location":"explanation/generators/clap_roar/","page":"ClaPROAR","title":"ClaPROAR","text":"for each counterfactual k where l denotes the loss function used to train M. This approach is based on the intuition that (endogenous) model shifts will be triggered by counterfactuals that increase classifier loss (Altmeyer et al. 2023).","category":"page"},{"location":"explanation/generators/clap_roar/#Usage","page":"ClaPROAR","title":"Usage","text":"","category":"section"},{"location":"explanation/generators/clap_roar/","page":"ClaPROAR","title":"ClaPROAR","text":"The approach can be used in our package as follows:","category":"page"},{"location":"explanation/generators/clap_roar/","page":"ClaPROAR","title":"ClaPROAR","text":"generator = ClaPROARGenerator()\nce = generate_counterfactual(x, target, counterfactual_data, M, generator)\nplot(ce)","category":"page"},{"location":"explanation/generators/clap_roar/","page":"ClaPROAR","title":"ClaPROAR","text":"(Image: )","category":"page"},{"location":"explanation/generators/clap_roar/#Comparison-to-GenericGenerator","page":"ClaPROAR","title":"Comparison to GenericGenerator","text":"","category":"section"},{"location":"explanation/generators/clap_roar/","page":"ClaPROAR","title":"ClaPROAR","text":"The figure below compares the outcome for the GenericGenerator and the ClaPROARGenerator.","category":"page"},{"location":"explanation/generators/clap_roar/","page":"ClaPROAR","title":"ClaPROAR","text":"(Image: )","category":"page"},{"location":"explanation/generators/clap_roar/#References","page":"ClaPROAR","title":"References","text":"","category":"section"},{"location":"explanation/generators/clap_roar/","page":"ClaPROAR","title":"ClaPROAR","text":"Altmeyer, Patrick, Giovan Angela, Aleksander Buszydlik, Karol Dobiczek, Arie van Deursen, and Cynthia Liem. 2023. “Endogenous Macrodynamics in Algorithmic Recourse.” In First IEEE Conference on Secure and Trustworthy Machine Learning. https://doi.org/10.1109/satml54575.2023.00036.","category":"page"},{"location":"explanation/generators/clap_roar/","page":"ClaPROAR","title":"ClaPROAR","text":"Upadhyay, Sohini, Shalmali Joshi, and Himabindu Lakkaraju. 2021. “Towards Robust and Reliable Algorithmic Recourse.” https://arxiv.org/abs/2102.13620.","category":"page"},{"location":"explanation/architecture/","page":"Package Architecture","title":"Package Architecture","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"explanation/architecture/#Package-Architecture","page":"Package Architecture","title":"Package Architecture","text":"","category":"section"},{"location":"explanation/architecture/","page":"Package Architecture","title":"Package Architecture","text":"The diagram below provides an overview of the package architecture. It is built around two core modules that are designed to be as extensible as possible through dispatch: 1) Models is concerned with making any arbitrary model compatible with the package; 2) Generators is used to implement arbitrary counterfactual search algorithms.[1]","category":"page"},{"location":"explanation/architecture/","page":"Package Architecture","title":"Package Architecture","text":"The core function of the package, generate_counterfactual, uses an instance of type AbstractFittedModel produced by the Models module and an instance of type AbstractGenerator produced by the Generators module.","category":"page"},{"location":"explanation/architecture/","page":"Package Architecture","title":"Package Architecture","text":"Metapackages from the Taija ecosystem provide additional functionality such as datasets, language interoperability, parallelization, and plotting. The CounterfactualExplanations package is designed to be used in conjunction with these metapackages, but can also be used as a standalone package.","category":"page"},{"location":"explanation/architecture/","page":"Package Architecture","title":"Package Architecture","text":"(Image: )","category":"page"},{"location":"explanation/architecture/","page":"Package Architecture","title":"Package Architecture","text":"[1] We have made an effort to keep the code base a flexible and extensible as possible, but cannot guarantee at this point that any counterfactual generator can be implemented without further adaptation.","category":"page"},{"location":"explanation/","page":"Overview","title":"Overview","text":"CurrentModule = CounterfactualExplanations","category":"page"},{"location":"explanation/#Explanation","page":"Overview","title":"Explanation","text":"","category":"section"},{"location":"explanation/","page":"Overview","title":"Overview","text":"In this section you will find detailed explanations about the methodology and code.","category":"page"},{"location":"explanation/","page":"Overview","title":"Overview","text":"Explanation clarifies, deepens and broadens the reader’s understanding of a subject.— Diátaxis","category":"page"},{"location":"explanation/","page":"Overview","title":"Overview","text":"In other words, you come here because you are interested in understanding how all of this actually works 🤓.","category":"page"},{"location":"extensions/laplace_redux/","page":"LaplaceRedux","title":"LaplaceRedux","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"extensions/laplace_redux/#[LaplaceRedux.jl](https://github.com/JuliaTrustworthyAI/LaplaceRedux.jl)","page":"LaplaceRedux","title":"LaplaceRedux.jl","text":"","category":"section"},{"location":"extensions/laplace_redux/","page":"LaplaceRedux","title":"LaplaceRedux","text":"LaplaceRedux.jl is one of Taija’s own packages that provides a framework for Effortless Bayesian Deep Learning through Laplace Approximation for Flux.jl neural networks. The methodology was first proposed by Immer, Korzepa, and Bauer (2020) and implemented in Python by Daxberger et al. (2021). This is relevant to the work on counterfactual explanations (CE), because research has shown that counterfactual explanations for Bayesian models are typically more plausible, because Bayesian models are able to capture the uncertainty in the data (Schut et al. 2021).","category":"page"},{"location":"extensions/laplace_redux/","page":"LaplaceRedux","title":"LaplaceRedux","text":"tip: Read More\nTo learn more about Laplace Redux, head over to the official documentation.","category":"page"},{"location":"extensions/laplace_redux/#Example","page":"LaplaceRedux","title":"Example","text":"","category":"section"},{"location":"extensions/laplace_redux/","page":"LaplaceRedux","title":"LaplaceRedux","text":"The extension will be loaded automatically when loading the LaplaceRedux package (assuming the CounterfactualExplanations package is also loaded).","category":"page"},{"location":"extensions/laplace_redux/","page":"LaplaceRedux","title":"LaplaceRedux","text":"using LaplaceRedux","category":"page"},{"location":"extensions/laplace_redux/","page":"LaplaceRedux","title":"LaplaceRedux","text":"Next, we will fit a neural network with Laplace Approximation to the moons dataset using our standard package API for doing so. By default, the Bayesian prior is optimized through empirical Bayes using the LaplaceRedux package.","category":"page"},{"location":"extensions/laplace_redux/","page":"LaplaceRedux","title":"LaplaceRedux","text":"# Fit model to data:\ndata = CounterfactualData(load_moons()...)\nM = fit_model(data, :LaplaceRedux; n_hidden=16)","category":"page"},{"location":"extensions/laplace_redux/","page":"LaplaceRedux","title":"LaplaceRedux","text":"LaplaceReduxExt.LaplaceReduxModel(Laplace(Chain(Dense(2 => 16, relu), Dense(16 => 2)), :classification, :all, nothing, :full, LaplaceRedux.Curvature.GGN(Chain(Dense(2 => 16, relu), Dense(16 => 2)), :classification, Flux.Losses.logitcrossentropy, Array{Float32}[[-1.3098596 0.59241515; 0.91760206 0.02950162; … ; -0.018356863 0.12850936; -0.5381665 -0.7872097], [-0.2581085, -0.90997887, -0.5418944, -0.23735572, 0.81020063, -0.3033359, -0.47902864, -0.6432098, -0.038013518, 0.028280666, 0.009903266, -0.8796683, 0.41090682, 0.011093224, -0.1580453, 0.7911349], [3.092321 -2.4660816 … -0.3446268 -1.465249; -2.9468734 3.167357 … 0.31758657 1.7140366], [-0.3107697, 0.31076983]], 1.0, :all, nothing), 1.0, 0.0, Float32[-1.3098596, 0.91760206, 0.5239727, -1.1579771, -0.851813, -1.9411169, 0.47409698, 0.6679365, 0.8944433, 0.663116 … -0.3172857, 0.15530388, 1.3264753, -0.3506721, -0.3446268, 0.31758657, -1.465249, 1.7140366, -0.3107697, 0.31076983], [0.10530027048093525 0.0 … 0.0 0.0; 0.0 0.10530027048093525 … 0.0 0.0; … ; 0.0 0.0 … 0.10530027048093525 0.0; 0.0 0.0 … 0.0 0.10530027048093525], [0.10066431429751965 0.0 … -0.030656783425475176 0.030656334963944154; 0.0 20.93513766443357 … -2.3185940232360736 2.3185965484008193; … ; -0.030656783425475176 -2.3185940232360736 … 1.0101450999063672 -1.0101448118057204; 0.030656334963944154 2.3185965484008193 … -1.0101448118057204 1.0101451389641771], [1.1006643142975197 0.0 … -0.030656783425475176 0.030656334963944154; 0.0 21.93513766443357 … -2.3185940232360736 2.3185965484008193; … ; -0.030656783425475176 -2.3185940232360736 … 2.0101450999063672 -1.0101448118057204; 0.030656334963944154 2.3185965484008193 … -1.0101448118057204 2.010145138964177], [0.9412600568016627 0.003106911671721699 … 0.003743740333409532 -0.003743452315572739; 0.003106912946573237 0.6539263732691709 … 0.0030385955287734246 -0.0030390041204196414; … ; 0.0037437406323562283 0.003038591829991259 … 0.9624905710233649 0.03750911813897676; -0.0037434526145225856 -0.0030390004216833593 … 0.03750911813898124 0.9624905774453485], 82, 250, 2, 997.8087484836578), :classification_multi)","category":"page"},{"location":"extensions/laplace_redux/","page":"LaplaceRedux","title":"LaplaceRedux","text":"Finally, we select a factual instance and generate a counterfactual explanation for it using the generic gradient-based CE method.","category":"page"},{"location":"extensions/laplace_redux/","page":"LaplaceRedux","title":"LaplaceRedux","text":"# Select a factual instance:\ntarget = 1\nfactual = 0\nchosen = rand(findall(predict_label(M, data) .== factual))\nx = select_factual(data, chosen)\n\n# Generate counterfactual explanation:\nη = 0.01\ngenerator = GenericGenerator(; opt=Descent(η), λ=0.01)\nconv = CounterfactualExplanations.Convergence.DecisionThresholdConvergence(;\n decision_threshold=0.9, max_iter=100\n)\nce = generate_counterfactual(x, target, data, M, generator; convergence=conv)\nplot(ce, alpha=0.1)","category":"page"},{"location":"extensions/laplace_redux/","page":"LaplaceRedux","title":"LaplaceRedux","text":"(Image: )","category":"page"},{"location":"extensions/laplace_redux/#References","page":"LaplaceRedux","title":"References","text":"","category":"section"},{"location":"extensions/laplace_redux/","page":"LaplaceRedux","title":"LaplaceRedux","text":"Daxberger, Erik, Agustinus Kristiadi, Alexander Immer, Runa Eschenhagen, Matthias Bauer, and Philipp Hennig. 2021. “Laplace Redux-Effortless Bayesian Deep Learning.” Advances in Neural Information Processing Systems 34.","category":"page"},{"location":"extensions/laplace_redux/","page":"LaplaceRedux","title":"LaplaceRedux","text":"Immer, Alexander, Maciej Korzepa, and Matthias Bauer. 2020. “Improving Predictions of Bayesian Neural Networks via Local Linearization.” https://arxiv.org/abs/2008.08400.","category":"page"},{"location":"extensions/laplace_redux/","page":"LaplaceRedux","title":"LaplaceRedux","text":"Schut, Lisa, Oscar Key, Rory Mc Grath, Luca Costabello, Bogdan Sacaleanu, Yarin Gal, et al. 2021. “Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertainties.” In International Conference on Artificial Intelligence and Statistics, 1756–64. PMLR.","category":"page"},{"location":"contribute/performance/","page":"-","title":"-","text":"Random.seed!(42)\n# Counteractual data and model:\ndata = TaijaData.load_linearly_separable()\ncounterfactual_data = DataPreprocessing.CounterfactualData(data...)\nM = fit_model(counterfactual_data, :Linear)\ntarget = 2\nfactual = 1\nchosen = rand(findall(predict_label(M, counterfactual_data) .== factual))\nx = select_factual(counterfactual_data, chosen)\n\n# Search:\ngenerator = GenericGenerator()\nce = generate_counterfactual(x, target, counterfactual_data, M, generator)","category":"page"},{"location":"contribute/performance/","page":"-","title":"-","text":"data_large = TaijaData.load_linearly_separable(100000)\ncounterfactual_data_large = DataPreprocessing.CounterfactualData(data_large...)","category":"page"},{"location":"contribute/performance/","page":"-","title":"-","text":"@time generate_counterfactual(x, target, counterfactual_data, M, generator)","category":"page"},{"location":"contribute/performance/","page":"-","title":"-","text":"@time generate_counterfactual(x, target, counterfactual_data_large, M, generator)","category":"page"},{"location":"explanation/generators/feature_tweak/","page":"FeatureTweak","title":"FeatureTweak","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"explanation/generators/feature_tweak/#FeatureTweakGenerator","page":"FeatureTweak","title":"FeatureTweakGenerator","text":"","category":"section"},{"location":"explanation/generators/feature_tweak/","page":"FeatureTweak","title":"FeatureTweak","text":"Feature Tweak refers to the generator introduced by Tolomei et al. (2017). Our implementation takes inspiration from the featureTweakPy library.","category":"page"},{"location":"explanation/generators/feature_tweak/#Description","page":"FeatureTweak","title":"Description","text":"","category":"section"},{"location":"explanation/generators/feature_tweak/","page":"FeatureTweak","title":"FeatureTweak","text":"Feature Tweak is a powerful recourse algorithm for ensembles of tree-based classifiers such as random forests. Though the problem of understanding how an input to an ensemble model could be transformed in such a way that the model changes its original prediction has been proven to be NP-hard (Tolomei et al. 2017), Feature Tweak provides an algorithm that manages to tractably solve this problem in multiple real-world applications. An example of a problem Feature Tweak is able to efficiently solve, explored in depth in Tolomei et al. (2017) is the problem of transforming an advertisement that has been classified by the ensemble model as a low-quality advertisement to a high-quality one through small changes to its features. With the help of Feature Tweak, advertisers can both learn about the reasons a particular ad was marked to have a low quality, as well as receive actionable suggestions about how to convert a low-quality ad into a high-quality one.","category":"page"},{"location":"explanation/generators/feature_tweak/","page":"FeatureTweak","title":"FeatureTweak","text":"Though Feature Tweak is a powerful way of avoiding brute-force search in an exponential search space, it does not come without disadvantages. The primary limitations of the approach are that it’s currently only applicable to tree-based classifiers and works only in the setting of binary classification. Another problem is that though the algorithm avoids exponential-time search, it is often still computationally expensive. The algorithm may be improved in the future to tackle all of these shortcomings.","category":"page"},{"location":"explanation/generators/feature_tweak/","page":"FeatureTweak","title":"FeatureTweak","text":"The following equation displays how a true negative instance x can be transformed into a positively predicted instance x’. To be more precise, x’ is the best possible transformation among all transformations **x***, computed with a cost function δ.","category":"page"},{"location":"explanation/generators/feature_tweak/","page":"FeatureTweak","title":"FeatureTweak","text":"beginaligned\nmathbfx^prime = arg_mathbfx^* min delta(mathbfx mathbfx^*) hatf(mathbfx) = -1 wedge hatf(mathbfx^*) = +1 \nendaligned","category":"page"},{"location":"explanation/generators/feature_tweak/#Example","page":"FeatureTweak","title":"Example","text":"","category":"section"},{"location":"explanation/generators/feature_tweak/","page":"FeatureTweak","title":"FeatureTweak","text":"In this example we apply the Feature Tweak algorithm to a decision tree and a random forest trained on the moons dataset. We first load the data and fit the models:","category":"page"},{"location":"explanation/generators/feature_tweak/","page":"FeatureTweak","title":"FeatureTweak","text":"n = 500\ncounterfactual_data = CounterfactualData(TaijaData.load_moons(n)...)\n\n# Classifiers\ndecision_tree = CounterfactualExplanations.Models.fit_model(\n counterfactual_data, :DecisionTree; max_depth=5, min_samples_leaf=3\n)\nforest = Models.fit_model(counterfactual_data, :RandomForest)","category":"page"},{"location":"explanation/generators/feature_tweak/","page":"FeatureTweak","title":"FeatureTweak","text":"Next, we select a point to explain and a target class to transform the point to. We then search for counterfactuals using the FeatureTweakGenerator:","category":"page"},{"location":"explanation/generators/feature_tweak/","page":"FeatureTweak","title":"FeatureTweak","text":"# Select a point to explain:\nx = float32.([1, -0.5])[:,:]\nfactual = Models.predict_label(forest, x)\ntarget = counterfactual_data.y_levels[findall(counterfactual_data.y_levels != factual)][1]\n\n# Search for counterfactuals:\ngenerator = FeatureTweakGenerator(ϵ=0.1)\ntree_counterfactual = generate_counterfactual(\n x, target, counterfactual_data, decision_tree, generator\n)\nforest_counterfactual = generate_counterfactual(\n x, target, counterfactual_data, forest, generator\n)","category":"page"},{"location":"explanation/generators/feature_tweak/","page":"FeatureTweak","title":"FeatureTweak","text":"The resulting counterfactuals are shown below:","category":"page"},{"location":"explanation/generators/feature_tweak/","page":"FeatureTweak","title":"FeatureTweak","text":"p1 = plot(\n tree_counterfactual;\n colorbar=false,\n title=\"Decision Tree\",\n)\n\np2 = plot(\n forest_counterfactual; title=\"Random Forest\",\n colorbar=false,\n)\n\ndisplay(plot(p1, p2; size=(800, 400)))","category":"page"},{"location":"explanation/generators/feature_tweak/","page":"FeatureTweak","title":"FeatureTweak","text":"(Image: )","category":"page"},{"location":"explanation/generators/feature_tweak/#References","page":"FeatureTweak","title":"References","text":"","category":"section"},{"location":"explanation/generators/feature_tweak/","page":"FeatureTweak","title":"FeatureTweak","text":"Tolomei, Gabriele, Fabrizio Silvestri, Andrew Haines, and Mounia Lalmas. 2017. “Interpretable Predictions of Tree-Based Ensembles via Actionable Feature Tweaking.” In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 465–74. https://doi.org/10.1145/3097983.3098039.","category":"page"},{"location":"explanation/generators/clue/","page":"CLUE","title":"CLUE","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"explanation/generators/clue/#CLUEGenerator","page":"CLUE","title":"CLUEGenerator","text":"","category":"section"},{"location":"explanation/generators/clue/","page":"CLUE","title":"CLUE","text":"In this tutorial, we introduce the CLUEGenerator, a counterfactual generator based on the Counterfactual Latent Uncertainty Explanations (CLUE) method proposed by Antorán et al. (2020).","category":"page"},{"location":"explanation/generators/clue/#Description","page":"CLUE","title":"Description","text":"","category":"section"},{"location":"explanation/generators/clue/","page":"CLUE","title":"CLUE","text":"The CLUEGenerator leverages differentiable probabilistic models, such as Bayesian Neural Networks (BNNs), to estimate uncertainty in predictions. It aims to provide interpretable counterfactual explanations by identifying input patterns that lead to predictive uncertainty. The generator utilizes a latent variable framework and employs a decoder from a variational autoencoder (VAE) to generate counterfactual samples in latent space.","category":"page"},{"location":"explanation/generators/clue/","page":"CLUE","title":"CLUE","text":"The CLUE algorithm minimizes a loss function that combines uncertainty estimates and the distance between the generated counterfactual and the original input. By optimizing this loss function iteratively, the CLUEGenerator generates counterfactuals that are similar to the original observation but assigned low uncertainty.","category":"page"},{"location":"explanation/generators/clue/","page":"CLUE","title":"CLUE","text":"The formula for predictive entropy is as follow:","category":"page"},{"location":"explanation/generators/clue/","page":"CLUE","title":"CLUE","text":"beginaligned\nH(y^*x^* D) = - sum_k=1^K p(y^*=c_kx^* D) log p(y^*=c_kx^* D)\nendaligned","category":"page"},{"location":"explanation/generators/clue/#Usage","page":"CLUE","title":"Usage","text":"","category":"section"},{"location":"explanation/generators/clue/","page":"CLUE","title":"CLUE","text":"While using one must keep in mind that the CLUE algorithim is meant to find a more robust datapoint of the same class, using CLUE generator without any additional penalties/losses will mean that it is not a counterfactual generator. The generated result will be of the same class as the original input, but a more robust datapoint.","category":"page"},{"location":"explanation/generators/clue/","page":"CLUE","title":"CLUE","text":"CLUE works best for BNN’s. The CLUEGenerator can be used with any differentiable probabilistic model, but the results may not be as good as with BNNs.","category":"page"},{"location":"explanation/generators/clue/","page":"CLUE","title":"CLUE","text":"The CLUEGenerator can be used in the following manner:","category":"page"},{"location":"explanation/generators/clue/","page":"CLUE","title":"CLUE","text":"generator = CLUEGenerator()\nM = fit_model(counterfactual_data, :DeepEnsemble)\nconv = CounterfactualExplanations.Convergence.MaxIterConvergence(max_iter=1000)\nce = generate_counterfactual(\n x, target, counterfactual_data, M, generator;\n convergence=conv)\nplot(ce)","category":"page"},{"location":"explanation/generators/clue/","page":"CLUE","title":"CLUE","text":"(Image: )","category":"page"},{"location":"explanation/generators/clue/","page":"CLUE","title":"CLUE","text":"Extra: The CLUE generator can also be used upon already having achieved a counterfactual with a different generator. In this case, you can use CLUE and make the counterfactual more robust.","category":"page"},{"location":"explanation/generators/clue/","page":"CLUE","title":"CLUE","text":"Note: The above documentation is based on the information provided in the CLUE paper. Please refer to the original paper for more detailed explanations and implementation specifics.","category":"page"},{"location":"explanation/generators/clue/#References","page":"CLUE","title":"References","text":"","category":"section"},{"location":"explanation/generators/clue/","page":"CLUE","title":"CLUE","text":"Antorán, Javier, Umang Bhatt, Tameem Adel, Adrian Weller, and José Miguel Hernández-Lobato. 2020. “Getting a Clue: A Method for Explaining Uncertainty Estimates.” https://arxiv.org/abs/2006.06848.","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"explanation/generators/revise/#REVISEGenerator","page":"REVISE","title":"REVISEGenerator","text":"","category":"section"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"REVISE is a Latent Space generator introduced by Joshi et al. (2019).","category":"page"},{"location":"explanation/generators/revise/#Description","page":"REVISE","title":"Description","text":"","category":"section"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"The current consensus in the literature is that Counterfactual Explanations should be realistic: the generated counterfactuals should look like they were generated by the data-generating process (DGP) that governs the problem at hand. With respect to Algorithmic Recourse, it is certainly true that counterfactuals should be realistic in order to be actionable for individuals.[1] To address this need, researchers have come up with various approaches in recent years. Among the most popular approaches is Latent Space Search, which was first proposed in Joshi et al. (2019): instead of traversing the feature space directly, this approach relies on a separate generative model that learns a latent space representation of the DGP. Assuming the generative model is well-specified, access to the learned latent embeddings of the data comes with two advantages:","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"Since the learned DGP is encoded in the latent space, the generated counterfactuals will respect the learned representation of the data. In practice, this means that counterfactuals will be realistic.\nThe latent space is typically a compressed (i.e. lower dimensional) version of the feature space. This makes the counterfactual search less costly.","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"There are also certain disadvantages though:","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"Learning generative models is (typically) an expensive task, which may well outweigh the benefits associated with utlimately traversing a lower dimensional space.\nIf the generative model is poorly specified, this will affect the quality of the counterfactuals.[2]","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"Anyway, traversing latent embeddings is a powerful idea that may be very useful depending on the specific context. This tutorial introduces the concept and how it is implemented in this package.","category":"page"},{"location":"explanation/generators/revise/#Usage","page":"REVISE","title":"Usage","text":"","category":"section"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"The approach can be used in our package as follows:","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"generator = REVISEGenerator()\nce = generate_counterfactual(x, target, counterfactual_data, M, generator)\nplot(ce)","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"(Image: )","category":"page"},{"location":"explanation/generators/revise/#Worked-2D-Examples","page":"REVISE","title":"Worked 2D Examples","text":"","category":"section"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"Below we load 2D data and train a VAE on it and plot the original samples against their reconstructions.","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"# output: true\n\ncounterfactual_data = CounterfactualData(load_overlapping()...)\nX = counterfactual_data.X\ny = counterfactual_data.y\ninput_dim = size(X, 1)\nusing CounterfactualExplanations.GenerativeModels: VAE, train!, reconstruct\nvae = VAE(input_dim; nll=Flux.Losses.mse, epochs=100, λ=0.01, latent_dim=2, hidden_dim=32)\nflux_training_params.verbose = true\ntrain!(vae, X, y)\nX̂ = reconstruct(vae, X)[1]\np0 = scatter(X[1, :], X[2, :], color=:blue, label=\"Original\", xlab=\"x₁\", ylab=\"x₂\")\nscatter!(X̂[1, :], X̂[2, :], color=:orange, label=\"Reconstructed\", xlab=\"x₁\", ylab=\"x₂\")\np1 = scatter(X[1, :], X̂[1, :], color=:purple, label=\"\", xlab=\"x₁\", ylab=\"x̂₁\")\np2 = scatter(X[2, :], X̂[2, :], color=:purple, label=\"\", xlab=\"x₂\", ylab=\"x̂₂\")\nplt2 = plot(p1,p2, layout=(1,2), size=(800, 400))\nplot(p0, plt2, layout=(2,1), size=(800, 600))","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"Next, we train a simple MLP for the classification task. Then we determine a target and factual class for our counterfactual search and select a random factual instance to explain.","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"M = fit_model(counterfactual_data, :MLP)\ntarget = 2\nfactual = 1\nchosen = rand(findall(predict_label(M, counterfactual_data) .== factual))\nx = select_factual(counterfactual_data, chosen)","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"Finally, we generate and visualize the generated counterfactual:","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"# Search:\ngenerator = REVISEGenerator()\nce = generate_counterfactual(x, target, counterfactual_data, M, generator)\nplot(ce)","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"(Image: )","category":"page"},{"location":"explanation/generators/revise/#3D-Example","page":"REVISE","title":"3D Example","text":"","category":"section"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"To illustrate the notion of Latent Space search, let’s look at an example involving 3-dimensional input data, which we can still visualize. The code chunk below loads the data and implements the counterfactual search.","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"# Data and Classifier:\ncounterfactual_data = CounterfactualData(load_blobs(k=3)...)\nX = counterfactual_data.X\nys = counterfactual_data.output_encoder.labels.refs\nM = fit_model(counterfactual_data, :MLP)\n\n# Randomly selected factual:\nx = select_factual(counterfactual_data,rand(1:size(counterfactual_data.X,2)))\ny = predict_label(M, counterfactual_data, x)[1]\ntarget = counterfactual_data.y_levels[counterfactual_data.y_levels .!= y][1]\n\n# Generate recourse:\nce = generate_counterfactual(x, target, counterfactual_data, M, generator)","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"The figure below demonstrates the idea of searching counterfactuals in a lower-dimensional latent space: on the left, we can see the counterfactual search in the 3-dimensional feature space, while on the right we can see the corresponding search in the latent space.","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"(Image: )","category":"page"},{"location":"explanation/generators/revise/#MNIST-data","page":"REVISE","title":"MNIST data","text":"","category":"section"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"Let’s carry the ideas introduced above over to a more complex example. The code below loads MNIST data as well as a pre-trained classifier and generative model for the data.","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"using CounterfactualExplanations.Models: load_mnist_mlp, load_mnist_ensemble, load_mnist_vae\ncounterfactual_data = CounterfactualData(load_mnist()...)\nX, y = CounterfactualExplanations.DataPreprocessing.unpack_data(counterfactual_data)\ninput_dim, n_obs = size(counterfactual_data.X)\nM = load_mnist_mlp()\nvae = load_mnist_vae()","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"The F1-score of our pre-trained image classifier on test data is: 0.94","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"Before continuing, we supply the pre-trained generative model to our data container:","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"counterfactual_data.generative_model = vae # assign generative model","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"Now let’s define a factual and target label:","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"# Randomly selected factual:\nRandom.seed!(2023)\nfactual_label = 8\nx = reshape(X[:,rand(findall(predict_label(M, counterfactual_data).==factual_label))],input_dim,1)\ntarget = 3\nfactual = predict_label(M, counterfactual_data, x)[1]","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"Using REVISE, we are going to turn a randomly drawn 8 into a 3.","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"The API call is the same as always:","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"γ = 0.95\nconv = \n CounterfactualExplanations.Convergence.DecisionThresholdConvergence(decision_threshold=γ)\n# Define generator:\ngenerator = REVISEGenerator(opt=Flux.Adam(0.1))\n# Generate recourse:\nce = generate_counterfactual(x, target, counterfactual_data, M, generator; convergence=conv)","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"The chart below shows the results:","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"(Image: )","category":"page"},{"location":"explanation/generators/revise/#References","page":"REVISE","title":"References","text":"","category":"section"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"Joshi, Shalmali, Oluwasanmi Koyejo, Warut Vijitbenjaronk, Been Kim, and Joydeep Ghosh. 2019. “Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems.” https://arxiv.org/abs/1907.09615.","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"[1] In general, we believe that there may be a trade-off between creating counterfactuals that respect the DGP vs. counterfactuals reflect the behaviour of the black-model in question - both accurately and complete.","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"[2] We believe that there is another potentially crucial disadvantage of relying on a separate generative model: it reallocates the task of learning realistic explanations for the data from the black-box model to the generative model.","category":"page"},{"location":"explanation/optimisers/overview/","page":"Overview","title":"Overview","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"explanation/optimisers/overview/#Optimisation-Rules","page":"Overview","title":"Optimisation Rules","text":"","category":"section"},{"location":"explanation/optimisers/overview/","page":"Overview","title":"Overview","text":"Counterfactual search is an optimization problem. Consequently, the choice of the optimisation rule affects the generated counterfactuals. In the short term, we aim to enable users to choose any of the available Flux optimisers. This has not been sufficiently tested yet, and you may run into issues.","category":"page"},{"location":"explanation/optimisers/overview/#Custom-Optimisation-Rules","page":"Overview","title":"Custom Optimisation Rules","text":"","category":"section"},{"location":"explanation/optimisers/overview/","page":"Overview","title":"Overview","text":"Flux optimisers are specifically designed for deep learning, and in particular, for learning model parameters. In counterfactual search, the features are the free parameters that we are optimising over. To this end, some custom optimisation rules are necessary to incorporate ideas presented in the literature. In the following, we introduce those rules.","category":"page"},{"location":"CHANGELOG/#Changelog","page":"Changelog","title":"Changelog","text":"","category":"section"},{"location":"CHANGELOG/","page":"Changelog","title":"Changelog","text":"All notable changes to this project will be documented in this file.","category":"page"},{"location":"CHANGELOG/","page":"Changelog","title":"Changelog","text":"The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.","category":"page"},{"location":"CHANGELOG/","page":"Changelog","title":"Changelog","text":"Note: We try to adhere to these practices as of version [v1.1.1].","category":"page"},{"location":"CHANGELOG/#Version-[v1.1.4]-2024-04-19","page":"Changelog","title":"Version [v1.1.4] - 2024-04-19","text":"","category":"section"},{"location":"CHANGELOG/#Changed","page":"Changelog","title":"Changed","text":"","category":"section"},{"location":"CHANGELOG/","page":"Changelog","title":"Changelog","text":"Refactors the encodings and decodings such that it is now more streamlined. Instead of conditional statements, encodings are now dispatched on the type of a new unifying data.input_encoder field. [#432]\nRefactors the check for redundancy. This is now based on the convergence type and done right before the counterfactual search begins, if not redundant. [#432]","category":"page"},{"location":"CHANGELOG/#Version-[v1.1.3]-2024-04-17","page":"Changelog","title":"Version [v1.1.3] - 2024-04-17","text":"","category":"section"},{"location":"CHANGELOG/#Added","page":"Changelog","title":"Added","text":"","category":"section"},{"location":"CHANGELOG/","page":"Changelog","title":"Changelog","text":"Adds a section on Convergence to the documentation, Changelog.jl functionality and a few doc tests. [#429]","category":"page"},{"location":"CHANGELOG/#Changed-2","page":"Changelog","title":"Changed","text":"","category":"section"},{"location":"CHANGELOG/","page":"Changelog","title":"Changelog","text":"Changes style of taking gradients for the counterfactual search from implicit to explicit. [#430]\nRemoved all implicit imports. [#430]","category":"page"},{"location":"CHANGELOG/#Removed","page":"Changelog","title":"Removed","text":"","category":"section"},{"location":"CHANGELOG/","page":"Changelog","title":"Changelog","text":"Removed CUDA.jl dependency, because redundant. [#430]\nRemoved Parameters.jl dependency, because redundant. [#430]","category":"page"},{"location":"CHANGELOG/#Version-[v1.1.2]-2024-04-16","page":"Changelog","title":"Version [v1.1.2] - 2024-04-16","text":"","category":"section"},{"location":"CHANGELOG/#Changed-3","page":"Changelog","title":"Changed","text":"","category":"section"},{"location":"CHANGELOG/","page":"Changelog","title":"Changelog","text":"Replaces the GIF in the README and introduction of docs for a static image. ","category":"page"},{"location":"CHANGELOG/#Version-[v1.1.1]-2024-04-15","page":"Changelog","title":"Version [v1.1.1] - 2024-04-15","text":"","category":"section"},{"location":"CHANGELOG/#Added-2","page":"Changelog","title":"Added","text":"","category":"section"},{"location":"CHANGELOG/","page":"Changelog","title":"Changelog","text":"Added tests for LaplaceRedux extension. Bumped upper compat bound for LaplaceRedux.jl. [#428]","category":"page"},{"location":"CHANGELOG/","page":"Changelog","title":"Changelog","text":"","category":"page"},{"location":"CHANGELOG/","page":"Changelog","title":"Changelog","text":"[#428]: https://github.com/juliatrustworthyai/CounterfactualExplanations.jl/issues/428 [#429]: https://github.com/juliatrustworthyai/CounterfactualExplanations.jl/issues/429","category":"page"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"tutorials/convergence/#convergence","page":"Convergence","title":"Convergence","text":"","category":"section"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"The search for counterfactuals can be seen as an optimization problem, where the goal is to find a point in the input space. One questions that has received surprisingly little attention is how to determine when the search has converged. In a recent paper, we have briefly discussed why it is important to consider convergence (Altmeyer et al. 2024):","category":"page"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"One intuitive way to specify convergence is in terms of threshold probabilities: once the predicted probability p(y^+x^prime) exceeds some user-defined threshold γ such that the counterfactual is valid, we could consider the search to have converged. In the binary case, for example, convergence could be defined as p(y^+x^prime) 05 in this sense. Note, however, how this can be expected to yield counterfactuals in the proximity of the decision boundary, a region characterized by high aleatoric uncertainty. In other words, counterfactuals generated in this way would generally not be plausible. To avoid this from happening, we specify convergence in terms of gradients approaching zero for all our experiments and all of our generators. This is allows us to get a cleaner read on how the different counterfactual search objectives affect counterfactual outcomes.","category":"page"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"In the paper, we were primarily interested in benchmarking counterfactuals generated by different search objectives. In other contexts, however, it may be more appropriate to specify convergence in terms of threshold probabilities. Our package allows you to specify convergence in terms of gradients, threshold probabilities or simply in terms of the total number of iterations. In this section, we will show you how to do this.","category":"page"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"using CounterfactualExplanations.Convergence\ngenerator = GenericGenerator(λ=0.01)","category":"page"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"GradientBasedGenerator(nothing, CounterfactualExplanations.Objectives.distance_l1, 0.01, false, false, Descent(0.1), NamedTuple())","category":"page"},{"location":"tutorials/convergence/#Convergence-in-terms-of-gradients","page":"Convergence","title":"Convergence in terms of gradients","text":"","category":"section"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"As gradients approach zero, the conditions defined by the search objective and hence the generator are satisfied. We therefore refere to this type of convergece criterium as GeneratorConditionsConvergence","category":"page"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"conv = GeneratorConditionsConvergence(gradient_tol=0.01, max_iter=1000)\nce_gen = generate_counterfactual(x, target, counterfactual_data, M, generator; convergence = conv)","category":"page"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"CounterfactualExplanation\nConvergence: ✅ after 179 steps.","category":"page"},{"location":"tutorials/convergence/#Convergence-in-terms-of-threshold-probabilities","page":"Convergence","title":"Convergence in terms of threshold probabilities","text":"","category":"section"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"In this case, the search is considered to have converged once the predicted probability p(y^+x^prime) exceeds some user-defined threshold γ such that the counterfactual is valid. We refer to this type of convergence criterium as DecisionThresholdConvergence.","category":"page"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"conv = DecisionThresholdConvergence(decision_threshold=0.75)\nce_dec = generate_counterfactual(x, target, counterfactual_data, M, generator; convergence = conv)","category":"page"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"CounterfactualExplanation\nConvergence: ✅ after 9 steps.","category":"page"},{"location":"tutorials/convergence/#Convergence-in-terms-of-the-total-number-of-iterations","page":"Convergence","title":"Convergence in terms of the total number of iterations","text":"","category":"section"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"In this case, the search is considered to have converged once the total number of iterations exceeds some user-defined threshold max_iter. We refer to this type of convergence criterium as MaxIterConvergence.","category":"page"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"conv = MaxIterConvergence(max_iter=25)\nce_max = generate_counterfactual(x, target, counterfactual_data, M, generator; convergence = conv)","category":"page"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"CounterfactualExplanation\nConvergence: ✅ after 25 steps.","category":"page"},{"location":"tutorials/convergence/#Comparison","page":"Convergence","title":"Comparison","text":"","category":"section"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"plts = []\nfor (ce, titl) in zip([ce_gen, ce_dec, ce_max], [\"Gradient Convergence\", \"Decision Threshold Convergence\", \"Max Iterations Convergence\"])\n push!(plts, plot(ce; title=titl, cbar=false))\nend\nplot(plts..., layout=(1,3), size=(1200, 380))","category":"page"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"(Image: )","category":"page"},{"location":"tutorials/convergence/#References","page":"Convergence","title":"References","text":"","category":"section"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"Altmeyer, Patrick, Mojtaba Farmanbar, Arie van Deursen, and Cynthia CS Liem. 2024. “Faithful Model Explanations Through Energy-Constrained Conformal Counterfactuals.” In Proceedings of the AAAI Conference on Artificial Intelligence, 38:10829–37. 10.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"tutorials/data_preprocessing/#Handling-Data","page":"Handling Data","title":"Handling Data","text":"","category":"section"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"The package works with custom data containers that contain the input and output data as well as information about the type and mutability of features. In this tutorial, we will see how data can be prepared for use with the package.","category":"page"},{"location":"tutorials/data_preprocessing/#Basic-Functionality","page":"Handling Data","title":"Basic Functionality","text":"","category":"section"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"To demonstrate the basic way to prepare data, let’s look at a standard benchmark dataset: Fisher’s classic iris dataset. We can use MLDatasets to load this data.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"dataset = Iris()","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"Our data constructor CounterfactualData needs at least two inputs: features X and targets y.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"X = dataset.features\ny = dataset.targets","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"Next, we convert the input data to a Tables.MatrixTable (following MLJ.jl) convention. Concerning the target variable, we just assign grab the first column of the data frame.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"X = table(Tables.matrix(X))\ny = y[:,1]","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"Now we can feed these two ingredients to our constructor:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"counterfactual_data = CounterfactualData(X, y)","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"Under the hood, the constructor performs basic preprocessing steps. For example, the output variable y is automatically one-hot encoded:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"counterfactual_data.y","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"3×150 Matrix{Bool}:\n 1 1 1 1 1 1 1 1 1 1 1 1 1 … 0 0 0 0 0 0 0 0 0 0 0 0\n 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"Similarly, a transformer used to scale continuous input features is automatically fitted:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"counterfactual_data.dt","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"ZScoreTransform{Float64, Vector{Float64}}(4, 2, [5.843333333333335, 3.0540000000000007, 3.7586666666666693, 1.1986666666666672], [0.8280661279778629, 0.4335943113621737, 1.7644204199522617, 0.7631607417008414])","category":"page"},{"location":"tutorials/data_preprocessing/#Categorical-Features","page":"Handling Data","title":"Categorical Features","text":"","category":"section"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"For the counterfactual search, it is important to distinguish between continuous and categorical features. This is because categorical features cannot be perturbed arbitrarily: they can take specific discrete values, but not just any value on the real line.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"Consider the following example:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"y = rand([1,0],4)\nX = (\n name=categorical([\"Danesh\", \"Lee\", \"Mary\", \"John\"]),\n grade=categorical([\"A\", \"B\", \"A\", \"C\"], ordered=true),\n sex=categorical([\"male\",\"female\",\"male\",\"male\"]),\n height=[1.85, 1.67, 1.5, 1.67],\n)\nschema(X)","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"┌────────┬──────────────────┬──────────────────────────────────┐\n│ names │ scitypes │ types │\n├────────┼──────────────────┼──────────────────────────────────┤\n│ name │ Multiclass{4} │ CategoricalValue{String, UInt32} │\n│ grade │ OrderedFactor{3} │ CategoricalValue{String, UInt32} │\n│ sex │ Multiclass{2} │ CategoricalValue{String, UInt32} │\n│ height │ Continuous │ Float64 │\n└────────┴──────────────────┴──────────────────────────────────┘","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"Typically, in the context of Unserpervised Learning, categorical features are one-hot or dummy encoded. To this end, we could use MLJ, for example:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"hot = OneHotEncoder()\nmach = MLJBase.fit!(machine(hot, X))\nW = MLJBase.transform(mach, X)\nX = permutedims(MLJBase.matrix(W))","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"In all likelihood, this pre-processing step already happens at the stage, when the supervised model is trained. Since our counterfactual generators need to work in the same feature domain as the model they are intended to explain, we assume that categorical features are already encoded.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"The CounterfactualData constructor takes two optional arguments that can be used to specify the indices of categorical and continuous features. By default, all features are assumed to be continuous. For categorical features, the constructor expects an array of arrays of integers (Vector{Vector{Int}}) where each subarray includes the indices of all one-hot encoded rows related to a single categorical feature. In the example above, the name feature is one-hot encoded across rows 1, 2, 3 and 4 of X, the grade feature is encoded across the following three rows, etc.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"schema(W)","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"┌──────────────┬────────────┬─────────┐\n│ names │ scitypes │ types │\n├──────────────┼────────────┼─────────┤\n│ name__Danesh │ Continuous │ Float64 │\n│ name__John │ Continuous │ Float64 │\n│ name__Lee │ Continuous │ Float64 │\n│ name__Mary │ Continuous │ Float64 │\n│ grade__A │ Continuous │ Float64 │\n│ grade__B │ Continuous │ Float64 │\n│ grade__C │ Continuous │ Float64 │\n│ sex__female │ Continuous │ Float64 │\n│ sex__male │ Continuous │ Float64 │\n│ height │ Continuous │ Float64 │\n└──────────────┴────────────┴─────────┘","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"The code chunk below assigns the categorical and continuous feature indices:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"features_categorical = [\n [1,2,3,4], # name\n [5,6,7], # grade\n [8,9] # sex\n]\nfeatures_continuous = [10]","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"When instantiating the data container, these indices just need to be supplied as keyword arguments:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"counterfactual_data = CounterfactualData(\n X,y;\n features_categorical = features_categorical,\n features_continuous = features_continuous\n)","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"This will ensure that the discrete domain of categorical features is respected in the counterfactual search. We achieve this through a form of Projected Gradient Descent and it works for any of our counterfactual generators.","category":"page"},{"location":"tutorials/data_preprocessing/#Example","page":"Handling Data","title":"Example","text":"","category":"section"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"To see this in action, let’s load some synthetic data using MLJ:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"N = 1000\nX, ys = MLJBase.make_blobs(N, 2; centers=2, as_table=false, center_box=(-5 => 5), cluster_std=0.5)\nys .= ys.==2","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"Next, we generate a synthetic categorical feature based on the output variable. First, we define the discrete levels:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"cat_values = [\"X\",\"Y\",\"Z\"]","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"Next, we impose that the categorical feature is most likely to take the first discrete level, namely X, whenever y is equal to 1.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"xcat = map(ys) do y\n if y==1\n x = sample(cat_values, Weights([0.8,0.1,0.1]))\n else\n x = sample(cat_values, Weights([0.1,0.1,0.8]))\n end\nend\nxcat = categorical(xcat)\nX = (\n x1 = X[:,1],\n x2 = X[:,2],\n x3 = xcat\n)\nschema(X)","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"As above, we use a OneHotEncoder to transform the data:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"hot = OneHotEncoder()\nmach = MLJBase.fit!(machine(hot, X))\nW = MLJBase.transform(mach, X)\nschema(W)\nX = permutedims(MLJBase.matrix(W))","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"Finally, we assign the categorical indices and instantiate our data container:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"features_categorical = [collect(3:size(X,1))]\ncounterfactual_data = CounterfactualData(\n X,ys';\n features_categorical = features_categorical,\n)","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"With the data pre-processed we can use the fit_model function to train a simple classifier:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"M = fit_model(counterfactual_data, :Linear)","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"Now it is finally time to generate counterfactuals. We first define 1 as our target and then choose a random sample from the non-target class:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"target = 1\nfactual = 0\nchosen = rand(findall(predict_label(M, counterfactual_data) .== factual))\nx = select_factual(counterfactual_data, chosen) ","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"5×1 Matrix{Float32}:\n -2.9433472\n 0.5782963\n 0.0\n 0.0\n 1.0","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"The factual x belongs to group Z.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"We generate a counterfactual for x using the standard API call:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"generator = GenericGenerator()\nce = generate_counterfactual(x, target, counterfactual_data, M, generator)","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"CounterfactualExplanation\nConvergence: ✅ after 7 steps.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"The search yields the following counterfactual:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"x′ = counterfactual(ce)","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"5-element Vector{Float32}:\n 1.1222683\n 0.7145791\n 0.0\n 0.0\n 1.0","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"It belongs to group Z.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"This is intuitive because by construction the categorical variable is most likely to take that value when y is equal to the target outcome.","category":"page"},{"location":"tutorials/data_preprocessing/#Immutable-Features","page":"Handling Data","title":"Immutable Features","text":"","category":"section"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"In practice, features usually cannot be perturbed arbitrarily. Suppose, for example, that one of the features used by a bank to predict the creditworthiness of its clients is gender. If a counterfactual explanation for the prediction model indicates that female clients should change their gender to improve their creditworthiness, then this is an interesting insight (it reveals gender bias), but it is not usually an actionable transformation in practice. In such cases, we may want to constrain the mutability of features to ensure actionable and realistic recourse.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"To illustrate how this can be implemented in CounterfactualExplanations.jl we will continue to work with the synthetic data from the previous section. Mutability of features can be defined in terms of four different options: 1) the feature is mutable in both directions, 2) the feature can only increase (e.g. age), 3) the feature can only decrease (e.g. time left until your next deadline) and 4) the feature is not mutable (e.g. skin colour, ethnicity, …). To specify which category a feature belongs to, you can pass a vector of symbols containing the mutability constraints at the pre-processing stage. For each feature you can choose from these four options: :both (mutable in both directions), :increase (only up), :decrease (only down) and :none (immutable). By default, nothing is passed to that keyword argument and it is assumed that all features are mutable in both directions.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"Below we impose that the second feature is immutable.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"counterfactual_data = CounterfactualData(load_linearly_separable()...)\nM = fit_model(counterfactual_data, :Linear)\ncounterfactual_data.mutability = [:both, :none]","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"target = 2\nfactual = 1\nchosen = rand(findall(predict_label(M, counterfactual_data) .== factual))\nx = select_factual(counterfactual_data, chosen) \nce = generate_counterfactual(x, target, counterfactual_data, M, generator)","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"The resulting counterfactual path is shown in the chart below. Since only the first feature can be perturbed, the sample can only move along the horizontal axis.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"plot(ce)","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"(Image: ) ","category":"page"},{"location":"explanation/generators/overview/","page":"Overview","title":"Overview","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"explanation/generators/overview/#generators_explanation","page":"Overview","title":"Counterfactual Generators","text":"","category":"section"},{"location":"explanation/generators/overview/","page":"Overview","title":"Overview","text":"Counterfactual generators form the very core of this package. The generator_catalogue can be used to inspect the available generators:","category":"page"},{"location":"explanation/generators/overview/","page":"Overview","title":"Overview","text":"generator_catalogue","category":"page"},{"location":"explanation/generators/overview/","page":"Overview","title":"Overview","text":"Dict{Symbol, Any} with 11 entries:\n :gravitational => GravitationalGenerator\n :growing_spheres => GrowingSpheresGenerator\n :revise => REVISEGenerator\n :clue => CLUEGenerator\n :probe => ProbeGenerator\n :dice => DiCEGenerator\n :feature_tweak => FeatureTweakGenerator\n :claproar => ClaPROARGenerator\n :wachter => WachterGenerator\n :generic => GenericGenerator\n :greedy => GreedyGenerator","category":"page"},{"location":"explanation/generators/overview/","page":"Overview","title":"Overview","text":"The following sections provide brief descriptions of all of them.","category":"page"},{"location":"explanation/generators/overview/#Gradient-based-Counterfactual-Generators","page":"Overview","title":"Gradient-based Counterfactual Generators","text":"","category":"section"},{"location":"explanation/generators/overview/","page":"Overview","title":"Overview","text":"At the time of writing, all generators are gradient-based: that is, counterfactuals are searched through gradient descent. In Altmeyer et al. (2023) we lay out a general methodological framework that can be applied to all of these generators:","category":"page"},{"location":"explanation/generators/overview/","page":"Overview","title":"Overview","text":"beginaligned\nmathbfs^prime = arg min_mathbfs^prime in mathcalS left textyloss(M(f(mathbfs^prime))y^*)+ lambda textcost(f(mathbfs^prime)) right \nendaligned ","category":"page"},{"location":"explanation/generators/overview/","page":"Overview","title":"Overview","text":"“Here mathbfs^prime=lefts_k^primeright_K is a K-dimensional array of counterfactual states and f mathcalS mapsto mathcalX maps from the counterfactual state space to the feature space.” (Altmeyer et al. 2023)","category":"page"},{"location":"explanation/generators/overview/","page":"Overview","title":"Overview","text":"For most generators, the state space is the feature space (f is the identity function) and the number of counterfactuals K is one. Latent Space generators instead search counterfactuals in some latent space mathcalS. In this case, f corresponds to the decoder part of the generative model, that is the function that maps back from the latent space to inputs.","category":"page"},{"location":"explanation/generators/overview/#References","page":"Overview","title":"References","text":"","category":"section"},{"location":"explanation/generators/overview/","page":"Overview","title":"Overview","text":"Altmeyer, Patrick, Giovan Angela, Aleksander Buszydlik, Karol Dobiczek, Arie van Deursen, and Cynthia Liem. 2023. “Endogenous Macrodynamics in Algorithmic Recourse.” In First IEEE Conference on Secure and Trustworthy Machine Learning. https://doi.org/10.1109/satml54575.2023.00036.","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"tutorials/model_catalogue/#Model-Catalogue","page":"Model Catalogue","title":"Model Catalogue","text":"","category":"section"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"While in general it is assumed that users will use this package to explain their pre-trained models, we provide out-of-the-box functionality to train various simple default models. In this tutorial, we will see how these models can be fitted to CounterfactualData.","category":"page"},{"location":"tutorials/model_catalogue/#Available-Models","page":"Model Catalogue","title":"Available Models","text":"","category":"section"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The standard_models_catalogue can be used to inspect the available default models:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"standard_models_catalogue","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Dict{Symbol, Any} with 4 entries:\n :Linear => Linear\n :LaplaceRedux => LaplaceReduxModel\n :DeepEnsemble => FluxEnsemble\n :MLP => FluxModel","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The dictionary keys correspond to the model names. In this case, the dictionary values are constructors that can be used called on instances of type CounterfactualData to fit the corresponding model. In most cases, users will find it most convenient to use the fit_model API call instead.","category":"page"},{"location":"tutorials/model_catalogue/#Fitting-Models","page":"Model Catalogue","title":"Fitting Models","text":"","category":"section"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Models from the standard model catalogue are a core part of the package and thus compatible with all offered counterfactual generators and functionalities.","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The all_models_catalogue can be used to inspect all models offered by the package:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"all_models_catalogue","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"However, when using models not included in the standard_models_catalogue, additional caution is advised: they might not be supported by all counterfactual generators or they might not be models native to Julia. Thus, a more thorough reading of their documentation may be necessary to make sure that they are used correctly.","category":"page"},{"location":"tutorials/model_catalogue/#Fitting-Flux-Models","page":"Model Catalogue","title":"Fitting Flux Models","text":"","category":"section"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"First, let’s load one of the synthetic datasets. For this, we’ll first need to import the TaijaData.jl package:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"n = 500\ndata = TaijaData.load_multi_class(n)\ncounterfactual_data = DataPreprocessing.CounterfactualData(data...)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"We could use a Deep Ensemble (Lakshminarayanan, Pritzel, and Blundell 2016) as follows:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"M = fit_model(counterfactual_data, :DeepEnsemble)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The returned object is an instance of type FluxEnsemble <: AbstractFittedModel and can be used in downstream tasks without further ado. For example, the resulting fit can be visualised using the generic plot() method as:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"plts = []\nfor target in counterfactual_data.y_levels\n plt = plot(M, counterfactual_data; target=target, title=\"p(y=$(target)|x,θ)\")\n plts = [plts..., plt]\nend\nplot(plts...)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"(Image: )","category":"page"},{"location":"tutorials/model_catalogue/#Importing-PyTorch-models","page":"Model Catalogue","title":"Importing PyTorch models","text":"","category":"section"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The package supports generating counterfactuals for any neural network that has been previously defined and trained using PyTorch, regardless of the specific architectural details of the model. To generate counterfactuals for a PyTorch model, save the model inside a .pt file and call the following function:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"model_loaded = TaijaInteroperability.pytorch_model_loader(\n \"$(pwd())/docs/src/tutorials/miscellaneous\",\n \"neural_network_class\",\n \"NeuralNetwork\",\n \"$(pwd())/docs/src/tutorials/miscellaneous/pretrained_model.pt\"\n)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The method pytorch_model_loader requires four arguments:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The path to the folder with a .py file where the PyTorch model is defined\nThe name of the file where the PyTorch model is defined\nThe name of the class of the PyTorch model\nThe path to the Pickle file that holds the model weights","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"In the above case:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The file defining the model is inside $(pwd())/docs/src/tutorials/miscellaneous\nThe name of the .py file holding the model definition is neural_network_class\nThe name of the model class is NeuralNetwork\nThe Pickle file is located at $(pwd())/docs/src/tutorials/miscellaneous/pretrained_model.pt","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Though the model file and Pickle file are inside the same directory in this tutorial, this does not necessarily have to be the case.","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The reason why the model file and Pickle file have to be provided separately is that the package expects an already trained PyTorch model as input. It is also possible to define new PyTorch models within the package, but since this is not the expected use of our package, special support is not offered for that. A guide for defining Python and PyTorch classes in Julia through PythonCall.jl can be found here.","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Once the PyTorch model has been loaded into the package, wrap it inside the PyTorchModel class:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"model_pytorch = TaijaInteroperability.PyTorchModel(model_loaded, counterfactual_data.likelihood)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"This model can now be passed into the generators like any other.","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Please note that the functionality for generating counterfactuals for Python models is only available if your Julia version is 1.8 or above. For Julia 1.7 users, we recommend upgrading the version to 1.8 or 1.9 before loading a PyTorch model into the package.","category":"page"},{"location":"tutorials/model_catalogue/#Importing-R-torch-models","page":"Model Catalogue","title":"Importing R torch models","text":"","category":"section"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"warning: Not fully tested\nPlease note that due to the incompatibility between RCall and PythonCall, it is not feasible to test both PyTorch and RTorch implementations within the same pipeline. While the RTorch implementation has been manually tested, we cannot ensure its consistent functionality as it is inherently susceptible to bugs.","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The CounterfactualExplanations package supports generating counterfactuals for neural networks that have been defined and trained using R torch. Regardless of the specific architectural details of the model, you can easily generate counterfactual explanations by following these steps.","category":"page"},{"location":"tutorials/model_catalogue/#Saving-the-R-torch-model","page":"Model Catalogue","title":"Saving the R torch model","text":"","category":"section"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"First, save your trained R torch model as a .pt file using the torch_save() function provided by the R torch library. This function allows you to serialize the model and save it to a file. For example:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"torch_save(model, file = \"$(pwd())/docs/src/tutorials/miscellaneous/r_model.pt\")","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Make sure to specify the correct file path where you want to save the model.","category":"page"},{"location":"tutorials/model_catalogue/#Loading-the-R-torch-model","page":"Model Catalogue","title":"Loading the R torch model","text":"","category":"section"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"To import the R torch model into the CounterfactualExplanations package, use the rtorch_model_loader() function. This function loads the model from the previously saved .pt file. Here is an example of how to load the R torch model:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"model_loaded = TaijaInteroperability.rtorch_model_loader(\"$(pwd())/docs/src/tutorials/miscellaneous/r_model.pt\")","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The rtorch_model_loader() function requires only one argument:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"model_path: The path to the .pt file that contains the trained R torch model.","category":"page"},{"location":"tutorials/model_catalogue/#Wrapping-the-R-torch-model","page":"Model Catalogue","title":"Wrapping the R torch model","text":"","category":"section"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Once the R torch model has been loaded into the package, wrap it inside the RTorchModel class. This step prepares the model to be used by the counterfactual generators. Here is an example:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"model_R = TaijaInteroperability.RTorchModel(model_loaded, counterfactual_data.likelihood)","category":"page"},{"location":"tutorials/model_catalogue/#Generating-counterfactuals-with-the-R-torch-model","page":"Model Catalogue","title":"Generating counterfactuals with the R torch model","text":"","category":"section"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Now that the R torch model has been wrapped inside the RTorchModel class, you can pass it into the counterfactual generators as you would with any other model.","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Please note that RCall is not fully compatible with PythonCall. Therefore, it is advisable not to import both R torch and PyTorch models within the same Julia session. Additionally, it’s worth mentioning that the R torch integration is still untested in the CounterfactualExplanations package.","category":"page"},{"location":"tutorials/model_catalogue/#Tuning-Flux-Models","page":"Model Catalogue","title":"Tuning Flux Models","text":"","category":"section"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"By default, model architectures are very simple. Through optional arguments, users have some control over the neural network architecture and can choose to impose regularization through dropout. Let’s tackle a more challenging dataset: MNIST (LeCun 1998).","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"data = TaijaData.load_mnist(10000)\ncounterfactual_data = DataPreprocessing.CounterfactualData(data...)\ntrain_data, test_data = \n CounterfactualExplanations.DataPreprocessing.train_test_split(counterfactual_data)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"(Image: )","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"In this case, we will use a Multi-Layer Perceptron (MLP) but we will adjust the model and training hyperparameters. Parameters related to training of Flux.jl models are currently stored in a mutable container:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"flux_training_params","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"CounterfactualExplanations.FluxModelParams\n loss: Symbol logitbinarycrossentropy\n opt: Symbol Adam\n n_epochs: Int64 100\n batchsize: Int64 1\n verbose: Bool false","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"In cases like this one, where model training can be expected to take a few moments, it can be useful to activate verbosity, so let’s set the corresponding field value to true. We’ll also impose mini-batch training:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"flux_training_params.verbose = true\nflux_training_params.batchsize = round(size(train_data.X,2)/10)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"To account for the fact that this is a slightly more challenging task, we will use an appropriate number of hidden neurons per layer. We will also activate dropout regularization. To scale networks up further, it is also possible to adjust the number of hidden layers, which we will not do here.","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"model_params = (\n n_hidden = 32,\n dropout = true\n)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The model_params can be supplied to the familiar API call:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"M = fit_model(train_data, :MLP; model_params...)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"FluxModel(Chain(Dense(784 => 32, relu), Dropout(0.25, active=false), Dense(32 => 10)), :classification_multi)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The model performance on our test set can be evaluated as follows:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"model_evaluation(M, test_data)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"1-element Vector{Float64}:\n 0.9185","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Finally, let’s restore the default training parameters:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"CounterfactualExplanations.reset!(flux_training_params)","category":"page"},{"location":"tutorials/model_catalogue/#Fitting-and-tuning-MLJ-models","page":"Model Catalogue","title":"Fitting and tuning MLJ models","text":"","category":"section"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Among models from the MLJ library, two models are integrated as part of the core functionality of the package:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"mlj_models_catalogue","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"These models are compatible with the Feature Tweak generator. Support for other generators has not been implemented, as both decision trees and random forests are non-differentiable tree-based models and thus, gradient-based generators don’t apply for them.","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Tuning MLJ models is very simple. As the first step, let’s reload the dataset:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"n = 500\ndata = TaijaData.load_moons(n)\ncounterfactual_data = DataPreprocessing.CounterfactualData(data...)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Using the usual procedure for fitting models, we can call the following method:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"tree = CounterfactualExplanations.Models.fit_model(counterfactual_data, :DecisionTree)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"However, it’s also possible to tune the DecisionTreeClassifier’s parameters. This can be done using the keyword arguments when calling fit_model() as follows:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"tree = CounterfactualExplanations.Models.fit_model(counterfactual_data, :DecisionTree; max_depth=2, min_samples_leaf=3)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"For all supported MLJ models, every tunable parameter they have is supported as a keyword argument. The tunable parameters for the DecisionTreeModel and the RandomForestModel can be found from the documentation of the DecisionTree.jl package under the Decision Tree Classifier and Random Forest Classifier sections.","category":"page"},{"location":"tutorials/model_catalogue/#Package-extension-models","page":"Model Catalogue","title":"Package extension models","text":"","category":"section"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The package also includes two models which don’t form a part of the core functionality of the package, but which can be accessed as package extensions. These are the EvoTreeModel from the MLJ library and the LaplaceReduxModel from LaplaceRedux.jl.","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"To trigger the package extensions, the weak dependency first has to be loaded with the using keyword:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"using EvoTrees","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Once this is done, the extension models can be used like any other model:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"M = fit_model(counterfactual_data, :EvoTree; model_params...)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"EvoTreesExt.EvoTreeModel(machine(EvoTreeClassifier{EvoTrees.MLogLoss}\n - nrounds: 100\n - L2: 0.0\n - lambda: 0.0\n - gamma: 0.0\n - eta: 0.1\n - max_depth: 6\n - min_weight: 1.0\n - rowsample: 1.0\n - colsample: 1.0\n - nbins: 64\n - alpha: 0.5\n - tree_type: binary\n - rng: MersenneTwister(123, (0, 9018, 8016, 884))\n, …), :classification_multi)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The tunable parameters for the EvoTreeModel can be found from the documentation of the EvoTrees.jl package under the EvoTreeClassifier section.","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Please note that support for counterfactual generation with both LaplaceReduxModel and EvoTreeModel is not yet fully implemented.","category":"page"},{"location":"tutorials/model_catalogue/#References","page":"Model Catalogue","title":"References","text":"","category":"section"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Lakshminarayanan, Balaji, Alexander Pritzel, and Charles Blundell. 2016. “Simple and Scalable Predictive Uncertainty Estimation Using Deep Ensembles.” https://arxiv.org/abs/1612.01474.","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"LeCun, Yann. 1998. “The MNIST Database of Handwritten Digits.”","category":"page"},{"location":"assets/resources/#Further-Resources","page":"📚 Additional Resources","title":"Further Resources","text":"","category":"section"},{"location":"assets/resources/#JuliaCon-2022","page":"📚 Additional Resources","title":"JuliaCon 2022","text":"","category":"section"},{"location":"assets/resources/","page":"📚 Additional Resources","title":"📚 Additional Resources","text":"Slides: link","category":"page"},{"location":"assets/resources/#JuliaCon-Proceedings-Paper","page":"📚 Additional Resources","title":"JuliaCon Proceedings Paper","text":"","category":"section"},{"location":"assets/resources/","page":"📚 Additional Resources","title":"📚 Additional Resources","text":"TBD","category":"page"},{"location":"reference/","page":"🧐 Reference","title":"🧐 Reference","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"reference/#Reference","page":"🧐 Reference","title":"Reference","text":"","category":"section"},{"location":"reference/","page":"🧐 Reference","title":"🧐 Reference","text":"In this reference, you will find a detailed overview of the package API.","category":"page"},{"location":"reference/","page":"🧐 Reference","title":"🧐 Reference","text":"Reference guides are technical descriptions of the machinery and how to operate it. Reference material is information-oriented.— Diátaxis","category":"page"},{"location":"reference/","page":"🧐 Reference","title":"🧐 Reference","text":"In other words, you come here because you want to take a very close look at the code 🧐.","category":"page"},{"location":"reference/#Content","page":"🧐 Reference","title":"Content","text":"","category":"section"},{"location":"reference/","page":"🧐 Reference","title":"🧐 Reference","text":"Pages = [\"reference.md\"]\nDepth = 2","category":"page"},{"location":"reference/#Exported-functions","page":"🧐 Reference","title":"Exported functions","text":"","category":"section"},{"location":"reference/","page":"🧐 Reference","title":"🧐 Reference","text":"Modules = [\n CounterfactualExplanations, \n CounterfactualExplanations.Convergence,\n CounterfactualExplanations.Evaluation,\n CounterfactualExplanations.DataPreprocessing,\n CounterfactualExplanations.Models,\n CounterfactualExplanations.GenerativeModels, \n CounterfactualExplanations.Generators, \n CounterfactualExplanations.Objectives\n]\nPrivate = false","category":"page"},{"location":"reference/#CounterfactualExplanations.RawOutputArrayType","page":"🧐 Reference","title":"CounterfactualExplanations.RawOutputArrayType","text":"RawOutputArrayType\n\nA type union for the allowed type for the output array y.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.RawTargetType","page":"🧐 Reference","title":"CounterfactualExplanations.RawTargetType","text":"RawTargetType\n\nA type union for the allowed types for the target variable.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.flux_training_params","page":"🧐 Reference","title":"CounterfactualExplanations.flux_training_params","text":"flux_training_params\n\nThe default training parameter for FluxModels etc.\n\n\n\n\n\n","category":"constant"},{"location":"reference/#CounterfactualExplanations.AbstractConvergence","page":"🧐 Reference","title":"CounterfactualExplanations.AbstractConvergence","text":"An abstract type that serves as the base type for convergence objects.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.AbstractCounterfactualExplanation","page":"🧐 Reference","title":"CounterfactualExplanations.AbstractCounterfactualExplanation","text":"Base type for counterfactual explanations.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.AbstractFittedModel","page":"🧐 Reference","title":"CounterfactualExplanations.AbstractFittedModel","text":"Base type for fitted models.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.AbstractGenerator","page":"🧐 Reference","title":"CounterfactualExplanations.AbstractGenerator","text":"An abstract type that serves as the base type for counterfactual generators.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.CounterfactualExplanation","page":"🧐 Reference","title":"CounterfactualExplanations.CounterfactualExplanation","text":"A struct that collects all information relevant to a specific counterfactual explanation for a single individual.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.CounterfactualExplanation-Tuple{AbstractArray, Union{Int64, AbstractFloat, String, Symbol}, CounterfactualData, AbstractFittedModel, AbstractGenerator}","page":"🧐 Reference","title":"CounterfactualExplanations.CounterfactualExplanation","text":"function CounterfactualExplanation(;\n\tx::AbstractArray,\n\ttarget::RawTargetType,\n\tdata::CounterfactualData,\n\tM::Models.AbstractFittedModel,\n\tgenerator::Generators.AbstractGenerator,\n\tnum_counterfactuals::Int = 1,\n\tinitialization::Symbol = :add_perturbation,\n convergence::Union{AbstractConvergence,Symbol}=:decision_threshold,\n)\n\nOuter method to construct a CounterfactualExplanation structure.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.EncodedOutputArrayType","page":"🧐 Reference","title":"CounterfactualExplanations.EncodedOutputArrayType","text":"EncodedOutputArrayType\n\nType of encoded output array.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.EncodedTargetType","page":"🧐 Reference","title":"CounterfactualExplanations.EncodedTargetType","text":"EncodedTargetType\n\nType of encoded target variable.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.OutputEncoder","page":"🧐 Reference","title":"CounterfactualExplanations.OutputEncoder","text":"OutputEncoder\n\nThe OutputEncoder takes a raw output array (y) and encodes it.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.OutputEncoder-Tuple{Union{Int64, AbstractFloat, String, Symbol}}","page":"🧐 Reference","title":"CounterfactualExplanations.OutputEncoder","text":"(encoder::OutputEncoder)(ynew::RawTargetType)\n\nWhen called on a new value ynew, the OutputEncoder encodes it based on the initial encoding.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.OutputEncoder-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.OutputEncoder","text":"(encoder::OutputEncoder)()\n\nOn call, the OutputEncoder returns the encoded output array.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.EvoTreeModel","page":"🧐 Reference","title":"CounterfactualExplanations.EvoTreeModel","text":"EvoTreeModel\n\nExposes the EvoTreeModel from the EvoTreesExt extension.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.LaplaceReduxModel","page":"🧐 Reference","title":"CounterfactualExplanations.LaplaceReduxModel","text":"LaplaceReduxModel\n\nExposes the LaplaceReduxModel from the LaplaceReduxExt extension.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.NeuroTreeModel","page":"🧐 Reference","title":"CounterfactualExplanations.NeuroTreeModel","text":"NeuroTreeModel\n\nExposes the NeuroTreeModel from the NeuroTreeExt extension.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.generate_counterfactual-Tuple{Base.Iterators.Zip, Union{Int64, AbstractFloat, String, Symbol}, CounterfactualData, AbstractFittedModel, AbstractGenerator}","page":"🧐 Reference","title":"CounterfactualExplanations.generate_counterfactual","text":"generate_counterfactual(\n x::Base.Iterators.Zip,\n target::RawTargetType,\n data::CounterfactualData,\n M::Models.AbstractFittedModel,\n generator::AbstractGenerator;\n kwargs...,\n)\n\nOverloads the generate_counterfactual method to accept a zip of factuals x and return a vector of counterfactuals.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.generate_counterfactual-Tuple{Matrix, Union{Int64, AbstractFloat, String, Symbol}, CounterfactualData, AbstractFittedModel, AbstractGenerator}","page":"🧐 Reference","title":"CounterfactualExplanations.generate_counterfactual","text":"generate_counterfactual(\n x::Matrix,\n target::RawTargetType,\n data::CounterfactualData,\n M::Models.AbstractFittedModel,\n generator::AbstractGenerator;\n num_counterfactuals::Int=1,\n initialization::Symbol=:add_perturbation,\n convergence::Union{AbstractConvergence,Symbol}=:decision_threshold,\n timeout::Union{Nothing,Real}=nothing,\n)\n\nThe core function that is used to run counterfactual search for a given factual x, target, counterfactual data, model and generator. Keywords can be used to specify the desired threshold for the predicted target class probability and the maximum number of iterations.\n\nArguments\n\nx::Matrix: Factual data point.\ntarget::RawTargetType: Target class.\ndata::CounterfactualData: Counterfactual data.\nM::Models.AbstractFittedModel: Fitted model.\ngenerator::AbstractGenerator: Generator.\nnum_counterfactuals::Int=1: Number of counterfactuals to generate for factual.\ninitialization::Symbol=:add_perturbation: Initialization method. By default, the initialization is done by adding a small random perturbation to the factual to achieve more robustness.\nconvergence::Union{AbstractConvergence,Symbol}=:decision_threshold: Convergence criterion. By default, the convergence is based on the decision threshold. Possible values are :decision_threshold, :max_iter, :generator_conditions or a conrete convergence object (e.g. DecisionThresholdConvergence). \ntimeout::Union{Nothing,Int}=nothing: Timeout in seconds.\n\nExamples\n\nGeneric generator\n\njulia> using CounterfactualExplanations\n\njulia> using TaijaData\n \n # Counteractual data and model:\n\njulia> counterfactual_data = CounterfactualData(load_linearly_separable()...);\n\njulia> M = fit_model(counterfactual_data, :Linear);\n\njulia> target = 2;\n\njulia> factual = 1;\n\njulia> chosen = rand(findall(predict_label(M, counterfactual_data) .== factual));\n\njulia> x = select_factual(counterfactual_data, chosen);\n \n # Search:\n\njulia> generator = Generators.GenericGenerator();\n\njulia> ce = generate_counterfactual(x, target, counterfactual_data, M, generator)\nCounterfactualExplanation\nConvergence: ✅ after 7 steps.\n\nBroadcasting\n\nThe generate_counterfactual method can also be broadcasted over a tuple containing an array. This allows for generating multiple counterfactuals in parallel. \n\njulia> chosen = rand(findall(predict_label(M, counterfactual_data) .== factual), 5);\n\njulia> xs = select_factual(counterfactual_data, chosen);\n\njulia> ces = generate_counterfactual.(xs, target, counterfactual_data, M, generator)\n5-element Vector{CounterfactualExplanation}:\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 6 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.generate_counterfactual-Tuple{Matrix, Union{Int64, AbstractFloat, String, Symbol}, CounterfactualData, AbstractFittedModel, GrowingSpheresGenerator}","page":"🧐 Reference","title":"CounterfactualExplanations.generate_counterfactual","text":"generate_counterfactual(\n x::Matrix,\n target::RawTargetType,\n data::DataPreprocessing.CounterfactualData,\n M::Models.AbstractFittedModel,\n generator::Generators.GrowingSpheresGenerator;\n num_counterfactuals::Int=1,\n convergence::Union{AbstractConvergence,Symbol}=Convergence.DecisionThresholdConvergence(;\n decision_threshold=(1 / length(data.y_levels)), max_iter=1000\n ),\n kwrgs...,\n)\n\nOverloads the generate_counterfactual for the GrowingSpheresGenerator generator.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.generate_counterfactual-Tuple{Tuple{AbstractArray}, Vararg{Any}}","page":"🧐 Reference","title":"CounterfactualExplanations.generate_counterfactual","text":"generate_counterfactual(x::Tuple{<:AbstractArray}, args...; kwargs...)\n\nOverloads the generate_counterfactual method to accept a tuple containing and array. This allows for broadcasting over Zip iterators.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.generate_counterfactual-Tuple{Vector{<:Matrix}, Union{Int64, AbstractFloat, String, Symbol}, CounterfactualData, AbstractFittedModel, AbstractGenerator}","page":"🧐 Reference","title":"CounterfactualExplanations.generate_counterfactual","text":"generate_counterfactual(\n x::Vector{<:Matrix},\n target::RawTargetType,\n data::CounterfactualData,\n M::Models.AbstractFittedModel,\n generator::AbstractGenerator;\n kwargs...,\n)\n\nOverloads the generate_counterfactual method to accept a vector of factuals x and return a vector of counterfactuals.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.get_target_index-Tuple{Any, Any}","page":"🧐 Reference","title":"CounterfactualExplanations.get_target_index","text":"get_target_index(y_levels, target)\n\nUtility that returns the index of target in y_levels.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.path-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.path","text":"path(ce::CounterfactualExplanation)\n\nA convenience method that returns the entire counterfactual path.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.target_probs","page":"🧐 Reference","title":"CounterfactualExplanations.target_probs","text":"target_probs(\n ce::CounterfactualExplanation,\n x::Union{AbstractArray,Nothing}=nothing,\n)\n\nReturns the predicted probability of the target class for x. If x is nothing, the predicted probability corresponding to the counterfactual value is returned.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.terminated-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.terminated","text":"terminated(ce::CounterfactualExplanation)\n\nA convenience method that checks if the counterfactual search has terminated.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.total_steps-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.total_steps","text":"total_steps(ce::CounterfactualExplanation)\n\nA convenience method that returns the total number of steps of the counterfactual search.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.update!-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.update!","text":"update!(ce::CounterfactualExplanation)\n\nAn important subroutine that updates the counterfactual explanation. It takes a snapshot of the current counterfactual search state and passes it to the generator. Based on the current state the generator generates perturbations. Various constraints are then applied to the proposed vector of feature perturbations. Finally, the counterfactual search state is updated.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Convergence.convergence_catalogue","page":"🧐 Reference","title":"CounterfactualExplanations.Convergence.convergence_catalogue","text":"convergence_catalogue\n\nA dictionary containing all convergence criteria.\n\n\n\n\n\n","category":"constant"},{"location":"reference/#CounterfactualExplanations.Convergence.DecisionThresholdConvergence","page":"🧐 Reference","title":"CounterfactualExplanations.Convergence.DecisionThresholdConvergence","text":"DecisionThresholdConvergence\n\nConvergence criterion based on the target class probability threshold. The search stops when the target class probability exceeds the predefined threshold.\n\nFields\n\ndecision_threshold::AbstractFloat: The predefined threshold for the target class probability.\nmax_iter::Int: The maximum number of iterations.\nmin_success_rate::AbstractFloat: The minimum success rate for the target class probability.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Convergence.GeneratorConditionsConvergence","page":"🧐 Reference","title":"CounterfactualExplanations.Convergence.GeneratorConditionsConvergence","text":"GeneratorConditionsConvergence\n\nConvergence criterion for counterfactual explanations based on the generator conditions. The search stops when the gradients of the search objective are below a certain threshold and the generator conditions are satisfied.\n\nFields\n\ndecision_threshold::AbstractFloat: The threshold for the decision probability.\ngradient_tol::AbstractFloat: The tolerance for the gradients of the search objective.\nmax_iter::Int: The maximum number of iterations.\nmin_success_rate::AbstractFloat: The minimum success rate for the generator conditions (across counterfactuals).\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Convergence.GeneratorConditionsConvergence-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Convergence.GeneratorConditionsConvergence","text":"GeneratorConditionsConvergence(; decision_threshold=0.5, gradient_tol=1e-2, max_iter=100, min_success_rate=0.75, y_levels=nothing)\n\nOuter constructor for GeneratorConditionsConvergence.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Convergence.MaxIterConvergence","page":"🧐 Reference","title":"CounterfactualExplanations.Convergence.MaxIterConvergence","text":"MaxIterConvergence\n\nConvergence criterion based on the maximum number of iterations.\n\nFields\n\nmax_iter::Int: The maximum number of iterations.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Convergence.converged","page":"🧐 Reference","title":"CounterfactualExplanations.Convergence.converged","text":"converged(\n convergence::DecisionThresholdConvergence,\n ce::AbstractCounterfactualExplanation,\n x::Union{AbstractArray,Nothing}=nothing,\n)\n\nChecks if the counterfactual search has converged when the convergence criterion is the decision threshold.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.Convergence.converged-2","page":"🧐 Reference","title":"CounterfactualExplanations.Convergence.converged","text":"converged(\n convergence::MaxIterConvergence,\n ce::AbstractCounterfactualExplanation,\n x::Union{AbstractArray,Nothing}=nothing,\n)\n\nChecks if the counterfactual search has converged when the convergence criterion is maximum iterations. This means the counterfactual search will not terminate until the maximum number of iterations has been reached independently of the other convergence criteria.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.Convergence.converged-3","page":"🧐 Reference","title":"CounterfactualExplanations.Convergence.converged","text":"converged(\n convergence::GeneratorConditionsConvergence,\n ce::AbstractCounterfactualExplanation,\n x::Union{AbstractArray,Nothing}=nothing,\n)\n\nChecks if the counterfactual search has converged when the convergence criterion is generator_conditions.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.Convergence.converged-4","page":"🧐 Reference","title":"CounterfactualExplanations.Convergence.converged","text":"converged(\n convergence::InvalidationRateConvergence,\n ce::AbstractCounterfactualExplanation,\n x::Union{AbstractArray,Nothing}=nothing,\n)\n\nChecks if the counterfactual search has converged when the convergence criterion is invalidation rate.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.Convergence.get_convergence_type-Tuple{AbstractConvergence, AbstractVector}","page":"🧐 Reference","title":"CounterfactualExplanations.Convergence.get_convergence_type","text":"get_convergence_type(convergence::AbstractConvergence)\n\nReturns the convergence object.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Convergence.get_convergence_type-Tuple{Symbol, AbstractVector}","page":"🧐 Reference","title":"CounterfactualExplanations.Convergence.get_convergence_type","text":"get_convergence_type(convergence::Symbol)\n\nReturns the convergence object from the dictionary of default convergence types.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Convergence.hinge_loss-Tuple{CounterfactualExplanations.Convergence.InvalidationRateConvergence, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Convergence.hinge_loss","text":"hinge_loss(convergence::InvalidationRateConvergence, ce::AbstractCounterfactualExplanation)\n\nCalculates the hinge loss of a counterfactual explanation.\n\nArguments\n\nconvergence::InvalidationRateConvergence: The convergence criterion to use.\nce::AbstractCounterfactualExplanation: The counterfactual explanation to calculate the hinge loss for.\n\nReturns\n\nThe hinge loss of the counterfactual explanation.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Convergence.invalidation_rate-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Convergence.invalidation_rate","text":"invalidation_rate(ce::AbstractCounterfactualExplanation)\n\nCalculates the invalidation rate of a counterfactual explanation.\n\nArguments\n\nce::AbstractCounterfactualExplanation: The counterfactual explanation to calculate the invalidation rate for.\nkwargs: Additional keyword arguments to pass to the function.\n\nReturns\n\nThe invalidation rate of the counterfactual explanation.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Convergence.threshold_reached","page":"🧐 Reference","title":"CounterfactualExplanations.Convergence.threshold_reached","text":"threshold_reached(ce::AbstractCounterfactualExplanation, x::Union{AbstractArray,Nothing}=nothing)\n\nDetermines if the predefined threshold for the target class probability has been reached.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.Evaluation.default_measures","page":"🧐 Reference","title":"CounterfactualExplanations.Evaluation.default_measures","text":"The default evaluation measures.\n\n\n\n\n\n","category":"constant"},{"location":"reference/#CounterfactualExplanations.Evaluation.Benchmark","page":"🧐 Reference","title":"CounterfactualExplanations.Evaluation.Benchmark","text":"A container for benchmarks of counterfactual explanations. Instead of subtyping DataFrame, it contains a DataFrame of evaluation measures (see this discussion for why we don't subtype DataFrame directly).\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Evaluation.Benchmark-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Evaluation.Benchmark","text":"(bmk::Benchmark)(; agg=mean)\n\nReturns a DataFrame containing evaluation measures aggregated by num_counterfactual.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Evaluation.benchmark-Tuple{CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.Evaluation.benchmark","text":"benchmark(\n data::CounterfactualData;\n models::Dict{<:Any,<:Any}=standard_models_catalogue,\n generators::Union{Nothing,Dict{<:Any,<:AbstractGenerator}}=nothing,\n measure::Union{Function,Vector{Function}}=default_measures,\n n_individuals::Int=5,\n suppress_training::Bool=false,\n factual::Union{Nothing,RawTargetType}=nothing,\n target::Union{Nothing,RawTargetType}=nothing,\n store_ce::Bool=false,\n parallelizer::Union{Nothing,AbstractParallelizer}=nothing,\n kwrgs...,\n)\n\nRuns the benchmarking exercise as follows:\n\nRandomly choose a factual and target label unless specified. \nIf no pretrained models are provided, it is assumed that a dictionary of callable model objects is provided (by default using the standard_models_catalogue). \nEach of these models is then trained on the data. \nFor each model separately choose n_individuals randomly from the non-target (factual) class. For each generator create a benchmark as in benchmark(xs::Union{AbstractArray,Base.Iterators.Zip}).\nFinally, concatenate the results.\n\nIf vertical_splits is specified to an integer, the computations are split vertically into vertical_splits chunks. In this case, the results are stored in a temporary directory and concatenated afterwards. \n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Evaluation.benchmark-Tuple{Union{AbstractArray, Base.Iterators.Zip}, Union{Int64, AbstractFloat, String, Symbol}, CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.Evaluation.benchmark","text":"benchmark(\n x::Union{AbstractArray,Base.Iterators.Zip},\n target::RawTargetType,\n data::CounterfactualData;\n models::Dict{<:Any,<:AbstractFittedModel},\n generators::Dict{<:Any,<:AbstractGenerator},\n measure::Union{Function,Vector{Function}}=default_measures,\n xids::Union{Nothing,AbstractArray}=nothing,\n dataname::Union{Nothing,Symbol,String}=nothing,\n verbose::Bool=true,\n store_ce::Bool=false,\n parallelizer::Union{Nothing,AbstractParallelizer}=nothing,\n kwrgs...,\n)\n\nFirst generates counterfactual explanations for factual x, the target and data using each of the provided models and generators. Then generates a Benchmark for the vector of counterfactual explanations as in benchmark(counterfactual_explanations::Vector{CounterfactualExplanation}).\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Evaluation.benchmark-Tuple{Vector{CounterfactualExplanation}}","page":"🧐 Reference","title":"CounterfactualExplanations.Evaluation.benchmark","text":"benchmark(\n counterfactual_explanations::Vector{CounterfactualExplanation};\n meta_data::Union{Nothing,<:Vector{<:Dict}}=nothing,\n measure::Union{Function,Vector{Function}}=default_measures,\n store_ce::Bool=false,\n)\n\nGenerates a Benchmark for a vector of counterfactual explanations. Optionally meta_data describing each individual counterfactual explanation can be supplied. This should be a vector of dictionaries of the same length as the vector of counterfactuals. If no meta_data is supplied, it will be automatically inferred. All measure functions are applied to each counterfactual explanation. If store_ce=true, the counterfactual explanations are stored in the benchmark.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Evaluation.evaluate","page":"🧐 Reference","title":"CounterfactualExplanations.Evaluation.evaluate","text":"evaluate(\n ce::CounterfactualExplanation;\n measure::Union{Function,Vector{Function}}=default_measures,\n agg::Function=mean,\n report_each::Bool=false,\n output_format::Symbol=:Vector,\n pivot_longer::Bool=true\n)\n\nJust computes evaluation measures for the counterfactual explanation. By default, no meta data is reported. For report_meta=true, meta data is automatically inferred, unless this overwritten by meta_data. The optional meta_data argument should be a vector of dictionaries of the same length as the vector of counterfactual explanations. \n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.Evaluation.redundancy-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Evaluation.redundancy","text":"redundancy(ce::CounterfactualExplanation)\n\nComputes the feature redundancy: that is, the number of features that remain unchanged from their original, factual values.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Evaluation.validity-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Evaluation.validity","text":"validity(ce::CounterfactualExplanation; γ=0.5)\n\nChecks of the counterfactual search has been successful with respect to the probability threshold γ. In case multiple counterfactuals were generated, the function returns the proportion of successful counterfactuals.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.CounterfactualData-Tuple{AbstractMatrix, Union{AbstractMatrix, AbstractVector}}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.CounterfactualData","text":"CounterfactualData(\n X::AbstractMatrix,\n y::RawOutputArrayType;\n mutability::Union{Vector{Symbol},Nothing}=nothing,\n domain::Union{Any,Nothing}=nothing,\n features_categorical::Union{Vector{Vector{Int}},Nothing}=nothing,\n features_continuous::Union{Vector{Int},Nothing}=nothing,\n input_encoder::Union{Nothing,InputTransformer,TypedInputTransformer}=nothing,\n)\n\nThis outer constructor method prepares features X and labels y to be used with the package. Mutability and domain constraints can be added for the features. The function also accepts arguments that specify which features are categorical and which are continues. These arguments are currently not used. \n\nExamples\n\nusing CounterfactualExplanations.Data\nx, y = toy_data_linear()\nX = hcat(x...)\ncounterfactual_data = CounterfactualData(X,y')\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.CounterfactualData-Tuple{Tables.MatrixTable, Union{AbstractMatrix, AbstractVector}}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.CounterfactualData","text":"function CounterfactualData(\n X::Tables.MatrixTable,\n y::RawOutputArrayType;\n kwrgs...\n)\n\nOuter constructor method that accepts a Tables.MatrixTable. By default, the indices of categorical and continuous features are automatically inferred the features' scitype.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.apply_domain_constraints-Tuple{CounterfactualData, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.apply_domain_constraints","text":"apply_domain_constraints(counterfactual_data::CounterfactualData, x::AbstractArray)\n\nA subroutine that is used to apply the predetermined domain constraints.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.select_factual-Tuple{CounterfactualData, Int64}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.select_factual","text":"select_factual(counterfactual_data::CounterfactualData, index::Int)\n\nA convenience method that can be used to access the feature matrix.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.select_factual-Tuple{CounterfactualData, Union{UnitRange{Int64}, Vector{Int64}}}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.select_factual","text":"select_factual(counterfactual_data::CounterfactualData, index::Union{Vector{Int},UnitRange{Int}})\n\nA convenience method that can be used to access the feature matrix.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.transformable_features-Tuple{CounterfactualData, Any}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.transformable_features","text":"transformable_features(counterfactual_data::CounterfactualData, input_encoder::Any)\n\nBy default, all continuous features are transformable. This function returns the indices of all continuous features.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.transformable_features-Tuple{CounterfactualData, Type{StatsBase.ZScoreTransform}}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.transformable_features","text":"transformable_features(\n counterfactual_data::CounterfactualData, input_encoder::Type{ZScoreTransform}\n)\n\nReturns the indices of all continuous features that can be transformed. For constant features ZScoreTransform returns NaN.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.transformable_features-Tuple{CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.transformable_features","text":"transformable_features(counterfactual_data::CounterfactualData)\n\nDispatches the transformable_features function to the appropriate method based on the type of the dt field.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.all_models_catalogue","page":"🧐 Reference","title":"CounterfactualExplanations.Models.all_models_catalogue","text":"all_models_catalogue\n\nA dictionary containing both differentiable and non-differentiable machine learning models.\n\n\n\n\n\n","category":"constant"},{"location":"reference/#CounterfactualExplanations.Models.mlj_models_catalogue","page":"🧐 Reference","title":"CounterfactualExplanations.Models.mlj_models_catalogue","text":"mlj_models_catalogue\n\nA dictionary containing all machine learning models from the MLJ model registry that the package supports.\n\n\n\n\n\n","category":"constant"},{"location":"reference/#CounterfactualExplanations.Models.standard_models_catalogue","page":"🧐 Reference","title":"CounterfactualExplanations.Models.standard_models_catalogue","text":"standard_models_catalogue\n\nA dictionary containing all differentiable machine learning models.\n\n\n\n\n\n","category":"constant"},{"location":"reference/#CounterfactualExplanations.Models.AbstractDifferentiableModel","page":"🧐 Reference","title":"CounterfactualExplanations.Models.AbstractDifferentiableModel","text":"Base type for differentiable models.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Models.FluxEnsemble","page":"🧐 Reference","title":"CounterfactualExplanations.Models.FluxEnsemble","text":"FluxEnsemble <: AbstractFluxModel\n\nConstructor for deep ensembles trained in Flux.jl. \n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Models.FluxModel","page":"🧐 Reference","title":"CounterfactualExplanations.Models.FluxModel","text":"FluxModel <: AbstractFluxModel\n\nConstructor for models trained in Flux.jl. \n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Models.FluxModel-Tuple{CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.FluxModel","text":"FluxModel(data::CounterfactualData; kwargs...)\n\nConstructs a multi-layer perceptron (MLP).\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.DecisionTreeModel-Tuple{CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.DecisionTreeModel","text":"DecisionTreeModel(data::CounterfactualData; kwargs...)\n\nConstructs a new TreeModel object wrapped around a decision tree from the data in a CounterfactualData object. Not called by the user directly.\n\nArguments\n\ndata::CounterfactualData: The CounterfactualData object containing the data to be used for training the model.\n\nReturns\n\nmodel::TreeModel: A TreeModel object.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.Linear-Tuple{CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.Linear","text":"Linear(data::CounterfactualData; kwargs...)\n\nConstructs a model with one linear layer. If the output is binary, this corresponds to logistic regression, since model outputs are passed through the sigmoid function. If the output is multi-class, this corresponds to multinomial logistic regression, since model outputs are passed through the softmax function.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.RandomForestModel-Tuple{CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.RandomForestModel","text":"RandomForestModel(data::CounterfactualData; kwargs...)\n\nConstructs a new TreeModel object wrapped around a random forest from the data in a CounterfactualData object. Not called by the user directly.\n\nArguments\n\ndata::CounterfactualData: The CounterfactualData object containing the data to be used for training the model.\n\nReturns\n\nmodel::TreeModel: A TreeModel object.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.fit_model","page":"🧐 Reference","title":"CounterfactualExplanations.Models.fit_model","text":"fit_model(\n counterfactual_data::CounterfactualData, model::Symbol=:MLP;\n kwrgs...\n)\n\nFits one of the available default models to the counterfactual_data. The model argument can be used to specify the desired model. The available values correspond to the keys of the all_models_catalogue dictionary.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.Models.logits-Tuple{AbstractFittedModel, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.logits","text":"logits(M::AbstractFittedModel, X::AbstractArray)\n\nGeneric method that is compulsory for all models. It returns the raw model predictions. In classification this is sometimes referred to as logits: the non-normalized predictions that are fed into a link function to produce predicted probabilities. In regression (not currently implemented) raw outputs typically correspond to final outputs. In other words, there is typically no normalization involved.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.logits-Tuple{CounterfactualExplanations.Models.TreeModel, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.logits","text":"logits(M::TreeModel, X::AbstractArray)\n\nCalculates the logit scores output by the model M for the input data X.\n\nArguments\n\nM::TreeModel: The model selected by the user.\nX::AbstractArray: The feature vector for which the logit scores are calculated.\n\nReturns\n\nlogits::Matrix: A matrix of logits for each output class for each data point in X.\n\nExample\n\nlogits = Models.logits(M, x) # calculates the logit scores for each output class for the data point x\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.model_evaluation-Tuple{AbstractFittedModel, CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.model_evaluation","text":"model_evaluation(M::AbstractFittedModel, test_data::CounterfactualData)\n\nHelper function to compute F-Score for AbstractFittedModel on a (test) data set. By default, it computes the accuracy. Any other measure, e.g. from the StatisticalMeasures package, can be passed as an argument. Currently, only measures applicable to classification tasks are supported.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.predict_label-Tuple{AbstractFittedModel, CounterfactualData, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.predict_label","text":"predict_label(M::AbstractFittedModel, counterfactual_data::CounterfactualData, X::AbstractArray)\n\nReturns the predicted output label for a given model M, data set counterfactual_data and input data X.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.predict_label-Tuple{AbstractFittedModel, CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.predict_label","text":"predict_label(M::AbstractFittedModel, counterfactual_data::CounterfactualData)\n\nReturns the predicted output labels for all data points of data set counterfactual_data for a given model M.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.predict_label-Tuple{CounterfactualExplanations.Models.TreeModel, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.predict_label","text":"predict_label(M::TreeModel, X::AbstractArray)\n\nReturns the predicted label for X.\n\nArguments\n\nM::TreeModel: The model selected by the user.\nX::AbstractArray: The input array for which the label is predicted.\n\nReturns\n\nlabels::AbstractArray: The predicted label for each data point in X.\n\nExample\n\nlabel = Models.predict_label(M, x) # returns the predicted label for each data point in x\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.predict_proba-Tuple{AbstractFittedModel, Union{Nothing, CounterfactualData}, Union{Nothing, AbstractArray}}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.predict_proba","text":"predict_proba(M::AbstractFittedModel, counterfactual_data::CounterfactualData, X::Union{Nothing,AbstractArray})\n\nReturns the predicted output probabilities for a given model M, data set counterfactual_data and input data X.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.probs-Tuple{AbstractFittedModel, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.probs","text":"probs(M::AbstractFittedModel, X::AbstractArray)\n\nGeneric method that is compulsory for all models. It returns the normalized model predictions, so the predicted probabilities in the case of classification. In regression (not currently implemented) this method is redundant. \n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.probs-Tuple{CounterfactualExplanations.Models.TreeModel, AbstractMatrix{<:Number}}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.probs","text":"probs(M::TreeModel, X::AbstractArray{<:Number, 2})\n\nCalculates the probability scores for each output class for the two-dimensional input data matrix X.\n\nArguments\n\nM::TreeModel: The TreeModel.\nX::AbstractArray: The feature vector for which the predictions are made.\n\nReturns\n\np::Matrix: A matrix of probability scores for each output class for each data point in X.\n\nExample\n\nprobabilities = Models.probs(M, X) # calculates the probability scores for each output class for each data point in X.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.probs-Tuple{CounterfactualExplanations.Models.TreeModel, AbstractVector{<:Number}}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.probs","text":"probs(M::TreeModel, X::AbstractArray{<:Number, 1})\n\nWorks the same way as the probs(M::TreeModel, X::AbstractArray{<:Number, 2}) method above, but handles 1-dimensional rather than 2-dimensional input data.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.generator_catalogue","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.generator_catalogue","text":"A dictionary containing the constructors of all available counterfactual generators.\n\n\n\n\n\n","category":"constant"},{"location":"reference/#CounterfactualExplanations.Generators.AbstractGradientBasedGenerator","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.AbstractGradientBasedGenerator","text":"AbstractGradientBasedGenerator\n\nAn abstract type that serves as the base type for gradient-based counterfactual generators. \n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Generators.AbstractNonGradientBasedGenerator","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.AbstractNonGradientBasedGenerator","text":"AbstractNonGradientBasedGenerator\n\nAn abstract type that serves as the base type for non gradient-based counterfactual generators. \n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Generators.FeatureTweakGenerator","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.FeatureTweakGenerator","text":"Feature Tweak counterfactual generator class.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Generators.FeatureTweakGenerator-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.FeatureTweakGenerator","text":"FeatureTweakGenerator(; penalty::Union{Nothing,Function,Vector{Function}}=Objectives.distance_l2, ϵ::AbstractFloat=0.1)\n\nConstructs a new Feature Tweak Generator object.\n\nUses the L2-norm as the penalty to measure the distance between the counterfactual and the factual. According to the paper by Tolomei et al., another recommended choice for the penalty in addition to the L2-norm is the L0-norm. The L0-norm simply minimizes the number of features that are changed through the tweak.\n\nArguments\n\npenalty::Union{Nothing,Function,Vector{Function}}: The penalty function to use for the generator. Defaults to distance_l2.\nϵ::AbstractFloat: The tolerance value for the feature tweaks. Described at length in Tolomei et al. (https://arxiv.org/pdf/1706.06691.pdf). Defaults to 0.1.\n\nReturns\n\ngenerator::FeatureTweakGenerator: A non-gradient-based generator that can be used to generate counterfactuals using the feature tweak method.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.GradientBasedGenerator","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.GradientBasedGenerator","text":"Base class for gradient-based counterfactual generators.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Generators.GradientBasedGenerator-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.GradientBasedGenerator","text":"GradientBasedGenerator(;\n\tloss::Union{Nothing,Function}=nothing,\n\tpenalty::Penalty=nothing,\n\tλ::Union{Nothing,AbstractFloat,Vector{AbstractFloat}}=nothing,\n\tlatent_space::Bool::false,\n\topt::Flux.Optimise.AbstractOptimiser=Flux.Descent(),\n generative_model_params::NamedTuple=(;),\n)\n\nDefault outer constructor for GradientBasedGenerator.\n\nArguments\n\nloss::Union{Nothing,Function}=nothing: The loss function used by the model.\npenalty::Penalty=nothing: A penalty function for the generator to penalize counterfactuals too far from the original point.\nλ::Union{Nothing,AbstractFloat,Vector{AbstractFloat}}=nothing: The weight of the penalty function.\nlatent_space::Bool=false: Whether to use the latent space of a generative model to generate counterfactuals.\nopt::Flux.Optimise.AbstractOptimiser=Flux.Descent(): The optimizer to use for the generator.\ngenerative_model_params::NamedTuple: The parameters of the generative model associated with the generator.\n\nReturns\n\ngenerator::GradientBasedGenerator: A gradient-based counterfactual generator.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.GrowingSpheresGenerator","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.GrowingSpheresGenerator","text":"Growing Spheres counterfactual generator class.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Generators.GrowingSpheresGenerator-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.GrowingSpheresGenerator","text":"GrowingSpheresGenerator(; n::Int=100, η::Float64=0.1, kwargs...)\n\nConstructs a new Growing Spheres Generator object.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.JSMADescent","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.JSMADescent","text":"An optimisation rule that can be used to implement a Jacobian-based Saliency Map Attack.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Generators.JSMADescent-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.JSMADescent","text":"Outer constructor for the JSMADescent rule.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.CLUEGenerator-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.CLUEGenerator","text":"Constructor for CLUEGenerator.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.ClaPROARGenerator-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.ClaPROARGenerator","text":"Constructor for ClaPGenerator.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.DiCEGenerator-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.DiCEGenerator","text":"Constructor for DiCEGenerator.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.GenericGenerator-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.GenericGenerator","text":"Constructor for GenericGenerator.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.GravitationalGenerator-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.GravitationalGenerator","text":"Constructor for GravitationalGenerator.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.GreedyGenerator-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.GreedyGenerator","text":"Constructor for GreedyGenerator.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.ProbeGenerator-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.ProbeGenerator","text":"Constructor for ProbeGenerator.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.REVISEGenerator-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.REVISEGenerator","text":"Constructor for REVISEGenerator.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.WachterGenerator-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.WachterGenerator","text":"Constructor for WachterGenerator.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.conditions_satisfied-Tuple{AbstractGradientBasedGenerator, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.conditions_satisfied","text":"conditions_satisfied(generator::AbstractGradientBasedGenerator, ce::AbstractCounterfactualExplanation)\n\nThe default method to check if the all conditions for convergence of the counterfactual search have been satisified for gradient-based generators. By default, gradient-based search is considered to have converged as soon as the proposed feature changes for all features are smaller than one percent of its standard deviation.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.feature_tweaking!-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.feature_tweaking!","text":"feature_tweaking!(ce::AbstractCounterfactualExplanation)\n\nReturns a counterfactual instance of ce.x based on the ensemble of classifiers provided.\n\nArguments\n\nce::AbstractCounterfactualExplanation: The counterfactual explanation object.\n\nReturns\n\nce::AbstractCounterfactualExplanation: The counterfactual explanation object.\n\nExample\n\nce = feature_tweaking!(ce) # returns a counterfactual inside the ce.s′ field based on the ensemble of classifiers provided\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.generate_perturbations-Tuple{AbstractGradientBasedGenerator, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.generate_perturbations","text":"generate_perturbations(generator::AbstractGradientBasedGenerator, ce::AbstractCounterfactualExplanation)\n\nThe default method to generate feature perturbations for gradient-based generators through simple gradient descent.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.generate_perturbations-Tuple{FeatureTweakGenerator, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.generate_perturbations","text":"generate_perturbations(generator::AbstractGradientBasedGenerator, ce::AbstractCounterfactualExplanation)\n\nThe default method to generate feature perturbations for gradient-based generators through simple gradient descent.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.hinge_loss-Tuple{AbstractConvergence, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.hinge_loss","text":"hinge_loss(convergence::AbstractConvergence, ce::AbstractCounterfactualExplanation)\n\nThe default hinge loss for any convergence criterion. Can be overridden inside the Convergence module as part of the definition of specific convergence criteria.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.@objective-Tuple{Any, Any}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.@objective","text":"objective(generator, ex)\n\nA macro that can be used to define the counterfactual search objective.\n\n\n\n\n\n","category":"macro"},{"location":"reference/#CounterfactualExplanations.Generators.@search_feature_space-Tuple{Any}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.@search_feature_space","text":"search_feature_space(generator)\n\nA simple macro that can be used to specify feature space search.\n\n\n\n\n\n","category":"macro"},{"location":"reference/#CounterfactualExplanations.Generators.@search_latent_space-Tuple{Any}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.@search_latent_space","text":"search_latent_space(generator)\n\nA simple macro that can be used to specify latent space search.\n\n\n\n\n\n","category":"macro"},{"location":"reference/#CounterfactualExplanations.Generators.@with_optimiser-Tuple{Any, Any}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.@with_optimiser","text":"with_optimiser(generator, optimiser)\n\nA simple macro that can be used to specify the optimiser to be used.\n\n\n\n\n\n","category":"macro"},{"location":"reference/#CounterfactualExplanations.Objectives.ddp_diversity-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Objectives.ddp_diversity","text":"ddp_diversity(\n ce::AbstractCounterfactualExplanation;\n perturbation_size=1e-5\n)\n\nEvaluates how diverse the counterfactuals are using a Determinantal Point Process (DDP).\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Objectives.distance-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Objectives.distance","text":"distance(ce::AbstractCounterfactualExplanation, p::Real=2)\n\nComputes the distance of the counterfactual to the original factual.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Objectives.distance_l0-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Objectives.distance_l0","text":"distance_l0(ce::AbstractCounterfactualExplanation)\n\nComputes the L0 distance of the counterfactual to the original factual.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Objectives.distance_l1-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Objectives.distance_l1","text":"distance_l1(ce::AbstractCounterfactualExplanation)\n\nComputes the L1 distance of the counterfactual to the original factual.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Objectives.distance_l2-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Objectives.distance_l2","text":"distance_l2(ce::AbstractCounterfactualExplanation)\n\nComputes the L2 (Euclidean) distance of the counterfactual to the original factual.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Objectives.distance_linf-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Objectives.distance_linf","text":"distance_linf(ce::AbstractCounterfactualExplanation)\n\nComputes the L-inf distance of the counterfactual to the original factual.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Objectives.distance_mad-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Objectives.distance_mad","text":"distance_mad(ce::AbstractCounterfactualExplanation; agg=mean)\n\nThis is the distance measure proposed by Wachter et al. (2017).\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Objectives.predictive_entropy-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Objectives.predictive_entropy","text":"predictive_entropy(ce::AbstractCounterfactualExplanation; agg=Statistics.mean)\n\nComputes the predictive entropy of the counterfactuals. Explained in https://arxiv.org/abs/1406.2541.\n\n\n\n\n\n","category":"method"},{"location":"reference/#Flux.Losses.logitbinarycrossentropy-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"Flux.Losses.logitbinarycrossentropy","text":"Flux.Losses.logitbinarycrossentropy(ce::AbstractCounterfactualExplanation)\n\nSimply extends the logitbinarycrossentropy method to work with objects of type AbstractCounterfactualExplanation.\n\n\n\n\n\n","category":"method"},{"location":"reference/#Flux.Losses.logitcrossentropy-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"Flux.Losses.logitcrossentropy","text":"Flux.Losses.logitcrossentropy(ce::AbstractCounterfactualExplanation)\n\nSimply extends the logitcrossentropy method to work with objects of type AbstractCounterfactualExplanation.\n\n\n\n\n\n","category":"method"},{"location":"reference/#Flux.Losses.mse-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"Flux.Losses.mse","text":"Flux.Losses.mse(ce::AbstractCounterfactualExplanation)\n\nSimply extends the mse method to work with objects of type AbstractCounterfactualExplanation.\n\n\n\n\n\n","category":"method"},{"location":"reference/#Internal-functions","page":"🧐 Reference","title":"Internal functions","text":"","category":"section"},{"location":"reference/","page":"🧐 Reference","title":"🧐 Reference","text":"Modules = [\n CounterfactualExplanations, \n CounterfactualExplanations.Convergence,\n CounterfactualExplanations.Evaluation,\n CounterfactualExplanations.DataPreprocessing,\n CounterfactualExplanations.Models, \n CounterfactualExplanations.GenerativeModels,\n CounterfactualExplanations.Generators, \n CounterfactualExplanations.Objectives\n]\nPublic = false","category":"page"},{"location":"reference/#CounterfactualExplanations.FluxModelParams","page":"🧐 Reference","title":"CounterfactualExplanations.FluxModelParams","text":"FluxModelParams\n\nDefault MLP training parameters.\n\n\n\n\n\n","category":"type"},{"location":"reference/#Base.Broadcast.broadcastable-Tuple{AbstractFittedModel}","page":"🧐 Reference","title":"Base.Broadcast.broadcastable","text":"Treat AbstractFittedModel as scalar when broadcasting.\n\n\n\n\n\n","category":"method"},{"location":"reference/#Base.Broadcast.broadcastable-Tuple{AbstractGenerator}","page":"🧐 Reference","title":"Base.Broadcast.broadcastable","text":"Treat AbstractGenerator as scalar when broadcasting.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.adjust_shape!-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.adjust_shape!","text":"adjust_shape!(ce::CounterfactualExplanation)\n\nA convenience method that adjusts the dimensions of the counterfactual state and related fields.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.adjust_shape-Tuple{CounterfactualExplanation, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.adjust_shape","text":"adjust_shape(\n ce::CounterfactualExplanation, \n x::AbstractArray\n)\n\nA convenience method that adjusts the dimensions of x.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.already_in_target_class-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.already_in_target_class","text":"already_in_target_class(ce::CounterfactualExplanation)\n\nCheck if the factual is already in the target class.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.apply_domain_constraints!-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.apply_domain_constraints!","text":"apply_domain_constraints!(ce::CounterfactualExplanation)\n\nWrapper function that applies underlying domain constraints.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.apply_mutability-Tuple{CounterfactualExplanation, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.apply_mutability","text":"apply_mutability(\n ce::CounterfactualExplanation,\n Δs′::AbstractArray,\n)\n\nA subroutine that applies mutability constraints to the proposed vector of feature perturbations.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.counterfactual-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.counterfactual","text":"counterfactual(ce::CounterfactualExplanation)\n\nA convenience method that returns the counterfactual.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.counterfactual_label-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.counterfactual_label","text":"counterfactual_label(ce::CounterfactualExplanation)\n\nA convenience method that returns the predicted label of the counterfactual.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.counterfactual_label_path-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.counterfactual_label_path","text":"counterfactual_label_path(ce::CounterfactualExplanation)\n\nReturns the counterfactual labels for each step of the search.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.counterfactual_probability","page":"🧐 Reference","title":"CounterfactualExplanations.counterfactual_probability","text":"counterfactual_probability(ce::CounterfactualExplanation)\n\nA convenience method that computes the class probabilities of the counterfactual.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.counterfactual_probability_path-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.counterfactual_probability_path","text":"counterfactual_probability_path(ce::CounterfactualExplanation)\n\nReturns the counterfactual probabilities for each step of the search.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.decode_array-Tuple{CounterfactualData, CounterfactualExplanations.GenerativeModels.AbstractGenerativeModel, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.decode_array","text":"decode_array(dt::GenerativeModels.AbstractGenerativeModel, x::AbstractArray)\n\nHelper function to decode an array x using a data transform dt::GenerativeModels.AbstractGenerativeModel.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.decode_array-Tuple{CounterfactualData, MultivariateStats.AbstractDimensionalityReduction, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.decode_array","text":"decode_array(dt::MultivariateStats.AbstractDimensionalityReduction, x::AbstractArray)\n\nHelper function to decode an array x using a data transform dt::MultivariateStats.AbstractDimensionalityReduction.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.decode_array-Tuple{CounterfactualData, Nothing, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.decode_array","text":"decode_array(dt::Nothing, x::AbstractArray)\n\nHelper function to decode an array x using a data transform dt::Nothing. This is a no-op.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.decode_array-Tuple{CounterfactualData, StatsBase.AbstractDataTransform, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.decode_array","text":"decode_array(dt::StatsBase.AbstractDataTransform, x::AbstractArray)\n\nHelper function to decode an array x using a data transform dt::StatsBase.AbstractDataTransform.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.decode_state","page":"🧐 Reference","title":"CounterfactualExplanations.decode_state","text":"function decode_state( ce::CounterfactualExplanation, x::Union{AbstractArray,Nothing}=nothing, )\n\nApplies all the applicable decoding functions:\n\nIf applicable, map the state variable back from the latent space to the feature space.\nIf and where applicable, inverse-transform features.\nReconstruct all categorical encodings.\n\nFinally, the decoded counterfactual is returned.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.decode_state!","page":"🧐 Reference","title":"CounterfactualExplanations.decode_state!","text":"decode_state!(ce::CounterfactualExplanation, x::Union{AbstractArray,Nothing}=nothing)\n\nIn-place version of decode_state.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.encode_array-Tuple{CounterfactualData, CounterfactualExplanations.GenerativeModels.AbstractGenerativeModel, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.encode_array","text":"encode_array(dt::GenerativeModels.AbstractGenerativeModel, x::AbstractArray)\n\nHelper function to encode an array x using a data transform dt::GenerativeModels.AbstractGenerativeModel.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.encode_array-Tuple{CounterfactualData, MultivariateStats.AbstractDimensionalityReduction, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.encode_array","text":"encode_array(dt::MultivariateStats.AbstractDimensionalityReduction, x::AbstractArray)\n\nHelper function to encode an array x using a data transform dt::MultivariateStats.AbstractDimensionalityReduction.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.encode_array-Tuple{CounterfactualData, Nothing, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.encode_array","text":"encode_array(dt::Nothing, x::AbstractArray)\n\nHelper function to encode an array x using a data transform dt::Nothing. This is a no-op.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.encode_array-Tuple{CounterfactualData, StatsBase.AbstractDataTransform, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.encode_array","text":"encode_array(dt::StatsBase.AbstractDataTransform, x::AbstractArray)\n\nHelper function to encode an array x using a data transform dt::StatsBase.AbstractDataTransform.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.encode_state","page":"🧐 Reference","title":"CounterfactualExplanations.encode_state","text":"function encode_state( ce::CounterfactualExplanation, x::Union{AbstractArray,Nothing} = nothing, )\n\nApplies all required encodings to x:\n\nIf applicable, it maps x to the latent space learned by the generative model.\nIf and where applicable, it rescales features. \n\nFinally, it returns the encoded state variable.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.encode_state!","page":"🧐 Reference","title":"CounterfactualExplanations.encode_state!","text":"encode_state!(ce::CounterfactualExplanation, x::Union{AbstractArray,Nothing}=nothing)\n\nIn-place version of encode_state.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.factual-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.factual","text":"factual(ce::CounterfactualExplanation)\n\nA convenience method to retrieve the factual x.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.factual_label-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.factual_label","text":"factual_label(ce::CounterfactualExplanation)\n\nA convenience method to get the predicted label associated with the factual.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.factual_probability-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.factual_probability","text":"factual_probability(ce::CounterfactualExplanation)\n\nA convenience method to compute the class probabilities of the factual.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.find_potential_neighbours-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.find_potential_neighbours","text":"find_potential_neighbors(ce::AbstractCounterfactualExplanation)\n\nFinds potential neighbors for the selected factual data point.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.get_meta-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.get_meta","text":"get_meta(ce::CounterfactualExplanation)\n\nReturns meta data for a counterfactual explanation.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.guess_likelihood-Tuple{Union{AbstractMatrix, AbstractVector}}","page":"🧐 Reference","title":"CounterfactualExplanations.guess_likelihood","text":"guess_likelihood(y::RawOutputArrayType)\n\nGuess the likelihood based on the scientific type of the output array. Returns a symbol indicating the guessed likelihood and the scientific type of the output array.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.guess_loss-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.guess_loss","text":"guess_loss(ce::CounterfactualExplanation)\n\nGuesses the loss function to be used for the counterfactual search in case likelihood field is specified for the AbstractFittedModel instance and no loss function was explicitly declared for AbstractGenerator instance.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.initialize_state!-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.initialize_state!","text":"initialize_state!(ce::CounterfactualExplanation)\n\nInitializes the starting point for the factual(s) in-place.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.initialize_state-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.initialize_state","text":"initialize_state(ce::CounterfactualExplanation)\n\nInitializes the starting point for the factual(s):\n\nIf ce.initialization is set to :identity or counterfactuals are searched in a latent space, then nothing is done.\nIf ce.initialization is set to :add_perturbation, then a random perturbation is added to the factual following following Slack (2021): https://arxiv.org/abs/2106.02666. The authors show that this improves adversarial robustness.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.output_dim-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.output_dim","text":"output_dim(ce::CounterfactualExplanation)\n\nA convenience method that returns the output dimension of the predictive model.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.reset!-Tuple{CounterfactualExplanations.FluxModelParams}","page":"🧐 Reference","title":"CounterfactualExplanations.reset!","text":"reset!(flux_training_params::FluxModelParams)\n\nRestores the default parameter values.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.steps_exhausted-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.steps_exhausted","text":"steps_exhausted(ce::CounterfactualExplanation)\n\nA convenience method that checks if the number of maximum iterations has been exhausted.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.target_probs_path-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.target_probs_path","text":"target_probs_path(ce::CounterfactualExplanation)\n\nReturns the target probabilities for each step of the search.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Evaluation.distance_measures","page":"🧐 Reference","title":"CounterfactualExplanations.Evaluation.distance_measures","text":"All distance measures.\n\n\n\n\n\n","category":"constant"},{"location":"reference/#Base.vcat-Tuple{CounterfactualExplanations.Evaluation.Benchmark, CounterfactualExplanations.Evaluation.Benchmark}","page":"🧐 Reference","title":"Base.vcat","text":"Base.vcat(bmk1::Benchmark, bmk2::Benchmark)\n\nVertically concatenates two Benchmark objects.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Evaluation.compute_measure-Tuple{CounterfactualExplanation, Function, Function}","page":"🧐 Reference","title":"CounterfactualExplanations.Evaluation.compute_measure","text":"compute_measure(ce::CounterfactualExplanation, measure::Function, agg::Function)\n\nComputes a single measure for a counterfactual explanation. The measure is applied to the counterfactual explanation ce and aggregated using the aggregation function agg.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Evaluation.to_dataframe-Tuple{Vector, Any, Bool, Bool, Bool, CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Evaluation.to_dataframe","text":"evaluate_dataframe(\n ce::CounterfactualExplanation,\n measure::Vector{Function},\n agg::Function,\n report_each::Bool,\n pivot_longer::Bool,\n store_ce::Bool,\n)\n\nEvaluates a counterfactual explanation and returns a dataframe of evaluation measures.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Evaluation.validity_strict-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Evaluation.validity_strict","text":"validity_strict(ce::CounterfactualExplanation)\n\nChecks if the counterfactual search has been strictly valid in the sense that it has converged with respect to the pre-specified target probability γ.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.InputTransformer","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.InputTransformer","text":"InputTransformer\n\nAbstract type for data transformers. This can be any of the following:\n\nStatsBase.AbstractDataTransform: A data transformation object from the StatsBase package.\nMultivariateStats.AbstractDimensionalityReduction: A dimensionality reduction object from the MultivariateStats package.\nGenerativeModels.AbstractGenerativeModel: A generative model object from the GenerativeModels module.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.TypedInputTransformer","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.TypedInputTransformer","text":"TypedInputTransformer\n\nAbstract type for data transformers.\n\n\n\n\n\n","category":"type"},{"location":"reference/#Base.Broadcast.broadcastable-Tuple{CounterfactualData}","page":"🧐 Reference","title":"Base.Broadcast.broadcastable","text":"Treat CounterfactualData as scalar when broadcasting.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing._subset-Tuple{CounterfactualData, Vector{Int64}}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing._subset","text":"_subset(data::CounterfactualData, idx::Vector{Int})\n\nCreates a subset of the data.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.convert_to_1d-Tuple{Matrix, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.convert_to_1d","text":"convert_to_1d(y::Matrix, y_levels::AbstractArray)\n\nHelper function to convert a one-hot encoded matrix to a vector of labels. This is necessary because MLJ models require the labels to be represented as a vector, but the synthetic datasets in this package hold the labels in one-hot encoded form.\n\nArguments\n\ny::Matrix: The one-hot encoded matrix.\ny_levels::AbstractArray: The levels of the categorical variable.\n\nReturns\n\nlabels: A vector of labels.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.fit_transformer-Tuple{CounterfactualData, Nothing}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.fit_transformer","text":"fit_transformer(data::CounterfactualData, input_encoder::Nothing; kwargs...)\n\nFit a transformer to the data. This is a no-op if input_encoder is Nothing.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.fit_transformer-Tuple{CounterfactualData, Type{<:CounterfactualExplanations.GenerativeModels.AbstractGenerativeModel}}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.fit_transformer","text":"fit_transformer(\n data::CounterfactualData,\n input_encoder::Type{GenerativeModels.AbstractGenerativeModel};\n kwargs...,\n)\n\nFit a transformer to the data for a GenerativeModels.AbstractGenerativeModel object.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.fit_transformer-Tuple{CounterfactualData, Type{<:MultivariateStats.AbstractDimensionalityReduction}}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.fit_transformer","text":"fit_transformer(\n data::CounterfactualData,\n input_encoder::Type{MultivariateStats.AbstractDimensionalityReduction};\n kwargs...,\n)\n\nFit a transformer to the data for a MultivariateStats.AbstractDimensionalityReduction object.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.fit_transformer-Tuple{CounterfactualData, Type{<:StatsBase.AbstractDataTransform}}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.fit_transformer","text":"fit_transformer(\n data::CounterfactualData,\n input_encoder::Type{StatsBase.AbstractDataTransform};\n kwargs...,\n)\n\nFit a transformer to the data for a StatsBase.AbstractDataTransform object.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.input_dim-Tuple{CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.input_dim","text":"input_dim(counterfactual_data::CounterfactualData)\n\nHelper function that returns the input dimension (number of features) of the data. \n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.mutability_constraints-Tuple{CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.mutability_constraints","text":"mutability_constraints(counterfactual_data::CounterfactualData)\n\nA convenience function that returns the mutability constraints. If none were specified, it is assumed that all features are mutable in :both directions.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.preprocess_data_for_mlj-Tuple{CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.preprocess_data_for_mlj","text":"preprocess_data_for_mlj(data::CounterfactualData)\n\nHelper function to preprocess data::CounterfactualData for MLJ models.\n\nArguments\n\ndata::CounterfactualData: The data to be preprocessed.\n\nReturns\n\n(df_x, y): A tuple containing the preprocessed data, with df_x being a DataFrame object and y being a categorical vector.\n\nExample\n\nX, y = preprocessdatafor_mlj(data)\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.reconstruct_cat_encoding-Tuple{CounterfactualData, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.reconstruct_cat_encoding","text":"reconstruct_cat_encoding(counterfactual_data::CounterfactualData, x::Vector)\n\nReconstruct the categorical encoding for a single instance.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.subsample-Tuple{CounterfactualData, Int64}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.subsample","text":"subsample(data::CounterfactualData, n::Int)\n\nHelper function to randomly subsample data::CounterfactualData.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.train_test_split-Tuple{CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.train_test_split","text":"train_test_split(data::CounterfactualData;test_size=0.2,keep_class_ratio=false)\n\nSplits data into train and test split.\n\nArguments\n\ndata::CounterfactualData: The data to be preprocessed.\ntest_size=0.2: Proportion of the data to be used for testing. \nkeep_class_ratio=false: Decides whether to sample equally from each class, or keep their relative size.\n\nReturns\n\n(train_data::CounterfactualData, test_data::CounterfactualData): A tuple containing the train and test splits.\n\nExample\n\ntrain, test = traintestsplit(data, testsize=0.1, keepclass_ratio=true)\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.unpack_data-Tuple{CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.unpack_data","text":"unpack_data(data::CounterfactualData)\n\nHelper function that unpacks data.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.AbstractCustomDifferentiableModel","page":"🧐 Reference","title":"CounterfactualExplanations.Models.AbstractCustomDifferentiableModel","text":"Base type for custom differentiable models.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Models.AbstractFluxModel","page":"🧐 Reference","title":"CounterfactualExplanations.Models.AbstractFluxModel","text":"Base type for differentiable models written in Flux.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Models.AbstractMLJModel","page":"🧐 Reference","title":"CounterfactualExplanations.Models.AbstractMLJModel","text":"Base type for differentiable models from the MLJ library.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Models.AbstractNonDifferentiableJuliaModel","page":"🧐 Reference","title":"CounterfactualExplanations.Models.AbstractNonDifferentiableJuliaModel","text":"Base type for non-differentiable models written in pure Julia.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Models.AbstractNonDifferentiableModel","page":"🧐 Reference","title":"CounterfactualExplanations.Models.AbstractNonDifferentiableModel","text":"Base type for non-differentiable models.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Models.FluxEnsembleParams","page":"🧐 Reference","title":"CounterfactualExplanations.Models.FluxEnsembleParams","text":"FluxModelParams\n\nDefault Deep Ensemble training parameters.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Models.TreeModel","page":"🧐 Reference","title":"CounterfactualExplanations.Models.TreeModel","text":"TreeModel <: AbstractNonDifferentiableJuliaModel\n\nConstructor for tree-based models from the MLJ library. \n\nArguments\n\nmodel::Any: The model selected by the user. Must be a model from the MLJ library.\nlikelihood::Symbol: The likelihood of the model. Must be one of [:classification_binary, :classification_multi].\n\nReturns\n\nTreeModel: A tree-based model from the MLJ library wrapped inside the TreeModel class.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Models.TreeModel-Tuple{Any}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.TreeModel","text":"Outer constructor method for TreeModel.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.binary_to_onehot-Tuple{Any}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.binary_to_onehot","text":"binary_to_onehot(p)\n\nHelper function to turn dummy-encoded variable into onehot-encoded variable.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.build_ensemble-Tuple{Int64}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.build_ensemble","text":"build_ensemble(K::Int;kw=(input_dim=2,n_hidden=32,output_dim=1))\n\nHelper function that builds an ensemble of K models.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.build_mlp-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.build_mlp","text":"build_mlp()\n\nHelper function to build simple MLP.\n\nExamples\n\nnn = build_mlp()\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.data_loader-Tuple{CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.data_loader","text":"data_loader(data::CounterfactualData)\n\nPrepares counterfactual data for training in Flux.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.get_individual_classifiers-Tuple{CounterfactualExplanations.Models.TreeModel}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.get_individual_classifiers","text":"get_individual_classifiers(M::TreeModel)\n\nReturns the individual classifiers in the forest. If the input is a decision tree, the method returns the decision tree itself inside an array.\n\nArguments\n\nM::TreeModel: The model selected by the user.\n\nReturns\n\nclassifiers::AbstractArray: An array of individual classifiers in the forest.\n\nExample\n\nclassifiers = Models.getindividualclassifiers(M) # returns the individual classifiers in the forest\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.train-Tuple{CounterfactualExplanations.Models.TreeModel, CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.train","text":"train(M::TreeModel, data::CounterfactualData; kwargs...)\n\nFits the model M to the data in the CounterfactualData object. This method is not called by the user directly.\n\nArguments\n\nM::TreeModel: The wrapper for a TreeModel.\ndata::CounterfactualData: The CounterfactualData object containing the data to be used for training the model.\n\nReturns\n\nM::TreeModel: The fitted TreeModel.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.train-Tuple{FluxEnsemble, CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.train","text":"train(M::FluxEnsemble, data::CounterfactualData; kwargs...)\n\nWrapper function to retrain.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.train-Tuple{FluxModel, CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.train","text":"train(M::FluxModel, data::CounterfactualData; kwargs...)\n\nWrapper function to retrain FluxModel.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.GenerativeModels.AbstractGMParams","page":"🧐 Reference","title":"CounterfactualExplanations.GenerativeModels.AbstractGMParams","text":"Base type of generative model hyperparameter container.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.GenerativeModels.AbstractGenerativeModel","page":"🧐 Reference","title":"CounterfactualExplanations.GenerativeModels.AbstractGenerativeModel","text":"Base type for generative model.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.GenerativeModels.Encoder","page":"🧐 Reference","title":"CounterfactualExplanations.GenerativeModels.Encoder","text":"Encoder\n\nConstructs encoder part of VAE: a simple Flux neural network with one hidden layer and two linear output layers for the first two moments of the latent distribution.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.GenerativeModels.VAE","page":"🧐 Reference","title":"CounterfactualExplanations.GenerativeModels.VAE","text":"VAE <: AbstractGenerativeModel\n\nConstructs the Variational Autoencoder. The VAE is a subtype of AbstractGenerativeModel. Any (sub-)type of AbstractGenerativeModel is accepted by latent space generators. \n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.GenerativeModels.VAE-Tuple{Any}","page":"🧐 Reference","title":"CounterfactualExplanations.GenerativeModels.VAE","text":"VAE(input_dim;kws...)\n\nOuter method for instantiating a VAE.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.GenerativeModels.VAEParams","page":"🧐 Reference","title":"CounterfactualExplanations.GenerativeModels.VAEParams","text":"VAEParams <: AbstractGMParams\n\nThe default VAE parameters describing both the encoder/decoder architecture and the training process.\n\n\n\n\n\n","category":"type"},{"location":"reference/#Base.rand","page":"🧐 Reference","title":"Base.rand","text":"Random.rand(encoder::Encoder, x, device=cpu)\n\nDraws random samples from the latent distribution.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.GenerativeModels.Decoder-Tuple{Int64, Int64, Int64}","page":"🧐 Reference","title":"CounterfactualExplanations.GenerativeModels.Decoder","text":"Decoder(input_dim::Int, latent_dim::Int, hidden_dim::Int; activation=relu)\n\nThe default decoder architecture is just a Flux Chain with one hidden layer and a linear output layer. \n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.GenerativeModels.decode-Tuple{CounterfactualExplanations.GenerativeModels.VAE, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.GenerativeModels.decode","text":"decode(generative_model::VAE, x::AbstractArray)\n\nDecodes an array x using the VAE decoder.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.GenerativeModels.encode-Tuple{CounterfactualExplanations.GenerativeModels.VAE, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.GenerativeModels.encode","text":"encode(generative_model::VAE, x::AbstractArray)\n\nEncodes an array x using the VAE encoder. Specifically, it samples from the latent distribution. It does so by first passing x through the encoder to obtain the mean and log-variance of the latent distribution. Then, it samples from the latent distribution using the reparameterization trick. See Random.rand(encoder::Encoder, x, device=cpu) for more details.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.GenerativeModels.get_data-Tuple{AbstractArray, Any}","page":"🧐 Reference","title":"CounterfactualExplanations.GenerativeModels.get_data","text":"get_data(X::AbstractArray, batch_size)\n\nPreparing data for mini-batch training .\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.GenerativeModels.reconstruct","page":"🧐 Reference","title":"CounterfactualExplanations.GenerativeModels.reconstruct","text":"reconstruct(generative_model::VAE, x, device=cpu)\n\nImplements a full pass of some input x through the VAE: x ↦ x̂.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.GenerativeModels.reparameterization_trick","page":"🧐 Reference","title":"CounterfactualExplanations.GenerativeModels.reparameterization_trick","text":"reparameterization_trick(μ,logσ,device=cpu)\n\nHelper function that implements the reparameterization trick: z ∼ 𝒩(μ,σ²) ⇔ z=μ + σ ⊙ ε, ε ∼ 𝒩(0,I).\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.Generators.Penalty","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.Penalty","text":"Type union for acceptable argument types for the penalty field of GradientBasedGenerator.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Generators._replace_nans","page":"🧐 Reference","title":"CounterfactualExplanations.Generators._replace_nans","text":"_replace_nans(Δs′::AbstractArray, old_new::Pair=(NaN => 0))\n\nHelper function to deal with exploding gradients. This is only a temporary fix and will be improved.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.Generators.calculate_delta-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.calculate_delta","text":"calculate_delta(ce::AbstractCounterfactualExplanation, penalty::Vector{Function})\n\nCalculates the penalty for the proposed feature tweak.\n\nArguments\n\nce::AbstractCounterfactualExplanation: The counterfactual explanation object.\n\nReturns\n\ndelta::Float64: The calculated penalty for the proposed feature tweak.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.esatisfactory_instance-Tuple{FeatureTweakGenerator, AbstractArray, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.esatisfactory_instance","text":"esatisfactory_instance(generator::FeatureTweakGenerator, x::AbstractArray, paths::Dict{String, Dict{String, Any}})\n\nReturns an epsilon-satisfactory counterfactual for x based on the paths provided.\n\nArguments\n\ngenerator::FeatureTweakGenerator: The feature tweak generator.\nx::AbstractArray: The factual instance.\npaths::Dict{String, Dict{String, Any}}: A list of paths to the leaves of the tree to be used for tweaking the feature.\n\nReturns\n\nesatisfactory::AbstractArray: The epsilon-satisfactory instance.\n\nExample\n\nesatisfactory = esatisfactory_instance(generator, x, paths) # returns an epsilon-satisfactory counterfactual for x based on the paths provided\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.feature_selection!-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.feature_selection!","text":"feature_selection!(ce::AbstractCounterfactualExplanation)\n\nPerform feature selection to find the dimension with the closest (but not equal) values between the ce.x (factual) and ce.s′ (counterfactual) arrays.\n\nArguments\n\nce::AbstractCounterfactualExplanation: An instance of the AbstractCounterfactualExplanation type representing the counterfactual explanation.\n\nReturns\n\nnothing\n\nThe function iteratively modifies the ce.s′ counterfactual array by updating its elements to match the corresponding elements in the ce.x factual array, one dimension at a time, until the predicted label of the modified ce.s′ matches the predicted label of the ce.x array.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.find_closest_dimension-Tuple{Any, Any}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.find_closest_dimension","text":"find_closest_dimension(factual, counterfactual)\n\nFind the dimension with the closest (but not equal) values between the factual and counterfactual arrays.\n\nArguments\n\nfactual: The factual array.\ncounterfactual: The counterfactual array.\n\nReturns\n\nclosest_dimension: The index of the dimension with the closest values.\n\nThe function iterates over the indices of the factual array and calculates the absolute difference between the corresponding elements in the factual and counterfactual arrays. It returns the index of the dimension with the smallest difference, excluding dimensions where the values in factual and counterfactual are equal.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.find_counterfactual-NTuple{4, Any}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.find_counterfactual","text":"find_counterfactual(model, factual_class, counterfactual_data, counterfactual_candidates)\n\nFind the first counterfactual index by predicting labels.\n\nArguments\n\nmodel: The fitted model used for prediction.\ntarget_class: Expected target class.\ncounterfactual_data: Data required for counterfactual generation.\ncounterfactual_candidates: The array of counterfactual candidates.\n\nReturns\n\ncounterfactual: The index of the first counterfactual found.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.growing_spheres_generation!-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.growing_spheres_generation!","text":"growing_spheres_generation(ce::AbstractCounterfactualExplanation)\n\nGenerate counterfactual candidates using the growing spheres generation algorithm.\n\nArguments\n\nce::AbstractCounterfactualExplanation: An instance of the AbstractCounterfactualExplanation type representing the counterfactual explanation.\n\nReturns\n\nnothing\n\nThis function applies the growing spheres generation algorithm to generate counterfactual candidates. It starts by generating random points uniformly on a sphere, gradually reducing the search space until no counterfactuals are found. Then it expands the search space until at least one counterfactual is found or the maximum number of iterations is reached.\n\nThe algorithm iteratively generates counterfactual candidates and predicts their labels using the model stored in ce.M. It checks if any of the predicted labels are different from the factual class. The process of reducing the search space involves halving the search radius, while the process of expanding the search space involves increasing the search radius.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.h-Tuple{AbstractGenerator, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.h","text":"h(generator::AbstractGenerator, ce::AbstractCounterfactualExplanation)\n\nDispatches to the appropriate complexity function for any generator.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.h-Tuple{AbstractGenerator, Function, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.h","text":"h(generator::AbstractGenerator, penalty::Function, ce::AbstractCounterfactualExplanation)\n\nOverloads the h function for the case where a single penalty function is provided.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.h-Tuple{AbstractGenerator, Nothing, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.h","text":"h(generator::AbstractGenerator, penalty::Nothing, ce::AbstractCounterfactualExplanation)\n\nOverloads the h function for the case where no penalty is provided.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.h-Tuple{AbstractGenerator, Vector{<:Tuple}, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.h","text":"h(generator::AbstractGenerator, penalty::Tuple, ce::AbstractCounterfactualExplanation)\n\nOverloads the h function for the case where a single penalty function is provided with additional keyword arguments.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.h-Tuple{AbstractGenerator, Vector{Function}, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.h","text":"h(generator::AbstractGenerator, penalty::Tuple, ce::AbstractCounterfactualExplanation)\n\nOverloads the h function for the case where a single penalty function is provided with additional keyword arguments.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.hyper_sphere_coordinates-Tuple{Integer, AbstractArray, AbstractFloat, AbstractFloat}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.hyper_sphere_coordinates","text":"hyper_sphere_coordinates(n_search_samples::Int, instance::Vector{Float64}, low::Int, high::Int; p_norm::Int=2)\n\nGenerates candidate counterfactuals using the growing spheres method based on hyper-sphere coordinates.\n\nThe implementation follows the Random Point Picking over a sphere algorithm described in the paper: \"Learning Counterfactual Explanations for Tabular Data\" by Pawelczyk, Broelemann & Kascneci (2020), presented at The Web Conference 2020 (WWW). It ensures that points are sampled uniformly at random using insights from: http://mathworld.wolfram.com/HyperspherePointPicking.html\n\nThe growing spheres method is originally proposed in the paper: \"Comparison-based Inverse Classification for Interpretability in Machine Learning\" by Thibaut Laugel et al (2018), presented at the International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems (2018).\n\nArguments\n\nn_search_samples::Int: The number of search samples (int > 0).\ninstance::AbstractArray: The input point array.\nlow::AbstractFloat: The lower bound (float >= 0, l < h).\nhigh::AbstractFloat: The upper bound (float >= 0, h > l).\np_norm::Integer: The norm parameter (int >= 1).\n\nReturns\n\ncandidate_counterfactuals::Array: An array of candidate counterfactuals.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.propose_state-Tuple{AbstractGradientBasedGenerator, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.propose_state","text":"propose_state(generator::AbstractGradientBasedGenerator, ce::AbstractCounterfactualExplanation)\n\nProposes new state based on backpropagation.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.search_path","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.search_path","text":"search_path(tree::Union{DecisionTree.Leaf, DecisionTree.Node}, target::RawTargetType, path::AbstractArray)\n\nReturn a path index list with the inequality symbols, thresholds and feature indices.\n\nArguments\n\ntree::Union{DecisionTree.Leaf, DecisionTree.Node}: The root node of a decision tree.\ntarget::RawTargetType: The target class.\npath::AbstractArray: A list containing the paths found thus far.\n\nReturns\n\npaths::AbstractArray: A list of paths to the leaves of the tree to be used for tweaking the feature.\n\nExample\n\npaths = search_path(tree, target) # returns a list of paths to the leaves of the tree to be used for tweaking the feature\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.Generators.ℓ-Tuple{AbstractGenerator, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.ℓ","text":"ℓ(generator::AbstractGenerator, ce::AbstractCounterfactualExplanation)\n\nDispatches to the appropriate loss function for any generator.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.ℓ-Tuple{AbstractGenerator, Function, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.ℓ","text":"ℓ(generator::AbstractGenerator, loss::Function, ce::AbstractCounterfactualExplanation)\n\nOverloads the ℓ function for the case where a single loss function is provided.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.ℓ-Tuple{AbstractGenerator, Nothing, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.ℓ","text":"ℓ(generator::AbstractGenerator, loss::Nothing, ce::AbstractCounterfactualExplanation)\n\nOverloads the ℓ function for the case where no loss function is provided.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.∂h-Tuple{AbstractGradientBasedGenerator, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.∂h","text":"∂h(generator::AbstractGradientBasedGenerator, ce::AbstractCounterfactualExplanation)\n\nThe default method to compute the gradient of the complexity penalty at the current counterfactual state for gradient-based generators. It assumes that Zygote.jl has gradient access. \n\nIf the penalty is not provided, it returns 0.0. By default, Zygote never works out the gradient for constants and instead returns 'nothing', so we need to add a manual step to override this behaviour. See here: https://discourse.julialang.org/t/zygote-gradient/26715.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.∂ℓ-Tuple{AbstractGradientBasedGenerator, AbstractDifferentiableModel, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.∂ℓ","text":"∂ℓ(generator::AbstractGradientBasedGenerator, M::Union{Models.LogisticModel, Models.BayesianLogisticModel}, ce::AbstractCounterfactualExplanation)\n\nThe default method to compute the gradient of the loss function at the current counterfactual state for gradient-based generators. It assumes that Zygote.jl has gradient access.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.∇-Tuple{AbstractGradientBasedGenerator, AbstractDifferentiableModel, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.∇","text":"∇(generator::AbstractGradientBasedGenerator, M::Models.AbstractDifferentiableModel, ce::AbstractCounterfactualExplanation)\n\nThe default method to compute the gradient of the counterfactual search objective for gradient-based generators. It simply computes the weighted sum over partial derivates. It assumes that Zygote.jl has gradient access. If the counterfactual is being generated using Probe, the hinge loss is added to the gradient.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Objectives.distance_from_target-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Objectives.distance_from_target","text":"distance_from_target(\n ce::AbstractCounterfactualExplanation;\n K::Int=50\n)\n\nComputes the distance of the counterfactual from a point in the target main.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Objectives.model_loss_penalty-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Objectives.model_loss_penalty","text":"function model_loss_penalty(\n ce::AbstractCounterfactualExplanation;\n agg=mean\n)\n\nAdditional penalty for ClaPROARGenerator.\n\n\n\n\n\n","category":"method"},{"location":"extensions/","page":"Overview","title":"Overview","text":"CurrentModule = CounterfactualExplanations","category":"page"},{"location":"extensions/#Extensions","page":"Overview","title":"⛓️ Extensions","text":"","category":"section"},{"location":"extensions/","page":"Overview","title":"Overview","text":"In this section, you will find information about package extensions of the CounterfactualExplanations package. Extensions are a relatively new feature of Julia that allows users to conditionally load code based on the presence of other packages. This is useful for creating packages that extend the functionality of other packages, without requiring the user to install the package being extended.","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"tutorials/evaluation/#Performance-Evaluation","page":"Evaluating Explanations","title":"Performance Evaluation","text":"","category":"section"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"Now that we know how to generate counterfactual explanations in Julia, you may have a few follow-up questions: How do I know if the counterfactual search has been successful? How good is my counterfactual explanation? What does ‘good’ even mean in this context? In this tutorial, we will see how counterfactual explanations can be evaluated with respect to their performance.","category":"page"},{"location":"tutorials/evaluation/#Default-Measures","page":"Evaluating Explanations","title":"Default Measures","text":"","category":"section"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"Numerous evaluation measures for counterfactual explanations have been proposed. In what follows, we will cover some of the most important measures.","category":"page"},{"location":"tutorials/evaluation/#Single-Measure,-Single-Counterfactual","page":"Evaluating Explanations","title":"Single Measure, Single Counterfactual","text":"","category":"section"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"One of the most important measures is validity, which simply determines whether or not a counterfactual explanation x^prime is valid in the sense that it yields the target prediction: M(x^prime)=t. We can evaluate the validity of a single counterfactual explanation ce using the Evaluation.evaluate function as follows:","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"using CounterfactualExplanations.Evaluation: evaluate, validity\nevaluate(ce; measure=validity)","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"1-element Vector{Vector{Float64}}:\n [1.0]","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"For a single counterfactual explanation, this evaluation measure can only take two values: it is either equal to 1, if the explanation is valid or 0 otherwise. Another important measure is distance, which relates to the distance between the factual x and the counterfactual x^prime. In the context of Algorithmic Recourse, higher distances are typically associated with higher costs to individuals seeking recourse.","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"using CounterfactualExplanations.Objectives: distance\nevaluate(ce; measure=distance)","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"1-element Vector{Vector{Float32}}:\n [3.2273161]","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"By default, distance computes the L2 (Euclidean) distance.","category":"page"},{"location":"tutorials/evaluation/#Multiple-Measures,-Single-Counterfactual","page":"Evaluating Explanations","title":"Multiple Measures, Single Counterfactual","text":"","category":"section"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"You might be interested in computing not just the L2 distance, but various LP norms. This can be done by supplying a vector of functions to the measure key argument. For convenience, all default distance measures have already been collected in a vector:","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"using CounterfactualExplanations.Evaluation: distance_measures\ndistance_measures","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"4-element Vector{Function}:\n distance_l0 (generic function with 1 method)\n distance_l1 (generic function with 1 method)\n distance_l2 (generic function with 1 method)\n distance_linf (generic function with 1 method)","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"We can use this vector of evaluation measures as follows:","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"evaluate(ce; measure=distance_measures)","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"4-element Vector{Vector{Float32}}:\n [2.0]\n [3.2273161]\n [2.7737978]\n [2.7285953]","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"If no measure is specified, the evaluate method will return all default measures,","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"evaluate(ce)","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"3-element Vector{Vector}:\n [1.0]\n Float32[3.2273161]\n [0.0]","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"which include:","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"CounterfactualExplanations.Evaluation.default_measures","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"3-element Vector{Function}:\n validity (generic function with 1 method)\n distance (generic function with 1 method)\n redundancy (generic function with 1 method)","category":"page"},{"location":"tutorials/evaluation/#Multiple-Measures-and-Counterfactuals","page":"Evaluating Explanations","title":"Multiple Measures and Counterfactuals","text":"","category":"section"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"We can also evaluate multiple counterfactual explanations at once:","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"generator = DiCEGenerator()\nces = generate_counterfactual(x, target, counterfactual_data, M, generator; num_counterfactuals=5)\nevaluate(ces)","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"3-element Vector{Vector}:\n [1.0]\n Float32[3.1955845]\n [[0.0, 0.0, 0.0, 0.0, 0.0]]","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"By default, each evaluation measure is aggregated across all counterfactual explanations. To return individual measures for each counterfactual explanation you can specify report_each=true","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"evaluate(ces; report_each=true)","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"3-element Vector{Vector}:\n BitVector[[1, 1, 1, 1, 1]]\n Vector{Float32}[[3.3671722, 3.1028512, 3.2829392, 3.0728922, 3.1520686]]\n [[0.0, 0.0, 0.0, 0.0, 0.0]]","category":"page"},{"location":"tutorials/evaluation/#Custom-Measures","page":"Evaluating Explanations","title":"Custom Measures","text":"","category":"section"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"A measure is just a method that takes a CounterfactualExplanation as its only positional argument and agg::Function as a key argument specifying how measures should be aggregated across counterfactuals. Defining custom measures is therefore straightforward. For example, we could define a measure to compute the inverse target probability as follows:","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"my_measure(ce::CounterfactualExplanation; agg=mean) = agg(1 .- CounterfactualExplanations.target_probs(ce))\nevaluate(ce; measure=my_measure)","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"1-element Vector{Vector{Float32}}:\n [0.41711217]","category":"page"},{"location":"tutorials/evaluation/#Tidy-Output","page":"Evaluating Explanations","title":"Tidy Output","text":"","category":"section"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"By default, evaluate returns vectors of evaluation measures. The optional key argument output_format::Symbol can be used to post-process the output in two ways: firstly, to return the output as a dictionary, specify output_format=:Dict:","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"evaluate(ces; output_format=:Dict, report_each=true)","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"Dict{Symbol, Vector} with 3 entries:\n :validity => BitVector[[1, 1, 1, 1, 1]]\n :redundancy => [[0.0, 0.0, 0.0, 0.0, 0.0]]\n :distance => Vector{Float32}[[3.36717, 3.10285, 3.28294, 3.07289, 3.15207]]","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"Secondly, to return the output as a data frame, specify output_format=:DataFrame.","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"evaluate(ces; output_format=:DataFrame, report_each=true)","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"By default, data frames are pivoted to long format using individual counterfactuals as the id column. This behaviour can be suppressed by specifying pivot_longer=false.","category":"page"},{"location":"tutorials/evaluation/#Multiple-Counterfactual-Explanations","page":"Evaluating Explanations","title":"Multiple Counterfactual Explanations","text":"","category":"section"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"It may be necessary to generate counterfactual explanations for multiple individuals.","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"Below, for example, we first select multiple samples (5) from the non-target class and then generate counterfactual explanations for all of them.","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"This can be done using broadcasting:","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"# Factual and target:\nids = rand(findall(predict_label(M, counterfactual_data) .== factual), n_individuals)\nxs = select_factual(counterfactual_data, ids)\nces = generate_counterfactual(xs, target, counterfactual_data, M, generator; num_counterfactuals=5)\nevaluation = evaluate.(ces)","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"5-element Vector{Vector{Vector}}:\n [[1.0], Float32[3.351181], [[0.0, 0.0, 0.0, 0.0, 0.0]]]\n [[1.0], Float32[2.6405892], [[0.0, 0.0, 0.0, 0.0, 0.0]]]\n [[1.0], Float32[2.935012], [[0.0, 0.0, 0.0, 0.0, 0.0]]]\n [[1.0], Float32[3.5348382], [[0.0, 0.0, 0.0, 0.0, 0.0]]]\n [[1.0], Float32[3.9373996], [[0.0, 0.0, 0.0, 0.0, 0.0]]]\n\nVector{Vector}[[[1.0], Float32[3.351181], [[0.0, 0.0, 0.0, 0.0, 0.0]]], [[1.0], Float32[2.6405892], [[0.0, 0.0, 0.0, 0.0, 0.0]]], [[1.0], Float32[2.935012], [[0.0, 0.0, 0.0, 0.0, 0.0]]], [[1.0], Float32[3.5348382], [[0.0, 0.0, 0.0, 0.0, 0.0]]], [[1.0], Float32[3.9373996], [[0.0, 0.0, 0.0, 0.0, 0.0]]]]","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"This leads us to our next topic: Performance Benchmarks.","category":"page"},{"location":"explanation/generators/generic/","page":"Generic","title":"Generic","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"explanation/generators/generic/#GenericGenerator","page":"Generic","title":"GenericGenerator","text":"","category":"section"},{"location":"explanation/generators/generic/","page":"Generic","title":"Generic","text":"We use the term generic to relate to the basic counterfactual generator proposed by Wachter, Mittelstadt, and Russell (2017) with L1-norm regularization. There is also a variant of this generator that uses the distance metric proposed in Wachter, Mittelstadt, and Russell (2017), which we call WachterGenerator.","category":"page"},{"location":"explanation/generators/generic/#Description","page":"Generic","title":"Description","text":"","category":"section"},{"location":"explanation/generators/generic/","page":"Generic","title":"Generic","text":"As the term indicates, this approach is simple: it forms the baseline approach for gradient-based counterfactual generators. Wachter, Mittelstadt, and Russell (2017) were among the first to realise that","category":"page"},{"location":"explanation/generators/generic/","page":"Generic","title":"Generic","text":"[…] explanations can, in principle, be offered without opening the “black box.”— Wachter, Mittelstadt, and Russell (2017)","category":"page"},{"location":"explanation/generators/generic/","page":"Generic","title":"Generic","text":"Gradient descent is performed directly in the feature space. Concerning the cost heuristic, the authors choose to penalize the distance of counterfactuals from the factual value. This is based on the intuitive notion that larger feature perturbations require greater effort.","category":"page"},{"location":"explanation/generators/generic/#Usage","page":"Generic","title":"Usage","text":"","category":"section"},{"location":"explanation/generators/generic/","page":"Generic","title":"Generic","text":"The approach can be used in our package as follows:","category":"page"},{"location":"explanation/generators/generic/","page":"Generic","title":"Generic","text":"generator = GenericGenerator()\nce = generate_counterfactual(x, target, counterfactual_data, M, generator)\nplot(ce)","category":"page"},{"location":"explanation/generators/generic/","page":"Generic","title":"Generic","text":"(Image: )","category":"page"},{"location":"explanation/generators/generic/#References","page":"Generic","title":"References","text":"","category":"section"},{"location":"explanation/generators/generic/","page":"Generic","title":"Generic","text":"Wachter, Sandra, Brent Mittelstadt, and Chris Russell. 2017. “Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR.” Harv. JL & Tech. 31: 841. https://doi.org/10.2139/ssrn.3063289.","category":"page"},{"location":"explanation/generators/greedy/","page":"Greedy","title":"Greedy","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"explanation/generators/greedy/#GreedyGenerator","page":"Greedy","title":"GreedyGenerator","text":"","category":"section"},{"location":"explanation/generators/greedy/","page":"Greedy","title":"Greedy","text":"We use the term greedy to describe the counterfactual generator introduced by Schut et al. (2021).","category":"page"},{"location":"explanation/generators/greedy/#Description","page":"Greedy","title":"Description","text":"","category":"section"},{"location":"explanation/generators/greedy/","page":"Greedy","title":"Greedy","text":"The Greedy generator works under the premise of generating realistic counterfactuals by minimizing predictive uncertainty. Schut et al. (2021) show that for models that incorporates predictive uncertainty in their predictions, maximizing the predictive probability corresponds to minimizing the predictive uncertainty: by construction, the generated counterfactual will therefore be realistic (low epistemic uncertainty) and unambiguous (low aleatoric uncertainty).","category":"page"},{"location":"explanation/generators/greedy/","page":"Greedy","title":"Greedy","text":"For the counterfactual search Schut et al. (2021) propose using a Jacobian-based Saliency Map Attack(JSMA). It is greedy in the sense that it is an “iterative algorithm that updates the most salient feature, i.e. the feature that has the largest influence on the classification, by delta at each step” (Schut et al. 2021).","category":"page"},{"location":"explanation/generators/greedy/#Usage","page":"Greedy","title":"Usage","text":"","category":"section"},{"location":"explanation/generators/greedy/","page":"Greedy","title":"Greedy","text":"The approach can be used in our package as follows:","category":"page"},{"location":"explanation/generators/greedy/","page":"Greedy","title":"Greedy","text":"M = fit_model(counterfactual_data, :DeepEnsemble)\ngenerator = GreedyGenerator()\nce = generate_counterfactual(x, target, counterfactual_data, M, generator)\nplot(ce)","category":"page"},{"location":"explanation/generators/greedy/","page":"Greedy","title":"Greedy","text":"(Image: )","category":"page"},{"location":"explanation/generators/greedy/#References","page":"Greedy","title":"References","text":"","category":"section"},{"location":"explanation/generators/greedy/","page":"Greedy","title":"Greedy","text":"Schut, Lisa, Oscar Key, Rory Mc Grath, Luca Costabello, Bogdan Sacaleanu, Yarin Gal, et al. 2021. “Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertainties.” In International Conference on Artificial Intelligence and Statistics, 1756–64. PMLR.","category":"page"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"explanation/generators/probe/#ProbeGenerator","page":"PROBE","title":"ProbeGenerator","text":"","category":"section"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"The ProbeGenerator is designed to navigate the trade-offs between costs and robustness in Algorithmic Recourse (Pawelczyk et al. 2022).","category":"page"},{"location":"explanation/generators/probe/#Description","page":"PROBE","title":"Description","text":"","category":"section"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"The goal of ProbeGenerator is to find a recourse x’ whose prediction at any point y within some set around x’ belongs to the positive class with probability 1 - r, where r is the recourse invalidation rate. It minimizes the gap between the achieved and desired recourse invalidation rates, minimizes recourse costs, and also ensures that the resulting recourse achieves a positive model prediction.","category":"page"},{"location":"explanation/generators/probe/#Explanation","page":"PROBE","title":"Explanation","text":"","category":"section"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"The loss function this generator is defined below. R is a hinge loss parameter which helps control for robustness. The loss and penalty functions can still be chosen freely.","category":"page"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"beginaligned\nR(x sigma^2 I) + l(f(x) s) + lambda d_c(x x)\nendaligned","category":"page"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"R uses the following formula to control for noise. It generates small perturbations and checks how often the counterfactual explanation flips back to a factual one, when small amounts of noise are added to it.","category":"page"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"beginaligned\nDelta(x^hatE) = E_varepsilonh(x^hatE) - h(x^hatE + varepsilon)\nendaligned","category":"page"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"The above formula is not differentiable. For this reason the generator uses the closed form version of the formula below.","category":"page"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"beginequation\nDelta tilde(x^hatE sigma^2 I) = 1 - Phi left(fracsqrtf(x^hatE)sqrtnabla f(x^hatE)^T sigma^2 I nabla f(x^hatE)right) \nendequation","category":"page"},{"location":"explanation/generators/probe/#Usage","page":"PROBE","title":"Usage","text":"","category":"section"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"Generating a counterfactual with the data loaded and generator chosen works as follows:","category":"page"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"Note: It is important to set the convergence to “:invalidation_rate” here.","category":"page"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"M = fit_model(counterfactual_data, :DeepEnsemble)\nopt = Descent(0.01)\ngenerator = CounterfactualExplanations.Generators.ProbeGenerator(opt=opt)\nconv = CounterfactualExplanations.Convergence.InvalidationRateConvergence(;invalidation_rate=0.5)\nce = generate_counterfactual(x, target, counterfactual_data, M, generator, convergence=conv)\nplot(ce)","category":"page"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"Choosing different invalidation rates makes the counterfactual more or less robust. The following plot shows the counterfactuals generated for different invalidation rates.","category":"page"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"(Image: )","category":"page"},{"location":"explanation/generators/probe/#References","page":"PROBE","title":"References","text":"","category":"section"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"Pawelczyk, Martin, Teresa Datta, Johannes van-den-Heuvel, Gjergji Kasneci, and Himabindu Lakkaraju. 2022. “Probabilistically Robust Recourse: Navigating the Trade-Offs Between Costs and Robustness in Algorithmic Recourse.” arXiv Preprint arXiv:2203.06768.","category":"page"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"explanation/generators/gravitational/#GravitationalGenerator","page":"Gravitational","title":"GravitationalGenerator","text":"","category":"section"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"The GravitationalGenerator was introduced in Altmeyer et al. (2023). It is named so because it generates counterfactuals that gravitate towards some sensible point in the target domain.","category":"page"},{"location":"explanation/generators/gravitational/#Description","page":"Gravitational","title":"Description","text":"","category":"section"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"Altmeyer et al. (2023) extend the general framework as follows,","category":"page"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"beginaligned\nmathbfs^prime = arg min_mathbfs^prime in mathcalS textyloss(M(f(mathbfs^prime))y^*) + lambda_1 textcost(f(mathbfs^prime)) + lambda_2 textextcost(f(mathbfs^prime)) \nendaligned ","category":"page"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"where textcost(f(mathbfs^prime)) denotes the proxy for costs faced by the individual. “The newly introduced term textextcost(f(mathbfs^prime)) is meant to capture and address external costs incurred by the collective of individuals in response to changes in mathbfs^prime.” (Altmeyer et al. 2023)","category":"page"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"For the GravitationalGenerator we have,","category":"page"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"beginaligned\ntextextcost(f(mathbfs^prime)) = textdist(f(mathbfs^prime)barx^*) \nendaligned","category":"page"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"where barx is some sensible point in the target domain, for example, the subsample average barx^*=textmean(x), x in mathcalD_1.","category":"page"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"There is a tradeoff then, between the distance of counterfactuals from their factual value and the chosen point in the target domain. The chart below illustrates how the counterfactual outcome changes as the penalty lambda_2 on the distance to the point in the target domain is increased from left to right (holding the other penalty term constant).","category":"page"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"(Image: )","category":"page"},{"location":"explanation/generators/gravitational/#Usage","page":"Gravitational","title":"Usage","text":"","category":"section"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"The approach can be used in our package as follows:","category":"page"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"generator = GravitationalGenerator()\nce = generate_counterfactual(x, target, counterfactual_data, M, generator)\ndisplay(plot(ce))","category":"page"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"(Image: )","category":"page"},{"location":"explanation/generators/gravitational/#Comparison-to-GenericGenerator","page":"Gravitational","title":"Comparison to GenericGenerator","text":"","category":"section"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"The figure below compares the outcome for the GenericGenerator and the GravitationalGenerator.","category":"page"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"(Image: )","category":"page"},{"location":"explanation/generators/gravitational/#References","page":"Gravitational","title":"References","text":"","category":"section"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"Altmeyer, Patrick, Giovan Angela, Aleksander Buszydlik, Karol Dobiczek, Arie van Deursen, and Cynthia Liem. 2023. “Endogenous Macrodynamics in Algorithmic Recourse.” In First IEEE Conference on Secure and Trustworthy Machine Learning. https://doi.org/10.1109/satml54575.2023.00036.","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"tutorials/benchmarking/#Performance-Benchmarks","page":"Benchmarking Explanations","title":"Performance Benchmarks","text":"","category":"section"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"In the previous tutorial, we have seen how counterfactual explanations can be evaluated. An important follow-up task is to compare the performance of different counterfactual generators is an important task. Researchers can use benchmarks to test new ideas they want to implement. Practitioners can find the right counterfactual generator for their specific use case through benchmarks. In this tutorial, we will see how to run benchmarks for counterfactual generators.","category":"page"},{"location":"tutorials/benchmarking/#Post-Hoc-Benchmarking","page":"Benchmarking Explanations","title":"Post Hoc Benchmarking","text":"","category":"section"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"We begin by continuing the discussion from the previous tutorial: suppose you have generated multiple counterfactual explanations for multiple individuals, like below:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"# Factual and target:\nn_individuals = 5\nids = rand(findall(predict_label(M, counterfactual_data) .== factual), n_individuals)\nxs = select_factual(counterfactual_data, ids)\nces = generate_counterfactual(xs, target, counterfactual_data, M, generator; num_counterfactuals=5)","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"You may be interested in comparing the outcomes across individuals. To benchmark the various counterfactual explanations using default evaluation measures, you can simply proceed as follows:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"bmk = benchmark(ces)","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Under the hood, the benchmark(counterfactual_explanations::Vector{CounterfactualExplanation}) uses CounterfactualExplanations.Evaluation.evaluate(ce::CounterfactualExplanation) to generate a Benchmark object, which contains the evaluation in its most granular form as a DataFrame.","category":"page"},{"location":"tutorials/benchmarking/#Working-with-Benchmarks","page":"Benchmarking Explanations","title":"Working with Benchmarks","text":"","category":"section"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"For convenience, the DataFrame containing the evaluation can be returned by simply calling the Benchmark object. By default, the aggregated evaluation measures across id (in line with the default behaviour of evaluate).","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"bmk()","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"15×7 DataFrame\n Row │ sample variable value generator ⋯\n │ Base.UUID String Float64 Symbol ⋯\n─────┼──────────────────────────────────────────────────────────────────────────\n 1 │ 239104d0-f59f-11ee-3d0c-d1db071927ff distance 3.17243 GradientBase ⋯\n 2 │ 239104d0-f59f-11ee-3d0c-d1db071927ff redundancy 0.0 GradientBase\n 3 │ 239104d0-f59f-11ee-3d0c-d1db071927ff validity 1.0 GradientBase\n 4 │ 2398b3e2-f59f-11ee-3323-13d53fb7e75b distance 3.07148 GradientBase\n 5 │ 2398b3e2-f59f-11ee-3323-13d53fb7e75b redundancy 0.0 GradientBase ⋯\n 6 │ 2398b3e2-f59f-11ee-3323-13d53fb7e75b validity 1.0 GradientBase\n 7 │ 2398b916-f59f-11ee-3f13-bd00858a39af distance 3.62159 GradientBase\n 8 │ 2398b916-f59f-11ee-3f13-bd00858a39af redundancy 0.0 GradientBase\n 9 │ 2398b916-f59f-11ee-3f13-bd00858a39af validity 1.0 GradientBase ⋯\n 10 │ 2398bce8-f59f-11ee-37c1-ef7c6de27b6b distance 2.62783 GradientBase\n 11 │ 2398bce8-f59f-11ee-37c1-ef7c6de27b6b redundancy 0.0 GradientBase\n 12 │ 2398bce8-f59f-11ee-37c1-ef7c6de27b6b validity 1.0 GradientBase\n 13 │ 2398c08a-f59f-11ee-175b-81c155750752 distance 2.91985 GradientBase ⋯\n 14 │ 2398c08a-f59f-11ee-175b-81c155750752 redundancy 0.0 GradientBase\n 15 │ 2398c08a-f59f-11ee-175b-81c155750752 validity 1.0 GradientBase\n 4 columns omitted","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"To retrieve the granular dataset, simply do:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"bmk(agg=nothing)","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"75×8 DataFrame\n Row │ sample num_counterfactual variable v ⋯\n │ Base.UUID Int64 String F ⋯\n─────┼──────────────────────────────────────────────────────────────────────────\n 1 │ 239104d0-f59f-11ee-3d0c-d1db071927ff 1 distance 3 ⋯\n 2 │ 239104d0-f59f-11ee-3d0c-d1db071927ff 2 distance 3\n 3 │ 239104d0-f59f-11ee-3d0c-d1db071927ff 3 distance 3\n 4 │ 239104d0-f59f-11ee-3d0c-d1db071927ff 4 distance 3\n 5 │ 239104d0-f59f-11ee-3d0c-d1db071927ff 5 distance 3 ⋯\n 6 │ 239104d0-f59f-11ee-3d0c-d1db071927ff 1 redundancy 0\n 7 │ 239104d0-f59f-11ee-3d0c-d1db071927ff 2 redundancy 0\n 8 │ 239104d0-f59f-11ee-3d0c-d1db071927ff 3 redundancy 0\n 9 │ 239104d0-f59f-11ee-3d0c-d1db071927ff 4 redundancy 0 ⋯\n 10 │ 239104d0-f59f-11ee-3d0c-d1db071927ff 5 redundancy 0\n 11 │ 239104d0-f59f-11ee-3d0c-d1db071927ff 1 validity 1\n ⋮ │ ⋮ ⋮ ⋮ ⋱\n 66 │ 2398c08a-f59f-11ee-175b-81c155750752 1 redundancy 0\n 67 │ 2398c08a-f59f-11ee-175b-81c155750752 2 redundancy 0 ⋯\n 68 │ 2398c08a-f59f-11ee-175b-81c155750752 3 redundancy 0\n 69 │ 2398c08a-f59f-11ee-175b-81c155750752 4 redundancy 0\n 70 │ 2398c08a-f59f-11ee-175b-81c155750752 5 redundancy 0\n 71 │ 2398c08a-f59f-11ee-175b-81c155750752 1 validity 1 ⋯\n 72 │ 2398c08a-f59f-11ee-175b-81c155750752 2 validity 1\n 73 │ 2398c08a-f59f-11ee-175b-81c155750752 3 validity 1\n 74 │ 2398c08a-f59f-11ee-175b-81c155750752 4 validity 1\n 75 │ 2398c08a-f59f-11ee-175b-81c155750752 5 validity 1 ⋯\n 5 columns and 54 rows omitted","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Since benchmarks return a DataFrame object on call, post-processing is straightforward. For example, we could use Tidier.jl:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"using Tidier\n@chain bmk() begin\n @filter(variable == \"distance\")\n @select(sample, variable, value)\nend","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"5×3 DataFrame\n Row │ sample variable value \n │ Base.UUID String Float64 \n─────┼─────────────────────────────────────────────────────────\n 1 │ 239104d0-f59f-11ee-3d0c-d1db071927ff distance 3.17243\n 2 │ 2398b3e2-f59f-11ee-3323-13d53fb7e75b distance 3.07148\n 3 │ 2398b916-f59f-11ee-3f13-bd00858a39af distance 3.62159\n 4 │ 2398bce8-f59f-11ee-37c1-ef7c6de27b6b distance 2.62783\n 5 │ 2398c08a-f59f-11ee-175b-81c155750752 distance 2.91985","category":"page"},{"location":"tutorials/benchmarking/#Metadata-for-Counterfactual-Explanations","page":"Benchmarking Explanations","title":"Metadata for Counterfactual Explanations","text":"","category":"section"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Benchmarks always report metadata for each counterfactual explanation, which is automatically inferred by default. The default metadata concerns the explained model and the employed generator. In the current example, we used the same model and generator for each individual:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"@chain bmk() begin\n @group_by(sample)\n @select(sample, model, generator)\n @summarize(model=first(model),generator=first(generator))\n @ungroup\nend","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"5×3 DataFrame\n Row │ sample model ⋯\n │ Base.UUID Symbol ⋯\n─────┼──────────────────────────────────────────────────────────────────────────\n 1 │ 239104d0-f59f-11ee-3d0c-d1db071927ff FluxModel(Chain(Dense(2 => 2)), … ⋯\n 2 │ 2398b3e2-f59f-11ee-3323-13d53fb7e75b FluxModel(Chain(Dense(2 => 2)), …\n 3 │ 2398b916-f59f-11ee-3f13-bd00858a39af FluxModel(Chain(Dense(2 => 2)), …\n 4 │ 2398bce8-f59f-11ee-37c1-ef7c6de27b6b FluxModel(Chain(Dense(2 => 2)), …\n 5 │ 2398c08a-f59f-11ee-175b-81c155750752 FluxModel(Chain(Dense(2 => 2)), … ⋯\n 1 column omitted","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Metadata can also be provided as an optional key argument.","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"meta_data = Dict(\n :generator => \"Generic\",\n :model => \"MLP\",\n)\nmeta_data = [meta_data for i in 1:length(ces)]\nbmk = benchmark(ces; meta_data=meta_data)\n@chain bmk() begin\n @group_by(sample)\n @select(sample, model, generator)\n @summarize(model=first(model),generator=first(generator))\n @ungroup\nend","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"5×3 DataFrame\n Row │ sample model generator \n │ Base.UUID String String \n─────┼─────────────────────────────────────────────────────────\n 1 │ 27fae496-f59f-11ee-2c30-f35d1025a6d4 MLP Generic\n 2 │ 27fdcc6a-f59f-11ee-030b-152c9794c5f1 MLP Generic\n 3 │ 27fdd04a-f59f-11ee-2010-e1732ff5d8d2 MLP Generic\n 4 │ 27fdd340-f59f-11ee-1d20-050a69dcacef MLP Generic\n 5 │ 27fdd5fc-f59f-11ee-02e8-d198e436abb3 MLP Generic","category":"page"},{"location":"tutorials/benchmarking/#Ad-Hoc-Benchmarking","page":"Benchmarking Explanations","title":"Ad Hoc Benchmarking","text":"","category":"section"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"So far we have assumed the following workflow:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Fit some machine learning model.\nGenerate counterfactual explanations for some individual(s) (generate_counterfactual).\nEvaluate and benchmark them (benchmark(ces::Vector{CounterfactualExplanation})).","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"In many cases, it may be preferable to combine these steps. To this end, we have added support for two scenarios of Ad Hoc Benchmarking.","category":"page"},{"location":"tutorials/benchmarking/#Pre-trained-Models","page":"Benchmarking Explanations","title":"Pre-trained Models","text":"","category":"section"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"In the first scenario, it is assumed that the machine learning models have been pre-trained and so the workflow can be summarized as follows:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Fit some machine learning model(s).\nGenerate counterfactual explanations and benchmark them.","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"We suspect that this is the most common workflow for practitioners who are interested in benchmarking counterfactual explanations for the pre-trained machine learning models. Let’s go through this workflow using a simple example. We first train some models and store them in a dictionary:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"models = Dict(\n :MLP => fit_model(counterfactual_data, :MLP),\n :Linear => fit_model(counterfactual_data, :Linear),\n)","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Next, we store the counterfactual generators of interest in a dictionary as well:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"generators = Dict(\n :Generic => GenericGenerator(),\n :Gravitational => GravitationalGenerator(),\n :Wachter => WachterGenerator(),\n :ClaPROAR => ClaPROARGenerator(),\n)","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Then we can run a benchmark for individual(s) x, a pre-specified target and counterfactual_data as follows:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"bmk = benchmark(x, target, counterfactual_data; models=models, generators=generators)","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"In this case, metadata is automatically inferred from the dictionaries:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"@chain bmk() begin\n @filter(variable == \"distance\")\n @select(sample, variable, value, model, generator)\nend","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"8×5 DataFrame\n Row │ sample variable value model ⋯\n │ Base.UUID String Float64 Tuple… ⋯\n─────┼──────────────────────────────────────────────────────────────────────────\n 1 │ 2cba5eee-f59f-11ee-1844-cbc7a8372a38 distance 4.38877 (:Linear, Flux ⋯\n 2 │ 2cd740fe-f59f-11ee-35c3-1157eb1b7583 distance 4.17021 (:Linear, Flux\n 3 │ 2cd741e2-f59f-11ee-2b09-0d55ef9892b9 distance 4.31145 (:Linear, Flux\n 4 │ 2cd7420c-f59f-11ee-1996-6fa75e23bb57 distance 4.17035 (:Linear, Flux\n 5 │ 2cd74234-f59f-11ee-0ad0-9f21949f5932 distance 5.73182 (:MLP, FluxMod ⋯\n 6 │ 2cd7425c-f59f-11ee-3eb4-af34f85ffd3d distance 5.50606 (:MLP, FluxMod\n 7 │ 2cd7427a-f59f-11ee-10d3-a1df6c8dc125 distance 5.2114 (:MLP, FluxMod\n 8 │ 2cd74298-f59f-11ee-32d1-f501c104fea8 distance 5.3623 (:MLP, FluxMod\n 2 columns omitted","category":"page"},{"location":"tutorials/benchmarking/#Everything-at-once","page":"Benchmarking Explanations","title":"Everything at once","text":"","category":"section"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Researchers, in particular, may be interested in combining all steps into one. This is the second scenario of Ad Hoc Benchmarking:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Fit some machine learning model(s), generate counterfactual explanations and benchmark them.","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"It involves calling benchmark directly on counterfactual data (the only positional argument):","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"bmk = benchmark(counterfactual_data)","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"This will use the default models from standard_models_catalogue and train them on the data. All available generators from generator_catalogue will also be used:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"@chain bmk() begin\n @filter(variable == \"validity\")\n @select(sample, variable, value, model, generator)\nend","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"200×5 DataFrame\n Row │ sample variable value model genera ⋯\n │ Base.UUID String Float64 Symbol Symbol ⋯\n─────┼──────────────────────────────────────────────────────────────────────────\n 1 │ 32d1817e-f59f-11ee-152f-a30b18c2e6f7 validity 1.0 Linear gravit ⋯\n 2 │ 32d1817e-f59f-11ee-152f-a30b18c2e6f7 validity 1.0 Linear growin\n 3 │ 32d1817e-f59f-11ee-152f-a30b18c2e6f7 validity 1.0 Linear revise\n 4 │ 32d1817e-f59f-11ee-152f-a30b18c2e6f7 validity 1.0 Linear clue\n 5 │ 32d1817e-f59f-11ee-152f-a30b18c2e6f7 validity 1.0 Linear probe ⋯\n 6 │ 32d1817e-f59f-11ee-152f-a30b18c2e6f7 validity 1.0 Linear dice\n 7 │ 32d1817e-f59f-11ee-152f-a30b18c2e6f7 validity 1.0 Linear clapro\n 8 │ 32d1817e-f59f-11ee-152f-a30b18c2e6f7 validity 1.0 Linear wachte\n 9 │ 32d1817e-f59f-11ee-152f-a30b18c2e6f7 validity 1.0 Linear generi ⋯\n 10 │ 32d1817e-f59f-11ee-152f-a30b18c2e6f7 validity 1.0 Linear greedy\n 11 │ 32d255e8-f59f-11ee-3e8d-a9e9f6e23ea8 validity 1.0 Linear gravit\n ⋮ │ ⋮ ⋮ ⋮ ⋮ ⋱\n 191 │ 3382d08a-f59f-11ee-10b3-f7d18cf7d3b5 validity 1.0 MLP gravit\n 192 │ 3382d08a-f59f-11ee-10b3-f7d18cf7d3b5 validity 1.0 MLP growin ⋯\n 193 │ 3382d08a-f59f-11ee-10b3-f7d18cf7d3b5 validity 1.0 MLP revise\n 194 │ 3382d08a-f59f-11ee-10b3-f7d18cf7d3b5 validity 1.0 MLP clue\n 195 │ 3382d08a-f59f-11ee-10b3-f7d18cf7d3b5 validity 1.0 MLP probe\n 196 │ 3382d08a-f59f-11ee-10b3-f7d18cf7d3b5 validity 1.0 MLP dice ⋯\n 197 │ 3382d08a-f59f-11ee-10b3-f7d18cf7d3b5 validity 1.0 MLP clapro\n 198 │ 3382d08a-f59f-11ee-10b3-f7d18cf7d3b5 validity 1.0 MLP wachte\n 199 │ 3382d08a-f59f-11ee-10b3-f7d18cf7d3b5 validity 1.0 MLP generi\n 200 │ 3382d08a-f59f-11ee-10b3-f7d18cf7d3b5 validity 1.0 MLP greedy ⋯\n 1 column and 179 rows omitted","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Optionally, you can instead provide a dictionary of models and generators as before. Each value in the models dictionary should be one of two things:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Either be an object M of type AbstractFittedModel that implements the Models.train method.\nOr a DataType that can be called on CounterfactualData to create an object M as in (a).","category":"page"},{"location":"tutorials/benchmarking/#Multiple-Datasets","page":"Benchmarking Explanations","title":"Multiple Datasets","text":"","category":"section"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Benchmarks are run on single instances of type CounterfactualData. This is our design choice for two reasons:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"We want to avoid the loops inside the benchmark method(s) from getting too nested and convoluted.\nWhile it is straightforward to infer metadata for models and generators, this is not the case for datasets.","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Fortunately, it is very easy to run benchmarks for multiple datasets anyway, since Benchmark instances can be concatenated. To see how, let’s consider an example involving multiple datasets, models and generators:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"# Data:\ndatasets = Dict(\n :moons => CounterfactualData(load_moons()...),\n :circles => CounterfactualData(load_circles()...),\n)\n\n# Models:\nmodels = Dict(\n :MLP => FluxModel,\n :Linear => Linear,\n)\n\n# Generators:\ngenerators = Dict(\n :Generic => GenericGenerator(),\n :Greedy => GreedyGenerator(),\n)","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Then we can simply loop over the datasets and eventually concatenate the results like so:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"using CounterfactualExplanations.Evaluation: distance_measures\nbmks = []\nfor (dataname, dataset) in datasets\n bmk = benchmark(dataset; models=models, generators=generators, measure=distance_measures)\n push!(bmks, bmk)\nend\nbmk = vcat(bmks[1], bmks[2]; ids=collect(keys(datasets)))","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"When ids are supplied, then a new id column is added to the evaluation data frame that contains unique identifiers for the different benchmarks. The optional idcol_name argument can be used to specify the name for that indicator column (defaults to \"dataset\"):","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"@chain bmk() begin\n @group_by(dataset, generator)\n @filter(model == :MLP)\n @filter(variable == \"distance_l1\")\n @summarize(L1_norm=mean(value))\n @ungroup\nend","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"4×3 DataFrame\n Row │ dataset generator L1_norm \n │ Symbol Symbol Float32 \n─────┼──────────────────────────────\n 1 │ moons Generic 1.56555\n 2 │ moons Greedy 0.819269\n 3 │ circles Generic 1.83524\n 4 │ circles Greedy 0.498953","category":"page"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"tutorials/whistle_stop/#Whistle-Stop-Tour","page":"Whiste-Stop Tour","title":"Whistle-Stop Tour","text":"","category":"section"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"In this tutorial, we will go through a slightly more complex example involving synthetic data. We will generate Counterfactual Explanations using different generators and visualize the results.","category":"page"},{"location":"tutorials/whistle_stop/#Data-and-Classifier","page":"Whiste-Stop Tour","title":"Data and Classifier","text":"","category":"section"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"# Choose some values for data and a model:\nn_dim = 2\nn_classes = 4\nn_samples = 400\nmodel_name = :MLP","category":"page"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"The code chunk below generates synthetic data and uses it to fit a classifier. The outcome variable counterfactual_data.y consists of 4 classes. The input data counterfactual_data.X consists of 2 features. We generate a total of 400 samples. On the model side, we have specified model_name = :MLP. The fit_model can be used to fit a number of default models.","category":"page"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"data = TaijaData.load_multi_class(n_samples)\ncounterfactual_data = DataPreprocessing.CounterfactualData(data...)\nM = fit_model(counterfactual_data, model_name)","category":"page"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"The chart below visualizes our data along with the model predictions. In particular, the contour indicates the predicted probabilities generated by our classifier. By default, these are the predicted probabilities for y=1, the first label. For multi-dimensional input data is compressed into two dimensions and the decision boundary is approximated using Nearest Neighbors (this is still somewhat experimental).","category":"page"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"plot(M, counterfactual_data)","category":"page"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"(Image: )","category":"page"},{"location":"tutorials/whistle_stop/#Counterfactual-Explanation","page":"Whiste-Stop Tour","title":"Counterfactual Explanation","text":"","category":"section"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"Next, we begin by specifying our target and factual label. We then draw a random sample from the non-target (factual) class.","category":"page"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"# Factual and target:\ntarget = 2\nfactual = 4\nchosen = rand(findall(predict_label(M, counterfactual_data) .== factual))\nx = select_factual(counterfactual_data, chosen)","category":"page"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"This sets the baseline for our counterfactual search: we plan to perturb the factual x to change the predicted label from y=4 to our target label target=2.","category":"page"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"Counterfactual generators accept several default parameters that can be used to adjust the counterfactual search at a high level: for example, a Flux.jl optimizer can be supplied to define how exactly gradient steps are performed. Importantly, one can also define the threshold probability at which the counterfactual search will converge. This relates to the probability predicted by the underlying black-box model, that the counterfactual belongs to the target class. A higher decision threshold typically prolongs the counterfactual search.","category":"page"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"# Search params:\ndecision_threshold = 0.75\nnum_counterfactuals = 3","category":"page"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"The code below runs the counterfactual search for each generator available in the generator_catalogue. In each case, we also call the generic plot() method on the generated instance of type CounterfactualExplanation. This generates a simple plot that visualizes the entire counterfactual path. The chart below shows the results for all counterfactual generators: Factual: 4 → Target: 2.","category":"page"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"ces = Dict()\nplts = []\nplottable_generators = filter(((k,v),) -> k ∉ [:growing_spheres, :feature_tweak], generator_catalogue)\n# Search:\nfor (key, Generator) in plottable_generators\n generator = Generator()\n ce = generate_counterfactual(\n x, target, counterfactual_data, M, generator;\n num_counterfactuals = num_counterfactuals,\n convergence=GeneratorConditionsConvergence(\n decision_threshold=decision_threshold\n )\n )\n ces[key] = ce\n plts = [plts..., plot(ce; title=key, colorbar=false)]\nend","category":"page"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"(Image: )","category":"page"},{"location":"how_to_guides/custom_models/","page":"... add custom models","title":"... add custom models","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"how_to_guides/custom_models/#How-to-add-Custom-Models","page":"... add custom models","title":"How to add Custom Models","text":"","category":"section"},{"location":"how_to_guides/custom_models/","page":"... add custom models","title":"... add custom models","text":"Adding custom models is possible and relatively straightforward, as we will demonstrate in this guide.","category":"page"},{"location":"how_to_guides/custom_models/#Custom-Models","page":"... add custom models","title":"Custom Models","text":"","category":"section"},{"location":"how_to_guides/custom_models/","page":"... add custom models","title":"... add custom models","text":"Apart from the default models you can use any arbitrary (differentiable) model and generate recourse in the same way as before. Only two steps are necessary to make your own Julia model compatible with this package:","category":"page"},{"location":"how_to_guides/custom_models/","page":"... add custom models","title":"... add custom models","text":"The model needs to be declared as a subtype of <:CounterfactualExplanations.Models.AbstractFittedModel.\nYou need to extend the functions CounterfactualExplanations.Models.logits and CounterfactualExplanations.Models.probs for your custom model.","category":"page"},{"location":"how_to_guides/custom_models/#How-FluxModel-was-added","page":"... add custom models","title":"How FluxModel was added","text":"","category":"section"},{"location":"how_to_guides/custom_models/","page":"... add custom models","title":"... add custom models","text":"To demonstrate how this can be done in practice, we will reiterate here how native support for Flux.jl models was enabled (Innes 2018). Once again we use synthetic data for an illustrative example. The code below loads the data and builds a simple model architecture that can be used for a multi-class prediction task. Note how outputs from the final layer are not passed through a softmax activation function, since the counterfactual loss is evaluated with respect to logits. The model is trained with dropout.","category":"page"},{"location":"how_to_guides/custom_models/","page":"... add custom models","title":"... add custom models","text":"# Data:\nN = 200\ndata = TaijaData.load_blobs(N; centers=4, cluster_std=0.5)\ncounterfactual_data = DataPreprocessing.CounterfactualData(data...)\ny = counterfactual_data.y\nX = counterfactual_data.X\n\n# Flux model setup: \nusing Flux\ndata = Flux.DataLoader((X,y), batchsize=1)\nn_hidden = 32\noutput_dim = size(y,1)\ninput_dim = 2\nactivation = σ\nmodel = Chain(\n Dense(input_dim, n_hidden, activation),\n Dropout(0.1),\n Dense(n_hidden, output_dim)\n) \nloss(x, y) = Flux.Losses.logitcrossentropy(model(x), y)\n\n# Flux model training:\nusing Flux.Optimise: update!, Adam\nopt = Adam()\nepochs = 50\nfor epoch = 1:epochs\n for d in data\n gs = gradient(Flux.params(model)) do\n l = loss(d...)\n end\n update!(opt, Flux.params(model), gs)\n end\nend","category":"page"},{"location":"how_to_guides/custom_models/","page":"... add custom models","title":"... add custom models","text":"The code below implements the two steps that were necessary to make Flux models compatible with the package. We first declare our new struct as a subtype of <:AbstractDifferentiableModel, which itself is an abstract subtype of <:AbstractFittedModel. Computing logits amounts to just calling the model on inputs. Predicted probabilities for labels can in this case be computed by passing predicted logits through the softmax function. Finally, we just instantiate our model in the same way as always.","category":"page"},{"location":"how_to_guides/custom_models/","page":"... add custom models","title":"... add custom models","text":"# Step 1)\nstruct MyFluxModel <: AbstractDifferentiableModel\n model::Any\n likelihood::Symbol\nend\n\n# Step 2)\n# import functions in order to extend\nimport CounterfactualExplanations.Models: logits\nimport CounterfactualExplanations.Models: probs \nlogits(M::MyFluxModel, X::AbstractArray) = M.model(X)\nprobs(M::MyFluxModel, X::AbstractArray) = softmax(logits(M, X))\nM = MyFluxModel(model, :classification_multi)","category":"page"},{"location":"how_to_guides/custom_models/","page":"... add custom models","title":"... add custom models","text":"The code below implements the counterfactual search and plots the results:","category":"page"},{"location":"how_to_guides/custom_models/","page":"... add custom models","title":"... add custom models","text":"factual_label = 4\ntarget = 2\nchosen = rand(findall(predict_label(M, counterfactual_data) .== factual_label))\nx = select_factual(counterfactual_data, chosen) \n\n# Counterfactual search:\ngenerator = GenericGenerator()\nce = generate_counterfactual(x, target, counterfactual_data, M, generator)\nplot(ce)","category":"page"},{"location":"how_to_guides/custom_models/","page":"... add custom models","title":"... add custom models","text":"(Image: )","category":"page"},{"location":"how_to_guides/custom_models/#References","page":"... add custom models","title":"References","text":"","category":"section"},{"location":"how_to_guides/custom_models/","page":"... add custom models","title":"... add custom models","text":"Innes, Mike. 2018. “Flux: Elegant Machine Learning with Julia.” Journal of Open Source Software 3 (25): 602. https://doi.org/10.21105/joss.00602.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"CurrentModule = CounterfactualExplanations","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"(Image: )","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Documentation for CounterfactualExplanations.jl.","category":"page"},{"location":"#CounterfactualExplanations","page":"🏠 Home","title":"CounterfactualExplanations","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Counterfactual Explanations and Algorithmic Recourse in Julia.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"(Image: Stable) (Image: Dev) (Image: Build Status) (Image: Coverage) (Image: Code Style: Blue) (Image: License) (Image: Package Downloads) (Image: Aqua QA)","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"CounterfactualExplanations.jl is a package for generating Counterfactual Explanations (CE) and Algorithmic Recourse (AR) for black-box algorithms. Both CE and AR are related tools for explainable artificial intelligence (XAI). While the package is written purely in Julia, it can be used to explain machine learning algorithms developed and trained in other popular programming languages like Python and R. See below for a short introduction and other resources or dive straight into the docs.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"There is also a corresponding paper, Explaining Black-Box Models through Counterfactuals, which has been published in JuliaCon Proceedings. Please consider citing the paper, if you use this package in your work:","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"(Image: DOI) (Image: DOI)","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"@article{Altmeyer2023,\n doi = {10.21105/jcon.00130},\n url = {https://doi.org/10.21105/jcon.00130},\n year = {2023},\n publisher = {The Open Journal},\n volume = {1},\n number = {1},\n pages = {130},\n author = {Patrick Altmeyer and Arie van Deursen and Cynthia C. s. Liem},\n title = {Explaining Black-Box Models through Counterfactuals},\n journal = {Proceedings of the JuliaCon Conferences}\n}","category":"page"},{"location":"#Installation","page":"🏠 Home","title":"🚩 Installation","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"You can install the stable release from Julia’s General Registry as follows:","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"using Pkg\nPkg.add(\"CounterfactualExplanations\")","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"CounterfactualExplanations.jl is under active development. To install the development version of the package you can run the following command:","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"using Pkg\nPkg.add(url=\"https://github.com/juliatrustworthyai/CounterfactualExplanations.jl\")","category":"page"},{"location":"#Background-and-Motivation","page":"🏠 Home","title":"🤔 Background and Motivation","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Machine learning models like Deep Neural Networks have become so complex, opaque and underspecified in the data that they are generally considered Black Boxes. Nonetheless, such models often play a key role in data-driven decision-making systems. This creates the following problem: human operators in charge of such systems have to rely on them blindly, while those individuals subject to them generally have no way of challenging an undesirable outcome:","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"“You cannot appeal to (algorithms). They do not listen. Nor do they bend.”— Cathy O’Neil in Weapons of Math Destruction, 2016","category":"page"},{"location":"#Enter:-Counterfactual-Explanations","page":"🏠 Home","title":"🔮 Enter: Counterfactual Explanations","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Counterfactual Explanations can help human stakeholders make sense of the systems they develop, use or endure: they explain how inputs into a system need to change for it to produce different decisions. Explainability benefits internal as well as external quality assurance.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Counterfactual Explanations have a few properties that are desirable in the context of Explainable Artificial Intelligence (XAI). These include:","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Full fidelity to the black-box model, since no proxy is involved.\nNo need for (reasonably) interpretable features as opposed to LIME and SHAP.\nClear link to Algorithmic Recourse and Causal Inference.\nLess susceptible to adversarial attacks than LIME and SHAP.","category":"page"},{"location":"#Example:-Give-Me-Some-Credit","page":"🏠 Home","title":"Example: Give Me Some Credit","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Consider the following real-world scenario: a retail bank is using a black-box model trained on their clients’ credit history to decide whether they will provide credit to new applicants. To simulate this scenario, we have pre-trained a binary classifier on the publicly available Give Me Some Credit dataset that ships with this package (Kaggle 2011).","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"The figure below shows counterfactuals for 10 randomly chosen individuals that would have been denied credit initially.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"(Image: )","category":"page"},{"location":"#Example:-MNIST","page":"🏠 Home","title":"Example: MNIST","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"The figure below shows a counterfactual generated for an image classifier trained on MNIST: in particular, it demonstrates which pixels need to change in order for the classifier to predict 3 instead of 8.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Since v0.1.9 counterfactual generators are fully composable. Here we have composed a generator that combines ideas from Wachter, Mittelstadt, and Russell (2017) and Altmeyer et al. (2023):","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"# Compose generator:\nusing CounterfactualExplanations.Objectives: distance_from_target\ngenerator = GradientBasedGenerator()\n@chain generator begin\n @objective logitcrossentropy + 0.1distance_mad + 0.1distance_from_target\n @with_optimiser Adam(0.1) \nend\ncounterfactual_data.generative_model = vae # assign generative model","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"(Image: )","category":"page"},{"location":"#Usage-example","page":"🏠 Home","title":"🔍 Usage example","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Generating counterfactuals will typically look like follows. Below we first fit a simple model to a synthetic dataset with linearly separable features and then draw a random sample:","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"# Data and Classifier:\ncounterfactual_data = CounterfactualData(load_linearly_separable()...)\nM = fit_model(counterfactual_data, :Linear)\n\n# Select random sample:\ntarget = 2\nfactual = 1\nchosen = rand(findall(predict_label(M, counterfactual_data) .== factual))\nx = select_factual(counterfactual_data, chosen)","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"To this end, we specify a counterfactual generator of our choice:","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"# Counterfactual search:\ngenerator = DiCEGenerator(λ=[0.1,0.3])","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Here, we have chosen to use the GradientBasedGenerator to move the individual from its factual label 1 to the target label 2.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"With all of our ingredients specified, we finally generate counterfactuals using a simple API call:","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"conv = conv = CounterfactualExplanations.Convergence.GeneratorConditionsConvergence()\nce = generate_counterfactual(\n x, target, counterfactual_data, M, generator; \n num_counterfactuals=3, convergence=conv,\n)","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"The plot below shows the resulting counterfactual path:","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"(Image: )","category":"page"},{"location":"#Implemented-Counterfactual-Generators","page":"🏠 Home","title":"☑️ Implemented Counterfactual Generators","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Currently, the following counterfactual generators are implemented:","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"ClaPROAR (Altmeyer et al. 2023)\nCLUE (Antorán et al. 2020)\nDiCE (Mothilal, Sharma, and Tan 2020)\nFeatureTweak (Tolomei et al. 2017)\nGeneric\nGravitationalGenerator (Altmeyer et al. 2023)\nGreedy (Schut et al. 2021)\nGrowingSpheres (Laugel et al. 2017)\nPROBE (Pawelczyk et al. 2023)\nREVISE (Joshi et al. 2019)\nWachter (Wachter, Mittelstadt, and Russell 2017)","category":"page"},{"location":"#Goals-and-limitations","page":"🏠 Home","title":"🎯 Goals and limitations","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"The goal of this library is to contribute to efforts towards trustworthy machine learning in Julia. The Julia language has an edge when it comes to trustworthiness: it is very transparent. Packages like this one are generally written in pure Julia, which makes it easy for users and developers to understand and contribute to open-source code. Eventually, this project aims to offer a one-stop-shop of counterfactual explanations.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Our ambition is to enhance the package through the following features:","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Support for all supervised machine learning models trained in MLJ.jl.\nSupport for regression models.","category":"page"},{"location":"#Contribute","page":"🏠 Home","title":"🛠 Contribute","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Contributions of any kind are very much welcome! Take a look at the issue to see what things we are currently working on. If you have an idea for a new feature or want to report a bug, please open a new issue.","category":"page"},{"location":"#Development","page":"🏠 Home","title":"Development","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"If your looking to contribute code, it may be helpful to check out the Explanation section of the docs.","category":"page"},{"location":"#Testing","page":"🏠 Home","title":"Testing","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Please always make sure to add tests for any new features or changes.","category":"page"},{"location":"#Documentation","page":"🏠 Home","title":"Documentation","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"If you add new features or change existing ones, please make sure to update the documentation accordingly. The documentation is written in Documenter.jl and is located in the docs/src folder.","category":"page"},{"location":"#Log-Changes","page":"🏠 Home","title":"Log Changes","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"As of version 1.1.1, we have tried to be more stringent about logging changes. Please make sure to add a note to the CHANGELOG.md file for any changes you make. It is sufficient to add a note under the Unreleased section.","category":"page"},{"location":"#General-Pointers","page":"🏠 Home","title":"General Pointers","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"There are also some general pointers for people looking to contribute to any of our Taija packages here.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Please follow the SciML ColPrac guide.","category":"page"},{"location":"#Citation","page":"🏠 Home","title":"🎓 Citation","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"If you want to use this codebase, please consider citing the corresponding paper:","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"@article{Altmeyer2023,\n doi = {10.21105/jcon.00130},\n url = {https://doi.org/10.21105/jcon.00130},\n year = {2023},\n publisher = {The Open Journal},\n volume = {1},\n number = {1},\n pages = {130},\n author = {Patrick Altmeyer and Arie van Deursen and Cynthia C. s. Liem},\n title = {Explaining Black-Box Models through Counterfactuals},\n journal = {Proceedings of the JuliaCon Conferences}\n}","category":"page"},{"location":"#References","page":"🏠 Home","title":"📚 References","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Altmeyer, Patrick, Giovan Angela, Aleksander Buszydlik, Karol Dobiczek, Arie van Deursen, and Cynthia CS Liem. 2023. “Endogenous Macrodynamics in Algorithmic Recourse.” In 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), 418–31. IEEE.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Antorán, Javier, Umang Bhatt, Tameem Adel, Adrian Weller, and José Miguel Hernández-Lobato. 2020. “Getting a Clue: A Method for Explaining Uncertainty Estimates.” https://arxiv.org/abs/2006.06848.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Joshi, Shalmali, Oluwasanmi Koyejo, Warut Vijitbenjaronk, Been Kim, and Joydeep Ghosh. 2019. “Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems.” https://arxiv.org/abs/1907.09615.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Kaggle. 2011. “Give Me Some Credit, Improve on the State of the Art in Credit Scoring by Predicting the Probability That Somebody Will Experience Financial Distress in the Next Two Years.” https://www.kaggle.com/c/GiveMeSomeCredit; Kaggle. https://www.kaggle.com/c/GiveMeSomeCredit.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Laugel, Thibault, Marie-Jeanne Lesot, Christophe Marsala, Xavier Renard, and Marcin Detyniecki. 2017. “Inverse Classification for Comparison-Based Interpretability in Machine Learning.” https://arxiv.org/abs/1712.08443.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Mothilal, Ramaravind K, Amit Sharma, and Chenhao Tan. 2020. “Explaining Machine Learning Classifiers Through Diverse Counterfactual Explanations.” In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 607–17. https://doi.org/10.1145/3351095.3372850.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Pawelczyk, Martin, Teresa Datta, Johannes van-den-Heuvel, Gjergji Kasneci, and Himabindu Lakkaraju. 2023. “Probabilistically Robust Recourse: Navigating the Trade-Offs Between Costs and Robustness in Algorithmic Recourse.” https://arxiv.org/abs/2203.06768.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Schut, Lisa, Oscar Key, Rory Mc Grath, Luca Costabello, Bogdan Sacaleanu, Yarin Gal, et al. 2021. “Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertainties.” In International Conference on Artificial Intelligence and Statistics, 1756–64. PMLR.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Tolomei, Gabriele, Fabrizio Silvestri, Andrew Haines, and Mounia Lalmas. 2017. “Interpretable Predictions of Tree-Based Ensembles via Actionable Feature Tweaking.” In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 465–74. https://doi.org/10.1145/3097983.3098039.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Wachter, Sandra, Brent Mittelstadt, and Chris Russell. 2017. “Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR.” Harv. JL & Tech. 31: 841. https://doi.org/10.2139/ssrn.3063289.","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"explanation/categorical/#Categorical-Features","page":"Categorical Features","title":"Categorical Features","text":"","category":"section"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"To illustrate how data is preprocessed under the hood, we consider a simple toy dataset with three categorical features (name, grade and sex) and one continuous feature (age):","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"X = (\n name=categorical([\"Danesh\", \"Lee\", \"Mary\", \"John\"]),\n grade=categorical([\"A\", \"B\", \"A\", \"C\"], ordered=true),\n sex=categorical([\"male\",\"female\",\"male\",\"male\"]),\n height=[1.85, 1.67, 1.5, 1.67],\n)\nschema(X)","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"Categorical features are expected to be one-hot or dummy encoded. To this end, we could use MLJ, for example:","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"hot = OneHotEncoder()\nmach = fit!(machine(hot, X))\nW = transform(mach, X)\nschema(W)","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"┌──────────────┬────────────┬─────────┐\n│ names │ scitypes │ types │\n├──────────────┼────────────┼─────────┤\n│ name__Danesh │ Continuous │ Float64 │\n│ name__John │ Continuous │ Float64 │\n│ name__Lee │ Continuous │ Float64 │\n│ name__Mary │ Continuous │ Float64 │\n│ grade__A │ Continuous │ Float64 │\n│ grade__B │ Continuous │ Float64 │\n│ grade__C │ Continuous │ Float64 │\n│ sex__female │ Continuous │ Float64 │\n│ sex__male │ Continuous │ Float64 │\n│ height │ Continuous │ Float64 │\n└──────────────┴────────────┴─────────┘","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"The matrix that will be perturbed during the counterfactual search looks as follows:","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"X = permutedims(MLJBase.matrix(W))","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"10×4 Matrix{Float64}:\n 1.0 0.0 0.0 0.0\n 0.0 0.0 0.0 1.0\n 0.0 1.0 0.0 0.0\n 0.0 0.0 1.0 0.0\n 1.0 0.0 1.0 0.0\n 0.0 1.0 0.0 0.0\n 0.0 0.0 0.0 1.0\n 0.0 1.0 0.0 0.0\n 1.0 0.0 1.0 1.0\n 1.85 1.67 1.5 1.67","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"The CounterfactualData constructor takes two optional arguments that can be used to specify the indices of categorical and continuous features. If nothing is supplied, all features are assumed to be continuous. For categorical features, the constructor expects and array of arrays of integers (Vector{Vector{Int}}) where each subarray includes the indices of a all one-hot encoded rows related to a single categorical feature. In the example above, the name feature is one-hot encoded across rows 1, 2 and 3 of X.","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"features_categorical = [\n [1,2,3,4], # name\n [5,6,7], # grade\n [8,9] # sex\n]\nfeatures_continuous = [10]","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"We propose the following simple logic for reconstructing categorical encodings after perturbations:","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"For one-hot encoded features with multiple classes, choose the maximum.\nFor binary features, clip the perturbed value to fall into 01 and round to the nearest of the two integers.","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"function reconstruct_cat_encoding(x)\n map(features_categorical) do cat_group_index\n if length(cat_group_index) > 1\n x[cat_group_index] = Int.(x[cat_group_index] .== maximum(x[cat_group_index]))\n if sum(x[cat_group_index]) > 1\n ties = findall(x[cat_group_index] .== 1)\n _x = zeros(length(x[cat_group_index]))\n winner = rand(ties,1)[1]\n _x[winner] = 1\n x[cat_group_index] = _x\n end\n else\n x[cat_group_index] = [round(clamp(x[cat_group_index][1],0,1))]\n end\n end\n return x\nend","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"Let’s look at a few simple examples to see how this function works. Firstly, consider the case of perturbing a single element:","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"x = X[:,1]\nx[1] = 1.1\nx","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"10-element Vector{Float64}:\n 1.1\n 0.0\n 0.0\n 0.0\n 1.0\n 0.0\n 0.0\n 0.0\n 1.0\n 1.85","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"The reconstructed one-hot-encoded vector will look like this:","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"reconstruct_cat_encoding(x)","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"10-element Vector{Float64}:\n 1.0\n 0.0\n 0.0\n 0.0\n 1.0\n 0.0\n 0.0\n 0.0\n 1.0\n 1.85","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"Next, consider the case of perturbing multiple elements:","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"x[2] = 1.1\nx[3] = -1.2\nx","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"10-element Vector{Float64}:\n 1.0\n 1.1\n -1.2\n 0.0\n 1.0\n 0.0\n 0.0\n 0.0\n 1.0\n 1.85","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"The reconstructed one-hot-encoded vector will look like this:","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"reconstruct_cat_encoding(x)","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"10-element Vector{Float64}:\n 0.0\n 1.0\n 0.0\n 0.0\n 1.0\n 0.0\n 0.0\n 0.0\n 1.0\n 1.85","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"Finally, let’s introduce a tie:","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"x[1] = 1.0\nx","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"10-element Vector{Float64}:\n 1.0\n 1.0\n 0.0\n 0.0\n 1.0\n 0.0\n 0.0\n 0.0\n 1.0\n 1.85","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"The reconstructed one-hot-encoded vector will look like this:","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"reconstruct_cat_encoding(x)","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"10-element Vector{Float64}:\n 0.0\n 1.0\n 0.0\n 0.0\n 1.0\n 0.0\n 0.0\n 0.0\n 1.0\n 1.85","category":"page"},{"location":"extensions/neurotree/","page":"NeuroTrees","title":"NeuroTrees","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"extensions/neurotree/#[NeuroTreeModels.jl](https://evovest.github.io/NeuroTreeModels.jl/dev/)","page":"NeuroTrees","title":"NeuroTreeModels.jl","text":"","category":"section"},{"location":"extensions/neurotree/","page":"NeuroTrees","title":"NeuroTrees","text":"NeuroTreeModels.jl is a package that provides a framework for training differentiable tree-based models. This is relevant to the work on counterfactual explanations (CE), which often assumes that the underlying black-box model is differentiable with respect to its input. The literature on CE therefore regularly focuses exclusively on explaining deep learning models. This is at odds with the fact that the literature also typically focuses on tabular data, which is often best modeled by tree-based models (Grinsztajn, Oyallon, and Varoquaux 2022). The extension for NeuroTreeModels.jl provides a way to bridge this gap by allowing users to apply existing gradient-based CE methods to differentiable tree-based models.","category":"page"},{"location":"extensions/neurotree/","page":"NeuroTrees","title":"NeuroTrees","text":"warning: Experimental Feature\nPlease note that this extension is still experimental. Neither the behaviour of differentiable tree-based models nor their interplay with counterfactual explanations is well understood at this point. If you encounter any issues, please report them to the package maintainers. Your feedback is highly appreciated.Please also note that this extension is only tested on Julia 1.9 and higher, due to compatibility issues.","category":"page"},{"location":"extensions/neurotree/#Example","page":"NeuroTrees","title":"Example","text":"","category":"section"},{"location":"extensions/neurotree/","page":"NeuroTrees","title":"NeuroTrees","text":"The extension will be loaded automatically when loading the NeuroTreeModels package (assuming the CounterfactualExplanations package is also loaded).","category":"page"},{"location":"extensions/neurotree/","page":"NeuroTrees","title":"NeuroTrees","text":"using NeuroTreeModels","category":"page"},{"location":"extensions/neurotree/","page":"NeuroTrees","title":"NeuroTrees","text":"Next, we will fit a NeuroTree model to the moons dataset using our standard package API for doing so.","category":"page"},{"location":"extensions/neurotree/","page":"NeuroTrees","title":"NeuroTrees","text":"# Fit model to data:\ndata = CounterfactualData(load_moons()...)\nM = fit_model(\n data, :NeuroTree; \n depth=2, lr=5e-2, nrounds=50, batchsize=10\n)","category":"page"},{"location":"extensions/neurotree/","page":"NeuroTrees","title":"NeuroTrees","text":"NeuroTreeExt.NeuroTreeModel(NeuroTreeRegressor(loss = mlogloss, …), :classification_multi, NeuroTreeModels.NeuroTreeModel{NeuroTreeModels.MLogLoss, Chain{Tuple{BatchNorm{typeof(identity), Vector{Float32}, Float32, Vector{Float32}}, NeuroTreeModels.StackTree}}}(NeuroTreeModels.MLogLoss, Chain(BatchNorm(2, active=false), NeuroTreeModels.StackTree(NeuroTree[NeuroTree{Matrix{Float32}, Vector{Float32}, Array{Float32, 3}}(Float32[1.8824593 -0.28222033; -2.680499 0.67347014; … ; -1.0722864 1.3651229; -2.0926774 1.63557], Float32[-3.4070241, 4.545113, 1.0882677, -0.3497498, -2.766766, 1.9072449, -0.9736261, 3.9750721, 1.726214, 3.7279263 … -0.0664266, -0.4214582, -2.3816268, -3.1371245, 0.76548636, 2.636373, 2.4558601, 0.893434, -1.9484522, 4.793434], Float32[3.44271 -6.334693 -0.6308845 3.385659; -3.4316056 6.297003 0.7254221 -3.3283486;;; -3.7011054 -0.17596768 0.15429471 2.270125; 3.4926674 0.026218029 -0.19753197 -2.2337704;;; 1.1795454 -4.315231 0.28486454 1.9995956; -0.9651108 4.0999455 -0.05312265 -1.8039354;;; … ;;; 2.5076811 -0.46358463 -3.5438805 0.0686823; -2.592356 0.47884527 3.781507 -0.022692114;;; -0.59115165 -3.234046 0.09896194 2.375202; 0.5592871 3.3082843 -0.014032216 -2.1876256;;; 2.039389 -0.10134532 2.6637273 -4.999703; -2.0289893 0.3368772 -2.5739825 5.069934], tanh)])), Dict{Symbol, Any}(:feature_names => [:x1, :x2], :nrounds => 50, :device => :cpu)))","category":"page"},{"location":"extensions/neurotree/","page":"NeuroTrees","title":"NeuroTrees","text":"Finally, we select a factual instance and generate a counterfactual explanation for it using the generic gradient-based CE method.","category":"page"},{"location":"extensions/neurotree/","page":"NeuroTrees","title":"NeuroTrees","text":"# Select a factual instance:\ntarget = 1\nfactual = 0\nchosen = rand(findall(predict_label(M, data) .== factual))\nx = select_factual(data, chosen)\n\n# Generate counterfactual explanation:\nη = 0.01\ngenerator = GenericGenerator(; opt=Descent(η), λ=0.01)\nconv = CounterfactualExplanations.Convergence.DecisionThresholdConvergence(;\n decision_threshold=0.9, max_iter=100\n)\nce = generate_counterfactual(x, target, data, M, generator; convergence=conv)\nplot(ce, alpha=0.1)","category":"page"},{"location":"extensions/neurotree/","page":"NeuroTrees","title":"NeuroTrees","text":"(Image: )","category":"page"},{"location":"extensions/neurotree/#References","page":"NeuroTrees","title":"References","text":"","category":"section"},{"location":"extensions/neurotree/","page":"NeuroTrees","title":"NeuroTrees","text":"Grinsztajn, Léo, Edouard Oyallon, and Gaël Varoquaux. 2022. “Why Do Tree-Based Models Still Outperform Deep Learning on Tabular Data?” https://arxiv.org/abs/2207.08815.","category":"page"}] +[{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"tutorials/models/#Handling-Models","page":"Handling Models","title":"Handling Models","text":"","category":"section"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"The typical use-case for Counterfactual Explanations and Algorithmic Recourse is as follows: users have trained some supervised model that is not inherently interpretable and are looking for a way to explain it. In this tutorial, we will see how pre-trained models can be used with this package.","category":"page"},{"location":"tutorials/models/#Models-trained-in-Flux.jl","page":"Handling Models","title":"Models trained in Flux.jl","text":"","category":"section"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"We will train a simple binary classifier in Flux.jl on the popular Moons dataset:","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"n = 500\ndata = TaijaData.load_moons(n)\ncounterfactual_data = DataPreprocessing.CounterfactualData(data...)\nX = counterfactual_data.X\ny = counterfactual_data.y\nplt = plot()\nscatter!(counterfactual_data)","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"(Image: )","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"The following code chunk sets up a Deep Neural Network for the task at hand:","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"data = Flux.DataLoader((X,y),batchsize=1)\ninput_dim = size(X,1)\nn_hidden = 32\nactivation = relu\noutput_dim = 2\nnn = Chain(\n Dense(input_dim, n_hidden, activation),\n Dropout(0.1),\n Dense(n_hidden, output_dim)\n)\nloss(yhat, y) = Flux.Losses.logitcrossentropy(nn(yhat), y)","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"Next, we fit the network to the data:","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"using Flux.Optimise: update!, Adam\nopt = Adam()\nepochs = 100\navg_loss(data) = mean(map(d -> loss(d[1],d[2]), data))\nshow_every = epochs/5\n# Training:\nfor epoch = 1:epochs\n for d in data\n gs = gradient(Flux.params(nn)) do\n l = loss(d...)\n end\n update!(opt, Flux.params(nn), gs)\n end\n if epoch % show_every == 0\n println(\"Epoch \" * string(epoch))\n @show avg_loss(data)\n end\nend","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"Epoch 20\navg_loss(data) = 0.1407434f0\nEpoch 40\navg_loss(data) = 0.11345118f0\nEpoch 60\navg_loss(data) = 0.046319224f0\nEpoch 80\navg_loss(data) = 0.011847609f0\nEpoch 100\navg_loss(data) = 0.007242911f0","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"To prepare the fitted model for use with our package, we need to wrap it inside a container. For plain-vanilla models trained in Flux.jl, the corresponding constructor is called FluxModel. There is also a separate constructor called FluxEnsemble, which applies to Deep Ensembles. Deep Ensembles are a popular approach to approximate Bayesian Deep Learning and have been shown to generate good predictive uncertainty estimates (Lakshminarayanan, Pritzel, and Blundell 2016).","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"The appropriate API call to wrap our simple network in a container follows below:","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"M = FluxModel(nn)","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"FluxModel(Chain(Dense(2 => 32, relu), Dropout(0.1, active=false), Dense(32 => 2)), :classification_binary)","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"The likelihood function of the output variable is automatically inferred from the data. The generic plot() method can be called on the model and data to visualise the results:","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"plot(M, counterfactual_data)","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"(Image: )","category":"page"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"Our model M is now ready for use with the package.","category":"page"},{"location":"tutorials/models/#References","page":"Handling Models","title":"References","text":"","category":"section"},{"location":"tutorials/models/","page":"Handling Models","title":"Handling Models","text":"Lakshminarayanan, Balaji, Alexander Pritzel, and Charles Blundell. 2016. “Simple and Scalable Predictive Uncertainty Estimation Using Deep Ensembles.” https://arxiv.org/abs/1612.01474.","category":"page"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"explanation/generators/dice/#DiCEGenerator","page":"DiCE","title":"DiCEGenerator","text":"","category":"section"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"The DiCEGenerator can be used to generate multiple diverse counterfactuals for a single factual.","category":"page"},{"location":"explanation/generators/dice/#Description","page":"DiCE","title":"Description","text":"","category":"section"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"Counterfactual Explanations are not unique and there are therefore many different ways through which valid counterfactuals can be generated. In the context of Algorithmic Recourse this can be leveraged to offer individuals not one, but possibly many different ways to change a negative outcome into a positive one. One might argue that it makes sense for those different options to be as diverse as possible. This idea is at the core of DiCE, a counterfactual generator introduce by Mothilal, Sharma, and Tan (2020) that generate a diverse set of counterfactual explanations.","category":"page"},{"location":"explanation/generators/dice/#Defining-Diversity","page":"DiCE","title":"Defining Diversity","text":"","category":"section"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"To ensure that the generated counterfactuals are diverse, Mothilal, Sharma, and Tan (2020) add a diversity constraint to the counterfactual search objective. In particular, diversity is explicitly proxied via Determinantal Point Processes (DDP).","category":"page"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"We can implement DDP in Julia as follows:[1]","category":"page"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"using LinearAlgebra\nfunction ddp_diversity(X::AbstractArray{<:Real, 3})\n xs = eachslice(X, dims = ndims(X))\n K = [1/(1 + norm(x .- y)) for x in xs, y in xs]\n return det(K)\nend","category":"page"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"Below we generate some random points in mathbbR^2 and apply gradient ascent on this function evaluated at the whole array of points. As we can see in the animation below, the points are sent away from each other. In other words, diversity across the array of points increases as we ascend the ddp_diversity function.","category":"page"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"lims = 5\nN = 5\nX = rand(2,1,N)\nT = 50\nη = 0.1\nanim = @animate for t in 1:T\n X .+= gradient(ddp_diversity, X)[1]\n Z = reshape(X,2,N)\n scatter(\n Z[1,:],Z[2,:],ms=25, \n xlims=(-lims,lims),ylims=(-lims,lims),\n label=\"\",colour=1:N,\n size=(500,500),\n title=\"Diverse Counterfactuals\"\n )\nend\ngif(anim, joinpath(www_path, \"dice_intro.gif\"))","category":"page"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"(Image: )","category":"page"},{"location":"explanation/generators/dice/#Usage","page":"DiCE","title":"Usage","text":"","category":"section"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"The approach can be used in our package as follows:","category":"page"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"generator = DiCEGenerator()\nconv = CounterfactualExplanations.Convergence.GeneratorConditionsConvergence()\nce = generate_counterfactual(\n x, target, counterfactual_data, M, generator; \n num_counterfactuals=5, convergence=conv\n)\nplot(ce)","category":"page"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"(Image: )","category":"page"},{"location":"explanation/generators/dice/#Effect-of-Penalty","page":"DiCE","title":"Effect of Penalty","text":"","category":"section"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"Λ₂ = [0.1, 1.0, 5.0]\nces = []\nn_cf = 5\nusing Flux\nfor λ₂ ∈ Λ₂ \n λ = [0.00, λ₂]\n generator = DiCEGenerator(λ=λ)\n ces = vcat(\n ces...,\n generate_counterfactual(\n x, target, counterfactual_data, M, generator; \n num_counterfactuals=n_cf, convergence=conv\n )\n )\nend","category":"page"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"The figure below shows the resulting counterfactual paths. As expected, the resulting counterfactuals are more dispersed across the feature domain for higher choices of lambda_2","category":"page"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"(Image: )","category":"page"},{"location":"explanation/generators/dice/#References","page":"DiCE","title":"References","text":"","category":"section"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"Mothilal, Ramaravind K, Amit Sharma, and Chenhao Tan. 2020. “Explaining Machine Learning Classifiers Through Diverse Counterfactual Explanations.” In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 607–17. https://doi.org/10.1145/3351095.3372850.","category":"page"},{"location":"explanation/generators/dice/","page":"DiCE","title":"DiCE","text":"[1] With thanks to the respondents on Discourse","category":"page"},{"location":"how_to_guides/","page":"Overview","title":"Overview","text":"CurrentModule = CounterfactualExplanations","category":"page"},{"location":"how_to_guides/#How-To-Guides","page":"Overview","title":"How-To Guides","text":"","category":"section"},{"location":"how_to_guides/","page":"Overview","title":"Overview","text":"In this section, you will find a series of how-to-guides that showcase specific use cases of counterfactual explanations (CE).","category":"page"},{"location":"how_to_guides/","page":"Overview","title":"Overview","text":"How-to guides are directions that take the reader through the steps required to solve a real-world problem. How-to guides are goal-oriented.— Diátaxis","category":"page"},{"location":"how_to_guides/","page":"Overview","title":"Overview","text":"In other words, you come here because you may have some particular problem in mind, would like to see how it can be solved using CE and then most likely head off again 🫡.","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"tutorials/generators/#Handling-Generators","page":"Handling Generators","title":"Handling Generators","text":"","category":"section"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"Generating Counterfactual Explanations can be seen as a generative modelling task because it involves generating samples in the input space: x sim mathcalX. In this tutorial, we will introduce how Counterfactual GradientBasedGenerators are used. They are discussed in more detail in the explanatory section of the documentation.","category":"page"},{"location":"tutorials/generators/#Composable-Generators","page":"Handling Generators","title":"Composable Generators","text":"","category":"section"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"warning: Breaking Changes Expected\nWork on this feature is still in its very early stages and breaking changes should be expected. ","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"One of the key objectives for this package is Composability. It turns out that many of the various counterfactual generators that have been proposed in the literature, essentially do the same thing: they optimize an objective function. Formally we have,","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"\nbeginaligned\nmathbfs^prime = arg min_mathbfs^prime in mathcalS left textyloss(M(f(mathbfs^prime))y^*)+ lambda textcost(f(mathbfs^prime)) right \nendaligned \n qquad(1)","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"where textyloss denotes the main loss function and textcost is a penalty term (Altmeyer et al. 2023).","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"Without going into further detail here, the important thing to mention is that Equation 1 very closely describes how counterfactual search is actually implemented in the package. In other words, all off-the-shelf generators currently implemented work with that same objective. They just vary in the way that penalties are defined, for example. This gives rise to an interesting idea:","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"Why not compose generators that combine ideas from different off-the-shelf generators?","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"The GradientBasedGenerator class provides a straightforward way to do this, without requiring users to build custom GradientBasedGenerators from scratch. It can be instantiated as follows:","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"generator = GradientBasedGenerator()","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"By default, this creates a generator that simply performs gradient descent without any penalties. To modify the behaviour of the generator, you can define the counterfactual search objective function using the @objective macro:","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"@objective(generator, logitbinarycrossentropy + 0.1distance_l2 + 1.0ddp_diversity)","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"Here we have essentially created a version of the DiCEGenerator:","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"ce = generate_counterfactual(x, target, counterfactual_data, M, generator; num_counterfactuals=5)\nplot(ce)","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"(Image: )","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"Multiple macros can be chained using Chains.jl making it easy to create entirely new flavours of counterfactual generators. The following generator, for example, combines ideas from DiCE (Mothilal, Sharma, and Tan 2020) and REVISE (Joshi et al. 2019):","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"@chain generator begin\n @objective logitcrossentropy + 1.0ddp_diversity # DiCE (Mothilal et al. 2020)\n @with_optimiser Flux.Adam(0.1) \n @search_latent_space # REVISE (Joshi et al. 2019)\nend","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"Let’s take this generator to our MNIST dataset and generate a counterfactual explanation for turning a 0 into a 8.","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"(Image: )","category":"page"},{"location":"tutorials/generators/#Off-the-Shelf-Generators","page":"Handling Generators","title":"Off-the-Shelf Generators","text":"","category":"section"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"Off-the-shelf generators are just default recipes for counterfactual generators. Currently, the following off-the-shelf counterfactual generators are implemented in the package:","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"generator_catalogue","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"Dict{Symbol, Any} with 11 entries:\n :gravitational => GravitationalGenerator\n :growing_spheres => GrowingSpheresGenerator\n :revise => REVISEGenerator\n :clue => CLUEGenerator\n :probe => ProbeGenerator\n :dice => DiCEGenerator\n :feature_tweak => FeatureTweakGenerator\n :claproar => ClaPROARGenerator\n :wachter => WachterGenerator\n :generic => GenericGenerator\n :greedy => GreedyGenerator","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"To specify the type of generator you want to use, you can simply instantiate it:","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"# Search:\ngenerator = GenericGenerator()\nce = generate_counterfactual(x, target, counterfactual_data, M, generator)\nplot(ce)","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"(Image: )","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"We generally make an effort to follow the literature as closely as possible when implementing off-the-shelf generators.","category":"page"},{"location":"tutorials/generators/#References","page":"Handling Generators","title":"References","text":"","category":"section"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"Altmeyer, Patrick, Giovan Angela, Aleksander Buszydlik, Karol Dobiczek, Arie van Deursen, and Cynthia Liem. 2023. “Endogenous Macrodynamics in Algorithmic Recourse.” In First IEEE Conference on Secure and Trustworthy Machine Learning. https://doi.org/10.1109/satml54575.2023.00036.","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"Joshi, Shalmali, Oluwasanmi Koyejo, Warut Vijitbenjaronk, Been Kim, and Joydeep Ghosh. 2019. “Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems.” https://arxiv.org/abs/1907.09615.","category":"page"},{"location":"tutorials/generators/","page":"Handling Generators","title":"Handling Generators","text":"Mothilal, Ramaravind K, Amit Sharma, and Chenhao Tan. 2020. “Explaining Machine Learning Classifiers Through Diverse Counterfactual Explanations.” In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 607–17. https://doi.org/10.1145/3351095.3372850.","category":"page"},{"location":"tutorials/simple_example/","page":"Simple Example","title":"Simple Example","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"tutorials/simple_example/#Simple-Example","page":"Simple Example","title":"Simple Example","text":"","category":"section"},{"location":"tutorials/simple_example/","page":"Simple Example","title":"Simple Example","text":"In this tutorial, we will go through a simple example involving synthetic data and a generic counterfactual generator.","category":"page"},{"location":"tutorials/simple_example/#Data-and-Classifier","page":"Simple Example","title":"Data and Classifier","text":"","category":"section"},{"location":"tutorials/simple_example/","page":"Simple Example","title":"Simple Example","text":"Below we generate some linearly separable data and fit a simple MLP classifier with batch normalization to it. For more information on generating data and models, refer to the Handling Data and Handling Models tutorials respectively.","category":"page"},{"location":"tutorials/simple_example/","page":"Simple Example","title":"Simple Example","text":"# Counteractual data and model:\nflux_training_params.batchsize = 10\ndata = TaijaData.load_linearly_separable()\ncounterfactual_data = DataPreprocessing.CounterfactualData(data...)\ncounterfactual_data.standardize = true\nM = fit_model(counterfactual_data, :MLP, batch_norm=true)","category":"page"},{"location":"tutorials/simple_example/#Counterfactual-Search","page":"Simple Example","title":"Counterfactual Search","text":"","category":"section"},{"location":"tutorials/simple_example/","page":"Simple Example","title":"Simple Example","text":"Next, determine a target and factual class for our counterfactual search and select a random factual instance to explain.","category":"page"},{"location":"tutorials/simple_example/","page":"Simple Example","title":"Simple Example","text":"target = 2\nfactual = 1\nchosen = rand(findall(predict_label(M, counterfactual_data) .== factual))\nx = select_factual(counterfactual_data, chosen)","category":"page"},{"location":"tutorials/simple_example/","page":"Simple Example","title":"Simple Example","text":"Finally, we generate and visualize the generated counterfactual:","category":"page"},{"location":"tutorials/simple_example/","page":"Simple Example","title":"Simple Example","text":"# Search:\ngenerator = WachterGenerator()\nce = generate_counterfactual(x, target, counterfactual_data, M, generator)\nplot(ce)","category":"page"},{"location":"tutorials/simple_example/","page":"Simple Example","title":"Simple Example","text":"(Image: )","category":"page"},{"location":"explanation/optimisers/jsma/","page":"JSMA","title":"JSMA","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"explanation/optimisers/jsma/#Jacobian-based-Saliency-Map-Attack","page":"JSMA","title":"Jacobian-based Saliency Map Attack","text":"","category":"section"},{"location":"explanation/optimisers/jsma/","page":"JSMA","title":"JSMA","text":"To search counterfactuals, Schut et al. (2021) propose to use a Jacobian-Based Saliency Map Attack (JSMA) inspired by the literature on adversarial attacks. It works by moving in the direction of the most salient feature at a fixed step size in each iteration. Schut et al. (2021) use this optimisation rule in the context of Bayesian classifiers and demonstrate good results in terms of plausibility — how realistic counterfactuals are — and redundancy — how sparse the proposed feature changes are.","category":"page"},{"location":"explanation/optimisers/jsma/#JSMADescent","page":"JSMA","title":"JSMADescent","text":"","category":"section"},{"location":"explanation/optimisers/jsma/","page":"JSMA","title":"JSMA","text":"To implement this approach in a reusable manner, we have added JSMA as a Flux optimiser. In particular, we have added a class JSMADescent<:Flux.Optimise.AbstractOptimiser, for which we have overloaded the Flux.Optimise.apply! method. This makes it possible to reuse JSMADescent as an optimiser in composable generators.","category":"page"},{"location":"explanation/optimisers/jsma/","page":"JSMA","title":"JSMA","text":"The optimiser can be used with with any generator as follows:","category":"page"},{"location":"explanation/optimisers/jsma/","page":"JSMA","title":"JSMA","text":"using CounterfactualExplanations.Generators: JSMADescent\ngenerator = GenericGenerator() |>\n gen -> @with_optimiser(gen,JSMADescent(;η=0.1))\nce = generate_counterfactual(x, target, counterfactual_data, M, generator)","category":"page"},{"location":"explanation/optimisers/jsma/","page":"JSMA","title":"JSMA","text":"The figure below compares the resulting counterfactual search outcome to the corresponding outcome with generic Descent.","category":"page"},{"location":"explanation/optimisers/jsma/","page":"JSMA","title":"JSMA","text":"plot(p1,p2,size=(1000,400))","category":"page"},{"location":"explanation/optimisers/jsma/","page":"JSMA","title":"JSMA","text":"(Image: )","category":"page"},{"location":"explanation/optimisers/jsma/","page":"JSMA","title":"JSMA","text":"Schut, Lisa, Oscar Key, Rory Mc Grath, Luca Costabello, Bogdan Sacaleanu, Yarin Gal, et al. 2021. “Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertainties.” In International Conference on Artificial Intelligence and Statistics, 1756–64. PMLR.","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"tutorials/data_catalogue/#Data-Catalogue","page":"Data Catalogue","title":"Data Catalogue","text":"","category":"section"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"To allow researchers and practitioners to test and compare counterfactual generators, the TAIJA environment includes the package TaijaData.jl which comes with pre-processed synthetic and real-world benchmark datasets from different domains. This page explains how to use TaijaData.jl in tandem with CounterfactualExplanations.jl.","category":"page"},{"location":"tutorials/data_catalogue/#Synthetic-Data","page":"Data Catalogue","title":"Synthetic Data","text":"","category":"section"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"The following dictionary can be used to inspect the available methods to generate synthetic datasets where the key indicates the name of the data and the value is the corresponding method:","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"TaijaData.data_catalogue[:synthetic]","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"Dict{Symbol, Function} with 6 entries:\n :overlapping => load_overlapping\n :linearly_separable => load_linearly_separable\n :blobs => load_blobs\n :moons => load_moons\n :circles => load_circles\n :multi_class => load_multi_class","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"The chart below shows the generated data using default parameters:","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"plts = []\n_height = 200\n_n = length(keys(data_catalogue[:synthetic]))\nfor (key, fun) in data_catalogue[:synthetic]\n data = fun()\n counterfactual_data = DataPreprocessing.CounterfactualData(data...)\n plt = plot()\n scatter!(counterfactual_data, title=key)\n plts = [plts..., plt]\nend\nplot(plts..., size=(_n * _height, _height), layout=(1, _n))","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"(Image: )","category":"page"},{"location":"tutorials/data_catalogue/#Real-World-Data","page":"Data Catalogue","title":"Real-World Data","text":"","category":"section"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"As for real-world data, the same dictionary can be used to inspect the available data from different domains.","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"TaijaData.data_catalogue[:tabular]","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"Dict{Symbol, Function} with 5 entries:\n :german_credit => load_german_credit\n :california_housing => load_california_housing\n :credit_default => load_credit_default\n :adult => load_uci_adult\n :gmsc => load_gmsc","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"TaijaData.data_catalogue[:vision]","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"Dict{Symbol, Function} with 3 entries:\n :fashion_mnist => load_fashion_mnist\n :mnist => load_mnist\n :cifar_10 => load_cifar_10","category":"page"},{"location":"tutorials/data_catalogue/#Loading-Data","page":"Data Catalogue","title":"Loading Data","text":"","category":"section"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"To load or generate any of the datasets listed above, you can just use the corresponding method, for example:","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"data = TaijaData.load_linearly_separable()\ncounterfactual_data = DataPreprocessing.CounterfactualData(data...)","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"Optionally, you can specify how many samples you want to generate like so:","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"n = 100\ndata = TaijaData.load_overlapping(n)\ncounterfactual_data = DataPreprocessing.CounterfactualData(data...)","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"This also applies to real-world datasets, which by default are loaded in their entirety. If n is supplied, the dataset will be randomly undersampled:","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"data = TaijaData.load_mnist(n)\ncounterfactual_data = DataPreprocessing.CounterfactualData(data...)","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"The undersampled dataset is automatically balanced:","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"sum(counterfactual_data.y; dims=2)","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"10×1 Matrix{Int64}:\n 10\n 10\n 10\n 10\n 10\n 10\n 10\n 10\n 10\n 10","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"We can also use a helper function to split the data into train and test sets:","category":"page"},{"location":"tutorials/data_catalogue/","page":"Data Catalogue","title":"Data Catalogue","text":"train_data, test_data = \n CounterfactualExplanations.DataPreprocessing.train_test_split(counterfactual_data)","category":"page"},{"location":"how_to_guides/custom_generators/","page":"... add custom generators","title":"... add custom generators","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"how_to_guides/custom_generators/#How-to-add-Custom-Generators","page":"... add custom generators","title":"How to add Custom Generators","text":"","category":"section"},{"location":"how_to_guides/custom_generators/","page":"... add custom generators","title":"... add custom generators","text":"As we will see in this short tutorial, building custom counterfactual generators is straightforward. We hope that this will facilitate contributions through the community.","category":"page"},{"location":"how_to_guides/custom_generators/#Generic-generator-with-dropout","page":"... add custom generators","title":"Generic generator with dropout","text":"","category":"section"},{"location":"how_to_guides/custom_generators/","page":"... add custom generators","title":"... add custom generators","text":"To illustrate how custom generators can be implemented we will consider a simple example of a generator that extends the functionality of our GenericGenerator. We have noted elsewhere that the effectiveness of counterfactual explanations depends to some degree on the quality of the fitted model. Another, perhaps trivial, thing to note is that counterfactual explanations are not unique: there are potentially many valid counterfactual paths. One interesting (or silly) idea following these two observations might be to introduce some form of regularization in the counterfactual search. For example, we could use dropout to randomly switch features on and off in each iteration. Without dwelling further on the usefulness of this idea, let us see how it can be implemented.","category":"page"},{"location":"how_to_guides/custom_generators/","page":"... add custom generators","title":"... add custom generators","text":"The first code chunk below implements two important steps: 1) create an abstract subtype of the AbstractGradientBasedGenerator and 2) create a constructor similar to the GenericConstructor, but with one additional field for the probability of dropout.","category":"page"},{"location":"how_to_guides/custom_generators/","page":"... add custom generators","title":"... add custom generators","text":"# Abstract suptype:\nabstract type AbstractDropoutGenerator <: AbstractGradientBasedGenerator end\n\n# Constructor:\nstruct DropoutGenerator <: AbstractDropoutGenerator\n loss::Function # loss function\n penalty::Function\n λ::AbstractFloat # strength of penalty\n latent_space::Bool\n opt::Any # optimizer\n generative_model_params::NamedTuple\n p_dropout::AbstractFloat # dropout rate\nend\n\n# Instantiate:\ngenerator = DropoutGenerator(\n Flux.logitbinarycrossentropy,\n CounterfactualExplanations.Objectives.distance_l1,\n 0.1,\n false,\n Flux.Optimise.Descent(0.1),\n (;),\n 0.5,\n)","category":"page"},{"location":"how_to_guides/custom_generators/","page":"... add custom generators","title":"... add custom generators","text":"Next, we define how feature perturbations are generated for our dropout generator: in particular, we extend the relevant function through a method that implemented the dropout logic.","category":"page"},{"location":"how_to_guides/custom_generators/","page":"... add custom generators","title":"... add custom generators","text":"using CounterfactualExplanations.Generators\nusing StatsBase\nfunction Generators.generate_perturbations(\n generator::AbstractDropoutGenerator, \n ce::CounterfactualExplanation\n)\n s′ = deepcopy(ce.s′)\n new_s′ = Generators.propose_state(generator, ce)\n Δs′ = new_s′ - s′ # gradient step\n\n # Dropout:\n set_to_zero = sample(\n 1:length(Δs′),\n Int(round(generator.p_dropout*length(Δs′))),\n replace=false\n )\n Δs′[set_to_zero] .= 0\n return Δs′\nend","category":"page"},{"location":"how_to_guides/custom_generators/","page":"... add custom generators","title":"... add custom generators","text":"Finally, we proceed to generate counterfactuals in the same way we always do:","category":"page"},{"location":"how_to_guides/custom_generators/","page":"... add custom generators","title":"... add custom generators","text":"# Data and Classifier:\nM = fit_model(counterfactual_data, :DeepEnsemble)\n\n# Factual and Target:\nyhat = predict_label(M, counterfactual_data)\ntarget = 2 # target label\ncandidates = findall(vec(yhat) .!= target)\nchosen = rand(candidates)\nx = select_factual(counterfactual_data, chosen)\n\n# Counterfactual search:\nce = generate_counterfactual(\n x, target, counterfactual_data, M, generator;\n num_counterfactuals=5)\n\nplot(ce)","category":"page"},{"location":"how_to_guides/custom_generators/","page":"... add custom generators","title":"... add custom generators","text":"(Image: )","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"tutorials/parallelization/#Parallelization","page":"Parallelization","title":"Parallelization","text":"","category":"section"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"Version 0.1.15 adds support for parallelization through multi-processing. Currently, the only available backend for parallelization is MPI.jl.","category":"page"},{"location":"tutorials/parallelization/#Available-functions","page":"Parallelization","title":"Available functions","text":"","category":"section"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"Parallelization is only available for certain functions. To check if a function is parallelizable, you can use parallelizable function:","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"using CounterfactualExplanations.Evaluation: evaluate, benchmark\nprintln(parallelizable(generate_counterfactual))\nprintln(parallelizable(evaluate))\nprintln(parallelizable(predict_label))","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"true\ntrue\nfalse","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"In the following, we will generate multiple counterfactuals and evaluate them in parallel:","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"chosen = rand(findall(predict_label(M, counterfactual_data) .== factual), 1000)\nxs = select_factual(counterfactual_data, chosen)","category":"page"},{"location":"tutorials/parallelization/#Multi-threading","page":"Parallelization","title":"Multi-threading","text":"","category":"section"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"We first instantiate an ThreadParallelizer object:","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"parallelizer = ThreadsParallelizer()","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"ThreadsParallelizer()","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"To generate counterfactuals in parallel, we use the parallelize function:","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"ces = @with_parallelizer parallelizer begin\n generate_counterfactual(\n xs,\n target,\n counterfactual_data,\n M,\n generator\n )\nend","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"Generating counterfactuals ... 0%| | ETA: 0:01:29 (89.14 ms/it)Generating counterfactuals ... 100%|███████| Time: 0:00:01 ( 1.59 ms/it)\n\n1000-element Vector{AbstractCounterfactualExplanation}:\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 6 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 6 steps.\n ⋮\n CounterfactualExplanation\nConvergence: ✅ after 9 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 6 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"To evaluate counterfactuals in parallel, we again use the parallelize function:","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"@with_parallelizer parallelizer evaluate(ces)","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"Evaluating counterfactuals ... 0%| | ETA: 0:07:03 ( 0.42 s/it)Evaluating counterfactuals ... 100%|███████| Time: 0:00:00 ( 0.86 ms/it)\n\n1000-element Vector{Any}:\n Vector[[1.0], Float32[3.2939816], [0.0]]\n Vector[[1.0], Float32[3.019046], [0.0]]\n Vector[[1.0], Float32[3.701171], [0.0]]\n Vector[[1.0], Float32[2.5611918], [0.0]]\n Vector[[1.0], Float32[2.9027307], [0.0]]\n Vector[[1.0], Float32[3.7893882], [0.0]]\n Vector[[1.0], Float32[3.5026522], [0.0]]\n Vector[[1.0], Float32[3.6317568], [0.0]]\n Vector[[1.0], Float32[3.084984], [0.0]]\n Vector[[1.0], Float32[3.2268934], [0.0]]\n Vector[[1.0], Float32[2.834947], [0.0]]\n Vector[[1.0], Float32[3.656587], [0.0]]\n Vector[[1.0], Float32[2.5985842], [0.0]]\n ⋮\n Vector[[1.0], Float32[4.067538], [0.0]]\n Vector[[1.0], Float32[3.02231], [0.0]]\n Vector[[1.0], Float32[2.748292], [0.0]]\n Vector[[1.0], Float32[2.9483426], [0.0]]\n Vector[[1.0], Float32[3.066149], [0.0]]\n Vector[[1.0], Float32[3.6018147], [0.0]]\n Vector[[1.0], Float32[3.0138078], [0.0]]\n Vector[[1.0], Float32[3.5724509], [0.0]]\n Vector[[1.0], Float32[3.117551], [0.0]]\n Vector[[1.0], Float32[2.9670508], [0.0]]\n Vector[[1.0], Float32[3.4107168], [0.0]]\n Vector[[1.0], Float32[3.0252533], [0.0]]","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"Benchmarks can also be run with parallelization by specifying parallelizer argument:","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"# Models:\nbmk = benchmark(counterfactual_data; parallelizer = parallelizer)","category":"page"},{"location":"tutorials/parallelization/#MPI","page":"Parallelization","title":"MPI","text":"","category":"section"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"note: Note\nTo use MPI, you need to have MPI installed on your machine. Running the following code straight from a running Julia session will work if you have MPI installed on your machine, but it will be run on a single process. To execute the code on multiple processes, you need to run it from the command line with mpirun or mpiexec. For example, to run a script on 4 processes, you can run the following command from the command line:\n\nmpiexecjl --project -n 4 julia -e 'include(\"docs/src/srcipts/mpi.jl\")'For more information, see MPI.jl. ","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"We first instantiate an MPIParallelizer object:","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"import MPI\nMPI.Init()\nparallelizer = MPIParallelizer(MPI.COMM_WORLD; threaded=true)","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"Precompiling MPIExt\n ✓ TaijaParallel → MPIExt\n 1 dependency successfully precompiled in 3 seconds. 255 already precompiled.\n[ Info: Precompiling MPIExt [48137b38-b316-530b-be8a-261f41e68c23]\n┌ Warning: Module TaijaParallel with build ID ffffffff-ffff-ffff-0001-2d458926c256 is missing from the cache.\n│ This may mean TaijaParallel [bf1c2c22-5e42-4e78-8b6b-92e6c673eeb0] does not support precompilation but is imported by a module that does.\n└ @ Base loading.jl:1948\n[ Info: Skipping precompilation since __precompile__(false). Importing MPIExt [48137b38-b316-530b-be8a-261f41e68c23].\n[ Info: Using `MPI.jl` for multi-processing.\n\nRunning on 1 processes.\n\nMPIExt.MPIParallelizer(MPI.Comm(1140850688), 0, 1, nothing, true)","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"To generate counterfactuals in parallel, we use the parallelize function:","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"ces = @with_parallelizer parallelizer begin\n generate_counterfactual(\n xs,\n target,\n counterfactual_data,\n M,\n generator\n )\nend","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"Generating counterfactuals ... 9%|▋ | ETA: 0:00:01 ( 1.15 ms/it)Generating counterfactuals ... 19%|█▍ | ETA: 0:00:01 ( 1.07 ms/it)Generating counterfactuals ... 29%|██ | ETA: 0:00:01 ( 1.10 ms/it)Generating counterfactuals ... 39%|██▊ | ETA: 0:00:01 ( 1.08 ms/it)Generating counterfactuals ... 49%|███▍ | ETA: 0:00:01 ( 1.08 ms/it)Generating counterfactuals ... 59%|████▏ | ETA: 0:00:00 ( 1.08 ms/it)Generating counterfactuals ... 69%|████▊ | ETA: 0:00:00 ( 1.08 ms/it)Generating counterfactuals ... 79%|█████▌ | ETA: 0:00:00 ( 1.07 ms/it)Generating counterfactuals ... 89%|██████▎| ETA: 0:00:00 ( 1.07 ms/it)Generating counterfactuals ... 99%|██████▉| ETA: 0:00:00 ( 1.06 ms/it)Generating counterfactuals ... 100%|███████| Time: 0:00:01 ( 1.06 ms/it)\n\n1000-element Vector{AbstractCounterfactualExplanation}:\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 6 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 6 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 6 steps.\n ⋮\n CounterfactualExplanation\nConvergence: ✅ after 9 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 6 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"To evaluate counterfactuals in parallel, we again use the parallelize function:","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"@with_parallelizer parallelizer evaluate(ces)","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"1000-element Vector{Any}:\n Vector[[1.0], Float32[3.0941274], [0.0]]\n Vector[[1.0], Float32[3.0894346], [0.0]]\n Vector[[1.0], Float32[3.5737448], [0.0]]\n Vector[[1.0], Float32[2.6201036], [0.0]]\n Vector[[1.0], Float32[2.8519764], [0.0]]\n Vector[[1.0], Float32[3.7762523], [0.0]]\n Vector[[1.0], Float32[3.4162796], [0.0]]\n Vector[[1.0], Float32[3.6095932], [0.0]]\n Vector[[1.0], Float32[3.1347957], [0.0]]\n Vector[[1.0], Float32[3.0313473], [0.0]]\n Vector[[1.0], Float32[2.7612567], [0.0]]\n Vector[[1.0], Float32[3.6191392], [0.0]]\n Vector[[1.0], Float32[2.610616], [0.0]]\n ⋮\n Vector[[1.0], Float32[4.0844703], [0.0]]\n Vector[[1.0], Float32[3.0119], [0.0]]\n Vector[[1.0], Float32[2.4461186], [0.0]]\n Vector[[1.0], Float32[3.071967], [0.0]]\n Vector[[1.0], Float32[3.132917], [0.0]]\n Vector[[1.0], Float32[3.5403214], [0.0]]\n Vector[[1.0], Float32[3.0588162], [0.0]]\n Vector[[1.0], Float32[3.5600657], [0.0]]\n Vector[[1.0], Float32[3.2205954], [0.0]]\n Vector[[1.0], Float32[2.896302], [0.0]]\n Vector[[1.0], Float32[3.2603998], [0.0]]\n Vector[[1.0], Float32[3.1369917], [0.0]]","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"tip: Tip\nNote that parallelizable processes can be supplied as input to the macro either as a block or directly as an expression.","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"Benchmarks can also be run with parallelization by specifying parallelizer argument:","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"# Models:\nbmk = benchmark(counterfactual_data; parallelizer = parallelizer)","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"The following code snippet shows a complete example script that uses MPI for running a benchmark in parallel:","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"using CounterfactualExplanations\nusing CounterfactualExplanations.Evaluation: benchmark\nusing CounterfactualExplanations.Models\nimport MPI\n\nMPI.Init()\n\ndata = TaijaData.load_linearly_separable()\ncounterfactual_data = DataPreprocessing.CounterfactualData(data...)\nM = fit_model(counterfactual_data, :Linear)\nfactual = 1\ntarget = 2\nchosen = rand(findall(predict_label(M, counterfactual_data) .== factual), 100)\nxs = select_factual(counterfactual_data, chosen)\ngenerator = GenericGenerator()\n\nparallelizer = MPIParallelizer(MPI.COMM_WORLD)\n\nbmk = benchmark(counterfactual_data; parallelizer=parallelizer)\n\nMPI.Finalize()","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"The file can be executed from the command line as follows:","category":"page"},{"location":"tutorials/parallelization/","page":"Parallelization","title":"Parallelization","text":"mpiexecjl --project -n 4 julia -e 'include(\"docs/src/srcipts/mpi.jl\")'","category":"page"},{"location":"release-notes/","page":"Release Notes","title":"Release Notes","text":"EditURL = \"https://github.com/juliatrustworthyai/CounterfactualExplanations.jl/blob/master/CHANGELOG.md\"","category":"page"},{"location":"release-notes/#Changelog","page":"Release Notes","title":"Changelog","text":"","category":"section"},{"location":"release-notes/","page":"Release Notes","title":"Release Notes","text":"All notable changes to this project will be documented in this file.","category":"page"},{"location":"release-notes/","page":"Release Notes","title":"Release Notes","text":"The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.","category":"page"},{"location":"release-notes/","page":"Release Notes","title":"Release Notes","text":"Note: We try to adhere to these practices as of version v1.1.1.","category":"page"},{"location":"release-notes/#Version-[v1.1.4](https://github.com/juliatrustworthyai/CounterfactualExplanations.jl/releases/tag/v1.1.4)-2024-04-24","page":"Release Notes","title":"Version v1.1.4 - 2024-04-24","text":"","category":"section"},{"location":"release-notes/#Changed","page":"Release Notes","title":"Changed","text":"","category":"section"},{"location":"release-notes/","page":"Release Notes","title":"Release Notes","text":"Refactors the encodings and decodings such that it is now more streamlined. Instead of conditional statements, encodings are now dispatched on the type of a new unifying data.input_encoder field. #432\nRefactors the check for redundancy. This is now based on the convergence type and done right before the counterfactual search begins, if not redundant. #432","category":"page"},{"location":"release-notes/#Version-[v1.1.3](https://github.com/juliatrustworthyai/CounterfactualExplanations.jl/releases/tag/v1.1.3)-2024-04-17","page":"Release Notes","title":"Version v1.1.3 - 2024-04-17","text":"","category":"section"},{"location":"release-notes/#Added","page":"Release Notes","title":"Added","text":"","category":"section"},{"location":"release-notes/","page":"Release Notes","title":"Release Notes","text":"Adds a section on Convergence to the documentation, Changelog.jl functionality and a few doc tests. #429","category":"page"},{"location":"release-notes/#Changed-2","page":"Release Notes","title":"Changed","text":"","category":"section"},{"location":"release-notes/","page":"Release Notes","title":"Release Notes","text":"Changes style of taking gradients for the counterfactual search from implicit to explicit. #430\nRemoved all implicit imports. #430","category":"page"},{"location":"release-notes/#Removed","page":"Release Notes","title":"Removed","text":"","category":"section"},{"location":"release-notes/","page":"Release Notes","title":"Release Notes","text":"Removed CUDA.jl dependency, because redundant. #430\nRemoved Parameters.jl dependency, because redundant. #430","category":"page"},{"location":"release-notes/#Version-[v1.1.2](https://github.com/juliatrustworthyai/CounterfactualExplanations.jl/releases/tag/v1.1.2)-2024-04-16","page":"Release Notes","title":"Version v1.1.2 - 2024-04-16","text":"","category":"section"},{"location":"release-notes/#Changed-3","page":"Release Notes","title":"Changed","text":"","category":"section"},{"location":"release-notes/","page":"Release Notes","title":"Release Notes","text":"Replaces the GIF in the README and introduction of docs for a static image. ","category":"page"},{"location":"release-notes/#Version-[v1.1.1](https://github.com/juliatrustworthyai/CounterfactualExplanations.jl/releases/tag/v1.1.1)-2024-04-15","page":"Release Notes","title":"Version v1.1.1 - 2024-04-15","text":"","category":"section"},{"location":"release-notes/#Added-2","page":"Release Notes","title":"Added","text":"","category":"section"},{"location":"release-notes/","page":"Release Notes","title":"Release Notes","text":"Added tests for LaplaceRedux extension. Bumped upper compat bound for LaplaceRedux.jl. #428","category":"page"},{"location":"tutorials/","page":"Overview","title":"Overview","text":"CurrentModule = CounterfactualExplanation","category":"page"},{"location":"tutorials/#Tutorials","page":"Overview","title":"Tutorials","text":"","category":"section"},{"location":"tutorials/","page":"Overview","title":"Overview","text":"In this section, you will find a series of tutorials that should help you gain a basic understanding of Conformal Prediction and how to apply it in Julia using this package.","category":"page"},{"location":"tutorials/","page":"Overview","title":"Overview","text":"Tutorials are lessons that take the reader by the hand through a series of steps to complete a project of some kind. Tutorials are learning-oriented.— Diátaxis","category":"page"},{"location":"tutorials/","page":"Overview","title":"Overview","text":"In other words, you come here because you are new to this topic and are looking for a first peek at the methodology and code 🫣.","category":"page"},{"location":"explanation/generators/growing_spheres/","page":"GrowingSpheres","title":"GrowingSpheres","text":"CurrentModule = CounterfactualExplanations","category":"page"},{"location":"explanation/generators/growing_spheres/#GrowingSpheres","page":"GrowingSpheres","title":"GrowingSpheres","text":"","category":"section"},{"location":"explanation/generators/growing_spheres/","page":"GrowingSpheres","title":"GrowingSpheres","text":"Growing Spheres refers to the generator introduced by Laugel et al. (2017). Our implementation takes inspiration from the CARLA library.","category":"page"},{"location":"explanation/generators/growing_spheres/#Principle-of-the-Proposed-Approach","page":"GrowingSpheres","title":"Principle of the Proposed Approach","text":"","category":"section"},{"location":"explanation/generators/growing_spheres/","page":"GrowingSpheres","title":"GrowingSpheres","text":"In order to interpret a prediction through comparison, the Growing Spheres algorithm focuses on finding an observation belonging to the other class and answers the question: “Considering an observation and a classifier, what is the minimal change we need to apply in order to change the prediction of this observation?”. This problem is similar to inverse classification but applied to interpretability.","category":"page"},{"location":"explanation/generators/growing_spheres/","page":"GrowingSpheres","title":"GrowingSpheres","text":"Explaining how to change a prediction can help the user understand what the model considers as locally important. The Growing Spheres approach provides insights into the classifier’s behavior without claiming any causal knowledge. It differs from other interpretability approaches and is not concerned with the global behavior of the model. Instead, it aims to provide local insights into the classifier’s decision-making process.","category":"page"},{"location":"explanation/generators/growing_spheres/","page":"GrowingSpheres","title":"GrowingSpheres","text":"The algorithm finds the closest “ennemy” observation, which is an observation classified into the other class than the input observation. The final explanation is the difference vector between the input observation and the ennemy.","category":"page"},{"location":"explanation/generators/growing_spheres/#Finding-the-Closest-Ennemy","page":"GrowingSpheres","title":"Finding the Closest Ennemy","text":"","category":"section"},{"location":"explanation/generators/growing_spheres/","page":"GrowingSpheres","title":"GrowingSpheres","text":"The algorithm solves the following minimization problem to find the closest ennemy:","category":"page"},{"location":"explanation/generators/growing_spheres/","page":"GrowingSpheres","title":"GrowingSpheres","text":"e^* = arg min_e in X c(x e) f(e) neq f(x) ","category":"page"},{"location":"explanation/generators/growing_spheres/","page":"GrowingSpheres","title":"GrowingSpheres","text":"The cost function c(x, e) is defined as:","category":"page"},{"location":"explanation/generators/growing_spheres/","page":"GrowingSpheres","title":"GrowingSpheres","text":"c(x e) = x - e_2 + gamma x - e_0","category":"page"},{"location":"explanation/generators/growing_spheres/","page":"GrowingSpheres","title":"GrowingSpheres","text":"where ||.||_2 is the Euclidean norm and ||.||_0 is the sparsity measure. The weight gamma balances the importance of sparsity in the cost function.","category":"page"},{"location":"explanation/generators/growing_spheres/","page":"GrowingSpheres","title":"GrowingSpheres","text":"To approximate the solution, the Growing Spheres algorithm uses a two-step heuristic approach. The first step is the Generation phase, where observations are generated in spherical layers around the input observation. The second step is the Feature Selection phase, where the generated observation with the smallest change in each feature is selected.","category":"page"},{"location":"explanation/generators/growing_spheres/#Example","page":"GrowingSpheres","title":"Example","text":"","category":"section"},{"location":"explanation/generators/growing_spheres/","page":"GrowingSpheres","title":"GrowingSpheres","text":"generator = GrowingSpheresGenerator()\nM = fit_model(counterfactual_data, :DeepEnsemble)\nce = generate_counterfactual(\n x, target, counterfactual_data, M, generator)\nplot(ce)","category":"page"},{"location":"explanation/generators/growing_spheres/","page":"GrowingSpheres","title":"GrowingSpheres","text":"(Image: )","category":"page"},{"location":"explanation/generators/growing_spheres/#References","page":"GrowingSpheres","title":"References","text":"","category":"section"},{"location":"explanation/generators/growing_spheres/","page":"GrowingSpheres","title":"GrowingSpheres","text":"Laugel, Thibault, Marie-Jeanne Lesot, Christophe Marsala, Xavier Renard, and Marcin Detyniecki. 2017. “Inverse Classification for Comparison-Based Interpretability in Machine Learning.” arXiv. https://doi.org/10.48550/arXiv.1712.08443.","category":"page"},{"location":"contribute/#Contribute","page":"🛠 Contribute","title":"🛠 Contribute","text":"","category":"section"},{"location":"contribute/","page":"🛠 Contribute","title":"🛠 Contribute","text":"Contributions of any kind are very much welcome! Take a look at the issue to see what things we are currently working on. If you have an idea for a new feature or want to report a bug, please open a new issue.","category":"page"},{"location":"contribute/#Development","page":"🛠 Contribute","title":"Development","text":"","category":"section"},{"location":"contribute/","page":"🛠 Contribute","title":"🛠 Contribute","text":"If your looking to contribute code, it may be helpful to check out the Explanation section of the docs.","category":"page"},{"location":"contribute/#Testing","page":"🛠 Contribute","title":"Testing","text":"","category":"section"},{"location":"contribute/","page":"🛠 Contribute","title":"🛠 Contribute","text":"Please always make sure to add tests for any new features or changes.","category":"page"},{"location":"contribute/#Documentation","page":"🛠 Contribute","title":"Documentation","text":"","category":"section"},{"location":"contribute/","page":"🛠 Contribute","title":"🛠 Contribute","text":"If you add new features or change existing ones, please make sure to update the documentation accordingly. The documentation is written in Documenter.jl and is located in the docs/src folder.","category":"page"},{"location":"contribute/#Log-Changes","page":"🛠 Contribute","title":"Log Changes","text":"","category":"section"},{"location":"contribute/","page":"🛠 Contribute","title":"🛠 Contribute","text":"As of version 1.1.1, we have tried to be more stringent about logging changes. Please make sure to add a note to the CHANGELOG.md file for any changes you make. It is sufficient to add a note under the Unreleased section.","category":"page"},{"location":"contribute/#General-Pointers","page":"🛠 Contribute","title":"General Pointers","text":"","category":"section"},{"location":"contribute/","page":"🛠 Contribute","title":"🛠 Contribute","text":"There are also some general pointers for people looking to contribute to any of our Taija packages here.","category":"page"},{"location":"contribute/","page":"🛠 Contribute","title":"🛠 Contribute","text":"Please follow the SciML ColPrac guide.","category":"page"},{"location":"explanation/generators/clap_roar/","page":"ClaPROAR","title":"ClaPROAR","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"explanation/generators/clap_roar/#ClaPROARGenerator","page":"ClaPROAR","title":"ClaPROARGenerator","text":"","category":"section"},{"location":"explanation/generators/clap_roar/","page":"ClaPROAR","title":"ClaPROAR","text":"The ClaPROARGenerator was introduced in Altmeyer et al. (2023).","category":"page"},{"location":"explanation/generators/clap_roar/#Description","page":"ClaPROAR","title":"Description","text":"","category":"section"},{"location":"explanation/generators/clap_roar/","page":"ClaPROAR","title":"ClaPROAR","text":"The acronym Clap stands for classifier-preserving. The approach is loosely inspired by ROAR (Upadhyay, Joshi, and Lakkaraju 2021). Altmeyer et al. (2023) propose to explicitly penalize the loss incurred by the classifer when evaluated on the counterfactual x^prime at given parameter values. Formally, we have","category":"page"},{"location":"explanation/generators/clap_roar/","page":"ClaPROAR","title":"ClaPROAR","text":"beginaligned\ntextextcost(f(mathbfs^prime)) = l(M(f(mathbfs^prime))y^prime)\nendaligned","category":"page"},{"location":"explanation/generators/clap_roar/","page":"ClaPROAR","title":"ClaPROAR","text":"for each counterfactual k where l denotes the loss function used to train M. This approach is based on the intuition that (endogenous) model shifts will be triggered by counterfactuals that increase classifier loss (Altmeyer et al. 2023).","category":"page"},{"location":"explanation/generators/clap_roar/#Usage","page":"ClaPROAR","title":"Usage","text":"","category":"section"},{"location":"explanation/generators/clap_roar/","page":"ClaPROAR","title":"ClaPROAR","text":"The approach can be used in our package as follows:","category":"page"},{"location":"explanation/generators/clap_roar/","page":"ClaPROAR","title":"ClaPROAR","text":"generator = ClaPROARGenerator()\nce = generate_counterfactual(x, target, counterfactual_data, M, generator)\nplot(ce)","category":"page"},{"location":"explanation/generators/clap_roar/","page":"ClaPROAR","title":"ClaPROAR","text":"(Image: )","category":"page"},{"location":"explanation/generators/clap_roar/#Comparison-to-GenericGenerator","page":"ClaPROAR","title":"Comparison to GenericGenerator","text":"","category":"section"},{"location":"explanation/generators/clap_roar/","page":"ClaPROAR","title":"ClaPROAR","text":"The figure below compares the outcome for the GenericGenerator and the ClaPROARGenerator.","category":"page"},{"location":"explanation/generators/clap_roar/","page":"ClaPROAR","title":"ClaPROAR","text":"(Image: )","category":"page"},{"location":"explanation/generators/clap_roar/#References","page":"ClaPROAR","title":"References","text":"","category":"section"},{"location":"explanation/generators/clap_roar/","page":"ClaPROAR","title":"ClaPROAR","text":"Altmeyer, Patrick, Giovan Angela, Aleksander Buszydlik, Karol Dobiczek, Arie van Deursen, and Cynthia Liem. 2023. “Endogenous Macrodynamics in Algorithmic Recourse.” In First IEEE Conference on Secure and Trustworthy Machine Learning. https://doi.org/10.1109/satml54575.2023.00036.","category":"page"},{"location":"explanation/generators/clap_roar/","page":"ClaPROAR","title":"ClaPROAR","text":"Upadhyay, Sohini, Shalmali Joshi, and Himabindu Lakkaraju. 2021. “Towards Robust and Reliable Algorithmic Recourse.” https://arxiv.org/abs/2102.13620.","category":"page"},{"location":"explanation/architecture/","page":"Package Architecture","title":"Package Architecture","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"explanation/architecture/#Package-Architecture","page":"Package Architecture","title":"Package Architecture","text":"","category":"section"},{"location":"explanation/architecture/","page":"Package Architecture","title":"Package Architecture","text":"The diagram below provides an overview of the package architecture. It is built around two core modules that are designed to be as extensible as possible through dispatch: 1) Models is concerned with making any arbitrary model compatible with the package; 2) Generators is used to implement arbitrary counterfactual search algorithms.[1]","category":"page"},{"location":"explanation/architecture/","page":"Package Architecture","title":"Package Architecture","text":"The core function of the package, generate_counterfactual, uses an instance of type AbstractFittedModel produced by the Models module and an instance of type AbstractGenerator produced by the Generators module.","category":"page"},{"location":"explanation/architecture/","page":"Package Architecture","title":"Package Architecture","text":"Metapackages from the Taija ecosystem provide additional functionality such as datasets, language interoperability, parallelization, and plotting. The CounterfactualExplanations package is designed to be used in conjunction with these metapackages, but can also be used as a standalone package.","category":"page"},{"location":"explanation/architecture/","page":"Package Architecture","title":"Package Architecture","text":"(Image: )","category":"page"},{"location":"explanation/architecture/","page":"Package Architecture","title":"Package Architecture","text":"[1] We have made an effort to keep the code base a flexible and extensible as possible, but cannot guarantee at this point that any counterfactual generator can be implemented without further adaptation.","category":"page"},{"location":"explanation/","page":"Overview","title":"Overview","text":"CurrentModule = CounterfactualExplanations","category":"page"},{"location":"explanation/#Explanation","page":"Overview","title":"Explanation","text":"","category":"section"},{"location":"explanation/","page":"Overview","title":"Overview","text":"In this section you will find detailed explanations about the methodology and code.","category":"page"},{"location":"explanation/","page":"Overview","title":"Overview","text":"Explanation clarifies, deepens and broadens the reader’s understanding of a subject.— Diátaxis","category":"page"},{"location":"explanation/","page":"Overview","title":"Overview","text":"In other words, you come here because you are interested in understanding how all of this actually works 🤓.","category":"page"},{"location":"extensions/laplace_redux/","page":"LaplaceRedux","title":"LaplaceRedux","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"extensions/laplace_redux/#[LaplaceRedux.jl](https://github.com/JuliaTrustworthyAI/LaplaceRedux.jl)","page":"LaplaceRedux","title":"LaplaceRedux.jl","text":"","category":"section"},{"location":"extensions/laplace_redux/","page":"LaplaceRedux","title":"LaplaceRedux","text":"LaplaceRedux.jl is one of Taija’s own packages that provides a framework for Effortless Bayesian Deep Learning through Laplace Approximation for Flux.jl neural networks. The methodology was first proposed by Immer, Korzepa, and Bauer (2020) and implemented in Python by Daxberger et al. (2021). This is relevant to the work on counterfactual explanations (CE), because research has shown that counterfactual explanations for Bayesian models are typically more plausible, because Bayesian models are able to capture the uncertainty in the data (Schut et al. 2021).","category":"page"},{"location":"extensions/laplace_redux/","page":"LaplaceRedux","title":"LaplaceRedux","text":"tip: Read More\nTo learn more about Laplace Redux, head over to the official documentation.","category":"page"},{"location":"extensions/laplace_redux/#Example","page":"LaplaceRedux","title":"Example","text":"","category":"section"},{"location":"extensions/laplace_redux/","page":"LaplaceRedux","title":"LaplaceRedux","text":"The extension will be loaded automatically when loading the LaplaceRedux package (assuming the CounterfactualExplanations package is also loaded).","category":"page"},{"location":"extensions/laplace_redux/","page":"LaplaceRedux","title":"LaplaceRedux","text":"using LaplaceRedux","category":"page"},{"location":"extensions/laplace_redux/","page":"LaplaceRedux","title":"LaplaceRedux","text":"Next, we will fit a neural network with Laplace Approximation to the moons dataset using our standard package API for doing so. By default, the Bayesian prior is optimized through empirical Bayes using the LaplaceRedux package.","category":"page"},{"location":"extensions/laplace_redux/","page":"LaplaceRedux","title":"LaplaceRedux","text":"# Fit model to data:\ndata = CounterfactualData(load_moons()...)\nM = fit_model(data, :LaplaceRedux; n_hidden=16)","category":"page"},{"location":"extensions/laplace_redux/","page":"LaplaceRedux","title":"LaplaceRedux","text":"LaplaceReduxExt.LaplaceReduxModel(Laplace(Chain(Dense(2 => 16, relu), Dense(16 => 2)), :classification, :all, nothing, :full, LaplaceRedux.Curvature.GGN(Chain(Dense(2 => 16, relu), Dense(16 => 2)), :classification, Flux.Losses.logitcrossentropy, Array{Float32}[[-1.3098596 0.59241515; 0.91760206 0.02950162; … ; -0.018356863 0.12850936; -0.5381665 -0.7872097], [-0.2581085, -0.90997887, -0.5418944, -0.23735572, 0.81020063, -0.3033359, -0.47902864, -0.6432098, -0.038013518, 0.028280666, 0.009903266, -0.8796683, 0.41090682, 0.011093224, -0.1580453, 0.7911349], [3.092321 -2.4660816 … -0.3446268 -1.465249; -2.9468734 3.167357 … 0.31758657 1.7140366], [-0.3107697, 0.31076983]], 1.0, :all, nothing), 1.0, 0.0, Float32[-1.3098596, 0.91760206, 0.5239727, -1.1579771, -0.851813, -1.9411169, 0.47409698, 0.6679365, 0.8944433, 0.663116 … -0.3172857, 0.15530388, 1.3264753, -0.3506721, -0.3446268, 0.31758657, -1.465249, 1.7140366, -0.3107697, 0.31076983], [0.10530027048093525 0.0 … 0.0 0.0; 0.0 0.10530027048093525 … 0.0 0.0; … ; 0.0 0.0 … 0.10530027048093525 0.0; 0.0 0.0 … 0.0 0.10530027048093525], [0.10066431429751965 0.0 … -0.030656783425475176 0.030656334963944154; 0.0 20.93513766443357 … -2.3185940232360736 2.3185965484008193; … ; -0.030656783425475176 -2.3185940232360736 … 1.0101450999063672 -1.0101448118057204; 0.030656334963944154 2.3185965484008193 … -1.0101448118057204 1.0101451389641771], [1.1006643142975197 0.0 … -0.030656783425475176 0.030656334963944154; 0.0 21.93513766443357 … -2.3185940232360736 2.3185965484008193; … ; -0.030656783425475176 -2.3185940232360736 … 2.0101450999063672 -1.0101448118057204; 0.030656334963944154 2.3185965484008193 … -1.0101448118057204 2.010145138964177], [0.9412600568016627 0.003106911671721699 … 0.003743740333409532 -0.003743452315572739; 0.003106912946573237 0.6539263732691709 … 0.0030385955287734246 -0.0030390041204196414; … ; 0.0037437406323562283 0.003038591829991259 … 0.9624905710233649 0.03750911813897676; -0.0037434526145225856 -0.0030390004216833593 … 0.03750911813898124 0.9624905774453485], 82, 250, 2, 997.8087484836578), :classification_multi)","category":"page"},{"location":"extensions/laplace_redux/","page":"LaplaceRedux","title":"LaplaceRedux","text":"Finally, we select a factual instance and generate a counterfactual explanation for it using the generic gradient-based CE method.","category":"page"},{"location":"extensions/laplace_redux/","page":"LaplaceRedux","title":"LaplaceRedux","text":"# Select a factual instance:\ntarget = 1\nfactual = 0\nchosen = rand(findall(predict_label(M, data) .== factual))\nx = select_factual(data, chosen)\n\n# Generate counterfactual explanation:\nη = 0.01\ngenerator = GenericGenerator(; opt=Descent(η), λ=0.01)\nconv = CounterfactualExplanations.Convergence.DecisionThresholdConvergence(;\n decision_threshold=0.9, max_iter=100\n)\nce = generate_counterfactual(x, target, data, M, generator; convergence=conv)\nplot(ce, alpha=0.1)","category":"page"},{"location":"extensions/laplace_redux/","page":"LaplaceRedux","title":"LaplaceRedux","text":"(Image: )","category":"page"},{"location":"extensions/laplace_redux/#References","page":"LaplaceRedux","title":"References","text":"","category":"section"},{"location":"extensions/laplace_redux/","page":"LaplaceRedux","title":"LaplaceRedux","text":"Daxberger, Erik, Agustinus Kristiadi, Alexander Immer, Runa Eschenhagen, Matthias Bauer, and Philipp Hennig. 2021. “Laplace Redux-Effortless Bayesian Deep Learning.” Advances in Neural Information Processing Systems 34.","category":"page"},{"location":"extensions/laplace_redux/","page":"LaplaceRedux","title":"LaplaceRedux","text":"Immer, Alexander, Maciej Korzepa, and Matthias Bauer. 2020. “Improving Predictions of Bayesian Neural Networks via Local Linearization.” https://arxiv.org/abs/2008.08400.","category":"page"},{"location":"extensions/laplace_redux/","page":"LaplaceRedux","title":"LaplaceRedux","text":"Schut, Lisa, Oscar Key, Rory Mc Grath, Luca Costabello, Bogdan Sacaleanu, Yarin Gal, et al. 2021. “Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertainties.” In International Conference on Artificial Intelligence and Statistics, 1756–64. PMLR.","category":"page"},{"location":"contribute/performance/","page":"-","title":"-","text":"Random.seed!(42)\n# Counteractual data and model:\ndata = TaijaData.load_linearly_separable()\ncounterfactual_data = DataPreprocessing.CounterfactualData(data...)\nM = fit_model(counterfactual_data, :Linear)\ntarget = 2\nfactual = 1\nchosen = rand(findall(predict_label(M, counterfactual_data) .== factual))\nx = select_factual(counterfactual_data, chosen)\n\n# Search:\ngenerator = GenericGenerator()\nce = generate_counterfactual(x, target, counterfactual_data, M, generator)","category":"page"},{"location":"contribute/performance/","page":"-","title":"-","text":"data_large = TaijaData.load_linearly_separable(100000)\ncounterfactual_data_large = DataPreprocessing.CounterfactualData(data_large...)","category":"page"},{"location":"contribute/performance/","page":"-","title":"-","text":"@time generate_counterfactual(x, target, counterfactual_data, M, generator)","category":"page"},{"location":"contribute/performance/","page":"-","title":"-","text":"@time generate_counterfactual(x, target, counterfactual_data_large, M, generator)","category":"page"},{"location":"explanation/generators/feature_tweak/","page":"FeatureTweak","title":"FeatureTweak","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"explanation/generators/feature_tweak/#FeatureTweakGenerator","page":"FeatureTweak","title":"FeatureTweakGenerator","text":"","category":"section"},{"location":"explanation/generators/feature_tweak/","page":"FeatureTweak","title":"FeatureTweak","text":"Feature Tweak refers to the generator introduced by Tolomei et al. (2017). Our implementation takes inspiration from the featureTweakPy library.","category":"page"},{"location":"explanation/generators/feature_tweak/#Description","page":"FeatureTweak","title":"Description","text":"","category":"section"},{"location":"explanation/generators/feature_tweak/","page":"FeatureTweak","title":"FeatureTweak","text":"Feature Tweak is a powerful recourse algorithm for ensembles of tree-based classifiers such as random forests. Though the problem of understanding how an input to an ensemble model could be transformed in such a way that the model changes its original prediction has been proven to be NP-hard (Tolomei et al. 2017), Feature Tweak provides an algorithm that manages to tractably solve this problem in multiple real-world applications. An example of a problem Feature Tweak is able to efficiently solve, explored in depth in Tolomei et al. (2017) is the problem of transforming an advertisement that has been classified by the ensemble model as a low-quality advertisement to a high-quality one through small changes to its features. With the help of Feature Tweak, advertisers can both learn about the reasons a particular ad was marked to have a low quality, as well as receive actionable suggestions about how to convert a low-quality ad into a high-quality one.","category":"page"},{"location":"explanation/generators/feature_tweak/","page":"FeatureTweak","title":"FeatureTweak","text":"Though Feature Tweak is a powerful way of avoiding brute-force search in an exponential search space, it does not come without disadvantages. The primary limitations of the approach are that it’s currently only applicable to tree-based classifiers and works only in the setting of binary classification. Another problem is that though the algorithm avoids exponential-time search, it is often still computationally expensive. The algorithm may be improved in the future to tackle all of these shortcomings.","category":"page"},{"location":"explanation/generators/feature_tweak/","page":"FeatureTweak","title":"FeatureTweak","text":"The following equation displays how a true negative instance x can be transformed into a positively predicted instance x’. To be more precise, x’ is the best possible transformation among all transformations **x***, computed with a cost function δ.","category":"page"},{"location":"explanation/generators/feature_tweak/","page":"FeatureTweak","title":"FeatureTweak","text":"beginaligned\nmathbfx^prime = arg_mathbfx^* min delta(mathbfx mathbfx^*) hatf(mathbfx) = -1 wedge hatf(mathbfx^*) = +1 \nendaligned","category":"page"},{"location":"explanation/generators/feature_tweak/#Example","page":"FeatureTweak","title":"Example","text":"","category":"section"},{"location":"explanation/generators/feature_tweak/","page":"FeatureTweak","title":"FeatureTweak","text":"In this example we apply the Feature Tweak algorithm to a decision tree and a random forest trained on the moons dataset. We first load the data and fit the models:","category":"page"},{"location":"explanation/generators/feature_tweak/","page":"FeatureTweak","title":"FeatureTweak","text":"n = 500\ncounterfactual_data = CounterfactualData(TaijaData.load_moons(n)...)\n\n# Classifiers\ndecision_tree = CounterfactualExplanations.Models.fit_model(\n counterfactual_data, :DecisionTree; max_depth=5, min_samples_leaf=3\n)\nforest = Models.fit_model(counterfactual_data, :RandomForest)","category":"page"},{"location":"explanation/generators/feature_tweak/","page":"FeatureTweak","title":"FeatureTweak","text":"Next, we select a point to explain and a target class to transform the point to. We then search for counterfactuals using the FeatureTweakGenerator:","category":"page"},{"location":"explanation/generators/feature_tweak/","page":"FeatureTweak","title":"FeatureTweak","text":"# Select a point to explain:\nx = float32.([1, -0.5])[:,:]\nfactual = Models.predict_label(forest, x)\ntarget = counterfactual_data.y_levels[findall(counterfactual_data.y_levels != factual)][1]\n\n# Search for counterfactuals:\ngenerator = FeatureTweakGenerator(ϵ=0.1)\ntree_counterfactual = generate_counterfactual(\n x, target, counterfactual_data, decision_tree, generator\n)\nforest_counterfactual = generate_counterfactual(\n x, target, counterfactual_data, forest, generator\n)","category":"page"},{"location":"explanation/generators/feature_tweak/","page":"FeatureTweak","title":"FeatureTweak","text":"The resulting counterfactuals are shown below:","category":"page"},{"location":"explanation/generators/feature_tweak/","page":"FeatureTweak","title":"FeatureTweak","text":"p1 = plot(\n tree_counterfactual;\n colorbar=false,\n title=\"Decision Tree\",\n)\n\np2 = plot(\n forest_counterfactual; title=\"Random Forest\",\n colorbar=false,\n)\n\ndisplay(plot(p1, p2; size=(800, 400)))","category":"page"},{"location":"explanation/generators/feature_tweak/","page":"FeatureTweak","title":"FeatureTweak","text":"(Image: )","category":"page"},{"location":"explanation/generators/feature_tweak/#References","page":"FeatureTweak","title":"References","text":"","category":"section"},{"location":"explanation/generators/feature_tweak/","page":"FeatureTweak","title":"FeatureTweak","text":"Tolomei, Gabriele, Fabrizio Silvestri, Andrew Haines, and Mounia Lalmas. 2017. “Interpretable Predictions of Tree-Based Ensembles via Actionable Feature Tweaking.” In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 465–74. https://doi.org/10.1145/3097983.3098039.","category":"page"},{"location":"explanation/generators/clue/","page":"CLUE","title":"CLUE","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"explanation/generators/clue/#CLUEGenerator","page":"CLUE","title":"CLUEGenerator","text":"","category":"section"},{"location":"explanation/generators/clue/","page":"CLUE","title":"CLUE","text":"In this tutorial, we introduce the CLUEGenerator, a counterfactual generator based on the Counterfactual Latent Uncertainty Explanations (CLUE) method proposed by Antorán et al. (2020).","category":"page"},{"location":"explanation/generators/clue/#Description","page":"CLUE","title":"Description","text":"","category":"section"},{"location":"explanation/generators/clue/","page":"CLUE","title":"CLUE","text":"The CLUEGenerator leverages differentiable probabilistic models, such as Bayesian Neural Networks (BNNs), to estimate uncertainty in predictions. It aims to provide interpretable counterfactual explanations by identifying input patterns that lead to predictive uncertainty. The generator utilizes a latent variable framework and employs a decoder from a variational autoencoder (VAE) to generate counterfactual samples in latent space.","category":"page"},{"location":"explanation/generators/clue/","page":"CLUE","title":"CLUE","text":"The CLUE algorithm minimizes a loss function that combines uncertainty estimates and the distance between the generated counterfactual and the original input. By optimizing this loss function iteratively, the CLUEGenerator generates counterfactuals that are similar to the original observation but assigned low uncertainty.","category":"page"},{"location":"explanation/generators/clue/","page":"CLUE","title":"CLUE","text":"The formula for predictive entropy is as follow:","category":"page"},{"location":"explanation/generators/clue/","page":"CLUE","title":"CLUE","text":"beginaligned\nH(y^*x^* D) = - sum_k=1^K p(y^*=c_kx^* D) log p(y^*=c_kx^* D)\nendaligned","category":"page"},{"location":"explanation/generators/clue/#Usage","page":"CLUE","title":"Usage","text":"","category":"section"},{"location":"explanation/generators/clue/","page":"CLUE","title":"CLUE","text":"While using one must keep in mind that the CLUE algorithim is meant to find a more robust datapoint of the same class, using CLUE generator without any additional penalties/losses will mean that it is not a counterfactual generator. The generated result will be of the same class as the original input, but a more robust datapoint.","category":"page"},{"location":"explanation/generators/clue/","page":"CLUE","title":"CLUE","text":"CLUE works best for BNN’s. The CLUEGenerator can be used with any differentiable probabilistic model, but the results may not be as good as with BNNs.","category":"page"},{"location":"explanation/generators/clue/","page":"CLUE","title":"CLUE","text":"The CLUEGenerator can be used in the following manner:","category":"page"},{"location":"explanation/generators/clue/","page":"CLUE","title":"CLUE","text":"generator = CLUEGenerator()\nM = fit_model(counterfactual_data, :DeepEnsemble)\nconv = CounterfactualExplanations.Convergence.MaxIterConvergence(max_iter=1000)\nce = generate_counterfactual(\n x, target, counterfactual_data, M, generator;\n convergence=conv)\nplot(ce)","category":"page"},{"location":"explanation/generators/clue/","page":"CLUE","title":"CLUE","text":"(Image: )","category":"page"},{"location":"explanation/generators/clue/","page":"CLUE","title":"CLUE","text":"Extra: The CLUE generator can also be used upon already having achieved a counterfactual with a different generator. In this case, you can use CLUE and make the counterfactual more robust.","category":"page"},{"location":"explanation/generators/clue/","page":"CLUE","title":"CLUE","text":"Note: The above documentation is based on the information provided in the CLUE paper. Please refer to the original paper for more detailed explanations and implementation specifics.","category":"page"},{"location":"explanation/generators/clue/#References","page":"CLUE","title":"References","text":"","category":"section"},{"location":"explanation/generators/clue/","page":"CLUE","title":"CLUE","text":"Antorán, Javier, Umang Bhatt, Tameem Adel, Adrian Weller, and José Miguel Hernández-Lobato. 2020. “Getting a Clue: A Method for Explaining Uncertainty Estimates.” https://arxiv.org/abs/2006.06848.","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"explanation/generators/revise/#REVISEGenerator","page":"REVISE","title":"REVISEGenerator","text":"","category":"section"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"REVISE is a Latent Space generator introduced by Joshi et al. (2019).","category":"page"},{"location":"explanation/generators/revise/#Description","page":"REVISE","title":"Description","text":"","category":"section"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"The current consensus in the literature is that Counterfactual Explanations should be realistic: the generated counterfactuals should look like they were generated by the data-generating process (DGP) that governs the problem at hand. With respect to Algorithmic Recourse, it is certainly true that counterfactuals should be realistic in order to be actionable for individuals.[1] To address this need, researchers have come up with various approaches in recent years. Among the most popular approaches is Latent Space Search, which was first proposed in Joshi et al. (2019): instead of traversing the feature space directly, this approach relies on a separate generative model that learns a latent space representation of the DGP. Assuming the generative model is well-specified, access to the learned latent embeddings of the data comes with two advantages:","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"Since the learned DGP is encoded in the latent space, the generated counterfactuals will respect the learned representation of the data. In practice, this means that counterfactuals will be realistic.\nThe latent space is typically a compressed (i.e. lower dimensional) version of the feature space. This makes the counterfactual search less costly.","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"There are also certain disadvantages though:","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"Learning generative models is (typically) an expensive task, which may well outweigh the benefits associated with utlimately traversing a lower dimensional space.\nIf the generative model is poorly specified, this will affect the quality of the counterfactuals.[2]","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"Anyway, traversing latent embeddings is a powerful idea that may be very useful depending on the specific context. This tutorial introduces the concept and how it is implemented in this package.","category":"page"},{"location":"explanation/generators/revise/#Usage","page":"REVISE","title":"Usage","text":"","category":"section"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"The approach can be used in our package as follows:","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"generator = REVISEGenerator()\nce = generate_counterfactual(x, target, counterfactual_data, M, generator)\nplot(ce)","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"(Image: )","category":"page"},{"location":"explanation/generators/revise/#Worked-2D-Examples","page":"REVISE","title":"Worked 2D Examples","text":"","category":"section"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"Below we load 2D data and train a VAE on it and plot the original samples against their reconstructions.","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"# output: true\n\ncounterfactual_data = CounterfactualData(load_overlapping()...)\nX = counterfactual_data.X\ny = counterfactual_data.y\ninput_dim = size(X, 1)\nusing CounterfactualExplanations.GenerativeModels: VAE, train!, reconstruct\nvae = VAE(input_dim; nll=Flux.Losses.mse, epochs=100, λ=0.01, latent_dim=2, hidden_dim=32)\nflux_training_params.verbose = true\ntrain!(vae, X, y)\nX̂ = reconstruct(vae, X)[1]\np0 = scatter(X[1, :], X[2, :], color=:blue, label=\"Original\", xlab=\"x₁\", ylab=\"x₂\")\nscatter!(X̂[1, :], X̂[2, :], color=:orange, label=\"Reconstructed\", xlab=\"x₁\", ylab=\"x₂\")\np1 = scatter(X[1, :], X̂[1, :], color=:purple, label=\"\", xlab=\"x₁\", ylab=\"x̂₁\")\np2 = scatter(X[2, :], X̂[2, :], color=:purple, label=\"\", xlab=\"x₂\", ylab=\"x̂₂\")\nplt2 = plot(p1,p2, layout=(1,2), size=(800, 400))\nplot(p0, plt2, layout=(2,1), size=(800, 600))","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"Next, we train a simple MLP for the classification task. Then we determine a target and factual class for our counterfactual search and select a random factual instance to explain.","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"M = fit_model(counterfactual_data, :MLP)\ntarget = 2\nfactual = 1\nchosen = rand(findall(predict_label(M, counterfactual_data) .== factual))\nx = select_factual(counterfactual_data, chosen)","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"Finally, we generate and visualize the generated counterfactual:","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"# Search:\ngenerator = REVISEGenerator()\nce = generate_counterfactual(x, target, counterfactual_data, M, generator)\nplot(ce)","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"(Image: )","category":"page"},{"location":"explanation/generators/revise/#3D-Example","page":"REVISE","title":"3D Example","text":"","category":"section"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"To illustrate the notion of Latent Space search, let’s look at an example involving 3-dimensional input data, which we can still visualize. The code chunk below loads the data and implements the counterfactual search.","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"# Data and Classifier:\ncounterfactual_data = CounterfactualData(load_blobs(k=3)...)\nX = counterfactual_data.X\nys = counterfactual_data.output_encoder.labels.refs\nM = fit_model(counterfactual_data, :MLP)\n\n# Randomly selected factual:\nx = select_factual(counterfactual_data,rand(1:size(counterfactual_data.X,2)))\ny = predict_label(M, counterfactual_data, x)[1]\ntarget = counterfactual_data.y_levels[counterfactual_data.y_levels .!= y][1]\n\n# Generate recourse:\nce = generate_counterfactual(x, target, counterfactual_data, M, generator)","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"The figure below demonstrates the idea of searching counterfactuals in a lower-dimensional latent space: on the left, we can see the counterfactual search in the 3-dimensional feature space, while on the right we can see the corresponding search in the latent space.","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"(Image: )","category":"page"},{"location":"explanation/generators/revise/#MNIST-data","page":"REVISE","title":"MNIST data","text":"","category":"section"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"Let’s carry the ideas introduced above over to a more complex example. The code below loads MNIST data as well as a pre-trained classifier and generative model for the data.","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"using CounterfactualExplanations.Models: load_mnist_mlp, load_mnist_ensemble, load_mnist_vae\ncounterfactual_data = CounterfactualData(load_mnist()...)\nX, y = CounterfactualExplanations.DataPreprocessing.unpack_data(counterfactual_data)\ninput_dim, n_obs = size(counterfactual_data.X)\nM = load_mnist_mlp()\nvae = load_mnist_vae()","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"The F1-score of our pre-trained image classifier on test data is: 0.94","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"Before continuing, we supply the pre-trained generative model to our data container:","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"counterfactual_data.generative_model = vae # assign generative model","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"Now let’s define a factual and target label:","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"# Randomly selected factual:\nRandom.seed!(2023)\nfactual_label = 8\nx = reshape(X[:,rand(findall(predict_label(M, counterfactual_data).==factual_label))],input_dim,1)\ntarget = 3\nfactual = predict_label(M, counterfactual_data, x)[1]","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"Using REVISE, we are going to turn a randomly drawn 8 into a 3.","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"The API call is the same as always:","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"γ = 0.95\nconv = \n CounterfactualExplanations.Convergence.DecisionThresholdConvergence(decision_threshold=γ)\n# Define generator:\ngenerator = REVISEGenerator(opt=Flux.Adam(0.1))\n# Generate recourse:\nce = generate_counterfactual(x, target, counterfactual_data, M, generator; convergence=conv)","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"The chart below shows the results:","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"(Image: )","category":"page"},{"location":"explanation/generators/revise/#References","page":"REVISE","title":"References","text":"","category":"section"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"Joshi, Shalmali, Oluwasanmi Koyejo, Warut Vijitbenjaronk, Been Kim, and Joydeep Ghosh. 2019. “Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems.” https://arxiv.org/abs/1907.09615.","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"[1] In general, we believe that there may be a trade-off between creating counterfactuals that respect the DGP vs. counterfactuals reflect the behaviour of the black-model in question - both accurately and complete.","category":"page"},{"location":"explanation/generators/revise/","page":"REVISE","title":"REVISE","text":"[2] We believe that there is another potentially crucial disadvantage of relying on a separate generative model: it reallocates the task of learning realistic explanations for the data from the black-box model to the generative model.","category":"page"},{"location":"explanation/optimisers/overview/","page":"Overview","title":"Overview","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"explanation/optimisers/overview/#Optimisation-Rules","page":"Overview","title":"Optimisation Rules","text":"","category":"section"},{"location":"explanation/optimisers/overview/","page":"Overview","title":"Overview","text":"Counterfactual search is an optimization problem. Consequently, the choice of the optimisation rule affects the generated counterfactuals. In the short term, we aim to enable users to choose any of the available Flux optimisers. This has not been sufficiently tested yet, and you may run into issues.","category":"page"},{"location":"explanation/optimisers/overview/#Custom-Optimisation-Rules","page":"Overview","title":"Custom Optimisation Rules","text":"","category":"section"},{"location":"explanation/optimisers/overview/","page":"Overview","title":"Overview","text":"Flux optimisers are specifically designed for deep learning, and in particular, for learning model parameters. In counterfactual search, the features are the free parameters that we are optimising over. To this end, some custom optimisation rules are necessary to incorporate ideas presented in the literature. In the following, we introduce those rules.","category":"page"},{"location":"CHANGELOG/#Changelog","page":"Changelog","title":"Changelog","text":"","category":"section"},{"location":"CHANGELOG/","page":"Changelog","title":"Changelog","text":"All notable changes to this project will be documented in this file.","category":"page"},{"location":"CHANGELOG/","page":"Changelog","title":"Changelog","text":"The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.","category":"page"},{"location":"CHANGELOG/","page":"Changelog","title":"Changelog","text":"Note: We try to adhere to these practices as of version [v1.1.1].","category":"page"},{"location":"CHANGELOG/#Version-[v1.1.4]-2024-04-24","page":"Changelog","title":"Version [v1.1.4] - 2024-04-24","text":"","category":"section"},{"location":"CHANGELOG/#Changed","page":"Changelog","title":"Changed","text":"","category":"section"},{"location":"CHANGELOG/","page":"Changelog","title":"Changelog","text":"Refactors the encodings and decodings such that it is now more streamlined. Instead of conditional statements, encodings are now dispatched on the type of a new unifying data.input_encoder field. [#432]\nRefactors the check for redundancy. This is now based on the convergence type and done right before the counterfactual search begins, if not redundant. [#432]","category":"page"},{"location":"CHANGELOG/#Version-[v1.1.3]-2024-04-17","page":"Changelog","title":"Version [v1.1.3] - 2024-04-17","text":"","category":"section"},{"location":"CHANGELOG/#Added","page":"Changelog","title":"Added","text":"","category":"section"},{"location":"CHANGELOG/","page":"Changelog","title":"Changelog","text":"Adds a section on Convergence to the documentation, Changelog.jl functionality and a few doc tests. [#429]","category":"page"},{"location":"CHANGELOG/#Changed-2","page":"Changelog","title":"Changed","text":"","category":"section"},{"location":"CHANGELOG/","page":"Changelog","title":"Changelog","text":"Changes style of taking gradients for the counterfactual search from implicit to explicit. [#430]\nRemoved all implicit imports. [#430]","category":"page"},{"location":"CHANGELOG/#Removed","page":"Changelog","title":"Removed","text":"","category":"section"},{"location":"CHANGELOG/","page":"Changelog","title":"Changelog","text":"Removed CUDA.jl dependency, because redundant. [#430]\nRemoved Parameters.jl dependency, because redundant. [#430]","category":"page"},{"location":"CHANGELOG/#Version-[v1.1.2]-2024-04-16","page":"Changelog","title":"Version [v1.1.2] - 2024-04-16","text":"","category":"section"},{"location":"CHANGELOG/#Changed-3","page":"Changelog","title":"Changed","text":"","category":"section"},{"location":"CHANGELOG/","page":"Changelog","title":"Changelog","text":"Replaces the GIF in the README and introduction of docs for a static image. ","category":"page"},{"location":"CHANGELOG/#Version-[v1.1.1]-2024-04-15","page":"Changelog","title":"Version [v1.1.1] - 2024-04-15","text":"","category":"section"},{"location":"CHANGELOG/#Added-2","page":"Changelog","title":"Added","text":"","category":"section"},{"location":"CHANGELOG/","page":"Changelog","title":"Changelog","text":"Added tests for LaplaceRedux extension. Bumped upper compat bound for LaplaceRedux.jl. [#428]","category":"page"},{"location":"CHANGELOG/","page":"Changelog","title":"Changelog","text":"","category":"page"},{"location":"CHANGELOG/","page":"Changelog","title":"Changelog","text":"[#428]: https://github.com/juliatrustworthyai/CounterfactualExplanations.jl/issues/428 [#429]: https://github.com/juliatrustworthyai/CounterfactualExplanations.jl/issues/429","category":"page"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"tutorials/convergence/#convergence","page":"Convergence","title":"Convergence","text":"","category":"section"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"The search for counterfactuals can be seen as an optimization problem, where the goal is to find a point in the input space. One questions that has received surprisingly little attention is how to determine when the search has converged. In a recent paper, we have briefly discussed why it is important to consider convergence (Altmeyer et al. 2024):","category":"page"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"One intuitive way to specify convergence is in terms of threshold probabilities: once the predicted probability p(y^+x^prime) exceeds some user-defined threshold γ such that the counterfactual is valid, we could consider the search to have converged. In the binary case, for example, convergence could be defined as p(y^+x^prime) 05 in this sense. Note, however, how this can be expected to yield counterfactuals in the proximity of the decision boundary, a region characterized by high aleatoric uncertainty. In other words, counterfactuals generated in this way would generally not be plausible. To avoid this from happening, we specify convergence in terms of gradients approaching zero for all our experiments and all of our generators. This is allows us to get a cleaner read on how the different counterfactual search objectives affect counterfactual outcomes.","category":"page"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"In the paper, we were primarily interested in benchmarking counterfactuals generated by different search objectives. In other contexts, however, it may be more appropriate to specify convergence in terms of threshold probabilities. Our package allows you to specify convergence in terms of gradients, threshold probabilities or simply in terms of the total number of iterations. In this section, we will show you how to do this.","category":"page"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"using CounterfactualExplanations.Convergence\ngenerator = GenericGenerator(λ=0.01)","category":"page"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"GradientBasedGenerator(nothing, CounterfactualExplanations.Objectives.distance_l1, 0.01, false, false, Descent(0.1), NamedTuple())","category":"page"},{"location":"tutorials/convergence/#Convergence-in-terms-of-gradients","page":"Convergence","title":"Convergence in terms of gradients","text":"","category":"section"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"As gradients approach zero, the conditions defined by the search objective and hence the generator are satisfied. We therefore refere to this type of convergece criterium as GeneratorConditionsConvergence","category":"page"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"conv = GeneratorConditionsConvergence(gradient_tol=0.01, max_iter=1000)\nce_gen = generate_counterfactual(x, target, counterfactual_data, M, generator; convergence = conv)","category":"page"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"CounterfactualExplanation\nConvergence: ✅ after 179 steps.","category":"page"},{"location":"tutorials/convergence/#Convergence-in-terms-of-threshold-probabilities","page":"Convergence","title":"Convergence in terms of threshold probabilities","text":"","category":"section"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"In this case, the search is considered to have converged once the predicted probability p(y^+x^prime) exceeds some user-defined threshold γ such that the counterfactual is valid. We refer to this type of convergence criterium as DecisionThresholdConvergence.","category":"page"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"conv = DecisionThresholdConvergence(decision_threshold=0.75)\nce_dec = generate_counterfactual(x, target, counterfactual_data, M, generator; convergence = conv)","category":"page"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"CounterfactualExplanation\nConvergence: ✅ after 9 steps.","category":"page"},{"location":"tutorials/convergence/#Convergence-in-terms-of-the-total-number-of-iterations","page":"Convergence","title":"Convergence in terms of the total number of iterations","text":"","category":"section"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"In this case, the search is considered to have converged once the total number of iterations exceeds some user-defined threshold max_iter. We refer to this type of convergence criterium as MaxIterConvergence.","category":"page"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"conv = MaxIterConvergence(max_iter=25)\nce_max = generate_counterfactual(x, target, counterfactual_data, M, generator; convergence = conv)","category":"page"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"CounterfactualExplanation\nConvergence: ✅ after 25 steps.","category":"page"},{"location":"tutorials/convergence/#Comparison","page":"Convergence","title":"Comparison","text":"","category":"section"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"plts = []\nfor (ce, titl) in zip([ce_gen, ce_dec, ce_max], [\"Gradient Convergence\", \"Decision Threshold Convergence\", \"Max Iterations Convergence\"])\n push!(plts, plot(ce; title=titl, cbar=false))\nend\nplot(plts..., layout=(1,3), size=(1200, 380))","category":"page"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"(Image: )","category":"page"},{"location":"tutorials/convergence/#References","page":"Convergence","title":"References","text":"","category":"section"},{"location":"tutorials/convergence/","page":"Convergence","title":"Convergence","text":"Altmeyer, Patrick, Mojtaba Farmanbar, Arie van Deursen, and Cynthia CS Liem. 2024. “Faithful Model Explanations Through Energy-Constrained Conformal Counterfactuals.” In Proceedings of the AAAI Conference on Artificial Intelligence, 38:10829–37. 10.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"tutorials/data_preprocessing/#Handling-Data","page":"Handling Data","title":"Handling Data","text":"","category":"section"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"The package works with custom data containers that contain the input and output data as well as information about the type and mutability of features. In this tutorial, we will see how data can be prepared for use with the package.","category":"page"},{"location":"tutorials/data_preprocessing/#Basic-Functionality","page":"Handling Data","title":"Basic Functionality","text":"","category":"section"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"To demonstrate the basic way to prepare data, let’s look at a standard benchmark dataset: Fisher’s classic iris dataset. We can use MLDatasets to load this data.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"dataset = Iris()","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"Our data constructor CounterfactualData needs at least two inputs: features X and targets y.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"X = dataset.features\ny = dataset.targets","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"Next, we convert the input data to a Tables.MatrixTable (following MLJ.jl) convention. Concerning the target variable, we just assign grab the first column of the data frame.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"X = table(Tables.matrix(X))\ny = y[:,1]","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"Now we can feed these two ingredients to our constructor:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"counterfactual_data = CounterfactualData(X, y)","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"Under the hood, the constructor performs basic preprocessing steps. For example, the output variable y is automatically one-hot encoded:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"counterfactual_data.y","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"3×150 Matrix{Bool}:\n 1 1 1 1 1 1 1 1 1 1 1 1 1 … 0 0 0 0 0 0 0 0 0 0 0 0\n 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"Similarly, a transformer used to scale continuous input features is automatically fitted:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"counterfactual_data.dt","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"ZScoreTransform{Float64, Vector{Float64}}(4, 2, [5.843333333333335, 3.0540000000000007, 3.7586666666666693, 1.1986666666666672], [0.8280661279778629, 0.4335943113621737, 1.7644204199522617, 0.7631607417008414])","category":"page"},{"location":"tutorials/data_preprocessing/#Categorical-Features","page":"Handling Data","title":"Categorical Features","text":"","category":"section"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"For the counterfactual search, it is important to distinguish between continuous and categorical features. This is because categorical features cannot be perturbed arbitrarily: they can take specific discrete values, but not just any value on the real line.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"Consider the following example:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"y = rand([1,0],4)\nX = (\n name=categorical([\"Danesh\", \"Lee\", \"Mary\", \"John\"]),\n grade=categorical([\"A\", \"B\", \"A\", \"C\"], ordered=true),\n sex=categorical([\"male\",\"female\",\"male\",\"male\"]),\n height=[1.85, 1.67, 1.5, 1.67],\n)\nschema(X)","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"┌────────┬──────────────────┬──────────────────────────────────┐\n│ names │ scitypes │ types │\n├────────┼──────────────────┼──────────────────────────────────┤\n│ name │ Multiclass{4} │ CategoricalValue{String, UInt32} │\n│ grade │ OrderedFactor{3} │ CategoricalValue{String, UInt32} │\n│ sex │ Multiclass{2} │ CategoricalValue{String, UInt32} │\n│ height │ Continuous │ Float64 │\n└────────┴──────────────────┴──────────────────────────────────┘","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"Typically, in the context of Unserpervised Learning, categorical features are one-hot or dummy encoded. To this end, we could use MLJ, for example:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"hot = OneHotEncoder()\nmach = MLJBase.fit!(machine(hot, X))\nW = MLJBase.transform(mach, X)\nX = permutedims(MLJBase.matrix(W))","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"In all likelihood, this pre-processing step already happens at the stage, when the supervised model is trained. Since our counterfactual generators need to work in the same feature domain as the model they are intended to explain, we assume that categorical features are already encoded.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"The CounterfactualData constructor takes two optional arguments that can be used to specify the indices of categorical and continuous features. By default, all features are assumed to be continuous. For categorical features, the constructor expects an array of arrays of integers (Vector{Vector{Int}}) where each subarray includes the indices of all one-hot encoded rows related to a single categorical feature. In the example above, the name feature is one-hot encoded across rows 1, 2, 3 and 4 of X, the grade feature is encoded across the following three rows, etc.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"schema(W)","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"┌──────────────┬────────────┬─────────┐\n│ names │ scitypes │ types │\n├──────────────┼────────────┼─────────┤\n│ name__Danesh │ Continuous │ Float64 │\n│ name__John │ Continuous │ Float64 │\n│ name__Lee │ Continuous │ Float64 │\n│ name__Mary │ Continuous │ Float64 │\n│ grade__A │ Continuous │ Float64 │\n│ grade__B │ Continuous │ Float64 │\n│ grade__C │ Continuous │ Float64 │\n│ sex__female │ Continuous │ Float64 │\n│ sex__male │ Continuous │ Float64 │\n│ height │ Continuous │ Float64 │\n└──────────────┴────────────┴─────────┘","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"The code chunk below assigns the categorical and continuous feature indices:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"features_categorical = [\n [1,2,3,4], # name\n [5,6,7], # grade\n [8,9] # sex\n]\nfeatures_continuous = [10]","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"When instantiating the data container, these indices just need to be supplied as keyword arguments:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"counterfactual_data = CounterfactualData(\n X,y;\n features_categorical = features_categorical,\n features_continuous = features_continuous\n)","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"This will ensure that the discrete domain of categorical features is respected in the counterfactual search. We achieve this through a form of Projected Gradient Descent and it works for any of our counterfactual generators.","category":"page"},{"location":"tutorials/data_preprocessing/#Example","page":"Handling Data","title":"Example","text":"","category":"section"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"To see this in action, let’s load some synthetic data using MLJ:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"N = 1000\nX, ys = MLJBase.make_blobs(N, 2; centers=2, as_table=false, center_box=(-5 => 5), cluster_std=0.5)\nys .= ys.==2","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"Next, we generate a synthetic categorical feature based on the output variable. First, we define the discrete levels:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"cat_values = [\"X\",\"Y\",\"Z\"]","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"Next, we impose that the categorical feature is most likely to take the first discrete level, namely X, whenever y is equal to 1.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"xcat = map(ys) do y\n if y==1\n x = sample(cat_values, Weights([0.8,0.1,0.1]))\n else\n x = sample(cat_values, Weights([0.1,0.1,0.8]))\n end\nend\nxcat = categorical(xcat)\nX = (\n x1 = X[:,1],\n x2 = X[:,2],\n x3 = xcat\n)\nschema(X)","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"As above, we use a OneHotEncoder to transform the data:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"hot = OneHotEncoder()\nmach = MLJBase.fit!(machine(hot, X))\nW = MLJBase.transform(mach, X)\nschema(W)\nX = permutedims(MLJBase.matrix(W))","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"Finally, we assign the categorical indices and instantiate our data container:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"features_categorical = [collect(3:size(X,1))]\ncounterfactual_data = CounterfactualData(\n X,ys';\n features_categorical = features_categorical,\n)","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"With the data pre-processed we can use the fit_model function to train a simple classifier:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"M = fit_model(counterfactual_data, :Linear)","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"Now it is finally time to generate counterfactuals. We first define 1 as our target and then choose a random sample from the non-target class:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"target = 1\nfactual = 0\nchosen = rand(findall(predict_label(M, counterfactual_data) .== factual))\nx = select_factual(counterfactual_data, chosen) ","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"5×1 Matrix{Float32}:\n -2.9433472\n 0.5782963\n 0.0\n 0.0\n 1.0","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"The factual x belongs to group Z.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"We generate a counterfactual for x using the standard API call:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"generator = GenericGenerator()\nce = generate_counterfactual(x, target, counterfactual_data, M, generator)","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"CounterfactualExplanation\nConvergence: ✅ after 7 steps.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"The search yields the following counterfactual:","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"x′ = counterfactual(ce)","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"5-element Vector{Float32}:\n 1.1222683\n 0.7145791\n 0.0\n 0.0\n 1.0","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"It belongs to group Z.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"This is intuitive because by construction the categorical variable is most likely to take that value when y is equal to the target outcome.","category":"page"},{"location":"tutorials/data_preprocessing/#Immutable-Features","page":"Handling Data","title":"Immutable Features","text":"","category":"section"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"In practice, features usually cannot be perturbed arbitrarily. Suppose, for example, that one of the features used by a bank to predict the creditworthiness of its clients is gender. If a counterfactual explanation for the prediction model indicates that female clients should change their gender to improve their creditworthiness, then this is an interesting insight (it reveals gender bias), but it is not usually an actionable transformation in practice. In such cases, we may want to constrain the mutability of features to ensure actionable and realistic recourse.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"To illustrate how this can be implemented in CounterfactualExplanations.jl we will continue to work with the synthetic data from the previous section. Mutability of features can be defined in terms of four different options: 1) the feature is mutable in both directions, 2) the feature can only increase (e.g. age), 3) the feature can only decrease (e.g. time left until your next deadline) and 4) the feature is not mutable (e.g. skin colour, ethnicity, …). To specify which category a feature belongs to, you can pass a vector of symbols containing the mutability constraints at the pre-processing stage. For each feature you can choose from these four options: :both (mutable in both directions), :increase (only up), :decrease (only down) and :none (immutable). By default, nothing is passed to that keyword argument and it is assumed that all features are mutable in both directions.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"Below we impose that the second feature is immutable.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"counterfactual_data = CounterfactualData(load_linearly_separable()...)\nM = fit_model(counterfactual_data, :Linear)\ncounterfactual_data.mutability = [:both, :none]","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"target = 2\nfactual = 1\nchosen = rand(findall(predict_label(M, counterfactual_data) .== factual))\nx = select_factual(counterfactual_data, chosen) \nce = generate_counterfactual(x, target, counterfactual_data, M, generator)","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"The resulting counterfactual path is shown in the chart below. Since only the first feature can be perturbed, the sample can only move along the horizontal axis.","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"plot(ce)","category":"page"},{"location":"tutorials/data_preprocessing/","page":"Handling Data","title":"Handling Data","text":"(Image: ) ","category":"page"},{"location":"explanation/generators/overview/","page":"Overview","title":"Overview","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"explanation/generators/overview/#generators_explanation","page":"Overview","title":"Counterfactual Generators","text":"","category":"section"},{"location":"explanation/generators/overview/","page":"Overview","title":"Overview","text":"Counterfactual generators form the very core of this package. The generator_catalogue can be used to inspect the available generators:","category":"page"},{"location":"explanation/generators/overview/","page":"Overview","title":"Overview","text":"generator_catalogue","category":"page"},{"location":"explanation/generators/overview/","page":"Overview","title":"Overview","text":"Dict{Symbol, Any} with 11 entries:\n :gravitational => GravitationalGenerator\n :growing_spheres => GrowingSpheresGenerator\n :revise => REVISEGenerator\n :clue => CLUEGenerator\n :probe => ProbeGenerator\n :dice => DiCEGenerator\n :feature_tweak => FeatureTweakGenerator\n :claproar => ClaPROARGenerator\n :wachter => WachterGenerator\n :generic => GenericGenerator\n :greedy => GreedyGenerator","category":"page"},{"location":"explanation/generators/overview/","page":"Overview","title":"Overview","text":"The following sections provide brief descriptions of all of them.","category":"page"},{"location":"explanation/generators/overview/#Gradient-based-Counterfactual-Generators","page":"Overview","title":"Gradient-based Counterfactual Generators","text":"","category":"section"},{"location":"explanation/generators/overview/","page":"Overview","title":"Overview","text":"At the time of writing, all generators are gradient-based: that is, counterfactuals are searched through gradient descent. In Altmeyer et al. (2023) we lay out a general methodological framework that can be applied to all of these generators:","category":"page"},{"location":"explanation/generators/overview/","page":"Overview","title":"Overview","text":"beginaligned\nmathbfs^prime = arg min_mathbfs^prime in mathcalS left textyloss(M(f(mathbfs^prime))y^*)+ lambda textcost(f(mathbfs^prime)) right \nendaligned ","category":"page"},{"location":"explanation/generators/overview/","page":"Overview","title":"Overview","text":"“Here mathbfs^prime=lefts_k^primeright_K is a K-dimensional array of counterfactual states and f mathcalS mapsto mathcalX maps from the counterfactual state space to the feature space.” (Altmeyer et al. 2023)","category":"page"},{"location":"explanation/generators/overview/","page":"Overview","title":"Overview","text":"For most generators, the state space is the feature space (f is the identity function) and the number of counterfactuals K is one. Latent Space generators instead search counterfactuals in some latent space mathcalS. In this case, f corresponds to the decoder part of the generative model, that is the function that maps back from the latent space to inputs.","category":"page"},{"location":"explanation/generators/overview/#References","page":"Overview","title":"References","text":"","category":"section"},{"location":"explanation/generators/overview/","page":"Overview","title":"Overview","text":"Altmeyer, Patrick, Giovan Angela, Aleksander Buszydlik, Karol Dobiczek, Arie van Deursen, and Cynthia Liem. 2023. “Endogenous Macrodynamics in Algorithmic Recourse.” In First IEEE Conference on Secure and Trustworthy Machine Learning. https://doi.org/10.1109/satml54575.2023.00036.","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"tutorials/model_catalogue/#Model-Catalogue","page":"Model Catalogue","title":"Model Catalogue","text":"","category":"section"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"While in general it is assumed that users will use this package to explain their pre-trained models, we provide out-of-the-box functionality to train various simple default models. In this tutorial, we will see how these models can be fitted to CounterfactualData.","category":"page"},{"location":"tutorials/model_catalogue/#Available-Models","page":"Model Catalogue","title":"Available Models","text":"","category":"section"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The standard_models_catalogue can be used to inspect the available default models:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"standard_models_catalogue","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Dict{Symbol, Any} with 4 entries:\n :Linear => Linear\n :LaplaceRedux => LaplaceReduxModel\n :DeepEnsemble => FluxEnsemble\n :MLP => FluxModel","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The dictionary keys correspond to the model names. In this case, the dictionary values are constructors that can be used called on instances of type CounterfactualData to fit the corresponding model. In most cases, users will find it most convenient to use the fit_model API call instead.","category":"page"},{"location":"tutorials/model_catalogue/#Fitting-Models","page":"Model Catalogue","title":"Fitting Models","text":"","category":"section"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Models from the standard model catalogue are a core part of the package and thus compatible with all offered counterfactual generators and functionalities.","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The all_models_catalogue can be used to inspect all models offered by the package:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"all_models_catalogue","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"However, when using models not included in the standard_models_catalogue, additional caution is advised: they might not be supported by all counterfactual generators or they might not be models native to Julia. Thus, a more thorough reading of their documentation may be necessary to make sure that they are used correctly.","category":"page"},{"location":"tutorials/model_catalogue/#Fitting-Flux-Models","page":"Model Catalogue","title":"Fitting Flux Models","text":"","category":"section"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"First, let’s load one of the synthetic datasets. For this, we’ll first need to import the TaijaData.jl package:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"n = 500\ndata = TaijaData.load_multi_class(n)\ncounterfactual_data = DataPreprocessing.CounterfactualData(data...)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"We could use a Deep Ensemble (Lakshminarayanan, Pritzel, and Blundell 2016) as follows:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"M = fit_model(counterfactual_data, :DeepEnsemble)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The returned object is an instance of type FluxEnsemble <: AbstractFittedModel and can be used in downstream tasks without further ado. For example, the resulting fit can be visualised using the generic plot() method as:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"plts = []\nfor target in counterfactual_data.y_levels\n plt = plot(M, counterfactual_data; target=target, title=\"p(y=$(target)|x,θ)\")\n plts = [plts..., plt]\nend\nplot(plts...)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"(Image: )","category":"page"},{"location":"tutorials/model_catalogue/#Importing-PyTorch-models","page":"Model Catalogue","title":"Importing PyTorch models","text":"","category":"section"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The package supports generating counterfactuals for any neural network that has been previously defined and trained using PyTorch, regardless of the specific architectural details of the model. To generate counterfactuals for a PyTorch model, save the model inside a .pt file and call the following function:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"model_loaded = TaijaInteroperability.pytorch_model_loader(\n \"$(pwd())/docs/src/tutorials/miscellaneous\",\n \"neural_network_class\",\n \"NeuralNetwork\",\n \"$(pwd())/docs/src/tutorials/miscellaneous/pretrained_model.pt\"\n)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The method pytorch_model_loader requires four arguments:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The path to the folder with a .py file where the PyTorch model is defined\nThe name of the file where the PyTorch model is defined\nThe name of the class of the PyTorch model\nThe path to the Pickle file that holds the model weights","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"In the above case:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The file defining the model is inside $(pwd())/docs/src/tutorials/miscellaneous\nThe name of the .py file holding the model definition is neural_network_class\nThe name of the model class is NeuralNetwork\nThe Pickle file is located at $(pwd())/docs/src/tutorials/miscellaneous/pretrained_model.pt","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Though the model file and Pickle file are inside the same directory in this tutorial, this does not necessarily have to be the case.","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The reason why the model file and Pickle file have to be provided separately is that the package expects an already trained PyTorch model as input. It is also possible to define new PyTorch models within the package, but since this is not the expected use of our package, special support is not offered for that. A guide for defining Python and PyTorch classes in Julia through PythonCall.jl can be found here.","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Once the PyTorch model has been loaded into the package, wrap it inside the PyTorchModel class:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"model_pytorch = TaijaInteroperability.PyTorchModel(model_loaded, counterfactual_data.likelihood)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"This model can now be passed into the generators like any other.","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Please note that the functionality for generating counterfactuals for Python models is only available if your Julia version is 1.8 or above. For Julia 1.7 users, we recommend upgrading the version to 1.8 or 1.9 before loading a PyTorch model into the package.","category":"page"},{"location":"tutorials/model_catalogue/#Importing-R-torch-models","page":"Model Catalogue","title":"Importing R torch models","text":"","category":"section"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"warning: Not fully tested\nPlease note that due to the incompatibility between RCall and PythonCall, it is not feasible to test both PyTorch and RTorch implementations within the same pipeline. While the RTorch implementation has been manually tested, we cannot ensure its consistent functionality as it is inherently susceptible to bugs.","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The CounterfactualExplanations package supports generating counterfactuals for neural networks that have been defined and trained using R torch. Regardless of the specific architectural details of the model, you can easily generate counterfactual explanations by following these steps.","category":"page"},{"location":"tutorials/model_catalogue/#Saving-the-R-torch-model","page":"Model Catalogue","title":"Saving the R torch model","text":"","category":"section"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"First, save your trained R torch model as a .pt file using the torch_save() function provided by the R torch library. This function allows you to serialize the model and save it to a file. For example:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"torch_save(model, file = \"$(pwd())/docs/src/tutorials/miscellaneous/r_model.pt\")","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Make sure to specify the correct file path where you want to save the model.","category":"page"},{"location":"tutorials/model_catalogue/#Loading-the-R-torch-model","page":"Model Catalogue","title":"Loading the R torch model","text":"","category":"section"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"To import the R torch model into the CounterfactualExplanations package, use the rtorch_model_loader() function. This function loads the model from the previously saved .pt file. Here is an example of how to load the R torch model:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"model_loaded = TaijaInteroperability.rtorch_model_loader(\"$(pwd())/docs/src/tutorials/miscellaneous/r_model.pt\")","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The rtorch_model_loader() function requires only one argument:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"model_path: The path to the .pt file that contains the trained R torch model.","category":"page"},{"location":"tutorials/model_catalogue/#Wrapping-the-R-torch-model","page":"Model Catalogue","title":"Wrapping the R torch model","text":"","category":"section"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Once the R torch model has been loaded into the package, wrap it inside the RTorchModel class. This step prepares the model to be used by the counterfactual generators. Here is an example:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"model_R = TaijaInteroperability.RTorchModel(model_loaded, counterfactual_data.likelihood)","category":"page"},{"location":"tutorials/model_catalogue/#Generating-counterfactuals-with-the-R-torch-model","page":"Model Catalogue","title":"Generating counterfactuals with the R torch model","text":"","category":"section"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Now that the R torch model has been wrapped inside the RTorchModel class, you can pass it into the counterfactual generators as you would with any other model.","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Please note that RCall is not fully compatible with PythonCall. Therefore, it is advisable not to import both R torch and PyTorch models within the same Julia session. Additionally, it’s worth mentioning that the R torch integration is still untested in the CounterfactualExplanations package.","category":"page"},{"location":"tutorials/model_catalogue/#Tuning-Flux-Models","page":"Model Catalogue","title":"Tuning Flux Models","text":"","category":"section"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"By default, model architectures are very simple. Through optional arguments, users have some control over the neural network architecture and can choose to impose regularization through dropout. Let’s tackle a more challenging dataset: MNIST (LeCun 1998).","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"data = TaijaData.load_mnist(10000)\ncounterfactual_data = DataPreprocessing.CounterfactualData(data...)\ntrain_data, test_data = \n CounterfactualExplanations.DataPreprocessing.train_test_split(counterfactual_data)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"(Image: )","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"In this case, we will use a Multi-Layer Perceptron (MLP) but we will adjust the model and training hyperparameters. Parameters related to training of Flux.jl models are currently stored in a mutable container:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"flux_training_params","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"CounterfactualExplanations.FluxModelParams\n loss: Symbol logitbinarycrossentropy\n opt: Symbol Adam\n n_epochs: Int64 100\n batchsize: Int64 1\n verbose: Bool false","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"In cases like this one, where model training can be expected to take a few moments, it can be useful to activate verbosity, so let’s set the corresponding field value to true. We’ll also impose mini-batch training:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"flux_training_params.verbose = true\nflux_training_params.batchsize = round(size(train_data.X,2)/10)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"To account for the fact that this is a slightly more challenging task, we will use an appropriate number of hidden neurons per layer. We will also activate dropout regularization. To scale networks up further, it is also possible to adjust the number of hidden layers, which we will not do here.","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"model_params = (\n n_hidden = 32,\n dropout = true\n)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The model_params can be supplied to the familiar API call:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"M = fit_model(train_data, :MLP; model_params...)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"FluxModel(Chain(Dense(784 => 32, relu), Dropout(0.25, active=false), Dense(32 => 10)), :classification_multi)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The model performance on our test set can be evaluated as follows:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"model_evaluation(M, test_data)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"1-element Vector{Float64}:\n 0.9185","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Finally, let’s restore the default training parameters:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"CounterfactualExplanations.reset!(flux_training_params)","category":"page"},{"location":"tutorials/model_catalogue/#Fitting-and-tuning-MLJ-models","page":"Model Catalogue","title":"Fitting and tuning MLJ models","text":"","category":"section"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Among models from the MLJ library, two models are integrated as part of the core functionality of the package:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"mlj_models_catalogue","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"These models are compatible with the Feature Tweak generator. Support for other generators has not been implemented, as both decision trees and random forests are non-differentiable tree-based models and thus, gradient-based generators don’t apply for them.","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Tuning MLJ models is very simple. As the first step, let’s reload the dataset:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"n = 500\ndata = TaijaData.load_moons(n)\ncounterfactual_data = DataPreprocessing.CounterfactualData(data...)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Using the usual procedure for fitting models, we can call the following method:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"tree = CounterfactualExplanations.Models.fit_model(counterfactual_data, :DecisionTree)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"However, it’s also possible to tune the DecisionTreeClassifier’s parameters. This can be done using the keyword arguments when calling fit_model() as follows:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"tree = CounterfactualExplanations.Models.fit_model(counterfactual_data, :DecisionTree; max_depth=2, min_samples_leaf=3)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"For all supported MLJ models, every tunable parameter they have is supported as a keyword argument. The tunable parameters for the DecisionTreeModel and the RandomForestModel can be found from the documentation of the DecisionTree.jl package under the Decision Tree Classifier and Random Forest Classifier sections.","category":"page"},{"location":"tutorials/model_catalogue/#Package-extension-models","page":"Model Catalogue","title":"Package extension models","text":"","category":"section"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The package also includes two models which don’t form a part of the core functionality of the package, but which can be accessed as package extensions. These are the EvoTreeModel from the MLJ library and the LaplaceReduxModel from LaplaceRedux.jl.","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"To trigger the package extensions, the weak dependency first has to be loaded with the using keyword:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"using EvoTrees","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Once this is done, the extension models can be used like any other model:","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"M = fit_model(counterfactual_data, :EvoTree; model_params...)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"EvoTreesExt.EvoTreeModel(machine(EvoTreeClassifier{EvoTrees.MLogLoss}\n - nrounds: 100\n - L2: 0.0\n - lambda: 0.0\n - gamma: 0.0\n - eta: 0.1\n - max_depth: 6\n - min_weight: 1.0\n - rowsample: 1.0\n - colsample: 1.0\n - nbins: 64\n - alpha: 0.5\n - tree_type: binary\n - rng: MersenneTwister(123, (0, 9018, 8016, 884))\n, …), :classification_multi)","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"The tunable parameters for the EvoTreeModel can be found from the documentation of the EvoTrees.jl package under the EvoTreeClassifier section.","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Please note that support for counterfactual generation with both LaplaceReduxModel and EvoTreeModel is not yet fully implemented.","category":"page"},{"location":"tutorials/model_catalogue/#References","page":"Model Catalogue","title":"References","text":"","category":"section"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"Lakshminarayanan, Balaji, Alexander Pritzel, and Charles Blundell. 2016. “Simple and Scalable Predictive Uncertainty Estimation Using Deep Ensembles.” https://arxiv.org/abs/1612.01474.","category":"page"},{"location":"tutorials/model_catalogue/","page":"Model Catalogue","title":"Model Catalogue","text":"LeCun, Yann. 1998. “The MNIST Database of Handwritten Digits.”","category":"page"},{"location":"assets/resources/#Further-Resources","page":"📚 Additional Resources","title":"Further Resources","text":"","category":"section"},{"location":"assets/resources/#JuliaCon-2022","page":"📚 Additional Resources","title":"JuliaCon 2022","text":"","category":"section"},{"location":"assets/resources/","page":"📚 Additional Resources","title":"📚 Additional Resources","text":"Slides: link","category":"page"},{"location":"assets/resources/#JuliaCon-Proceedings-Paper","page":"📚 Additional Resources","title":"JuliaCon Proceedings Paper","text":"","category":"section"},{"location":"assets/resources/","page":"📚 Additional Resources","title":"📚 Additional Resources","text":"TBD","category":"page"},{"location":"reference/","page":"🧐 Reference","title":"🧐 Reference","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"reference/#Reference","page":"🧐 Reference","title":"Reference","text":"","category":"section"},{"location":"reference/","page":"🧐 Reference","title":"🧐 Reference","text":"In this reference, you will find a detailed overview of the package API.","category":"page"},{"location":"reference/","page":"🧐 Reference","title":"🧐 Reference","text":"Reference guides are technical descriptions of the machinery and how to operate it. Reference material is information-oriented.— Diátaxis","category":"page"},{"location":"reference/","page":"🧐 Reference","title":"🧐 Reference","text":"In other words, you come here because you want to take a very close look at the code 🧐.","category":"page"},{"location":"reference/#Content","page":"🧐 Reference","title":"Content","text":"","category":"section"},{"location":"reference/","page":"🧐 Reference","title":"🧐 Reference","text":"Pages = [\"reference.md\"]\nDepth = 2","category":"page"},{"location":"reference/#Exported-functions","page":"🧐 Reference","title":"Exported functions","text":"","category":"section"},{"location":"reference/","page":"🧐 Reference","title":"🧐 Reference","text":"Modules = [\n CounterfactualExplanations, \n CounterfactualExplanations.Convergence,\n CounterfactualExplanations.Evaluation,\n CounterfactualExplanations.DataPreprocessing,\n CounterfactualExplanations.Models,\n CounterfactualExplanations.GenerativeModels, \n CounterfactualExplanations.Generators, \n CounterfactualExplanations.Objectives\n]\nPrivate = false","category":"page"},{"location":"reference/#CounterfactualExplanations.RawOutputArrayType","page":"🧐 Reference","title":"CounterfactualExplanations.RawOutputArrayType","text":"RawOutputArrayType\n\nA type union for the allowed type for the output array y.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.RawTargetType","page":"🧐 Reference","title":"CounterfactualExplanations.RawTargetType","text":"RawTargetType\n\nA type union for the allowed types for the target variable.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.flux_training_params","page":"🧐 Reference","title":"CounterfactualExplanations.flux_training_params","text":"flux_training_params\n\nThe default training parameter for FluxModels etc.\n\n\n\n\n\n","category":"constant"},{"location":"reference/#CounterfactualExplanations.AbstractConvergence","page":"🧐 Reference","title":"CounterfactualExplanations.AbstractConvergence","text":"An abstract type that serves as the base type for convergence objects.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.AbstractCounterfactualExplanation","page":"🧐 Reference","title":"CounterfactualExplanations.AbstractCounterfactualExplanation","text":"Base type for counterfactual explanations.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.AbstractFittedModel","page":"🧐 Reference","title":"CounterfactualExplanations.AbstractFittedModel","text":"Base type for fitted models.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.AbstractGenerator","page":"🧐 Reference","title":"CounterfactualExplanations.AbstractGenerator","text":"An abstract type that serves as the base type for counterfactual generators.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.CounterfactualExplanation","page":"🧐 Reference","title":"CounterfactualExplanations.CounterfactualExplanation","text":"A struct that collects all information relevant to a specific counterfactual explanation for a single individual.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.CounterfactualExplanation-Tuple{AbstractArray, Union{Int64, AbstractFloat, String, Symbol}, CounterfactualData, AbstractFittedModel, AbstractGenerator}","page":"🧐 Reference","title":"CounterfactualExplanations.CounterfactualExplanation","text":"function CounterfactualExplanation(;\n\tx::AbstractArray,\n\ttarget::RawTargetType,\n\tdata::CounterfactualData,\n\tM::Models.AbstractFittedModel,\n\tgenerator::Generators.AbstractGenerator,\n\tnum_counterfactuals::Int = 1,\n\tinitialization::Symbol = :add_perturbation,\n convergence::Union{AbstractConvergence,Symbol}=:decision_threshold,\n)\n\nOuter method to construct a CounterfactualExplanation structure.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.EncodedOutputArrayType","page":"🧐 Reference","title":"CounterfactualExplanations.EncodedOutputArrayType","text":"EncodedOutputArrayType\n\nType of encoded output array.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.EncodedTargetType","page":"🧐 Reference","title":"CounterfactualExplanations.EncodedTargetType","text":"EncodedTargetType\n\nType of encoded target variable.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.OutputEncoder","page":"🧐 Reference","title":"CounterfactualExplanations.OutputEncoder","text":"OutputEncoder\n\nThe OutputEncoder takes a raw output array (y) and encodes it.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.OutputEncoder-Tuple{Union{Int64, AbstractFloat, String, Symbol}}","page":"🧐 Reference","title":"CounterfactualExplanations.OutputEncoder","text":"(encoder::OutputEncoder)(ynew::RawTargetType)\n\nWhen called on a new value ynew, the OutputEncoder encodes it based on the initial encoding.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.OutputEncoder-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.OutputEncoder","text":"(encoder::OutputEncoder)()\n\nOn call, the OutputEncoder returns the encoded output array.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.EvoTreeModel","page":"🧐 Reference","title":"CounterfactualExplanations.EvoTreeModel","text":"EvoTreeModel\n\nExposes the EvoTreeModel from the EvoTreesExt extension.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.LaplaceReduxModel","page":"🧐 Reference","title":"CounterfactualExplanations.LaplaceReduxModel","text":"LaplaceReduxModel\n\nExposes the LaplaceReduxModel from the LaplaceReduxExt extension.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.NeuroTreeModel","page":"🧐 Reference","title":"CounterfactualExplanations.NeuroTreeModel","text":"NeuroTreeModel\n\nExposes the NeuroTreeModel from the NeuroTreeExt extension.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.generate_counterfactual-Tuple{Base.Iterators.Zip, Union{Int64, AbstractFloat, String, Symbol}, CounterfactualData, AbstractFittedModel, AbstractGenerator}","page":"🧐 Reference","title":"CounterfactualExplanations.generate_counterfactual","text":"generate_counterfactual(\n x::Base.Iterators.Zip,\n target::RawTargetType,\n data::CounterfactualData,\n M::Models.AbstractFittedModel,\n generator::AbstractGenerator;\n kwargs...,\n)\n\nOverloads the generate_counterfactual method to accept a zip of factuals x and return a vector of counterfactuals.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.generate_counterfactual-Tuple{Matrix, Union{Int64, AbstractFloat, String, Symbol}, CounterfactualData, AbstractFittedModel, AbstractGenerator}","page":"🧐 Reference","title":"CounterfactualExplanations.generate_counterfactual","text":"generate_counterfactual(\n x::Matrix,\n target::RawTargetType,\n data::CounterfactualData,\n M::Models.AbstractFittedModel,\n generator::AbstractGenerator;\n num_counterfactuals::Int=1,\n initialization::Symbol=:add_perturbation,\n convergence::Union{AbstractConvergence,Symbol}=:decision_threshold,\n timeout::Union{Nothing,Real}=nothing,\n)\n\nThe core function that is used to run counterfactual search for a given factual x, target, counterfactual data, model and generator. Keywords can be used to specify the desired threshold for the predicted target class probability and the maximum number of iterations.\n\nArguments\n\nx::Matrix: Factual data point.\ntarget::RawTargetType: Target class.\ndata::CounterfactualData: Counterfactual data.\nM::Models.AbstractFittedModel: Fitted model.\ngenerator::AbstractGenerator: Generator.\nnum_counterfactuals::Int=1: Number of counterfactuals to generate for factual.\ninitialization::Symbol=:add_perturbation: Initialization method. By default, the initialization is done by adding a small random perturbation to the factual to achieve more robustness.\nconvergence::Union{AbstractConvergence,Symbol}=:decision_threshold: Convergence criterion. By default, the convergence is based on the decision threshold. Possible values are :decision_threshold, :max_iter, :generator_conditions or a conrete convergence object (e.g. DecisionThresholdConvergence). \ntimeout::Union{Nothing,Int}=nothing: Timeout in seconds.\n\nExamples\n\nGeneric generator\n\njulia> using CounterfactualExplanations\n\njulia> using TaijaData\n \n # Counteractual data and model:\n\njulia> counterfactual_data = CounterfactualData(load_linearly_separable()...);\n\njulia> M = fit_model(counterfactual_data, :Linear);\n\njulia> target = 2;\n\njulia> factual = 1;\n\njulia> chosen = rand(findall(predict_label(M, counterfactual_data) .== factual));\n\njulia> x = select_factual(counterfactual_data, chosen);\n \n # Search:\n\njulia> generator = Generators.GenericGenerator();\n\njulia> ce = generate_counterfactual(x, target, counterfactual_data, M, generator)\nCounterfactualExplanation\nConvergence: ✅ after 7 steps.\n\nBroadcasting\n\nThe generate_counterfactual method can also be broadcasted over a tuple containing an array. This allows for generating multiple counterfactuals in parallel. \n\njulia> chosen = rand(findall(predict_label(M, counterfactual_data) .== factual), 5);\n\njulia> xs = select_factual(counterfactual_data, chosen);\n\njulia> ces = generate_counterfactual.(xs, target, counterfactual_data, M, generator)\n5-element Vector{CounterfactualExplanation}:\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n CounterfactualExplanation\nConvergence: ✅ after 8 steps.\n CounterfactualExplanation\nConvergence: ✅ after 6 steps.\n CounterfactualExplanation\nConvergence: ✅ after 7 steps.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.generate_counterfactual-Tuple{Matrix, Union{Int64, AbstractFloat, String, Symbol}, CounterfactualData, AbstractFittedModel, GrowingSpheresGenerator}","page":"🧐 Reference","title":"CounterfactualExplanations.generate_counterfactual","text":"generate_counterfactual(\n x::Matrix,\n target::RawTargetType,\n data::DataPreprocessing.CounterfactualData,\n M::Models.AbstractFittedModel,\n generator::Generators.GrowingSpheresGenerator;\n num_counterfactuals::Int=1,\n convergence::Union{AbstractConvergence,Symbol}=Convergence.DecisionThresholdConvergence(;\n decision_threshold=(1 / length(data.y_levels)), max_iter=1000\n ),\n kwrgs...,\n)\n\nOverloads the generate_counterfactual for the GrowingSpheresGenerator generator.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.generate_counterfactual-Tuple{Tuple{AbstractArray}, Vararg{Any}}","page":"🧐 Reference","title":"CounterfactualExplanations.generate_counterfactual","text":"generate_counterfactual(x::Tuple{<:AbstractArray}, args...; kwargs...)\n\nOverloads the generate_counterfactual method to accept a tuple containing and array. This allows for broadcasting over Zip iterators.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.generate_counterfactual-Tuple{Vector{<:Matrix}, Union{Int64, AbstractFloat, String, Symbol}, CounterfactualData, AbstractFittedModel, AbstractGenerator}","page":"🧐 Reference","title":"CounterfactualExplanations.generate_counterfactual","text":"generate_counterfactual(\n x::Vector{<:Matrix},\n target::RawTargetType,\n data::CounterfactualData,\n M::Models.AbstractFittedModel,\n generator::AbstractGenerator;\n kwargs...,\n)\n\nOverloads the generate_counterfactual method to accept a vector of factuals x and return a vector of counterfactuals.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.get_target_index-Tuple{Any, Any}","page":"🧐 Reference","title":"CounterfactualExplanations.get_target_index","text":"get_target_index(y_levels, target)\n\nUtility that returns the index of target in y_levels.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.path-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.path","text":"path(ce::CounterfactualExplanation)\n\nA convenience method that returns the entire counterfactual path.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.target_probs","page":"🧐 Reference","title":"CounterfactualExplanations.target_probs","text":"target_probs(\n ce::CounterfactualExplanation,\n x::Union{AbstractArray,Nothing}=nothing,\n)\n\nReturns the predicted probability of the target class for x. If x is nothing, the predicted probability corresponding to the counterfactual value is returned.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.terminated-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.terminated","text":"terminated(ce::CounterfactualExplanation)\n\nA convenience method that checks if the counterfactual search has terminated.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.total_steps-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.total_steps","text":"total_steps(ce::CounterfactualExplanation)\n\nA convenience method that returns the total number of steps of the counterfactual search.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.update!-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.update!","text":"update!(ce::CounterfactualExplanation)\n\nAn important subroutine that updates the counterfactual explanation. It takes a snapshot of the current counterfactual search state and passes it to the generator. Based on the current state the generator generates perturbations. Various constraints are then applied to the proposed vector of feature perturbations. Finally, the counterfactual search state is updated.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Convergence.convergence_catalogue","page":"🧐 Reference","title":"CounterfactualExplanations.Convergence.convergence_catalogue","text":"convergence_catalogue\n\nA dictionary containing all convergence criteria.\n\n\n\n\n\n","category":"constant"},{"location":"reference/#CounterfactualExplanations.Convergence.DecisionThresholdConvergence","page":"🧐 Reference","title":"CounterfactualExplanations.Convergence.DecisionThresholdConvergence","text":"DecisionThresholdConvergence\n\nConvergence criterion based on the target class probability threshold. The search stops when the target class probability exceeds the predefined threshold.\n\nFields\n\ndecision_threshold::AbstractFloat: The predefined threshold for the target class probability.\nmax_iter::Int: The maximum number of iterations.\nmin_success_rate::AbstractFloat: The minimum success rate for the target class probability.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Convergence.GeneratorConditionsConvergence","page":"🧐 Reference","title":"CounterfactualExplanations.Convergence.GeneratorConditionsConvergence","text":"GeneratorConditionsConvergence\n\nConvergence criterion for counterfactual explanations based on the generator conditions. The search stops when the gradients of the search objective are below a certain threshold and the generator conditions are satisfied.\n\nFields\n\ndecision_threshold::AbstractFloat: The threshold for the decision probability.\ngradient_tol::AbstractFloat: The tolerance for the gradients of the search objective.\nmax_iter::Int: The maximum number of iterations.\nmin_success_rate::AbstractFloat: The minimum success rate for the generator conditions (across counterfactuals).\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Convergence.GeneratorConditionsConvergence-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Convergence.GeneratorConditionsConvergence","text":"GeneratorConditionsConvergence(; decision_threshold=0.5, gradient_tol=1e-2, max_iter=100, min_success_rate=0.75, y_levels=nothing)\n\nOuter constructor for GeneratorConditionsConvergence.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Convergence.MaxIterConvergence","page":"🧐 Reference","title":"CounterfactualExplanations.Convergence.MaxIterConvergence","text":"MaxIterConvergence\n\nConvergence criterion based on the maximum number of iterations.\n\nFields\n\nmax_iter::Int: The maximum number of iterations.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Convergence.converged","page":"🧐 Reference","title":"CounterfactualExplanations.Convergence.converged","text":"converged(\n convergence::MaxIterConvergence,\n ce::AbstractCounterfactualExplanation,\n x::Union{AbstractArray,Nothing}=nothing,\n)\n\nChecks if the counterfactual search has converged when the convergence criterion is maximum iterations. This means the counterfactual search will not terminate until the maximum number of iterations has been reached independently of the other convergence criteria.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.Convergence.converged-2","page":"🧐 Reference","title":"CounterfactualExplanations.Convergence.converged","text":"converged(\n convergence::GeneratorConditionsConvergence,\n ce::AbstractCounterfactualExplanation,\n x::Union{AbstractArray,Nothing}=nothing,\n)\n\nChecks if the counterfactual search has converged when the convergence criterion is generator_conditions.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.Convergence.converged-3","page":"🧐 Reference","title":"CounterfactualExplanations.Convergence.converged","text":"converged(\n convergence::InvalidationRateConvergence,\n ce::AbstractCounterfactualExplanation,\n x::Union{AbstractArray,Nothing}=nothing,\n)\n\nChecks if the counterfactual search has converged when the convergence criterion is invalidation rate.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.Convergence.converged-4","page":"🧐 Reference","title":"CounterfactualExplanations.Convergence.converged","text":"converged(\n convergence::DecisionThresholdConvergence,\n ce::AbstractCounterfactualExplanation,\n x::Union{AbstractArray,Nothing}=nothing,\n)\n\nChecks if the counterfactual search has converged when the convergence criterion is the decision threshold.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.Convergence.get_convergence_type-Tuple{AbstractConvergence, AbstractVector}","page":"🧐 Reference","title":"CounterfactualExplanations.Convergence.get_convergence_type","text":"get_convergence_type(convergence::AbstractConvergence)\n\nReturns the convergence object.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Convergence.get_convergence_type-Tuple{Symbol, AbstractVector}","page":"🧐 Reference","title":"CounterfactualExplanations.Convergence.get_convergence_type","text":"get_convergence_type(convergence::Symbol)\n\nReturns the convergence object from the dictionary of default convergence types.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Convergence.hinge_loss-Tuple{CounterfactualExplanations.Convergence.InvalidationRateConvergence, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Convergence.hinge_loss","text":"hinge_loss(convergence::InvalidationRateConvergence, ce::AbstractCounterfactualExplanation)\n\nCalculates the hinge loss of a counterfactual explanation.\n\nArguments\n\nconvergence::InvalidationRateConvergence: The convergence criterion to use.\nce::AbstractCounterfactualExplanation: The counterfactual explanation to calculate the hinge loss for.\n\nReturns\n\nThe hinge loss of the counterfactual explanation.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Convergence.invalidation_rate-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Convergence.invalidation_rate","text":"invalidation_rate(ce::AbstractCounterfactualExplanation)\n\nCalculates the invalidation rate of a counterfactual explanation.\n\nArguments\n\nce::AbstractCounterfactualExplanation: The counterfactual explanation to calculate the invalidation rate for.\nkwargs: Additional keyword arguments to pass to the function.\n\nReturns\n\nThe invalidation rate of the counterfactual explanation.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Convergence.threshold_reached","page":"🧐 Reference","title":"CounterfactualExplanations.Convergence.threshold_reached","text":"threshold_reached(ce::AbstractCounterfactualExplanation, x::Union{AbstractArray,Nothing}=nothing)\n\nDetermines if the predefined threshold for the target class probability has been reached.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.Evaluation.default_measures","page":"🧐 Reference","title":"CounterfactualExplanations.Evaluation.default_measures","text":"The default evaluation measures.\n\n\n\n\n\n","category":"constant"},{"location":"reference/#CounterfactualExplanations.Evaluation.Benchmark","page":"🧐 Reference","title":"CounterfactualExplanations.Evaluation.Benchmark","text":"A container for benchmarks of counterfactual explanations. Instead of subtyping DataFrame, it contains a DataFrame of evaluation measures (see this discussion for why we don't subtype DataFrame directly).\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Evaluation.Benchmark-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Evaluation.Benchmark","text":"(bmk::Benchmark)(; agg=mean)\n\nReturns a DataFrame containing evaluation measures aggregated by num_counterfactual.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Evaluation.benchmark-Tuple{CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.Evaluation.benchmark","text":"benchmark(\n data::CounterfactualData;\n models::Dict{<:Any,<:Any}=standard_models_catalogue,\n generators::Union{Nothing,Dict{<:Any,<:AbstractGenerator}}=nothing,\n measure::Union{Function,Vector{Function}}=default_measures,\n n_individuals::Int=5,\n suppress_training::Bool=false,\n factual::Union{Nothing,RawTargetType}=nothing,\n target::Union{Nothing,RawTargetType}=nothing,\n store_ce::Bool=false,\n parallelizer::Union{Nothing,AbstractParallelizer}=nothing,\n kwrgs...,\n)\n\nRuns the benchmarking exercise as follows:\n\nRandomly choose a factual and target label unless specified. \nIf no pretrained models are provided, it is assumed that a dictionary of callable model objects is provided (by default using the standard_models_catalogue). \nEach of these models is then trained on the data. \nFor each model separately choose n_individuals randomly from the non-target (factual) class. For each generator create a benchmark as in benchmark(xs::Union{AbstractArray,Base.Iterators.Zip}).\nFinally, concatenate the results.\n\nIf vertical_splits is specified to an integer, the computations are split vertically into vertical_splits chunks. In this case, the results are stored in a temporary directory and concatenated afterwards. \n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Evaluation.benchmark-Tuple{Union{AbstractArray, Base.Iterators.Zip}, Union{Int64, AbstractFloat, String, Symbol}, CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.Evaluation.benchmark","text":"benchmark(\n x::Union{AbstractArray,Base.Iterators.Zip},\n target::RawTargetType,\n data::CounterfactualData;\n models::Dict{<:Any,<:AbstractFittedModel},\n generators::Dict{<:Any,<:AbstractGenerator},\n measure::Union{Function,Vector{Function}}=default_measures,\n xids::Union{Nothing,AbstractArray}=nothing,\n dataname::Union{Nothing,Symbol,String}=nothing,\n verbose::Bool=true,\n store_ce::Bool=false,\n parallelizer::Union{Nothing,AbstractParallelizer}=nothing,\n kwrgs...,\n)\n\nFirst generates counterfactual explanations for factual x, the target and data using each of the provided models and generators. Then generates a Benchmark for the vector of counterfactual explanations as in benchmark(counterfactual_explanations::Vector{CounterfactualExplanation}).\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Evaluation.benchmark-Tuple{Vector{CounterfactualExplanation}}","page":"🧐 Reference","title":"CounterfactualExplanations.Evaluation.benchmark","text":"benchmark(\n counterfactual_explanations::Vector{CounterfactualExplanation};\n meta_data::Union{Nothing,<:Vector{<:Dict}}=nothing,\n measure::Union{Function,Vector{Function}}=default_measures,\n store_ce::Bool=false,\n)\n\nGenerates a Benchmark for a vector of counterfactual explanations. Optionally meta_data describing each individual counterfactual explanation can be supplied. This should be a vector of dictionaries of the same length as the vector of counterfactuals. If no meta_data is supplied, it will be automatically inferred. All measure functions are applied to each counterfactual explanation. If store_ce=true, the counterfactual explanations are stored in the benchmark.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Evaluation.evaluate","page":"🧐 Reference","title":"CounterfactualExplanations.Evaluation.evaluate","text":"evaluate(\n ce::CounterfactualExplanation;\n measure::Union{Function,Vector{Function}}=default_measures,\n agg::Function=mean,\n report_each::Bool=false,\n output_format::Symbol=:Vector,\n pivot_longer::Bool=true\n)\n\nJust computes evaluation measures for the counterfactual explanation. By default, no meta data is reported. For report_meta=true, meta data is automatically inferred, unless this overwritten by meta_data. The optional meta_data argument should be a vector of dictionaries of the same length as the vector of counterfactual explanations. \n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.Evaluation.redundancy-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Evaluation.redundancy","text":"redundancy(ce::CounterfactualExplanation)\n\nComputes the feature redundancy: that is, the number of features that remain unchanged from their original, factual values.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Evaluation.validity-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Evaluation.validity","text":"validity(ce::CounterfactualExplanation; γ=0.5)\n\nChecks of the counterfactual search has been successful with respect to the probability threshold γ. In case multiple counterfactuals were generated, the function returns the proportion of successful counterfactuals.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.CounterfactualData-Tuple{AbstractMatrix, Union{AbstractMatrix, AbstractVector}}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.CounterfactualData","text":"CounterfactualData(\n X::AbstractMatrix,\n y::RawOutputArrayType;\n mutability::Union{Vector{Symbol},Nothing}=nothing,\n domain::Union{Any,Nothing}=nothing,\n features_categorical::Union{Vector{Vector{Int}},Nothing}=nothing,\n features_continuous::Union{Vector{Int},Nothing}=nothing,\n input_encoder::Union{Nothing,InputTransformer,TypedInputTransformer}=nothing,\n)\n\nThis outer constructor method prepares features X and labels y to be used with the package. Mutability and domain constraints can be added for the features. The function also accepts arguments that specify which features are categorical and which are continues. These arguments are currently not used. \n\nExamples\n\nusing CounterfactualExplanations.Data\nx, y = toy_data_linear()\nX = hcat(x...)\ncounterfactual_data = CounterfactualData(X,y')\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.CounterfactualData-Tuple{Tables.MatrixTable, Union{AbstractMatrix, AbstractVector}}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.CounterfactualData","text":"function CounterfactualData(\n X::Tables.MatrixTable,\n y::RawOutputArrayType;\n kwrgs...\n)\n\nOuter constructor method that accepts a Tables.MatrixTable. By default, the indices of categorical and continuous features are automatically inferred the features' scitype.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.apply_domain_constraints-Tuple{CounterfactualData, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.apply_domain_constraints","text":"apply_domain_constraints(counterfactual_data::CounterfactualData, x::AbstractArray)\n\nA subroutine that is used to apply the predetermined domain constraints.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.select_factual-Tuple{CounterfactualData, Int64}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.select_factual","text":"select_factual(counterfactual_data::CounterfactualData, index::Int)\n\nA convenience method that can be used to access the feature matrix.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.select_factual-Tuple{CounterfactualData, Union{UnitRange{Int64}, Vector{Int64}}}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.select_factual","text":"select_factual(counterfactual_data::CounterfactualData, index::Union{Vector{Int},UnitRange{Int}})\n\nA convenience method that can be used to access the feature matrix.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.transformable_features-Tuple{CounterfactualData, Any}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.transformable_features","text":"transformable_features(counterfactual_data::CounterfactualData, input_encoder::Any)\n\nBy default, all continuous features are transformable. This function returns the indices of all continuous features.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.transformable_features-Tuple{CounterfactualData, Type{StatsBase.ZScoreTransform}}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.transformable_features","text":"transformable_features(\n counterfactual_data::CounterfactualData, input_encoder::Type{ZScoreTransform}\n)\n\nReturns the indices of all continuous features that can be transformed. For constant features ZScoreTransform returns NaN.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.transformable_features-Tuple{CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.transformable_features","text":"transformable_features(counterfactual_data::CounterfactualData)\n\nDispatches the transformable_features function to the appropriate method based on the type of the dt field.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.all_models_catalogue","page":"🧐 Reference","title":"CounterfactualExplanations.Models.all_models_catalogue","text":"all_models_catalogue\n\nA dictionary containing both differentiable and non-differentiable machine learning models.\n\n\n\n\n\n","category":"constant"},{"location":"reference/#CounterfactualExplanations.Models.mlj_models_catalogue","page":"🧐 Reference","title":"CounterfactualExplanations.Models.mlj_models_catalogue","text":"mlj_models_catalogue\n\nA dictionary containing all machine learning models from the MLJ model registry that the package supports.\n\n\n\n\n\n","category":"constant"},{"location":"reference/#CounterfactualExplanations.Models.standard_models_catalogue","page":"🧐 Reference","title":"CounterfactualExplanations.Models.standard_models_catalogue","text":"standard_models_catalogue\n\nA dictionary containing all differentiable machine learning models.\n\n\n\n\n\n","category":"constant"},{"location":"reference/#CounterfactualExplanations.Models.AbstractDifferentiableModel","page":"🧐 Reference","title":"CounterfactualExplanations.Models.AbstractDifferentiableModel","text":"Base type for differentiable models.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Models.FluxEnsemble","page":"🧐 Reference","title":"CounterfactualExplanations.Models.FluxEnsemble","text":"FluxEnsemble <: AbstractFluxModel\n\nConstructor for deep ensembles trained in Flux.jl. \n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Models.FluxModel","page":"🧐 Reference","title":"CounterfactualExplanations.Models.FluxModel","text":"FluxModel <: AbstractFluxModel\n\nConstructor for models trained in Flux.jl. \n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Models.FluxModel-Tuple{CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.FluxModel","text":"FluxModel(data::CounterfactualData; kwargs...)\n\nConstructs a multi-layer perceptron (MLP).\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.DecisionTreeModel-Tuple{CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.DecisionTreeModel","text":"DecisionTreeModel(data::CounterfactualData; kwargs...)\n\nConstructs a new TreeModel object wrapped around a decision tree from the data in a CounterfactualData object. Not called by the user directly.\n\nArguments\n\ndata::CounterfactualData: The CounterfactualData object containing the data to be used for training the model.\n\nReturns\n\nmodel::TreeModel: A TreeModel object.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.Linear-Tuple{CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.Linear","text":"Linear(data::CounterfactualData; kwargs...)\n\nConstructs a model with one linear layer. If the output is binary, this corresponds to logistic regression, since model outputs are passed through the sigmoid function. If the output is multi-class, this corresponds to multinomial logistic regression, since model outputs are passed through the softmax function.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.RandomForestModel-Tuple{CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.RandomForestModel","text":"RandomForestModel(data::CounterfactualData; kwargs...)\n\nConstructs a new TreeModel object wrapped around a random forest from the data in a CounterfactualData object. Not called by the user directly.\n\nArguments\n\ndata::CounterfactualData: The CounterfactualData object containing the data to be used for training the model.\n\nReturns\n\nmodel::TreeModel: A TreeModel object.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.fit_model","page":"🧐 Reference","title":"CounterfactualExplanations.Models.fit_model","text":"fit_model(\n counterfactual_data::CounterfactualData, model::Symbol=:MLP;\n kwrgs...\n)\n\nFits one of the available default models to the counterfactual_data. The model argument can be used to specify the desired model. The available values correspond to the keys of the all_models_catalogue dictionary.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.Models.logits-Tuple{AbstractFittedModel, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.logits","text":"logits(M::AbstractFittedModel, X::AbstractArray)\n\nGeneric method that is compulsory for all models. It returns the raw model predictions. In classification this is sometimes referred to as logits: the non-normalized predictions that are fed into a link function to produce predicted probabilities. In regression (not currently implemented) raw outputs typically correspond to final outputs. In other words, there is typically no normalization involved.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.logits-Tuple{CounterfactualExplanations.Models.TreeModel, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.logits","text":"logits(M::TreeModel, X::AbstractArray)\n\nCalculates the logit scores output by the model M for the input data X.\n\nArguments\n\nM::TreeModel: The model selected by the user.\nX::AbstractArray: The feature vector for which the logit scores are calculated.\n\nReturns\n\nlogits::Matrix: A matrix of logits for each output class for each data point in X.\n\nExample\n\nlogits = Models.logits(M, x) # calculates the logit scores for each output class for the data point x\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.model_evaluation-Tuple{AbstractFittedModel, CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.model_evaluation","text":"model_evaluation(M::AbstractFittedModel, test_data::CounterfactualData)\n\nHelper function to compute F-Score for AbstractFittedModel on a (test) data set. By default, it computes the accuracy. Any other measure, e.g. from the StatisticalMeasures package, can be passed as an argument. Currently, only measures applicable to classification tasks are supported.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.predict_label-Tuple{AbstractFittedModel, CounterfactualData, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.predict_label","text":"predict_label(M::AbstractFittedModel, counterfactual_data::CounterfactualData, X::AbstractArray)\n\nReturns the predicted output label for a given model M, data set counterfactual_data and input data X.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.predict_label-Tuple{AbstractFittedModel, CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.predict_label","text":"predict_label(M::AbstractFittedModel, counterfactual_data::CounterfactualData)\n\nReturns the predicted output labels for all data points of data set counterfactual_data for a given model M.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.predict_label-Tuple{CounterfactualExplanations.Models.TreeModel, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.predict_label","text":"predict_label(M::TreeModel, X::AbstractArray)\n\nReturns the predicted label for X.\n\nArguments\n\nM::TreeModel: The model selected by the user.\nX::AbstractArray: The input array for which the label is predicted.\n\nReturns\n\nlabels::AbstractArray: The predicted label for each data point in X.\n\nExample\n\nlabel = Models.predict_label(M, x) # returns the predicted label for each data point in x\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.predict_proba-Tuple{AbstractFittedModel, Union{Nothing, CounterfactualData}, Union{Nothing, AbstractArray}}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.predict_proba","text":"predict_proba(M::AbstractFittedModel, counterfactual_data::CounterfactualData, X::Union{Nothing,AbstractArray})\n\nReturns the predicted output probabilities for a given model M, data set counterfactual_data and input data X.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.probs-Tuple{AbstractFittedModel, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.probs","text":"probs(M::AbstractFittedModel, X::AbstractArray)\n\nGeneric method that is compulsory for all models. It returns the normalized model predictions, so the predicted probabilities in the case of classification. In regression (not currently implemented) this method is redundant. \n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.probs-Tuple{CounterfactualExplanations.Models.TreeModel, AbstractMatrix{<:Number}}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.probs","text":"probs(M::TreeModel, X::AbstractArray{<:Number, 2})\n\nCalculates the probability scores for each output class for the two-dimensional input data matrix X.\n\nArguments\n\nM::TreeModel: The TreeModel.\nX::AbstractArray: The feature vector for which the predictions are made.\n\nReturns\n\np::Matrix: A matrix of probability scores for each output class for each data point in X.\n\nExample\n\nprobabilities = Models.probs(M, X) # calculates the probability scores for each output class for each data point in X.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.probs-Tuple{CounterfactualExplanations.Models.TreeModel, AbstractVector{<:Number}}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.probs","text":"probs(M::TreeModel, X::AbstractArray{<:Number, 1})\n\nWorks the same way as the probs(M::TreeModel, X::AbstractArray{<:Number, 2}) method above, but handles 1-dimensional rather than 2-dimensional input data.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.generator_catalogue","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.generator_catalogue","text":"A dictionary containing the constructors of all available counterfactual generators.\n\n\n\n\n\n","category":"constant"},{"location":"reference/#CounterfactualExplanations.Generators.AbstractGradientBasedGenerator","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.AbstractGradientBasedGenerator","text":"AbstractGradientBasedGenerator\n\nAn abstract type that serves as the base type for gradient-based counterfactual generators. \n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Generators.AbstractNonGradientBasedGenerator","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.AbstractNonGradientBasedGenerator","text":"AbstractNonGradientBasedGenerator\n\nAn abstract type that serves as the base type for non gradient-based counterfactual generators. \n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Generators.FeatureTweakGenerator","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.FeatureTweakGenerator","text":"Feature Tweak counterfactual generator class.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Generators.FeatureTweakGenerator-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.FeatureTweakGenerator","text":"FeatureTweakGenerator(; penalty::Union{Nothing,Function,Vector{Function}}=Objectives.distance_l2, ϵ::AbstractFloat=0.1)\n\nConstructs a new Feature Tweak Generator object.\n\nUses the L2-norm as the penalty to measure the distance between the counterfactual and the factual. According to the paper by Tolomei et al., another recommended choice for the penalty in addition to the L2-norm is the L0-norm. The L0-norm simply minimizes the number of features that are changed through the tweak.\n\nArguments\n\npenalty::Union{Nothing,Function,Vector{Function}}: The penalty function to use for the generator. Defaults to distance_l2.\nϵ::AbstractFloat: The tolerance value for the feature tweaks. Described at length in Tolomei et al. (https://arxiv.org/pdf/1706.06691.pdf). Defaults to 0.1.\n\nReturns\n\ngenerator::FeatureTweakGenerator: A non-gradient-based generator that can be used to generate counterfactuals using the feature tweak method.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.GradientBasedGenerator","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.GradientBasedGenerator","text":"Base class for gradient-based counterfactual generators.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Generators.GradientBasedGenerator-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.GradientBasedGenerator","text":"GradientBasedGenerator(;\n\tloss::Union{Nothing,Function}=nothing,\n\tpenalty::Penalty=nothing,\n\tλ::Union{Nothing,AbstractFloat,Vector{AbstractFloat}}=nothing,\n\tlatent_space::Bool::false,\n\topt::Flux.Optimise.AbstractOptimiser=Flux.Descent(),\n generative_model_params::NamedTuple=(;),\n)\n\nDefault outer constructor for GradientBasedGenerator.\n\nArguments\n\nloss::Union{Nothing,Function}=nothing: The loss function used by the model.\npenalty::Penalty=nothing: A penalty function for the generator to penalize counterfactuals too far from the original point.\nλ::Union{Nothing,AbstractFloat,Vector{AbstractFloat}}=nothing: The weight of the penalty function.\nlatent_space::Bool=false: Whether to use the latent space of a generative model to generate counterfactuals.\nopt::Flux.Optimise.AbstractOptimiser=Flux.Descent(): The optimizer to use for the generator.\ngenerative_model_params::NamedTuple: The parameters of the generative model associated with the generator.\n\nReturns\n\ngenerator::GradientBasedGenerator: A gradient-based counterfactual generator.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.GrowingSpheresGenerator","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.GrowingSpheresGenerator","text":"Growing Spheres counterfactual generator class.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Generators.GrowingSpheresGenerator-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.GrowingSpheresGenerator","text":"GrowingSpheresGenerator(; n::Int=100, η::Float64=0.1, kwargs...)\n\nConstructs a new Growing Spheres Generator object.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.JSMADescent","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.JSMADescent","text":"An optimisation rule that can be used to implement a Jacobian-based Saliency Map Attack.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Generators.JSMADescent-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.JSMADescent","text":"Outer constructor for the JSMADescent rule.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.CLUEGenerator-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.CLUEGenerator","text":"Constructor for CLUEGenerator.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.ClaPROARGenerator-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.ClaPROARGenerator","text":"Constructor for ClaPGenerator.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.DiCEGenerator-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.DiCEGenerator","text":"Constructor for DiCEGenerator.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.GenericGenerator-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.GenericGenerator","text":"Constructor for GenericGenerator.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.GravitationalGenerator-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.GravitationalGenerator","text":"Constructor for GravitationalGenerator.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.GreedyGenerator-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.GreedyGenerator","text":"Constructor for GreedyGenerator.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.ProbeGenerator-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.ProbeGenerator","text":"Constructor for ProbeGenerator.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.REVISEGenerator-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.REVISEGenerator","text":"Constructor for REVISEGenerator.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.WachterGenerator-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.WachterGenerator","text":"Constructor for WachterGenerator.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.conditions_satisfied-Tuple{AbstractGradientBasedGenerator, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.conditions_satisfied","text":"conditions_satisfied(generator::AbstractGradientBasedGenerator, ce::AbstractCounterfactualExplanation)\n\nThe default method to check if the all conditions for convergence of the counterfactual search have been satisified for gradient-based generators. By default, gradient-based search is considered to have converged as soon as the proposed feature changes for all features are smaller than one percent of its standard deviation.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.feature_tweaking!-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.feature_tweaking!","text":"feature_tweaking!(ce::AbstractCounterfactualExplanation)\n\nReturns a counterfactual instance of ce.x based on the ensemble of classifiers provided.\n\nArguments\n\nce::AbstractCounterfactualExplanation: The counterfactual explanation object.\n\nReturns\n\nce::AbstractCounterfactualExplanation: The counterfactual explanation object.\n\nExample\n\nce = feature_tweaking!(ce) # returns a counterfactual inside the ce.s′ field based on the ensemble of classifiers provided\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.generate_perturbations-Tuple{AbstractGradientBasedGenerator, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.generate_perturbations","text":"generate_perturbations(generator::AbstractGradientBasedGenerator, ce::AbstractCounterfactualExplanation)\n\nThe default method to generate feature perturbations for gradient-based generators through simple gradient descent.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.generate_perturbations-Tuple{FeatureTweakGenerator, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.generate_perturbations","text":"generate_perturbations(generator::AbstractGradientBasedGenerator, ce::AbstractCounterfactualExplanation)\n\nThe default method to generate feature perturbations for gradient-based generators through simple gradient descent.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.hinge_loss-Tuple{AbstractConvergence, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.hinge_loss","text":"hinge_loss(convergence::AbstractConvergence, ce::AbstractCounterfactualExplanation)\n\nThe default hinge loss for any convergence criterion. Can be overridden inside the Convergence module as part of the definition of specific convergence criteria.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.@objective-Tuple{Any, Any}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.@objective","text":"objective(generator, ex)\n\nA macro that can be used to define the counterfactual search objective.\n\n\n\n\n\n","category":"macro"},{"location":"reference/#CounterfactualExplanations.Generators.@search_feature_space-Tuple{Any}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.@search_feature_space","text":"search_feature_space(generator)\n\nA simple macro that can be used to specify feature space search.\n\n\n\n\n\n","category":"macro"},{"location":"reference/#CounterfactualExplanations.Generators.@search_latent_space-Tuple{Any}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.@search_latent_space","text":"search_latent_space(generator)\n\nA simple macro that can be used to specify latent space search.\n\n\n\n\n\n","category":"macro"},{"location":"reference/#CounterfactualExplanations.Generators.@with_optimiser-Tuple{Any, Any}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.@with_optimiser","text":"with_optimiser(generator, optimiser)\n\nA simple macro that can be used to specify the optimiser to be used.\n\n\n\n\n\n","category":"macro"},{"location":"reference/#CounterfactualExplanations.Objectives.ddp_diversity-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Objectives.ddp_diversity","text":"ddp_diversity(\n ce::AbstractCounterfactualExplanation;\n perturbation_size=1e-5\n)\n\nEvaluates how diverse the counterfactuals are using a Determinantal Point Process (DDP).\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Objectives.distance-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Objectives.distance","text":"distance(ce::AbstractCounterfactualExplanation, p::Real=2)\n\nComputes the distance of the counterfactual to the original factual.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Objectives.distance_l0-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Objectives.distance_l0","text":"distance_l0(ce::AbstractCounterfactualExplanation)\n\nComputes the L0 distance of the counterfactual to the original factual.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Objectives.distance_l1-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Objectives.distance_l1","text":"distance_l1(ce::AbstractCounterfactualExplanation)\n\nComputes the L1 distance of the counterfactual to the original factual.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Objectives.distance_l2-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Objectives.distance_l2","text":"distance_l2(ce::AbstractCounterfactualExplanation)\n\nComputes the L2 (Euclidean) distance of the counterfactual to the original factual.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Objectives.distance_linf-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Objectives.distance_linf","text":"distance_linf(ce::AbstractCounterfactualExplanation)\n\nComputes the L-inf distance of the counterfactual to the original factual.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Objectives.distance_mad-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Objectives.distance_mad","text":"distance_mad(ce::AbstractCounterfactualExplanation; agg=mean)\n\nThis is the distance measure proposed by Wachter et al. (2017).\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Objectives.predictive_entropy-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Objectives.predictive_entropy","text":"predictive_entropy(ce::AbstractCounterfactualExplanation; agg=Statistics.mean)\n\nComputes the predictive entropy of the counterfactuals. Explained in https://arxiv.org/abs/1406.2541.\n\n\n\n\n\n","category":"method"},{"location":"reference/#Flux.Losses.logitbinarycrossentropy-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"Flux.Losses.logitbinarycrossentropy","text":"Flux.Losses.logitbinarycrossentropy(ce::AbstractCounterfactualExplanation)\n\nSimply extends the logitbinarycrossentropy method to work with objects of type AbstractCounterfactualExplanation.\n\n\n\n\n\n","category":"method"},{"location":"reference/#Flux.Losses.logitcrossentropy-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"Flux.Losses.logitcrossentropy","text":"Flux.Losses.logitcrossentropy(ce::AbstractCounterfactualExplanation)\n\nSimply extends the logitcrossentropy method to work with objects of type AbstractCounterfactualExplanation.\n\n\n\n\n\n","category":"method"},{"location":"reference/#Flux.Losses.mse-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"Flux.Losses.mse","text":"Flux.Losses.mse(ce::AbstractCounterfactualExplanation)\n\nSimply extends the mse method to work with objects of type AbstractCounterfactualExplanation.\n\n\n\n\n\n","category":"method"},{"location":"reference/#Internal-functions","page":"🧐 Reference","title":"Internal functions","text":"","category":"section"},{"location":"reference/","page":"🧐 Reference","title":"🧐 Reference","text":"Modules = [\n CounterfactualExplanations, \n CounterfactualExplanations.Convergence,\n CounterfactualExplanations.Evaluation,\n CounterfactualExplanations.DataPreprocessing,\n CounterfactualExplanations.Models, \n CounterfactualExplanations.GenerativeModels,\n CounterfactualExplanations.Generators, \n CounterfactualExplanations.Objectives\n]\nPublic = false","category":"page"},{"location":"reference/#CounterfactualExplanations.FluxModelParams","page":"🧐 Reference","title":"CounterfactualExplanations.FluxModelParams","text":"FluxModelParams\n\nDefault MLP training parameters.\n\n\n\n\n\n","category":"type"},{"location":"reference/#Base.Broadcast.broadcastable-Tuple{AbstractFittedModel}","page":"🧐 Reference","title":"Base.Broadcast.broadcastable","text":"Treat AbstractFittedModel as scalar when broadcasting.\n\n\n\n\n\n","category":"method"},{"location":"reference/#Base.Broadcast.broadcastable-Tuple{AbstractGenerator}","page":"🧐 Reference","title":"Base.Broadcast.broadcastable","text":"Treat AbstractGenerator as scalar when broadcasting.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.adjust_shape!-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.adjust_shape!","text":"adjust_shape!(ce::CounterfactualExplanation)\n\nA convenience method that adjusts the dimensions of the counterfactual state and related fields.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.adjust_shape-Tuple{CounterfactualExplanation, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.adjust_shape","text":"adjust_shape(\n ce::CounterfactualExplanation, \n x::AbstractArray\n)\n\nA convenience method that adjusts the dimensions of x.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.already_in_target_class-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.already_in_target_class","text":"already_in_target_class(ce::CounterfactualExplanation)\n\nCheck if the factual is already in the target class.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.apply_domain_constraints!-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.apply_domain_constraints!","text":"apply_domain_constraints!(ce::CounterfactualExplanation)\n\nWrapper function that applies underlying domain constraints.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.apply_mutability-Tuple{CounterfactualExplanation, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.apply_mutability","text":"apply_mutability(\n ce::CounterfactualExplanation,\n Δs′::AbstractArray,\n)\n\nA subroutine that applies mutability constraints to the proposed vector of feature perturbations.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.counterfactual-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.counterfactual","text":"counterfactual(ce::CounterfactualExplanation)\n\nA convenience method that returns the counterfactual.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.counterfactual_label-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.counterfactual_label","text":"counterfactual_label(ce::CounterfactualExplanation)\n\nA convenience method that returns the predicted label of the counterfactual.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.counterfactual_label_path-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.counterfactual_label_path","text":"counterfactual_label_path(ce::CounterfactualExplanation)\n\nReturns the counterfactual labels for each step of the search.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.counterfactual_probability","page":"🧐 Reference","title":"CounterfactualExplanations.counterfactual_probability","text":"counterfactual_probability(ce::CounterfactualExplanation)\n\nA convenience method that computes the class probabilities of the counterfactual.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.counterfactual_probability_path-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.counterfactual_probability_path","text":"counterfactual_probability_path(ce::CounterfactualExplanation)\n\nReturns the counterfactual probabilities for each step of the search.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.decode_array-Tuple{CounterfactualData, CounterfactualExplanations.GenerativeModels.AbstractGenerativeModel, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.decode_array","text":"decode_array(dt::GenerativeModels.AbstractGenerativeModel, x::AbstractArray)\n\nHelper function to decode an array x using a data transform dt::GenerativeModels.AbstractGenerativeModel.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.decode_array-Tuple{CounterfactualData, MultivariateStats.AbstractDimensionalityReduction, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.decode_array","text":"decode_array(dt::MultivariateStats.AbstractDimensionalityReduction, x::AbstractArray)\n\nHelper function to decode an array x using a data transform dt::MultivariateStats.AbstractDimensionalityReduction.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.decode_array-Tuple{CounterfactualData, Nothing, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.decode_array","text":"decode_array(dt::Nothing, x::AbstractArray)\n\nHelper function to decode an array x using a data transform dt::Nothing. This is a no-op.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.decode_array-Tuple{CounterfactualData, StatsBase.AbstractDataTransform, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.decode_array","text":"decode_array(dt::StatsBase.AbstractDataTransform, x::AbstractArray)\n\nHelper function to decode an array x using a data transform dt::StatsBase.AbstractDataTransform.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.decode_state","page":"🧐 Reference","title":"CounterfactualExplanations.decode_state","text":"function decode_state( ce::CounterfactualExplanation, x::Union{AbstractArray,Nothing}=nothing, )\n\nApplies all the applicable decoding functions:\n\nIf applicable, map the state variable back from the latent space to the feature space.\nIf and where applicable, inverse-transform features.\nReconstruct all categorical encodings.\n\nFinally, the decoded counterfactual is returned.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.decode_state!","page":"🧐 Reference","title":"CounterfactualExplanations.decode_state!","text":"decode_state!(ce::CounterfactualExplanation, x::Union{AbstractArray,Nothing}=nothing)\n\nIn-place version of decode_state.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.encode_array-Tuple{CounterfactualData, CounterfactualExplanations.GenerativeModels.AbstractGenerativeModel, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.encode_array","text":"encode_array(dt::GenerativeModels.AbstractGenerativeModel, x::AbstractArray)\n\nHelper function to encode an array x using a data transform dt::GenerativeModels.AbstractGenerativeModel.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.encode_array-Tuple{CounterfactualData, MultivariateStats.AbstractDimensionalityReduction, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.encode_array","text":"encode_array(dt::MultivariateStats.AbstractDimensionalityReduction, x::AbstractArray)\n\nHelper function to encode an array x using a data transform dt::MultivariateStats.AbstractDimensionalityReduction.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.encode_array-Tuple{CounterfactualData, Nothing, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.encode_array","text":"encode_array(dt::Nothing, x::AbstractArray)\n\nHelper function to encode an array x using a data transform dt::Nothing. This is a no-op.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.encode_array-Tuple{CounterfactualData, StatsBase.AbstractDataTransform, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.encode_array","text":"encode_array(dt::StatsBase.AbstractDataTransform, x::AbstractArray)\n\nHelper function to encode an array x using a data transform dt::StatsBase.AbstractDataTransform.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.encode_state","page":"🧐 Reference","title":"CounterfactualExplanations.encode_state","text":"function encode_state( ce::CounterfactualExplanation, x::Union{AbstractArray,Nothing} = nothing, )\n\nApplies all required encodings to x:\n\nIf applicable, it maps x to the latent space learned by the generative model.\nIf and where applicable, it rescales features. \n\nFinally, it returns the encoded state variable.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.encode_state!","page":"🧐 Reference","title":"CounterfactualExplanations.encode_state!","text":"encode_state!(ce::CounterfactualExplanation, x::Union{AbstractArray,Nothing}=nothing)\n\nIn-place version of encode_state.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.factual-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.factual","text":"factual(ce::CounterfactualExplanation)\n\nA convenience method to retrieve the factual x.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.factual_label-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.factual_label","text":"factual_label(ce::CounterfactualExplanation)\n\nA convenience method to get the predicted label associated with the factual.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.factual_probability-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.factual_probability","text":"factual_probability(ce::CounterfactualExplanation)\n\nA convenience method to compute the class probabilities of the factual.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.find_potential_neighbours-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.find_potential_neighbours","text":"find_potential_neighbors(ce::AbstractCounterfactualExplanation)\n\nFinds potential neighbors for the selected factual data point.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.get_meta-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.get_meta","text":"get_meta(ce::CounterfactualExplanation)\n\nReturns meta data for a counterfactual explanation.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.guess_likelihood-Tuple{Union{AbstractMatrix, AbstractVector}}","page":"🧐 Reference","title":"CounterfactualExplanations.guess_likelihood","text":"guess_likelihood(y::RawOutputArrayType)\n\nGuess the likelihood based on the scientific type of the output array. Returns a symbol indicating the guessed likelihood and the scientific type of the output array.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.guess_loss-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.guess_loss","text":"guess_loss(ce::CounterfactualExplanation)\n\nGuesses the loss function to be used for the counterfactual search in case likelihood field is specified for the AbstractFittedModel instance and no loss function was explicitly declared for AbstractGenerator instance.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.initialize_state!-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.initialize_state!","text":"initialize_state!(ce::CounterfactualExplanation)\n\nInitializes the starting point for the factual(s) in-place.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.initialize_state-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.initialize_state","text":"initialize_state(ce::CounterfactualExplanation)\n\nInitializes the starting point for the factual(s):\n\nIf ce.initialization is set to :identity or counterfactuals are searched in a latent space, then nothing is done.\nIf ce.initialization is set to :add_perturbation, then a random perturbation is added to the factual following following Slack (2021): https://arxiv.org/abs/2106.02666. The authors show that this improves adversarial robustness.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.output_dim-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.output_dim","text":"output_dim(ce::CounterfactualExplanation)\n\nA convenience method that returns the output dimension of the predictive model.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.reset!-Tuple{CounterfactualExplanations.FluxModelParams}","page":"🧐 Reference","title":"CounterfactualExplanations.reset!","text":"reset!(flux_training_params::FluxModelParams)\n\nRestores the default parameter values.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.steps_exhausted-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.steps_exhausted","text":"steps_exhausted(ce::CounterfactualExplanation)\n\nA convenience method that checks if the number of maximum iterations has been exhausted.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.target_probs_path-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.target_probs_path","text":"target_probs_path(ce::CounterfactualExplanation)\n\nReturns the target probabilities for each step of the search.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Evaluation.distance_measures","page":"🧐 Reference","title":"CounterfactualExplanations.Evaluation.distance_measures","text":"All distance measures.\n\n\n\n\n\n","category":"constant"},{"location":"reference/#Base.vcat-Tuple{CounterfactualExplanations.Evaluation.Benchmark, CounterfactualExplanations.Evaluation.Benchmark}","page":"🧐 Reference","title":"Base.vcat","text":"Base.vcat(bmk1::Benchmark, bmk2::Benchmark)\n\nVertically concatenates two Benchmark objects.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Evaluation.compute_measure-Tuple{CounterfactualExplanation, Function, Function}","page":"🧐 Reference","title":"CounterfactualExplanations.Evaluation.compute_measure","text":"compute_measure(ce::CounterfactualExplanation, measure::Function, agg::Function)\n\nComputes a single measure for a counterfactual explanation. The measure is applied to the counterfactual explanation ce and aggregated using the aggregation function agg.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Evaluation.to_dataframe-Tuple{Vector, Any, Bool, Bool, Bool, CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Evaluation.to_dataframe","text":"evaluate_dataframe(\n ce::CounterfactualExplanation,\n measure::Vector{Function},\n agg::Function,\n report_each::Bool,\n pivot_longer::Bool,\n store_ce::Bool,\n)\n\nEvaluates a counterfactual explanation and returns a dataframe of evaluation measures.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Evaluation.validity_strict-Tuple{CounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Evaluation.validity_strict","text":"validity_strict(ce::CounterfactualExplanation)\n\nChecks if the counterfactual search has been strictly valid in the sense that it has converged with respect to the pre-specified target probability γ.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.InputTransformer","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.InputTransformer","text":"InputTransformer\n\nAbstract type for data transformers. This can be any of the following:\n\nStatsBase.AbstractDataTransform: A data transformation object from the StatsBase package.\nMultivariateStats.AbstractDimensionalityReduction: A dimensionality reduction object from the MultivariateStats package.\nGenerativeModels.AbstractGenerativeModel: A generative model object from the GenerativeModels module.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.TypedInputTransformer","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.TypedInputTransformer","text":"TypedInputTransformer\n\nAbstract type for data transformers.\n\n\n\n\n\n","category":"type"},{"location":"reference/#Base.Broadcast.broadcastable-Tuple{CounterfactualData}","page":"🧐 Reference","title":"Base.Broadcast.broadcastable","text":"Treat CounterfactualData as scalar when broadcasting.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing._subset-Tuple{CounterfactualData, Vector{Int64}}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing._subset","text":"_subset(data::CounterfactualData, idx::Vector{Int})\n\nCreates a subset of the data.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.convert_to_1d-Tuple{Matrix, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.convert_to_1d","text":"convert_to_1d(y::Matrix, y_levels::AbstractArray)\n\nHelper function to convert a one-hot encoded matrix to a vector of labels. This is necessary because MLJ models require the labels to be represented as a vector, but the synthetic datasets in this package hold the labels in one-hot encoded form.\n\nArguments\n\ny::Matrix: The one-hot encoded matrix.\ny_levels::AbstractArray: The levels of the categorical variable.\n\nReturns\n\nlabels: A vector of labels.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.fit_transformer-Tuple{CounterfactualData, Nothing}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.fit_transformer","text":"fit_transformer(data::CounterfactualData, input_encoder::Nothing; kwargs...)\n\nFit a transformer to the data. This is a no-op if input_encoder is Nothing.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.fit_transformer-Tuple{CounterfactualData, Type{<:CounterfactualExplanations.GenerativeModels.AbstractGenerativeModel}}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.fit_transformer","text":"fit_transformer(\n data::CounterfactualData,\n input_encoder::Type{GenerativeModels.AbstractGenerativeModel};\n kwargs...,\n)\n\nFit a transformer to the data for a GenerativeModels.AbstractGenerativeModel object.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.fit_transformer-Tuple{CounterfactualData, Type{<:MultivariateStats.AbstractDimensionalityReduction}}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.fit_transformer","text":"fit_transformer(\n data::CounterfactualData,\n input_encoder::Type{MultivariateStats.AbstractDimensionalityReduction};\n kwargs...,\n)\n\nFit a transformer to the data for a MultivariateStats.AbstractDimensionalityReduction object.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.fit_transformer-Tuple{CounterfactualData, Type{<:StatsBase.AbstractDataTransform}}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.fit_transformer","text":"fit_transformer(\n data::CounterfactualData,\n input_encoder::Type{StatsBase.AbstractDataTransform};\n kwargs...,\n)\n\nFit a transformer to the data for a StatsBase.AbstractDataTransform object.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.input_dim-Tuple{CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.input_dim","text":"input_dim(counterfactual_data::CounterfactualData)\n\nHelper function that returns the input dimension (number of features) of the data. \n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.mutability_constraints-Tuple{CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.mutability_constraints","text":"mutability_constraints(counterfactual_data::CounterfactualData)\n\nA convenience function that returns the mutability constraints. If none were specified, it is assumed that all features are mutable in :both directions.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.preprocess_data_for_mlj-Tuple{CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.preprocess_data_for_mlj","text":"preprocess_data_for_mlj(data::CounterfactualData)\n\nHelper function to preprocess data::CounterfactualData for MLJ models.\n\nArguments\n\ndata::CounterfactualData: The data to be preprocessed.\n\nReturns\n\n(df_x, y): A tuple containing the preprocessed data, with df_x being a DataFrame object and y being a categorical vector.\n\nExample\n\nX, y = preprocessdatafor_mlj(data)\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.reconstruct_cat_encoding-Tuple{CounterfactualData, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.reconstruct_cat_encoding","text":"reconstruct_cat_encoding(counterfactual_data::CounterfactualData, x::Vector)\n\nReconstruct the categorical encoding for a single instance.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.subsample-Tuple{CounterfactualData, Int64}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.subsample","text":"subsample(data::CounterfactualData, n::Int)\n\nHelper function to randomly subsample data::CounterfactualData.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.train_test_split-Tuple{CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.train_test_split","text":"train_test_split(data::CounterfactualData;test_size=0.2,keep_class_ratio=false)\n\nSplits data into train and test split.\n\nArguments\n\ndata::CounterfactualData: The data to be preprocessed.\ntest_size=0.2: Proportion of the data to be used for testing. \nkeep_class_ratio=false: Decides whether to sample equally from each class, or keep their relative size.\n\nReturns\n\n(train_data::CounterfactualData, test_data::CounterfactualData): A tuple containing the train and test splits.\n\nExample\n\ntrain, test = traintestsplit(data, testsize=0.1, keepclass_ratio=true)\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.DataPreprocessing.unpack_data-Tuple{CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.DataPreprocessing.unpack_data","text":"unpack_data(data::CounterfactualData)\n\nHelper function that unpacks data.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.AbstractCustomDifferentiableModel","page":"🧐 Reference","title":"CounterfactualExplanations.Models.AbstractCustomDifferentiableModel","text":"Base type for custom differentiable models.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Models.AbstractFluxModel","page":"🧐 Reference","title":"CounterfactualExplanations.Models.AbstractFluxModel","text":"Base type for differentiable models written in Flux.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Models.AbstractMLJModel","page":"🧐 Reference","title":"CounterfactualExplanations.Models.AbstractMLJModel","text":"Base type for differentiable models from the MLJ library.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Models.AbstractNonDifferentiableJuliaModel","page":"🧐 Reference","title":"CounterfactualExplanations.Models.AbstractNonDifferentiableJuliaModel","text":"Base type for non-differentiable models written in pure Julia.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Models.AbstractNonDifferentiableModel","page":"🧐 Reference","title":"CounterfactualExplanations.Models.AbstractNonDifferentiableModel","text":"Base type for non-differentiable models.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Models.FluxEnsembleParams","page":"🧐 Reference","title":"CounterfactualExplanations.Models.FluxEnsembleParams","text":"FluxModelParams\n\nDefault Deep Ensemble training parameters.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Models.TreeModel","page":"🧐 Reference","title":"CounterfactualExplanations.Models.TreeModel","text":"TreeModel <: AbstractNonDifferentiableJuliaModel\n\nConstructor for tree-based models from the MLJ library. \n\nArguments\n\nmodel::Any: The model selected by the user. Must be a model from the MLJ library.\nlikelihood::Symbol: The likelihood of the model. Must be one of [:classification_binary, :classification_multi].\n\nReturns\n\nTreeModel: A tree-based model from the MLJ library wrapped inside the TreeModel class.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Models.TreeModel-Tuple{Any}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.TreeModel","text":"Outer constructor method for TreeModel.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.binary_to_onehot-Tuple{Any}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.binary_to_onehot","text":"binary_to_onehot(p)\n\nHelper function to turn dummy-encoded variable into onehot-encoded variable.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.build_ensemble-Tuple{Int64}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.build_ensemble","text":"build_ensemble(K::Int;kw=(input_dim=2,n_hidden=32,output_dim=1))\n\nHelper function that builds an ensemble of K models.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.build_mlp-Tuple{}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.build_mlp","text":"build_mlp()\n\nHelper function to build simple MLP.\n\nExamples\n\nnn = build_mlp()\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.data_loader-Tuple{CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.data_loader","text":"data_loader(data::CounterfactualData)\n\nPrepares counterfactual data for training in Flux.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.get_individual_classifiers-Tuple{CounterfactualExplanations.Models.TreeModel}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.get_individual_classifiers","text":"get_individual_classifiers(M::TreeModel)\n\nReturns the individual classifiers in the forest. If the input is a decision tree, the method returns the decision tree itself inside an array.\n\nArguments\n\nM::TreeModel: The model selected by the user.\n\nReturns\n\nclassifiers::AbstractArray: An array of individual classifiers in the forest.\n\nExample\n\nclassifiers = Models.getindividualclassifiers(M) # returns the individual classifiers in the forest\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.train-Tuple{CounterfactualExplanations.Models.TreeModel, CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.train","text":"train(M::TreeModel, data::CounterfactualData; kwargs...)\n\nFits the model M to the data in the CounterfactualData object. This method is not called by the user directly.\n\nArguments\n\nM::TreeModel: The wrapper for a TreeModel.\ndata::CounterfactualData: The CounterfactualData object containing the data to be used for training the model.\n\nReturns\n\nM::TreeModel: The fitted TreeModel.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.train-Tuple{FluxEnsemble, CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.train","text":"train(M::FluxEnsemble, data::CounterfactualData; kwargs...)\n\nWrapper function to retrain.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Models.train-Tuple{FluxModel, CounterfactualData}","page":"🧐 Reference","title":"CounterfactualExplanations.Models.train","text":"train(M::FluxModel, data::CounterfactualData; kwargs...)\n\nWrapper function to retrain FluxModel.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.GenerativeModels.AbstractGMParams","page":"🧐 Reference","title":"CounterfactualExplanations.GenerativeModels.AbstractGMParams","text":"Base type of generative model hyperparameter container.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.GenerativeModels.AbstractGenerativeModel","page":"🧐 Reference","title":"CounterfactualExplanations.GenerativeModels.AbstractGenerativeModel","text":"Base type for generative model.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.GenerativeModels.Encoder","page":"🧐 Reference","title":"CounterfactualExplanations.GenerativeModels.Encoder","text":"Encoder\n\nConstructs encoder part of VAE: a simple Flux neural network with one hidden layer and two linear output layers for the first two moments of the latent distribution.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.GenerativeModels.VAE","page":"🧐 Reference","title":"CounterfactualExplanations.GenerativeModels.VAE","text":"VAE <: AbstractGenerativeModel\n\nConstructs the Variational Autoencoder. The VAE is a subtype of AbstractGenerativeModel. Any (sub-)type of AbstractGenerativeModel is accepted by latent space generators. \n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.GenerativeModels.VAE-Tuple{Any}","page":"🧐 Reference","title":"CounterfactualExplanations.GenerativeModels.VAE","text":"VAE(input_dim;kws...)\n\nOuter method for instantiating a VAE.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.GenerativeModels.VAEParams","page":"🧐 Reference","title":"CounterfactualExplanations.GenerativeModels.VAEParams","text":"VAEParams <: AbstractGMParams\n\nThe default VAE parameters describing both the encoder/decoder architecture and the training process.\n\n\n\n\n\n","category":"type"},{"location":"reference/#Base.rand","page":"🧐 Reference","title":"Base.rand","text":"Random.rand(encoder::Encoder, x, device=cpu)\n\nDraws random samples from the latent distribution.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.GenerativeModels.Decoder-Tuple{Int64, Int64, Int64}","page":"🧐 Reference","title":"CounterfactualExplanations.GenerativeModels.Decoder","text":"Decoder(input_dim::Int, latent_dim::Int, hidden_dim::Int; activation=relu)\n\nThe default decoder architecture is just a Flux Chain with one hidden layer and a linear output layer. \n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.GenerativeModels.decode-Tuple{CounterfactualExplanations.GenerativeModels.VAE, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.GenerativeModels.decode","text":"decode(generative_model::VAE, x::AbstractArray)\n\nDecodes an array x using the VAE decoder.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.GenerativeModels.encode-Tuple{CounterfactualExplanations.GenerativeModels.VAE, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.GenerativeModels.encode","text":"encode(generative_model::VAE, x::AbstractArray)\n\nEncodes an array x using the VAE encoder. Specifically, it samples from the latent distribution. It does so by first passing x through the encoder to obtain the mean and log-variance of the latent distribution. Then, it samples from the latent distribution using the reparameterization trick. See Random.rand(encoder::Encoder, x, device=cpu) for more details.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.GenerativeModels.get_data-Tuple{AbstractArray, Any}","page":"🧐 Reference","title":"CounterfactualExplanations.GenerativeModels.get_data","text":"get_data(X::AbstractArray, batch_size)\n\nPreparing data for mini-batch training .\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.GenerativeModels.reconstruct","page":"🧐 Reference","title":"CounterfactualExplanations.GenerativeModels.reconstruct","text":"reconstruct(generative_model::VAE, x, device=cpu)\n\nImplements a full pass of some input x through the VAE: x ↦ x̂.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.GenerativeModels.reparameterization_trick","page":"🧐 Reference","title":"CounterfactualExplanations.GenerativeModels.reparameterization_trick","text":"reparameterization_trick(μ,logσ,device=cpu)\n\nHelper function that implements the reparameterization trick: z ∼ 𝒩(μ,σ²) ⇔ z=μ + σ ⊙ ε, ε ∼ 𝒩(0,I).\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.Generators.Penalty","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.Penalty","text":"Type union for acceptable argument types for the penalty field of GradientBasedGenerator.\n\n\n\n\n\n","category":"type"},{"location":"reference/#CounterfactualExplanations.Generators._replace_nans","page":"🧐 Reference","title":"CounterfactualExplanations.Generators._replace_nans","text":"_replace_nans(Δs′::AbstractArray, old_new::Pair=(NaN => 0))\n\nHelper function to deal with exploding gradients. This is only a temporary fix and will be improved.\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.Generators.calculate_delta-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.calculate_delta","text":"calculate_delta(ce::AbstractCounterfactualExplanation, penalty::Vector{Function})\n\nCalculates the penalty for the proposed feature tweak.\n\nArguments\n\nce::AbstractCounterfactualExplanation: The counterfactual explanation object.\n\nReturns\n\ndelta::Float64: The calculated penalty for the proposed feature tweak.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.esatisfactory_instance-Tuple{FeatureTweakGenerator, AbstractArray, AbstractArray}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.esatisfactory_instance","text":"esatisfactory_instance(generator::FeatureTweakGenerator, x::AbstractArray, paths::Dict{String, Dict{String, Any}})\n\nReturns an epsilon-satisfactory counterfactual for x based on the paths provided.\n\nArguments\n\ngenerator::FeatureTweakGenerator: The feature tweak generator.\nx::AbstractArray: The factual instance.\npaths::Dict{String, Dict{String, Any}}: A list of paths to the leaves of the tree to be used for tweaking the feature.\n\nReturns\n\nesatisfactory::AbstractArray: The epsilon-satisfactory instance.\n\nExample\n\nesatisfactory = esatisfactory_instance(generator, x, paths) # returns an epsilon-satisfactory counterfactual for x based on the paths provided\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.feature_selection!-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.feature_selection!","text":"feature_selection!(ce::AbstractCounterfactualExplanation)\n\nPerform feature selection to find the dimension with the closest (but not equal) values between the ce.x (factual) and ce.s′ (counterfactual) arrays.\n\nArguments\n\nce::AbstractCounterfactualExplanation: An instance of the AbstractCounterfactualExplanation type representing the counterfactual explanation.\n\nReturns\n\nnothing\n\nThe function iteratively modifies the ce.s′ counterfactual array by updating its elements to match the corresponding elements in the ce.x factual array, one dimension at a time, until the predicted label of the modified ce.s′ matches the predicted label of the ce.x array.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.find_closest_dimension-Tuple{Any, Any}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.find_closest_dimension","text":"find_closest_dimension(factual, counterfactual)\n\nFind the dimension with the closest (but not equal) values between the factual and counterfactual arrays.\n\nArguments\n\nfactual: The factual array.\ncounterfactual: The counterfactual array.\n\nReturns\n\nclosest_dimension: The index of the dimension with the closest values.\n\nThe function iterates over the indices of the factual array and calculates the absolute difference between the corresponding elements in the factual and counterfactual arrays. It returns the index of the dimension with the smallest difference, excluding dimensions where the values in factual and counterfactual are equal.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.find_counterfactual-NTuple{4, Any}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.find_counterfactual","text":"find_counterfactual(model, factual_class, counterfactual_data, counterfactual_candidates)\n\nFind the first counterfactual index by predicting labels.\n\nArguments\n\nmodel: The fitted model used for prediction.\ntarget_class: Expected target class.\ncounterfactual_data: Data required for counterfactual generation.\ncounterfactual_candidates: The array of counterfactual candidates.\n\nReturns\n\ncounterfactual: The index of the first counterfactual found.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.growing_spheres_generation!-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.growing_spheres_generation!","text":"growing_spheres_generation(ce::AbstractCounterfactualExplanation)\n\nGenerate counterfactual candidates using the growing spheres generation algorithm.\n\nArguments\n\nce::AbstractCounterfactualExplanation: An instance of the AbstractCounterfactualExplanation type representing the counterfactual explanation.\n\nReturns\n\nnothing\n\nThis function applies the growing spheres generation algorithm to generate counterfactual candidates. It starts by generating random points uniformly on a sphere, gradually reducing the search space until no counterfactuals are found. Then it expands the search space until at least one counterfactual is found or the maximum number of iterations is reached.\n\nThe algorithm iteratively generates counterfactual candidates and predicts their labels using the model stored in ce.M. It checks if any of the predicted labels are different from the factual class. The process of reducing the search space involves halving the search radius, while the process of expanding the search space involves increasing the search radius.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.h-Tuple{AbstractGenerator, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.h","text":"h(generator::AbstractGenerator, ce::AbstractCounterfactualExplanation)\n\nDispatches to the appropriate complexity function for any generator.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.h-Tuple{AbstractGenerator, Function, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.h","text":"h(generator::AbstractGenerator, penalty::Function, ce::AbstractCounterfactualExplanation)\n\nOverloads the h function for the case where a single penalty function is provided.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.h-Tuple{AbstractGenerator, Nothing, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.h","text":"h(generator::AbstractGenerator, penalty::Nothing, ce::AbstractCounterfactualExplanation)\n\nOverloads the h function for the case where no penalty is provided.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.h-Tuple{AbstractGenerator, Vector{<:Tuple}, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.h","text":"h(generator::AbstractGenerator, penalty::Tuple, ce::AbstractCounterfactualExplanation)\n\nOverloads the h function for the case where a single penalty function is provided with additional keyword arguments.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.h-Tuple{AbstractGenerator, Vector{Function}, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.h","text":"h(generator::AbstractGenerator, penalty::Tuple, ce::AbstractCounterfactualExplanation)\n\nOverloads the h function for the case where a single penalty function is provided with additional keyword arguments.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.hyper_sphere_coordinates-Tuple{Integer, AbstractArray, AbstractFloat, AbstractFloat}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.hyper_sphere_coordinates","text":"hyper_sphere_coordinates(n_search_samples::Int, instance::Vector{Float64}, low::Int, high::Int; p_norm::Int=2)\n\nGenerates candidate counterfactuals using the growing spheres method based on hyper-sphere coordinates.\n\nThe implementation follows the Random Point Picking over a sphere algorithm described in the paper: \"Learning Counterfactual Explanations for Tabular Data\" by Pawelczyk, Broelemann & Kascneci (2020), presented at The Web Conference 2020 (WWW). It ensures that points are sampled uniformly at random using insights from: http://mathworld.wolfram.com/HyperspherePointPicking.html\n\nThe growing spheres method is originally proposed in the paper: \"Comparison-based Inverse Classification for Interpretability in Machine Learning\" by Thibaut Laugel et al (2018), presented at the International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems (2018).\n\nArguments\n\nn_search_samples::Int: The number of search samples (int > 0).\ninstance::AbstractArray: The input point array.\nlow::AbstractFloat: The lower bound (float >= 0, l < h).\nhigh::AbstractFloat: The upper bound (float >= 0, h > l).\np_norm::Integer: The norm parameter (int >= 1).\n\nReturns\n\ncandidate_counterfactuals::Array: An array of candidate counterfactuals.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.propose_state-Tuple{AbstractGradientBasedGenerator, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.propose_state","text":"propose_state(generator::AbstractGradientBasedGenerator, ce::AbstractCounterfactualExplanation)\n\nProposes new state based on backpropagation.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.search_path","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.search_path","text":"search_path(tree::Union{DecisionTree.Leaf, DecisionTree.Node}, target::RawTargetType, path::AbstractArray)\n\nReturn a path index list with the inequality symbols, thresholds and feature indices.\n\nArguments\n\ntree::Union{DecisionTree.Leaf, DecisionTree.Node}: The root node of a decision tree.\ntarget::RawTargetType: The target class.\npath::AbstractArray: A list containing the paths found thus far.\n\nReturns\n\npaths::AbstractArray: A list of paths to the leaves of the tree to be used for tweaking the feature.\n\nExample\n\npaths = search_path(tree, target) # returns a list of paths to the leaves of the tree to be used for tweaking the feature\n\n\n\n\n\n","category":"function"},{"location":"reference/#CounterfactualExplanations.Generators.ℓ-Tuple{AbstractGenerator, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.ℓ","text":"ℓ(generator::AbstractGenerator, ce::AbstractCounterfactualExplanation)\n\nDispatches to the appropriate loss function for any generator.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.ℓ-Tuple{AbstractGenerator, Function, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.ℓ","text":"ℓ(generator::AbstractGenerator, loss::Function, ce::AbstractCounterfactualExplanation)\n\nOverloads the ℓ function for the case where a single loss function is provided.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.ℓ-Tuple{AbstractGenerator, Nothing, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.ℓ","text":"ℓ(generator::AbstractGenerator, loss::Nothing, ce::AbstractCounterfactualExplanation)\n\nOverloads the ℓ function for the case where no loss function is provided.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.∂h-Tuple{AbstractGradientBasedGenerator, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.∂h","text":"∂h(generator::AbstractGradientBasedGenerator, ce::AbstractCounterfactualExplanation)\n\nThe default method to compute the gradient of the complexity penalty at the current counterfactual state for gradient-based generators. It assumes that Zygote.jl has gradient access. \n\nIf the penalty is not provided, it returns 0.0. By default, Zygote never works out the gradient for constants and instead returns 'nothing', so we need to add a manual step to override this behaviour. See here: https://discourse.julialang.org/t/zygote-gradient/26715.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.∂ℓ-Tuple{AbstractGradientBasedGenerator, AbstractDifferentiableModel, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.∂ℓ","text":"∂ℓ(generator::AbstractGradientBasedGenerator, M::Union{Models.LogisticModel, Models.BayesianLogisticModel}, ce::AbstractCounterfactualExplanation)\n\nThe default method to compute the gradient of the loss function at the current counterfactual state for gradient-based generators. It assumes that Zygote.jl has gradient access.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Generators.∇-Tuple{AbstractGradientBasedGenerator, AbstractDifferentiableModel, AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Generators.∇","text":"∇(generator::AbstractGradientBasedGenerator, M::Models.AbstractDifferentiableModel, ce::AbstractCounterfactualExplanation)\n\nThe default method to compute the gradient of the counterfactual search objective for gradient-based generators. It simply computes the weighted sum over partial derivates. It assumes that Zygote.jl has gradient access. If the counterfactual is being generated using Probe, the hinge loss is added to the gradient.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Objectives.distance_from_target-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Objectives.distance_from_target","text":"distance_from_target(\n ce::AbstractCounterfactualExplanation;\n K::Int=50\n)\n\nComputes the distance of the counterfactual from a point in the target main.\n\n\n\n\n\n","category":"method"},{"location":"reference/#CounterfactualExplanations.Objectives.model_loss_penalty-Tuple{AbstractCounterfactualExplanation}","page":"🧐 Reference","title":"CounterfactualExplanations.Objectives.model_loss_penalty","text":"function model_loss_penalty(\n ce::AbstractCounterfactualExplanation;\n agg=mean\n)\n\nAdditional penalty for ClaPROARGenerator.\n\n\n\n\n\n","category":"method"},{"location":"extensions/","page":"Overview","title":"Overview","text":"CurrentModule = CounterfactualExplanations","category":"page"},{"location":"extensions/#Extensions","page":"Overview","title":"⛓️ Extensions","text":"","category":"section"},{"location":"extensions/","page":"Overview","title":"Overview","text":"In this section, you will find information about package extensions of the CounterfactualExplanations package. Extensions are a relatively new feature of Julia that allows users to conditionally load code based on the presence of other packages. This is useful for creating packages that extend the functionality of other packages, without requiring the user to install the package being extended.","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"tutorials/evaluation/#Performance-Evaluation","page":"Evaluating Explanations","title":"Performance Evaluation","text":"","category":"section"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"Now that we know how to generate counterfactual explanations in Julia, you may have a few follow-up questions: How do I know if the counterfactual search has been successful? How good is my counterfactual explanation? What does ‘good’ even mean in this context? In this tutorial, we will see how counterfactual explanations can be evaluated with respect to their performance.","category":"page"},{"location":"tutorials/evaluation/#Default-Measures","page":"Evaluating Explanations","title":"Default Measures","text":"","category":"section"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"Numerous evaluation measures for counterfactual explanations have been proposed. In what follows, we will cover some of the most important measures.","category":"page"},{"location":"tutorials/evaluation/#Single-Measure,-Single-Counterfactual","page":"Evaluating Explanations","title":"Single Measure, Single Counterfactual","text":"","category":"section"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"One of the most important measures is validity, which simply determines whether or not a counterfactual explanation x^prime is valid in the sense that it yields the target prediction: M(x^prime)=t. We can evaluate the validity of a single counterfactual explanation ce using the Evaluation.evaluate function as follows:","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"using CounterfactualExplanations.Evaluation: evaluate, validity\nevaluate(ce; measure=validity)","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"1-element Vector{Vector{Float64}}:\n [1.0]","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"For a single counterfactual explanation, this evaluation measure can only take two values: it is either equal to 1, if the explanation is valid or 0 otherwise. Another important measure is distance, which relates to the distance between the factual x and the counterfactual x^prime. In the context of Algorithmic Recourse, higher distances are typically associated with higher costs to individuals seeking recourse.","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"using CounterfactualExplanations.Objectives: distance\nevaluate(ce; measure=distance)","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"1-element Vector{Vector{Float32}}:\n [3.2273161]","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"By default, distance computes the L2 (Euclidean) distance.","category":"page"},{"location":"tutorials/evaluation/#Multiple-Measures,-Single-Counterfactual","page":"Evaluating Explanations","title":"Multiple Measures, Single Counterfactual","text":"","category":"section"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"You might be interested in computing not just the L2 distance, but various LP norms. This can be done by supplying a vector of functions to the measure key argument. For convenience, all default distance measures have already been collected in a vector:","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"using CounterfactualExplanations.Evaluation: distance_measures\ndistance_measures","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"4-element Vector{Function}:\n distance_l0 (generic function with 1 method)\n distance_l1 (generic function with 1 method)\n distance_l2 (generic function with 1 method)\n distance_linf (generic function with 1 method)","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"We can use this vector of evaluation measures as follows:","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"evaluate(ce; measure=distance_measures)","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"4-element Vector{Vector{Float32}}:\n [2.0]\n [3.2273161]\n [2.7737978]\n [2.7285953]","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"If no measure is specified, the evaluate method will return all default measures,","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"evaluate(ce)","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"3-element Vector{Vector}:\n [1.0]\n Float32[3.2273161]\n [0.0]","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"which include:","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"CounterfactualExplanations.Evaluation.default_measures","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"3-element Vector{Function}:\n validity (generic function with 1 method)\n distance (generic function with 1 method)\n redundancy (generic function with 1 method)","category":"page"},{"location":"tutorials/evaluation/#Multiple-Measures-and-Counterfactuals","page":"Evaluating Explanations","title":"Multiple Measures and Counterfactuals","text":"","category":"section"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"We can also evaluate multiple counterfactual explanations at once:","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"generator = DiCEGenerator()\nces = generate_counterfactual(x, target, counterfactual_data, M, generator; num_counterfactuals=5)\nevaluate(ces)","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"3-element Vector{Vector}:\n [1.0]\n Float32[3.1955845]\n [[0.0, 0.0, 0.0, 0.0, 0.0]]","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"By default, each evaluation measure is aggregated across all counterfactual explanations. To return individual measures for each counterfactual explanation you can specify report_each=true","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"evaluate(ces; report_each=true)","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"3-element Vector{Vector}:\n BitVector[[1, 1, 1, 1, 1]]\n Vector{Float32}[[3.3671722, 3.1028512, 3.2829392, 3.0728922, 3.1520686]]\n [[0.0, 0.0, 0.0, 0.0, 0.0]]","category":"page"},{"location":"tutorials/evaluation/#Custom-Measures","page":"Evaluating Explanations","title":"Custom Measures","text":"","category":"section"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"A measure is just a method that takes a CounterfactualExplanation as its only positional argument and agg::Function as a key argument specifying how measures should be aggregated across counterfactuals. Defining custom measures is therefore straightforward. For example, we could define a measure to compute the inverse target probability as follows:","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"my_measure(ce::CounterfactualExplanation; agg=mean) = agg(1 .- CounterfactualExplanations.target_probs(ce))\nevaluate(ce; measure=my_measure)","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"1-element Vector{Vector{Float32}}:\n [0.41711217]","category":"page"},{"location":"tutorials/evaluation/#Tidy-Output","page":"Evaluating Explanations","title":"Tidy Output","text":"","category":"section"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"By default, evaluate returns vectors of evaluation measures. The optional key argument output_format::Symbol can be used to post-process the output in two ways: firstly, to return the output as a dictionary, specify output_format=:Dict:","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"evaluate(ces; output_format=:Dict, report_each=true)","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"Dict{Symbol, Vector} with 3 entries:\n :validity => BitVector[[1, 1, 1, 1, 1]]\n :redundancy => [[0.0, 0.0, 0.0, 0.0, 0.0]]\n :distance => Vector{Float32}[[3.36717, 3.10285, 3.28294, 3.07289, 3.15207]]","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"Secondly, to return the output as a data frame, specify output_format=:DataFrame.","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"evaluate(ces; output_format=:DataFrame, report_each=true)","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"By default, data frames are pivoted to long format using individual counterfactuals as the id column. This behaviour can be suppressed by specifying pivot_longer=false.","category":"page"},{"location":"tutorials/evaluation/#Multiple-Counterfactual-Explanations","page":"Evaluating Explanations","title":"Multiple Counterfactual Explanations","text":"","category":"section"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"It may be necessary to generate counterfactual explanations for multiple individuals.","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"Below, for example, we first select multiple samples (5) from the non-target class and then generate counterfactual explanations for all of them.","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"This can be done using broadcasting:","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"# Factual and target:\nids = rand(findall(predict_label(M, counterfactual_data) .== factual), n_individuals)\nxs = select_factual(counterfactual_data, ids)\nces = generate_counterfactual(xs, target, counterfactual_data, M, generator; num_counterfactuals=5)\nevaluation = evaluate.(ces)","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"5-element Vector{Vector{Vector}}:\n [[1.0], Float32[3.351181], [[0.0, 0.0, 0.0, 0.0, 0.0]]]\n [[1.0], Float32[2.6405892], [[0.0, 0.0, 0.0, 0.0, 0.0]]]\n [[1.0], Float32[2.935012], [[0.0, 0.0, 0.0, 0.0, 0.0]]]\n [[1.0], Float32[3.5348382], [[0.0, 0.0, 0.0, 0.0, 0.0]]]\n [[1.0], Float32[3.9373996], [[0.0, 0.0, 0.0, 0.0, 0.0]]]\n\nVector{Vector}[[[1.0], Float32[3.351181], [[0.0, 0.0, 0.0, 0.0, 0.0]]], [[1.0], Float32[2.6405892], [[0.0, 0.0, 0.0, 0.0, 0.0]]], [[1.0], Float32[2.935012], [[0.0, 0.0, 0.0, 0.0, 0.0]]], [[1.0], Float32[3.5348382], [[0.0, 0.0, 0.0, 0.0, 0.0]]], [[1.0], Float32[3.9373996], [[0.0, 0.0, 0.0, 0.0, 0.0]]]]","category":"page"},{"location":"tutorials/evaluation/","page":"Evaluating Explanations","title":"Evaluating Explanations","text":"This leads us to our next topic: Performance Benchmarks.","category":"page"},{"location":"explanation/generators/generic/","page":"Generic","title":"Generic","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"explanation/generators/generic/#GenericGenerator","page":"Generic","title":"GenericGenerator","text":"","category":"section"},{"location":"explanation/generators/generic/","page":"Generic","title":"Generic","text":"We use the term generic to relate to the basic counterfactual generator proposed by Wachter, Mittelstadt, and Russell (2017) with L1-norm regularization. There is also a variant of this generator that uses the distance metric proposed in Wachter, Mittelstadt, and Russell (2017), which we call WachterGenerator.","category":"page"},{"location":"explanation/generators/generic/#Description","page":"Generic","title":"Description","text":"","category":"section"},{"location":"explanation/generators/generic/","page":"Generic","title":"Generic","text":"As the term indicates, this approach is simple: it forms the baseline approach for gradient-based counterfactual generators. Wachter, Mittelstadt, and Russell (2017) were among the first to realise that","category":"page"},{"location":"explanation/generators/generic/","page":"Generic","title":"Generic","text":"[…] explanations can, in principle, be offered without opening the “black box.”— Wachter, Mittelstadt, and Russell (2017)","category":"page"},{"location":"explanation/generators/generic/","page":"Generic","title":"Generic","text":"Gradient descent is performed directly in the feature space. Concerning the cost heuristic, the authors choose to penalize the distance of counterfactuals from the factual value. This is based on the intuitive notion that larger feature perturbations require greater effort.","category":"page"},{"location":"explanation/generators/generic/#Usage","page":"Generic","title":"Usage","text":"","category":"section"},{"location":"explanation/generators/generic/","page":"Generic","title":"Generic","text":"The approach can be used in our package as follows:","category":"page"},{"location":"explanation/generators/generic/","page":"Generic","title":"Generic","text":"generator = GenericGenerator()\nce = generate_counterfactual(x, target, counterfactual_data, M, generator)\nplot(ce)","category":"page"},{"location":"explanation/generators/generic/","page":"Generic","title":"Generic","text":"(Image: )","category":"page"},{"location":"explanation/generators/generic/#References","page":"Generic","title":"References","text":"","category":"section"},{"location":"explanation/generators/generic/","page":"Generic","title":"Generic","text":"Wachter, Sandra, Brent Mittelstadt, and Chris Russell. 2017. “Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR.” Harv. JL & Tech. 31: 841. https://doi.org/10.2139/ssrn.3063289.","category":"page"},{"location":"explanation/generators/greedy/","page":"Greedy","title":"Greedy","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"explanation/generators/greedy/#GreedyGenerator","page":"Greedy","title":"GreedyGenerator","text":"","category":"section"},{"location":"explanation/generators/greedy/","page":"Greedy","title":"Greedy","text":"We use the term greedy to describe the counterfactual generator introduced by Schut et al. (2021).","category":"page"},{"location":"explanation/generators/greedy/#Description","page":"Greedy","title":"Description","text":"","category":"section"},{"location":"explanation/generators/greedy/","page":"Greedy","title":"Greedy","text":"The Greedy generator works under the premise of generating realistic counterfactuals by minimizing predictive uncertainty. Schut et al. (2021) show that for models that incorporates predictive uncertainty in their predictions, maximizing the predictive probability corresponds to minimizing the predictive uncertainty: by construction, the generated counterfactual will therefore be realistic (low epistemic uncertainty) and unambiguous (low aleatoric uncertainty).","category":"page"},{"location":"explanation/generators/greedy/","page":"Greedy","title":"Greedy","text":"For the counterfactual search Schut et al. (2021) propose using a Jacobian-based Saliency Map Attack(JSMA). It is greedy in the sense that it is an “iterative algorithm that updates the most salient feature, i.e. the feature that has the largest influence on the classification, by delta at each step” (Schut et al. 2021).","category":"page"},{"location":"explanation/generators/greedy/#Usage","page":"Greedy","title":"Usage","text":"","category":"section"},{"location":"explanation/generators/greedy/","page":"Greedy","title":"Greedy","text":"The approach can be used in our package as follows:","category":"page"},{"location":"explanation/generators/greedy/","page":"Greedy","title":"Greedy","text":"M = fit_model(counterfactual_data, :DeepEnsemble)\ngenerator = GreedyGenerator()\nce = generate_counterfactual(x, target, counterfactual_data, M, generator)\nplot(ce)","category":"page"},{"location":"explanation/generators/greedy/","page":"Greedy","title":"Greedy","text":"(Image: )","category":"page"},{"location":"explanation/generators/greedy/#References","page":"Greedy","title":"References","text":"","category":"section"},{"location":"explanation/generators/greedy/","page":"Greedy","title":"Greedy","text":"Schut, Lisa, Oscar Key, Rory Mc Grath, Luca Costabello, Bogdan Sacaleanu, Yarin Gal, et al. 2021. “Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertainties.” In International Conference on Artificial Intelligence and Statistics, 1756–64. PMLR.","category":"page"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"explanation/generators/probe/#ProbeGenerator","page":"PROBE","title":"ProbeGenerator","text":"","category":"section"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"The ProbeGenerator is designed to navigate the trade-offs between costs and robustness in Algorithmic Recourse (Pawelczyk et al. 2022).","category":"page"},{"location":"explanation/generators/probe/#Description","page":"PROBE","title":"Description","text":"","category":"section"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"The goal of ProbeGenerator is to find a recourse x’ whose prediction at any point y within some set around x’ belongs to the positive class with probability 1 - r, where r is the recourse invalidation rate. It minimizes the gap between the achieved and desired recourse invalidation rates, minimizes recourse costs, and also ensures that the resulting recourse achieves a positive model prediction.","category":"page"},{"location":"explanation/generators/probe/#Explanation","page":"PROBE","title":"Explanation","text":"","category":"section"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"The loss function this generator is defined below. R is a hinge loss parameter which helps control for robustness. The loss and penalty functions can still be chosen freely.","category":"page"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"beginaligned\nR(x sigma^2 I) + l(f(x) s) + lambda d_c(x x)\nendaligned","category":"page"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"R uses the following formula to control for noise. It generates small perturbations and checks how often the counterfactual explanation flips back to a factual one, when small amounts of noise are added to it.","category":"page"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"beginaligned\nDelta(x^hatE) = E_varepsilonh(x^hatE) - h(x^hatE + varepsilon)\nendaligned","category":"page"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"The above formula is not differentiable. For this reason the generator uses the closed form version of the formula below.","category":"page"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"beginequation\nDelta tilde(x^hatE sigma^2 I) = 1 - Phi left(fracsqrtf(x^hatE)sqrtnabla f(x^hatE)^T sigma^2 I nabla f(x^hatE)right) \nendequation","category":"page"},{"location":"explanation/generators/probe/#Usage","page":"PROBE","title":"Usage","text":"","category":"section"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"Generating a counterfactual with the data loaded and generator chosen works as follows:","category":"page"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"Note: It is important to set the convergence to “:invalidation_rate” here.","category":"page"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"M = fit_model(counterfactual_data, :DeepEnsemble)\nopt = Descent(0.01)\ngenerator = CounterfactualExplanations.Generators.ProbeGenerator(opt=opt)\nconv = CounterfactualExplanations.Convergence.InvalidationRateConvergence(;invalidation_rate=0.5)\nce = generate_counterfactual(x, target, counterfactual_data, M, generator, convergence=conv)\nplot(ce)","category":"page"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"Choosing different invalidation rates makes the counterfactual more or less robust. The following plot shows the counterfactuals generated for different invalidation rates.","category":"page"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"(Image: )","category":"page"},{"location":"explanation/generators/probe/#References","page":"PROBE","title":"References","text":"","category":"section"},{"location":"explanation/generators/probe/","page":"PROBE","title":"PROBE","text":"Pawelczyk, Martin, Teresa Datta, Johannes van-den-Heuvel, Gjergji Kasneci, and Himabindu Lakkaraju. 2022. “Probabilistically Robust Recourse: Navigating the Trade-Offs Between Costs and Robustness in Algorithmic Recourse.” arXiv Preprint arXiv:2203.06768.","category":"page"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"explanation/generators/gravitational/#GravitationalGenerator","page":"Gravitational","title":"GravitationalGenerator","text":"","category":"section"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"The GravitationalGenerator was introduced in Altmeyer et al. (2023). It is named so because it generates counterfactuals that gravitate towards some sensible point in the target domain.","category":"page"},{"location":"explanation/generators/gravitational/#Description","page":"Gravitational","title":"Description","text":"","category":"section"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"Altmeyer et al. (2023) extend the general framework as follows,","category":"page"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"beginaligned\nmathbfs^prime = arg min_mathbfs^prime in mathcalS textyloss(M(f(mathbfs^prime))y^*) + lambda_1 textcost(f(mathbfs^prime)) + lambda_2 textextcost(f(mathbfs^prime)) \nendaligned ","category":"page"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"where textcost(f(mathbfs^prime)) denotes the proxy for costs faced by the individual. “The newly introduced term textextcost(f(mathbfs^prime)) is meant to capture and address external costs incurred by the collective of individuals in response to changes in mathbfs^prime.” (Altmeyer et al. 2023)","category":"page"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"For the GravitationalGenerator we have,","category":"page"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"beginaligned\ntextextcost(f(mathbfs^prime)) = textdist(f(mathbfs^prime)barx^*) \nendaligned","category":"page"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"where barx is some sensible point in the target domain, for example, the subsample average barx^*=textmean(x), x in mathcalD_1.","category":"page"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"There is a tradeoff then, between the distance of counterfactuals from their factual value and the chosen point in the target domain. The chart below illustrates how the counterfactual outcome changes as the penalty lambda_2 on the distance to the point in the target domain is increased from left to right (holding the other penalty term constant).","category":"page"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"(Image: )","category":"page"},{"location":"explanation/generators/gravitational/#Usage","page":"Gravitational","title":"Usage","text":"","category":"section"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"The approach can be used in our package as follows:","category":"page"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"generator = GravitationalGenerator()\nce = generate_counterfactual(x, target, counterfactual_data, M, generator)\ndisplay(plot(ce))","category":"page"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"(Image: )","category":"page"},{"location":"explanation/generators/gravitational/#Comparison-to-GenericGenerator","page":"Gravitational","title":"Comparison to GenericGenerator","text":"","category":"section"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"The figure below compares the outcome for the GenericGenerator and the GravitationalGenerator.","category":"page"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"(Image: )","category":"page"},{"location":"explanation/generators/gravitational/#References","page":"Gravitational","title":"References","text":"","category":"section"},{"location":"explanation/generators/gravitational/","page":"Gravitational","title":"Gravitational","text":"Altmeyer, Patrick, Giovan Angela, Aleksander Buszydlik, Karol Dobiczek, Arie van Deursen, and Cynthia Liem. 2023. “Endogenous Macrodynamics in Algorithmic Recourse.” In First IEEE Conference on Secure and Trustworthy Machine Learning. https://doi.org/10.1109/satml54575.2023.00036.","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"tutorials/benchmarking/#Performance-Benchmarks","page":"Benchmarking Explanations","title":"Performance Benchmarks","text":"","category":"section"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"In the previous tutorial, we have seen how counterfactual explanations can be evaluated. An important follow-up task is to compare the performance of different counterfactual generators is an important task. Researchers can use benchmarks to test new ideas they want to implement. Practitioners can find the right counterfactual generator for their specific use case through benchmarks. In this tutorial, we will see how to run benchmarks for counterfactual generators.","category":"page"},{"location":"tutorials/benchmarking/#Post-Hoc-Benchmarking","page":"Benchmarking Explanations","title":"Post Hoc Benchmarking","text":"","category":"section"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"We begin by continuing the discussion from the previous tutorial: suppose you have generated multiple counterfactual explanations for multiple individuals, like below:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"# Factual and target:\nn_individuals = 5\nids = rand(findall(predict_label(M, counterfactual_data) .== factual), n_individuals)\nxs = select_factual(counterfactual_data, ids)\nces = generate_counterfactual(xs, target, counterfactual_data, M, generator; num_counterfactuals=5)","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"You may be interested in comparing the outcomes across individuals. To benchmark the various counterfactual explanations using default evaluation measures, you can simply proceed as follows:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"bmk = benchmark(ces)","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Under the hood, the benchmark(counterfactual_explanations::Vector{CounterfactualExplanation}) uses CounterfactualExplanations.Evaluation.evaluate(ce::CounterfactualExplanation) to generate a Benchmark object, which contains the evaluation in its most granular form as a DataFrame.","category":"page"},{"location":"tutorials/benchmarking/#Working-with-Benchmarks","page":"Benchmarking Explanations","title":"Working with Benchmarks","text":"","category":"section"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"For convenience, the DataFrame containing the evaluation can be returned by simply calling the Benchmark object. By default, the aggregated evaluation measures across id (in line with the default behaviour of evaluate).","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"bmk()","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"15×7 DataFrame\n Row │ sample variable value generator ⋯\n │ Base.UUID String Float64 Symbol ⋯\n─────┼──────────────────────────────────────────────────────────────────────────\n 1 │ 239104d0-f59f-11ee-3d0c-d1db071927ff distance 3.17243 GradientBase ⋯\n 2 │ 239104d0-f59f-11ee-3d0c-d1db071927ff redundancy 0.0 GradientBase\n 3 │ 239104d0-f59f-11ee-3d0c-d1db071927ff validity 1.0 GradientBase\n 4 │ 2398b3e2-f59f-11ee-3323-13d53fb7e75b distance 3.07148 GradientBase\n 5 │ 2398b3e2-f59f-11ee-3323-13d53fb7e75b redundancy 0.0 GradientBase ⋯\n 6 │ 2398b3e2-f59f-11ee-3323-13d53fb7e75b validity 1.0 GradientBase\n 7 │ 2398b916-f59f-11ee-3f13-bd00858a39af distance 3.62159 GradientBase\n 8 │ 2398b916-f59f-11ee-3f13-bd00858a39af redundancy 0.0 GradientBase\n 9 │ 2398b916-f59f-11ee-3f13-bd00858a39af validity 1.0 GradientBase ⋯\n 10 │ 2398bce8-f59f-11ee-37c1-ef7c6de27b6b distance 2.62783 GradientBase\n 11 │ 2398bce8-f59f-11ee-37c1-ef7c6de27b6b redundancy 0.0 GradientBase\n 12 │ 2398bce8-f59f-11ee-37c1-ef7c6de27b6b validity 1.0 GradientBase\n 13 │ 2398c08a-f59f-11ee-175b-81c155750752 distance 2.91985 GradientBase ⋯\n 14 │ 2398c08a-f59f-11ee-175b-81c155750752 redundancy 0.0 GradientBase\n 15 │ 2398c08a-f59f-11ee-175b-81c155750752 validity 1.0 GradientBase\n 4 columns omitted","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"To retrieve the granular dataset, simply do:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"bmk(agg=nothing)","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"75×8 DataFrame\n Row │ sample num_counterfactual variable v ⋯\n │ Base.UUID Int64 String F ⋯\n─────┼──────────────────────────────────────────────────────────────────────────\n 1 │ 239104d0-f59f-11ee-3d0c-d1db071927ff 1 distance 3 ⋯\n 2 │ 239104d0-f59f-11ee-3d0c-d1db071927ff 2 distance 3\n 3 │ 239104d0-f59f-11ee-3d0c-d1db071927ff 3 distance 3\n 4 │ 239104d0-f59f-11ee-3d0c-d1db071927ff 4 distance 3\n 5 │ 239104d0-f59f-11ee-3d0c-d1db071927ff 5 distance 3 ⋯\n 6 │ 239104d0-f59f-11ee-3d0c-d1db071927ff 1 redundancy 0\n 7 │ 239104d0-f59f-11ee-3d0c-d1db071927ff 2 redundancy 0\n 8 │ 239104d0-f59f-11ee-3d0c-d1db071927ff 3 redundancy 0\n 9 │ 239104d0-f59f-11ee-3d0c-d1db071927ff 4 redundancy 0 ⋯\n 10 │ 239104d0-f59f-11ee-3d0c-d1db071927ff 5 redundancy 0\n 11 │ 239104d0-f59f-11ee-3d0c-d1db071927ff 1 validity 1\n ⋮ │ ⋮ ⋮ ⋮ ⋱\n 66 │ 2398c08a-f59f-11ee-175b-81c155750752 1 redundancy 0\n 67 │ 2398c08a-f59f-11ee-175b-81c155750752 2 redundancy 0 ⋯\n 68 │ 2398c08a-f59f-11ee-175b-81c155750752 3 redundancy 0\n 69 │ 2398c08a-f59f-11ee-175b-81c155750752 4 redundancy 0\n 70 │ 2398c08a-f59f-11ee-175b-81c155750752 5 redundancy 0\n 71 │ 2398c08a-f59f-11ee-175b-81c155750752 1 validity 1 ⋯\n 72 │ 2398c08a-f59f-11ee-175b-81c155750752 2 validity 1\n 73 │ 2398c08a-f59f-11ee-175b-81c155750752 3 validity 1\n 74 │ 2398c08a-f59f-11ee-175b-81c155750752 4 validity 1\n 75 │ 2398c08a-f59f-11ee-175b-81c155750752 5 validity 1 ⋯\n 5 columns and 54 rows omitted","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Since benchmarks return a DataFrame object on call, post-processing is straightforward. For example, we could use Tidier.jl:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"using Tidier\n@chain bmk() begin\n @filter(variable == \"distance\")\n @select(sample, variable, value)\nend","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"5×3 DataFrame\n Row │ sample variable value \n │ Base.UUID String Float64 \n─────┼─────────────────────────────────────────────────────────\n 1 │ 239104d0-f59f-11ee-3d0c-d1db071927ff distance 3.17243\n 2 │ 2398b3e2-f59f-11ee-3323-13d53fb7e75b distance 3.07148\n 3 │ 2398b916-f59f-11ee-3f13-bd00858a39af distance 3.62159\n 4 │ 2398bce8-f59f-11ee-37c1-ef7c6de27b6b distance 2.62783\n 5 │ 2398c08a-f59f-11ee-175b-81c155750752 distance 2.91985","category":"page"},{"location":"tutorials/benchmarking/#Metadata-for-Counterfactual-Explanations","page":"Benchmarking Explanations","title":"Metadata for Counterfactual Explanations","text":"","category":"section"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Benchmarks always report metadata for each counterfactual explanation, which is automatically inferred by default. The default metadata concerns the explained model and the employed generator. In the current example, we used the same model and generator for each individual:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"@chain bmk() begin\n @group_by(sample)\n @select(sample, model, generator)\n @summarize(model=first(model),generator=first(generator))\n @ungroup\nend","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"5×3 DataFrame\n Row │ sample model ⋯\n │ Base.UUID Symbol ⋯\n─────┼──────────────────────────────────────────────────────────────────────────\n 1 │ 239104d0-f59f-11ee-3d0c-d1db071927ff FluxModel(Chain(Dense(2 => 2)), … ⋯\n 2 │ 2398b3e2-f59f-11ee-3323-13d53fb7e75b FluxModel(Chain(Dense(2 => 2)), …\n 3 │ 2398b916-f59f-11ee-3f13-bd00858a39af FluxModel(Chain(Dense(2 => 2)), …\n 4 │ 2398bce8-f59f-11ee-37c1-ef7c6de27b6b FluxModel(Chain(Dense(2 => 2)), …\n 5 │ 2398c08a-f59f-11ee-175b-81c155750752 FluxModel(Chain(Dense(2 => 2)), … ⋯\n 1 column omitted","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Metadata can also be provided as an optional key argument.","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"meta_data = Dict(\n :generator => \"Generic\",\n :model => \"MLP\",\n)\nmeta_data = [meta_data for i in 1:length(ces)]\nbmk = benchmark(ces; meta_data=meta_data)\n@chain bmk() begin\n @group_by(sample)\n @select(sample, model, generator)\n @summarize(model=first(model),generator=first(generator))\n @ungroup\nend","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"5×3 DataFrame\n Row │ sample model generator \n │ Base.UUID String String \n─────┼─────────────────────────────────────────────────────────\n 1 │ 27fae496-f59f-11ee-2c30-f35d1025a6d4 MLP Generic\n 2 │ 27fdcc6a-f59f-11ee-030b-152c9794c5f1 MLP Generic\n 3 │ 27fdd04a-f59f-11ee-2010-e1732ff5d8d2 MLP Generic\n 4 │ 27fdd340-f59f-11ee-1d20-050a69dcacef MLP Generic\n 5 │ 27fdd5fc-f59f-11ee-02e8-d198e436abb3 MLP Generic","category":"page"},{"location":"tutorials/benchmarking/#Ad-Hoc-Benchmarking","page":"Benchmarking Explanations","title":"Ad Hoc Benchmarking","text":"","category":"section"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"So far we have assumed the following workflow:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Fit some machine learning model.\nGenerate counterfactual explanations for some individual(s) (generate_counterfactual).\nEvaluate and benchmark them (benchmark(ces::Vector{CounterfactualExplanation})).","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"In many cases, it may be preferable to combine these steps. To this end, we have added support for two scenarios of Ad Hoc Benchmarking.","category":"page"},{"location":"tutorials/benchmarking/#Pre-trained-Models","page":"Benchmarking Explanations","title":"Pre-trained Models","text":"","category":"section"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"In the first scenario, it is assumed that the machine learning models have been pre-trained and so the workflow can be summarized as follows:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Fit some machine learning model(s).\nGenerate counterfactual explanations and benchmark them.","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"We suspect that this is the most common workflow for practitioners who are interested in benchmarking counterfactual explanations for the pre-trained machine learning models. Let’s go through this workflow using a simple example. We first train some models and store them in a dictionary:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"models = Dict(\n :MLP => fit_model(counterfactual_data, :MLP),\n :Linear => fit_model(counterfactual_data, :Linear),\n)","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Next, we store the counterfactual generators of interest in a dictionary as well:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"generators = Dict(\n :Generic => GenericGenerator(),\n :Gravitational => GravitationalGenerator(),\n :Wachter => WachterGenerator(),\n :ClaPROAR => ClaPROARGenerator(),\n)","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Then we can run a benchmark for individual(s) x, a pre-specified target and counterfactual_data as follows:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"bmk = benchmark(x, target, counterfactual_data; models=models, generators=generators)","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"In this case, metadata is automatically inferred from the dictionaries:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"@chain bmk() begin\n @filter(variable == \"distance\")\n @select(sample, variable, value, model, generator)\nend","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"8×5 DataFrame\n Row │ sample variable value model ⋯\n │ Base.UUID String Float64 Tuple… ⋯\n─────┼──────────────────────────────────────────────────────────────────────────\n 1 │ 2cba5eee-f59f-11ee-1844-cbc7a8372a38 distance 4.38877 (:Linear, Flux ⋯\n 2 │ 2cd740fe-f59f-11ee-35c3-1157eb1b7583 distance 4.17021 (:Linear, Flux\n 3 │ 2cd741e2-f59f-11ee-2b09-0d55ef9892b9 distance 4.31145 (:Linear, Flux\n 4 │ 2cd7420c-f59f-11ee-1996-6fa75e23bb57 distance 4.17035 (:Linear, Flux\n 5 │ 2cd74234-f59f-11ee-0ad0-9f21949f5932 distance 5.73182 (:MLP, FluxMod ⋯\n 6 │ 2cd7425c-f59f-11ee-3eb4-af34f85ffd3d distance 5.50606 (:MLP, FluxMod\n 7 │ 2cd7427a-f59f-11ee-10d3-a1df6c8dc125 distance 5.2114 (:MLP, FluxMod\n 8 │ 2cd74298-f59f-11ee-32d1-f501c104fea8 distance 5.3623 (:MLP, FluxMod\n 2 columns omitted","category":"page"},{"location":"tutorials/benchmarking/#Everything-at-once","page":"Benchmarking Explanations","title":"Everything at once","text":"","category":"section"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Researchers, in particular, may be interested in combining all steps into one. This is the second scenario of Ad Hoc Benchmarking:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Fit some machine learning model(s), generate counterfactual explanations and benchmark them.","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"It involves calling benchmark directly on counterfactual data (the only positional argument):","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"bmk = benchmark(counterfactual_data)","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"This will use the default models from standard_models_catalogue and train them on the data. All available generators from generator_catalogue will also be used:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"@chain bmk() begin\n @filter(variable == \"validity\")\n @select(sample, variable, value, model, generator)\nend","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"200×5 DataFrame\n Row │ sample variable value model genera ⋯\n │ Base.UUID String Float64 Symbol Symbol ⋯\n─────┼──────────────────────────────────────────────────────────────────────────\n 1 │ 32d1817e-f59f-11ee-152f-a30b18c2e6f7 validity 1.0 Linear gravit ⋯\n 2 │ 32d1817e-f59f-11ee-152f-a30b18c2e6f7 validity 1.0 Linear growin\n 3 │ 32d1817e-f59f-11ee-152f-a30b18c2e6f7 validity 1.0 Linear revise\n 4 │ 32d1817e-f59f-11ee-152f-a30b18c2e6f7 validity 1.0 Linear clue\n 5 │ 32d1817e-f59f-11ee-152f-a30b18c2e6f7 validity 1.0 Linear probe ⋯\n 6 │ 32d1817e-f59f-11ee-152f-a30b18c2e6f7 validity 1.0 Linear dice\n 7 │ 32d1817e-f59f-11ee-152f-a30b18c2e6f7 validity 1.0 Linear clapro\n 8 │ 32d1817e-f59f-11ee-152f-a30b18c2e6f7 validity 1.0 Linear wachte\n 9 │ 32d1817e-f59f-11ee-152f-a30b18c2e6f7 validity 1.0 Linear generi ⋯\n 10 │ 32d1817e-f59f-11ee-152f-a30b18c2e6f7 validity 1.0 Linear greedy\n 11 │ 32d255e8-f59f-11ee-3e8d-a9e9f6e23ea8 validity 1.0 Linear gravit\n ⋮ │ ⋮ ⋮ ⋮ ⋮ ⋱\n 191 │ 3382d08a-f59f-11ee-10b3-f7d18cf7d3b5 validity 1.0 MLP gravit\n 192 │ 3382d08a-f59f-11ee-10b3-f7d18cf7d3b5 validity 1.0 MLP growin ⋯\n 193 │ 3382d08a-f59f-11ee-10b3-f7d18cf7d3b5 validity 1.0 MLP revise\n 194 │ 3382d08a-f59f-11ee-10b3-f7d18cf7d3b5 validity 1.0 MLP clue\n 195 │ 3382d08a-f59f-11ee-10b3-f7d18cf7d3b5 validity 1.0 MLP probe\n 196 │ 3382d08a-f59f-11ee-10b3-f7d18cf7d3b5 validity 1.0 MLP dice ⋯\n 197 │ 3382d08a-f59f-11ee-10b3-f7d18cf7d3b5 validity 1.0 MLP clapro\n 198 │ 3382d08a-f59f-11ee-10b3-f7d18cf7d3b5 validity 1.0 MLP wachte\n 199 │ 3382d08a-f59f-11ee-10b3-f7d18cf7d3b5 validity 1.0 MLP generi\n 200 │ 3382d08a-f59f-11ee-10b3-f7d18cf7d3b5 validity 1.0 MLP greedy ⋯\n 1 column and 179 rows omitted","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Optionally, you can instead provide a dictionary of models and generators as before. Each value in the models dictionary should be one of two things:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Either be an object M of type AbstractFittedModel that implements the Models.train method.\nOr a DataType that can be called on CounterfactualData to create an object M as in (a).","category":"page"},{"location":"tutorials/benchmarking/#Multiple-Datasets","page":"Benchmarking Explanations","title":"Multiple Datasets","text":"","category":"section"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Benchmarks are run on single instances of type CounterfactualData. This is our design choice for two reasons:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"We want to avoid the loops inside the benchmark method(s) from getting too nested and convoluted.\nWhile it is straightforward to infer metadata for models and generators, this is not the case for datasets.","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Fortunately, it is very easy to run benchmarks for multiple datasets anyway, since Benchmark instances can be concatenated. To see how, let’s consider an example involving multiple datasets, models and generators:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"# Data:\ndatasets = Dict(\n :moons => CounterfactualData(load_moons()...),\n :circles => CounterfactualData(load_circles()...),\n)\n\n# Models:\nmodels = Dict(\n :MLP => FluxModel,\n :Linear => Linear,\n)\n\n# Generators:\ngenerators = Dict(\n :Generic => GenericGenerator(),\n :Greedy => GreedyGenerator(),\n)","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"Then we can simply loop over the datasets and eventually concatenate the results like so:","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"using CounterfactualExplanations.Evaluation: distance_measures\nbmks = []\nfor (dataname, dataset) in datasets\n bmk = benchmark(dataset; models=models, generators=generators, measure=distance_measures)\n push!(bmks, bmk)\nend\nbmk = vcat(bmks[1], bmks[2]; ids=collect(keys(datasets)))","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"When ids are supplied, then a new id column is added to the evaluation data frame that contains unique identifiers for the different benchmarks. The optional idcol_name argument can be used to specify the name for that indicator column (defaults to \"dataset\"):","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"@chain bmk() begin\n @group_by(dataset, generator)\n @filter(model == :MLP)\n @filter(variable == \"distance_l1\")\n @summarize(L1_norm=mean(value))\n @ungroup\nend","category":"page"},{"location":"tutorials/benchmarking/","page":"Benchmarking Explanations","title":"Benchmarking Explanations","text":"4×3 DataFrame\n Row │ dataset generator L1_norm \n │ Symbol Symbol Float32 \n─────┼──────────────────────────────\n 1 │ moons Generic 1.56555\n 2 │ moons Greedy 0.819269\n 3 │ circles Generic 1.83524\n 4 │ circles Greedy 0.498953","category":"page"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"tutorials/whistle_stop/#Whistle-Stop-Tour","page":"Whiste-Stop Tour","title":"Whistle-Stop Tour","text":"","category":"section"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"In this tutorial, we will go through a slightly more complex example involving synthetic data. We will generate Counterfactual Explanations using different generators and visualize the results.","category":"page"},{"location":"tutorials/whistle_stop/#Data-and-Classifier","page":"Whiste-Stop Tour","title":"Data and Classifier","text":"","category":"section"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"# Choose some values for data and a model:\nn_dim = 2\nn_classes = 4\nn_samples = 400\nmodel_name = :MLP","category":"page"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"The code chunk below generates synthetic data and uses it to fit a classifier. The outcome variable counterfactual_data.y consists of 4 classes. The input data counterfactual_data.X consists of 2 features. We generate a total of 400 samples. On the model side, we have specified model_name = :MLP. The fit_model can be used to fit a number of default models.","category":"page"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"data = TaijaData.load_multi_class(n_samples)\ncounterfactual_data = DataPreprocessing.CounterfactualData(data...)\nM = fit_model(counterfactual_data, model_name)","category":"page"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"The chart below visualizes our data along with the model predictions. In particular, the contour indicates the predicted probabilities generated by our classifier. By default, these are the predicted probabilities for y=1, the first label. For multi-dimensional input data is compressed into two dimensions and the decision boundary is approximated using Nearest Neighbors (this is still somewhat experimental).","category":"page"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"plot(M, counterfactual_data)","category":"page"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"(Image: )","category":"page"},{"location":"tutorials/whistle_stop/#Counterfactual-Explanation","page":"Whiste-Stop Tour","title":"Counterfactual Explanation","text":"","category":"section"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"Next, we begin by specifying our target and factual label. We then draw a random sample from the non-target (factual) class.","category":"page"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"# Factual and target:\ntarget = 2\nfactual = 4\nchosen = rand(findall(predict_label(M, counterfactual_data) .== factual))\nx = select_factual(counterfactual_data, chosen)","category":"page"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"This sets the baseline for our counterfactual search: we plan to perturb the factual x to change the predicted label from y=4 to our target label target=2.","category":"page"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"Counterfactual generators accept several default parameters that can be used to adjust the counterfactual search at a high level: for example, a Flux.jl optimizer can be supplied to define how exactly gradient steps are performed. Importantly, one can also define the threshold probability at which the counterfactual search will converge. This relates to the probability predicted by the underlying black-box model, that the counterfactual belongs to the target class. A higher decision threshold typically prolongs the counterfactual search.","category":"page"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"# Search params:\ndecision_threshold = 0.75\nnum_counterfactuals = 3","category":"page"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"The code below runs the counterfactual search for each generator available in the generator_catalogue. In each case, we also call the generic plot() method on the generated instance of type CounterfactualExplanation. This generates a simple plot that visualizes the entire counterfactual path. The chart below shows the results for all counterfactual generators: Factual: 4 → Target: 2.","category":"page"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"ces = Dict()\nplts = []\nplottable_generators = filter(((k,v),) -> k ∉ [:growing_spheres, :feature_tweak], generator_catalogue)\n# Search:\nfor (key, Generator) in plottable_generators\n generator = Generator()\n ce = generate_counterfactual(\n x, target, counterfactual_data, M, generator;\n num_counterfactuals = num_counterfactuals,\n convergence=GeneratorConditionsConvergence(\n decision_threshold=decision_threshold\n )\n )\n ces[key] = ce\n plts = [plts..., plot(ce; title=key, colorbar=false)]\nend","category":"page"},{"location":"tutorials/whistle_stop/","page":"Whiste-Stop Tour","title":"Whiste-Stop Tour","text":"(Image: )","category":"page"},{"location":"how_to_guides/custom_models/","page":"... add custom models","title":"... add custom models","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"how_to_guides/custom_models/#How-to-add-Custom-Models","page":"... add custom models","title":"How to add Custom Models","text":"","category":"section"},{"location":"how_to_guides/custom_models/","page":"... add custom models","title":"... add custom models","text":"Adding custom models is possible and relatively straightforward, as we will demonstrate in this guide.","category":"page"},{"location":"how_to_guides/custom_models/#Custom-Models","page":"... add custom models","title":"Custom Models","text":"","category":"section"},{"location":"how_to_guides/custom_models/","page":"... add custom models","title":"... add custom models","text":"Apart from the default models you can use any arbitrary (differentiable) model and generate recourse in the same way as before. Only two steps are necessary to make your own Julia model compatible with this package:","category":"page"},{"location":"how_to_guides/custom_models/","page":"... add custom models","title":"... add custom models","text":"The model needs to be declared as a subtype of <:CounterfactualExplanations.Models.AbstractFittedModel.\nYou need to extend the functions CounterfactualExplanations.Models.logits and CounterfactualExplanations.Models.probs for your custom model.","category":"page"},{"location":"how_to_guides/custom_models/#How-FluxModel-was-added","page":"... add custom models","title":"How FluxModel was added","text":"","category":"section"},{"location":"how_to_guides/custom_models/","page":"... add custom models","title":"... add custom models","text":"To demonstrate how this can be done in practice, we will reiterate here how native support for Flux.jl models was enabled (Innes 2018). Once again we use synthetic data for an illustrative example. The code below loads the data and builds a simple model architecture that can be used for a multi-class prediction task. Note how outputs from the final layer are not passed through a softmax activation function, since the counterfactual loss is evaluated with respect to logits. The model is trained with dropout.","category":"page"},{"location":"how_to_guides/custom_models/","page":"... add custom models","title":"... add custom models","text":"# Data:\nN = 200\ndata = TaijaData.load_blobs(N; centers=4, cluster_std=0.5)\ncounterfactual_data = DataPreprocessing.CounterfactualData(data...)\ny = counterfactual_data.y\nX = counterfactual_data.X\n\n# Flux model setup: \nusing Flux\ndata = Flux.DataLoader((X,y), batchsize=1)\nn_hidden = 32\noutput_dim = size(y,1)\ninput_dim = 2\nactivation = σ\nmodel = Chain(\n Dense(input_dim, n_hidden, activation),\n Dropout(0.1),\n Dense(n_hidden, output_dim)\n) \nloss(x, y) = Flux.Losses.logitcrossentropy(model(x), y)\n\n# Flux model training:\nusing Flux.Optimise: update!, Adam\nopt = Adam()\nepochs = 50\nfor epoch = 1:epochs\n for d in data\n gs = gradient(Flux.params(model)) do\n l = loss(d...)\n end\n update!(opt, Flux.params(model), gs)\n end\nend","category":"page"},{"location":"how_to_guides/custom_models/","page":"... add custom models","title":"... add custom models","text":"The code below implements the two steps that were necessary to make Flux models compatible with the package. We first declare our new struct as a subtype of <:AbstractDifferentiableModel, which itself is an abstract subtype of <:AbstractFittedModel. Computing logits amounts to just calling the model on inputs. Predicted probabilities for labels can in this case be computed by passing predicted logits through the softmax function. Finally, we just instantiate our model in the same way as always.","category":"page"},{"location":"how_to_guides/custom_models/","page":"... add custom models","title":"... add custom models","text":"# Step 1)\nstruct MyFluxModel <: AbstractDifferentiableModel\n model::Any\n likelihood::Symbol\nend\n\n# Step 2)\n# import functions in order to extend\nimport CounterfactualExplanations.Models: logits\nimport CounterfactualExplanations.Models: probs \nlogits(M::MyFluxModel, X::AbstractArray) = M.model(X)\nprobs(M::MyFluxModel, X::AbstractArray) = softmax(logits(M, X))\nM = MyFluxModel(model, :classification_multi)","category":"page"},{"location":"how_to_guides/custom_models/","page":"... add custom models","title":"... add custom models","text":"The code below implements the counterfactual search and plots the results:","category":"page"},{"location":"how_to_guides/custom_models/","page":"... add custom models","title":"... add custom models","text":"factual_label = 4\ntarget = 2\nchosen = rand(findall(predict_label(M, counterfactual_data) .== factual_label))\nx = select_factual(counterfactual_data, chosen) \n\n# Counterfactual search:\ngenerator = GenericGenerator()\nce = generate_counterfactual(x, target, counterfactual_data, M, generator)\nplot(ce)","category":"page"},{"location":"how_to_guides/custom_models/","page":"... add custom models","title":"... add custom models","text":"(Image: )","category":"page"},{"location":"how_to_guides/custom_models/#References","page":"... add custom models","title":"References","text":"","category":"section"},{"location":"how_to_guides/custom_models/","page":"... add custom models","title":"... add custom models","text":"Innes, Mike. 2018. “Flux: Elegant Machine Learning with Julia.” Journal of Open Source Software 3 (25): 602. https://doi.org/10.21105/joss.00602.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"CurrentModule = CounterfactualExplanations","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"(Image: )","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Documentation for CounterfactualExplanations.jl.","category":"page"},{"location":"#CounterfactualExplanations","page":"🏠 Home","title":"CounterfactualExplanations","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Counterfactual Explanations and Algorithmic Recourse in Julia.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"(Image: Stable) (Image: Dev) (Image: Build Status) (Image: Coverage) (Image: Code Style: Blue) (Image: License) (Image: Package Downloads) (Image: Aqua QA)","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"CounterfactualExplanations.jl is a package for generating Counterfactual Explanations (CE) and Algorithmic Recourse (AR) for black-box algorithms. Both CE and AR are related tools for explainable artificial intelligence (XAI). While the package is written purely in Julia, it can be used to explain machine learning algorithms developed and trained in other popular programming languages like Python and R. See below for a short introduction and other resources or dive straight into the docs.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"There is also a corresponding paper, Explaining Black-Box Models through Counterfactuals, which has been published in JuliaCon Proceedings. Please consider citing the paper, if you use this package in your work:","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"(Image: DOI) (Image: DOI)","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"@article{Altmeyer2023,\n doi = {10.21105/jcon.00130},\n url = {https://doi.org/10.21105/jcon.00130},\n year = {2023},\n publisher = {The Open Journal},\n volume = {1},\n number = {1},\n pages = {130},\n author = {Patrick Altmeyer and Arie van Deursen and Cynthia C. s. Liem},\n title = {Explaining Black-Box Models through Counterfactuals},\n journal = {Proceedings of the JuliaCon Conferences}\n}","category":"page"},{"location":"#Installation","page":"🏠 Home","title":"🚩 Installation","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"You can install the stable release from Julia’s General Registry as follows:","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"using Pkg\nPkg.add(\"CounterfactualExplanations\")","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"CounterfactualExplanations.jl is under active development. To install the development version of the package you can run the following command:","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"using Pkg\nPkg.add(url=\"https://github.com/juliatrustworthyai/CounterfactualExplanations.jl\")","category":"page"},{"location":"#Background-and-Motivation","page":"🏠 Home","title":"🤔 Background and Motivation","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Machine learning models like Deep Neural Networks have become so complex, opaque and underspecified in the data that they are generally considered Black Boxes. Nonetheless, such models often play a key role in data-driven decision-making systems. This creates the following problem: human operators in charge of such systems have to rely on them blindly, while those individuals subject to them generally have no way of challenging an undesirable outcome:","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"“You cannot appeal to (algorithms). They do not listen. Nor do they bend.”— Cathy O’Neil in Weapons of Math Destruction, 2016","category":"page"},{"location":"#Enter:-Counterfactual-Explanations","page":"🏠 Home","title":"🔮 Enter: Counterfactual Explanations","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Counterfactual Explanations can help human stakeholders make sense of the systems they develop, use or endure: they explain how inputs into a system need to change for it to produce different decisions. Explainability benefits internal as well as external quality assurance.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Counterfactual Explanations have a few properties that are desirable in the context of Explainable Artificial Intelligence (XAI). These include:","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Full fidelity to the black-box model, since no proxy is involved.\nNo need for (reasonably) interpretable features as opposed to LIME and SHAP.\nClear link to Algorithmic Recourse and Causal Inference.\nLess susceptible to adversarial attacks than LIME and SHAP.","category":"page"},{"location":"#Example:-Give-Me-Some-Credit","page":"🏠 Home","title":"Example: Give Me Some Credit","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Consider the following real-world scenario: a retail bank is using a black-box model trained on their clients’ credit history to decide whether they will provide credit to new applicants. To simulate this scenario, we have pre-trained a binary classifier on the publicly available Give Me Some Credit dataset that ships with this package (Kaggle 2011).","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"The figure below shows counterfactuals for 10 randomly chosen individuals that would have been denied credit initially.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"(Image: )","category":"page"},{"location":"#Example:-MNIST","page":"🏠 Home","title":"Example: MNIST","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"The figure below shows a counterfactual generated for an image classifier trained on MNIST: in particular, it demonstrates which pixels need to change in order for the classifier to predict 3 instead of 8.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Since v0.1.9 counterfactual generators are fully composable. Here we have composed a generator that combines ideas from Wachter, Mittelstadt, and Russell (2017) and Altmeyer et al. (2023):","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"# Compose generator:\nusing CounterfactualExplanations.Objectives: distance_from_target\ngenerator = GradientBasedGenerator()\n@chain generator begin\n @objective logitcrossentropy + 0.1distance_mad + 0.1distance_from_target\n @with_optimiser Adam(0.1) \nend\ncounterfactual_data.generative_model = vae # assign generative model","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"(Image: )","category":"page"},{"location":"#Usage-example","page":"🏠 Home","title":"🔍 Usage example","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Generating counterfactuals will typically look like follows. Below we first fit a simple model to a synthetic dataset with linearly separable features and then draw a random sample:","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"# Data and Classifier:\ncounterfactual_data = CounterfactualData(load_linearly_separable()...)\nM = fit_model(counterfactual_data, :Linear)\n\n# Select random sample:\ntarget = 2\nfactual = 1\nchosen = rand(findall(predict_label(M, counterfactual_data) .== factual))\nx = select_factual(counterfactual_data, chosen)","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"To this end, we specify a counterfactual generator of our choice:","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"# Counterfactual search:\ngenerator = DiCEGenerator(λ=[0.1,0.3])","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Here, we have chosen to use the GradientBasedGenerator to move the individual from its factual label 1 to the target label 2.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"With all of our ingredients specified, we finally generate counterfactuals using a simple API call:","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"conv = conv = CounterfactualExplanations.Convergence.GeneratorConditionsConvergence()\nce = generate_counterfactual(\n x, target, counterfactual_data, M, generator; \n num_counterfactuals=3, convergence=conv,\n)","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"The plot below shows the resulting counterfactual path:","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"(Image: )","category":"page"},{"location":"#Implemented-Counterfactual-Generators","page":"🏠 Home","title":"☑️ Implemented Counterfactual Generators","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Currently, the following counterfactual generators are implemented:","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"ClaPROAR (Altmeyer et al. 2023)\nCLUE (Antorán et al. 2020)\nDiCE (Mothilal, Sharma, and Tan 2020)\nFeatureTweak (Tolomei et al. 2017)\nGeneric\nGravitationalGenerator (Altmeyer et al. 2023)\nGreedy (Schut et al. 2021)\nGrowingSpheres (Laugel et al. 2017)\nPROBE (Pawelczyk et al. 2023)\nREVISE (Joshi et al. 2019)\nWachter (Wachter, Mittelstadt, and Russell 2017)","category":"page"},{"location":"#Goals-and-limitations","page":"🏠 Home","title":"🎯 Goals and limitations","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"The goal of this library is to contribute to efforts towards trustworthy machine learning in Julia. The Julia language has an edge when it comes to trustworthiness: it is very transparent. Packages like this one are generally written in pure Julia, which makes it easy for users and developers to understand and contribute to open-source code. Eventually, this project aims to offer a one-stop-shop of counterfactual explanations.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Our ambition is to enhance the package through the following features:","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Support for all supervised machine learning models trained in MLJ.jl.\nSupport for regression models.","category":"page"},{"location":"#Contribute","page":"🏠 Home","title":"🛠 Contribute","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Contributions of any kind are very much welcome! Take a look at the issue to see what things we are currently working on. If you have an idea for a new feature or want to report a bug, please open a new issue.","category":"page"},{"location":"#Development","page":"🏠 Home","title":"Development","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"If your looking to contribute code, it may be helpful to check out the Explanation section of the docs.","category":"page"},{"location":"#Testing","page":"🏠 Home","title":"Testing","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Please always make sure to add tests for any new features or changes.","category":"page"},{"location":"#Documentation","page":"🏠 Home","title":"Documentation","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"If you add new features or change existing ones, please make sure to update the documentation accordingly. The documentation is written in Documenter.jl and is located in the docs/src folder.","category":"page"},{"location":"#Log-Changes","page":"🏠 Home","title":"Log Changes","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"As of version 1.1.1, we have tried to be more stringent about logging changes. Please make sure to add a note to the CHANGELOG.md file for any changes you make. It is sufficient to add a note under the Unreleased section.","category":"page"},{"location":"#General-Pointers","page":"🏠 Home","title":"General Pointers","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"There are also some general pointers for people looking to contribute to any of our Taija packages here.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Please follow the SciML ColPrac guide.","category":"page"},{"location":"#Citation","page":"🏠 Home","title":"🎓 Citation","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"If you want to use this codebase, please consider citing the corresponding paper:","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"@article{Altmeyer2023,\n doi = {10.21105/jcon.00130},\n url = {https://doi.org/10.21105/jcon.00130},\n year = {2023},\n publisher = {The Open Journal},\n volume = {1},\n number = {1},\n pages = {130},\n author = {Patrick Altmeyer and Arie van Deursen and Cynthia C. s. Liem},\n title = {Explaining Black-Box Models through Counterfactuals},\n journal = {Proceedings of the JuliaCon Conferences}\n}","category":"page"},{"location":"#References","page":"🏠 Home","title":"📚 References","text":"","category":"section"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Altmeyer, Patrick, Giovan Angela, Aleksander Buszydlik, Karol Dobiczek, Arie van Deursen, and Cynthia CS Liem. 2023. “Endogenous Macrodynamics in Algorithmic Recourse.” In 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), 418–31. IEEE.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Antorán, Javier, Umang Bhatt, Tameem Adel, Adrian Weller, and José Miguel Hernández-Lobato. 2020. “Getting a Clue: A Method for Explaining Uncertainty Estimates.” https://arxiv.org/abs/2006.06848.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Joshi, Shalmali, Oluwasanmi Koyejo, Warut Vijitbenjaronk, Been Kim, and Joydeep Ghosh. 2019. “Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems.” https://arxiv.org/abs/1907.09615.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Kaggle. 2011. “Give Me Some Credit, Improve on the State of the Art in Credit Scoring by Predicting the Probability That Somebody Will Experience Financial Distress in the Next Two Years.” https://www.kaggle.com/c/GiveMeSomeCredit; Kaggle. https://www.kaggle.com/c/GiveMeSomeCredit.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Laugel, Thibault, Marie-Jeanne Lesot, Christophe Marsala, Xavier Renard, and Marcin Detyniecki. 2017. “Inverse Classification for Comparison-Based Interpretability in Machine Learning.” https://arxiv.org/abs/1712.08443.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Mothilal, Ramaravind K, Amit Sharma, and Chenhao Tan. 2020. “Explaining Machine Learning Classifiers Through Diverse Counterfactual Explanations.” In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 607–17. https://doi.org/10.1145/3351095.3372850.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Pawelczyk, Martin, Teresa Datta, Johannes van-den-Heuvel, Gjergji Kasneci, and Himabindu Lakkaraju. 2023. “Probabilistically Robust Recourse: Navigating the Trade-Offs Between Costs and Robustness in Algorithmic Recourse.” https://arxiv.org/abs/2203.06768.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Schut, Lisa, Oscar Key, Rory Mc Grath, Luca Costabello, Bogdan Sacaleanu, Yarin Gal, et al. 2021. “Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertainties.” In International Conference on Artificial Intelligence and Statistics, 1756–64. PMLR.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Tolomei, Gabriele, Fabrizio Silvestri, Andrew Haines, and Mounia Lalmas. 2017. “Interpretable Predictions of Tree-Based Ensembles via Actionable Feature Tweaking.” In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 465–74. https://doi.org/10.1145/3097983.3098039.","category":"page"},{"location":"","page":"🏠 Home","title":"🏠 Home","text":"Wachter, Sandra, Brent Mittelstadt, and Chris Russell. 2017. “Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR.” Harv. JL & Tech. 31: 841. https://doi.org/10.2139/ssrn.3063289.","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"explanation/categorical/#Categorical-Features","page":"Categorical Features","title":"Categorical Features","text":"","category":"section"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"To illustrate how data is preprocessed under the hood, we consider a simple toy dataset with three categorical features (name, grade and sex) and one continuous feature (age):","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"X = (\n name=categorical([\"Danesh\", \"Lee\", \"Mary\", \"John\"]),\n grade=categorical([\"A\", \"B\", \"A\", \"C\"], ordered=true),\n sex=categorical([\"male\",\"female\",\"male\",\"male\"]),\n height=[1.85, 1.67, 1.5, 1.67],\n)\nschema(X)","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"Categorical features are expected to be one-hot or dummy encoded. To this end, we could use MLJ, for example:","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"hot = OneHotEncoder()\nmach = fit!(machine(hot, X))\nW = transform(mach, X)\nschema(W)","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"┌──────────────┬────────────┬─────────┐\n│ names │ scitypes │ types │\n├──────────────┼────────────┼─────────┤\n│ name__Danesh │ Continuous │ Float64 │\n│ name__John │ Continuous │ Float64 │\n│ name__Lee │ Continuous │ Float64 │\n│ name__Mary │ Continuous │ Float64 │\n│ grade__A │ Continuous │ Float64 │\n│ grade__B │ Continuous │ Float64 │\n│ grade__C │ Continuous │ Float64 │\n│ sex__female │ Continuous │ Float64 │\n│ sex__male │ Continuous │ Float64 │\n│ height │ Continuous │ Float64 │\n└──────────────┴────────────┴─────────┘","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"The matrix that will be perturbed during the counterfactual search looks as follows:","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"X = permutedims(MLJBase.matrix(W))","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"10×4 Matrix{Float64}:\n 1.0 0.0 0.0 0.0\n 0.0 0.0 0.0 1.0\n 0.0 1.0 0.0 0.0\n 0.0 0.0 1.0 0.0\n 1.0 0.0 1.0 0.0\n 0.0 1.0 0.0 0.0\n 0.0 0.0 0.0 1.0\n 0.0 1.0 0.0 0.0\n 1.0 0.0 1.0 1.0\n 1.85 1.67 1.5 1.67","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"The CounterfactualData constructor takes two optional arguments that can be used to specify the indices of categorical and continuous features. If nothing is supplied, all features are assumed to be continuous. For categorical features, the constructor expects and array of arrays of integers (Vector{Vector{Int}}) where each subarray includes the indices of a all one-hot encoded rows related to a single categorical feature. In the example above, the name feature is one-hot encoded across rows 1, 2 and 3 of X.","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"features_categorical = [\n [1,2,3,4], # name\n [5,6,7], # grade\n [8,9] # sex\n]\nfeatures_continuous = [10]","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"We propose the following simple logic for reconstructing categorical encodings after perturbations:","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"For one-hot encoded features with multiple classes, choose the maximum.\nFor binary features, clip the perturbed value to fall into 01 and round to the nearest of the two integers.","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"function reconstruct_cat_encoding(x)\n map(features_categorical) do cat_group_index\n if length(cat_group_index) > 1\n x[cat_group_index] = Int.(x[cat_group_index] .== maximum(x[cat_group_index]))\n if sum(x[cat_group_index]) > 1\n ties = findall(x[cat_group_index] .== 1)\n _x = zeros(length(x[cat_group_index]))\n winner = rand(ties,1)[1]\n _x[winner] = 1\n x[cat_group_index] = _x\n end\n else\n x[cat_group_index] = [round(clamp(x[cat_group_index][1],0,1))]\n end\n end\n return x\nend","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"Let’s look at a few simple examples to see how this function works. Firstly, consider the case of perturbing a single element:","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"x = X[:,1]\nx[1] = 1.1\nx","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"10-element Vector{Float64}:\n 1.1\n 0.0\n 0.0\n 0.0\n 1.0\n 0.0\n 0.0\n 0.0\n 1.0\n 1.85","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"The reconstructed one-hot-encoded vector will look like this:","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"reconstruct_cat_encoding(x)","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"10-element Vector{Float64}:\n 1.0\n 0.0\n 0.0\n 0.0\n 1.0\n 0.0\n 0.0\n 0.0\n 1.0\n 1.85","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"Next, consider the case of perturbing multiple elements:","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"x[2] = 1.1\nx[3] = -1.2\nx","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"10-element Vector{Float64}:\n 1.0\n 1.1\n -1.2\n 0.0\n 1.0\n 0.0\n 0.0\n 0.0\n 1.0\n 1.85","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"The reconstructed one-hot-encoded vector will look like this:","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"reconstruct_cat_encoding(x)","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"10-element Vector{Float64}:\n 0.0\n 1.0\n 0.0\n 0.0\n 1.0\n 0.0\n 0.0\n 0.0\n 1.0\n 1.85","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"Finally, let’s introduce a tie:","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"x[1] = 1.0\nx","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"10-element Vector{Float64}:\n 1.0\n 1.0\n 0.0\n 0.0\n 1.0\n 0.0\n 0.0\n 0.0\n 1.0\n 1.85","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"The reconstructed one-hot-encoded vector will look like this:","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"reconstruct_cat_encoding(x)","category":"page"},{"location":"explanation/categorical/","page":"Categorical Features","title":"Categorical Features","text":"10-element Vector{Float64}:\n 0.0\n 1.0\n 0.0\n 0.0\n 1.0\n 0.0\n 0.0\n 0.0\n 1.0\n 1.85","category":"page"},{"location":"extensions/neurotree/","page":"NeuroTrees","title":"NeuroTrees","text":"CurrentModule = CounterfactualExplanations ","category":"page"},{"location":"extensions/neurotree/#[NeuroTreeModels.jl](https://evovest.github.io/NeuroTreeModels.jl/dev/)","page":"NeuroTrees","title":"NeuroTreeModels.jl","text":"","category":"section"},{"location":"extensions/neurotree/","page":"NeuroTrees","title":"NeuroTrees","text":"NeuroTreeModels.jl is a package that provides a framework for training differentiable tree-based models. This is relevant to the work on counterfactual explanations (CE), which often assumes that the underlying black-box model is differentiable with respect to its input. The literature on CE therefore regularly focuses exclusively on explaining deep learning models. This is at odds with the fact that the literature also typically focuses on tabular data, which is often best modeled by tree-based models (Grinsztajn, Oyallon, and Varoquaux 2022). The extension for NeuroTreeModels.jl provides a way to bridge this gap by allowing users to apply existing gradient-based CE methods to differentiable tree-based models.","category":"page"},{"location":"extensions/neurotree/","page":"NeuroTrees","title":"NeuroTrees","text":"warning: Experimental Feature\nPlease note that this extension is still experimental. Neither the behaviour of differentiable tree-based models nor their interplay with counterfactual explanations is well understood at this point. If you encounter any issues, please report them to the package maintainers. Your feedback is highly appreciated.Please also note that this extension is only tested on Julia 1.9 and higher, due to compatibility issues.","category":"page"},{"location":"extensions/neurotree/#Example","page":"NeuroTrees","title":"Example","text":"","category":"section"},{"location":"extensions/neurotree/","page":"NeuroTrees","title":"NeuroTrees","text":"The extension will be loaded automatically when loading the NeuroTreeModels package (assuming the CounterfactualExplanations package is also loaded).","category":"page"},{"location":"extensions/neurotree/","page":"NeuroTrees","title":"NeuroTrees","text":"using NeuroTreeModels","category":"page"},{"location":"extensions/neurotree/","page":"NeuroTrees","title":"NeuroTrees","text":"Next, we will fit a NeuroTree model to the moons dataset using our standard package API for doing so.","category":"page"},{"location":"extensions/neurotree/","page":"NeuroTrees","title":"NeuroTrees","text":"# Fit model to data:\ndata = CounterfactualData(load_moons()...)\nM = fit_model(\n data, :NeuroTree; \n depth=2, lr=5e-2, nrounds=50, batchsize=10\n)","category":"page"},{"location":"extensions/neurotree/","page":"NeuroTrees","title":"NeuroTrees","text":"NeuroTreeExt.NeuroTreeModel(NeuroTreeRegressor(loss = mlogloss, …), :classification_multi, NeuroTreeModels.NeuroTreeModel{NeuroTreeModels.MLogLoss, Chain{Tuple{BatchNorm{typeof(identity), Vector{Float32}, Float32, Vector{Float32}}, NeuroTreeModels.StackTree}}}(NeuroTreeModels.MLogLoss, Chain(BatchNorm(2, active=false), NeuroTreeModels.StackTree(NeuroTree[NeuroTree{Matrix{Float32}, Vector{Float32}, Array{Float32, 3}}(Float32[1.8824593 -0.28222033; -2.680499 0.67347014; … ; -1.0722864 1.3651229; -2.0926774 1.63557], Float32[-3.4070241, 4.545113, 1.0882677, -0.3497498, -2.766766, 1.9072449, -0.9736261, 3.9750721, 1.726214, 3.7279263 … -0.0664266, -0.4214582, -2.3816268, -3.1371245, 0.76548636, 2.636373, 2.4558601, 0.893434, -1.9484522, 4.793434], Float32[3.44271 -6.334693 -0.6308845 3.385659; -3.4316056 6.297003 0.7254221 -3.3283486;;; -3.7011054 -0.17596768 0.15429471 2.270125; 3.4926674 0.026218029 -0.19753197 -2.2337704;;; 1.1795454 -4.315231 0.28486454 1.9995956; -0.9651108 4.0999455 -0.05312265 -1.8039354;;; … ;;; 2.5076811 -0.46358463 -3.5438805 0.0686823; -2.592356 0.47884527 3.781507 -0.022692114;;; -0.59115165 -3.234046 0.09896194 2.375202; 0.5592871 3.3082843 -0.014032216 -2.1876256;;; 2.039389 -0.10134532 2.6637273 -4.999703; -2.0289893 0.3368772 -2.5739825 5.069934], tanh)])), Dict{Symbol, Any}(:feature_names => [:x1, :x2], :nrounds => 50, :device => :cpu)))","category":"page"},{"location":"extensions/neurotree/","page":"NeuroTrees","title":"NeuroTrees","text":"Finally, we select a factual instance and generate a counterfactual explanation for it using the generic gradient-based CE method.","category":"page"},{"location":"extensions/neurotree/","page":"NeuroTrees","title":"NeuroTrees","text":"# Select a factual instance:\ntarget = 1\nfactual = 0\nchosen = rand(findall(predict_label(M, data) .== factual))\nx = select_factual(data, chosen)\n\n# Generate counterfactual explanation:\nη = 0.01\ngenerator = GenericGenerator(; opt=Descent(η), λ=0.01)\nconv = CounterfactualExplanations.Convergence.DecisionThresholdConvergence(;\n decision_threshold=0.9, max_iter=100\n)\nce = generate_counterfactual(x, target, data, M, generator; convergence=conv)\nplot(ce, alpha=0.1)","category":"page"},{"location":"extensions/neurotree/","page":"NeuroTrees","title":"NeuroTrees","text":"(Image: )","category":"page"},{"location":"extensions/neurotree/#References","page":"NeuroTrees","title":"References","text":"","category":"section"},{"location":"extensions/neurotree/","page":"NeuroTrees","title":"NeuroTrees","text":"Grinsztajn, Léo, Edouard Oyallon, and Gaël Varoquaux. 2022. “Why Do Tree-Based Models Still Outperform Deep Learning on Tabular Data?” https://arxiv.org/abs/2207.08815.","category":"page"}] } diff --git a/dev/tutorials/benchmarking/index.html b/dev/tutorials/benchmarking/index.html index 36f16a731..d78edc9f4 100644 --- a/dev/tutorials/benchmarking/index.html +++ b/dev/tutorials/benchmarking/index.html @@ -180,4 +180,4 @@ 1 │ moons Generic 1.56555 2 │ moons Greedy 0.819269 3 │ circles Generic 1.83524 - 4 │ circles Greedy 0.498953 + 4 │ circles Greedy 0.498953 diff --git a/dev/tutorials/convergence/index.html b/dev/tutorials/convergence/index.html index 13f9771bc..287deac06 100644 --- a/dev/tutorials/convergence/index.html +++ b/dev/tutorials/convergence/index.html @@ -10,4 +10,4 @@ for (ce, titl) in zip([ce_gen, ce_dec, ce_max], ["Gradient Convergence", "Decision Threshold Convergence", "Max Iterations Convergence"]) push!(plts, plot(ce; title=titl, cbar=false)) end -plot(plts..., layout=(1,3), size=(1200, 380))

References

Altmeyer, Patrick, Mojtaba Farmanbar, Arie van Deursen, and Cynthia CS Liem. 2024. “Faithful Model Explanations Through Energy-Constrained Conformal Counterfactuals.” In Proceedings of the AAAI Conference on Artificial Intelligence, 38:10829–37. 10.

+plot(plts..., layout=(1,3), size=(1200, 380))

References

Altmeyer, Patrick, Mojtaba Farmanbar, Arie van Deursen, and Cynthia CS Liem. 2024. “Faithful Model Explanations Through Energy-Constrained Conformal Counterfactuals.” In Proceedings of the AAAI Conference on Artificial Intelligence, 38:10829–37. 10.

diff --git a/dev/tutorials/data_catalogue/index.html b/dev/tutorials/data_catalogue/index.html index 94d8bdea9..87749bfdd 100644 --- a/dev/tutorials/data_catalogue/index.html +++ b/dev/tutorials/data_catalogue/index.html @@ -38,4 +38,4 @@ 10 10 10

We can also use a helper function to split the data into train and test sets:

train_data, test_data = 
-    CounterfactualExplanations.DataPreprocessing.train_test_split(counterfactual_data)
+ CounterfactualExplanations.DataPreprocessing.train_test_split(counterfactual_data) diff --git a/dev/tutorials/data_preprocessing/index.html b/dev/tutorials/data_preprocessing/index.html index 2d69ca48c..423e3f096 100644 --- a/dev/tutorials/data_preprocessing/index.html +++ b/dev/tutorials/data_preprocessing/index.html @@ -90,4 +90,4 @@ ce = generate_counterfactual(x, target, counterfactual_data, M, generator)

The resulting counterfactual path is shown in the chart below. Since only the first feature can be perturbed, the sample can only move along the horizontal axis.

plot(ce)

<!– ## Domain constraints &#10;In some cases, we may also want to constrain the domain of some feature. For example, age as a feature is constrained to a range from 0 to some upper bound corresponding perhaps to the average life expectancy of humans. Below, for example, we impose an upper bound of $0.5$ for our two features. &#10;```{.julia} counterfactualdata.mutability = [:both, :both] counterfactualdata.domain = [(0,0) for var in counterfactualdata.featurescontinuous]

&#10;This results in the counterfactual path shown below: since features are not allowed to be perturbed beyond the upper bound, the resulting counterfactual falls just short of the threshold probability $\gamma$.
 &#10;```{.julia}
 ce = generate_counterfactual(x, target, counterfactual_data, M, generator)
-plot(ce)

–>

+plot(ce)

–>

diff --git a/dev/tutorials/evaluation/index.html b/dev/tutorials/evaluation/index.html index f1a1f3d81..e41b7ba7f 100644 --- a/dev/tutorials/evaluation/index.html +++ b/dev/tutorials/evaluation/index.html @@ -42,4 +42,4 @@ [[1.0], Float32[3.5348382], [[0.0, 0.0, 0.0, 0.0, 0.0]]] [[1.0], Float32[3.9373996], [[0.0, 0.0, 0.0, 0.0, 0.0]]] -Vector{Vector}[[[1.0], Float32[3.351181], [[0.0, 0.0, 0.0, 0.0, 0.0]]], [[1.0], Float32[2.6405892], [[0.0, 0.0, 0.0, 0.0, 0.0]]], [[1.0], Float32[2.935012], [[0.0, 0.0, 0.0, 0.0, 0.0]]], [[1.0], Float32[3.5348382], [[0.0, 0.0, 0.0, 0.0, 0.0]]], [[1.0], Float32[3.9373996], [[0.0, 0.0, 0.0, 0.0, 0.0]]]]

This leads us to our next topic: Performance Benchmarks.

+Vector{Vector}[[[1.0], Float32[3.351181], [[0.0, 0.0, 0.0, 0.0, 0.0]]], [[1.0], Float32[2.6405892], [[0.0, 0.0, 0.0, 0.0, 0.0]]], [[1.0], Float32[2.935012], [[0.0, 0.0, 0.0, 0.0, 0.0]]], [[1.0], Float32[3.5348382], [[0.0, 0.0, 0.0, 0.0, 0.0]]], [[1.0], Float32[3.9373996], [[0.0, 0.0, 0.0, 0.0, 0.0]]]]

This leads us to our next topic: Performance Benchmarks.

diff --git a/dev/tutorials/generators/index.html b/dev/tutorials/generators/index.html index 48c62292a..da48e6747 100644 --- a/dev/tutorials/generators/index.html +++ b/dev/tutorials/generators/index.html @@ -22,4 +22,4 @@ :greedy => GreedyGenerator

To specify the type of generator you want to use, you can simply instantiate it:

# Search:
 generator = GenericGenerator()
 ce = generate_counterfactual(x, target, counterfactual_data, M, generator)
-plot(ce)

We generally make an effort to follow the literature as closely as possible when implementing off-the-shelf generators.

References

Altmeyer, Patrick, Giovan Angela, Aleksander Buszydlik, Karol Dobiczek, Arie van Deursen, and Cynthia Liem. 2023. “Endogenous Macrodynamics in Algorithmic Recourse.” In First IEEE Conference on Secure and Trustworthy Machine Learning. https://doi.org/10.1109/satml54575.2023.00036.

Joshi, Shalmali, Oluwasanmi Koyejo, Warut Vijitbenjaronk, Been Kim, and Joydeep Ghosh. 2019. “Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems.” https://arxiv.org/abs/1907.09615.

Mothilal, Ramaravind K, Amit Sharma, and Chenhao Tan. 2020. “Explaining Machine Learning Classifiers Through Diverse Counterfactual Explanations.” In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 607–17. https://doi.org/10.1145/3351095.3372850.

+plot(ce)

We generally make an effort to follow the literature as closely as possible when implementing off-the-shelf generators.

References

Altmeyer, Patrick, Giovan Angela, Aleksander Buszydlik, Karol Dobiczek, Arie van Deursen, and Cynthia Liem. 2023. “Endogenous Macrodynamics in Algorithmic Recourse.” In First IEEE Conference on Secure and Trustworthy Machine Learning. https://doi.org/10.1109/satml54575.2023.00036.

Joshi, Shalmali, Oluwasanmi Koyejo, Warut Vijitbenjaronk, Been Kim, and Joydeep Ghosh. 2019. “Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems.” https://arxiv.org/abs/1907.09615.

Mothilal, Ramaravind K, Amit Sharma, and Chenhao Tan. 2020. “Explaining Machine Learning Classifiers Through Diverse Counterfactual Explanations.” In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 607–17. https://doi.org/10.1145/3351095.3372850.

diff --git a/dev/tutorials/index.html b/dev/tutorials/index.html index 4497dd439..fd439caa8 100644 --- a/dev/tutorials/index.html +++ b/dev/tutorials/index.html @@ -1,2 +1,2 @@ -Overview · CounterfactualExplanations.jl
GitHub

Tutorials

In this section, you will find a series of tutorials that should help you gain a basic understanding of Conformal Prediction and how to apply it in Julia using this package.

Tutorials are lessons that take the reader by the hand through a series of steps to complete a project of some kind. Tutorials are learning-oriented.

— Diátaxis

In other words, you come here because you are new to this topic and are looking for a first peek at the methodology and code 🫣.

+Overview · CounterfactualExplanations.jl
GitHub

Tutorials

In this section, you will find a series of tutorials that should help you gain a basic understanding of Conformal Prediction and how to apply it in Julia using this package.

Tutorials are lessons that take the reader by the hand through a series of steps to complete a project of some kind. Tutorials are learning-oriented.

— Diátaxis

In other words, you come here because you are new to this topic and are looking for a first peek at the methodology and code 🫣.

diff --git a/dev/tutorials/model_catalogue/index.html b/dev/tutorials/model_catalogue/index.html index 6fc81fdd4..dce18fb38 100644 --- a/dev/tutorials/model_catalogue/index.html +++ b/dev/tutorials/model_catalogue/index.html @@ -44,4 +44,4 @@ - alpha: 0.5 - tree_type: binary - rng: MersenneTwister(123, (0, 9018, 8016, 884)) -, …), :classification_multi)

The tunable parameters for the EvoTreeModel can be found from the documentation of the EvoTrees.jl package under the EvoTreeClassifier section.

Please note that support for counterfactual generation with both LaplaceReduxModel and EvoTreeModel is not yet fully implemented.

References

Lakshminarayanan, Balaji, Alexander Pritzel, and Charles Blundell. 2016. “Simple and Scalable Predictive Uncertainty Estimation Using Deep Ensembles.” https://arxiv.org/abs/1612.01474.

LeCun, Yann. 1998. “The MNIST Database of Handwritten Digits.”

+, …), :classification_multi)

The tunable parameters for the EvoTreeModel can be found from the documentation of the EvoTrees.jl package under the EvoTreeClassifier section.

Please note that support for counterfactual generation with both LaplaceReduxModel and EvoTreeModel is not yet fully implemented.

References

Lakshminarayanan, Balaji, Alexander Pritzel, and Charles Blundell. 2016. “Simple and Scalable Predictive Uncertainty Estimation Using Deep Ensembles.” https://arxiv.org/abs/1612.01474.

LeCun, Yann. 1998. “The MNIST Database of Handwritten Digits.”

diff --git a/dev/tutorials/models/index.html b/dev/tutorials/models/index.html index dfd24769a..c1c2bd8e7 100644 --- a/dev/tutorials/models/index.html +++ b/dev/tutorials/models/index.html @@ -41,4 +41,4 @@ Epoch 80 avg_loss(data) = 0.011847609f0 Epoch 100 -avg_loss(data) = 0.007242911f0

To prepare the fitted model for use with our package, we need to wrap it inside a container. For plain-vanilla models trained in Flux.jl, the corresponding constructor is called FluxModel. There is also a separate constructor called FluxEnsemble, which applies to Deep Ensembles. Deep Ensembles are a popular approach to approximate Bayesian Deep Learning and have been shown to generate good predictive uncertainty estimates (Lakshminarayanan, Pritzel, and Blundell 2016).

The appropriate API call to wrap our simple network in a container follows below:

M = FluxModel(nn)
FluxModel(Chain(Dense(2 => 32, relu), Dropout(0.1, active=false), Dense(32 => 2)), :classification_binary)

The likelihood function of the output variable is automatically inferred from the data. The generic plot() method can be called on the model and data to visualise the results:

plot(M, counterfactual_data)

Our model M is now ready for use with the package.

References

Lakshminarayanan, Balaji, Alexander Pritzel, and Charles Blundell. 2016. “Simple and Scalable Predictive Uncertainty Estimation Using Deep Ensembles.” https://arxiv.org/abs/1612.01474.

+avg_loss(data) = 0.007242911f0

To prepare the fitted model for use with our package, we need to wrap it inside a container. For plain-vanilla models trained in Flux.jl, the corresponding constructor is called FluxModel. There is also a separate constructor called FluxEnsemble, which applies to Deep Ensembles. Deep Ensembles are a popular approach to approximate Bayesian Deep Learning and have been shown to generate good predictive uncertainty estimates (Lakshminarayanan, Pritzel, and Blundell 2016).

The appropriate API call to wrap our simple network in a container follows below:

M = FluxModel(nn)
FluxModel(Chain(Dense(2 => 32, relu), Dropout(0.1, active=false), Dense(32 => 2)), :classification_binary)

The likelihood function of the output variable is automatically inferred from the data. The generic plot() method can be called on the model and data to visualise the results:

plot(M, counterfactual_data)

Our model M is now ready for use with the package.

References

Lakshminarayanan, Balaji, Alexander Pritzel, and Charles Blundell. 2016. “Simple and Scalable Predictive Uncertainty Estimation Using Deep Ensembles.” https://arxiv.org/abs/1612.01474.

diff --git a/dev/tutorials/parallelization/index.html b/dev/tutorials/parallelization/index.html index d543a7eb7..49ea49727 100644 --- a/dev/tutorials/parallelization/index.html +++ b/dev/tutorials/parallelization/index.html @@ -219,4 +219,4 @@ bmk = benchmark(counterfactual_data; parallelizer=parallelizer) -MPI.Finalize()

The file can be executed from the command line as follows:

mpiexecjl --project -n 4 julia -e 'include("docs/src/srcipts/mpi.jl")'
+MPI.Finalize()

The file can be executed from the command line as follows:

mpiexecjl --project -n 4 julia -e 'include("docs/src/srcipts/mpi.jl")'
diff --git a/dev/tutorials/simple_example/index.html b/dev/tutorials/simple_example/index.html index 08fb92d95..dda608b2e 100644 --- a/dev/tutorials/simple_example/index.html +++ b/dev/tutorials/simple_example/index.html @@ -10,4 +10,4 @@ x = select_factual(counterfactual_data, chosen)

Finally, we generate and visualize the generated counterfactual:

# Search:
 generator = WachterGenerator()
 ce = generate_counterfactual(x, target, counterfactual_data, M, generator)
-plot(ce)

+plot(ce)

diff --git a/dev/tutorials/whistle_stop/index.html b/dev/tutorials/whistle_stop/index.html index 690e31798..62ea85c44 100644 --- a/dev/tutorials/whistle_stop/index.html +++ b/dev/tutorials/whistle_stop/index.html @@ -26,4 +26,4 @@ ) ces[key] = ce plts = [plts..., plot(ce; title=key, colorbar=false)] -end

+end