Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Jacobian adjustment #265

Open
avehtari opened this issue Apr 5, 2024 · 0 comments
Open

Add support for Jacobian adjustment #265

avehtari opened this issue Apr 5, 2024 · 0 comments

Comments

@avehtari
Copy link
Collaborator

avehtari commented Apr 5, 2024

Related to discussion in a Stan discourse post

ROS book and some Stan discourse answers show how to make Jacobian adjustment for elpd_loo values if different models use different transformations of the continuous target. I had tested that a simple approach of adjusting the pointwise values as

loo_2_with_jacobian <- loo_2
loo_2_with_jacobian$pointwise[,1] <- loo_2_with_jacobian$pointwise[,1] - log(2*sqrt(data$y))

works when comparing two models. In case of more than two models, the summary values in $estimates slot are used to sort the models, which causes problems if the order is different after Jacobian adjustment. A quick fix is to update the summary, too

loo_2_with_jacobian$estimates["elpd_loo",] <- loo:::table_of_estimates(loo_2_with_jacobian$pointwise[,"elpd_loo", drop=FALSE])

It would be better to make a function that would properly update all the slots, or probably even better to add jacobian argument to loo() function.

This update would not be big, doesn't cause backward compatibility issues, and would make the life of users easier

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant