Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Basic rewrite of the package 2023 edition Part II: Location-Scale Families #50

Closed
wants to merge 214 commits into from
Closed

Basic rewrite of the package 2023 edition Part II: Location-Scale Families #50

wants to merge 214 commits into from

Conversation

Red-Portal
Copy link
Member

This is a partial pull request of #45.
(The original PR is now severely outdated.)

The content of Part II is as follows:

  • Add the location-scale variational family
  • Add constructors for Gaussian variational families.
  • Optimize AD paths for the location-scale family. (Refer to the original PR for benchmarks.)

@Red-Portal
refactor ADVI, change gradient operation interface
b49cf3e
@Red-Portal
remove unused file, remove unused dependency
88e0b79
@Red-Portal
fix ADVI elbo computation more efficiently
c2fb3f8
@Red-Portal
fix missing entropy regularization term
83161fd
@Red-Portal
add LogDensityProblem interface
efa8106
@Red-Portal
refactor use bijectors directly instead of transformed distributions
4ae2fbf 
This is to avoid having to reconstruct transformed distributions all
the time. The direct use of bijectors also avoids going through lots
of abstraction layers that could break.

Instead, transformed distributions could be constructed only once when
returing the VI result.
@Red-Portal
Merge branch 'master' of https://github.com/TuringLang/AdvancedVI.jl …
2bf2a42 
…into rewriting_advancedvi
@Red-Portal
fix type restrictions
1cadb51
@Red-Portal
remove unused file
3474e8d
@Red-Portal
fix use of with_logabsdet_jacobian
03a2767
@Red-Portal
restructure project; move the main VI routine to its own file
09c44fb
@Red-Portal
remove redundant import
b7407ce
@Red-Portal
restructure project into more modular objective estimators
4040149
@Red-Portal
migrate to AbstractDifferentiation
2a4514e
@Red-Portal
add location scale pre-packaged variational family, add functors
93a16d8
@Red-Portal
Revert "migrate to AbstractDifferentiation"
2b6e9eb 
This reverts commit 2a4514e.
@Red-Portal
fix use optimized MvNormal specialization, add logpdf for Loc.Scale.
1bfec36
@Red-Portal
remove dead code
1003606
@Red-Portal
fix location-scale logpdf
60a9987 
- Full Monte Carlo ELBO estimation now works. I checked.
@Red-Portal
add sticking-the-landing (STL) estimator
cd84f02
@Red-Portal
migrate to Optimisers.jl
768641b
@Red-Portal
remove execution time measurement (replace later with somethin else)
ca02fa3
@Red-Portal
fix use multiple dispatch for deciding whether to stop entropy grad.
a48377f
@Red-Portal
add termination decision, callback arguments
0b40ccf
@Red-Portal
add Base.show to modules
21db3fb
@Red-Portal
add interface calling restructure, rename rebuild -> restructure
25c51b4
@Red-Portal
add estimator state interface, add control variate interface to ADVI
fc20046
@Red-Portal
fix show(advi) to show control variate
6faa807
@Red-Portal
fix simplify show(advi.control_variate)
7095d27
@Red-Portal
fix type piracy by wrapping location-scale bijected distribution
9169ae2
Red-Portal and others added 24 commits October 23, 2023 01:31
@Red-Portal
add compat entry for stdlib in test/
7a92708
@Red-Portal
fix rng argument position in tests
5dd434d
@Red-Portal
refactor change name of inference test
a764d9b
@Red-Portal
fix documentation for optimize
8af8a5f
@Red-Portal
refactor rewrite the documentation for the global interfaces
5f1fb52
@Red-Portal
fix compat error
2491c64
@Red-Portal
fix documentation for optimize to be single line
92d1489
@Red-Portal
refactor remove begin end for one-liner
a03e955
@Red-Portal
refactor create unified interface for estimating objectives
ff83c03
@Red-Portal
refactor unify interface for entropy estimator, fix advi docs
aecc655
@Red-Portal
fix STL estimator to use manually stopped gradients instead
a8d532a
@Red-Portal
add inference test for a non-bijector model
65e9b12
@Red-Portal
refactor add indirections to handle STL and bijectors in ADVI
3691f16
@Red-Portal
refactor split inference tests for advi+distributionsad
a063583
@Red-Portal
refactor rename advi to repgradelbo and not use bijectors directly
316b629
@Red-Portal
fix documentation for estimate_objective
13b2088
@Red-Portal
refactor add indirection in repgradelbo for interacting with q
b0e1be1
@Red-Portal
add TransformedDistribution support as extension
7361ed4
@Red-Portal @torfjelde
Update src/objectives/elbo/repgradelbo.jl
d2e7614 
Co-authored-by: Tor Erlend Fjelde <[email protected]>
@Red-Portal
fix docstring for entropy estimator
77686b5
@Red-Portal
fix reparam_with_entropy specialization for bijectors
8461b43
@Red-Portal
Merge branch 'rewriting_advancedvi_optimize' of github.com:Red-Portal…
8c559e3 
…/AdvancedVI.jl into rewriting_advancedvi_optimize
@Red-Portal
add location scale family
8a17b22
@Red-Portal
refactor test with bijector use locscale, enable reverssediff
862876e
@Red-Portal
Copy link
Member Author

Sorry I messed up the git commit history. I'll reopen it after fixing this.

@Red-Portal Red-Portal closed this Dec 9, 2023
@Red-Portal Red-Portal deleted the rewriting_advancedvi_locscale branch December 9, 2023 08:10
@yebai yebai mentioned this pull request Dec 22, 2023
Closed
13 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yebai yebai Awaiting requested review from yebai

@torfjelde torfjelde Awaiting requested review from torfjelde

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Red-Portal @yebai