Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Developer continuous #84

Merged
merged 73 commits into from
Jun 7, 2024
Merged

Developer continuous #84

merged 73 commits into from
Jun 7, 2024

Conversation

mpielies
Copy link
Member

No description provided.

ri-heme and others added 30 commits January 5, 2023 16:26
@ri-heme
🔊 Capture warnings
50bfa61
@ri-heme
🐛 VAE and data objects handle missing data type
63f7c2f
@ri-heme
🚧 Update tasks for missing data type
5c0c90b
@mpielies @ri-heme
🚧 Committing provisional changes (not finished)
d831a49
@mpielies @ri-heme
🚧 min max feature pertubation
b5e1044 
- new preprocessing
- work on cont. associations
@mpielies @ri-heme
Last batch of changes (unfinished)
ff8f8aa
@mpielies @ri-heme
preprocessing changes
26bb304
@mpielies @ri-heme
Version that reaches training
aa28897
@mpielies @ri-heme
last version
0826212
@mpielies @ri-heme
Changes on writing the final files
98a5dfa
@mpielies @ri-heme
Last commit:
c838976 
- checking different config settings.

Highlights:
- preprocessing.py: feature_min_max function
-perturbations.py:
perturb...extended:
change target_dataset feature by feature for min/max
- identify_associations.py: branched flow depending on target_value
@mpielies @ri-heme
# Added plus_std and minus_std as target values
9d560bc 
- *preprocessing.py:* feature_min_max to feature_stats (added std)
-* perturbations.py:* added std (1 for now) feature by feature

-*identify_associations.py:*
added predefined list of cont target values
@mpielies @ri-heme
New changes for plotting new features
081e737
@mpielies @ri-heme
Note in funtion description updated
85ce21f
@mpielies @ri-heme
Perturbation visualization added:
337ddc5 
It is used in:
- encode data (show before and after preprocessing)
- identify associations (plot each feature after perturbation)
@mpielies @ri-heme
:build: Perturbation visualization
c7b4f1f
@mpielies @ri-heme
🎨 ✨
f02c63e 
Reorganizing the file identify_associations:
- ttest and bayes functions put outside
- dataloader preparation defined in functions
- save results function added
- single identify_associations function
- Most comments addressed
@mpielies @ri-heme
📝 Make output_subpath optional
5248274 
Reformat constant CONTINUOUS_TARGET_VALUES
@ri-heme
⏪ Revert styling in conflicting files
79b2849
@ri-heme
🎨 Rename results directory
1036656
@mpielies @ri-heme
🎨
71dd894 
Reused code put in main function:
Identify associations

Working branch for both modes (Continuous assoc finds self correlations)
@mpielies @ri-heme
✨ Random continuous dataset added.
c9cf123 
- Tested bayes and ttest on new data
- Added config files for continuous test
TODO:
Create folder with files to create synthetic datasets
@ri-heme
🎨 Type hinting
a3ddf48
@ri-heme
✨ Update ID associations module
1c64573 
- Add GPU compatibility
- Fix typing
- Make test dataloader batch size configurable
- Add extra columns to output table
@ri-heme
🎨 ♻️ Refactoring/styling
59a6d7a
@ri-heme
🔧 Update default config
0530139
@mpielies
✨ 🚧 🧐
6fd7589 
- Exploring bayes behaviour on continuous pert
- Plot feature associations and vae architecture as graphs
- Use VSCode debugger (json edited)
@mpielies
Added random_basic config files
c83fb42
@mpielies
🚧
3300a86 
Added ks method to calculate distances
@mpielies
✨ 🚧
2cac930 
Functioning Kolmogorov Smirnov method

- QC of features is given as separate csv from KS scores
- schema: feature names of feature to visualize added
- Basic visualization functions added to dataset_distribution.py
Henry added 3 commits May 16, 2024 18:09
Henry added 4 commits May 31, 2024 16:45
ri-heme
ri-heme requested changes Jun 3, 2024
View reviewed changes
Copy link
Collaborator

@ri-heme ri-heme left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the help, Henry!

tutorial/config/task/random_continuous__id_assoc_ks.yaml Outdated Show resolved Hide resolved
tutorial/config/task/random_small__latent.yaml Outdated Show resolved Hide resolved
src/move/visualization/style.py Outdated Show resolved Hide resolved
ri-heme
ri-heme reviewed Jun 4, 2024
View reviewed changes
setup.cfg Outdated Show resolved Hide resolved
Henry and others added 11 commits June 4, 2024 15:19
⚡ reduce action runtime by configuring tasks
09ca32f
@ri-heme
🎨 Make sure correct colormap is used
6a02d0e
@ri-heme
Merge branch 'developer-continuous-v3' of https://github.com/Rasmusse…
2a12ed1 
…nLab/MOVE into developer-continuous-v3
🔧 udpate flake8 configuration according to defaults
2e1c6ba 
https://black.readthedocs.io/en/stable/guides/using_black_with_other_tools.html#flake8
✨ dry-run continuous configuration sample data
6207350 
- play also with sample data configuration
🎨 make line 88 characters long, remove some expections from flake8
e4e2283
🐛 error due to black formatting?
5e23447 
- f-string formatting error.
✨ run continous example
7796f5e 
- check if less latent dimensions reduces the t-test runtime
🐛 four latent dimensions are required for t-test
48a76b7
⚡ run latent job, try to increase speed for t-test
5533eca
⚡ speed up and balance both jobs' runtime
d0a4409
@enryH enryH merged commit 6f5737a into developer Jun 7, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@enryH enryH enryH left review comments

@ri-heme ri-heme ri-heme requested changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Perturbations with continuous data
3 participants
@mpielies @enryH @ri-heme