- new preprocessing
- work on cont. associations

- checking different config settings.

Highlights:
- preprocessing.py: feature_min_max function
-perturbations.py:
perturb...extended:
change target_dataset feature by feature for min/max
- identify_associations.py: branched flow depending on target_value

- *preprocessing.py:* feature_min_max to feature_stats (added std)
-* perturbations.py:* added std (1 for now) feature by feature

-*identify_associations.py:*
added predefined list of cont target values

It is used in:
- encode data (show before and after preprocessing)
- identify associations (plot each feature after perturbation)

Reorganizing the file identify_associations:
- ttest and bayes functions put outside
- dataloader preparation defined in functions
- save results function added
- single identify_associations function
- Most comments addressed

Reformat constant CONTINUOUS_TARGET_VALUES

Reused code put in main function:
Identify associations

Working branch for both modes (Continuous assoc finds self correlations)

- Tested bayes and ttest on new data
- Added config files for continuous test
TODO:
Create folder with files to create synthetic datasets

- Add GPU compatibility
- Fix typing
- Make test dataloader batch size configurable
- Add extra columns to output table

- Exploring bayes behaviour on continuous pert
- Plot feature associations and vae architecture as graphs
- Use VSCode debugger (json edited)

Added ks method to calculate distances

Functioning Kolmogorov Smirnov method

- QC of features is given as separate csv from KS scores
- schema: feature names of feature to visualize added
- Basic visualization functions added to dataset_distribution.py

Updated identify associations:
- Latest KS version

New folder: supplementary_material
- Create datasets py file
- Performance evaluation py file

Clean MOVE continuous branch
- To ease eventual merging with main
- Add tutorial files for continuous perturbations
- Append content to supplementary material folder:
Random continuous dataset (originally called realistic) added.
Final predictions saved for 150 feature model.
This is to run performance_evaluation directly.

Clean code to merge with developer:
- Add hotfix changes for plotting
- Uncouple visualize latent and VAE from id_assoc_ks.
- id_assoc_ks stores latent repr before and after pert. in 3D npy file
- 2 new files in supplementary_files: 3D_latent and VAE visualization.

- Relocated get_2nd... function to analysis - metrics .
- Generalized node colormap for graph visualization
- Enforced MOVE default style
- Add some type hints and docstrings

Allowing multiple hidden layers when tuning reconstruction:
- Solved parsing issue with results.tsv file's title

If a row (sample) had all its values as NaNs, this would eliminate it from the metrics output. This caused a mismatch in # of labels/scores.

- 🐛 fix incomplete publishing package file
- 🎨 format src files

- set max-line-length to 120

- in case of no categorical variables, the error surfaced: input_config has to be used!

double check type hints
- also FloatArray is not very informative about the layout of the array (2D, 3D, 4D?)

…nLab/MOVE into developer-continuous-v3

https://black.readthedocs.io/en/stable/guides/using_black_with_other_tools.html#flake8

- play also with sample data configuration

- f-string formatting error.

- check if less latent dimensions reduces the t-test runtime