Changes to openff-qcsubmit to use next QCFractal

[dotsdl]

Description

The next major iteration of QCFractal is coming soon, and these changes are those necessary for QCSubmit to be used with it.

• Easy (straightforward/syntax) fixes in test_submissions
• Figure out how to wire pcm back in in test_submissions.py::test_optimization_submissions_with_pcm
• Easy (straightforward/syntax) fixes in test_datasets
• Hard (info content/architecture) fixes in test_datasets
• Easy (straightforward/syntax) fixes in tests/results/
  • Note: An example of quadrupole info is in records_and_molecules[0][0].properties['scf quadrupole'] after calling records_and_molecules = collection.to_records() on the COH single point set
• Why is test_results.py::test_collection_from_server[BasicResultCollection-OpenFF BCC Refit Study COH v1.0-spec_1-91-191] returning 190 results when it used to return 191? The spec names were lost and one of the two specs had an errored calc
• Figure out a new way for maxiter and scf keywords to get in submissions. Previously these were added by the QCSpec class from common_structures (which has values for those as defaults)
• Figure out how to do torsiondrive mocking in test_results.py::test_to_records[collection2-record2]
• Re-add find_existing to _BaseDataset.submit (datasets.py around line 220) (or open an issue to do in a separate PR)
• Un-ignore tests/results
• Add pre-commit config back
• Rename specs in single point datasets (cc Some datasets have placeholder spec names after migration MolSSI/QCFractal#739)
• Is s99f now unloadable via openmmforcefields?

Release notes

• Note that QCF record IDs now fail equality checks with str, but succeed with int.
• QCSubmit's dataset.get_qc_spec method is gone now, including its kwarg keyword_id (it's not clear to me what this means or where it's used)
• OptimizationDataset driver is now deferred (used to be gradient)
• pcm is currently unstable, so the test is marked as xfail
• Many tests now just run short jobs instead of mocking data from the QCA server
• cmiles is now found on all entries created in new submissions
• All to_records methods now warn about entries with invalid cmiles and then exclude them from the returned values.
• OptimizationDataset.to_basic_result_collection will now warn if it's unable to find results with the desired driver and will exclude them from the returned values.

Dev install instructions

mamba env create -n qcsubmit-dev -f basic.yaml
mamba install -c conda-forge -c openeye openeye-toolkits jupyterlab
mamba install -c conda-forge -c conda-forge/label/libint_dev psi4

In a different folder

git clone -b next [email protected]:MolSSI/QCFractal.git # really important to check out this branch directly, otherwise you'll get a bunch of empty folders preventing pip installation
cd QCFractal
pip install ./qcportal ./qcarchivetesting ./qcfractalcompute ./qcfractal

To prevent posgresql's "not enough shared memory" issue:

sudo sysctl -w kern.sysv.shmall=65536
sudo sysctl -w kern.sysv.shmmax=16777216

Finally in the openff-qcsubmit folder

pip install -e ./

When running pytest, sometimes I'll get a postgres issue about a temporary file path being too long. I can fix that by setting --basetemp in the pytest invocation to something short-ish:

pytest openff/qcsubmit/tests/test_submissions.py --basetemp="/Users/jeffreywagner/temp"  -vvvvv

QCF next docs: https://molssi.github.io/QCFractal/user_guide/records/base.html#qcportal.dataset_models.BaseDataset.submit

Status

• Ready to go

[lgtm-com]

This pull request introduces 1 alert when merging ce56294 into fff7590 - view on LGTM.com

new alerts:

• 1 for Unused import

[codecov]

Codecov Report

Merging #195 (73136c3) into main (c4d3313) will increase coverage by 0.20%.
Report is 2 commits behind head on main.
The diff coverage is 95.55%.

Additional details and impacted files

[dotsdl]

Live testing with this PR: openforcefield/qca-dataset-submission-next-test#4

[lgtm-com]

This pull request introduces 1 alert when merging 20ee05e into fff7590 - view on LGTM.com

new alerts:

• 1 for Unused import

[lgtm-com]

This pull request introduces 1 alert when merging c926347 into fff7590 - view on LGTM.com

new alerts:

• 1 for Unused import

[lgtm-com]

This pull request introduces 1 alert when merging 64dbf12 into fff7590 - view on LGTM.com

new alerts:

• 1 for Unused import

[lgtm-com]

This pull request introduces 1 alert when merging e3b15e6 into fff7590 - view on LGTM.com

new alerts:

• 1 for Unused import

[lgtm-com]

This pull request introduces 1 alert when merging f36c9e2 into fff7590 - view on LGTM.com

new alerts:

• 1 for Unused import

[lgtm-com]

This pull request introduces 1 alert when merging 667ed24 into fff7590 - view on LGTM.com

new alerts:

• 1 for Unused import

[lgtm-com]

This pull request introduces 1 alert when merging e6147ab into fff7590 - view on LGTM.com

new alerts:

• 1 for Unused import

[lgtm-com]

This pull request introduces 1 alert when merging 0956d90 into fff7590 - view on LGTM.com

new alerts:

• 1 for Unused import

[lgtm-com]

This pull request introduces 1 alert when merging e44eaa9 into fff7590 - view on LGTM.com

new alerts:

• 1 for Unused import

[lgtm-com]

This pull request introduces 2 alerts when merging f142658 into e4d5add - view on LGTM.com

new alerts:

• 2 for Unused import

[lgtm-com]

This pull request introduces 3 alerts when merging e2eaf09 into e4d5add - view on LGTM.com

new alerts:

• 3 for Unused import

[lgtm-com]

This pull request introduces 3 alerts when merging 727c8f9 into e4d5add - view on LGTM.com

new alerts:

• 3 for Unused import

[lgtm-com]

This pull request introduces 3 alerts when merging a85d635 into e4d5add - view on LGTM.com

new alerts:

• 3 for Unused import

[j-wags]

Gonna go ahead and merge and make an 0.50.0rc1 release, then will build a non-main label package to make sure things don't go sideways!

[jthorton]

Thanks so much, everyone, fantastic to qcsumit back online!