Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Complete removal of "core" from the idiomatic API #314

Merged
merged 27 commits into from
Sep 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
93f8484
added http-error handling to api cmder; added apicmd equality to astr…
hemidactylus Sep 10, 2024
99ffd5a
Database has switched to APICommander
hemidactylus Sep 10, 2024
9a5f119
wip through collections
hemidactylus Sep 11, 2024
cb3435e
collections.py is core-free and tested
hemidactylus Sep 12, 2024
d47cbdf
sync cursor is core-free
hemidactylus Sep 12, 2024
a8d220a
cursors are core-free (no prefetch support)
hemidactylus Sep 12, 2024
db9b7f0
remove unused error recast-decorators
hemidactylus Sep 12, 2024
9563d26
wip on full decoupling of defaults, constants, utils
hemidactylus Sep 12, 2024
b490f94
user-agent and payload transformation libs brought to idiomatic
hemidactylus Sep 12, 2024
7b9402d
request tools is core-less. Only ops remain now
hemidactylus Sep 12, 2024
21cff34
mirrored exception tree for devops; WIP, started core-freeing admin
hemidactylus Sep 12, 2024
52b9567
admin part I: fetch db info uses apicommander
hemidactylus Sep 13, 2024
9e20f99
api_commander got additional_path param in request methods
hemidactylus Sep 13, 2024
09157ee
wip before cleaning AstraDBDatabaseAdmin
hemidactylus Sep 13, 2024
c8556be
admin is core-free
hemidactylus Sep 13, 2024
191f23e
removed the last recast-decorators. Idiomatic is fully core-free
hemidactylus Sep 13, 2024
b7a0769
core is completely sealed away
hemidactylus Sep 13, 2024
60bf7b9
deprecation of core (with a test thereof); update README changesfile,…
hemidactylus Sep 16, 2024
1a7f4c6
more linter-induced style fixes
hemidactylus Sep 16, 2024
5f51191
replace collections.abc.Iterator with typing.Iterator for subscriptab…
hemidactylus Sep 17, 2024
15e1eb5
update README; update AstraDBDatabaseAdmin unit testing; add DataAPID…
hemidactylus Sep 17, 2024
5e0a04d
basic unit test for apicommander
hemidactylus Sep 17, 2024
d2f1f4c
remove TRACE logging level; reduce code dup in apicommander
hemidactylus Sep 17, 2024
f4a49be
unit tests for APICommander
hemidactylus Sep 17, 2024
89ebb26
add test of insert_many with empty doc list
hemidactylus Sep 17, 2024
f94fca5
final changesfile
hemidactylus Sep 18, 2024
b413894
Merge branch 'SL-database-uses-api-commander' of https://github.com/d…
hemidactylus Sep 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions CHANGES
Original file line number Diff line number Diff line change
@@ -1,3 +1,27 @@
master
======
Full removal of "core library" from the current API:
- DevOps API accessed through APICommander everywhere
- Admin objects use APICommander consistently
- [Async]Database and [Async]Collection directly use APICommander
- Cursor library uses APICommander directly
- Core library imports triggers a submodule-wide deprecation warning
Widened exception hierarchy with:
- DevOpsAPIHttpException
- DevOpsAPITimeoutException
- DevOpsAPIFaultyResponseException
Rearrangement into separate modules for:
- constants, strings, magic numbers and settings
- request low-level tools
- payload/response transformations
- (sometimes with temporary duplication to avoid depending on 'core')
Testing:
- added tests for APICommander
- improved tests for admin classes
Logging of API requests made more uniform and easier to read
Replaced collections.abc.Iterator => typing.Iterator for python3.8 compatibility


v. 1.4.2
========
Method 'update_one' of [Async]Collection: now invokes the corresponding API command.
Expand Down
36 changes: 25 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,8 +81,8 @@ for result in cursor:
Next steps:

- More info and usage patterns are given in the docstrings of classes and methods
- [Data API reference](https://docs.datastax.com/en/astra/astra-db-vector/api-reference/overview.html)
- [AstraPy reference](https://docs.datastax.com/en/astra/astra-db-vector/api-reference/dataapiclient.html)
- [Data API reference](https://docs.datastax.com/en/astra-db-serverless/api-reference/overview.html)
- [AstraPy reference](https://docs.datastax.com/en/astra-api-docs/_attachments/python-client/astrapy/index.html)
- Package on [PyPI](https://pypi.org/project/astrapy/)

### Usage with HCD and other non-Astra installations
Expand Down Expand Up @@ -251,19 +251,19 @@ naming convention and module structure).

Tests are grouped in three _blocks_ (in as many subdirs of `tests/`):

- **core**: pre-1.0 classes
- **idiomatic**: all 1.0+ classes and APIs, except...
- **vectorize**: ... everything making use of `$vectorize` (within the idiomatic classes)
- _(core: pre-1.0 classes). Frozen as of v1.5, deprecated for removal in v2.0_

Actually, for convenience, _sub-blocks_ of tests are considered:

- **core regular**: everything except DevOps interactions
- **core ops**: core DevOps operations
- **idiomatic regular**: everything except the admin parts
- **idiomatic admin Astra**: the Astra-specific admin operations
- **idiomatic admin nonAstra**: the nonAstra-specific admin operations
- **vectorize in-depth**: many Data API interactions for a single choice of provider/model. This is mostly test the client
- **vectorize all-providers**: a slightly more shallow test repeated for all providers, models, auth methods etc. This is mostly testing the API
- _(core regular: everything except DevOps interactions)_
- _(core ops: core DevOps operations)_

Tests can be run on three types of Data API _targets_ (with slight differences in what is applicable):

Expand Down Expand Up @@ -373,10 +373,10 @@ Remove logging noise with:
poetry run pytest [...] -o log_cli=0
```

Increase logging level to `TRACE` (i.e. level `5`):
Increase logging level to `DEBUG` (i.e. level `10`):

```
poetry run pytest [...] -o log_cli=1 --log-cli-level=5
poetry run pytest [...] -o log_cli=1 --log-cli-level=10
```

Do not drop collections (valid for core):
Expand Down Expand Up @@ -488,7 +488,10 @@ from astrapy.exceptions import (
DeleteManyException,
DevOpsAPIErrorDescriptor,
DevOpsAPIException,
DevOpsAPIFaultyResponseException,
DevOpsAPIHttpException,
DevOpsAPIResponseException,
DevOpsAPITimeoutException,
InsertManyException,
TooManyDocumentsToCountException,
UpdateManyException,
Expand Down Expand Up @@ -538,12 +541,24 @@ from astrapy.cursors import (

### Appendix B: compatibility with pre-1.0.0 library

If your code uses the pre-1.0.0 astrapy (i.e. `from astrapy.db import Database, Collection` and so on) you are strongly advised to migrate to the current API.
If your code still uses the pre-1.0.0 astrapy (i.e. `from astrapy.db import AstraDB, AstraDBCollection` and so on)
you are strongly advised to migrate to the current API, which has more capabilities and improved interfaces.

All of the astrapy pre-1.0 API (now dubbed "core") works throughout *astrapy v1*, albeit with a deprecation warning
on astrapy v. 1.5.

Version 1.5 (the first to not wrap internally "core" as the engine of its own "idiomatic" API) introduces
several deprecation notices (nothing is retired yet), including a submodule-wide deprecation of "core".

**Version 2 of astrapy will finally remove "core" entirely (along with a few other things).**

#### v1 is fully compatible with "core", i.e. with pre-1.0.0

That being said, there are no known breakings of backward compatibility:
**legacy code would run with a newest astrapy version just as well.**
Here is a recap of the minor changes that came _to the old API_ with 1.0.0 (and beyond):
**legacy code would run with astrapy v1 just as well**
Here is a recap of the minor changes that came _to the old API_ with 1.0.0 (and beyond, up to 1.5):

- added a submodule-wide deprecation warning of the whole "core" library (v 1.5)
- added 'options' parameter to [Async]AstraDBCollection.update_one (v. 1.4.2+)
- prefetched find iterators: fix second-thread hangups in some cases (v. 1.4.2+)
- Added support for null tokens (with the effect of no authentication/token header in requests)
Expand Down Expand Up @@ -571,4 +586,3 @@ Here is a recap of the minor changes that came _to the old API_ with 1.0.0 (and
- Method `create_collection` of `AstraDB` relaxes checks on passing `dimensions` for vector collections
- AstraDBOps core class acquired async methods: `async_get_databases`, `async_get_database`, `async_create_database`, `async_terminate_database`, `async_create_keyspace`, `async_delete_keyspace`

Keep in mind that the pre-1.0.0 library, now dubbed "core", is what the current 1.0.0 API ("idiomatic") builds on.
6 changes: 5 additions & 1 deletion astrapy/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,11 @@ def get_version() -> str:
"api_options": False,
"core": False,
"db": False,
"defaults": False,
"ids": False,
"meta": False,
"ops": False,
"ids": False,
"request_tools": False,
"transform_payload": False,
"user_agents": False,
}
Loading
Loading