-
Notifications
You must be signed in to change notification settings - Fork 252
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Move docker #938
Move docker #938
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 Looks good to me! Reviewed everything up to bf5e78a in 25 seconds
More details
- Looked at
480
lines of code in10
files - Skipped
0
files when reviewing. - Skipped posting
8
drafted comments based on config settings.
1. py/.dockerignore:20
- Draft comment:
Consider adding 'venv' and 'dist' to the ignore list to prevent unnecessary files from being included in the Docker context. - Reason this comment was not posted:
Confidence changes required:50%
The .dockerignore file is well-structured and includes common patterns to ignore. However, it might be beneficial to add 'venv' and 'dist' to the ignore list to prevent unnecessary files from being included in the Docker context.
2. py/.flake8:3
- Draft comment:
Be cautious with ignoring E722 (do not use bare 'except') as it can lead to catching unexpected exceptions. Ensure this is intentional and reviewed. - Reason this comment was not posted:
Confidence changes required:50%
The .flake8 configuration file is excluding several directories and ignoring multiple error codes. While this might be intentional, it's important to ensure that the ignored error codes do not hide potential issues. Specifically, ignoring E722 (do not use bare 'except') can lead to catching unexpected exceptions, which might be problematic.
3. py/.isort.cfg:10
- Draft comment:
Consider adding '.venv' to the 'skip' list to ensure all virtual environments are excluded. - Reason this comment was not posted:
Confidence changes required:50%
The .isort.cfg file is well-configured for the most part. However, the 'skip' option includes 'venv' and 'env', but not '.venv'. It's a common practice to include '.venv' as well to ensure all virtual environments are skipped.
4. py/.pre-commit-config.yaml:15
- Draft comment:
Using 'bash -c' might not be portable across all systems. Consider specifying a dependency on bash or using a more portable shell command. - Reason this comment was not posted:
Confidence changes required:50%
The pre-commit configuration is generally well-structured. However, the use of 'bash -c' in the entry for isort and black might not be portable across all systems, especially if bash is not available. Consider using a more portable shell command or specifying a dependency on bash.
5. py/Dockerfile.unstructured:24
- Draft comment:
Using 'poetry add' in a Dockerfile modifies the pyproject.toml file, which might not be intended. Consider specifying all dependencies in the pyproject.toml file beforehand. - Reason this comment was not posted:
Confidence changes required:50%
The Dockerfile is generally well-structured, but there is a potential issue with the use of 'poetry add' in the Dockerfile.unstructured. This command modifies the pyproject.toml file, which might not be the intended behavior in a Docker build context. It's better to specify all dependencies in the pyproject.toml file beforehand.
6. py/Dockerfile.unstructured:52
- Draft comment:
Downloading NLTK packages and initializing models might be time-consuming. Consider caching these downloads if possible to optimize build time. - Reason this comment was not posted:
Confidence changes required:50%
The Dockerfile.unstructured includes a command to download NLTK packages and initialize models. This step might be time-consuming and could be optimized by caching these downloads if possible.
7. py/compose.neo4j.yaml:17
- Draft comment:
Using a default password for Neo4j is a security risk. Ensure strong, unique passwords are used in production environments. - Reason this comment was not posted:
Confidence changes required:50%
The compose.neo4j.yaml file uses a default password for Neo4j, which is a potential security risk. It's important to ensure that strong, unique passwords are used in production environments.
8. py/compose.postgres.yaml:14
- Draft comment:
Using default credentials for PostgreSQL is a security risk. Ensure strong, unique credentials are used in production environments. - Reason this comment was not posted:
Confidence changes required:50%
The compose.postgres.yaml file uses default credentials for PostgreSQL, which is a security risk. It's important to ensure that strong, unique credentials are used in production environments.
Workflow ID: wflow_I4TXOdLSjo9eahY4
You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet
mode, and more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 Looks good to me! Incremental review on 38b7af5 in 19 seconds
More details
- Looked at
459
lines of code in10
files - Skipped
0
files when reviewing. - Skipped posting
0
drafted comments based on config settings.
Workflow ID: wflow_1Ax9cv6FhYp9pcRE
You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet
mode, and more.
* Feature/merge graphrag group mgmt (#876) * add group ids to document abstraction, first steps * extend group permissions * up * add tests for new group features * up * fixup auth * onboard extensive regression tests * adding regression tests * finish tests * rm selenium * test observability * uncomment tests * checkin first set of group tests * modify search, passing vector tests * checkin work * full delete logic * update search to use new filters * check in * Clean up * Check in * add search * tests/test_end_to_end.py::test_ingest_txt_document passing * cleanup logging * make schemas explicit * move to run logger abstraction * cleanup some test workflows * revive tests * tweak to pass tests * tweak rrf * finish hybrid search cleanup * fixup on regr tests, regen payloads * refresh payloads * refactor api model * Feature/refactor api model (#868) * cleanup imports * flake and cleanup * coherent global import / export structure * add ingestion response models * add management response models * cleanups * checkin work on routes * remove request models * last fixes * merge * add user / group gating * working test groups * updating client --------- Co-authored-by: NolanTrem <[email protected]> * Clean up API (#878) * Get running * fixes in sdk * Add in more fixes * Feature/merge dev owen changes (#880) * add group ids to document abstraction, first steps * extend group permissions * up * add tests for new group features * up * fixup auth * onboard extensive regression tests * adding regression tests * finish tests * rm selenium * test observability * uncomment tests * checkin first set of group tests * modify search, passing vector tests * checkin work * full delete logic * update search to use new filters * check in * Clean up * Check in * add search * tests/test_end_to_end.py::test_ingest_txt_document passing * cleanup logging * make schemas explicit * move to run logger abstraction * cleanup some test workflows * revive tests * tweak to pass tests * tweak rrf * finish hybrid search cleanup * fixup on regr tests, regen payloads * refresh payloads * refactor api model * Feature/refactor api model (#868) * cleanup imports * flake and cleanup * coherent global import / export structure * add ingestion response models * add management response models * cleanups * checkin work on routes * remove request models * last fixes * merge * add user / group gating * working test groups * updating client * rename service to restructure * add get documents for group endpoint * fix client bugs * return delete format * merge cleanups * merge * finalize --------- Co-authored-by: NolanTrem <[email protected]> * Shreyas/graphrag test (#881) * add group ids to document abstraction, first steps * extend group permissions * up * add tests for new group features * up * fixup auth * onboard extensive regression tests * adding regression tests * finish tests * rm selenium * test observability * uncomment tests * checkin first set of group tests * modify search, passing vector tests * checkin work * full delete logic * update search to use new filters * check in * Clean up * Check in * add search * tests/test_end_to_end.py::test_ingest_txt_document passing * cleanup logging * make schemas explicit * move to run logger abstraction * cleanup some test workflows * revive tests * tweak to pass tests * tweak rrf * finish hybrid search cleanup * fixup on regr tests, regen payloads * refresh payloads * refactor api model * Feature/refactor api model (#868) * cleanup imports * flake and cleanup * coherent global import / export structure * add ingestion response models * add management response models * cleanups * checkin work on routes * remove request models * last fixes * merge * add user / group gating * sync * enrich * up * fix global search * rag * remove client.py * rm configs * rm configs --------- Co-authored-by: emrgnt-cmplxty <[email protected]> Co-authored-by: NolanTrem <[email protected]> Co-authored-by: emrgnt-cmplxty <[email protected]> * Feature/fix embedding pipe (#882) * up * fixup concurrency * fix ollama embeddings * fix batching with ollama * checkin all cleanups * rm kg cruft (#884) * rm kg cruft * tweaks * tweak 2 (#885) * Feature/fix retrieval endpoint cruft (#887) * tweak 2 * fix retrieval endpoint descriptions * Python SDK (#886) Clean up Python SDK and routes * Separate out SDK, add js and go sdk to monorepo (#888) * Add r2r-js sdk * Add go sdk * Pull out python sdk * remove venv * Update packages * Check in fixes * Remove alembic dependencies * Feature/merge w nolan (#894) * cleanup hybrid search * cleanups in * Fix structure * Make graspologic optional * fix rag stream (#895) * add py r2r (#896) * Clean up (#897) * fix agent (#898) * define `RAGAgentResponse` (#899) * Shreyas/unstructured (#900) * api + oss lib * rm pdb * rm poetry lock * update version * fixes * Feature/cleanup client obj logic (#901) * define `RAGAgentResponse` * cleanup client logic * Shreyas/tests (#889) * init * tests * rename service * api model * add * merge * rm restructure router * print descriptions * Refactor CLI (#903) * Rm files readded by git (#904) * Remove Execution Wrapper (#905) * Rm files readded by git * Fix merge botch * Feature/fix auth revive tests rebased (#906) * adding the client touch ups * fix auth, revive tests * add back tests * uncomment run auth workflow * decruft * refresh test kg * fixup toml (#908) * Feature/fix ingestion update (#909) * fixup toml * fix update * Fix CLI Tests (#912) Fix CLI tests * Shreyas/kg runtime cfg (#913) add kg runtime config * rename kgenrichmentresponse (#914) * Feature/add nltk hybrid expansion rebased (#917) * expand hybrid search with nltk * cleanups * cleanup hybrid search * format * add setup.py * update * add script (#918) * Fix bug in document chunks (#921) * Fix bug in update files (#923) * Shreyas/unstructured (#922) * fix dockerfiles * adding config * fix paths * mv unstructured dep to docker * clean * Update docker_utils.py * Update unstructured_parsing.py * Update r2r_chunking.py * Update app_entry.py * Feature/repair logging (#925) * fixing logs * fix * rm double logging (#929) * Configs (#926) * Fix config logic * Update config * Clean up cli entry point * Disable SSL when installing nltk wordnet (#930) * Fix analytics endpoint * Update OpenAI sdk calls (#933) * Feature/revive advanced rag (#932) * rm double logging * revive advanced rag examples * merge (#934) * sync model (#935) * Feature/remove version from ingestion end pt (#936) * sync model * remove ability to set version * tweak versions impl * fix version bug * Move docker (#938) * Move docker * remove from root * Clean up sdk/restructure.py * Fix js tests, completion scoring (#939) * Shreyas/unstructured docker image (#940) unstructured docker image * Update JS (#941) * Update models (#942) * Feature/complete group logic (#945) * fix group logic * up * Fix Dockerbuild, Symlink Readme (#944) * Add back tast prompt override and include title if availible * Fix docker, sym link readme * Fix compose file path * Shreyas/KG Search Result model (#937) * return type to kg_search_result * add model * local and global results * modify config * refresh should not be gated by auth (#946) * Linting sync (#947) * Remove email from refresh (#948) * Fix link to image * Feature/rm print cruft rebase (#953) * refresh should not be gated by auth * rm print cruft * black and sort * merge * rm * update api return type * Update Actions (#954) * Update Github Actions (#956) * Update Actions * Update actions * Shreyas/kgsearchresult model (#957) * return type to kg_search_result * add model * local and global results * modify config * add models * up * fix config path * fix models * Login and refresh token bug (#959) * Update Actions * Fix bug in login with refresh token * Point pytest to linux (#960) * collection docs (#955) * Feature/merge dev to main (#962) * merge dev and main * git rm * add back collection fix * fix docker builds (#963) * Running unstructured docker + code cleanups (#964) * Small bugfixes on prompts, return types (#965) * Fix failing CLI tests * NPM publish action * remove tarball * Feature/fix dev tests (#966) * update auth tests * fix tests * back and sort * decruft * revert back to gpt-4o --------- Co-authored-by: NolanTrem <[email protected]> Co-authored-by: Shreyas Pimpalgaonkar <[email protected]>
Summary:
Reorganized Docker and related configuration files into
py/
directory and added comprehensive Docker and Docker Compose setups for Python applications.Key points:
py/
directory for better project structure.py/.dockerignore
to exclude unnecessary files from Docker builds.py/.flake8
for linting configuration, excluding specific directories and ignoring certain errors.py/.isort.cfg
for import sorting configuration withblack
profile.py/.pre-commit-config.yaml
for pre-commit hooks includingisort
andblack
.py/Dockerfile
for building a Python application withpoetry
,gunicorn
, anduvicorn
.py/Dockerfile.unstructured
for building with additional dependencies likeunstructured
andOpenCV
.py/compose.neo4j.yaml
for setting up a Neo4j service with health checks and volume mounts.py/compose.ollama.yaml
for setting up an Ollama service with health checks.py/compose.postgres.yaml
for setting up a Postgres service with health checks and volume mounts.py/compose.yaml
for orchestrating services with Traefik for routing and health checks.Generated with ❤️ by ellipsis.dev