Syncing latest changes from master for noobaa-core #13
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Romy <[email protected]>
NSFS | NC | Refactor Bucketspace_fs.delete_bucket() and others
* Utilize the correct certificates for the STS HTTPS server
* Fix log-based replication Prometheus field
* Add missing parameter to STS server init * Add missing `await` Signed-off-by: Ben <[email protected]>
We need to change the log prefix restriction and accept it in accordance with AWS S3 guidelines. As we do that, it is better to fetch the log prefix from the configuration of the source bucket instead of depending on log object names. It's because the log prefix can have any character in it, and it would be difficult to have a delimiter that can segregate the log prefix from other strings in the log object name. Signed-off-by: Ashish Pandey <[email protected]>
…ange Bucket Logging: Accept Log Prefix as per S3 bucket logging
At present we are creating log objects and giving them name based on log_prefix and log_bucket. There are some constraint to that and one is that a user can set same log bucket and log prefix for two different source buckets. That will cause logs from two different source buckets to go to same log objects. In addition to that, source bucket name in log file name will help to get it easily without processing log file. This source bucket name can be used to get more logging configuration, which can be helpful in further processing. Signed-off-by: Ashish Pandey <[email protected]>
Bucket_logging: Change log object name format.
Signed-off-by: jackyalbo <[email protected]>
Adding debug option to manage_nsfs
Signed-off-by: Romy <[email protected]>
…check NSFS | MPU | Disable ctime check on load_multipart()
Bumps [ws](https://github.com/websockets/ws) from 8.17.0 to 8.17.1. - [Release notes](https://github.com/websockets/ws/releases) - [Commits](websockets/ws@8.17.0...8.17.1) --- updated-dependencies: - dependency-name: ws dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]>
…-8.17.1 Bump ws from 8.17.0 to 8.17.1
1. Implement the functions related to access keys in AccountSpaceFS (CRUD). 2. Add more properties to nsfs_account_schema inside access_key object (all of them are not required): - creation_date = the date the access-key object was created. - deactivated = boolean (we need to translate it to status which is 'Active' or 'Inactive'). 3. Clean the account cache after updating the account config using the IAM API. 4. Remove unused errors that were copied from STS' errors in IamErrors. 5. Change the param name from user_name to username in the ops. 6. Remove the mock variables that we used in the IAM boilerplate implementation. 7. Update unit test for access_keys function in accountspace_fs. 8. Add unit test in account_schema_validation related to the schema changes. 9. Update Get Started section that would be the demo for IAM access keys management (docs/dev_guide/nc_nsfs_iam_developer_doc.md). 10. Changes in nc_master_key_manager to allow an access key object with more properties and add tests for the edited functions (encrypt_access_keys and decrypt_access_keys of account). 11. Add the support for denying a request whose access key status is Inactive (in all services: S3, STS, IAM). 12. Instead of copying the master_key_id from the root account, we now take the most updated master_key_id. 13. Add a new RPC code for a deactivated access key. 14. Edit the function authorize_request_account_by_token adding the access key deactivated check, but also throwing errors on every missing component instead of using optional chaining. Signed-off-by: shirady <[email protected]>
NSFS | NC | IAM Service - Access Keys CRUD API Implementation
…h clause in sts_sdk Signed-off-by: shirady <[email protected]>
Signed-off-by: shirady <[email protected]>
…esting_account Fix Throw Error in `load_requesting_account` (object_sdk, account_sdk)
1. Change the require to the file name (instead of destructor in the require with the functions name). 2. Change all functions which start with "verify" to "validate". 3. Create a function for checking the identifier for each command. 4. Remove a duplicate test (cli account status without name and access_key). 5. Remove dead code inside validate_bucket_args and validate_account_args. 6. Change the operator in the function has_access_keys to match its name. Signed-off-by: shirady <[email protected]>
…on-part2 NSFS | NC | CLI Refactor Validation (Partial) - Part 2
…NV = 'true' set Signed-off-by: Romy <[email protected]>
NC | nsfs.js | Lower fork_utils dependancy down after process.env.NC_NSFS_NO_DB_ENV=true setting
…oobaa#8169) * Fix STS SessionToken generation and refactor JWT `make_auth_token` Signed-off-by: Ben <[email protected]>
Signed-off-by: Romy <[email protected]>
NC | Add `--debug` flag to health CLI
If a newly generated tier name already exists, regenerate it instead of failing the entire operation. Signed-off-by: Danny Zaken <[email protected]>
Avoid throwing an error in tier_server.update_bucket_class
…ons using the IAM API) 1. Add more properties to nsfs_account_schema (not required): - iam_operate_on_root_account = boolean (if an account was set with true. then it is a roots accounts manager). 2. Add the option of iam_operate_on_root_account in noobaa cli. 3. Edit all the functions in AccountSpaceFS (CRUD) to operate on root accounts when the requesting account is a root accounts manager. Signed-off-by: shirady <[email protected]>
…-object-with-object-id NC | NSFS | Versioning | Fix Bug 8333 | Delete Object with Version ID | GPFS
Signed-off-by: shirady <[email protected]>
…-objects NC | NSFS | Versioning | Delete Objects | Add a Test | GPFS
1. Add manual instructions for testing Ceph S3 tests in NC deployment. 2. Fix the small mistake in all principals example in bucket policy. Signed-off-by: shirady <[email protected]>
…y-fix-and-ceph-update NC | Docs | Update Ceph S3 Tests Instructions in NC + Fix Bucket Policy All Principals
Signed-off-by: nadav mizrahi <[email protected]>
…n-twice NSFS | GPFS | fix delete latest object twice on GPFS
Signed-off-by: shirady <[email protected]>
CI | Update Ceph S3 Tests Days (Temporary Solution)
…dd health access check configuration Signed-off-by: Romy <[email protected]>
…guration NC | Change Check Access Behavior
Signed-off-by: Romy <[email protected]>
…ect-delete NC | GPFS | Threaded Multiple Object Delete fix
Signed-off-by: Romy <[email protected]>
…ip-fix NC | Copy based on link/same inode overrides existing xattr
Signed-off-by: naveenpaul1 <[email protected]>
NSFS | NOOBAA_LOG_LEVEL is not working as expected
…or nodejs Signed-off-by: Aayush Chouhan <[email protected]>
Added a fix for prometheusDuplicateTimestamp error
Signed-off-by: nadav mizrahi <[email protected]>
NSFS | GPFS | fix put object over null version in suspended mode
Bumping deps to avoid CVE Signed-off-by: liranmauda <[email protected]>
Bumping deps to avoid CVE
Signed-off-by: Romy <[email protected]>
…ect-delete NSFS | Add parentheses to should retry
1. In test_bucketspace_versioning.js in the object tagging tests - Add Header check related to HeadObject with tagging, as was manually tested in PR NC | NSFS | Fix Bug | Head Object on a Tagged Object Does Not Return x-amz-tagging-count Header noobaa#8357. - Change the bucket from suspended_bucket_name to bucket_name (the bucket that we use with versioning Enabled). 2. In test_bucketspace_versioning.js add headers check, as was manually tested in PR NC | NSFS | Versioning | Fix Bug | Return 405 for Get/Head Specific Delete-Marker noobaa#8338 in the following tests: - head object, with version enabled, version id specified delete marker - should throw error with code 405 - get object, with version enabled, version id specified delete marker - should throw error with code 405 Signed-off-by: shirady <[email protected]>
…s-check NC | NSFS | Versioning | Headers Check in Unit Tests
1. Add 2 cases to our concurrency tests: - Key with 5 versions, concurrent delete of 3 latest versions (no veID) -> 3 delete markers - Start with 1 version, 10 Puts + 10 Reads (latest) concurrent nothing fails -> 11 version IDs 2. Refactor the file test_versioning_concurrency for reuse: move tmp_fs_path and nsfs as global variables, add the helper function _upload_versions from existing code to avoid code duplication. Signed-off-by: shirady <[email protected]>
…rency-tests NC | NSFS | Add Concurrency Tests
Signed-off-by: Utkarsh Srivastava <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR containing the latest commits from master branch