Backend deployment failed due to Resource is not in the state stackUpdateComplete

[lucajung]

How did you install the Amplify CLI?

npm

If applicable, what version of Node.js are you using?

v18.16.1

Amplify CLI Version

12.8.2

What operating system are you using?

macOS 12.7

Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.

No manual changes made

Describe the bug

When I try to deploy my changes I get sometimes the following error:

Resource Name: LambdaLayerPermissionAwsAccountsXXXXXXXXXXXXfa1cebacLegacy8 (AWS::Lambda::LayerVersionPermission)
Event Type: create
Reason: Resource handler returned message: "1 validation error detected: Value 'XXXXXXXXXXXXfa1cebac' at 'principal' failed to satisfy constraint: Member must satisfy regular expression pattern: \d{12}|\*|arn:(aws[a-zA-Z-]*):iam::\d{12}:root (Service: AWSLambdaInternal; Status Code: 400; Error Code: ValidationException; Request ID: a22b990d-a19e-4dbd-98dd-687f28f66031; Proxy: null)" (RequestToken: d9e4115d-40db-4c09-3aff-c2c27b8823b0, HandlerErrorCode: GeneralServiceException)

Expected behavior

The error should not happen

Reproduction steps

The error occurs in a non-deterministic way

Project Identifier

No response

Log output

# Put your logs below this line

Additional information

No response

Before submitting, please confirm:

• I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
• I have removed any sensitive information from my code snippets and submission.

[ykethan]

Hey @lucajung, thank you for reaching out. Diving into the error message it appears the principal on the the Lambda layer does not meet the expects constraints. Looking at LambdaLayerPermissionAwsAccounts the layer has been configured to use only access specific AWS accounts.
From XXXXXXXXXXXXfa1cebac the trailing this doesnt appear to be a account number.
under the backendfunction/<layer-name> folder, could you check the CloudFormation template for Principal which should contain similar to

"LambdaLayerPermissionAwsAccountsbdac368": {
      "Type": "AWS::Lambda::LayerVersionPermission",
      "Properties": {
        "Action": "lambda:GetLayerVersion",
        "LayerVersionArn": {
          "Ref": "LambdaLayerVersion2bdac368"
        },
        "Principal": "<account-id>"
      }
    },
    "LambdaLayerPermissionPrivate2bdac368": {
      "Type": "AWS::Lambda::LayerVersionPermission",
      "Properties": {
        "Action": "lambda:GetLayerVersion",
        "LayerVersionArn": {
          "Ref": "LambdaLayerVersion2bdac368"
        },
        "Principal": {
          "Ref": "AWS::AccountId"
        }
      }
    }

to update the layer, could you run amplify update function and update the Specific AWS accounts values then push.

[lucajung]

Hi @ykethan
Thanks for reaching out. Updating the permissions actually works. But why does this happen all the time?
Maybe the following will help you debugging:
Each layer version which contains an invalid principal id had a slightly different Object key. They do all contain a Legacy in it like LambdaLayerPermissionPrivate**Legacy**1 and LambdaLayerPermissionAwsAccountsXXXXXXXXXXXX**Legacy**1.

[ykethan]

@lucajung it appears the lambda layer was created in a older version of Amplify CLI before 5.0. When triggering a update on the layer and running push the layer should be migrated to the newer version.
https://docs.amplify.aws/react/tools/cli/migration/lambda-layers-update/

[lucajung]

@ykethan Actually no. The layer was created around one month ago. Since then I pushed around 200 new versions. It suddenly breaks within the same amplify version without upgrading the amplify version.

[ykethan]

@lucajung could you run amplify diagnose --send-report and provide us the Project Identifier output. This would assist in reproducing the issue.

[lucajung]

@ykethan sure. The Project Identifier is a491d1928efc92c62fe18602bfa5c29a

[ykethan]

Hey @lucajung, thank you for the information. On diving into the cloudformation template notices the layer version 8 and 1 contained legacy in the names and the latest version till version 34 did not contain legacy.
If the error message occurs intermittently on these specific versions(1,8) on a push, to mitigate this we could try removing the older versions(if the versions are not utilized) by running amplify remove function, select the layers(1 and 8) and then run push.
Tried reproducing the issue but did not observe the behavior in my env.
Do let us know if the error reoccurs when creating a version of the layer.

[lucajung]

@ykethan Many thanks for your efforts! I deleted some of the layers via the AWS CLI and not the amplify cli. I will keep an eye on it and get back to you if necessary. Thanks!

[ykethan]

@lucajung i understand feel free in reaching out to us. As a side note I would recommend not deleting them directly on the console or AWS CLI as the CloudFormation template may still have reference to the layer and run into CloudFormation push failures.

[lucajung]

@ykethan yes I'll delete them via the amplify cli in the future. A option would be helpful to always keep only the latest layer version

[ykethan]

@lucajung sounds good, i will be closing the issue for now. Do reach out to us if the behaviour reoccurs.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.