We host in AWS at work, which allows us to use the AWS CDK for IaC and CI/CD Pipelines (so many acronyms). Our DevOps team crafted the original CDK pipeline files for us and handed them over to live in our repository, but when it came time to change the architecture of our application I figured I could use what we already had combined with a little help from Copilot and/or ChatGPT and fix it up to do what I need. I was eventually able to do that, but not without a major challenge in the form of a cryptic error message that took me nearly two weeks to figure out and resolve.
TL;DR: AWS caps the length of the handler of a Lambda's build to 128 characters. I had 129 characters so it failed, but didn't give me a useful error message. I had to pull the template from the pipeline and try to manually create it via the UI to get to the error message.
The architecture change was to go from one single Lambda to two Lambdas and an S3 bucket where the first Lambda accepts a file upload request and generates a pre-signed URL to upload the file to the S3 bucket. The client uploads the file using the pre-signed URL and the second Lambda is notified to process the file. It's a fairly straightforward design and all the pieces to implement it exist directly in AWS. It should have been easy.
I added everything and ran the pipeline and it failed with the message in the title: "Stack with id null does not exist". There was another error with the message "ChangeSet [arn:aws:cloudformation:{region}:{target account}:changeSet/PipelineChange/{guid}] does not exist", but no additional information. So the pipeline was running, which should have generated a stack and a changeset on that stack, but neither was being created. Why not? That's where my two weeks went. describe-events didn't tell me anything. Logs didn't tell me anything. I was getting nowhere so I started over and just created the S3 bucket: success. No issues whatsoever. I was able to generate a pre-signed URL and upload a file to it. So I added the second Lambda back in. More failures.
I don't remember all the troubleshooting steps, but at various points I found the bucket creation failed because it already exists and learned that bucket names are globally unique. Not per account or region or customer/subscription. Globally unique. Solved that and still had the same error. ChatGPT or Copilot thought there must be a circular reference so I revamped all the new CDK files to make sure there wasn't. Same failure. So I started over again. This time I just added the second Lambda and did nothing with the bucket and immediately got the same failure again. Obviously, the problem was in the definition of the new Lambda, right? Yes, but no.
The AWS CLI allowed me to generate the template that would be used by the pipeline to create the stack, and then validate that template. Everything was fine. Validation passed, but when I ran the actual pipeline, it failed. Then ChatGPT suggested pulling the template that was generated during the pipeline build (using `aws cloudformation get-template --change-set-name PipelineChange --stack-name {stack name} --region {region} --query 'TemplateBody' --output json > {filename}.json`) and using the template to manually create the Lambda via the AWS UI. That finally led me to an error message that indicated my resource value was too long. My handler was 129 characters and the limit is 128 characters. One too many characters cost me two weeks because AWS wasn't giving me a useful error message.