Bug: print of IAM policy on gitlab CI not working

[neylsoncrepalde]

Hi guys. I ran rony's CI on gitlab but the policy does not get printed as it should by iamlive. Below is a screenshot of the final logs of the pipeline:

aws_s3_bucket_object.job_spark: Creation complete after 1s [id=emr-code/pyspark/spark_censo_escolar_to_parquet.py]
aws_lambda_function.decompresss3: Still creating... [10s elapsed]
aws_lambda_function.decompresss3: Still creating... [20s elapsed]
aws_lambda_function.decompresss3: Creation complete after 22s [id=prod-prod_censo_ingestion_igti-ney]
Apply complete! Resources: 11 added, 0 changed, 0 destroyed.
PERMISSIONS USED:
cat: ./policy.json: No such file or directory
Cleaning up file based variables
00:00
ERROR: Job failed: exit code 1

Any help? Many thanks!!!

[neylsoncrepalde]

UPDATE

I have modified apply.sh to this

#!/bin/bash
set -e

#### Terraform Apply #####

./CI/scripts/pre_terraform.sh

export AWS_CSM_ENABLED=true
export AWS_CSM_PORT=31000
export AWS_CSM_HOST=127.0.0.1

cd infrastructure/aws/

(iamlive --output-file './policy.json' > /dev/null &)

terraform init
terraform workspace select $1 || terraform workspace new $1
terraform apply $2

pkill iamlive

sleep 1
echo ""
GREEN='\033[0;32m'
NC='\033[0m' # No Color

echo -e "${GREEN}Files found:${NC}"
ls

echo -e "${GREEN}PERMISSIONS USED:${NC}"
#cat ./policy.json

cd ../..

###########################

in order just to print the found files and not cat the policy.json file. It has been correctly created:

null_resource.fn_example_script: Destroying... [id=1563616191854048973]
null_resource.fn_example_script: Destruction complete after 0s
null_resource.fn_example_script: Creating...
null_resource.fn_example_script: Provisioning with 'local-exec'...
null_resource.fn_example_script (local-exec): Executing: ["/bin/sh" "-c" "zip -rj ../../functions/fn_example_script.zip ../../functions/fn_example_script"]
null_resource.fn_example_script (local-exec):   adding: main.py (stored 0%)
null_resource.fn_example_script (local-exec):   adding: handler.py (deflated 55%)
null_resource.fn_example_script: Creation complete after 0s [id=8498242253945208677]
Apply complete! Resources: 1 added, 0 changed, 1 destroyed.
Files found:
backend.tf
ecr.tf
function.tf
iam.tf
permissions
policy.json
provider.tf
s3_files.tf
storage.tf
variables.tf
PERMISSIONS USED:
Cleaning up file based variables
00:01
Job succeeded

[neylsoncrepalde]

Update 2

This is very interesting... it seems that sometimes, policy.json file is created, sometimes it is not. I am not sure why this happens...

[RodrigoATorres]

I was not able to reproduce this issue, but I think it might be related to the fact that IAM live takes some time to write the policy file after its process is killed.

What the current code does to address this, is to sleep for 1 second after killing IAMLive. I guess it might not be enough in some cases (maybe on more complex projects). For a simple project running locally, it usually takes 0.4 seconds to create the file.

Could you try running the pipeline with this new apply.sh? If you share your code I can make some tests as well.

What this new code does is to wait for the file to be created (up to 60 seconds).
This code prints the timeout, for debugging purposes. Later we should remove it.
If it works, we should also update ./CI/scripts/destroy.sh

#!/bin/bash
set -e

#### Terraform Apply #####

./CI/scripts/pre_terraform.sh

export AWS_CSM_ENABLED=true
export AWS_CSM_PORT=31000
export AWS_CSM_HOST=127.0.0.1


cd infrastructure/aws/

[ -e "./policy.json" ] && rm "./policy.json"

(iamlive --output-file './policy.json' > /dev/null &)

terraform init
terraform workspace select $1 || terraform workspace new $1
terraform apply $2

pkill iamlive

# Wait 60 seconds for the file policy.json to be generated
export timeout=600
while [ ! -f './policy.json' ]
do
  if [ "$timeout" == 0 ]; then
    echo "ERROR: Timeout while waiting for the file permissions file to be generated"
    break
  fi
  timeout=$((timeout-1))
  echo $timeout
  sleep 0.1
done
##########################

if [ -f './policy.json' ]; then
  echo ""
  GREEN='\033[0;32m'
  NC='\033[0m' # No Color
  echo -e "${GREEN}PERMISSIONS USED:${NC}"
  cat ./policy.json
  cd ../..
fi

###########################