Skip to content

v10.0.0-beta

Pre-release
Pre-release
Compare
Choose a tag to compare
@rizbihassan rizbihassan released this 13 Jun 20:46
· 14 commits to master since this release
v10.0.0-beta
435e173

Release v10.0.0-beta

Migration Notes

Remove the s3_access_key and s3_secret_key variables from your orca.tf file.

Post V2 Upgrade Comparison

Once the Aurora V1 database has been migrated/upgrade to Aurora V2 you can verify data integrity of the ORCA database by deploying the EC2 comparison instance which can be found at modules/db_compare_instance/main.tf

  • Deployment Steps
    1. Fill in the variables in modules/db_compare_instance/scripts/db_config.sh
      • archive_bucket - ORCA Archive Bucket Name IMPORTANT: use underscores in place of dashes e.g. zrtest_orca_archive
      • v1_endpoint - Endpoint of the V1 cluster e.g. orcaV1.cluster-c1xufm1sp0ux.us-west-2.rds.amazonaws.com
      • v1_database - Database of the V1 cluster e.g. orca_db
      • v1_user - Username of the V1 cluster e.g orcaV1_user
      • v1_password - Password for the V1 user e.g. OrcaDBPass_4
      • v2_endpoint - Endpoint of the V2 cluster e.g. orcaV2.cluster-c1xufm1sp0ux.us-west-2.rds.amazonaws.com
      • v2_database - Database of the V2 cluster e.g. orca_db2
      • v2_user - Username of the V2 cluster e.g orcaV2_user
      • v2_password - Password for the V2 user e.g. OrcaDB2Pass_9
    2. cd to modules/db_compare_instance
    3. Run terraform init
    4. Run terraform apply
    5. Once the instance is deployed add an inbound rule to both the V1 and V2 database security groups with the private IP of the EC2 instance.
      • The private IP of the instance can be found via the console or AWS CLI by running the command: aws ec2 describe-instances --filters "Name=instance-state-name,Values=running" "Name=instance-id,Values=<INSTANCE_ID>" --query 'Reservations[*].Instances[*].[PrivateIpAddress]' --output text
      • This needs to be performed on BOTH V1 and V2 Security Groups The inbound rule can be added via the AWS console or AWS CLI by running the command: aws ec2 authorize-security-group-ingress --group-id <DB_SECURITY_GROUP_ID> --protocol tcp --port 5432 --cidr <INSTANCE_PRIVATE_IP>/32
    6. Now you can connect to the EC2 via the AWS console or AWS CLI with the command: aws ssm start-session --target <INSTANCE_ID>
    7. Once connected run the command cd /home
    8. Once at the /home directory run the command: sh db_compare.sh
    9. When the script completes it will output two tables:
      • v1_cluster - This table is count of data in the ORCA database of each table in the V1 cluster.
      • v2_cluster - This table is count of data in the ORCA database of each table in the V2 cluster.
    10. Verify that the output of the V2 database matches that of the V1 database to ensure no data was lost during the migration.
    11. Once verified the EC2 instance can be destroyed by running terraform destroy Verify you are in the modules/db_compare_instance directory
    12. This needs to be performed on BOTH V1 and V2 Security Groups Remove the added inbound rules that were added in step 5 either in the AWS Console or AWS CLI by running the command: aws ec2 revoke-security-group-ingress --group-id <DB_SECURITY_GROUP_ID> --protocol tcp --port 5432 --cidr <INSTANCE_PRIVATE_IP>/32
    13. Delete the V1 database.
    14. Remove the snapshot identifier from the Terraform (If Applicable)
    15. In the AWS console navigate to RDS -> Snapshots and delete the snapshot the V2 database was restored from.

Added

  • ORCA-845 - Created IAM role for RDS S3 import needed for Aurora v2 upgrade.
  • ORCA-792 - Added DB comparison script at modules/db_compare_instance/scripts/db_compare.sh for the temporary EC2 to compare databases post migration.
  • ORCA-868 - Added EC2 instance for DB comparison after migration under modules/db_compare_instance/main.tf

Changed

  • ORCA-832 - Modified pyscopg2 installation to allow for SSL connections to database.
  • ORCA-795 - Modified Graphql task policy to allow for S3 imports.
  • ORCA-797 - Removed s3 credential variables from deployment-with-cumulus.md and s3-credentials.md documentations since they are no longer used in Aurora v2 DB.
  • ORCA-873 - Modified build task script to copy schemas into a schema folder to resolve errors.
  • ORCA-872 - Updated grapql version, modified policy in modules/iam/main.tf to resolve errors, and added DB role attachment to modules/graphql_0/main.tf

Deprecated

Removed

  • ORCA-793 - Removed s3_access_key and s3_secret_key variables from terraform.
  • ORCA-795 - Removed s3_access_key and s3_secret_key variables from Graphql code and from get_current_archive_list task.
  • ORCA-798 - Removed s3_access_key and s3_secret_key variables from integration tests.
  • ORCA-783 - Removed tasks/copy_to_archive_adapter and tasks/orca_recovery_adapter as they are handled by Cumulus.

Fixed

  • ORCA-835 - Fixed ORCA documentation bamboo CI/CD pipeline showing node package import errors.
  • ORCA-864 - Updated ORCA archive bucket policy and IAM role to fix access denied error during backup/recovery process.

Security

  • ORCA-851 - Updated bandit libraries to fix Snyk vulnerabilities.
Assets 3
Loading