Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix sanity 7 #14

Closed
wants to merge 344 commits into from
Closed

Fix sanity 7 #14

wants to merge 344 commits into from

Conversation

alinabuzachis
Copy link
Owner

No description provided.

tremble and others added 30 commits October 6, 2023 14:09
* Black prep

* Black

* changelog

* Fix pylint unused-import in tests

* Split SSM connection plugin changes

* disable glue tests - bucket's missing

* Disable s3_logging and s3_sync tests

This commit was initially merged in https://github.com/ansible-collections/community.aws
See: ansible-collections/community.aws@2c4575c
Bulk migration to Python 3.6 f-strings

SUMMARY
We've dropped support for Python <3.6, bulk migrate to fstrings and perform some general string cleanup
A combination of

black --preview
flynt
some manual cleanup

ISSUE TYPE

Feature Pull Request

COMPONENT NAME
plugins/
tests/
ADDITIONAL INFORMATION

Reviewed-by: Alina Buzachis

This commit was initially merged in https://github.com/ansible-collections/community.aws
See: ansible-collections/community.aws@de33821
Various ARN handling fixes

Depends-On: ansible-collections#1619
SUMMARY
fixes: ansible-collections#1846
Various modules had hard-coded ARN handling which assumed the use of the main  partition.  This causes problems for folks using Gov Cloud (and aws-cn)
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
plugins/modules/batch_compute_environment.py
plugins/modules/ec2_launch_template.py
plugins/modules/elasticache_info.py
plugins/modules/iam_group.py
plugins/modules/iam_role.py
plugins/modules/msk_config.py
plugins/modules/redshift.py
plugins/modules/sns_topic.py
ADDITIONAL INFORMATION

Reviewed-by: Alina Buzachis

This commit was initially merged in https://github.com/ansible-collections/community.aws
See: ansible-collections/community.aws@25a636c
…ollections#1921)

Mass update of docs and tests (credentials/session tokens)

SUMMARY
We had a cleanup of credentials/session parameters which included a batch of deprecations and renames.
Ensure that all of our tests and docs are using the 'canonical' names
ISSUE TYPE

Docs Pull Request

COMPONENT NAME
plugins/modules/batch_compute_environment.py
plugins/modules/cloudformation_exports_info.py
plugins/modules/ec2_vpc_vpn.py
plugins/modules/elasticache.py
plugins/modules/elasticache_parameter_group.py
plugins/modules/elasticache_snapshot.py
plugins/modules/ses_rule_set.py
plugins/modules/sts_assume_role.py
plugins/modules/sts_session_token.py
tests/integration
ADDITIONAL INFORMATION
See also

ansible-collections#1172
ansible-collections#1714

Reviewed-by: Alina Buzachis

This commit was initially merged in https://github.com/ansible-collections/community.aws
See: ansible-collections/community.aws@4a5b50e
Co-authored-by: Bikouo Aubin <79859644+abikouo@users.noreply.github.com>
Signed-off-by: Alina Buzachis <abuzachis@redhat.com>
Signed-off-by: Alina Buzachis <abuzachis@redhat.com>
Signed-off-by: Alina Buzachis <abuzachis@redhat.com>
…e_iam_managed_policy

Promote iam_managed_policy
* Rename core collection

Rename references to ansible.amazon to amazon.aws.

* Rename community.amazon to community.aws
Fix pep8 line lengths for rewritten amazon.aws imports

* Missed a path in shippable.sh
* Dependency repos moved

This commit was initially merged in https://github.com/ansible-collections/community.aws
See: ansible-collections/community.aws@235c5db
* Remove ANSIBLE_METADATA entirely, see ansible/ansible/pull/69454.
Remove `license` field from galaxy.yml, in favor of `license_file`.

This commit was initially merged in https://github.com/ansible-collections/community.aws
See: ansible-collections/community.aws@05672a6
* Update module deprecations

Switch version to `removed_at_date`

* Don't install amazon.aws from galaxy

We've been using galaxy to install amazon.aws in shippable, but that
doesn't really work if we aren't publising faster. Get that collection
from git so it is most up to date.

* We need to declare python test deps now

* missed a python dep

This commit was initially merged in https://github.com/ansible-collections/community.aws
See: ansible-collections/community.aws@7cd211e
Updated module examples with FQCN

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>

This commit was initially merged in https://github.com/ansible-collections/community.aws
See: ansible-collections/community.aws@98173ae
* Update docs

Remove .git from repo url so links in readme will generate correctly
Add required ansible version
Run latest version of add_docs.py
Add version_added string to modules

* galaxy.yml was missing authors

This commit was initially merged in https://github.com/ansible-collections/community.aws
See: ansible-collections/community.aws@96ee268
* Update comments to reference AnsibleAWSModule rather than AnsibleModule
* Bulk re-order imports and split onto one from import per-line.
* Add AnsibleAWSModule imports
* Migrate boto 2 based modules to AnsibleAWSModule
* Move boto3-only modules over to AnsibleAWSModule
* Remove extra ec2_argument_spec calls - not needed now we're using AnsibleAWSModule
* Remove most HAS_BOTO3 code, it's handled by AnsibleAWSModule
* Handle missing Boto 2 consistently  (HAS_BOTO)
* Remove AnsibleModule imports
* Changelog fragment

This commit was initially merged in https://github.com/ansible-collections/community.aws
See: ansible-collections/community.aws@818c6d2
…r(e) or e.me… (ansible-collections#26)

* Py3 compat error handling: use to_native(e) instead of str(e) or e.message
* PR comment changes, use fail_json_aws and is_boto3_error_code

This commit was initially merged in https://github.com/ansible-collections/community.aws
See: ansible-collections/community.aws@ffe14f9
…ollections#188)

* Migrate from boto3_conn to module.client
* Simplify error handling when creating connections
* Simplify Region handling
* Remove unused imports
* Changelog

This commit was initially merged in https://github.com/ansible-collections/community.aws
See: ansible-collections/community.aws@6bdf00d
* Split imports and reorder
* Import camel_dict_to_snake_dict and snake_dict_to_camel_dict direct from ansible.module_utils.common.dict_transformations
* Remove unused imports
* Route53 Info was migrated to Boto3 drop the HAS_BOTO check and import
* changelog

This commit was initially merged in https://github.com/ansible-collections/community.aws
See: ansible-collections/community.aws@130cf3c
timdiggins and others added 29 commits November 29, 2023 18:16
allow s3_object put to specify metadata again

fix erroneous change to meaning
fixes ansible-collections#1881
optional extra refactorings as separate commits (happy to drop these if unwanted)

Reviewed-by: Mark Chappell
…e-core >= 2.17 (ansible-collections#1893)

ec2_metadata_facts - Skip tests against 2.7 hosts when running Ansible-core >= 2.17

SUMMARY
ansible-core 2.17 will drop support for Python 2.7.  While in general we require Python > 3.7, we've had an exception for ec2_metadata_facts since it doesn't actually need botocore and it's used outside of the AWS ecosystem.
Unfortunately it's no longer possible to test against 2.7 targets using the devel branch, so we'll have to skip those tests in CI (can still be run locally if folks are using ansible-core <2.17
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
ec2_metadata_facts
ADDITIONAL INFORMATION

Reviewed-by: Alina Buzachis
Reviewed-by: Mike Graves <mgraves@redhat.com>
iam_group - add support for setting the path

SUMMARY
Adds support for the 'path' option on IAM Groups.
Adds attached_policies return value.
Also refactors create_or_update_group because it was getting large and unwieldy.
Splits up test file, and adds tests for policy management
ISSUE TYPE

Feature Pull Request

COMPONENT NAME
iam_group
ADDITIONAL INFORMATION

Reviewed-by: Milan Zink <zeten30@gmail.com>
Reviewed-by: Alina Buzachis
)

Avoid unsafe conditionals in integration tests

SUMMARY
Update a batch of assertions in integration tests to avoid templating
See: ansible/ansible-documentation#866 and ansible/ansible-documentation#866 for more information
This PR covers about half of the impacted test tasks for this repo but is already going to be a lot to get through CI so additional PRs will follow.
Affected tasks identified with:
https://gist.github.com/sivel/7b012ae2ebf4ab4cb6225379a72cc502
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
tests
ADDITIONAL INFORMATION
There are unsupported tests in this change.  The cloudtrail tests seem to be broken independent of this change, iam_group tests pass for me locally

Reviewed-by: Bikouo Aubin
Reviewed-by: Mark Chappell
Reviewed-by: Alina Buzachis
Reviewed-by: Mike Graves <mgraves@redhat.com>
SUMMARY

This PR fixes the integration tests to use FQCN throughout. Every change has been generated by running ansible-lint --fix with the write_list limited to fqcn. The one exception is the manual removal of the tests/integration/targets/aws_region_info/main.yml which ansible-lint was failing on. This seems to be unused? It attempts to include tasks/tests.yml which does not exist.

The ansible-lint config used to generate this changeset:

---
exclude_paths:
  - plugins/
write_list:
  - fqcn

I had to exclude the plugins/ directory because ansible-lint was crashing, I suspect because of docstrings. It did not provide enough information in debug or traceback to identify which file was causing the problem.

ISSUE TYPE

    Bugfix Pull Request

COMPONENT NAME

ADDITIONAL INFORMATION
…ollections#1907)

revert changelog update - it's auto-generated by antsibull

SUMMARY
revert changelog linting update - it's auto-generated by antsibull
ISSUE TYPE

Docs Pull Request

COMPONENT NAME
changelogs/changelog.yaml
ADDITIONAL INFORMATION

Reviewed-by: Helen Bailey <hebailey@redhat.com>
Reviewed-by: Alina Buzachis
…ctions#1884)

module_utils/elbv2 - supports update for TLS listeners

SUMMARY

closes ansible-collections#1198

ISSUE TYPE


Feature Pull Request

COMPONENT NAME

module_utils/elbv2
ADDITIONAL INFORMATION

Reviewed-by: Helen Bailey <hebailey@redhat.com>
Reviewed-by: Bikouo Aubin
Reviewed-by: GomathiselviS
Reviewed-by: Alina Buzachis
…collections#1912)

iam_user - Add support for setting path and boundary policy

SUMMARY
Add support for setting path and boundary policy

Code refactor to reduce complexity
Adds support for setting path
Adds support for setting boundary policy
returns attached_policies

ISSUE TYPE

Feature Pull Request

COMPONENT NAME
iam_user
ADDITIONAL INFORMATION

Reviewed-by: Alina Buzachis
…1913)

Update changelog after releases 7.1.0 and 6.5.1

SUMMARY

Update changelog after releases 7.1.0 and 6.5.1

ISSUE TYPE


Bugfix Pull Request
Docs Pull Request
Feature Pull Request
New Module Pull Request

COMPONENT NAME

ADDITIONAL INFORMATION

Reviewed-by: Mark Chappell
Fix additional FQCNs

SUMMARY

ansible-collections#1902 attempted to fix FQCNs using ansible-lint --fix, but this apparently missed quite a few instances. This hopefully fixes the rest.

ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME

ADDITIONAL INFORMATION

Reviewed-by: Mark Chappell
Reviewed-by: Alina Buzachis
kms_key - integration test race condition

SUMMARY
Because kms_key doesn't have any waiters we sometimes hit the race condition with the deletion test that we return before the service fully registers that a deletion is in progress.
This change uses kms_key_info to check that the deletion and relevant dates are correctly set after a short pause to allow propagation.
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
kms_key
ADDITIONAL INFORMATION

Reviewed-by: Alina Buzachis
…ible-collections#1922)

Remove useless tests/sanity/ignore-2.*.txt files lower than 2.14

SUMMARY

Remove useless tests/sanity/ignore-2.*.txt files lower than 2.14

ISSUE TYPE


Bugfix Pull Request
Docs Pull Request
Feature Pull Request
New Module Pull Request

COMPONENT NAME

ADDITIONAL INFORMATION

Reviewed-by: Mark Chappell
…le-collections#1921)

Bump MariaDB versions for test - Amazon dropped old ones again

SUMMARY
Bump MariaDB versions for test - Amazon dropped old ones again
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
rds_instance_upgrade
ADDITIONAL INFORMATION

Reviewed-by: Alina Buzachis
…s#1900)

ansible-lint (documentation) cleanup for plugins/

SUMMARY
Fixes an array of ansible-lint failures in plugins/
Adds ansible-lint plugins/ to tox -m lint
ISSUE TYPE

Docs Pull Request

COMPONENT NAME
plugins/
ADDITIONAL INFORMATION
docs changes only (no changelog fragment needed)

Reviewed-by: Alina Buzachis
changelog - fix formatting

SUMMARY
Fixes an extra backtick that snuck into a changelog fragment.
ISSUE TYPE

Docs Pull Request

COMPONENT NAME
changelog.yml
ADDITIONAL INFORMATION

Reviewed-by: Alina Buzachis
…e-collections#1839)

rds_cluster: add support for ServerlessV2ScalingConfiguration

SUMMARY

Fixes ansible-collections#1476
Add support for ServerlessV2ScalingConfiguration in create_db_cluster and modify_db_cluster
https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_ServerlessV2ScalingConfiguration.html
https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless-v2.html

ISSUE TYPE


Feature Pull Request

COMPONENT NAME

rds_cluster
ADDITIONAL INFORMATION

Reviewed-by: Alina Buzachis
Reviewed-by: Helen Bailey <hebailey@redhat.com>
Reviewed-by: GomathiselviS
…ible-collections#1883)

cloudwatchevent_rule: Fix json input handling for input_template

SUMMARY

Fixes ansible-collections#1842
This PR


Moves _snakify() out of class CloudWatchEventRule https://github.com/ansible-collections/amazon.aws/pull/1883/files#diff-3a2f223edc4e34ea28b9859827a830844d92a8cd97f1300a2d16b1703a083bc8R213-R215


Fixes conditional in function _targets_to_put() to avoid wrong results due to _snakify() https://github.com/ansible-collections/amazon.aws/pull/1883/files#diff-3a2f223edc4e34ea28b9859827a830844d92a8cd97f1300a2d16b1703a083bc8R461-R463


Updates code to add quotes " to input_template only when the given input is not JSON to avoid quotes being added to provided JSON input https://github.com/ansible-collections/amazon.aws/pull/1883/files#diff-3a2f223edc4e34ea28b9859827a830844d92a8cd97f1300a2d16b1703a083bc8R452-R458



ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME

cloudwatchevent_rule
ADDITIONAL INFORMATION

Reviewed-by: Bikouo Aubin
Reviewed-by: Helen Bailey <hebailey@redhat.com>
Reviewed-by: Alina Buzachis
Reviewed-by: Mandar Kulkarni <mandar242@gmail.com>
…nsible-collections#1919)

s3_bucket_info - add support for bucket versioning in bucket_facts

SUMMARY
Module s3_bucket_info should be able to return the bucket versioning into bucket_facts
ISSUE TYPE


Feature Pull Request

COMPONENT NAME

s3_bucket_info
ADDITIONAL INFORMATION



- amazon.aws.s3_bucket_info:
   name: mybucket
   bucket_facts:
    bucket_versioninig: true
expecting output
{
    "bucket_name": "mybucket",
    "buckets": [
         {
            "bucket_versioning": {"Status": "Enabled"},    ## New information in output
           "creation_date": "2023-12-11T15:48:42+00:00",
           "name": "mybucket"
         }
    ],
   "changed": false,
   "msg": "Retrieved s3 info."
}

Reviewed-by: Alina Buzachis
Reviewed-by: GomathiselviS
Reviewed-by: Bikouo Aubin
Reviewed-by: Helen Bailey <hebailey@redhat.com>
avoid unsafe conditions for integration tests

SUMMARY
Remove all jinja templating expression from asserts condition
ISSUE TYPE


Bugfix Pull Request

COMPONENT NAME

integration tests

Reviewed-by: Helen Bailey <hebailey@redhat.com>
Reviewed-by: Alina Buzachis
…sible-collections#1918)

ec2_instance: add support for modifying instance metadata options

SUMMARY

Fixes ansible-collections#1864
This PR add support for modifying instance metadata options.
Uses client. modify_instance_metadata_options()

ISSUE TYPE


Feature Pull Request

COMPONENT NAME

ec2_instance
ADDITIONAL INFORMATION

Reviewed-by: Helen Bailey <hebailey@redhat.com>
Reviewed-by: Mark Chappell
Reviewed-by: Bikouo Aubin
Reviewed-by: GomathiselviS
Reviewed-by: Mandar Kulkarni <mandar242@gmail.com>
Various IAM module cleanup

SUMMARY

Consistently use "path" and "name" for IAM modules (with "path_prefix", "prefix" and "TYPE_name" as aliases)
Consistently use "path_prefix" and "name" for IAM info modules (with "path", "prefix" and "TYPE_name" as aliases)
Consistently test "path" and "name" for validity
Spit out warning when we would update path, but it's not supported by the APIs (could become an error at a later date)
Do not set "/" as the explicit default for paths

ISSUE TYPE

Feature Pull Request

COMPONENT NAME
plugins/module_utils/iam.py
plugins/modules/iam_group.py
plugins/modules/iam_instance_profile.py
plugins/modules/iam_managed_policy.py
plugins/modules/iam_role.py
plugins/modules/iam_role_info.py
plugins/modules/iam_user.py
plugins/modules/iam_user_info.py
ADDITIONAL INFORMATION

Reviewed-by: Mandar Kulkarni <mandar242@gmail.com>
Reviewed-by: Mark Chappell
Reviewed-by: Bikouo Aubin
…tions#1936)

iam_managed_policy - add support for path and tagging

SUMMARY

Fixes minor bug where description was only returned if changes were made.
Adds support for setting the path on IAM Managed Policies.
Adds support for tagging Managed Policies

ISSUE TYPE

Bugfix Pull Request
Feature Pull Request

COMPONENT NAME
iam_managed_policy
ADDITIONAL INFORMATION

Reviewed-by: Helen Bailey <hebailey@redhat.com>
Reviewed-by: Mark Chappell
…sible-collections#1939)

lambda: add support for container image as alternative to runtime

SUMMARY
Fixes ansible-collections#1417
Supersedes ansible-collections#1438
ISSUE TYPE

Feature Pull Request

COMPONENT NAME
lambda
ADDITIONAL INFORMATION
The original PR appears to have been abandoned by its author. I have taken the liberty of salvaging it and resolving current merge conflicts as conservatively as possible.
I hope I have correctly merged the required_if and required_together arrays; it seems to make sense to me, and it seems to run correctly from my local library.
I have also targeted release 6.5.2 in the documentation snippet, if that is not too presumptive.
Please request changes. I will endeavor to oblige. Thank you.

Reviewed-by: Mark Chappell
Reviewed-by: Alina Buzachis
…ions#1944)

Pull 7.2.0 changelogs into main branch

SUMMARY
Pull 7.2.0 changelogs into main branch
ISSUE TYPE

Docs Pull Request

COMPONENT NAME
CHANGELOG.rst
changelogs/changelog.yaml
ADDITIONAL INFORMATION
(partial port of ansible-collections#1941)

Reviewed-by: Alina Buzachis
…ible-collections#1947)

Fix version_released for changes that weren't backported in time (7.2.0 -> 7.3.0)

SUMMARY
Fix version_released for changes that weren't backported in time
ISSUE TYPE

Docs Pull Request

COMPONENT NAME
plugins/modules/rds_cluster.py
plugins/modules/s3_bucket_info.py
ADDITIONAL INFORMATION

Reviewed-by: Alina Buzachis
… multiple certificates (ansible-collections#1950)

elb_application_lb: add support for creating alb with listener having multiple certificates

SUMMARY

Fixes ansible-collections#1349
This PR adds support for adding multiple certificates to a listener during Application Load Balancer creation.

ISSUE TYPE


Feature Pull Request

COMPONENT NAME

elb_application_lb
ADDITIONAL INFORMATION


As per boto3 documentation

only 1 certificate can be provided during call to create_listener
Multiple certificates can be added using add_listener_certificates, however important to note that only 1 certificate can be specified per call.

Reviewed-by: Bikouo Aubin
Reviewed-by: Alina Buzachis
Bugfix aws_ec2 inventory use_ssm_inventory

SUMMARY
Fix bug where aws_ec2 inventory plugin was failing with use_ssm_inventory: true if there are more than 40 inventories.
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
aws_ec2
ADDITIONAL INFORMATION
Fixes ansible-collections#1713

Reviewed-by: Bikouo Aubin
Reviewed-by: Mark Chappell
…le-collections#1951)

Add simple decorator for handling common botocore/boto3 errors

SUMMARY
Adds a handler for common IAM API errors
ISSUE TYPE

Feature Pull Request

COMPONENT NAME
plugins/module_utils/iam.py
plugins/modules/iam_user.py
ADDITIONAL INFORMATION
TODO:

  Changelog

Reviewed-by: Alina Buzachis
Reviewed-by: Mark Chappell
Reviewed-by: Bikouo Aubin
Reviewed-by: Helen Bailey <hebailey@redhat.com>
Signed-off-by: Alina Buzachis <abuzachis@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.