Releases: ExpediaGroup/apiary-data-lake
Releases · ExpediaGroup/apiary-data-lake
v5.2.0: Added expiration days (if declared) to Apiary S3 bucket (#145)
* added bucket expiry days * change variable declaration to number * change variable declaration to number * change variable declaration to number * change variable declaration to number * change variable declaration to number * null variable check * null variable check * removed and set 365 days expiry * test expiry days with checkpoint variable * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * test expiry days * added expiry variable * PR comment changes * removed s3_object_expiration_days variable * Update CHANGELOG.md Co-Authored-By: Scott Barnhart <github@barnharts4.com> * Update VARIABLES.md Co-Authored-By: Adrian Woodhead <massdosage@gmail.com> * Update VARIABLES.md Co-Authored-By: Adrian Woodhead <massdosage@gmail.com> * updated varibale.md & changelog.md * updated varibale.md & changelog.md * updated varibale.md & changelog.md * Update VARIABLES.md Co-Authored-By: Scott Barnhart <github@barnharts4.com> * Update VARIABLES.md Co-Authored-By: Scott Barnhart <github@barnharts4.com> * Update release date in CHANGELOG.md Co-authored-by: Baskar Jayakumar <bjayakumar@BELC02Q47S2G8WN.sea.corp.expecn.com> Co-authored-by: Scott Barnhart <github@barnharts4.com> Co-authored-by: Adrian Woodhead <massdosage@gmail.com>
Variable to configure S3 inventory output format
variable to configure s3 inventory output format (#141) * variable to configure s3 inventory output format * setting ORC as default * update VARIABLES.md Co-authored-by: Raj Poluri <rpoluri@expediagroup.com> Co-authored-by: Scott Barnhart <sbarnhart@expedia.com>
v4.4.1
v4.4.0: Manage S3 logs bucket. (#137)
* manage s3 logs bucket * make apiary_log_bucket variable optional * fix bucket logging * fix bucket logging * enable encryption and configure log expiry * update changelog * PR comment updates
Added Prometheus scrape annotations to Kubernetes deployments
Enable prometheus metrics when running on Kubernetes (#135) * add prometheus scrape annotations * only create cloudwatch dashboard when running on ecs * update changelog * Update CHANGELOG.md Co-Authored-By: Adrian Woodhead <awoodhead@expediagroup.com> Co-authored-by: Adrian Woodhead <massdosage@gmail.com>
v4.2.0: apiary kafka metastore listener (#134)
* configure kafka bootstrap url and topic name * update kafka bootstrap variable name * update changelog * update changelog
Reduce k8s Hive Metastore process heapsize from 90 to 85 percent of container memory limit.
Reduce k8s Hive Metastore process heapsize from 90 to 85 percent (#128) * Reduce k8s Hive Metastore process heapsize from 90 to 85 percent of container memory limit * update changelog
Add variable to enable Atlas hive-bridge metastore listener.
Adding support of a new variable for Atlas hive sync (#127) * Adding support of a new variable Fix line Fix syntax * Pr comments * Adding to changelog and variables.md * PR comments * Making zookeeper and kafka configurable * PR comments - Removing extra full stop * update changelog and variable names * update changelog * update changelog
deploy metastore services on kubernetes
deploy metastore services on kubernetes (#126) * configure metastore route53 using load balancer alias * k8s.tf * add required environment variables * add image_pull_secrets * annotate k8s deployment with iam role * add kiam assume policy * default to admin arn when kiam_arn is blank * add AWS_DEFAULT_REGION * configure iam.amazonaws.com/role annotation as role name instead of arn * move annotations to pod template * add k8s readonly service * fix ro iam role * add hms-readwrite k8s service * add hms_readonly k8s service * configure k8s services as NodePort * configure 90% memory as heapsize for k8s containers * create LoadBalancers for metastore services * configure metastore dns as cname of k8s load balancers * Revert "configure metastore dns as cname of k8s load balancers" This reverts commit 7182dbce39d7c7966a4ee396fedbf29f783cb434. * configure metastore dns as cname of k8s load balancers * set ttl to fix dns records * cleanup ec2 deployment * create kubernetes_deployment only when instance_type is k8s * fix * make ecs LBs optional * cloudwatch fix * route53 fix * vpce fix * make nlb optional in cloudwatch dashboards * add k8s lb to vpc endpoint service * update k8s lb reference * update changelog and add load_balancer_source_ranges * fix load_balancer_source_ranges * add load_balancer_source_ranges to k8s-readwrite * configure load_balancer_source_ranges and ingress_cidr * update VARIABLES.md and license headers * add env variables * update changelog * cleanup root_vol_type,root_vol_size variables
Add support for deploying on EC2
Add limited support for running Hive Metastore in EC2. (#114) * add instance_type variable and logic to disable ecs specific resources * configure hive metastore on EC2 instances * lambda to update hive db locations * configure using SSM * configure SNS event listener on EC2 nodes * terraform fmt * update changelog * update variable.md * PR comment updates * PR comment updates * add iam_name_root variable * update iam role policy resource names * add Name tag to S3 bucket * update changelog