Observed issues on Fortigate Autoscale code in AWS

[ManikantaNandyala]

Observed issues on this code.

1. Lambda Function Default Memory on Template is 128 MB which is not a variable parameter to specify the modify Memory Details while creation of stack, is there any specific reason to keep 128 MB as default for Lambda. although the Lambda function does work when memory is set to default 128MB which is completely utilized sometimes the returned value would cause a spike in the heartbeat total duration which eventually would result in instance/Firewalls termination.

2. Add the pre-requisites on the Github readme, for Example about the heartbeat parameters/metrics, Lambda Memory to changes these metrics based on customer requirements.

3. However, we are having the Ingress Autoscale & Egress traffic is through the Primary Firewall Instance using this template, but we expect that Egress traffic should also be there with HA/Autoscale between available firewalls instances instead of only one Primary Firewall.

4. There is a VPC Endpoint for API Gateway to communicate within the VPC (With Firewalls) but Lambda Function is in Public(Internet) If we integrate Lambda ASG Handler Function with VPC endpoint to make secure communication between all resources within the VPC there will be no latency issues between API Gateway & Lambda Function. And also sometimes observing the latency due to that Heartbeat Interval/HeartBeat Delay Allowance are getting increase which causes the Firewalls Termination.

[JaydenLiang]

Hi @ManikantaNandyala

1. Investigation is needed to evaluate the outcome of running the lambda fucntion in different percentage of Memory utilization. e.g. < 50%, < 100%, = 100%

2. we could improve the 'heartbeat parameters' portion in the future. The metric and Lambda memory usage portions can be adjusted by any AWS Solution Architect or whoever has AWS knowledge because the metric is AWS provided service, and documentations are availalbe on AWS. Recommended to adjust it based on the performance of individual working environments.

3. Not expected for the current Autoscale design. The egress traffic in this way was to allow for internal web services vm to be able to get necessary OS update from the Internet. It isn't designed for heavy egress traffic for other cases.

4. The connections are truly private via the private VPC Endpoint. Please refer to the first paragraph of this documentation: VPC Endpoints.

[ManikantaNandyala]

Hi @JaydenLiang

1. Please investigate & provide the best outcome for Lambda Function Memory.

2. Provide your suggestions for heartbeat parameters based on internal QA/Dev Testing.

3. For Egress traffic, we suggest you include the gateway load balancer towards the Fortigate Private ENI for the Available Zone. Note: In this scenario, Hybrid/BYOL or PAYG ASG should be with the Desired count of 2 Min.

4. Agreed with your comments on the Private VPC Endpoint. However, if we integrate the Private Endpoint to ASG Handler of Lambda Function it will reduce the delay/latency. Please let us know if you have any concerns or issues with private endpoint integration with ASG Handler of Lambda Function.

[JaydenLiang]

Hi @JaydenLiang

1. Please investigate & provide the best outcome for Lambda Function Memory.
2. Provide your suggestions for heartbeat parameters based on internal QA/Dev Testing.
3. For Egress traffic, we suggest you include the gateway load balancer towards the Fortigate Private ENI for the Available Zone. Note: In this scenario, Hybrid/BYOL or PAYG ASG should be with the Desired count of 2 Min.
4. Agreed with your comments on the Private VPC Endpoint. However, if we integrate the Private Endpoint to ASG Handler of Lambda Function it will reduce the delay/latency. Please let us know if you have any concerns or issues with private endpoint integration with ASG Handler of Lambda Function.

Hi @ManikantaNandyala , please find my comments as below:

Regarding p1, here is the helpful documentation for you: configuration-memory-optimization-accept
There is one overall best outcome: let your function run in 50% memory utilization. When it comes to your case, I suggest that you adjust your function configuration according to your needs in your environment. Configuring Lambda function options is a good guide for you.

Regarding p2, my suggestion: heartbeat interval = 30 seconds, heartbeat loss count = 10 times, heartbeat delay allowance = 2 seconds

Regarding p3, thanks for your suggestion. I don't have any comment on this for now since our project doesn't support this feature in the current version. GWLB is a different scenario. It might be discussed in the future project.

Regarding p4, please provide with me any AWS documentation that "integrate the Private Endpoint to ASG Handler of Lambda Function" refers to. My current undertanding of your saying is APIGateway private endpoints, which we already use it in the current version. Is there anything still missing to me? Please feel free to point it out to me with the related AWS documentation, thanks!