"Keep Alive" config for Hasura hosted on AWS ECS

[willtpwise]

We’ve been trying to find the appropriate documentation for Hasura used on AWS ECS, specifically relating to keep alive timeout configuration between the Hasura process running on ECS and the AWS ALB. There’s a great doc which here but it doesn’t cover the specific topic around keep alive timeouts.

Why I’m asking is that we recently noticed our Hasura instance is seeing sporadic 502 errors at the load balancer. The timing of the 502s is random and doesn't line up with any resource issues or scaling events. See screenshot below with several 502s over a 12 hour period.

We also noticed another ALB/ECS setup with similar issues. This infrastructure was running a Next.js app and we learnt that the recommended configuration for an AWS ALB is to set the application’s timeout (i.e. Hasura) to be longer than the ALB timeout (default of 60s). As soon as we increased the keep alive timeout at the Next.js app to 61s the 502 errors stopped.

[manasag]

This is a known issue on our side, and unfortunately with no solution. Hasura is written in Haskell, and the http server library dosn't support configurable keep-alive timeouts yet. Here is Hasura's comment on the still open issue on the haskell library.