CrashLoopBackoff caused by error reading k8s api

[calbot]

I get CrashLoopBackoff in the controller because the following error:

It seems to be a transient error so I don't think the controller should be stopped when it happens. Can hairpin proxy be changed to report unhealthy instead of quitting the controller?

Happens when controller loads ingresses: @k8s.api(api_version).resource("ingresses").list

/usr/local/lib/ruby/2.7.0/openssl/buffering.rb:182:in `sysread_nonblock': Connection reset by peer (Errno::ECONNRESET) (Excon::Error::Socket)
	from /usr/local/lib/ruby/2.7.0/openssl/buffering.rb:182:in `read_nonblock'
	from /usr/local/bundle/gems/excon-0.79.0/lib/excon/socket.rb:179:in `read_nonblock'
	from /usr/local/bundle/gems/excon-0.79.0/lib/excon/socket.rb:63:in `readline'
	from /usr/local/bundle/gems/excon-0.79.0/lib/excon/response.rb:64:in `block in parse'
	from /usr/local/bundle/gems/excon-0.79.0/lib/excon/response.rb:63:in `loop'
	from /usr/local/bundle/gems/excon-0.79.0/lib/excon/response.rb:63:in `parse'
	from /usr/local/bundle/gems/excon-0.79.0/lib/excon/middlewares/response_parser.rb:7:in `response_call'
	from /usr/local/bundle/gems/excon-0.79.0/lib/excon/middlewares/redirect_follower.rb:82:in `response_call'
	from /usr/local/bundle/gems/excon-0.79.0/lib/excon/connection.rb:456:in `response'
	from /usr/local/bundle/gems/excon-0.79.0/lib/excon/connection.rb:287:in `request'
	from /usr/local/bundle/gems/k8s-ruby-0.10.5/lib/k8s/transport.rb:284:in `request'
	from /usr/local/bundle/gems/k8s-ruby-0.10.5/lib/k8s/transport.rb:363:in `get'
	from /usr/local/bundle/gems/k8s-ruby-0.10.5/lib/k8s/api_client.rb:46:in `api_resources!'
	from /usr/local/bundle/gems/k8s-ruby-0.10.5/lib/k8s/api_client.rb:54:in `api_resources'
	from /usr/local/bundle/gems/k8s-ruby-0.10.5/lib/k8s/api_client.rb:61:in `find_api_resource'
	from /usr/local/bundle/gems/k8s-ruby-0.10.5/lib/k8s/api_client.rb:73:in `resource'
	from /app/src/main.rb:30:in `block in fetch_ingress_hosts'
	from /app/src/main.rb:28:in `map'
	from /app/src/main.rb:28:in `fetch_ingress_hosts'
	from /app/src/main.rb:62:in `check_and_rewrite_coredns'
	from /app/src/main.rb:134:in `block in main_loop'
	from /app/src/main.rb:132:in `loop'
	from /app/src/main.rb:132:in `main_loop'
	from /app/src/main.rb:144:in `<main>'
/usr/local/lib/ruby/2.7.0/openssl/buffering.rb:182:in `sysread_nonblock': Connection reset by peer (Errno::ECONNRESET)
	from /usr/local/lib/ruby/2.7.0/openssl/buffering.rb:182:in `read_nonblock'
	from /usr/local/bundle/gems/excon-0.79.0/lib/excon/socket.rb:179:in `read_nonblock'
	from /usr/local/bundle/gems/excon-0.79.0/lib/excon/socket.rb:63:in `readline'
	from /usr/local/bundle/gems/excon-0.79.0/lib/excon/response.rb:64:in `block in parse'
	from /usr/local/bundle/gems/excon-0.79.0/lib/excon/response.rb:63:in `loop'
	from /usr/local/bundle/gems/excon-0.79.0/lib/excon/response.rb:63:in `parse'
	from /usr/local/bundle/gems/excon-0.79.0/lib/excon/middlewares/response_parser.rb:7:in `response_call'
	from /usr/local/bundle/gems/excon-0.79.0/lib/excon/middlewares/redirect_follower.rb:82:in `response_call'
	from /usr/local/bundle/gems/excon-0.79.0/lib/excon/connection.rb:456:in `response'
	from /usr/local/bundle/gems/excon-0.79.0/lib/excon/connection.rb:287:in `request'
	from /usr/local/bundle/gems/k8s-ruby-0.10.5/lib/k8s/transport.rb:284:in `request'
	from /usr/local/bundle/gems/k8s-ruby-0.10.5/lib/k8s/transport.rb:363:in `get'
	from /usr/local/bundle/gems/k8s-ruby-0.10.5/lib/k8s/api_client.rb:46:in `api_resources!'
	from /usr/local/bundle/gems/k8s-ruby-0.10.5/lib/k8s/api_client.rb:54:in `api_resources'
	from /usr/local/bundle/gems/k8s-ruby-0.10.5/lib/k8s/api_client.rb:61:in `find_api_resource'
	from /usr/local/bundle/gems/k8s-ruby-0.10.5/lib/k8s/api_client.rb:73:in `resource'
	from /app/src/main.rb:30:in `block in fetch_ingress_hosts'
	from /app/src/main.rb:28:in `map'
	from /app/src/main.rb:28:in `fetch_ingress_hosts'
	from /app/src/main.rb:62:in `check_and_rewrite_coredns'
	from /app/src/main.rb:134:in `block in main_loop'
	from /app/src/main.rb:132:in `loop'
	from /app/src/main.rb:132:in `main_loop'
	from /app/src/main.rb:144:in `<main>'