topic Sticky connections to Viya compute server via HAProxy in Administration and Deployment

Sticky connections to Viya compute server via HAProxy

jklaverstijn — Wed, 30 Mar 2022 07:10:07 GMT

Hi all,

We have users that want to use EG to connect to a Viya compute server. While advertising StudioV there are still EG strongholds in our organization. In order to avoid them all connecting to the same compute host (we have anywhere between 2 and 5 depending on the deployment size) we want to utilize a load-balancer. For now (we hope to get an F5 later this year) HAProxy seems a good choice for this. But we are struggling to get the connections "sticky" enough. As long as one uses a single EG session there is no problem. But having multiple EG connections simultaneously we get connection errors ("connection lost") when switching between the EG's. This appears to be caused by HAProxy sending a request from session A to the host assigned to session B. We can probably solve this but at the cost of tying a client pc to always the same compute host. Something we like to avoid.

Maybe HAProxy is too much designed for HTTP and TCP has come as an afterthought? Has anyone been successful in configuring TCP (Layer4) stickiness in HAProxy for this scenario? Would you be willing to share your config?

Kind regards,

- Jan.

Re: Sticky connections to Viya compute server via HAProxy

gwootton — Wed, 30 Mar 2022 13:04:39 GMT

It seems HAProxy supports source IP affinity. This might be a solution for you if the problem is that the same client should prefer to go to the same host in separate sessions.

https://www.haproxy.com/blog/client-ip-persistence-or-source-ip-hash-load-balancing/

Re: Sticky connections to Viya compute server via HAProxy

jklaverstijn — Thu, 31 Mar 2022 14:12:22 GMT

Thanks for the link Greg. I will point our proxy people to this resource. Looks like it must be possible given a certain amount of experimentation and determination. Will report back once we have result.

Cheers,
- Jan.

Re: Sticky connections to Viya compute server via HAProxy

jklaverstijn — Wed, 22 Jun 2022 10:06:26 GMT

Our proxy people fixed this. They implemented the roundrobbin approach. This caused the proper behaviour of choosing a new host for every new EG connection. This didn't work at first as a new submit after a minute of idle time resulted in a "lost connection" error. In fact the connection wasn't lost but removed from the proxy list and a new host (wrong) was assigned. In HAProxy this is managed by some time-out value that defaults to 30s. That is very short but only for stateful connections where HAProxy was designed with stateless in mind. So bumping the default to 18 hours resulted in the desired behaviour: every new EG connection gets a new host assigned from the list, even when coming from the same IP address. If anyone wants to know I can ask the colleagues for the exact config.

Re: Sticky connections to Viya compute server via HAProxy

JuanS_OCS — Wed, 22 Jun 2022 10:27:47 GMT

Hi Jan,

this is wonderful, it makes sense to me. Personally, I would love to know about the exact configuration that was implemented.

Thank you in advance!

Best regards

Juan

Re: Sticky connections to Viya compute server via HAProxy

jklaverstijn — Wed, 22 Jun 2022 11:02:53 GMT

Here it is @JuanS_OCS :

defaults
  mode tcp
  log  global

  option http-server-close

  timeout client 18h
  timeout server 18h
  timeout connect 4s
  timeout queue  60s

  timeout client-fin 1s
  timeout server-fin 1s

  timeout http-request 10s
  timeout http-keep-alive 10s

  maxconn 250
  fullconn 25

frontend frontend_tcp_in
   mode tcp
   bind 10.235.41.32:16000-32000
   default_backend  backend_tcp_out

backend backend_tcp_out
   mode tcp
   server spre1 10.235.41.225
   server spre2 10.235.41.224