Exposing Event Broker Services to External Traffic

As shown in the Deployment Architecture for Kubernetes, event broker services run in pods on the worker nodes in your Kubernetes cluster.

Each event broker service consists of three software event brokers deployed across three pods. Each of these pods has a unique IP address (pod IP) which can't be exposed outside the cluster except through a Kubernetes service. A Kubernetes service is an object that abstracts and exposes an application running on a set of pods as a network service. The Mission Control Agent configures the Kubernetes service to route external traffic to the software event brokers running inside the Kubernetes cluster. With external traffic exposed, clients outside the Kubernetes cluster can interact with event broker services, either from other customer VPCs or, optionally, from the Internet.

Kubernetes services can be exposed in different ways by using different ServiceTypes (specified in the type property in the Kubernetes Service Specification). There are three ServiceTypes you (the customer) can use to expose your event broker services outside the cluster:

  • LoadBalancer: Exposes the Kubernetes service externally using a cloud provider's load balancer. This is the simplest way to expose a service but requires an L4 load balancer supported by the cluster's cloud provider. Load balancer is the only ServiceType that supports Public/Private endpoint connectivity. See Configuring Connection Endpoints and Ports for Event Broker Services for more information. Load balancer supports port customization and the ability to disable and enable individual protocols.
  • NodePort: Exposes the Kubernetes service on each Node's IP address at a static port (the NodePort). A LoadBalancer, which routes to the NodePort Service, must be manually created (usually by a network administrator) to front the NodePort service. You can contact the NodePort Service, from outside the cluster, by requesting <NodeIP>:<NodePort>. NodePort does not allow port customization, but supports the ability to disable and enable individual protocols.
  • ClusterIP: Exposes the Kubernetes service on a cluster-internal IP address. Choosing this value makes the Kubernetes service reachable only from within the cluster. This is the default ServiceType. ClusterIP supports port customization and the ability to disable and enable individual protocols.

For more details about Kubernetes services, load balancing, and networking, see the Kubernetes documentation.

Consider the following when deciding which of these strategies to use:

  • Solace recommends that you use an integrated load balancer (Kubernetes service type of LoadBalancer). Specifically, integrated load balancer solutions must use an external network load balancer with an L4 load balancing strategy.
  • If an integrated load balancer is not available, you can use NodePort or ClusterIP with External IP to expose the event broker service within the cluster to external traffic.
  • If you don't need to expose the event broker service outside of its cluster (that is, clients connect from the same cluster), then consider using ClusterIP only. For more information, see Not Exposing Event Broker Services to External Traffic.

If you need more help to decide, see the considerations in Questionnaire: Deploying in a Customer-Controlled Region.

The options for exposing your event broker services are summarized below, in order of preference.

Option 1: Integrated Load Balancer

This is the recommended solution that permits each event broker service to be accessible via its own endpoint (IP address and optional port). This solution specifies that each event broker service that's exposed has its own network load balancer. In this solution, the Kubernetes Service type is set to LoadBalancer. From a high-level perspective, this solution permits connectivity from outside the cluster to the event broker service, as shown in the following diagram.

Considerations:

  • Solace recommends that you use an load balancer solution that's integrated with Kubernetes (e.g.,MetalLB, VMWare NST) to expose the cluster to external traffic. These integrated solutions automatically create a network load balancer for you to simplify deployment.
  • Determine with the Kubernetes administrator whether services of type LoadBalancer are supported and whether any special annotations are needed to create a network load balancer.
  • To support an integrated load balancer setup, ensure that you determine with your Kubernetes administrator that an integrated load balancer is available.
  • For planning purposes, this solution requires one load balancer and that load balancer consumes one IP address. For example, if you have three services, they require three load balancers and three IP addresses.
  • Solace recommends setting the load balancer's externalTrafficPolicy to local. The local policy setting preserves the source IP of connecting client applications. For more information see our traffic policy recommendation in Using an Integrated Load Balancer Solution.

For more in details about the architecture and advantages of this solution, see Using an Integrated Load Balancer Solution.

Option 2: NodePort with an External Network Load Balancer

This is a solution that's well-known and available across all Kubernetes clusters. This approach requires that customers configure the Kubernetes Service type to NodePort and manually provision an external network load balancer that redirects traffic to the Kubernetes cluster.

Considerations:

  • You must use an external network load balancer in this configuration for robustness and fault-tolerant routing, but you can use any means to map an external IP address to an IP address and port within your private network.
  • This approach is similar to the integrated load balancer solution, but you don't have one network load balancer per service and services are accessed using port numbers.

For more in details about the architecture and advantages of this solution, see Using NodePort.

Option 3: ClusterIP with External IP

You can use this solution when it is possible to set up a private IP addresses within your private network and have those private IP addresses assigned to worker nodes using Border Gateway Protocol (BGP).

Considerations:

  • This option permits you to assign IP addresses to a group of services, which allows traffic be routed to the Kubernetes worker node with the external IP address set as the destination.
  • This option doesn't require a load balancer and is simple to deploy if you use BGP.

For more in details about the architecture and advantages of this solution, see Using ClusterIP with External IP.