WIP: Document hash-based routing

hash-based-routing.html.md.erb

@@ -0,0 +1,49 @@
+---
+title: Hash-Based Routing
+owner: CF for VMs Networking
+---
+
+## <a id="purpose"></a> Purpose
+
+Hash-Based Routing is a load-balancing algorithm that distributes incoming requests to application instances based on a hash of a specific HTTP header value,  e.g., `X-Resource-ID` or `Tenant-ID`. This ensures consistent routing behavior where requests containing the same header value are always directed to the same instance.
+
+## <a id="key-features"></a> Key Features
+
+- **Configurable via Per-Route Options**: Hash-Based load-balancing setup through application route options
+- **Configurable Hash Header**: The HTTP header to consider for Hash-Based Routing is configurable for each route
+- **Session Affinity Precedence**: Session affinity (sticky sessions) is prioritized over Hash-Based Routing
+- **No availability zones preference**: The global properties `locallyOptimistic` and `localAvailabilityZone` are ignored when using Hash-Based Routing
+- **Consistent Hashing**: Implementation of the Maglev Algorithm (see [Maglev: A Fast and Reliable Software Network Load Balancer](https://storage.googleapis.com/gweb-research2023-media/pubtools/2904.pdf) for details)
+- **Minimal Rehashing**: Usage of the Maglev lookup table to map application instances by hash ensures that hash positions are shifted to other instances as little as possible when application instances are added or removed
+- **Handling imbalanced loads**: Detection and mitigation of higher request loads on individual instances prevents overloading while keeping instances for a particular hash at a minimum.
+
+Hash-Based Routing implements a clear precedence hierarchy:
+
+1. **Sticky Sessions**: First checks if a sticky session is used and the sticky session endpoint is available
+2. **Hash-Based Routing**: Calculates the hash of the specified HTTP header value and routes the requests to a pre-determined application instance
+3. **Default Load Balancing**: Falls back to default load balancing if the header configured in the application route for Hash-Based Routing is absent in the request
+
+
+## <a id="hash-based-vs-session-affinity"></a> Hash-Based Routing vs. Session Affinity
+
+Using Session Affinity, requests from a particular client are always routed to the same app instance. This allows apps to store session data specific to a user session (see more details in the [Session Affinity](https://docs.cloudfoundry.org/concepts/http-routing.html#-session-affinity) documentation).
+
+While Session Affinity offers a solution for maintaining session consistency, it poses scalability challenges, as Session Affinity is confined to a single instance. It increases the risk that large customers could be routed to the same instance by chance. On the other hand, enabling Session Affinity requires additional implementation effort on the application side to return a sticky session cookie in responses.
+
+In contrast, Hash-Based Routing provides a more scalable and balanced approach by consistently distributing requests based on a hash of a specific HTTP header value. This hash value determines the appropriate application instance for each request, ensuring that requests with the same hash are consistently routed to the same instance, but might be routed to another predetermined instance when the current one is saturated (find more about Handling imbalanced loads below). In contrast to Session Affinity, no code change is required.
+
+This makes it especially suitable for applications requiring high scalability and performance, such as microservices architectures or multi-tenant applications, when dealing with limited or memory-intensive resources in backend services.
+
+
+## <a id="handling-imbalance-loads"></a> Handling imbalanced loads
+
+The implementation of Hash-Based Routing includes mechanisms to detect imbalanced load across application instances. Imbalanced load can occur when certain hashes receive more traffic, such as when a particular tenant generates a large number of requests, resulting in heavier use of their mapped instances compared to others. Moreover, multiple high-traffic targets could be assigned to the same instance.
+
+To prevent overloading specific instances while others remain under-utilized, the acceptable threshold for load imbalance can be configured using the `hash_balance_factor` property.
+This factor determines whether an instance is handling more traffic than its fair share compared to the average load across all application instances, measured by the number of in-flight requests. For instance, with a balance factor of 1.25, no single instance should exceed 125% of the average number of in-flight requests across all instances managed by the current router. Consequently, the router redirects the next requests to other predetermined instances that are not overloaded.
+
+This ensures that a minimum number of instances process requests for a particular hash while preventing any single instance from becoming overloaded.
+
+## <a id="retries-in-hash-based"></a> Retries in Hash-Based Routing
+
+For idempotent requests, the retry mechanism is supported in Hash-Based Routing. When a request fails due to a network error or a 5xx response from the application instance, the router retries the request with a different predetermined application instance. The next entry in the predefined Maglev lookup table determines this instance. This is the same approach used in handling imbalanced loads. This ensures that the retry mechanism adheres to the principles of Hash-Based Routing while providing resilience against transient failures.