From 69b33d007a72669d4d22a0a38cad828de9efd348 Mon Sep 17 00:00:00 2001 From: Tamara Boehm Date: Mon, 3 Nov 2025 14:24:49 +0100 Subject: [PATCH 1/5] WIP: Document hash-based routing --- hash-based-routing.html.md.erb | 47 ++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) create mode 100644 hash-based-routing.html.md.erb diff --git a/hash-based-routing.html.md.erb b/hash-based-routing.html.md.erb new file mode 100644 index 00000000..4f2655f4 --- /dev/null +++ b/hash-based-routing.html.md.erb @@ -0,0 +1,47 @@ +--- +title: Hash-Based Routing +owner: CF for VMs Networking +--- + +## Purpose + +Hash-Based Routing is a load-balancing algorithm that distributes incoming requests to application instances based on a hash of a specific HTTP header value. This ensures consistent routing behavior where requests containing the same header value are always directed to the same instance. + +## Key Features + +- **Configurable via Per-Route Options**: Easily set up Hash-Based Routing using application route options +- **Consistent Hashing**: Implements the Maglev consistent hashing algorithm as outlined in the paper "Maglev: A Fast and Reliable Software Network Load Balancer" (https://storage.googleapis.com/gweb-research2023-media/pubtools/2904.pdf) +- **Minimal Rehashing**: Uses the Maglev lookup table to map application instances by hash, minimizing the need to recalculate the entire table when instances are added or removed +- **Configurable Hash Header**: Allows specifying which HTTP header to use for hashing via the `hash_header` property (e.g., `X-Resource-ID`) +- **Sticky Session Precedence**: Prioritizes sticky sessions over hash-based routing when available +- **Handling imbalanced loads**: Implements balance factor-based overload detection and mitigation +- **No availability zones preference**: The global properties `locallyOptimistic` and `localAvailabilityZone` will be ignored when using Hash-Based Routing + +Hash-Based Routing implements a clear precedence hierarchy: + +1. **Sticky Sessions**: First checks if a sticky session is used and the sticky session endpoint is available +2. **Hash-Based Routing**: Calculates the hash of the specified HTTP header value and routes the requests to a pre-determined application instance +3. **Default Load Balancing**: Falls back to default load balancing (round-robin) if the header configured in the application route for hash-based routing is absent in the request + + +## Hash-Based Routing vs. Session Affinity + +Using Session Affinity, requests from a particular client are always routed to the same app instance. This allows apps to store session data specific to a user session (see more details in the [Session Affinity](https://docs.cloudfoundry.org/concepts/http-routing.html#-session-affinity) documentation). + +While Session Affinity offers a solution for maintaining session consistency, it poses scalability challenges, as Session Affinity is confined to a single instance. It increases the risk that large customers could be routed to the same instance by chance. On the other hand, enabling Session Affinity requires additional implementation effort on the application side to return a sticky session cookie in responses. + +In contrast, Hash-Based Routing provides a more scalable and balanced approach by consistently distributing requests based on a hash of a specific HTTP header value (e.g., `X-Resource-ID` or `Tenant-ID`). This hash value determines the appropriate application instance for each request, ensuring that requests with the same hash are consistently routed to the same instance, but might be routed to another predetermined instance when the current one is saturated (find more about Handling imbalanced loads below). + +This makes it especially suitable for applications requiring high scalability and performance, such as microservices architectures or multi-tenant applications, when dealing with limited or memory-intensive resources in backend services. + + +## Handling imbalanced loads + +The implementation of the hash-based routing includes mechanisms to detect imbalanced loads across application instances. This is achieved through a balance factor that defines the acceptable threshold for load imbalance, which is configurable via the `hash_balance_factor` property. + +When an application instance is being considered for selection, it will be checked whether it is handling more traffic than its fair share compared to the average load across all application instances. This load is measured by the number of in-flight requests. For example, with a balance factor of 1.25, no application instance should exceed 125% of the average number of in-flight requests across all application instances initiated by the current router. Consequently, requests must be distributed to different pre-determined application instance that is not overloaded. + + +## Retries in Hash-Based Routing + +For the idempotent requests, the retry mechanism is supported in Hash-Based Routing. When a request fails due to a network error or a 5xx response from the application instance, the router retries the request with a different predetermined application instance. The next entry in the predefined Maglev lookup table determines this instance. This is the same approach used in handling imbalanced loads. This ensures that the retry mechanism adheres to the principles of Hash-Based Routing while providing resilience against transient failures. From 2681f4b2dcfb0a70d2974e2c9788fe6b83ec8ee3 Mon Sep 17 00:00:00 2001 From: Tamara Boehm <34028368+b1tamara@users.noreply.github.com> Date: Tue, 4 Nov 2025 11:02:10 +0100 Subject: [PATCH 2/5] Apply suggestions from code review Co-authored-by: Alexander Lais --- hash-based-routing.html.md.erb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/hash-based-routing.html.md.erb b/hash-based-routing.html.md.erb index 4f2655f4..c2a7e274 100644 --- a/hash-based-routing.html.md.erb +++ b/hash-based-routing.html.md.erb @@ -14,7 +14,7 @@ Hash-Based Routing is a load-balancing algorithm that distributes incoming reque - **Minimal Rehashing**: Uses the Maglev lookup table to map application instances by hash, minimizing the need to recalculate the entire table when instances are added or removed - **Configurable Hash Header**: Allows specifying which HTTP header to use for hashing via the `hash_header` property (e.g., `X-Resource-ID`) - **Sticky Session Precedence**: Prioritizes sticky sessions over hash-based routing when available -- **Handling imbalanced loads**: Implements balance factor-based overload detection and mitigation +- **Handling imbalanced loads**: Implements detection and mitigation of higher request loads on single instances due to different usage patterns for specific hashes. This avoids overloading a single instance while keeping instances for a particular hash at a minimum. - **No availability zones preference**: The global properties `locallyOptimistic` and `localAvailabilityZone` will be ignored when using Hash-Based Routing Hash-Based Routing implements a clear precedence hierarchy: @@ -44,4 +44,4 @@ When an application instance is being considered for selection, it will be check ## Retries in Hash-Based Routing -For the idempotent requests, the retry mechanism is supported in Hash-Based Routing. When a request fails due to a network error or a 5xx response from the application instance, the router retries the request with a different predetermined application instance. The next entry in the predefined Maglev lookup table determines this instance. This is the same approach used in handling imbalanced loads. This ensures that the retry mechanism adheres to the principles of Hash-Based Routing while providing resilience against transient failures. +For idempotent requests, the retry mechanism is supported in Hash-Based Routing. When a request fails due to a network error or a 5xx response from the application instance, the router retries the request with a different predetermined application instance. The next entry in the predefined Maglev lookup table determines this instance. This is the same approach used in handling imbalanced loads. This ensures that the retry mechanism adheres to the principles of Hash-Based Routing while providing resilience against transient failures. From 58cd0d51ebcdae00c05909480f5bcb5cf1d7d394 Mon Sep 17 00:00:00 2001 From: Tamara Boehm Date: Tue, 4 Nov 2025 12:44:36 +0100 Subject: [PATCH 3/5] Apply review feedback --- hash-based-routing.html.md.erb | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/hash-based-routing.html.md.erb b/hash-based-routing.html.md.erb index c2a7e274..33f09bdf 100644 --- a/hash-based-routing.html.md.erb +++ b/hash-based-routing.html.md.erb @@ -5,15 +5,15 @@ owner: CF for VMs Networking ## Purpose -Hash-Based Routing is a load-balancing algorithm that distributes incoming requests to application instances based on a hash of a specific HTTP header value. This ensures consistent routing behavior where requests containing the same header value are always directed to the same instance. +Hash-Based Routing is a load-balancing algorithm that distributes incoming requests to application instances based on a hash of a specific HTTP header value, e.g., `X-Resource-ID` or `Tenant-ID`. This ensures consistent routing behavior where requests containing the same header value are always directed to the same instance. ## Key Features - **Configurable via Per-Route Options**: Easily set up Hash-Based Routing using application route options - **Consistent Hashing**: Implements the Maglev consistent hashing algorithm as outlined in the paper "Maglev: A Fast and Reliable Software Network Load Balancer" (https://storage.googleapis.com/gweb-research2023-media/pubtools/2904.pdf) -- **Minimal Rehashing**: Uses the Maglev lookup table to map application instances by hash, minimizing the need to recalculate the entire table when instances are added or removed -- **Configurable Hash Header**: Allows specifying which HTTP header to use for hashing via the `hash_header` property (e.g., `X-Resource-ID`) -- **Sticky Session Precedence**: Prioritizes sticky sessions over hash-based routing when available +- **Minimal Rehashing**: Uses the Maglev lookup table to map application instances by hash, minimizing changes to hash assignments when application instances are added or removed +- **Configurable Hash Header**: Allows specifying which HTTP header to use for hashing via the `hash_header` property +- **Session Affinity Precedence**: Prioritizes session affinity, or sticky sessions, over hash-based routing when available - **Handling imbalanced loads**: Implements detection and mitigation of higher request loads on single instances due to different usage patterns for specific hashes. This avoids overloading a single instance while keeping instances for a particular hash at a minimum. - **No availability zones preference**: The global properties `locallyOptimistic` and `localAvailabilityZone` will be ignored when using Hash-Based Routing @@ -21,7 +21,7 @@ Hash-Based Routing implements a clear precedence hierarchy: 1. **Sticky Sessions**: First checks if a sticky session is used and the sticky session endpoint is available 2. **Hash-Based Routing**: Calculates the hash of the specified HTTP header value and routes the requests to a pre-determined application instance -3. **Default Load Balancing**: Falls back to default load balancing (round-robin) if the header configured in the application route for hash-based routing is absent in the request +3. **Default Load Balancing**: Falls back to default load balancing if the header configured in the application route for hash-based routing is absent in the request ## Hash-Based Routing vs. Session Affinity @@ -30,17 +30,19 @@ Using Session Affinity, requests from a particular client are always routed to t While Session Affinity offers a solution for maintaining session consistency, it poses scalability challenges, as Session Affinity is confined to a single instance. It increases the risk that large customers could be routed to the same instance by chance. On the other hand, enabling Session Affinity requires additional implementation effort on the application side to return a sticky session cookie in responses. -In contrast, Hash-Based Routing provides a more scalable and balanced approach by consistently distributing requests based on a hash of a specific HTTP header value (e.g., `X-Resource-ID` or `Tenant-ID`). This hash value determines the appropriate application instance for each request, ensuring that requests with the same hash are consistently routed to the same instance, but might be routed to another predetermined instance when the current one is saturated (find more about Handling imbalanced loads below). +In contrast, Hash-Based Routing provides a more scalable and balanced approach by consistently distributing requests based on a hash of a specific HTTP header value. This hash value determines the appropriate application instance for each request, ensuring that requests with the same hash are consistently routed to the same instance, but might be routed to another predetermined instance when the current one is saturated (find more about Handling imbalanced loads below). This makes it especially suitable for applications requiring high scalability and performance, such as microservices architectures or multi-tenant applications, when dealing with limited or memory-intensive resources in backend services. ## Handling imbalanced loads -The implementation of the hash-based routing includes mechanisms to detect imbalanced loads across application instances. This is achieved through a balance factor that defines the acceptable threshold for load imbalance, which is configurable via the `hash_balance_factor` property. +The implementation of the hash-based routing includes mechanisms to detect imbalanced loads across application instances. Imbalanced loads can occur when certain hashes receive more traffic, such as when a particular tenant generates a large number of requests, resulting in heavier use of their mapped instances compared to others. Moreover, multiple high-traffic targets could be assigned to the same instance. -When an application instance is being considered for selection, it will be checked whether it is handling more traffic than its fair share compared to the average load across all application instances. This load is measured by the number of in-flight requests. For example, with a balance factor of 1.25, no application instance should exceed 125% of the average number of in-flight requests across all application instances initiated by the current router. Consequently, requests must be distributed to different pre-determined application instance that is not overloaded. +To prevent overloading specific instances while others remain under-utilized, the acceptable threshold for load imbalance can be configured using the `hash_balance_factor` property. +This factor determines whether an instance is handling more traffic than its fair share compared to the average load across all application instances, measured by the number of in-flight requests. For instance, with a balance factor of 1.25, no single instance should exceed 125% of the average number of in-flight requests across all instances managed by the current router. Consequently, the router redirected the next requests to other predetermined instances that are not overloaded. +This ensures that a minimum number of instances process requests for a particular hash while preventing any single instance from becoming overloaded. ## Retries in Hash-Based Routing From 73de4a82574fa4ebf317a9f9433fba184fcfc91a Mon Sep 17 00:00:00 2001 From: Tamara Boehm <34028368+b1tamara@users.noreply.github.com> Date: Fri, 7 Nov 2025 09:12:53 +0100 Subject: [PATCH 4/5] Apply suggestions from code review Co-authored-by: Clemens Hoffmann --- hash-based-routing.html.md.erb | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/hash-based-routing.html.md.erb b/hash-based-routing.html.md.erb index 33f09bdf..2126e04a 100644 --- a/hash-based-routing.html.md.erb +++ b/hash-based-routing.html.md.erb @@ -12,16 +12,16 @@ Hash-Based Routing is a load-balancing algorithm that distributes incoming reque - **Configurable via Per-Route Options**: Easily set up Hash-Based Routing using application route options - **Consistent Hashing**: Implements the Maglev consistent hashing algorithm as outlined in the paper "Maglev: A Fast and Reliable Software Network Load Balancer" (https://storage.googleapis.com/gweb-research2023-media/pubtools/2904.pdf) - **Minimal Rehashing**: Uses the Maglev lookup table to map application instances by hash, minimizing changes to hash assignments when application instances are added or removed -- **Configurable Hash Header**: Allows specifying which HTTP header to use for hashing via the `hash_header` property -- **Session Affinity Precedence**: Prioritizes session affinity, or sticky sessions, over hash-based routing when available +- **Configurable Hash Header**: The HTTP Header to consider for Hash-Based Routing is configurable for each route +- **Session Affinity Precedence**: Session affinity (sticky sessions) is prioritized over Hash-Based Routing. - **Handling imbalanced loads**: Implements detection and mitigation of higher request loads on single instances due to different usage patterns for specific hashes. This avoids overloading a single instance while keeping instances for a particular hash at a minimum. -- **No availability zones preference**: The global properties `locallyOptimistic` and `localAvailabilityZone` will be ignored when using Hash-Based Routing +- **No availability zones preference**: The global properties `locallyOptimistic` and `localAvailabilityZone` are ignored when using Hash-Based Routing Hash-Based Routing implements a clear precedence hierarchy: 1. **Sticky Sessions**: First checks if a sticky session is used and the sticky session endpoint is available 2. **Hash-Based Routing**: Calculates the hash of the specified HTTP header value and routes the requests to a pre-determined application instance -3. **Default Load Balancing**: Falls back to default load balancing if the header configured in the application route for hash-based routing is absent in the request +3. **Default Load Balancing**: Falls back to default load balancing if the header configured in the application route for Hash-Based Routing is absent in the request ## Hash-Based Routing vs. Session Affinity @@ -30,17 +30,17 @@ Using Session Affinity, requests from a particular client are always routed to t While Session Affinity offers a solution for maintaining session consistency, it poses scalability challenges, as Session Affinity is confined to a single instance. It increases the risk that large customers could be routed to the same instance by chance. On the other hand, enabling Session Affinity requires additional implementation effort on the application side to return a sticky session cookie in responses. -In contrast, Hash-Based Routing provides a more scalable and balanced approach by consistently distributing requests based on a hash of a specific HTTP header value. This hash value determines the appropriate application instance for each request, ensuring that requests with the same hash are consistently routed to the same instance, but might be routed to another predetermined instance when the current one is saturated (find more about Handling imbalanced loads below). +In contrast, Hash-Based Routing provides a more scalable and balanced approach by consistently distributing requests based on a hash of a specific HTTP header value. This hash value determines the appropriate application instance for each request, ensuring that requests with the same hash are consistently routed to the same instance, but might be routed to another predetermined instance when the current one is saturated (find more about Handling imbalanced loads below). In contrast to Session Affinity, no code change is required. This makes it especially suitable for applications requiring high scalability and performance, such as microservices architectures or multi-tenant applications, when dealing with limited or memory-intensive resources in backend services. ## Handling imbalanced loads -The implementation of the hash-based routing includes mechanisms to detect imbalanced loads across application instances. Imbalanced loads can occur when certain hashes receive more traffic, such as when a particular tenant generates a large number of requests, resulting in heavier use of their mapped instances compared to others. Moreover, multiple high-traffic targets could be assigned to the same instance. +The implementation of Hash-Based Routing includes mechanisms to detect imbalanced load across application instances. Imbalanced load can occur when certain hashes receive more traffic, such as when a particular tenant generates a large number of requests, resulting in heavier use of their mapped instances compared to others. Moreover, multiple high-traffic targets could be assigned to the same instance. To prevent overloading specific instances while others remain under-utilized, the acceptable threshold for load imbalance can be configured using the `hash_balance_factor` property. -This factor determines whether an instance is handling more traffic than its fair share compared to the average load across all application instances, measured by the number of in-flight requests. For instance, with a balance factor of 1.25, no single instance should exceed 125% of the average number of in-flight requests across all instances managed by the current router. Consequently, the router redirected the next requests to other predetermined instances that are not overloaded. +This factor determines whether an instance is handling more traffic than its fair share compared to the average load across all application instances, measured by the number of in-flight requests. For instance, with a balance factor of 1.25, no single instance should exceed 125% of the average number of in-flight requests across all instances managed by the current router. Consequently, the router redirects the next requests to other predetermined instances that are not overloaded. This ensures that a minimum number of instances process requests for a particular hash while preventing any single instance from becoming overloaded. From 6d7a71cfc771a551ab1e4734f3bc9546cafefa22 Mon Sep 17 00:00:00 2001 From: Tamara Boehm Date: Fri, 7 Nov 2025 09:35:35 +0100 Subject: [PATCH 5/5] Rephrase --- hash-based-routing.html.md.erb | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/hash-based-routing.html.md.erb b/hash-based-routing.html.md.erb index 2126e04a..18e13b25 100644 --- a/hash-based-routing.html.md.erb +++ b/hash-based-routing.html.md.erb @@ -9,13 +9,13 @@ Hash-Based Routing is a load-balancing algorithm that distributes incoming reque ## Key Features -- **Configurable via Per-Route Options**: Easily set up Hash-Based Routing using application route options -- **Consistent Hashing**: Implements the Maglev consistent hashing algorithm as outlined in the paper "Maglev: A Fast and Reliable Software Network Load Balancer" (https://storage.googleapis.com/gweb-research2023-media/pubtools/2904.pdf) -- **Minimal Rehashing**: Uses the Maglev lookup table to map application instances by hash, minimizing changes to hash assignments when application instances are added or removed -- **Configurable Hash Header**: The HTTP Header to consider for Hash-Based Routing is configurable for each route -- **Session Affinity Precedence**: Session affinity (sticky sessions) is prioritized over Hash-Based Routing. -- **Handling imbalanced loads**: Implements detection and mitigation of higher request loads on single instances due to different usage patterns for specific hashes. This avoids overloading a single instance while keeping instances for a particular hash at a minimum. +- **Configurable via Per-Route Options**: Hash-Based load-balancing setup through application route options +- **Configurable Hash Header**: The HTTP header to consider for Hash-Based Routing is configurable for each route +- **Session Affinity Precedence**: Session affinity (sticky sessions) is prioritized over Hash-Based Routing - **No availability zones preference**: The global properties `locallyOptimistic` and `localAvailabilityZone` are ignored when using Hash-Based Routing +- **Consistent Hashing**: Implementation of the Maglev Algorithm (see [Maglev: A Fast and Reliable Software Network Load Balancer](https://storage.googleapis.com/gweb-research2023-media/pubtools/2904.pdf) for details) +- **Minimal Rehashing**: Usage of the Maglev lookup table to map application instances by hash ensures that hash positions are shifted to other instances as little as possible when application instances are added or removed +- **Handling imbalanced loads**: Detection and mitigation of higher request loads on individual instances prevents overloading while keeping instances for a particular hash at a minimum. Hash-Based Routing implements a clear precedence hierarchy: