Implement Hash-Based Routing by hoffmaen · Pull Request #519 · cloudfoundry/routing-release

hoffmaen · 2025-11-11T15:22:59Z

Read the Contributing document.

Summary

This Pull-Request implements #505.

Implement hash-based routing in Gorouter
- Gorouter routes requests to endpoints based on the value of a configured request header
- Gorouter distributes requests to different predefined application instances, when the current request count to an app instance exceeds the balance factor
- Gorouter routes requests to the "next instance" in the lookup table on retries
Follow decisions documented in this comment

Backward Compatibility

Breaking Change? Yes/No

src/code.cloudfoundry.org/gorouter/proxy/round_tripper/proxy_round_tripper.go

src/code.cloudfoundry.org/gorouter/route/maglev.go

src/code.cloudfoundry.org/gorouter/route/pool.go

peanball

First set of comments. I have not looked at the implementation of maglev and the hash based instance determination.

I've focussed on the integration with the existing Gorouter code, route lookup, LB algo handling, etc.

src/code.cloudfoundry.org/gorouter/route/pool.go

peanball · 2025-12-09T09:08:19Z

src/code.cloudfoundry.org/gorouter/route/pool.go

 }

-func (p *EndpointPool) Endpoints(logger *slog.Logger, initial string, mustBeSticky bool, azPreference string, az string) EndpointIterator {
+func (p *EndpointPool) Endpoints(logger *slog.Logger, initial string, mustBeSticky bool, azPreference string, az string, globalLB string, request *http.Request) EndpointIterator {


you're passing in the full request to then dynamically fetch the hash based header. Why not pass an optional reference to *HashRoutingProperties? Parsing then can be done at the call site where you have the request anyway.

src/code.cloudfoundry.org/gorouter/route/pool.go

peanball · 2025-12-09T09:14:17Z

src/code.cloudfoundry.org/gorouter/route/pool.go

+		logger.Debug("endpoint-iterator-with-round-robin-lb-algo")
+		return NewRoundRobin(logger, p, initial, mustBeSticky, azPreference == config.AZ_PREF_LOCAL, az)
+	}
+	return NewRoundRobin(logger, p, initial, mustBeSticky, azPreference == config.AZ_PREF_LOCAL, az)


you could also move this to the default: to make it clear in the switch. There is also no debug log for this branch.

src/code.cloudfoundry.org/gorouter/route/pool.go

src/code.cloudfoundry.org/gorouter/route/hash_based.go

src/code.cloudfoundry.org/gorouter/registry/registry_test.go

src/code.cloudfoundry.org/gorouter/handlers/helpers.go

peanball · 2026-01-16T08:55:17Z

src/code.cloudfoundry.org/gorouter/route/hash_based.go

+	if e != nil {
+		e.RLock()
+		defer e.RUnlock()
+		return e.endpoint


e might not be valid anymore, as it is retrieved without lock. You then acquire a lock and should re-fetch the value to be sure that it's still there.

src/code.cloudfoundry.org/gorouter/route/pool.go

This commit provides the basic implementation for hash-based routing. It does not consider the balance factor yet. Co-authored-by: Clemens Hoffmann <clemens.hoffmann@sap.com> Co-authored-by: Tamara Boehm <tamara.boehm@sap.com> Co-authored-by: Soha Alboghdady <soha.alboghdady@sap.com>

peanball

Some more comments, now on the hash_based.go

src/code.cloudfoundry.org/gorouter/route/hash_based.go

peanball · 2026-01-16T15:39:17Z

src/code.cloudfoundry.org/gorouter/route/hash_based.go

+	balanceFactor := h.pool.HashRoutingProperties.BalanceFactor
+
+	if isEndpointOverloaded {
+		h.logger.Debug("hash-based-routing-endpoint-overloaded", slog.String("host", h.pool.host), slog.String("endpoint-id", endpoint.PrivateInstanceId), slog.Int64("endpoint-connections", currentInFlightRequestCount))


as debug log is usually not on, for performance's sake it's better to guard it by checking if debug level is enabled at all in the logger.

something like

if !h.logger.Enabled(context.Background(), slog.LevelDebug) { h.logger.Debug(...) }

src/code.cloudfoundry.org/gorouter/route/hash_based.go

peanball · 2026-01-16T16:03:30Z

src/code.cloudfoundry.org/gorouter/route/hash_based.go

+			return nil
+		}
+
+		if !h.mustBeSticky {


this set of conditions could be refactored to be more readable.

The code is also almost 1:1 from leastconnection.go (Next()), maybe we can consolidate into one piece of code?

See commit: 6b7daaa

src/code.cloudfoundry.org/gorouter/route/roundrobin.go

src/code.cloudfoundry.org/gorouter/route/pool.go

peanball · 2026-01-22T12:40:56Z

LGTM. Once we have the performance tests, we should move to community review.

hoffmaen · 2026-01-28T13:20:26Z

Good news - we evaluated the performance of gorouter with hash-based routing support.

Our test setup constists of scaled out HAProxy (as a load balancer in front of gorouter), scaled out Diego Cells and a very simple and performant application with 30 instances. This allows us to performance test gorouter via external requests, where gorouter is the bottleneck.

We compared the performance of the latest stable routing-release, and the same release including changes from this Pull-Request. Using oha (inofficial successor of hey), we put various load patterns on the application and measured throughput, CPU usage of gorouter, and latency. In summary,

both versions performed absolutely the same when using round-robin
we achieved the same performance using hash-based routing and round-robin, when requests are distributed across all instances (achieved by running oha with multiple instances, each using a different hash_header value)
various mixed load scenarios (load on a route with round-robin parallel to load on a route with hash) showed no impact on performance

We also explored that requests are routed to next instances as expected, when the max. connection limit of one app instance was reached. The hash_balance factor is effective and allows distribution of requests to next instances before overloading it.

Please feel free to reach out for details on the results!

Changes were addressed. As this is still subject to change, I cannot give "final" approval yet.

b1tamara · 2026-02-05T08:20:45Z

During performance tests, we figured out that the Gorouter memory consumption is higher than expected if we keep the permutation table (table which is used to fill the final maglev lookup table) in memory. We want to investigate this finding further and optimize it.
Moving PR back to Draft.

hoffmaen mentioned this pull request Nov 11, 2025

[DRAFT] Implement Hash-Based Routing #518

Closed

1 task

cf-foundation-community-automation bot moved this to Inbox in Application Runtime Platform Working Group Nov 11, 2025

cf-foundation-community-automation bot added this to Application Runtime Platform Working Group Nov 11, 2025

b1tamara force-pushed the hash-based-routing-with-overload branch from 7c44bfe to 2c23efd Compare November 13, 2025 13:05

hoffmaen force-pushed the hash-based-routing-with-overload branch from 2c23efd to 91f5968 Compare November 13, 2025 13:47

b1tamara mentioned this pull request Nov 14, 2025

Implement Hash-Based Routing Support in Cloud Controller cloudfoundry/capi-release#589

Closed

maxmoehl reviewed Nov 27, 2025

View reviewed changes

src/code.cloudfoundry.org/gorouter/proxy/round_tripper/proxy_round_tripper.go Outdated Show resolved Hide resolved

src/code.cloudfoundry.org/gorouter/route/maglev.go Outdated Show resolved Hide resolved

src/code.cloudfoundry.org/gorouter/route/pool.go Show resolved Hide resolved

b1tamara force-pushed the hash-based-routing-with-overload branch from e9cbe0c to 7b456f8 Compare December 3, 2025 09:32

hoffmaen force-pushed the hash-based-routing-with-overload branch from 9f21f6b to 513c575 Compare December 9, 2025 08:24

peanball reviewed Dec 9, 2025

View reviewed changes

peanball requested changes Jan 16, 2026

View reviewed changes

a18e and others added 13 commits January 16, 2026 14:28

Add LICENSE information for maglev.go

4c6982f

Implement overflow traffic

5981bcb

* Minor improvements and refactoring

25d9395

Refactor pool.Endpoints to apply review feedback

bda2dbf

Fix bug related to all endpoints are overloaded

fbbfaef

Fix compiler errors

350038a

Update RouteSchema struct in route-emitter

95edd7b

Add json string property to hash_balance

f7910d7

Refactor route/pool Endpoints function

b8b2607

Refactor locallyOptimistic and pass request header to Endpoints

d23f285

WIP: Introduce routing properties

52f0f7c

Apply review feedback

560f935

hoffmaen force-pushed the hash-based-routing-with-overload branch from 5030bbc to 560f935 Compare January 16, 2026 14:29

peanball previously requested changes Jan 16, 2026

View reviewed changes

b1tamara added 3 commits January 19, 2026 10:48

Refactor hash-based.go

b6811af

refactor looking for endpoint for sticky session

6b7daaa

Refactor hash_based.go

c518e14

b1tamara requested a review from peanball January 21, 2026 07:22

peanball reviewed Jan 22, 2026

View reviewed changes

src/code.cloudfoundry.org/gorouter/route/roundrobin.go Show resolved Hide resolved

Add function for debug logs

b414a38

peanball reviewed Jan 22, 2026

View reviewed changes

src/code.cloudfoundry.org/gorouter/route/pool.go Show resolved Hide resolved

This comment was marked as outdated.

Sign in to view

b1tamara requested a review from peanball January 28, 2026 13:35

hoffmaen marked this pull request as ready for review January 28, 2026 14:53

hoffmaen requested a review from a team as a code owner January 28, 2026 14:53

a18e force-pushed the hash-based-routing-with-overload branch from 21cc8b7 to b414a38 Compare January 30, 2026 10:31

b1tamara marked this pull request as draft February 5, 2026 08:21

Conversation

hoffmaen commented Nov 11, 2025 • edited by b1tamara Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Backward Compatibility

Uh oh!

Uh oh!

Uh oh!

Uh oh!

peanball left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

peanball Dec 9, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

peanball Dec 9, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

peanball Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

peanball left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

peanball Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

peanball Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

b1tamara Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

peanball commented Jan 22, 2026

Uh oh!

hoffmaen commented Jan 28, 2026 • edited by peanball Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

b1tamara commented Feb 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

hoffmaen commented Nov 11, 2025 •

edited by b1tamara

Loading

hoffmaen commented Jan 28, 2026 •

edited by peanball

Loading