Class RateLimiterRegistry

java.lang.Object

ai.redouble.nucleo.harness.admission.RateLimiterRegistry

public class RateLimiterRegistry
extends Object

Lookup facade for per-model accounts: the TokenBucketRateLimiter of an entry bounded by a quota window, the ModelGate of an entry bounded by a concurrency. Shared across all clients of a given model so global limits are honored even when many jobs use the same model concurrently. Delegates to RateLimiterFactory for the actual creation and lifecycle.

Since:

0.1 (2025-08-12)

Author:

Andrey Santrosyan

Method Summary

Modifier and Type	Method	Description
void	clear()	Clears all rate limiters (useful for testing).
ModelGate	gate(ModelSpec model)	Gets or creates the gate for a model whose entry declares max_concurrent.
static RateLimiterRegistry	getInstance()	Gets the singleton instance.
TokenBucketRateLimiter	getRateLimiter(ModelSpec model)	Gets or creates a rate limiter for the given model.
TokenBucketStatus	getStatus(ModelSpec model)	Gets the current status of a model's rate limiter.
void	updateLimits(ModelSpec model, int requestsPerMinute, int tokensPerMinute)	Updates rate limits based on actual API response headers.

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

getInstance

public static RateLimiterRegistry getInstance()

Gets the singleton instance.

getRateLimiter

public TokenBucketRateLimiter getRateLimiter(ModelSpec model)

Gets or creates a rate limiter for the given model.

Parameters:

model - the model

Returns:

the rate limiter for this model

gate

public ModelGate gate(ModelSpec model)

Gets or creates the gate for a model whose entry declares max_concurrent.

Parameters:

model - the model

Returns:

the gate for this model

updateLimits

public void updateLimits(ModelSpec model,
 int requestsPerMinute,
 int tokensPerMinute)

Updates rate limits based on actual API response headers. This allows dynamic adjustment based on real limits.

Parameters:

model - the model

requestsPerMinute - new RPM limit

tokensPerMinute - new TPM limit

getStatus

public TokenBucketStatus getStatus(ModelSpec model)

Gets the current status of a model's rate limiter.

clear

public void clear()

Clears all rate limiters (useful for testing).