Interface RateLimiter<T>

Type Parameters:

T - the amount type: Integer for token counts, Void for unit permits, a routing key for multi-bucket limiters

All Known Implementing Classes:

AbstractRateLimiter, CountingGate, DatabaseGate, ElasticWindowRateLimiter, EPORateLimiter, HttpConnectionGate, MemoryPressureGate, ModelGate, PubMedRateLimiter, STDIOEndpointRateLimiter, TokenBucketRateLimiter, UsptoOdpRateLimiter

public interface RateLimiter<T>

One admission account: a resource counter with a replenishment kind that a job's Demand names and that Admission debits.

An account never blocks and never parks anyone. The wait belongs to Admission, which evaluates a job's whole demand as one conjunctive guard and takes every entry in one critical section or none of them. The account answers three questions about amounts and performs two movements:

• fits(List, List): would mine fit right now with reservedAhead set aside. Pure with respect to admission state (clocked accounts may advance their own bookkeeping). Refusal is judged on mine alone: throw an UncorrectableRuntimeLLMException when mine can never fit under the current configuration or when the account is refusing everyone; when mine fits alone but not together with reservedAhead, the answer is false.
• tryTake(List, List): the debit. Take mine if mine and reservedAhead fit together, atomically under the account's own lock, so a cap or circuit change on a client thread cannot land between the check and the take.
• give(List): the credit, at close for RateLimiter.Replenishment.RELEASE accounts, at rollback and as compensation inside a failed grant for every account.
• earliestFit(List, List): the System.nanoTime() instant at which fits(List, List) would become true from the clock alone, or null when only an external event can change the answer.

mine are the asking job's amounts on this account; reservedAhead are the amounts the head of the admission queue holds reserved on it, empty for the head itself. Each account decides how amounts combine: tokens sum, unit permits count, a router routes each to its bucket, a predicate account ignores them. Amount lists may contain null for Void-typed accounts.

Since:

0.1 (2025-11-14)

Author:

Andrey Santrosyan

Nested Class Summary

Nested Classes

Modifier and Type	Interface	Description
static enum	RateLimiter.Replenishment	How a permit taken from this limiter comes back, which decides whether Admission may return it when the job finishes.

Method Summary

Modifier and Type	Method	Description
LimiterIdentity	accountFor(T amount)	The identity the given amount debits: this limiter for a plain account, the routed bucket for a router.
Long	earliestFit(List<T> mine, List<T> reservedAhead)	The System.nanoTime() instant at which fits(List, List) would become true for these lists from the clock alone; null when only an external event can change the answer - always for RateLimiter.Replenishment.RELEASE accounts, and for any account when mine plus reservedAhead exceeds its capacity.
boolean	fits(List<T> mine, List<T> reservedAhead)	Would mine fit right now with reservedAhead set aside.
void	give(List<T> amounts)	Credits previously taken amounts.
default void	observeHeld(IntSupplier held)	Installs a lock-free view of how many jobs admission is holding on this account right now: the waiters this account has refused and not yet released, which is this account's own line and nobody else's.
void	onCapacityChange(Runnable wake)	Installs the single wake this account runs after any self-change that can only increase capacity: a throttle relaxation, a cap raised from headers, a circuit closing, a pushed fleet allotment.
void	onRateLimitError(UpstreamFailure failure)	Notifies the rate limiter that an upstream request failed.
default void	onRateLimitError(UpstreamFailure failure, T input)	Per-input variant of onRateLimitError(UpstreamFailure).
void	onSuccess()	Notifies the rate limiter of a successful operation.
default void	onSuccess(T input)	Notifies the rate limiter of a successful operation, scoped to the specific input that was acquired.
RateLimiter.Replenishment	replenishment()	Which replenishment model this limiter follows.
default boolean	requiresHttpConnection()	Whether this rate limiter's operations require HTTP connections.
boolean	tryTake(List<T> mine, List<T> reservedAhead)	Takes mine if mine and reservedAhead fit together; the check and the debit are one atomic step under the account's own lock.

Method Details

replenishment

RateLimiter.Replenishment replenishment()

Which replenishment model this limiter follows. Deliberately has no default: the answer is a property of the limiter's nature, and guessing it wrong is silent in both directions - a leaked permit one way, a disabled rate limit the other.

fits

boolean fits(List<T> mine,
 List<T> reservedAhead)

Would mine fit right now with reservedAhead set aside.

Throws:

UncorrectableRuntimeLLMException - when mine can never fit under the current configuration, or when the account is refusing everyone

tryTake

boolean tryTake(List<T> mine,
 List<T> reservedAhead)

Takes mine if mine and reservedAhead fit together; the check and the debit are one atomic step under the account's own lock.

Returns:

true when taken, false when the amounts do not fit at this instant

Throws:

UncorrectableRuntimeLLMException - under the same conditions as fits(List, List)

give

void give(List<T> amounts)

Credits previously taken amounts. For a RateLimiter.Replenishment.TIME account this is a refund of a debit whose request never went upstream.

earliestFit

Long earliestFit(List<T> mine,
 List<T> reservedAhead)

The System.nanoTime() instant at which fits(List, List) would become true for these lists from the clock alone; null when only an external event can change the answer - always for RateLimiter.Replenishment.RELEASE accounts, and for any account when mine plus reservedAhead exceeds its capacity. A past instant means now.

onCapacityChange

void onCapacityChange(Runnable wake)

Installs the single wake this account runs after any self-change that can only increase capacity: a throttle relaxation, a cap raised from headers, a circuit closing, a pushed fleet allotment. The wake is a bare unpark of the admission evaluator and never re-enters an account or the monitor, so it may be run from anywhere, under any lock. null clears the slot. Decreases never wake anyone.

observeHeld

default void observeHeld(IntSupplier held)

Installs a lock-free view of how many jobs admission is holding on this account right now: the waiters this account has refused and not yet released, which is this account's own line and nobody else's. An account that drains its line at a rate needs to know whether it has one: with nothing held here a lone arrival passes at once, with a line the releases are spaced, and a line held on some other account is not this account's to pace. Accounts whose answer does not depend on their line ignore this.

accountFor

LimiterIdentity accountFor(T amount)

The identity the given amount debits: this limiter for a plain account, the routed bucket for a router. Head reservation, waiter counts and events all key on it.

onSuccess

void onSuccess()

Notifies the rate limiter of a successful operation. Used by adaptive strategies to relax throttling.

onSuccess

default void onSuccess(T input)

Notifies the rate limiter of a successful operation, scoped to the specific input that was acquired. Default implementation delegates to onSuccess() for limiters that don't care about per-input bucketing. Multi-bucket limiters (e.g. one elastic window per upstream sub-service) override this to signal the correct bucket.

Parameters:

input - The same value that was passed as an amount in the demand

onRateLimitError

void onRateLimitError(UpstreamFailure failure)

Notifies the rate limiter that an upstream request failed. Used by adaptive strategies to tighten throttling, and - just as importantly - to record what the upstream actually said, so anything the limiter subsequently emits can name the cause.

Parameters:

failure - the upstream's status and response detail; never null

onRateLimitError

default void onRateLimitError(UpstreamFailure failure,
 T input)

Per-input variant of onRateLimitError(UpstreamFailure). Default implementation delegates for limiters that don't care about per-input bucketing. Multi-bucket limiters override this to tighten only the bucket that failed.

Parameters:

failure - the upstream's status and response detail; never null

input - The same value that was passed as an amount in the demand

requiresHttpConnection

default boolean requiresHttpConnection()

Whether this rate limiter's operations require HTTP connections. A demand that names such a limiter also names the shared HTTP connection gate.

Returns:

true if operations require HTTP, false otherwise