tiennm99/java-design-patterns
1
0 Fork 0
mirror of https://github.com/tiennm99/java-design-patterns.git synced 2026-10-13 14:16:02 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: update throttling

Browse Source
This commit is contained in:
Ilkka Seppälä committed 2024-05-22 15:17:05 +03:00
1 parent 16fbd26a42
commit 628ec1548c
3 files changed
+82 -62
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

throttling/README.md
+80 -57
View File
@@ -1,24 +1,28 @@
--- ---
title: Throttling title: Throttling
category: Behavioral category: Resource management
language: en language: en
tag: tag:
- Performance - API design
- Cloud distributed - Fault tolerance
- Performance
- Resilience
- Scalability
--- ---
## Also known as
* Rate Limiting
## Intent ## Intent
Ensure that a given client is not able to access service resources more than the assigned limit. Throttling limits the number of requests a system can process within a given time frame to prevent overload and ensure stability.
## Explanation ## Explanation
Real-world example Real-world example
> A young human and an old dwarf walk into a bar. They start ordering beers from the bartender. > Imagine a popular amusement park that limits the number of visitors who can enter per hour to prevent overcrowding. This ensures that all visitors can enjoy the park without long wait times and maintain a pleasant experience. Similarly, the Throttling design pattern in software controls the rate of requests to a system, preventing it from being overwhelmed and ensuring consistent performance for all users.
> The bartender immediately sees that the young human shouldn't consume too many drinks too fast
> and refuses to serve if enough time has not passed. For the old dwarf, the serving rate can
> be higher.
In plain words In plain words
@@ -26,21 +30,20 @@ In plain words
[Microsoft documentation](https://docs.microsoft.com/en-us/azure/architecture/patterns/throttling) says [Microsoft documentation](https://docs.microsoft.com/en-us/azure/architecture/patterns/throttling) says
> Control the consumption of resources used by an instance of an application, an individual tenant, > Control the consumption of resources used by an instance of an application, an individual tenant, or an entire service. This can allow the system to continue to function and meet service level agreements, even when an increase in demand places an extreme load on resources.
> or an entire service. This can allow the system to continue to function and meet service level
> agreements, even when an increase in demand places an extreme load on resources.
**Programmatic Example** **Programmatic Example**
`BarCustomer` class presents the clients of the `Bartender` API. `CallsCount` tracks the number of In this example a young human and an old dwarf walk into a bar. They start ordering beers from the bartender. The bartender immediately sees that the young human shouldn't consume too many drinks too fast and refuses to serve if enough time has not passed. For the old dwarf, the serving rate can be higher.
calls per `BarCustomer`.
`BarCustomer` class presents the clients of the `Bartender` API. `CallsCount` tracks the number of calls per `BarCustomer`.
```java ```java
@Getter
public class BarCustomer { public class BarCustomer {
@Getter
private final String name; private final String name;
@Getter
private final int allowedCallsPerSecond; private final int allowedCallsPerSecond;
public BarCustomer(String name, int allowedCallsPerSecond, CallsCount callsCount) { public BarCustomer(String name, int allowedCallsPerSecond, CallsCount callsCount) {
@@ -55,60 +58,57 @@ public class BarCustomer {
@Slf4j @Slf4j
public final class CallsCount { public final class CallsCount {
private final Map<String, AtomicLong> tenantCallsCount = new ConcurrentHashMap<>(); private final Map<String, AtomicLong> tenantCallsCount = new ConcurrentHashMap<>();
public void addTenant(String tenantName) { public void addTenant(String tenantName) {
tenantCallsCount.putIfAbsent(tenantName, new AtomicLong(0)); tenantCallsCount.putIfAbsent(tenantName, new AtomicLong(0));
} }
public void incrementCount(String tenantName) { public void incrementCount(String tenantName) {
tenantCallsCount.get(tenantName).incrementAndGet(); tenantCallsCount.get(tenantName).incrementAndGet();
} }
public long getCount(String tenantName) { public long getCount(String tenantName) {
return tenantCallsCount.get(tenantName).get(); return tenantCallsCount.get(tenantName).get();
} }
public void reset() { public void reset() {
tenantCallsCount.replaceAll((k, v) -> new AtomicLong(0)); tenantCallsCount.replaceAll((k, v) -> new AtomicLong(0));
LOGGER.info("reset counters"); LOGGER.info("reset counters");
} }
} }
``` ```
Next, the service that the tenants are calling is introduced. To track the call count, a throttler Next, the service that the tenants are calling is introduced. To track the call count, a throttler timer is used.
timer is used.
```java ```java
public interface Throttler { public interface Throttler {
void start();
void start();
} }
public class ThrottleTimerImpl implements Throttler { public class ThrottleTimerImpl implements Throttler {
private final int throttlePeriod; private final int throttlePeriod;
private final CallsCount callsCount; private final CallsCount callsCount;
public ThrottleTimerImpl(int throttlePeriod, CallsCount callsCount) { public ThrottleTimerImpl(int throttlePeriod, CallsCount callsCount) {
this.throttlePeriod = throttlePeriod; this.throttlePeriod = throttlePeriod;
this.callsCount = callsCount; this.callsCount = callsCount;
} }
@Override @Override
public void start() { public void start() {
new Timer(true).schedule(new TimerTask() { new Timer(true).schedule(new TimerTask() {
@Override @Override
public void run() { public void run() {
callsCount.reset(); callsCount.reset();
} }
}, 0, throttlePeriod); }, 0, throttlePeriod);
} }
} }
``` ```
`Bartender` offers the `orderDrink` service to the `BarCustomer`s. The customers probably don't `Bartender` offers the `orderDrink` service to the `BarCustomer`s. The customers probably don't know that the beer serving rate is limited by their appearances.
know that the beer serving rate is limited by their appearances.
```java ```java
class Bartender { class Bartender {
@@ -129,7 +129,7 @@ class Bartender {
return -1; return -1;
} }
callsCount.incrementCount(tenantName); callsCount.incrementCount(tenantName);
LOGGER.debug("Serving beer to {} : [{} consumed] ", barCustomer.getName(), count+1); LOGGER.debug("Serving beer to {} : [{} consumed] ", barCustomer.getName(), count + 1);
return getRandomCustomerId(); return getRandomCustomerId();
} }
@@ -139,8 +139,7 @@ class Bartender {
} }
``` ```
Now it is possible to see the full example in action. `BarCustomer` young human is rate-limited to 2 Now it is possible to see the full example in action. `BarCustomer` young human is rate-limited to 2 calls per second and the old dwarf to 4.
calls per second and the old dwarf to 4.
```java ```java
public static void main(String[] args) { public static void main(String[] args) {
@@ -205,14 +204,38 @@ An excerpt from the example's console output:
## Class diagram ## Class diagram
![alt text](./etc/throttling_urm.png "Throttling pattern class diagram") ![Throttling](./etc/throttling_urm.png "Throttling pattern class diagram")
## Applicability ## Applicability
The Throttling pattern should be used: * You need to protect resources from being overwhelmed by too many requests.
* You want to ensure fair usage of a service among multiple users.
* You need to maintain the quality of service under high load conditions.
* When service access needs to be restricted not to have high impact on the performance of the service. ## Known Uses
* When multiple clients are consuming the same service resources and restriction has to be made according to the usage per client.
* APIs of major cloud providers like AWS, Google Cloud, and Azure use throttling to manage resource usage.
* Web services to prevent denial-of-service (DoS) attacks by limiting the number of requests from a single IP address.
* Online platforms like social media sites and e-commerce websites to ensure even distribution of server load.
## Consequences
Benefits:
* Prevents resource exhaustion, ensuring system stability.
* Helps in maintaining consistent performance and quality of service.
* Improves fault tolerance by avoiding system crashes under high load.
Trade-offs:
* May cause increased latency or delay in request processing.
* Requires careful tuning to balance between resource protection and user experience.
* Could lead to denial of service to legitimate users if not configured correctly.
## Related Patterns
* [Circuit Breaker](https://java-design-patterns.com/patterns/circuit-breaker/): Works in tandem with throttling to prevent repeated attempts to access an overloaded service.
* Bulkhead: Isolates different parts of the system to limit the impact of throttling on other components.
## Credits ## Credits
throttling/src/main/java/com/iluwatar/throttling/BarCustomer.java
+1 -2
View File
@@ -30,11 +30,10 @@ import lombok.Getter;
/** /**
* BarCustomer is a tenant with a name and a number of allowed calls per second. * BarCustomer is a tenant with a name and a number of allowed calls per second.
*/ */
@Getter
public class BarCustomer { public class BarCustomer {
@Getter
private final String name; private final String name;
@Getter
private final int allowedCallsPerSecond; private final int allowedCallsPerSecond;
/** /**
throttling/src/test/java/com/iluwatar/throttling/BarCustomerTest.java
+1 -3
View File
@@ -36,8 +36,6 @@ class BarCustomerTest {
@Test @Test
void constructorTest() { void constructorTest() {
assertThrows(InvalidParameterException.class, () -> { assertThrows(InvalidParameterException.class, () -> new BarCustomer("sirBrave", -1, new CallsCount()));
new BarCustomer("sirBrave", -1, new CallsCount());
});
} }
} }