Build an In-Memory Token-Bucket Rate Limiter in Rust - Structs and Methods
1 min read
In our previous article, we learned about ownership and borrowing by building a log-line parser. This time, the main subject is structs: how to group related data, create values of a new type, and define methods that work on those values
The project that we will be building is a Rust library with a TokenBucket struct that represents one bucket in memory. You choose its capacity and an interval that restores one whole token.
But as usual, we will learn the concepts first and then focus on building the project
A Rust Struct Groups Named Fields into One Type
A struct defines a type whose fields have names and types. An instance is a value of that type, with a value supplied for every field (I’ll explain with an example, this statement might have confused you a little). We access a field with a dot followed by its name insource.
struct Book { title: String, pages: u32,}fn main() { let book = Book { title: String::from("The Hobbit"), pages: 310, }; println!("{} has {} pages", book.title, book.pages);}
So here the struct Book defines the type, Book { ... } creates one instance, this is very similar to other languages like C and Go where they have structs. The field names tell us what the two values represent. book.pages reads the page count without requiring us to remember a tuple position
Try removing pages: 310 from the struct literal and run cargo check. Rust tells you that the instance is missing a field. Put it back, then give pages a string instead of a u32. The compiler reports the type mismatch. A struct definition fixes the names and types that each instance must provide.
We already learned that Rust bindings are immutable by default. That rule also applies here: write let mut book if you later need to assign to book.pages. There is no separate mut marker on an individual field in this struct definition insource.
Field Init Shorthand Uses a Matching Variable Name
When a field and a variable have the same name, Rust lets us write that name once in a struct literal. Writing width there is shorthand for width: widthinsource.
So, the function receives variables named width and height and the struct has fields with those names, so Rectangle {width, height} fills both the fields from the variables. It creates the same vale as Rectangle { width: width, height: height }
you can try to change the parameter name from width to new_width without changing the struct literal, now if you’ll do cargo check it can’t find a variable named width. Write width: new_width and it’ll work again
Rust Struct Update Syntax Reuses Fields from Another Instance
The ..existing syntax fills unspecified fields of a new struct instance from an existing instance of the same type. It still follows the ownership rules we learned last time: a non-Copy field can move into the new value insource
Here, we are creating a struct Profile with name, city and age as fields. Then in main, we are creating an instance of it called original with some values for its fields, then we are creating another instance of this struct called updated and in this we are specifying city field value but for the rest of the fields we are writing ..original, this moves the string from original.name to updated.name but for original.age, it gets copied to updated.age as its a u8.
If you want to confirm that move operation actually happened, try adding println!("{}", original.name); after creating the updated and run cargo check, you’ll get an error. Rust rejects that line because the name moved. Struct update syntax saves repetition; it does not clone the reused fields.
Rust Tuple Structs Give Positional Data a Distinct Type
A tuple struct has a name and positional fields. Each tuple struct definition creates its own type, even when another tuple struct contains the same field types insource
struct Meters(u32);struct Seconds(u32);fn print_distance(distance: Meters) { println!("{} meters", distance.0);}fn main() { let distance = Meters(100); let time = Seconds(20); print_distance(distance); println!("{} seconds", time.0);}
We access a tuple struct field by position, so distance.0 is its first field. Meters(100) and Seconds(20) both contain a u32, but Meters and Seconds are different types. Replace print_distance(distance) with print_distance(time) and run cargo check to see the mismatch.
A named-field struct helps when the individual fields need descriptive names. A tuple struct helps when the type name carries the meaning and the fields’ positions are clear.
A new struct does not automatically have a display format. If we want to inspect its fields while developing, we can derive Debug and print it with the {:?} formatter insource
#[derive(Debug)]struct Movie { title: String, year: u16,}fn main() { let movie = Movie { title: String::from("Arrival"), year: 2016, }; println!("{movie:?}");}
If you’ll run this, you’ll get Movie { title: "Arrival", year: 2016 }
Remove #[derive(Debug)] and run cargo check. Rust tells us that Movie does not implement Debug. The derive attribute asks Rust to generate that implementation. Debug is useful for inspecting values during development; a polished format for readers is a separate choice.
A method is defined in an impl block and is called on an instance with dot syntax. Its first parameter is a form of self, which represents the instance receiving the call insource.
This must feel very similar to other languages that support struct or classes and methods
impl Rectangle groups this behavior with the Rectangle type. The area method reads the fields of the particular instance on which we call it. We write rectangle.area() rather than passing rectangle as an explicit argument. &self means the method borrows that instance, so the next line can still use rectangle
A Method Receiver in Rust Says How It Uses the Instance
The receiver can be &self to read, &mut self to change the instance, or self to take ownership of it. These are the borrowing and moving rules from the previous article applied to a method’s first parameter insource.
current only reads the counter. add changes it, so the binding in main must be mutable. into_value consumes the counter and returns its stored value. Try calling counter.current() after into_value(); the compiler rejects it because counter was moved into that method.
This example is about choosing a receiver for a method. The underlying read, mutable borrow, and move behavior is the same behavior we already studied with ordinary functions.
An Associated Function in Rust Does Not Need an Instance
A function inside an impl block that does not have a self parameter is an associated function rather than a method. We call it with Type::function(...). Constructors are often named new, but Rust does not treat that name specially insource.
struct Circle { radius: u32,}impl Circle { fn new(radius: u32) -> Self { Self { radius } }}fn main() { let circle = Circle::new(8); println!("Radius: {}", circle.radius);}
Inside impl Circle, Self means Circle. There is no circle instance yet when we call Circle::new(8), so the function constructs and returns one.
This can feel like class methods in some other languages
Rust Project - Build a Token-Bucket Rate Limiter Library
Let’s define what I mean by the title of this project before coding. A bucket has a capacity and a refill interval. It starts full and each accepted request uses one token. Every complete refill interval restores one token but the count never rises above capacity. If the bucket is empty, a request is rejected and the caller gets the remaining wait until the next token
For example, capacity 3 and a two-second refill interval mean that three requests arriving at the start can pass. A fourth request at that same instant must wait two seconds. After one second, it still has one second to wait. At two seconds, one token is available again
This is a deliberately small token bucket: it grants whole tokens at fixed intervals, has no networking or shared concurrent state, and does not sleep. The caller supplies the current Instant on each request. That lets us test time behaviour without waiting for a real clock.
Create The Project
Let’s start by creating or intialising our library package:
cargo new token-bucket-limiter --libcd token-bucket-limitercargo test
Cargo creates src/lib.rs for the library. We will later add src/main.rs as a small program that calls it. Cargo recognizes those locations by convention source: inThe Cargo Book
The package name contains hyphens, while Rust code refers to the library as token_bucket_limiter. We will use that name when the demo imports our types inThe Cargo Book
For now, open src/lib.rs and replace Cargo’s sample code
Give the configuration a struct and a constructor
Add this to src/lib.rs:
use std::time::Duration;#[derive(Debug)]pub struct RateLimitConfig { capacity: u32, refill_every: Duration,}impl RateLimitConfig { pub fn new(capacity: u32, refill_every: Duration) -> Result<Self, &'static str> { if capacity == 0 { return Err("capacity must be greater than zero"); } if refill_every.is_zero() { return Err("refill interval must be greater than zero"); } Ok(Self { capacity, refill_every, }) } pub fn capacity(&self) -> u32 { self.capacity } pub fn refill_every(&self) -> Duration { self.refill_every }}
RateLimitConfig groups the two settings. Its new function is the associated-function pattern we just studied: it creates the struct after checking the inputs. We already used Result in the guessing-game article, so here we use it to return either a valid configuration or an error message. The fields are private, which means callers must use the validating constructor rather than fill in arbitrary field values.
The two getter methods use &self. They let a caller read the configuration without gaining permission to change its fields. Notice the field init shorthand inside Ok(Self { capacity, refill_every }).
Run:
cargo check
At this point, the library compiles, even though it does not make any rate-limit decisions yet.
Model the bucket’s changing state
Change the import at the top of src/lib.rs to include Instant:
use std::time::{Duration, Instant};
Then add this below RateLimitConfig and its impl block:
config contains the fixed rules. available is the changing token count. last_refill records the instant from which we measure the next refill. The constructor starts the bucket full by setting available to the capacity.
We read config.capacity before moving config into the new bucket, following the ownership rule from article three. Instant lets us measure elapsed time rather than a calendar date, which is what this limiter needs.
Decision is a named result. allowed says what happened, remaining reports the tokens left, and retry_after contains a duration only when a request was rejected. Option is the standard type for a value that may be present or absent. We will study enums, including Option, properly in the next article; here we only need Some(duration) and None to report the result insource.
Add this method inside impl TokenBucket after new:
The receiver is &mut self because accepting a request changes available. In the first branch, we spend one token and return a decision with the new count. In the second branch, we reject the request. For the moment we return the full interval as the retry time; this is accurate when all requests arrive at the starting instant. The now parameter is not used yet, so cargo check may warn about it. We will use it in the next step.
Run cargo check. If you change &mut self to &self, Rust rejects self.available -= 1 because a shared borrow cannot mutate the bucket. Put &mut self back before continuing.
Try a burst of requests
Create src/main.rs:
use std::time::{Duration, Instant};use token_bucket_limiter::{RateLimitConfig, TokenBucket};fn main() { let start = Instant::now(); let config = RateLimitConfig::new(3, Duration::from_secs(2)).expect("valid rate limit configuration"); let mut bucket = TokenBucket::new(config, start); for request in 1..=4 { let decision = bucket.try_acquire(start); println!( "request {request}: allowed={}, remaining={}, retry_after={:?}", decision.allowed, decision.remaining, decision.retry_after ); }}
bucket is mutable because each call to try_acquire takes &mut self. The four calls deliberately use the same start instant, so no time has passed between them. Run:
The bucket enforces its initial burst, but it cannot refill yet. That is the next method we will add.
Refill according to elapsed time
A rate limiter needs to remember time as well as tokens. Add a private refill method inside impl TokenBucket:
fn refill(&mut self, now: Instant) { if self.available == self.config.capacity { if now > self.last_refill { self.last_refill = now; } return; } let elapsed = now.saturating_duration_since(self.last_refill); let intervals = elapsed.as_nanos() / self.config.refill_every.as_nanos(); if intervals == 0 { return; } let missing = self.config.capacity - self.available; if intervals >= u128::from(missing) { self.available = self.config.capacity; self.last_refill = now; } else { let added = intervals as u32; self.available += added; self.last_refill += self.config.refill_every * added; }}
There are several decisions in this method, so let’s go through them one at a time.
First, a full bucket cannot store extra tokens or extra refill credit. If it has stayed full until now, we reset last_refill to now. Otherwise, a request that spends a token after a long idle period could get that token back almost immediately from time accumulated while the bucket was already full. This reset is part of our bucket’s behavior, not a special Rust rule.
Next, saturating_duration_since measures how much time has passed since last_refill. If a caller supplies an earlier instant, it returns zero rather than a negative duration. In normal use, callers should supply successive readings from Instant::now(); the zero behavior keeps our simple library from inventing tokens if an earlier value is passed.
as_nanos() gives us each duration’s total nanoseconds as a u128. Dividing elapsed nanoseconds by interval nanoseconds counts complete refill intervals. A one-second wait does not yet produce a token when the interval is two seconds.
If enough intervals have passed to fill every missing token, we set available to capacity. We cannot exceed that capacity. Otherwise, we add only the complete intervals and move last_refill forward by exactly those intervals. The unfinished part of the time stays available for the next call. For example, after five seconds with a two-second interval, two tokens can be restored and one second remains toward another token.
The intervals as u32 conversion appears only in the branch where intervals is less than missing, and missing is a u32. That condition makes the conversion fit. Run cargo check after adding the method.
Make each request refill before deciding
Replace the earlier try_acquire method with this version:
Now now has a job. The method refills before it asks whether a token can be spent. When the bucket is empty, last_refill points to the start of the current unfinished interval. Subtracting the elapsed part from refill_every gives the remaining wait. Because refill has already credited every complete interval, the elapsed part in this rejection branch is shorter than one interval.
Notice that the caller cannot alter available directly. The &mut self method is the one place where spending a token and reporting the result happen together. Our struct keeps its own rule: available stays between zero and capacity.
Check a refill without sleeping
Replace src/main.rs with this version:
use std::time::{Duration, Instant};use token_bucket_limiter::{RateLimitConfig, TokenBucket};fn main() { let start = Instant::now(); let config = RateLimitConfig::new(3, Duration::from_secs(2)).expect("valid rate limit configuration"); let mut bucket = TokenBucket::new(config, start); for request in 1..=4 { let decision = bucket.try_acquire(start); println!( "request {request}: allowed={}, remaining={}, retry_after={:?}", decision.allowed, decision.remaining, decision.retry_after ); } let halfway = bucket.try_acquire(start + Duration::from_secs(1)); println!( "after 1s: allowed={}, retry_after={:?}", halfway.allowed, halfway.retry_after ); let refilled = bucket.try_acquire(start + Duration::from_secs(2)); println!( "after 2s: allowed={}, remaining={}", refilled.allowed, refilled.remaining );}
Run cargo run. The program supplies instants representing one and two seconds after start; it does not wait in real time. That is why this check is immediate and repeatable.
The first three requests spend the starting tokens. The fourth is rejected. One second later there is still no complete refill interval; two seconds later one token has returned and the request spends it.
Test the boundaries
We want to check more than the happy path. Add the #[cfg(test)] module shown in the complete src/lib.rs below, then run:
cargo test
The tests check that invalid settings are rejected, a burst stops at capacity, partial time is preserved, a long idle period cannot overfill the bucket, time spent while full is discarded, and an earlier timestamp never refills it. These cases matter because each one exercises a state change in the struct. The tests supply Instant values directly, so they finish without sleeping.
In particular, predict this case before reading its assertion: a capacity-one bucket starts full; we wait one second, spend its token, then ask again one second later. With a two-second refill interval, should that second request pass? In our design, no. The first second passed while the bucket was full, so the refill countdown starts when the token is spent.
Complete program
Here is the finished src/lib.rs. It contains the configuration, decision, bucket, and boundary tests in one file:
use std::time::{Duration, Instant};#[derive(Debug)]pub struct RateLimitConfig { capacity: u32, refill_every: Duration,}impl RateLimitConfig { pub fn new(capacity: u32, refill_every: Duration) -> Result<Self, &'static str> { if capacity == 0 { return Err("capacity must be greater than zero"); } if refill_every.is_zero() { return Err("refill interval must be greater than zero"); } Ok(Self { capacity, refill_every, }) } pub fn capacity(&self) -> u32 { self.capacity } pub fn refill_every(&self) -> Duration { self.refill_every }}#[derive(Debug)]pub struct Decision { pub allowed: bool, pub remaining: u32, pub retry_after: Option<Duration>,}#[derive(Debug)]pub struct TokenBucket { config: RateLimitConfig, available: u32, last_refill: Instant,}impl TokenBucket { pub fn new(config: RateLimitConfig, now: Instant) -> Self { let available = config.capacity; Self { config, available, last_refill: now, } } pub fn try_acquire(&mut self, now: Instant) -> Decision { self.refill(now); if self.available > 0 { self.available -= 1; Decision { allowed: true, remaining: self.available, retry_after: None, } } else { let elapsed = now.saturating_duration_since(self.last_refill); Decision { allowed: false, remaining: 0, retry_after: Some(self.config.refill_every - elapsed), } } } fn refill(&mut self, now: Instant) { if self.available == self.config.capacity { if now > self.last_refill { self.last_refill = now; } return; } let elapsed = now.saturating_duration_since(self.last_refill); let intervals = elapsed.as_nanos() / self.config.refill_every.as_nanos(); if intervals == 0 { return; } let missing = self.config.capacity - self.available; if intervals >= u128::from(missing) { self.available = self.config.capacity; self.last_refill = now; } else { let added = intervals as u32; self.available += added; self.last_refill += self.config.refill_every * added; } }}#[cfg(test)]mod tests { use super::*; #[test] fn rejects_invalid_configuration() { assert!(RateLimitConfig::new(0, Duration::from_secs(1)).is_err()); assert!(RateLimitConfig::new(2, Duration::ZERO).is_err()); } #[test] fn allows_only_the_initial_burst() { let start = Instant::now(); let config = RateLimitConfig::new(3, Duration::from_secs(2)).unwrap(); let mut bucket = TokenBucket::new(config, start); assert_eq!(bucket.try_acquire(start).remaining, 2); assert_eq!(bucket.try_acquire(start).remaining, 1); assert_eq!(bucket.try_acquire(start).remaining, 0); let rejected = bucket.try_acquire(start); assert!(!rejected.allowed); assert_eq!(rejected.retry_after, Some(Duration::from_secs(2))); } #[test] fn keeps_partial_time_between_requests() { let start = Instant::now(); let config = RateLimitConfig::new(1, Duration::from_secs(2)).unwrap(); let mut bucket = TokenBucket::new(config, start); assert!(bucket.try_acquire(start).allowed); let after_one_second = bucket.try_acquire(start + Duration::from_secs(1)); assert!(!after_one_second.allowed); assert_eq!(after_one_second.retry_after, Some(Duration::from_secs(1))); assert!(bucket.try_acquire(start + Duration::from_secs(2)).allowed); } #[test] fn long_idle_period_never_exceeds_capacity() { let start = Instant::now(); let config = RateLimitConfig::new(2, Duration::from_secs(1)).unwrap(); let mut bucket = TokenBucket::new(config, start); bucket.try_acquire(start); bucket.try_acquire(start); let later = start + Duration::from_secs(20); assert!(bucket.try_acquire(later).allowed); assert!(bucket.try_acquire(later).allowed); assert!(!bucket.try_acquire(later).allowed); } #[test] fn time_spent_full_does_not_count_toward_a_new_token() { let start = Instant::now(); let config = RateLimitConfig::new(1, Duration::from_secs(2)).unwrap(); let mut bucket = TokenBucket::new(config, start); assert!(bucket.try_acquire(start + Duration::from_secs(1)).allowed); let rejected = bucket.try_acquire(start + Duration::from_secs(2)); assert!(!rejected.allowed); assert_eq!(rejected.retry_after, Some(Duration::from_secs(1))); assert!(bucket.try_acquire(start + Duration::from_secs(3)).allowed); } #[test] fn an_earlier_timestamp_does_not_refill() { let earlier = Instant::now(); let start = earlier + Duration::from_secs(1); let config = RateLimitConfig::new(1, Duration::from_secs(2)).unwrap(); let mut bucket = TokenBucket::new(config, start); assert!(bucket.try_acquire(start).allowed); let rejected = bucket.try_acquire(earlier); assert!(!rejected.allowed); assert_eq!(rejected.retry_after, Some(Duration::from_secs(2))); }}
And here is the complete src/main.rs client:
use std::time::{Duration, Instant};use token_bucket_limiter::{RateLimitConfig, TokenBucket};fn main() { let start = Instant::now(); let config = RateLimitConfig::new(3, Duration::from_secs(2)).expect("valid rate limit configuration"); let mut bucket = TokenBucket::new(config, start); for request in 1..=4 { let decision = bucket.try_acquire(start); println!( "request {request}: allowed={}, remaining={}, retry_after={:?}", decision.allowed, decision.remaining, decision.retry_after ); } let halfway = bucket.try_acquire(start + Duration::from_secs(1)); println!( "after 1s: allowed={}, retry_after={:?}", halfway.allowed, halfway.retry_after ); let refilled = bucket.try_acquire(start + Duration::from_secs(2)); println!( "after 2s: allowed={}, remaining={}", refilled.allowed, refilled.remaining );}
Run the finished package with:
cargo testcargo run
I hope you liked it, in the next one we will learn about Rust Enums and Pattern Matching in Rust by building a process supervisor state machine. See you in the next one, till then have a great life!