abstract class RingBuffer[A] extends MutableQueueFieldsPadding[A] with Serializable
A lock-free array-based bounded queue. It is thread-safe and can be used in multiple-producer/multiple-consumer (MPMC) setting.
Main concepts
A simple array-based queue of size N uses an array buf
of size N as an
underlying storage. There are 2 pointers head
and tail
. The element is
enqueued into buf
at position tail % N
and dequeued from head % N
. Each
time an enqueue happens tail
is incremented, similarly when dequeue happens
head
is incremented.
Since pointers wrap around the array as they get incremented such data structure is also called a circular buffer or a ring buffer.
Because queue is bounded, enqueue and dequeue may fail, which is captured in
the semantics of offer
and poll
methods.
Using offer
as an example, the algorithm can be broken down roughly into
three steps:
- Find a place to insert an element.
- Reserve this place, put an element and make it visible to other threads (store and publish).
- If there was no place on step 1 return false, otherwise returns true.
Steps 1 and 2 are usually done in a loop to accommodate the possibility of failure due to race. Depending on the implementation of these steps the resulting queue will have different characteristics. For instance, the more sub-steps are between reserve and publish in step 2, the higher is the chance that one thread will delay other threads due to being descheduled.
Notes on the design
The queue uses a buf
array to store elements. It uses seq
array to store
longs which serve as:
- an indicator to producer/consumer threads whether the slot is right for enqueue/dequeue,
- an indicator whether the queue is empty/full,
- a mechanism to publish changes to
buf
via volatile write (can even be relaxed to ordered store).
See comments in offer
/poll
methods for more details on seq
.
The benefit of using seq
+ head
/tail
counters is that there are no
allocations during enqueue/dequeue and very little overhead. The downside is
it doubles (on 64bit) or triples (compressed OOPs) the amount of memory
needed for a queue.
Concurrent enqueues and concurrent dequeues are possible. However there is no helping, so threads can delay other threads, and thus the queue doesn't provide full set of lock-free guarantees. In practice it's usually not a problem, since benefits are simplicity, zero GC pressure and speed.
There are 2 implementations of a RingBuffer:
RingBufferArb
that supports queues with arbitrary capacity;RingBufferPow2
that supports queues with only power of 2 capacities.
The reason is head % N
and tail % N
are rather cheap when can be done as
a simple mask (N is pow 2), and pretty expensive when involve an idiv
instruction. The difference is especially pronounced in tight loops (see.
RoundtripBenchmark).
To ensure good performance reads/writes to head
and tail
fields need to
be independent, e.g. they shouldn't fall on the same (adjacent) cache-line.
We can make those counters regular volatile long fields and space them out,
but we still need a way to do CAS on them. The only way to do this except
Unsafe
is to use AtomicLongFieldUpdater
, which is exactly what we have
here.
- See also
zio.internal.MutableQueueFieldsPadding for more details on padding and object's memory layout. The design is heavily inspired by such libraries as https://github.com/LMAX-Exchange/disruptor and https://github.com/JCTools/JCTools which is based off D. Vyukov's design http://www.1024cores.net/home/lock-free-algorithms/queues/bounded-mpmc-queue Compared to JCTools this implementation doesn't rely on
sun.misc.Unsafe
, so it is arguably more portable, and should be easier to read. It's also very extensively commented, including reasoning, assumptions, and hacks.Alternative designs
There is an alternative design described in the paper A Portable Lock-Free Bounded Queue by Pirkelbauer et al. It provides full lock-free guarantees, which generally means that one out of many contending threads is guaranteed to make progress in a finite number of steps. The design thus is not susceptible to threads delaying other threads. However the helping scheme is rather involved and cannot be implemented without allocations (at least I couldn't come up with a way yet). This translates into worse performance on average, and better performance in some very specific situations.
- Alphabetic
- By Inheritance
- RingBuffer
- MutableQueueFieldsPadding
- TailPadding
- PreTailPadding
- HeadPadding
- ClassFieldsPadding
- Serializable
- MutableConcurrentQueue
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Concrete Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- final val capacity: Int
The maximum number of elements that a queue can hold.
The maximum number of elements that a queue can hold.
- Definition Classes
- RingBuffer → MutableConcurrentQueue
- Note
that unbounded queues can still implement this interface with
capacity = MAX_INT
.
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native()
- final def dequeuedCount(): Long
- returns
the number of elements that have ever been taken from the queue.
- Definition Classes
- RingBuffer → MutableConcurrentQueue
- Note
if you know how much time the queue is alive, you can calculate the rate at which elements are being dequeued.
- final def enqueuedCount(): Long
- returns
the number of elements that have ever been added to the queue.
- Definition Classes
- RingBuffer → MutableConcurrentQueue
- Note
that scala.Long is used here, since scala.Int will be overflowed really quickly for busy queues.
,if you know how much time the queue is alive, you can calculate the rate at which elements are being enqueued.
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable])
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- final def isEmpty(): Boolean
- Definition Classes
- RingBuffer → MutableConcurrentQueue
- final def isFull(): Boolean
- Definition Classes
- RingBuffer → MutableConcurrentQueue
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def offer(a: A): Boolean
A non-blocking enqueue.
A non-blocking enqueue.
- returns
whether the enqueue was successful or not.
- Definition Classes
- RingBuffer → MutableConcurrentQueue
- final def offerAll(as: Iterable[A]): Chunk[A]
A non-blocking enqueue of multiple elements.
A non-blocking enqueue of multiple elements.
- Definition Classes
- RingBuffer → MutableConcurrentQueue
- final def poll(default: A): A
A non-blocking dequeue.
A non-blocking dequeue.
- returns
either an element from the queue, or the
default
param.
- Definition Classes
- RingBuffer → MutableConcurrentQueue
- Note
that if there's no meaningful default for your type, you can always use
poll(null)
. Not the best, but reasonable price to pay for lower heap churn from not using scala.Option here.
- final def pollUpTo(n: Int): Chunk[A]
A non-blocking dequeue of multiple elements.
A non-blocking dequeue of multiple elements.
- Definition Classes
- RingBuffer → MutableConcurrentQueue
- final def size(): Int
- returns
the current number of elements inside the queue.
- Definition Classes
- RingBuffer → MutableConcurrentQueue
- Note
that this method can be non-atomic and return the approximate number in a concurrent setting.
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()