executor

Type Members

1. abstract class AsyncBase[I, O, S, D, RC] extends Serializable with OperationContainer[I, O, S, D, RC]

   

2. class AtomicStateTransformer[T] extends AnyRef

   

3. class FinalFlatMap[Event, Key, Value, S <: InputState[_], D, RC] extends AsyncBase[Event, (Int, Map[Key, Value]), S, D, RC]

   

4. case class InputState[T](state: T) extends Product with Serializable

   

5. class IntermediateFlatMap[T, U, S, D, RC] extends AsyncBase[T, U, S, D, RC]

   

6. trait OperationContainer[Input, Output, State, WireFmt, RuntimeContext] extends AnyRef

   

7. class Summer[Key, Value, Event, S, D, RC] extends AsyncBase[(Int, Map[Key, Value]), Event, InputState[S], D, RC]

   

   The SummerBolt takes two related options: CacheSize and MaxWaitingFutures.

   The SummerBolt takes two related options: CacheSize and MaxWaitingFutures. CacheSize sets the number of key-value pairs that the SinkBolt will accept (and sum into an internal map) before committing out to the online store.

   To perform this commit, the SinkBolt iterates through the map of aggregated kv pairs and performs a "+" on the store for each pair, sequencing these "+" calls together using the Future monad. If the store has high latency, these calls might take a bit of time to complete.

   MaxWaitingFutures(count) handles this problem by realizing a future representing the "+" of kv-pair n only when kvpair n + 100 shows up in the bolt, effectively pushing back against latency bumps in the host.

   The allowed latency before a future is forced is equal to (MaxWaitingFutures * execute latency).