Packages

  • package root
    Definition Classes
    root
  • package com
    Definition Classes
    root
  • package criteo
    Definition Classes
    com
  • package cuttle

    Core cuttle concepts are defined here.

    Core cuttle concepts are defined here.

    Definition Classes
    criteo
  • package platforms

    Defines useful ExecutionPlatforms:

    Defines useful ExecutionPlatforms:

    • local to fork process locally in a managed way.
    • http to make HTTP calls with rate limiting.
    Definition Classes
    cuttle
  • package http

    Allow to make HTTP calls in a managed way with rate limiting.

    Allow to make HTTP calls in a managed way with rate limiting.

    It uses lolhttp as HTTP client.

    Definition Classes
    platforms
  • package local

    Allow to fork process locally in a managed way.

    Allow to fork process locally in a managed way.

    It provides the exec interpolator that you can use to fork shell scripts:

    exec"""hdfs dfs -cp /from /to"""()

    The script your provide will be forked into another process. Note that if you provide several commands separated by ;' only the first one will be forked and the other ones will be ignored.

    If you really need to run several commands, you can use one of the followed ways:

    1. Chain multiple execs in a for-comprehension. Because of platform mechanism it will require several platform allocations to run completely.

    2. Wrap it in one sh -c' _In this case we cannot guarantee that all process spawned inside this fork will be successfully killed when we kill the parent process._

    Definition Classes
    platforms
  • ExecutionPool
  • RateLimiter
  • WaitingExecutionQueue

class RateLimiter extends WaitingExecutionQueue

An rate limiter pool backed by a priority queue. It rate limits the executions and the priority queue is ordered by the Ordering defined on the SchedulingContext.

The implementation is based on the tokens bucket algorithm.

Linear Supertypes
WaitingExecutionQueue, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. RateLimiter
  2. WaitingExecutionQueue
  3. AnyRef
  4. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new RateLimiter(tokens: Int, refillRateInMs: Int)

    tokens

    Maximum (and initial) number of tokens.

    refillRateInMs

    A token is added to the bucket every refillRateInMs milliseconds.

Type Members

  1. case class DelayedResult [A](effect: () ⇒ Future[A], effectDebug: String, promise: Promise[A], submitted: Instant, started: Ref[Option[Instant]] = Ref(None)) extends Product with Serializable
    Definition Classes
    WaitingExecutionQueue

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. lazy val _running: Ref[Set[(Execution[_], DelayedResult[_])]]
    Definition Classes
    WaitingExecutionQueue
  5. lazy val _waiting: Ref[SortedSet[(Execution[_], DelayedResult[_])]]
    Definition Classes
    WaitingExecutionQueue
  6. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  7. def canRunNextCondition(implicit txn: InTxn): Boolean
    Definition Classes
    RateLimiter → WaitingExecutionQueue
  8. def clone(): AnyRef
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  9. def doRunNext()(implicit txn: InTxn): Unit
    Definition Classes
    RateLimiter → WaitingExecutionQueue
  10. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  11. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  12. def finalize(): Unit
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  13. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
  14. def hashCode(): Int
    Definition Classes
    AnyRef → Any
  15. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  16. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  17. final def notify(): Unit
    Definition Classes
    AnyRef
  18. final def notifyAll(): Unit
    Definition Classes
    AnyRef
  19. def routes(urlPrefix: String): PartialFunction[Request, Future[Response]]
    Definition Classes
    RateLimiter → WaitingExecutionQueue
  20. def run[A, S <: Scheduling](execution: Execution[S], debug: String)(f: () ⇒ Future[A]): Future[A]
    Definition Classes
    WaitingExecutionQueue
  21. def runNext(): Unit
    Definition Classes
    WaitingExecutionQueue
  22. def running: Set[Execution[_]]
    Definition Classes
    WaitingExecutionQueue
  23. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  24. implicit def taskEncoder[A]: Encoder[DelayedResult[A]]
    Definition Classes
    WaitingExecutionQueue
  25. def toString(): String
    Definition Classes
    AnyRef → Any
  26. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  27. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  28. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  29. def waiting: Set[Execution[_]]
    Definition Classes
    WaitingExecutionQueue

Inherited from WaitingExecutionQueue

Inherited from AnyRef

Inherited from Any

Ungrouped