A Beam that writes all messages to a fixed set of Druid tasks.
Describes rollup (dimensions, aggregators, index granularity) desired for a Druid datasource.
Exceptions that are permanent in nature, and are useless to retry externally.
Exceptions that are permanent in nature, and are useless to retry externally. The assumption is that all other exceptions may be transient.
Exceptions that indicate transient indexing service failures.
Exceptions that indicate transient indexing service failures. Can be retried if desired.
Client for a single Druid task.
Builds Beams or Finagle services that send events to the Druid indexing service.
Builds Beams or Finagle services that send events to the Druid indexing service.
val curator = CuratorFrameworkFactory.newClient("localhost:2181", new BoundedExponentialBackoffRetry(100, 30000, 30)) curator.start() val dataSource = "foo" val dimensions = Seq("bar") val aggregators = Seq(new LongSumAggregatorFactory("baz", "baz")) val sender = DruidBeams .builder[Map[String, Any]](eventMap => new DateTime(eventMap("timestamp"))) .curator(curator) .discoveryPath("/test/discovery") .location(DruidLocation(new DruidEnvironment("druid:local:indexer", "druid:local:firehose:%s"), dataSource)) .rollup(DruidRollup(dimensions, aggregators, QueryGranularities.MINUTE)) .tuning(new ClusteredBeamTuning(Granularity.HOUR, 10.minutes, 1, 1)) .buildTranquilizer() val future = sender.send(Map("timestamp" -> "2010-01-02T03:04:05.678Z", "bar" -> "hey", "baz" -> 3)) println("result = %s" format Await.result(future))
Your event type (in this case, Map[String, Any]
) must be serializable via Jackson to JSON that Druid can
understand. If Jackson is not an appropriate choice, you can provide an ObjectWriter via .objectWriter(...)
.
Describes rollup (dimensions, aggregators, index granularity) desired for a Druid datasource. Java users should use the create methods on DruidRollup$, as those accepts Java collections rather than Scala ones.
See DruidDimensions.specific, DruidDimensions.schemaless, and DruidDimensions.schemalessWithExclusions for three common ways of creating druid dimensions objects.