A Beam that writes all events to a fixed set of Druid tasks.
Exceptions that are permanent in nature, and are useless to retry externally.
Exceptions that indicate transient indexing service failures.
Exceptions that indicate transient indexing service failures. Can be retried if desired.
Client for a single Druid task.
Builds Beams or Finagle services that send events to the Druid indexing service.
Builds Beams or Finagle services that send events to the Druid indexing service.
val curator = CuratorFrameworkFactory.newClient("localhost:2181", new BoundedExponentialBackoffRetry(100, 30000, 30)) curator.start() val dataSource = "foo" val dimensions = Seq("bar") val aggregators = Seq(new LongSumAggregatorFactory("baz", "baz")) val service = DruidBeams .builder[Map[String, Any]](eventMap => new DateTime(eventMap("timestamp"))) .curator(curator) .discoveryPath("/test/discovery") .location(DruidLocation(new DruidEnvironment("druid:local:indexer", "druid:local:firehose:%s"), dataSource)) .rollup(DruidRollup(dimensions, aggregators, QueryGranularity.MINUTE)) .tuning(new ClusteredBeamTuning(Granularity.HOUR, 10.minutes, 1, 1)) .buildService() val future = service(Seq(Map("timestamp" -> "2010-01-02T03:04:05.678Z", "bar" -> "hey", "baz" -> 3))) println("result = %s" format Await.result(future))
Your event type (in this case,
Map[String, Any]
must be serializable via Jackson to JSON that Druid can understand. If Jackson is not an appropriate choice, you can provide an ObjectWriter via
.objectWriter(...)
.
.objectWriter(...) }}}
Map[String, Any] }}} understand. If Jackson is not an appropriate choice, you can provide an ObjectWriter via
.objectWriter(...)
.
.objectWriter(...) }}}
Exceptions that are permanent in nature, and are useless to retry externally. The assumption is that all other exceptions may be transient.