K
- The counter key type. This type must have good implementations of
Object.hashCode()
and Object.equals(Object)
.public abstract class CountingSummarizer<K> extends Object implements Summarizer
During collection and summarization this class will use the functions from converter()
and encoder()
. For each key/value the function from converter()
will be called
to create zero or more counter objects. A counter associated with each counter object will be
incremented, as long as there are not too many counters and the counter object is not too long.
When Summarizer.Collector.summarize(Summarizer.StatisticConsumer)
is called, the function
from encoder()
will be used to convert counter objects to strings. These strings will be
used to emit statistics. Overriding encoder()
is optional. One reason to override is if
the counter object contains binary or special data. For example, a function that base64 encodes
counter objects could be created.
If the counter key type is mutable, then consider overriding copier()
.
The function returned by converter()
will be called frequently and should be very
efficient. The function returned by encoder()
will be called less frequently and can be
more expensive. The reason these two functions exists is to avoid the conversion to string for
each key value, if that conversion is unnecessary.
Below is an example implementation that counts column visibilities. This example avoids
converting column visibility to string for each key/value. This example shows the source code for
VisibilitySummarizer
.
public class VisibilitySummarizer extends CountingSummarizer<ByteSequence> {
@Override
protected UnaryOperator<ByteSequence> copier() {
// ByteSequences are mutable, so override and provide a copy function
return ArrayByteSequence::new;
}
@Override
protected Converter<ByteSequence> converter() {
return (key, val, consumer) -> consumer.accept(key.getColumnVisibilityData());
}
}
CounterSummary
Modifier and Type | Class and Description |
---|---|
static interface |
CountingSummarizer.Converter<K>
A function that converts key values to zero or more counter objects.
|
Summarizer.Collector, Summarizer.Combiner, Summarizer.StatisticConsumer
Modifier and Type | Field and Description |
---|---|
static String |
COUNTER_STAT_PREFIX
This prefixes all counters when emitting statistics in
Summarizer.Collector.summarize(Summarizer.StatisticConsumer) . |
static String |
DELETES_IGNORED_STAT
This is the name of the statistic that tracks the total number of deleted keys seen.
|
static String |
EMITTED_STAT
This is the name of the statistic that tracks the total number of counter objects emitted by
the
CountingSummarizer.Converter . |
static String |
INGNORE_DELETES_DEFAULT |
static String |
INGNORE_DELETES_OPT
A configuration option to determine if delete keys should be counted.
|
static String |
MAX_CKL_DEFAULT |
static String |
MAX_COUNTER_DEFAULT |
static String |
MAX_COUNTER_LEN_OPT
A configuration option for specifying the maximum length of an individual counter key.
|
static String |
MAX_COUNTERS_OPT
A configuration option for specifying the maximum number of unique counters an instance of this
summarizer should track.
|
static String |
SEEN_STAT
This tracks the total number of key/values seen by the
Summarizer.Collector |
static String |
TOO_LONG_STAT
This is the name of the statistic that tracks how many counter objects were ignored because
they were too long.
|
static String |
TOO_MANY_STAT
This is the name of the statistic that tracks how many counters objects were ignored because
the number of unique counters was exceeded.
|
Constructor and Description |
---|
CountingSummarizer() |
Modifier and Type | Method and Description |
---|---|
Summarizer.Collector |
collector(SummarizerConfiguration sc)
Factory method that creates a
Summarizer.Collector based on configuration. |
Summarizer.Combiner |
combiner(SummarizerConfiguration sc)
Factory method that creates a
Summarizer.Combiner . |
protected abstract CountingSummarizer.Converter<K> |
converter() |
protected UnaryOperator<K> |
copier()
Override this if your key type is mutable and subject to change.
|
protected Function<K,String> |
encoder() |
public static final String MAX_COUNTERS_OPT
public static final String MAX_COUNTER_LEN_OPT
public static final String INGNORE_DELETES_OPT
CountingSummarizer.Converter
and the statistic
"deletesIgnored" will track the number of deleted ignored. This options defaults
to "true".public static final String COUNTER_STAT_PREFIX
Summarizer.Collector.summarize(Summarizer.StatisticConsumer)
.public static final String TOO_MANY_STAT
MAX_COUNTERS_OPT
.public static final String TOO_LONG_STAT
MAX_COUNTER_LEN_OPT
.public static final String EMITTED_STAT
CountingSummarizer.Converter
. This includes emitted Counter objects that were ignored.public static final String DELETES_IGNORED_STAT
public static final String SEEN_STAT
Summarizer.Collector
public static final String MAX_COUNTER_DEFAULT
public static final String MAX_CKL_DEFAULT
public static final String INGNORE_DELETES_DEFAULT
protected abstract CountingSummarizer.Converter<K> converter()
protected Function<K,String> encoder()
Object.toString()
on the counter object.protected UnaryOperator<K> copier()
UnaryOperator.identity()
function.public Summarizer.Collector collector(SummarizerConfiguration sc)
Summarizer
Summarizer.Collector
based on configuration. Each Summarizer.Collector
created by this method should be independent and have its own internal state. Accumulo uses a
Collector to generate summary statistics about a sequence of key values written to a file.collector
in interface Summarizer
public Summarizer.Combiner combiner(SummarizerConfiguration sc)
Summarizer
Summarizer.Combiner
. Accumulo will only use the created Combiner to
merge data from Summarizer.Collector
s created using the same SummarizerConfiguration
.combiner
in interface Summarizer
Copyright © 2011–2019 The Apache Software Foundation. All rights reserved.