Package alluxio.metrics

Class MetricsSystem

  • @ThreadSafe
public final class MetricsSystem
extends java.lang.Object
    A MetricsSystem is created by a specific instance(master, worker). It polls the metrics sources periodically and pass the data to the sinks. The syntax of the metrics configuration file is: sink.[name].[options]=[value]

    • Field Summary

      Fields 
      Modifier and Type Field Description
      static java.lang.String CLUSTER  
      static java.lang.management.BufferPoolMXBean DIRECT_BUFFER_POOL  
      static com.codahale.metrics.MetricRegistry METRIC_REGISTRY  
      static java.lang.String SINK_REGEX  

    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method Description
      static java.util.Map<java.lang.String,​alluxio.grpc.MetricValue> allMetrics()  
      static void checkMinimalPollingPeriod​(java.util.concurrent.TimeUnit pollUnit, int pollPeriod)
      Checks if the poll period is smaller that the minimal poll period which is 1 second.
      static void clearAllMetrics()
      Resets the metric registry and removes all the metrics.
      static com.codahale.metrics.Counter counter​(java.lang.String name)
      Get or add counter with the given name.
      static com.codahale.metrics.Counter counterWithTags​(java.lang.String name, boolean shouldReport, java.lang.String... tags)
      Get or add counter with the given name with tags.
      static java.lang.String escape​(AlluxioURI uri)
      Escapes a URI, replacing "/" with "%2F".
      static InstrumentedExecutorService executorService​(java.util.concurrent.ExecutorService delegate, java.lang.String name)
      Add or replace the instrumented executor service metrics for the given name and executor service.
      static long getDirectMemUsed()  
      static java.lang.String getJobWorkerMetricName​(java.lang.String name)
      Builds metric registry name for job worker instance.
      static java.lang.String getMasterMetricName​(java.lang.String name)
      Builds metric registry names for master instance.
      static java.util.Map<java.lang.String,​java.util.Set<Metric>> getMasterMetrics​(java.util.Set<java.lang.String> metricNames)
      Gets all the master metrics belongs to the given metric names.
      static java.lang.String getMetricName​(java.lang.String name)
      Converts a simple string to a qualified metric name based on the process type.
      static Metric getMetricValue​(java.lang.String fullName)
      Gets metric with the given full metric name.
      static int getNumSinks()  
      static java.lang.String getPluginMetricName​(java.lang.String name)
      Builds metric registry name for plugin instance.
      static java.lang.String getResourcePoolMetricName​(java.lang.Object obj)
      Get metrics name based on class.
      static void initShouldReportMetrics​(MetricsSystem.InstanceType instanceType)
      Initialize the SHOULD_REPORT_METRICS.
      static boolean isStarted()  
      static com.codahale.metrics.Meter meter​(java.lang.String name)
      Get or add meter with the given name.
      static com.codahale.metrics.Meter meterWithTags​(java.lang.String name, boolean shouldReport, java.lang.String... tags)
      Get or add meter with the given name.
      static void registerAggregatedCachedGaugeIfAbsent​(java.lang.String name, java.util.Set<MetricKey> metrics, long timeout, java.util.concurrent.TimeUnit timeUnit)
      Created a gauge that aggregates the value of existing gauges.
      static <T> void registerCachedGaugeIfAbsent​(java.lang.String name, com.codahale.metrics.Gauge<T> metric)
      Registers a cached gauge if it has not been registered.
      static <T> void registerCachedGaugeIfAbsent​(java.lang.String name, com.codahale.metrics.Gauge<T> metric, long timeout, java.util.concurrent.TimeUnit unit)
      Registers a cached gauge if it has not been registered.
      static <T> void registerGaugeIfAbsent​(java.lang.String name, com.codahale.metrics.Gauge<T> metric)
      Registers a gauge if it has not been registered.
      static boolean removeMetrics​(java.lang.String name)
      Removes the metric with the given name.
      static java.util.List<alluxio.grpc.Metric> reportClientMetrics()  
      static java.util.List<alluxio.grpc.Metric> reportWorkerMetrics()  
      static void resetAllMetrics()
      Resets all the metrics in the MetricsSystem.
      static void resetCountersAndGauges()
      Resets all counters to 0 and unregisters gauges for testing.
      static void startSinks​(java.lang.String metricsConfFile)
      Starts sinks specified in the configuration.
      static void startSinksFromConfig​(MetricsConfig config)
      Starts sinks from a given metrics configuration.
      static void stopSinks()
      Stops all the sinks.
      static java.lang.String stripInstanceAndHost​(java.lang.String metricsName)
      Removes the instance and host from the given metric name, returning the result.
      static com.codahale.metrics.Timer timer​(java.lang.String name)
      Get or add timer with the given name.
      static java.lang.String unescape​(java.lang.String uri)
      Unescapes a URI, reverts it to before the escape, to display it correctly.
      static com.codahale.metrics.Timer uniformTimer​(java.lang.String name)
      Same with timer(java.lang.String) but with UnirformReservoir for sampling.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Field Detail

      • DIRECT_BUFFER_POOL

        public static final java.lang.management.BufferPoolMXBean DIRECT_BUFFER_POOL

      • METRIC_REGISTRY

        public static final com.codahale.metrics.MetricRegistry METRIC_REGISTRY

    • Method Detail

      • getDirectMemUsed

        public static long getDirectMemUsed()
        Returns:
        the used direct memory

      • startSinks

        public static void startSinks​(java.lang.String metricsConfFile)
        Starts sinks specified in the configuration. This is an no-op if the sinks have already been started. Note: This has to be called after Alluxio configuration is initialized.
        Parameters:
        metricsConfFile - the location of the metrics configuration file

      • startSinksFromConfig

        public static void startSinksFromConfig​(MetricsConfig config)
        Starts sinks from a given metrics configuration. This is made public for unit test.
        Parameters:
        config - the metrics config

      • stopSinks

        public static void stopSinks()
        Stops all the sinks.

      • isStarted

        public static boolean isStarted()
        Returns:
        true if the metric system is started, false otherwise

      • getNumSinks

        public static int getNumSinks()
        Returns:
        the number of sinks started

      • getResourcePoolMetricName

        public static java.lang.String getResourcePoolMetricName​(java.lang.Object obj)
        Get metrics name based on class.
        Parameters:
        obj - object for the resource pool
        Returns:
        metrics string

      • getMetricName

        public static java.lang.String getMetricName​(java.lang.String name)
        Converts a simple string to a qualified metric name based on the process type.
        Parameters:
        name - the name of the metric
        Returns:
        the metric with instance and id tags

      • getMasterMetricName

        public static java.lang.String getMasterMetricName​(java.lang.String name)
        Builds metric registry names for master instance. The pattern is instance.metricName.
        Parameters:
        name - the metric name
        Returns:
        the metric registry name

      • getJobWorkerMetricName

        public static java.lang.String getJobWorkerMetricName​(java.lang.String name)
        Builds metric registry name for job worker instance. The pattern is instance.uniqueId.metricName.
        Parameters:
        name - the metric name
        Returns:
        the metric registry name

      • getPluginMetricName

        public static java.lang.String getPluginMetricName​(java.lang.String name)
        Builds metric registry name for plugin instance. The pattern is instance.uniqueId.metricName.
        Parameters:
        name - the metric name
        Returns:
        the metric registry name

      • checkMinimalPollingPeriod

        public static void checkMinimalPollingPeriod​(java.util.concurrent.TimeUnit pollUnit,
                                             int pollPeriod)
                                      throws java.lang.IllegalArgumentException
        Checks if the poll period is smaller that the minimal poll period which is 1 second.
        Parameters:
        pollUnit - the polling unit
        pollPeriod - the polling period
        Throws:
        java.lang.IllegalArgumentException - if the polling period is invalid

      • stripInstanceAndHost

        public static java.lang.String stripInstanceAndHost​(java.lang.String metricsName)
        Removes the instance and host from the given metric name, returning the result.
        Parameters:
        metricsName - the long metrics name with instance and host name
        Returns:
        the metrics name without instance and host name

      • escape

        public static java.lang.String escape​(AlluxioURI uri)
        Escapes a URI, replacing "/" with "%2F". Replaces "." with "%2E" because dots are used as tag separators in metric names. Replaces "%" with "%25" because it now has special use. So when the URI is used in a metric name, the "." and "/" won't be interpreted as path separators unescaped and interfere with the internal logic of AlluxioURI.
        Parameters:
        uri - the URI to escape
        Returns:
        the string representing the escaped URI

      • unescape

        public static java.lang.String unescape​(java.lang.String uri)
        Unescapes a URI, reverts it to before the escape, to display it correctly.
        Parameters:
        uri - the escaped URI to unescape
        Returns:
        the string representing the unescaped original URI

      • executorService

        public static InstrumentedExecutorService executorService​(java.util.concurrent.ExecutorService delegate,
                                                          java.lang.String name)
        Add or replace the instrumented executor service metrics for the given name and executor service.
        Parameters:
        delegate - the executor service delegate that will be instrumented with metrics
        name - the name of the metric
        Returns:
        the instrumented executor service

      • counter

        public static com.codahale.metrics.Counter counter​(java.lang.String name)
        Get or add counter with the given name. The counter stores in the metrics system is never removed but may reset to zero.
        Parameters:
        name - the name of the metric
        Returns:
        a counter object with the qualified metric name

      • counterWithTags

        public static com.codahale.metrics.Counter counterWithTags​(java.lang.String name,
                                                           boolean shouldReport,
                                                           java.lang.String... tags)
        Get or add counter with the given name with tags. The counter stores in the metrics system is never removed but may reset to zero. If this metric can be aggregated at cluster level and should report to leading master, add it to the should report metrics map. This method is added to add worker metrics with ufs tags into the should report metrics map.
        Parameters:
        name - the metric name
        shouldReport - whether this metric should be reported
        tags - the tag name and tag value pairs
        Returns:
        a counter object with the qualified metric name

      • meter

        public static com.codahale.metrics.Meter meter​(java.lang.String name)
        Get or add meter with the given name. Please don't save the Meter instance since the returned Meter instance may not be used due to resetAllMetrics()
        Parameters:
        name - the name of the metric
        Returns:
        a meter object with the qualified metric name

      • meterWithTags

        public static com.codahale.metrics.Meter meterWithTags​(java.lang.String name,
                                                       boolean shouldReport,
                                                       java.lang.String... tags)
        Get or add meter with the given name. The returned meter may be changed due to resetAllMetrics() If this metric can be aggregated at cluster level and should report to leading master, add it to the should report metrics map. This method is added to add worker metrics with ufs tags into the should report metrics map.
        Parameters:
        name - the name of the metric
        shouldReport - whether this metric should be reported
        tags - the tag name and tag value pairs
        Returns:
        a meter object with the qualified metric name

      • timer

        public static com.codahale.metrics.Timer timer​(java.lang.String name)
        Get or add timer with the given name. Please don't save the Timer instance since the returned Timer instance may not be used due to resetAllMetrics()
        Parameters:
        name - the name of the metric
        Returns:
        a timer object with the qualified metric name

      • uniformTimer

        public static com.codahale.metrics.Timer uniformTimer​(java.lang.String name)
        Same with timer(java.lang.String) but with UnirformReservoir for sampling.
        Parameters:
        name - the name of the metric
        Returns:
        a timer object with the qualified metric name

      • registerGaugeIfAbsent

        public static <T> void registerGaugeIfAbsent​(java.lang.String name,
                                             com.codahale.metrics.Gauge<T> metric)
        Registers a gauge if it has not been registered.
        Type Parameters:
        T - the type
        Parameters:
        name - the gauge name
        metric - the gauge

      • registerCachedGaugeIfAbsent

        public static <T> void registerCachedGaugeIfAbsent​(java.lang.String name,
                                                   com.codahale.metrics.Gauge<T> metric)
        Registers a cached gauge if it has not been registered.
        Type Parameters:
        T - the type
        Parameters:
        name - the gauge name
        metric - the gauge

      • registerCachedGaugeIfAbsent

        public static <T> void registerCachedGaugeIfAbsent​(java.lang.String name,
                                                   com.codahale.metrics.Gauge<T> metric,
                                                   long timeout,
                                                   java.util.concurrent.TimeUnit unit)
        Registers a cached gauge if it has not been registered.
        Type Parameters:
        T - the type
        Parameters:
        name - the gauge name
        metric - the gauge
        timeout - the cache gauge timeout
        unit - the unit of timeout

      • registerAggregatedCachedGaugeIfAbsent

        public static void registerAggregatedCachedGaugeIfAbsent​(java.lang.String name,
                                                         java.util.Set<MetricKey> metrics,
                                                         long timeout,
                                                         java.util.concurrent.TimeUnit timeUnit)
        Created a gauge that aggregates the value of existing gauges.
        Parameters:
        name - the gauge name
        metrics - the set of metric values to be aggregated
        timeout - the cached gauge timeout
        timeUnit - the unit of timeout

      • removeMetrics

        public static boolean removeMetrics​(java.lang.String name)
        Removes the metric with the given name.
        Parameters:
        name - the metric name
        Returns:
        true if the metric was removed, false otherwise

      • reportWorkerMetrics

        public static java.util.List<alluxio.grpc.Metric> reportWorkerMetrics()
        Returns:
        the worker metrics to send via RPC

      • reportClientMetrics

        public static java.util.List<alluxio.grpc.Metric> reportClientMetrics()
        Returns:
        the client metrics to send via RPC

      • getMasterMetrics

        public static java.util.Map<java.lang.String,​java.util.Set<Metric>> getMasterMetrics​(java.util.Set<java.lang.String> metricNames)
        Gets all the master metrics belongs to the given metric names.
        Parameters:
        metricNames - the name of the metrics to get
        Returns:
        a metric map from metric name to metrics with this name

      • getMetricValue

        @Nullable
public static Metric getMetricValue​(java.lang.String fullName)
        Gets metric with the given full metric name.
        Parameters:
        fullName - the full name of the metric to get
        Returns:
        a metric set with the master metric of the given metric name

      • allMetrics

        public static java.util.Map<java.lang.String,​alluxio.grpc.MetricValue> allMetrics()
        Returns:
        a map of all metrics stored in the current node from metric name to MetricValue

      • initShouldReportMetrics

        public static void initShouldReportMetrics​(MetricsSystem.InstanceType instanceType)
        Initialize the SHOULD_REPORT_METRICS. This should be called only once. Note that this method is able to catch most of the should report metrics except worker metrics with ufs tags.
        Parameters:
        instanceType - the instance type

      • resetAllMetrics

        public static void resetAllMetrics()
        Resets all the metrics in the MetricsSystem. This method is not thread-safe and should be used sparingly.

      • clearAllMetrics

        public static void clearAllMetrics()
        Resets the metric registry and removes all the metrics.

      • resetCountersAndGauges

        public static void resetCountersAndGauges()
        Resets all counters to 0 and unregisters gauges for testing.