Package org.apache.accumulo.server.master.balancer

Class HostRegexTableLoadBalancer

  • All Implemented Interfaces:
    ConfigurationObserver
    public class HostRegexTableLoadBalancer
extends TableLoadBalancer
implements ConfigurationObserver
    This balancer creates groups of tablet servers using user-provided regular expressions over the tablet server hostnames. Then it delegates to the table balancer to balance the tablets within the resulting group of tablet servers. All tablet servers that do not match a regex are grouped into a default group.
    Regex properties for this balancer are specified as:
    table.custom.balancer.host.regex.<tablename>=<regex>
    Periodically (default 5m) this balancer will check to see if a tablet server is hosting tablets that it should not be according to the regex configuration. If this occurs then the offending tablets will be reassigned. This would cover the case where the configuration is changed and the master is restarted while the tablet servers are up. To change the out of bounds check time period, set the following property:
    table.custom.balancer.host.regex.oob.period=5m
    Regex matching can be based on either the host name (default) or host ip address. To set this balancer to match the regular expressions to the tablet server IP address, then set the following property:
    table.custom.balancer.host.regex.is.ip=true
    It's possible that this balancer may create a lot of migrations. To limit the number of migrations that are created during a balance call, set the following property (default 250):
    table.custom.balancer.host.regex.concurrent.migrations This balancer can continue balancing even if there are outstanding migrations. To limit the number of outstanding migrations in which this balancer will continue balancing, set the following property (default 0):
    table.custom.balancer.host.regex.max.outstanding.migrations

    • Field Detail

      • HOST_BALANCER_PREFIX

        public static final String HOST_BALANCER_PREFIX

      • HOST_BALANCER_OOB_CHECK_KEY

        public static final String HOST_BALANCER_OOB_CHECK_KEY

      • HOST_BALANCER_REGEX_USING_IPS_KEY

        public static final String HOST_BALANCER_REGEX_USING_IPS_KEY

      • HOST_BALANCER_REGEX_MAX_MIGRATIONS_KEY

        public static final String HOST_BALANCER_REGEX_MAX_MIGRATIONS_KEY

      • HOST_BALANCER_OUTSTANDING_MIGRATIONS_KEY

        public static final String HOST_BALANCER_OUTSTANDING_MIGRATIONS_KEY

      • oobCheckMillis

        protected long oobCheckMillis

    • Constructor Detail

      • HostRegexTableLoadBalancer

        public HostRegexTableLoadBalancer()

    • Method Detail

      • splitCurrentByRegex

        protected Map<String,​SortedMap<TServerInstance,​TabletServerStatus>> splitCurrentByRegex​(SortedMap<TServerInstance,​TabletServerStatus> current)
        Group the set of current tservers by pool name. Tservers that don't match a regex are put into a default pool. This could be expensive in the terms of the amount of time to recompute the groups, so HOST_BALANCER_POOL_RECHECK_KEY should be specified in the terms of minutes, not seconds or less.
        Parameters:
        current - map of current tservers
        Returns:
        current servers grouped by pool name, if not a match it is put into a default pool.

      • getPoolNamesForHost

        protected List<String> getPoolNamesForHost​(String host)
        Matches host against the regexes and returns the matching pool names
        Parameters:
        host - tablet server host
        Returns:
        pool names, will return default pool if host matches more no regex

      • getPoolNameForTable

        protected String getPoolNameForTable​(String tableName)
        Matches table name against pool names, returns matching pool name or DEFAULT_POOL.
        Parameters:
        tableName - name of table
        Returns:
        tablet server pool name (table name or DEFAULT_POOL)

      • parseConfiguration

        protected void parseConfiguration​(ServerConfiguration conf)
        Parse configuration and extract properties
        Parameters:
        conf - server configuration

      • getPoolNameToRegexPattern

        public Map<String,​Pattern> getPoolNameToRegexPattern()

      • getMaxMigrations

        public int getMaxMigrations()

      • getMaxOutstandingMigrations

        public int getMaxOutstandingMigrations()

      • getOobCheckMillis

        public long getOobCheckMillis()

      • isIpBasedRegex

        public boolean isIpBasedRegex()

      • init

        public void init​(ServerContext context)
        Description copied from class: TabletBalancer
        Initialize the TabletBalancer. This gives the balancer the opportunity to read the configuration.
        Overrides:
        init in class TabletBalancer

      • getAssignments

        public void getAssignments​(SortedMap<TServerInstance,​TabletServerStatus> current,
                           Map<KeyExtent,​TServerInstance> unassigned,
                           Map<KeyExtent,​TServerInstance> assignments)
        Description copied from class: TabletBalancer
        Assign tablets to tablet servers. This method is called whenever the master finds tablets that are unassigned.
        Overrides:
        getAssignments in class TableLoadBalancer
        Parameters:
        current - The current table-summary state of all the online tablet servers. Read-only. The TabletServerStatus for each server may be null if the tablet server has not yet responded to a recent request for status.
        unassigned - A map from unassigned tablet to the last known tablet server. Read-only.
        assignments - A map from tablet to assigned server. Write-only.

      • balance

        public long balance​(SortedMap<TServerInstance,​TabletServerStatus> current,
                    Set<KeyExtent> migrations,
                    List<TabletMigration> migrationsOut)
        Description copied from class: TabletBalancer
        Ask the balancer if any migrations are necessary. If the balancer is going to self-abort due to some environmental constraint (e.g. it requires some minimum number of tservers, or a maximum number of outstanding migrations), it should issue a log message to alert operators. The message should be at WARN normally and at ERROR if the balancer knows that the problem can not self correct. It should not issue these messages more than once a minute.
        Overrides:
        balance in class TableLoadBalancer
        Parameters:
        current - The current table-summary state of all the online tablet servers. Read-only.
        migrations - the current set of migrations. Read-only.
        migrationsOut - new migrations to perform; should not contain tablets in the current set of migrations. Write-only.
        Returns:
        the time, in milliseconds, to wait before re-balancing. This method will not be called when there are unassigned tablets.