Class HelixAccountService
- java.lang.Object
-
- com.github.ambry.account.HelixAccountService
-
- All Implemented Interfaces:
com.github.ambry.account.AccountService
,java.io.Closeable
,java.lang.AutoCloseable
public class HelixAccountService extends java.lang.Object implements com.github.ambry.account.AccountService
An implementation of
AccountService
that employs aHelixPropertyStore
as its underlying storage. It has two differentAccountMetadataStore
implementations. UseHelixAccountServiceConfig.useNewZNodePath
to control which one to use. When the configuration is true, it uses theRouterStore
. Otherwise, it usesLegacyMetadataStore
. In bothAccountMetadataStore
, the latest fullAccount
metadata will be cached locally, so serving anAccount
query will not incur any calls to remoteZooKeeper
orAmbryServer
.When a
HelixAccountService
starts up, it will automatically fetch a full set ofAccount
metadata and store them in its local cache. It also takes aNotifier
, and subscribes to aACCOUNT_METADATA_CHANGE_TOPIC
. Each time when receiving aFULL_ACCOUNT_METADATA_CHANGE_MESSAGE
, it will fetch the updated fullAccount
metadata, and refresh its local cache. After every successful operation for updating a collection ofAccount
s, it publishes aFULL_ACCOUNT_METADATA_CHANGE_MESSAGE
forACCOUNT_METADATA_CHANGE_TOPIC
through theNotifier
. If the remoteAccount
metadata is corrupted or has conflict,HelixAccountService
will not update its local cache.Previously,
HelixAccountService
only keeps a backup when there is an updateAccount
HTTP request received by this instance. It doesn't backup mutations made by other instances. Since HTTP requests to updateAccount
are rare, latest backup file often holds a out-of-date view of theAccount
metadata at lots of instances. In order to keep backup file up-to-date, in the new implementation, each mutation to theAccount
metadata will be persisted withBackupFileManager
. This is achieved by the fact theHelixAccountService
will publish each mutation toZooKeeper
and upon receiving the mutation message, all theHelixAccountService
instances will fetch the latestAccount
metadata. And when it does, it also persists the latestAccount
metadata in the backup. Every time a mutation of account metadata is performed, a new backup file will be created for locally, in a directory specified by theHelixAccountServiceConfig.backupDir
. A backup file is created when the helix listener onACCOUNT_METADATA_CHANGE_TOPIC
is notified. And backup's filename could contain version and last modified time information of the znode that stores the account metadata. The flow to update account looks likeupdateAccounts(Collection)
is invokedAccountMetadataStore
merges the incoming account mutation request with the current account set and write it back to helixNotifier
publish this action to helix by writing toACCOUNT_METADATA_CHANGE_TOPIC
- Listeners listening on
ACCOUNT_METADATA_CHANGE_TOPIC
fetches the new account metadata set - Listeners persists this new set as a backup file through
BackupFileManager
BackupFileManager
would start removing the oldest backup file. The number can be configured throughHelixAccountServiceConfig.maxBackupFileCount
.The full set of the
Account
metadata are stored in a singleZNRecord
inLegacyMetadataStore
or a single blob inRouterStore
, as a simple map from a string account id to theAccount
content in json as a string.Limited by
HelixPropertyStore
, the total size ofAccount
data stored on a singleZNRecord
cannot exceed 1MB. If the serialized json string exceeded 1MB limitation, it will then be compressed before saving back to zookeeper.
-
-
Field Summary
Fields Modifier and Type Field Description protected java.util.concurrent.atomic.AtomicReference<com.github.ambry.account.AccountInfoMap>
accountInfoMapRef
protected AccountServiceMetrics
accountServiceMetrics
protected java.util.concurrent.CopyOnWriteArraySet<java.util.function.Consumer<java.util.Collection<com.github.ambry.account.Account>>>
accountUpdateConsumers
protected java.util.concurrent.locks.ReentrantLock
lock
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description boolean
addAccountUpdateConsumer(java.util.function.Consumer<java.util.Collection<com.github.ambry.account.Account>> accountUpdateConsumer)
protected void
checkOpen()
Ensures the account service is ready to process requests.void
close()
com.github.ambry.account.Account
getAccountById(short id)
com.github.ambry.account.Account
getAccountByName(java.lang.String accountName)
java.util.Collection<com.github.ambry.account.Account>
getAllAccounts()
protected void
notifyAccountUpdateConsumers(java.util.Collection<com.github.ambry.account.Account> updatedAccounts, boolean isCalledFromListener)
Logs and notifies account updateConsumer
s about any new account changes/creations.boolean
removeAccountUpdateConsumer(java.util.function.Consumer<java.util.Collection<com.github.ambry.account.Account>> accountUpdateConsumer)
void
selectInactiveContainersAndMarkInStore(com.github.ambry.server.StatsSnapshot statsSnapshot)
SelectsContainer
s to be marked as INACTIVE and marked in underlying account store.void
setupRouter(com.github.ambry.router.Router router)
set the router to the given one.void
updateAccounts(java.util.Collection<com.github.ambry.account.Account> accounts)
java.util.Collection<com.github.ambry.account.Container>
updateContainers(java.lang.String accountName, java.util.Collection<com.github.ambry.account.Container> containers)
protected void
updateResolvedContainers(com.github.ambry.account.Account account, java.util.Collection<com.github.ambry.account.Container> resolvedContainers)
Update the containers that have been vetted as non-duplicates and populated with ids.
-
-
-
Field Detail
-
accountInfoMapRef
protected final java.util.concurrent.atomic.AtomicReference<com.github.ambry.account.AccountInfoMap> accountInfoMapRef
-
lock
protected final java.util.concurrent.locks.ReentrantLock lock
-
accountUpdateConsumers
protected final java.util.concurrent.CopyOnWriteArraySet<java.util.function.Consumer<java.util.Collection<com.github.ambry.account.Account>>> accountUpdateConsumers
-
accountServiceMetrics
protected final AccountServiceMetrics accountServiceMetrics
-
-
Method Detail
-
setupRouter
public void setupRouter(com.github.ambry.router.Router router) throws java.lang.IllegalStateException
set the router to the given one. This is a blocking call. This function would block until it fetches theAccount
metadata from ambry server.- Parameters:
router
- The router to set.- Throws:
java.lang.IllegalStateException
- when the router already set up.
-
updateAccounts
public void updateAccounts(java.util.Collection<com.github.ambry.account.Account> accounts) throws com.github.ambry.account.AccountServiceException
This call is blocking until it completes the operation of updating account metadata to
HelixPropertyStore
.There is a slight chance that an
Account
could be updated successfully, but its value was set based on an outdatedAccount
. This can be fixed aftergenerationId
is introduced toAccount
.- Specified by:
updateAccounts
in interfacecom.github.ambry.account.AccountService
- Throws:
com.github.ambry.account.AccountServiceException
-
close
public void close()
- Specified by:
close
in interfacejava.lang.AutoCloseable
- Specified by:
close
in interfacejava.io.Closeable
-
checkOpen
protected void checkOpen()
Ensures the account service is ready to process requests. Throws an unchecked exception (for exampleIllegalStateException
) if the account service is not ready to process requests.
-
getAccountByName
public com.github.ambry.account.Account getAccountByName(java.lang.String accountName)
- Specified by:
getAccountByName
in interfacecom.github.ambry.account.AccountService
-
getAccountById
public com.github.ambry.account.Account getAccountById(short id)
- Specified by:
getAccountById
in interfacecom.github.ambry.account.AccountService
-
getAllAccounts
public java.util.Collection<com.github.ambry.account.Account> getAllAccounts()
- Specified by:
getAllAccounts
in interfacecom.github.ambry.account.AccountService
-
updateContainers
public java.util.Collection<com.github.ambry.account.Container> updateContainers(java.lang.String accountName, java.util.Collection<com.github.ambry.account.Container> containers) throws com.github.ambry.account.AccountServiceException
- Specified by:
updateContainers
in interfacecom.github.ambry.account.AccountService
- Throws:
com.github.ambry.account.AccountServiceException
-
updateResolvedContainers
protected void updateResolvedContainers(com.github.ambry.account.Account account, java.util.Collection<com.github.ambry.account.Container> resolvedContainers) throws com.github.ambry.account.AccountServiceException
Update the containers that have been vetted as non-duplicates and populated with ids.- Parameters:
account
- the account owning the containers.resolvedContainers
- the resolved containers.- Throws:
com.github.ambry.account.AccountServiceException
-
addAccountUpdateConsumer
public boolean addAccountUpdateConsumer(java.util.function.Consumer<java.util.Collection<com.github.ambry.account.Account>> accountUpdateConsumer)
- Specified by:
addAccountUpdateConsumer
in interfacecom.github.ambry.account.AccountService
-
removeAccountUpdateConsumer
public boolean removeAccountUpdateConsumer(java.util.function.Consumer<java.util.Collection<com.github.ambry.account.Account>> accountUpdateConsumer)
- Specified by:
removeAccountUpdateConsumer
in interfacecom.github.ambry.account.AccountService
-
notifyAccountUpdateConsumers
protected void notifyAccountUpdateConsumers(java.util.Collection<com.github.ambry.account.Account> updatedAccounts, boolean isCalledFromListener)
Logs and notifies account updateConsumer
s about any new account changes/creations.- Parameters:
updatedAccounts
- collection of updated accountsisCalledFromListener
-true
if the caller is the account update listener,false
otherwise.
-
selectInactiveContainersAndMarkInStore
public void selectInactiveContainersAndMarkInStore(com.github.ambry.server.StatsSnapshot statsSnapshot)
SelectsContainer
s to be marked as INACTIVE and marked in underlying account store.- Specified by:
selectInactiveContainersAndMarkInStore
in interfacecom.github.ambry.account.AccountService
-
-