Package com.github.ambry.cloud.azure

Class StorageClient

java.lang.Object

com.github.ambry.cloud.azure.StorageClient

Direct Known Subclasses:

ADAuthBasedStorageClient, ConnectionStringBasedStorageClient

public abstract class StorageClient
extends java.lang.Object

Abstract class encapsulation ABS client operations.

Field Summary

Fields

Modifier and Type	Field	Description
protected AzureMetrics	azureMetrics

Constructor Summary

Constructors

Constructor	Description
StorageClient(com.azure.storage.blob.BlobServiceClient blobServiceClient, com.azure.storage.blob.batch.BlobBatchClient blobBatchClient, AzureMetrics azureMetrics, AzureBlobLayoutStrategy blobLayoutStrategy)	Constructor for StorageClient.
StorageClient(com.github.ambry.config.CloudConfig cloudConfig, AzureCloudConfig azureCloudConfig, AzureMetrics azureMetrics, AzureBlobLayoutStrategy blobLayoutStrategy)	Constructor for StorageClient.

Method Summary

Modifier and Type	Method	Description
protected abstract com.azure.storage.blob.BlobServiceClient	buildBlobServiceClient(com.azure.core.http.HttpClient httpClient, com.azure.core.util.Configuration configuration, com.azure.storage.common.policy.RequestRetryOptions retryOptions, AzureCloudConfig azureCloudConfig)	Build BlobServiceClient.
protected com.azure.storage.blob.BlobServiceClient	createBlobStorageClient()	Create the BlobServiceClient object.
java.util.List<com.azure.core.http.rest.Response<java.lang.Void>>	deleteBatch(java.util.List<CloudBlobMetadata> batchOfBlobs, java.time.Duration timeout)	Deletes a list of blobs.
com.azure.storage.blob.models.BlobDownloadResponse	downloadWithResponse(java.lang.String containerName, java.lang.String blobName, boolean autoCreateContainer, java.io.OutputStream stream, com.azure.storage.blob.models.BlobRange range, com.azure.storage.blob.models.DownloadRetryOptions options, com.azure.storage.blob.models.BlobRequestConditions requestConditions, boolean getRangeContentMd5, java.time.Duration timeout)	Downloads a range of bytes from a blob into an output stream.
com.azure.storage.blob.models.BlobProperties	getPropertiesWithResponse(com.github.ambry.commons.BlobId blobId, com.azure.storage.blob.models.BlobRequestConditions requestConditions, java.time.Duration timeout)	Returns the blob's metadata and properties.
com.azure.storage.blob.BlobServiceClient	getStorageClient()	Visible for testing.
protected abstract boolean	handleExceptionAndHintRetry(com.azure.storage.blob.models.BlobStorageException blobStorageException)	Check if the exception can be handled and return a flag indicating if it can be retried.
protected void	setClientReferences(com.azure.storage.blob.BlobServiceClient blobServiceClient)	Set the references for storage and blob clients atomically.
void	setMetadataWithResponse(com.github.ambry.commons.BlobId blobId, java.util.Map<java.lang.String,java.lang.String> metadata, com.azure.storage.blob.models.BlobRequestConditions requestConditions, java.time.Duration timeout, com.azure.core.util.Context context)	Changes a blob's metadata.
com.azure.core.http.rest.Response<com.azure.storage.blob.models.BlockBlobItem>	uploadWithResponse(com.github.ambry.commons.BlobId blobId, java.io.InputStream data, long length, com.azure.storage.blob.models.BlobHttpHeaders headers, java.util.Map<java.lang.String,java.lang.String> metadata, com.azure.storage.blob.models.AccessTier tier, byte[] contentMd5, com.azure.storage.blob.models.BlobRequestConditions requestConditions, java.time.Duration timeout)	Creates a new block blob, or updates the content of an existing block blob.
com.azure.core.http.rest.Response<com.azure.storage.blob.models.BlockBlobItem>	uploadWithResponse(java.lang.String containerName, java.lang.String blobName, boolean autoCreateContainer, java.io.InputStream data, long length, com.azure.storage.blob.models.BlobHttpHeaders headers, java.util.Map<java.lang.String,java.lang.String> metadata, com.azure.storage.blob.models.AccessTier tier, byte[] contentMd5, com.azure.storage.blob.models.BlobRequestConditions requestConditions, java.time.Duration timeout)	Creates a new block blob, or updates the content of an existing block blob.
protected abstract void	validateABSAuthConfigs(AzureCloudConfig azureCloudConfig)	Validate that all the required configs for ABS authentication are present.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

azureMetrics

protected final AzureMetrics azureMetrics

Constructor Detail

StorageClient

public StorageClient(com.github.ambry.config.CloudConfig cloudConfig,
                     AzureCloudConfig azureCloudConfig,
                     AzureMetrics azureMetrics,
                     AzureBlobLayoutStrategy blobLayoutStrategy)

Constructor for StorageClient.

Parameters:

cloudConfig - CloudConfig object.

azureCloudConfig - AzureCloudConfig object.

azureMetrics - AzureMetrics object.

blobLayoutStrategy - AzureBlobLayoutStrategy object.

StorageClient

public StorageClient(com.azure.storage.blob.BlobServiceClient blobServiceClient,
                     com.azure.storage.blob.batch.BlobBatchClient blobBatchClient,
                     AzureMetrics azureMetrics,
                     AzureBlobLayoutStrategy blobLayoutStrategy)

Constructor for StorageClient.

Parameters:

blobServiceClient - BlobServiceClient object.

blobBatchClient - BlobBatchClient object.

azureMetrics - AzureMetrics object.

blobLayoutStrategy - AzureBlobLayoutStrategy object.

Method Detail

getStorageClient

public com.azure.storage.blob.BlobServiceClient getStorageClient()

Visible for testing.

Returns:

the underlying BlobServiceClient.

uploadWithResponse

public com.azure.core.http.rest.Response<com.azure.storage.blob.models.BlockBlobItem> uploadWithResponse(com.github.ambry.commons.BlobId blobId,
                                                                                                         java.io.InputStream data,
                                                                                                         long length,
                                                                                                         com.azure.storage.blob.models.BlobHttpHeaders headers,
                                                                                                         java.util.Map<java.lang.String,java.lang.String> metadata,
                                                                                                         com.azure.storage.blob.models.AccessTier tier,
                                                                                                         byte[] contentMd5,
                                                                                                         com.azure.storage.blob.models.BlobRequestConditions requestConditions,
                                                                                                         java.time.Duration timeout)

Creates a new block blob, or updates the content of an existing block blob.

Parameters:

blobId - BlobId of the blob to upload.

data - The data to write to the blob.

length - The exact length of the data. It is important that this value match precisely the length of the data provided in the InputStream.

headers - BlobHttpHeaders

metadata - Metadata to associate with the blob.

tier - AccessTier for the destination blob.

contentMd5 - An MD5 hash of the block content.

requestConditions - BlobRequestConditions

timeout - An optional timeout value beyond which a RuntimeException will be raised.

Returns:

The information of the uploaded block blob.

Throws:

com.azure.core.exception.UnexpectedLengthException - when the length of data does not match the input length.

java.lang.NullPointerException - if the input data is null.

java.io.UncheckedIOException - If an I/O error occurs

uploadWithResponse

public com.azure.core.http.rest.Response<com.azure.storage.blob.models.BlockBlobItem> uploadWithResponse(java.lang.String containerName,
                                                                                                         java.lang.String blobName,
                                                                                                         boolean autoCreateContainer,
                                                                                                         java.io.InputStream data,
                                                                                                         long length,
                                                                                                         com.azure.storage.blob.models.BlobHttpHeaders headers,
                                                                                                         java.util.Map<java.lang.String,java.lang.String> metadata,
                                                                                                         com.azure.storage.blob.models.AccessTier tier,
                                                                                                         byte[] contentMd5,
                                                                                                         com.azure.storage.blob.models.BlobRequestConditions requestConditions,
                                                                                                         java.time.Duration timeout)

Creates a new block blob, or updates the content of an existing block blob.

Parameters:

containerName - name of the Azure container where the blob lives.

blobName - name of the blob.

autoCreateContainer - flag indicating whether to create the container if it does not exist.

data - The data to write to the blob.

length - The exact length of the data. It is important that this value match precisely the length of the data provided in the InputStream.

headers - BlobHttpHeaders

metadata - Metadata to associate with the blob.

tier - AccessTier for the destination blob.

contentMd5 - An MD5 hash of the block content.

requestConditions - BlobRequestConditions

timeout - An optional timeout value beyond which a RuntimeException will be raised.

Returns:

The information of the uploaded block blob.

Throws:

com.azure.core.exception.UnexpectedLengthException - when the length of data does not match the input length.

java.lang.NullPointerException - if the input data is null.

java.io.UncheckedIOException - If an I/O error occurs

downloadWithResponse

public com.azure.storage.blob.models.BlobDownloadResponse downloadWithResponse(java.lang.String containerName,
                                                                               java.lang.String blobName,
                                                                               boolean autoCreateContainer,
                                                                               java.io.OutputStream stream,
                                                                               com.azure.storage.blob.models.BlobRange range,
                                                                               com.azure.storage.blob.models.DownloadRetryOptions options,
                                                                               com.azure.storage.blob.models.BlobRequestConditions requestConditions,
                                                                               boolean getRangeContentMd5,
                                                                               java.time.Duration timeout)

Downloads a range of bytes from a blob into an output stream.

Parameters:

containerName - name of the Azure container where the blob lives.

blobName - name of the blob.

autoCreateContainer - flag indicating whether to create the container if it does not exist.

stream - A non-null OutputStream instance where the downloaded data will be written.

range - BlobRange

options - DownloadRetryOptions

requestConditions - BlobRequestConditions

getRangeContentMd5 - Whether the contentMD5 for the specified blob range should be returned.

timeout - An optional timeout value beyond which a RuntimeException will be raised.

Returns:

A response containing status code and HTTP headers.

Throws:

java.io.UncheckedIOException - If an I/O error occurs.

java.lang.NullPointerException - if stream is null

getPropertiesWithResponse

public com.azure.storage.blob.models.BlobProperties getPropertiesWithResponse(com.github.ambry.commons.BlobId blobId,
                                                                              com.azure.storage.blob.models.BlobRequestConditions requestConditions,
                                                                              java.time.Duration timeout)

Returns the blob's metadata and properties.

Parameters:

blobId - BlobId

requestConditions - BlobRequestConditions

timeout - An optional timeout value beyond which a RuntimeException will be raised.

Returns:

The blob properties and metadata.

setMetadataWithResponse

public void setMetadataWithResponse(com.github.ambry.commons.BlobId blobId,
                                    java.util.Map<java.lang.String,java.lang.String> metadata,
                                    com.azure.storage.blob.models.BlobRequestConditions requestConditions,
                                    java.time.Duration timeout,
                                    com.azure.core.util.Context context)

Changes a blob's metadata. The specified metadata in this method will replace existing metadata. If old values must be preserved, they must be downloaded and included in the call to this method.

Parameters:

blobId - BlobId object.

metadata - Metadata to associate with the blob.

requestConditions - BlobRequestConditions

timeout - An optional timeout value beyond which a RuntimeException will be raised.

context - Additional context that is passed through the Http pipeline during the service call.

deleteBatch

public java.util.List<com.azure.core.http.rest.Response<java.lang.Void>> deleteBatch(java.util.List<CloudBlobMetadata> batchOfBlobs,
                                                                                     java.time.Duration timeout)

Deletes a list of blobs.

Parameters:

batchOfBlobs - List of CloudBlobMetadata objects.

timeout - An optional timeout value beyond which a RuntimeException will be raised.

Returns:

List of Responses for the blobs in the batch.

Throws:

java.lang.RuntimeException - If the timeout duration completes before a response is returned.

com.azure.storage.blob.models.BlobStorageException - If the batch request is malformed.

com.azure.storage.blob.batch.BlobBatchStorageException - If throwOnAnyFailure is true and any request in the BlobBatch failed.

createBlobStorageClient

protected com.azure.storage.blob.BlobServiceClient createBlobStorageClient()

Create the BlobServiceClient object.

Returns:

BlobServiceClient object.

setClientReferences

protected void setClientReferences(com.azure.storage.blob.BlobServiceClient blobServiceClient)

Set the references for storage and blob clients atomically. Note this method is not thread safe and must always be called within a thread safe context.

Parameters:

blobServiceClient - BlobServiceClient object.

validateABSAuthConfigs

protected abstract void validateABSAuthConfigs(AzureCloudConfig azureCloudConfig)

Validate that all the required configs for ABS authentication are present.

Parameters:

azureCloudConfig - AzureCloudConfig object.

buildBlobServiceClient

protected abstract com.azure.storage.blob.BlobServiceClient buildBlobServiceClient(com.azure.core.http.HttpClient httpClient,
                                                                                   com.azure.core.util.Configuration configuration,
                                                                                   com.azure.storage.common.policy.RequestRetryOptions retryOptions,
                                                                                   AzureCloudConfig azureCloudConfig)
                                                                            throws java.net.MalformedURLException,
                                                                                   java.lang.InterruptedException,
                                                                                   java.util.concurrent.ExecutionException

Build BlobServiceClient.

Parameters:

httpClient - HttpClient object.

configuration - Configuration object.

retryOptions - RetryOptions object.

azureCloudConfig - AzureCloudConfig object.

Returns:

BlobServiceClient object.

Throws:

java.net.MalformedURLException

java.lang.InterruptedException

java.util.concurrent.ExecutionException

handleExceptionAndHintRetry

protected abstract boolean handleExceptionAndHintRetry(com.azure.storage.blob.models.BlobStorageException blobStorageException)

Check if the exception can be handled and return a flag indicating if it can be retried. Note that if this method changes state of this class, then it should do it in a thread safe way.

Parameters:

blobStorageException - BlobStorageException object.

Returns:

true if the operation can be retried. false otherwise.