Azure SDK for Java Reference Documentation
Azure Monitor Ingestion client library for Java
The Azure Monitor Ingestion client library is used to send custom logs to Azure Monitor using the Logs Ingestion API.
This library allows you to send data from virtually any source to supported built-in tables or to custom tables that you create in Log Analytics workspace. You can even extend the schema of built-in tables with custom columns.
Getting started
Prerequisites
- A Java Development Kit (JDK), version 8 or later.
- Here are details about Java 8 client compatibility with Azure Certificate Authority.
- Azure Subscription
- A Data Collection Endpoint
- A Data Collection Rule
- A Log Analytics workspace
Include the package
Include the BOM file
Please include the azure-sdk-bom to your project to take a dependency on the latest stable version of the library. In
the following snippet, replace the {bom_version_to_target} placeholder with the version number.
To learn more about the BOM, see the AZURE SDK BOM README.
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-sdk-bom</artifactId>
<version>{bom_version_to_target}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
and then include the direct dependency in the dependencies section without the version tag as shown below.
<dependencies>
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-monitor-ingestion</artifactId>
</dependency>
</dependencies>
Include direct dependency
If you want to take dependency on a particular version of the library that is not present in the BOM, add the direct dependency to your project as follows.
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-monitor-ingestion</artifactId>
<version>1.2.5</version>
</dependency>
Create the client
An authenticated client is required to upload logs to Azure Monitor. The library includes both synchronous and asynchronous forms of the clients. To authenticate,
the following examples use DefaultAzureCredentialBuilder from the azure-identity package.
Authenticating using Azure Active Directory
You can authenticate with Azure Active Directory using the Azure Identity library.
To use the DefaultAzureCredential provider shown below, or other credential providers provided with the Azure SDK, please include the azure-identity package:
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-identity</artifactId>
<version>1.13.1</version>
</dependency>
Set the values of the client ID, tenant ID, and client secret of the AAD application as environment variables: AZURECLIENTID, AZURETENANTID, AZURECLIENTSECRET.
Synchronous Logs Ingestion client
DefaultAzureCredential tokenCredential = new DefaultAzureCredentialBuilder().build();
LogsIngestionClient client = new LogsIngestionClientBuilder()
.endpoint("<data-collection-endpoint>")
.credential(tokenCredential)
.buildClient();
Asynchronous Logs Ingestion client
DefaultAzureCredential tokenCredential = new DefaultAzureCredentialBuilder().build();
LogsIngestionAsyncClient asyncClient = new LogsIngestionClientBuilder()
.endpoint("<data-collection-endpoint>")
.credential(tokenCredential)
.buildAsyncClient();
Key concepts
Data Collection Endpoint
Data Collection Endpoints (DCEs) allow you to uniquely configure ingestion settings for Azure Monitor. This article provides an overview of data collection endpoints including their contents and structure and how you can create and work with them.
Data Collection Rule
Data collection rules (DCR) define data collected by Azure Monitor and specify how and where that data should be sent or stored. The REST API call must specify a DCR to use. A single DCE can support multiple DCRs, so you can specify a different DCR for different sources and target tables.
The DCR must understand the structure of the input data and the structure of the target table. If the two don't match, it can use a transformation to convert the source data to match the target table. You may also use the transform to filter source data and perform any other calculations or conversions.
For more details, see Data collection rules in Azure Monitor. For information on how to retrieve a DCR ID, see this tutorial.
Log Analytics Workspace Tables
Custom logs can send data to any custom table that you create and to certain built-in tables in your Log Analytics workspace. The target table must exist before you can send data to it. The following built-in tables are currently supported:
Logs retrieval
The logs that were uploaded using this library can be queried using the Azure Monitor Query client library.
Examples
Upload custom logs
DefaultAzureCredential tokenCredential = new DefaultAzureCredentialBuilder().build();
LogsIngestionClient client = new LogsIngestionClientBuilder()
.endpoint("<data-collection-endpoint>")
.credential(tokenCredential)
.buildClient();
List<Object> logs = getLogs();
client.upload("<data-collection-rule-id>", "<stream-name>", logs);
System.out.println("Logs uploaded successfully");
Upload custom logs with max concurrency
If the in input logs collection is too large, the client will split the input into multiple smaller requests. These
requests are sent serially, by default, but by configuring the max concurrency in LogsUploadOptions, these requests
can be concurrently sent to the service as shown in the example below.
DefaultAzureCredential tokenCredential = new DefaultAzureCredentialBuilder().build();
LogsIngestionClient client = new LogsIngestionClientBuilder()
.endpoint("<data-collection-endpoint>")
.credential(tokenCredential)
.buildClient();
List<Object> logs = getLogs();
LogsUploadOptions logsUploadOptions = new LogsUploadOptions()
.setMaxConcurrency(3);
client.upload("<data-collection-rule-id>", "<stream-name>", logs, logsUploadOptions,
Context.NONE);
System.out.println("Logs uploaded successfully");
Upload custom logs with error handling
When uploading large collection of logs, the client splits the input into multiple smaller service requests. The upload method provides an option to handle individual service errors through an error handler as shown in the example below. This error handler include the exception details and the list of all logs that failed to upload. If an error handler is not provided, the upload method will throw an aggregate exception that includes all the service errors.
DefaultAzureCredential tokenCredential = new DefaultAzureCredentialBuilder().build();
LogsIngestionClient client = new LogsIngestionClientBuilder()
.endpoint("<data-collection-endpoint>")
.credential(tokenCredential)
.buildClient();
List<Object> logs = getLogs();
LogsUploadOptions logsUploadOptions = new LogsUploadOptions()
.setLogsUploadErrorConsumer(uploadLogsError -> {
System.out.println("Error message " + uploadLogsError.getResponseException().getMessage());
System.out.println("Total logs failed to upload = " + uploadLogsError.getFailedLogs().size());
// throw the exception here to abort uploading remaining logs
// throw uploadLogsError.getResponseException();
});
client.upload("<data-collection-rule-id>", "<stream-name>", logs, logsUploadOptions,
Context.NONE);
Troubleshooting
For details on diagnosing various failure scenarios, see our troubleshooting guide.
Next steps
More samples can be found here.
Contributing
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.
When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.