com.google.cloud.bigquery.storage.v1alpha2

Class JsonStreamWriter

java.lang.Object

com.google.cloud.bigquery.storage.v1alpha2.JsonStreamWriter

All Implemented Interfaces:

AutoCloseable

public class JsonStreamWriter
extends Object
implements AutoCloseable

A StreamWriter that can write JSON data (JSONObjects) to BigQuery tables. The JsonStreamWriter is built on top of a StreamWriter, and it simply converts all JSON data to protobuf messages then calls StreamWriter's append() method to write to BigQuery tables. It maintains all StreamWriter functions, but also provides an additional feature: schema update support, where if the BigQuery table schema is updated, users will be able to ingest data on the new schema after some time (in order of minutes).

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	JsonStreamWriter.Builder

Method Summary

Modifier and Type	Method and Description
com.google.api.core.ApiFuture<Storage.AppendRowsResponse>	append(org.json.JSONArray jsonArr, boolean allowUnknownFields) Writes a JSONArray that contains JSONObjects to the BigQuery table by first converting the JSON data to protobuf messages, then using StreamWriter's append() to write the data.
com.google.api.core.ApiFuture<Storage.AppendRowsResponse>	append(org.json.JSONArray jsonArr, long offset, boolean allowUnknownFields) Writes a JSONArray that contains JSONObjects to the BigQuery table by first converting the JSON data to protobuf messages, then using StreamWriter's append() to write the data.
void	close() Closes the underlying StreamWriter.
com.google.protobuf.Descriptors.Descriptor	getDescriptor() Gets current descriptor
String	getStreamName() Gets streamName
static JsonStreamWriter.Builder	newBuilder(String streamName, Table.TableSchema tableSchema) newBuilder that constructs a JsonStreamWriter builder with BigQuery client being initialized by StreamWriter by default.
static JsonStreamWriter.Builder	newBuilder(String streamName, Table.TableSchema tableSchema, BigQueryWriteClient client) newBuilder that constructs a JsonStreamWriter builder.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

append

public com.google.api.core.ApiFuture<Storage.AppendRowsResponse> append(org.json.JSONArray jsonArr,
                                                                        boolean allowUnknownFields)

Writes a JSONArray that contains JSONObjects to the BigQuery table by first converting the JSON data to protobuf messages, then using StreamWriter's append() to write the data. If there is a schema update, the OnSchemaUpdateRunnable will be used to determine what actions to perform.

Parameters:

jsonArr - The JSON array that contains JSONObjects to be written

allowUnknownFields - if true, json data can have fields unknown to the BigQuery table.

Returns:

ApiFuture returns an AppendRowsResponse message wrapped in an ApiFuture

append

public com.google.api.core.ApiFuture<Storage.AppendRowsResponse> append(org.json.JSONArray jsonArr,
                                                                        long offset,
                                                                        boolean allowUnknownFields)

Writes a JSONArray that contains JSONObjects to the BigQuery table by first converting the JSON data to protobuf messages, then using StreamWriter's append() to write the data. If there is a schema update, the OnSchemaUpdateRunnable will be used to determine what actions to perform.

Parameters:

jsonArr - The JSON array that contains JSONObjects to be written

offset - Offset for deduplication

allowUnknownFields - if true, json data can have fields unknown to the BigQuery table.

Returns:

ApiFuture returns an AppendRowsResponse message wrapped in an ApiFuture

getStreamName

public String getStreamName()

Gets streamName

Returns:

String

getDescriptor

public com.google.protobuf.Descriptors.Descriptor getDescriptor()

Gets current descriptor

Returns:

Descriptor

newBuilder

public static JsonStreamWriter.Builder newBuilder(String streamName,
                                                  Table.TableSchema tableSchema)

newBuilder that constructs a JsonStreamWriter builder with BigQuery client being initialized by StreamWriter by default.

Parameters:

streamName - name of the stream that must follow "projects/[^/]+/datasets/[^/]+/tables/[^/]+/streams/[^/]+"

tableSchema - The schema of the table when the stream was created, which is passed back through WriteStream

Returns:

Builder

newBuilder

public static JsonStreamWriter.Builder newBuilder(String streamName,
                                                  Table.TableSchema tableSchema,
                                                  BigQueryWriteClient client)

newBuilder that constructs a JsonStreamWriter builder.

Parameters:

streamName - name of the stream that must follow "projects/[^/]+/datasets/[^/]+/tables/[^/]+/streams/[^/]+"

tableSchema - The schema of the table when the stream was created, which is passed back through WriteStream

client -

Returns:

Builder

close

public void close()

Closes the underlying StreamWriter.

Specified by:

close in interface AutoCloseable