Package | Description |
---|---|
com.aliyun.odps.tunnel |
ODPS Tunnel服务用于上传、下载数据到ODPS
|
com.aliyun.odps.tunnel.impl | |
com.aliyun.odps.tunnel.io |
提供通过Tunnel读写数据的Reader、Writer
|
com.aliyun.odps.tunnel.io.streaming.utils | |
com.aliyun.odps.tunnel.streams |
Modifier and Type | Method and Description |
---|---|
void |
TableTunnel.UpsertSession.abort()
中止UpsertSession
|
abstract TableTunnel.StreamUploadSession |
TableTunnel.StreamUploadSession.Builder.build() |
TableTunnel.UpsertSession |
TableTunnel.UpsertSession.Builder.build() |
TableTunnel.DownloadSession |
TableTunnel.DownloadSessionBuilder.build() |
void |
TableTunnel.UploadSession.commit()
不进行校验的会话提交
|
void |
TableTunnel.UpsertSession.commit(boolean async)
提交UpsertSession
|
void |
TableTunnel.UploadSession.commit(Long[] blocks)
提交本次上传的所有数据块
|
void |
VolumeTunnel.UploadSession.commit(String[] files)
提交本次上传的所有file
|
void |
VolumeFSTunnel.commit(String project,
String path,
String sessionId)
Commit upload session
|
InstanceTunnel.DownloadSession |
InstanceTunnel.createDirectDownloadSession(String projectName,
String instanceID,
String taskName,
int queryId)
Deprecated.
|
InstanceTunnel.DownloadSession |
InstanceTunnel.createDirectDownloadSession(String projectName,
String instanceID,
String taskName,
int queryId,
boolean limitEnabled)
在 Instance 上创建下载long polling会话
非法情况:
1.
|
InstanceTunnel.DownloadSession |
InstanceTunnel.createDownloadSession(String projectName,
String instanceID)
在 Instance 上创建下载会话
非法情况:
1.
|
TableTunnel.DownloadSession |
TableTunnel.createDownloadSession(String projectName,
String tableName)
Deprecated.
|
InstanceTunnel.DownloadSession |
InstanceTunnel.createDownloadSession(String projectName,
String instanceID,
boolean limitEnabled)
在 Instance 上创建下载会话
非法情况:
1.
|
TableTunnel.DownloadSession |
TableTunnel.createDownloadSession(String projectName,
String tableName,
boolean async)
Deprecated.
|
TableTunnel.DownloadSession |
TableTunnel.createDownloadSession(String projectName,
String tableName,
long shardId)
Deprecated.
|
TableTunnel.DownloadSession |
TableTunnel.createDownloadSession(String projectName,
String tableName,
PartitionSpec partitionSpec)
Deprecated.
|
TableTunnel.DownloadSession |
TableTunnel.createDownloadSession(String projectName,
String tableName,
PartitionSpec partitionSpec,
boolean async)
Deprecated.
|
TableTunnel.DownloadSession |
TableTunnel.createDownloadSession(String projectName,
String tableName,
PartitionSpec partitionSpec,
long shardId)
Deprecated.
|
TableTunnel.DownloadSession |
TableTunnel.createDownloadSession(String projectName,
String schemaName,
String tableName,
boolean async)
Deprecated.
|
TableTunnel.DownloadSession |
TableTunnel.createDownloadSession(String projectName,
String schemaName,
String tableName,
PartitionSpec partitionSpec,
boolean async)
Deprecated.
|
VolumeTunnel.DownloadSession |
VolumeTunnel.createDownloadSession(String projectName,
String volumeName,
String partition,
String fileName)
构造一个新的
VolumeTunnel.DownloadSession 对象。 |
TableTunnel.UploadSession |
TableTunnel.createUploadSession(String projectName,
String tableName)
在非分区表上创建上传会话
|
TableTunnel.UploadSession |
TableTunnel.createUploadSession(String projectName,
String tableName,
boolean overwrite)
在非分区表上创建上传会话
|
TableTunnel.UploadSession |
TableTunnel.createUploadSession(String projectName,
String tableName,
PartitionSpec partitionSpec)
在分区表上创建上传会话
|
TableTunnel.UploadSession |
TableTunnel.createUploadSession(String projectName,
String tableName,
PartitionSpec partitionSpec,
boolean overwrite)
在分区表上创建上传会话
|
VolumeTunnel.UploadSession |
VolumeTunnel.createUploadSession(String projectName,
String volumeName,
String partition)
构造一个新的
VolumeTunnel.UploadSession 对象。 |
TableTunnel.UploadSession |
TableTunnel.createUploadSession(String projectName,
String schemaName,
String tableName,
boolean overwrite)
Create an upload session of a non-partitioned table.
|
TableTunnel.UploadSession |
TableTunnel.createUploadSession(String projectName,
String schemaName,
String tableName,
PartitionSpec partitionSpec,
boolean overwrite)
Create an upload session of a partitioned table.
|
Long[] |
TableTunnel.UploadSession.getBlockList()
获取当前会话已经上传成功的数据块列表
|
TableTunnel.DownloadSession |
TableTunnel.getDownloadSession(String projectName,
String tableName,
long shardId,
String id)
Deprecated.
|
TableTunnel.DownloadSession |
TableTunnel.getDownloadSession(String projectName,
String tableName,
PartitionSpec partitionSpec,
long shardId,
String id)
Deprecated.
|
TableTunnel.DownloadSession |
TableTunnel.getDownloadSession(String projectName,
String tableName,
PartitionSpec partitionSpec,
String id)
获得在分区表上创建的下载会话
|
TableTunnel.DownloadSession |
TableTunnel.getDownloadSession(String projectName,
String tableName,
String id)
获得在非分区表上创建的下载会话
|
TableTunnel.DownloadSession |
TableTunnel.getDownloadSession(String projectName,
String schemaName,
String tableName,
PartitionSpec partitionSpec,
String id)
Get the download session specified by the download session ID.
|
TableTunnel.DownloadSession |
TableTunnel.getDownloadSession(String projectName,
String schemaName,
String tableName,
String id)
Get the download session specified by the download session ID.
|
VolumeTunnel.DownloadSession |
VolumeTunnel.getDownloadSession(String projectName,
String volumeName,
String partition,
String fileName,
String downloadId)
根据已有downloadId获取一个
VolumeTunnel.DownloadSession 对象。 |
URI |
Configuration.getEndpoint(String projectName)
取得指定Project的Tunnel服务入口地址
|
Long |
VolumeTunnel.UploadSession.getFileLength(String fileName) |
String[] |
VolumeTunnel.UploadSession.getFileList()
获取当前
VolumeTunnel.UploadSession 已成功上传的files |
InstanceTunnel.DownloadStatus |
InstanceTunnel.DownloadSession.getStatus()
获取会话状态
|
String |
TableTunnel.UpsertSession.getStatus()
获取Session状态
|
TableTunnel.UploadStatus |
TableTunnel.UploadSession.getStatus()
获取会话状态
|
TableTunnel.DownloadStatus |
TableTunnel.DownloadSession.getStatus()
获取会话状态
|
VolumeTunnel.UploadStatus |
VolumeTunnel.UploadSession.getStatus()
获取当前
VolumeTunnel.UploadSession 的Status |
VolumeTunnel.DownloadStatus |
VolumeTunnel.DownloadSession.getStatus()
获取当前
VolumeTunnel.DownloadSession 的Status |
TableTunnel.UploadSession |
TableTunnel.getUploadSession(String projectName,
String tableName,
PartitionSpec partitionSpec,
String id)
获得在分区表上创建的上传会话
|
TableTunnel.UploadSession |
TableTunnel.getUploadSession(String projectName,
String tableName,
PartitionSpec partitionSpec,
String id,
long shares,
long shareId)
获得在分区表的上传会话,且该会话将要使用
TunnelBufferedWriter 进行数据上传。
当有多个这样的会话实例(多进程或多线程)共享会话 ID 时,需要同时声明此会话实例的唯一标识(shareId)和共享的会话实例个数(shares)。 |
TableTunnel.UploadSession |
TableTunnel.getUploadSession(String projectName,
String tableName,
String id)
获得在非分区表上创建的上传会话
|
TableTunnel.UploadSession |
TableTunnel.getUploadSession(String projectName,
String tableName,
String id,
long shares,
long shareId)
获得在非分区表的上传会话,且该会话将要使用
TunnelBufferedWriter 进行数据上传。
当有多个这样的会话实例(多进程或多线程)共享会话 ID 时,需要同时声明此会话实例的唯一标识(shareId)和共享的会话实例个数(shares)。 |
TableTunnel.UploadSession |
TableTunnel.getUploadSession(String projectName,
String schemaName,
String tableName,
PartitionSpec partitionSpec,
String id)
Get the upload session specified by the upload session ID.
|
TableTunnel.UploadSession |
TableTunnel.getUploadSession(String projectName,
String schemaName,
String tableName,
PartitionSpec partitionSpec,
String id,
long shares,
long shareId)
Get the upload session specified by the upload session ID.
|
TableTunnel.UploadSession |
TableTunnel.getUploadSession(String projectName,
String schemaName,
String tableName,
String id)
Get the upload session specified by the upload session ID.
|
VolumeTunnel.UploadSession |
VolumeTunnel.getUploadSession(String projectName,
String volumeName,
String partition,
String uploadId)
根据已有uploadId获取一个
VolumeTunnel.UploadSession 对象。 |
static String |
VolumeFSTunnel.getUploadSessionId(VolumeOutputStream outputStream)
Get upload sessionId from outputStream
|
void |
TunnelException.loadFromJson(InputStream is)
Deprecated.
|
TableTunnel.StreamRecordPack |
TableTunnel.StreamUploadSession.newRecordPack(CompressOption compressOption)
创建一个对象
|
ArrowRecordReader |
TableTunnel.DownloadSession.openArrowRecordReader(long start,
long count) |
ArrowRecordReader |
TableTunnel.DownloadSession.openArrowRecordReader(long start,
long count,
org.apache.arrow.memory.BufferAllocator allocator) |
ArrowRecordReader |
TableTunnel.DownloadSession.openArrowRecordReader(long start,
long count,
CompressOption compress) |
ArrowRecordReader |
TableTunnel.DownloadSession.openArrowRecordReader(long start,
long count,
List<Column> columns) |
ArrowRecordReader |
TableTunnel.DownloadSession.openArrowRecordReader(long start,
long count,
List<Column> columns,
org.apache.arrow.memory.BufferAllocator allocator) |
ArrowRecordReader |
TableTunnel.DownloadSession.openArrowRecordReader(long start,
long count,
List<Column> columns,
org.apache.arrow.memory.BufferAllocator allocator,
CompressOption compress) |
ArrowRecordWriter |
TableTunnel.UploadSession.openArrowRecordWriter(long blockId) |
ArrowRecordWriter |
TableTunnel.UploadSession.openArrowRecordWriter(long blockId,
CompressOption option) |
RecordWriter |
TableTunnel.UploadSession.openBufferedWriter()
打开一个无压缩
TunnelBufferedWriter 用来写入数据 |
RecordWriter |
TableTunnel.UploadSession.openBufferedWriter(boolean compress)
打开
TunnelBufferedWriter 用来写入数据 |
RecordWriter |
TableTunnel.UploadSession.openBufferedWriter(CompressOption compressOption)
打开
TunnelBufferedWriter 用来写入数据 |
RecordWriter |
TableTunnel.UploadSession.openBufferedWriter(CompressOption compressOption,
long timeout)
打开
TunnelBufferedWriter 用来写入数据 |
InputStream |
VolumeTunnel.DownloadSession.openInputStream()
创建
InputStream 用来将指定File以数据流的形式输出。 |
InputStream |
VolumeTunnel.DownloadSession.openInputStream(boolean compress)
创建
InputStream 用来将指定File以数据流的形式输出。 |
InputStream |
VolumeTunnel.DownloadSession.openInputStream(long start,
long length)
创建
InputStream 用来将指定File以数据流的形式输出。 |
InputStream |
VolumeTunnel.DownloadSession.openInputStream(long start,
long length,
boolean compress)
创建
InputStream 用来将指定File以数据流的形式输出。 |
InputStream |
VolumeFSTunnel.openInputStream(String project,
String path,
Long start,
Long end,
CompressOption compressOption)
Create an InputStream at the indicated Path
|
OutputStream |
VolumeTunnel.UploadSession.openOutputStream(String fileName)
创建
OutputStream 用来将数据流入到指定File。 |
OutputStream |
VolumeTunnel.UploadSession.openOutputStream(String fileName,
boolean compress)
创建
OutputStream 用来将数据流入到指定File。 |
OutputStream |
VolumeTunnel.UploadSession.openOutputStream(String fileName,
boolean compress,
boolean append)
创建
OutputStream 用来将数据流入到指定File。 |
OutputStream |
VolumeFSTunnel.openOutputStream(String project,
String path,
Integer replication,
CompressOption compressOption)
Create an OutputStream at the indicated Path
|
TunnelRecordReader |
InstanceTunnel.DownloadSession.openRecordReader(long start,
long count)
打开
RecordReader 用来读取记录 |
TunnelRecordReader |
TableTunnel.DownloadSession.openRecordReader(long start,
long count)
打开
RecordReader 用来读取记录 |
TunnelRecordReader |
InstanceTunnel.DownloadSession.openRecordReader(long start,
long count,
boolean compress)
打开
RecordReader 用来读取记录 |
TunnelRecordReader |
TableTunnel.DownloadSession.openRecordReader(long start,
long count,
boolean compress)
打开
RecordReader 用来读取记录 |
TunnelRecordReader |
InstanceTunnel.DownloadSession.openRecordReader(long start,
long count,
boolean compress,
List<Column> columns)
打开
RecordReader 用来读取记录 |
TunnelRecordReader |
TableTunnel.DownloadSession.openRecordReader(long start,
long count,
boolean compress,
List<Column> columns)
打开
RecordReader 用来读取记录 |
TunnelRecordReader |
InstanceTunnel.DownloadSession.openRecordReader(long start,
long count,
CompressOption compress)
打开
RecordReader 用来读取记录 |
TunnelRecordReader |
TableTunnel.DownloadSession.openRecordReader(long start,
long count,
CompressOption compress)
打开
RecordReader 用来读取记录 |
TunnelRecordReader |
InstanceTunnel.DownloadSession.openRecordReader(long start,
long count,
CompressOption compress,
List<Column> columns)
打开
RecordReader 用来读取记录 |
TunnelRecordReader |
TableTunnel.DownloadSession.openRecordReader(long start,
long count,
CompressOption compress,
List<Column> columns)
打开
RecordReader 用来读取记录 |
TunnelRecordReader |
InstanceTunnel.DownloadSession.openRecordReader(long start,
long count,
long sizeLimit)
打开
RecordReader 用来读取记录 |
TunnelRecordReader |
InstanceTunnel.DownloadSession.openRecordReader(long start,
long count,
long sizeLimit,
boolean compress)
打开
RecordReader 用来读取记录 |
TunnelRecordReader |
InstanceTunnel.DownloadSession.openRecordReader(long start,
long count,
long sizeLimit,
boolean compress,
List<Column> columns)
打开
RecordReader 用来读取记录 |
TunnelRecordReader |
InstanceTunnel.DownloadSession.openRecordReader(long start,
long count,
long sizeLimit,
CompressOption compress,
List<Column> columns)
打开
RecordReader 用来读取记录 |
RecordWriter |
TableTunnel.UploadSession.openRecordWriter(long blockId)
打开
RecordWriter 用来写入数据 |
RecordWriter |
TableTunnel.UploadSession.openRecordWriter(long blockId,
boolean compress)
打开
RecordWriter 用来写入数据 |
RecordWriter |
TableTunnel.UploadSession.openRecordWriter(long blockId,
CompressOption compress)
打开
RecordWriter 用来写入数据 |
Constructor and Description |
---|
DownloadSession(String projectName,
String instanceID,
String downloadId)
根据已有downloadId构造一个
InstanceTunnel.DownloadSession 对象。 |
DownloadSession(String projectName,
String volumeName,
String partitionSpec,
String fileName)
构造一个新的
VolumeTunnel.DownloadSession 。 |
DownloadSession(String projectName,
String volumeName,
String partitionSpec,
String fileName,
String downloadId)
根据已有downloadId构造一个
VolumeTunnel.DownloadSession 对象。 |
UploadSession(String projectName,
String volumeName,
String partitionSpec)
构造一个新的
VolumeTunnel.UploadSession 对象。 |
UploadSession(String projectName,
String volumeName,
String partitionSpec,
String uploadId)
根据已有的uploadId构造一个
VolumeTunnel.UploadSession 对象。 |
Modifier and Type | Method and Description |
---|---|
void |
StreamUploadSessionImpl.abort() |
void |
UpsertSessionImpl.abort() |
UpsertStream |
UpsertStreamImpl.Builder.build() |
TableTunnel.StreamUploadSession |
StreamUploadSessionImpl.Builder.build() |
UpsertSessionImpl |
UpsertSessionImpl.Builder.build() |
void |
UpsertStreamImpl.close() |
void |
UpsertSessionImpl.commit(boolean async) |
void |
UpsertStreamImpl.delete(Record record) |
void |
UpsertStreamImpl.flush() |
String |
UpsertSessionImpl.getStatus() |
static com.aliyun.odps.rest.RestClient |
Util.newRestClient(Configuration conf,
String projectName) |
com.aliyun.odps.rest.RestClient |
ConfigurationImpl.newRestClient(String projectName) |
static List<com.aliyun.odps.tunnel.impl.Slot> |
Util.parseSlots(com.google.gson.JsonArray slotElements) |
void |
StreamUploadSessionImpl.reloadSlots(com.aliyun.odps.tunnel.impl.Slot slot,
String server,
int slotNum) |
void |
UpsertStreamImpl.upsert(Record record) |
Constructor and Description |
---|
StreamUploadSessionImpl(ConfigurationImpl conf,
String projectName,
String schemaName,
String tableName,
String partitionSpec,
boolean cretaPartition,
long slotNum,
List<Column> zorderColumns) |
UpsertSessionImpl(UpsertSessionImpl.Builder builder) |
UpsertStreamImpl(UpsertStreamImpl.Builder builder) |
Modifier and Type | Method and Description |
---|---|
static RawTunnelRecordReader |
RawTunnelRecordReader.createInstanceTunnelReader(long start,
long count,
CompressOption compress,
List<Column> columns,
com.aliyun.odps.rest.RestClient restClient,
InstanceTunnel.DownloadSession session,
boolean longPolling) |
static RawTunnelRecordReader |
RawTunnelRecordReader.createInstanceTunnelReader(long start,
long count,
long sizeLimit,
CompressOption compress,
List<Column> columns,
com.aliyun.odps.rest.RestClient restClient,
InstanceTunnel.DownloadSession session,
boolean longPolling) |
static RawTunnelRecordReader |
RawTunnelRecordReader.createTableTunnelReader(long start,
long count,
CompressOption compress,
List<Column> columns,
com.aliyun.odps.rest.RestClient restClient,
TableTunnel.DownloadSession session)
创建不可靠的 tunnel record reader
|
Constructor and Description |
---|
ArrowTunnelRecordReader(long start,
long count,
List<Column> columns,
com.aliyun.odps.rest.RestClient tunnelRestClient,
TableTunnel.DownloadSession session,
org.apache.arrow.memory.BufferAllocator allocator,
CompressOption compress) |
TunnelRecordReader(long start,
long count,
List<Column> columns,
CompressOption option,
com.aliyun.odps.rest.RestClient tunnelRestClient,
InstanceTunnel.DownloadSession session)
构造此类对象
|
TunnelRecordReader(long start,
long count,
List<Column> columns,
CompressOption option,
com.aliyun.odps.rest.RestClient tunnelRestClient,
TableTunnel.DownloadSession session)
构造此类对象
|
TunnelRecordReader(long start,
long count,
long sizeLimit,
List<Column> columns,
CompressOption option,
com.aliyun.odps.rest.RestClient tunnelRestClient,
InstanceTunnel.DownloadSession session)
构造此类对象
|
Modifier and Type | Method and Description |
---|---|
static byte[] |
CastUtils.toByteArray(Object value,
OdpsType type)
用于将record value转为字节数组
|
Modifier and Type | Method and Description |
---|---|
UpsertStream |
UpsertStream.Builder.build() |
void |
UpsertStream.close()
关闭UpsertStream, 调用后upsert/delete和flush接口均不可用
|
void |
UpsertStream.delete(Record record)
按delete操作,写入一条
Record 对象到缓冲区 |
void |
UpsertStream.flush()
发送缓冲区数据到服务端
|
void |
UpsertStream.upsert(Record record)
按upsert操作,写入一条
Record 对象到缓冲区 |
Copyright © 2023 Alibaba Cloud Computing. All rights reserved.