Package org.apache.flink.runtime.blob

Interface BlobWriter

  • All Known Implementing Classes:
    BlobServer, VoidBlobWriter
    public interface BlobWriter
    BlobWriter is used to upload data to the BLOB store.

    • Field Summary

      Fields 
      Modifier and Type Field Description
      static org.slf4j.Logger LOG  

    • Method Summary

      All Methods Static Methods Instance Methods Abstract Methods 
      Modifier and Type Method Description
      boolean deletePermanent​(org.apache.flink.api.common.JobID jobId, PermanentBlobKey permanentBlobKey)
      Delete the uploaded data with the given JobID and PermanentBlobKey.
      int getMinOffloadingSize()
      Returns the min size before data will be offloaded to the BLOB store.
      static <T> org.apache.flink.types.Either<org.apache.flink.util.SerializedValue<T>,​PermanentBlobKey> offloadWithException​(org.apache.flink.util.SerializedValue<T> serializedValue, org.apache.flink.api.common.JobID jobId, BlobWriter blobWriter)  
      PermanentBlobKey putPermanent​(org.apache.flink.api.common.JobID jobId, byte[] value)
      Uploads the data of the given byte array for the given job to the BLOB server and makes it a permanent BLOB.
      PermanentBlobKey putPermanent​(org.apache.flink.api.common.JobID jobId, InputStream inputStream)
      Uploads the data from the given input stream for the given job to the BLOB server and makes it a permanent BLOB.
      static <T> org.apache.flink.types.Either<org.apache.flink.util.SerializedValue<T>,​PermanentBlobKey> serializeAndTryOffload​(T value, org.apache.flink.api.common.JobID jobId, BlobWriter blobWriter)
      Serializes the given value and offloads it to the BlobServer if its size exceeds the minimum offloading size of the BlobServer.
      static <T> org.apache.flink.types.Either<org.apache.flink.util.SerializedValue<T>,​PermanentBlobKey> tryOffload​(org.apache.flink.util.SerializedValue<T> serializedValue, org.apache.flink.api.common.JobID jobId, BlobWriter blobWriter)  

    • Field Detail

      • LOG

        static final org.slf4j.Logger LOG

    • Method Detail

      • putPermanent

        PermanentBlobKey putPermanent​(org.apache.flink.api.common.JobID jobId,
                              byte[] value)
                       throws IOException
        Uploads the data of the given byte array for the given job to the BLOB server and makes it a permanent BLOB.
        Parameters:
        jobId - the ID of the job the BLOB belongs to
        value - the buffer to upload
        Returns:
        the computed BLOB key identifying the BLOB on the server
        Throws:
        IOException - thrown if an I/O error occurs while writing it to a local file, or uploading it to the HA store

      • putPermanent

        PermanentBlobKey putPermanent​(org.apache.flink.api.common.JobID jobId,
                              InputStream inputStream)
                       throws IOException
        Uploads the data from the given input stream for the given job to the BLOB server and makes it a permanent BLOB.
        Parameters:
        jobId - ID of the job this blob belongs to
        inputStream - the input stream to read the data from
        Returns:
        the computed BLOB key identifying the BLOB on the server
        Throws:
        IOException - thrown if an I/O error occurs while reading the data from the input stream, writing it to a local file, or uploading it to the HA store

      • deletePermanent

        boolean deletePermanent​(org.apache.flink.api.common.JobID jobId,
                        PermanentBlobKey permanentBlobKey)
        Delete the uploaded data with the given JobID and PermanentBlobKey.
        Parameters:
        jobId - ID of the job this blob belongs to
        permanentBlobKey - the key of this blob

      • getMinOffloadingSize

        int getMinOffloadingSize()
        Returns the min size before data will be offloaded to the BLOB store.
        Returns:
        minimum offloading size

      • serializeAndTryOffload

        static <T> org.apache.flink.types.Either<org.apache.flink.util.SerializedValue<T>,​PermanentBlobKey> serializeAndTryOffload​(T value,
                                                                                                                                 org.apache.flink.api.common.JobID jobId,
                                                                                                                                 BlobWriter blobWriter)
                                                                                                                          throws IOException
        Serializes the given value and offloads it to the BlobServer if its size exceeds the minimum offloading size of the BlobServer.
        Type Parameters:
        T - type of the value to serialize
        Parameters:
        value - to serialize
        jobId - to which the value belongs.
        blobWriter - to use to offload the serialized value
        Returns:
        Either the serialized value or the stored blob key
        Throws:
        IOException - if the data cannot be serialized

      • tryOffload

        static <T> org.apache.flink.types.Either<org.apache.flink.util.SerializedValue<T>,​PermanentBlobKey> tryOffload​(org.apache.flink.util.SerializedValue<T> serializedValue,
                                                                                                                     org.apache.flink.api.common.JobID jobId,
                                                                                                                     BlobWriter blobWriter)

      • offloadWithException

        static <T> org.apache.flink.types.Either<org.apache.flink.util.SerializedValue<T>,​PermanentBlobKey> offloadWithException​(org.apache.flink.util.SerializedValue<T> serializedValue,
                                                                                                                               org.apache.flink.api.common.JobID jobId,
                                                                                                                               BlobWriter blobWriter)