Package org.apache.accumulo.core.client.rfile

Class RFileWriter

  • All Implemented Interfaces:
    AutoCloseable
    public class RFileWriter
extends Object
implements AutoCloseable
    This class provides an API for writing RFiles. It can be used to create file for bulk import into Accumulo using TableOperations.importDirectory(String)

    A RFileWriter has the following constraints. Violating these constraints will result in runtime exceptions.

    • Keys must be appended in sorted order within a locality group.
    • Locality groups must have a mutually exclusive set of column families.
    • The default locality group must be started last.

    Below is an example of using RFileWriter 

     
     Iterable<Entry<Key, Value>> localityGroup1Data = ...
     Iterable<Entry<Key, Value>> localityGroup2Data = ...
     Iterable<Entry<Key, Value>> defaultGroupData = ...

     try(RFileWriter writer = RFile.newWriter().to(file).build()) {

       // Start a locality group before appending data.
       writer.startNewLocalityGroup("groupA", "columnFam1", "columnFam2");
       // Append data to the locality group that was started above. Must append in sorted order.
       writer.append(localityGroup1Data);

       // Add another locality group.
       writer.startNewLocalityGroup("groupB", "columnFam3", "columnFam4");
       writer.append(localityGroup2Data);

       // The default locality group must be started last.
       // The column families for the default group do not need to be specified.
       writer.startDefaultLocalityGroup();
       // Data appended here can not contain any
       // column families specified in previous locality groups.
       writer.append(defaultGroupData);

       // This is a try-with-resources so the writer is closed here at the end of the code block.
     }

    Create instances by calling RFile.newWriter()

    Since:
    1.8.0

    • Method Detail

      • startNewLocalityGroup

        public void startNewLocalityGroup​(String name,
                                  List<byte[]> families)
                           throws IOException
        Before appending any data, a locality group must be started. The default locality group must be started last.
        Parameters:
        name - locality group name, used for informational purposes
        families - the column families the locality group can contain
        Throws:
        IllegalStateException - When default locality group already started.
        IOException

      • startDefaultLocalityGroup

        public void startDefaultLocalityGroup()
                               throws IOException
        A locality group in which the column families do not need to specified. The locality group must be started after all other locality groups. Can not append column families that were in a previous locality group. If no locality groups were started, then the first append will start the default locality group.
        Throws:
        IllegalStateException - When default locality group already started.
        IOException

      • append

        public void append​(Key key,
                   Value val)
            throws IOException
        Append the key and value to the last locality group that was started. If no locality group was started, then the default group will automatically be started.
        Parameters:
        key - This key must be greater than or equal to the last key appended. For non-default locality groups, the keys column family must be one of the column families specified when calling startNewLocalityGroup(). Must be non-null.
        val - value to append, must be non-null.
        Throws:
        IllegalArgumentException - This is thrown when data is appended out of order OR when the key contains a invalid visibility OR when a column family is not valid for a locality group.
        IOException

      • append

        public void append​(Iterable<Map.Entry<Key,​Value>> keyValues)
            throws IOException
        Append the keys and values to the last locality group that was started.
        Parameters:
        keyValues - The keys must be in sorted order. The first key returned by the iterable must be greater than or equal to the last key appended. For non-default locality groups, the keys column family must be one of the column families specified when calling startNewLocalityGroup(). Must be non-null. If no locality group was started, then the default group will automatically be started.
        Throws:
        IllegalArgumentException - This is thrown when data is appended out of order OR when the key contains a invalid visibility OR when a column family is not valid for a locality group.
        IOException