Package org.roaringbitmap.buffer

Class MutableRoaringBitmap

java.lang.Object
org.roaringbitmap.buffer.ImmutableRoaringBitmap
org.roaringbitmap.buffer.MutableRoaringBitmap
All Implemented Interfaces:
Externalizable, Serializable, Cloneable, Iterable<Integer>, AppendableStorage<MappeableContainer>, BitmapDataProvider, ImmutableBitmapDataProvider
public class MutableRoaringBitmap extends ImmutableRoaringBitmap implements Cloneable, Serializable, Iterable<Integer>, Externalizable, BitmapDataProvider, AppendableStorage<MappeableContainer>
MutableRoaringBitmap, a compressed alternative to the BitSet. It is similar to org.roaringbitmap.RoaringBitmap, but it differs in that it can interact with ImmutableRoaringBitmap objects in the sense that MutableRoaringBitmap is derived from MutableRoaringBitmap. A MutableRoaringBitmap is an instance of an ImmutableRoaringBitmap (where methods like "serialize" are implemented). That is, they both share the same core (immutable) methods, but a MutableRoaringBitmap adds methods that allow you to modify the object. This design allows us to use MutableRoaringBitmap as ImmutableRoaringBitmap instances when needed. MutableRoaringBitmap instances can be casted to an ImmutableRoaringBitmap instance in constant time which means that code written for ImmutableRoaringBitmap instances run at full speed (without copies) on MutableRoaringBitmap instances. A MutableRoaringBitmap can be used much like an org.roaringbitmap.RoaringBitmap instance, and they serialize to the same output. The RoaringBitmap instance will be faster since it does not carry the overhead of a ByteBuffer back-end, but the MutableRoaringBitmap can be used as an ImmutableRoaringBitmap instance. Thus, if you use ImmutableRoaringBitmap, you probably need to use MutableRoaringBitmap instances as well; if you do not use ImmutableRoaringBitmap, you probably want to use only RoaringBitmap instances. 
 
      import org.roaringbitmap.buffer.*;

      //...

      MutableRoaringBitmap rr = MutableRoaringBitmap.bitmapOf(1,2,3,1000);
      MutableRoaringBitmap rr2 = new MutableRoaringBitmap();
      for(int k = 4000; k<4255;++k) rr2.add(k);

      RoaringBitmap rror = RoaringBitmap.or(rr, rr2);

      //...
      DataOutputStream wheretoserialize = ...
      rr.runOptimize(); // can help compression
      rr.serialize(wheretoserialize);
Integers are added in unsigned sorted order. That is, they are treated as unsigned integers (see Java 8's Integer.toUnsignedLong function). Up to 4294967296 integers can be stored.
See Also:

  • Constructor Details

    • MutableRoaringBitmap

      public MutableRoaringBitmap()
      Create an empty bitmap

    • MutableRoaringBitmap

      public MutableRoaringBitmap(MutableRoaringArray highLowContainer)

    • MutableRoaringBitmap

      public MutableRoaringBitmap(RoaringBitmap rb)
      Create a MutableRoaringBitmap from a RoaringBitmap. The RoaringBitmap is not modified.
      Parameters:
      rb - the original bitmap

  • Method Details

    • addOffset

      public static MutableRoaringBitmap addOffset(ImmutableRoaringBitmap x, long offset)
      Generate a copy of the provided bitmap, but with all its values incremented by offset. The parameter offset can be negative. Values that would fall outside of the valid 32-bit range are discarded so that the result can have lower cardinality. This method can be relatively expensive when offset is not divisible by 65536. Use sparingly.
      Parameters:
      x - source bitmap
      offset - increment (can be negative)
      Returns:
      a new bitmap

    • add

      public static MutableRoaringBitmap add(MutableRoaringBitmap rb, long rangeStart, long rangeEnd)
      Generate a new bitmap with all integers in [rangeStart,rangeEnd) added.
      Parameters:
      rb - initial bitmap (will not be modified)
      rangeStart - inclusive beginning of range
      rangeEnd - exclusive ending of range
      Returns:
      new bitmap

    • add

      @Deprecated public static MutableRoaringBitmap add(MutableRoaringBitmap rb, int rangeStart, int rangeEnd)
      Deprecated.
      use the version where longs specify the range
      Generate a new bitmap with all integers in [rangeStart,rangeEnd) added.
      Parameters:
      rb - initial bitmap (will not be modified)
      rangeStart - inclusive beginning of range
      rangeEnd - exclusive ending of range
      Returns:
      new bitmap

    • and

      public static MutableRoaringBitmap and(MutableRoaringBitmap x1, MutableRoaringBitmap x2)
      Bitwise AND (intersection) operation. The provided bitmaps are *not* modified. This operation is thread-safe as long as the provided bitmaps remain unchanged.
      Parameters:
      x1 - first bitmap
      x2 - other bitmap
      Returns:
      result of the operation

    • andNot

      public static MutableRoaringBitmap andNot(MutableRoaringBitmap x1, MutableRoaringBitmap x2)
      Bitwise ANDNOT (difference) operation. The provided bitmaps are *not* modified. This operation is thread-safe as long as the provided bitmaps remain unchanged.
      Parameters:
      x1 - first bitmap
      x2 - other bitmap
      Returns:
      result of the operation

    • add

      public void add(int... dat)
      Set all the specified values to true. This can be expected to be slightly faster than calling "add" repeatedly. The provided integers values don't have to be in sorted order, but it may be preferable to sort them from a performance point of view.
      Parameters:
      dat - set values

    • addN

      public void addN(int[] dat, int offset, int n)
      Set the specified values to true, within given boundaries. This can be expected to be slightly faster than calling "add" repeatedly on the values dat[offset], dat[offset+1],..., dat[offset+n-1]. The provided integers values don't have to be in sorted order, but it may be preferable to sort them from a performance point of view.
      Parameters:
      dat - set values
      offset - from which index the values should be set to true
      n - how many values should be set to true

    • bitmapOf

      public static MutableRoaringBitmap bitmapOf(int... dat)
      Generate a bitmap with the specified values set to true. The provided integers values don't have to be in sorted order, but it may be preferable to sort them from a performance point of view.
      Parameters:
      dat - set values
      Returns:
      a new bitmap

    • rangeSanityCheck

      protected static void rangeSanityCheck(long rangeStart, long rangeEnd)

    • flip

      public static MutableRoaringBitmap flip(MutableRoaringBitmap bm, long rangeStart, long rangeEnd)
      Complements the bits in the given range, from rangeStart (inclusive) rangeEnd (exclusive). The given bitmap is unchanged.
      Parameters:
      bm - bitmap being negated
      rangeStart - inclusive beginning of range
      rangeEnd - exclusive ending of range
      Returns:
      a new Bitmap

    • flip

      @Deprecated public static MutableRoaringBitmap flip(MutableRoaringBitmap rb, int rangeStart, int rangeEnd)
      Deprecated.
      use the version where longs specify the range
      Complements the bits in the given range, from rangeStart (inclusive) rangeEnd (exclusive). The given bitmap is unchanged.
      Parameters:
      rb - bitmap being negated
      rangeStart - inclusive beginning of range
      rangeEnd - exclusive ending of range
      Returns:
      a new Bitmap

    • lazyorfromlazyinputs

      protected static MutableRoaringBitmap lazyorfromlazyinputs(MutableRoaringBitmap x1, MutableRoaringBitmap x2)

    • or

      public static MutableRoaringBitmap or(ImmutableRoaringBitmap... bitmaps)
      Compute overall OR between bitmaps. (Effectively calls BufferFastAggregation.or(org.roaringbitmap.buffer.ImmutableRoaringBitmap...))
      Parameters:
      bitmaps - input bitmaps
      Returns:
      aggregated bitmap

    • or

      public static MutableRoaringBitmap or(MutableRoaringBitmap x1, MutableRoaringBitmap x2)
      Bitwise OR (union) operation. The provided bitmaps are *not* modified. This operation is thread-safe as long as the provided bitmaps remain unchanged.
      Parameters:
      x1 - first bitmap
      x2 - other bitmap
      Returns:
      result of the operation

    • remove

      public static MutableRoaringBitmap remove(MutableRoaringBitmap rb, long rangeStart, long rangeEnd)
      Generate a new bitmap with all integers in [rangeStart,rangeEnd) removed.
      Parameters:
      rb - initial bitmap (will not be modified)
      rangeStart - inclusive beginning of range
      rangeEnd - exclusive ending of range
      Returns:
      new bitmap

    • remove

      @Deprecated public static MutableRoaringBitmap remove(MutableRoaringBitmap rb, int rangeStart, int rangeEnd)
      Deprecated.
      use the version where longs specify the range
      Generate a new bitmap with all integers in [rangeStart,rangeEnd) removed.
      Parameters:
      rb - initial bitmap (will not be modified)
      rangeStart - inclusive beginning of range
      rangeEnd - exclusive ending of range
      Returns:
      new bitmap

    • xor

      public static MutableRoaringBitmap xor(MutableRoaringBitmap x1, MutableRoaringBitmap x2)
      Bitwise XOR (symmetric difference) operation. The provided bitmaps are *not* modified. This operation is thread-safe as long as the provided bitmaps remain unchanged.
      Parameters:
      x1 - first bitmap
      x2 - other bitmap
      Returns:
      result of the operation

    • add

      public void add(int x)
      Add the value to the container (set the value to "true"), whether it already appears or not. Java lacks native unsigned integers but the x argument is considered to be unsigned. Within bitmaps, numbers are ordered according toInteger.compareUnsigned(int, int). We order the numbers like 0, 1, ..., 2147483647, -2147483648, -2147483647,..., -1.
      Specified by:
      add in interface BitmapDataProvider
      Parameters:
      x - integer value

    • add

      public void add(long rangeStart, long rangeEnd)
      Add to the current bitmap all integers in [rangeStart,rangeEnd).
      Specified by:
      add in interface BitmapDataProvider
      Parameters:
      rangeStart - inclusive beginning of range
      rangeEnd - exclusive ending of range

    • add

      @Deprecated public void add(int rangeStart, int rangeEnd)
      Deprecated.
      use the version where longs specify the range
      Add to the current bitmap all integers in [rangeStart,rangeEnd).
      Parameters:
      rangeStart - inclusive beginning of range
      rangeEnd - exclusive ending of range

    • and

      public void and(ImmutableRoaringBitmap array)
      In-place bitwise AND (intersection) operation. The current bitmap is modified.
      Parameters:
      array - other bitmap

    • andNot

      public void andNot(ImmutableRoaringBitmap x2)
      In-place bitwise ANDNOT (difference) operation. The current bitmap is modified.
      Parameters:
      x2 - other bitmap

    • orNot

      public void orNot(ImmutableRoaringBitmap other, long rangeEnd)
      In-place bitwise ORNOT operation. The current bitmap is modified.
      Parameters:
      other - the other bitmap
      rangeEnd - end point of the range (exclusive).

    • checkedAdd

      public boolean checkedAdd(int x)
      Add the value to the container (set the value to "true"), whether it already appears or not.
      Parameters:
      x - integer value
      Returns:
      true if the added int wasn't already contained in the bitmap. False otherwise.

    • checkedRemove

      public boolean checkedRemove(int x)
      If present remove the specified integer (effectively, sets its bit value to false)
      Parameters:
      x - integer value representing the index in a bitmap
      Returns:
      true if the unset bit was already in the bitmap

    • clear

      public void clear()
      reset to an empty bitmap; result occupies as much space a newly created bitmap.

    • clone

      public MutableRoaringBitmap clone()
      Overrides:
      clone in class ImmutableRoaringBitmap

    • deserialize

      public void deserialize(DataInput in) throws IOException
      Deserialize the bitmap (retrieve from the input stream). The current bitmap is overwritten. See format specification at https://github.com/RoaringBitmap/RoaringFormatSpec
      Parameters:
      in - the DataInput stream
      Throws:
      IOException - Signals that an I/O exception has occurred.

    • deserialize

      public void deserialize(ByteBuffer buffer) throws IOException
      Deserialize (retrieve) this bitmap. See format specification at https://github.com/RoaringBitmap/RoaringFormatSpec The current bitmap is overwritten. It is not necessary that limit() on the input ByteBuffer indicates the end of the serialized data. After loading this RoaringBitmap, you can advance to the rest of the data (if there is more) by setting bbf.position(bbf.position() + bitmap.serializedSizeInBytes()); Note that the input ByteBuffer is effectively copied (with the slice operation) so you should expect the provided ByteBuffer to remain unchanged.
      Parameters:
      buffer - the byte buffer (can be mapped, direct, array backed etc.
      Throws:
      IOException - Signals that an I/O exception has occurred.

    • flip

      public void flip(int x)
      Add the value if it is not already present, otherwise remove it.
      Parameters:
      x - integer value

    • flip

      public void flip(long rangeStart, long rangeEnd)
      Modifies the current bitmap by complementing the bits in the given range, from rangeStart (inclusive) rangeEnd (exclusive).
      Parameters:
      rangeStart - inclusive beginning of range
      rangeEnd - exclusive ending of range

    • flip

      @Deprecated public void flip(int rangeStart, int rangeEnd)
      Deprecated.
      use the version where longs specify the range
      Modifies the current bitmap by complementing the bits in the given range, from rangeStart (inclusive) rangeEnd (exclusive).
      Parameters:
      rangeStart - inclusive beginning of range
      rangeEnd - exclusive ending of range

    • getMappeableRoaringArray

      public MutableRoaringArray getMappeableRoaringArray()
      Returns:
      a mutable copy of this bitmap

    • iterator

      public Iterator<Integer> iterator()
      iterate over the positions of the true values.
      Specified by:
      iterator in interface Iterable<Integer>
      Overrides:
      iterator in class ImmutableRoaringBitmap
      Returns:
      the iterator

    • lazyor

      protected void lazyor(ImmutableRoaringBitmap x2)

    • naivelazyor

      protected void naivelazyor(ImmutableRoaringBitmap x2)

    • or

      public void or(ImmutableRoaringBitmap x2)
      In-place bitwise OR (union) operation. The current bitmap is modified.
      Parameters:
      x2 - other bitmap

    • readExternal

      public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException
      Specified by:
      readExternal in interface Externalizable
      Throws:
      IOException
      ClassNotFoundException

    • remove

      public void remove(int x)
      If present remove the specified integers (effectively, sets its bit value to false)
      Specified by:
      remove in interface BitmapDataProvider
      Parameters:
      x - integer value representing the index in a bitmap

    • remove

      public void remove(long rangeStart, long rangeEnd)
      Remove from the current bitmap all integers in [rangeStart,rangeEnd).
      Parameters:
      rangeStart - inclusive beginning of range
      rangeEnd - exclusive ending of range

    • remove

      @Deprecated public void remove(int rangeStart, int rangeEnd)
      Deprecated.
      use the version where longs specify the range
      Remove from the current bitmap all integers in [rangeStart,rangeEnd).
      Parameters:
      rangeStart - inclusive beginning of range
      rangeEnd - exclusive ending of range

    • removeRunCompression

      public boolean removeRunCompression()
      Remove run-length encoding even when it is more space efficient
      Returns:
      whether a change was applied

    • repairAfterLazy

      protected void repairAfterLazy()

    • runOptimize

      public boolean runOptimize()
      Use a run-length encoding where it is estimated as more space efficient
      Returns:
      whether a change was applied

    • toImmutableRoaringBitmap

      public ImmutableRoaringBitmap toImmutableRoaringBitmap()
      Convenience method, effectively casts the object to an object of class ImmutableRoaringBitmap. This function is equivalent to : 
       
       (ImmutableRoaringBitmap) bitmap
      Some users would prefer to generate a hard copy of the data. The following code illustrates how to proceed, but note that the resulting copy can be expected to perform significantly worse than the original: the toImmutableRoaringBitmap method is almost free, it uses less memory and it produces a much faster bitmap. 
       
      /////////////
      // Code to create a hard copy of MutableRoaringBitmap to an
      // ImmutableRoaringBitmap.
      // Usage of this code is discouraged because it is expensive
      // and it creates a copy that
      // suffers from more performance overhead than the original.
      /////////////
      import org.roaringbitmap.buffer.*;

      //...

      MutableRoaringBitmap rr = ... // some bitmap
      rr.runOptimize(); // can help compression

      // we are going to create an immutable copy of rr
      ByteBuffer outbb = ByteBuffer.allocate(mrb.serializedSizeInBytes());
      mrb.serialize(outbb);
      outbb.flip();
      ImmutableRoaringBitmap irb = new ImmutableRoaringBitmap(outbb);
      Returns:
      a cast of this object

    • trim

      public void trim()
      Recover allocated but unused memory.
      Specified by:
      trim in interface BitmapDataProvider

    • writeExternal

      public void writeExternal(ObjectOutput out) throws IOException
      Specified by:
      writeExternal in interface Externalizable
      Throws:
      IOException

    • xor

      public void xor(ImmutableRoaringBitmap x2)
      In-place bitwise XOR (symmetric difference) operation. The current bitmap is modified.
      Parameters:
      x2 - other bitmap

    • maximumSerializedSize

      public static long maximumSerializedSize(int cardinality, int universe_size)
      Assume that one wants to store "cardinality" integers in [0, universe_size), this function returns an upper bound on the serialized size in bytes. This function is identical to RoaringBitmap.maximumSerializedSize.
      Parameters:
      cardinality - maximal cardinality
      universe_size - maximal value
      Returns:
      upper bound on the serialized size in bytes of the bitmap

    • append

      public void append(char key, MappeableContainer container)
      Description copied from interface: AppendableStorage
      Appends the key and container to the storage, throws if the key is less than the current mark.
      Specified by:
      append in interface AppendableStorage<MappeableContainer>
      Parameters:
      key - the key to append
      container - the data to append