E
- element type of the sets, created by this builderpublic final class ChronicleSetBuilder<E> extends Object implements ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
ChronicleSetBuilder
manages the whole set of ChronicleSet
configurations, could
be used as a classic builder and/or factory.
ChronicleMapBuilder
is mutable, see a note in ChronicleHashBuilder
interface documentation.
ChronicleSet
,
ChronicleMapBuilder
Modifier and Type | Method and Description |
---|---|
ChronicleSetBuilder<E> |
actualEntriesPerSegment(long actualEntriesPerSegment) |
ChronicleSetBuilder<E> |
actualSegments(int actualSegments) |
ChronicleSetBuilder<E> |
bytesMarshallerFactory(BytesMarshallerFactory bytesMarshallerFactory)
Configures a
BytesMarshallerFactory to be used with BytesMarshallableSerializer , which is a default ObjectSerializer ,
to serialize/deserialize data to/from off-heap memory in hash containers, created by this
builder. |
ChronicleSetBuilder<E> |
clone()
Clones this builder.
|
ChronicleSetBuilder<E> |
constantKeySizeBySample(E sampleKey)
Configures the constant number of bytes, taken by serialized form of keys, put into hash
containers, created by this builder.
|
ChronicleSet<E> |
create()
Creates a new hash container, storing it's data in off-heap memory, not mapped to any file.
|
ChronicleSet<E> |
createPersistedTo(File file)
Opens a hash container residing the specified file, or creates a new one if the file not yet
exists and maps its off-heap memory to the file.
|
ChronicleSet<E> |
createStatelessClient(InetSocketAddress serverAddress) |
ChronicleSetBuilder<E> |
entries(long entries)
Configures the maximum number of "entry size chunks", which
could be taken by the maximum number of entries, inserted into the hash containers, created
by this builder.
|
ChronicleSetBuilder<E> |
entrySize(int entrySize)
Configures the size in bytes of allocation unit of hash container instances, created by this
builder.
|
boolean |
equals(Object o) |
ChronicleSetBuilder<E> |
errorListener(ChronicleHashErrorListener errorListener) |
int |
hashCode() |
ChronicleSetBuilder<E> |
immutableKeys()
Specifies that key objects, queried with the hash containers, created by this builder, are
inherently immutable.
|
ChronicleHashInstanceConfig<ChronicleSet<E>> |
instance() |
ChronicleSetBuilder<E> |
keyDeserializationFactory(ObjectFactory<E> keyDeserializationFactory)
Configures factory which is used to create a new key instance, if key class is either
Byteable , BytesMarshallable or Externalizable subclass, or key type is
eligible for data value generation, or configured custom key reader implements DeserializationFactoryConfigurableBytesReader , in maps, created by this builder. |
ChronicleSetBuilder<E> |
keyMarshaller(BytesMarshaller<? super E> keyMarshaller)
Configures the
BytesMarshaller used to serialize/deserialize keys to/from off-heap
memory in hash containers, created by this builder. |
ChronicleSetBuilder<E> |
keyMarshallers(BytesWriter<E> keyWriter,
BytesReader<E> keyReader)
Configures the marshallers, used to serialize/deserialize keys to/from off-heap memory in
hash containers, created by this builder.
|
ChronicleSetBuilder<E> |
keySize(int keySize)
Configures the optimal number of bytes, taken by serialized form of keys, put into hash
containers, created by this builder.
|
ChronicleSetBuilder<E> |
keySizeMarshaller(SizeMarshaller keySizeMarshaller)
Configures the marshaller used to serialize actual key sizes to off-heap memory in hash
containers, created by this builder.
|
ChronicleSetBuilder<E> |
lockTimeOut(long lockTimeOut,
TimeUnit unit)
Configures timeout of locking on segments of hash
containers, created by this builder, when performing any queries, as well as bulk operations
like iteration.
|
ChronicleSetBuilder<E> |
maxEntryOversizeFactor(int maxEntryOversizeFactor)
Configures how much the actual entry size is allowed to be larger than configured or derived
entry size.
|
ChronicleSetBuilder<E> |
metaDataBytes(int metaDataBytes) |
ChronicleSetBuilder<E> |
minSegments(int minSegments)
Set minimum number of segments in hash containers, constructed by this builder.
|
ChronicleSetBuilder<E> |
objectSerializer(ObjectSerializer objectSerializer)
Configures the serializer used to serialize/deserialize data to/from off-heap memory, when
specified class doesn't implement a specific serialization interface like
Externalizable or BytesMarshallable (for example, if data is loosely typed and just
Object is specified as the data class), or nullable data, and if custom marshaller is
not configured, in hash containers, created by
this builder. |
static <K> ChronicleSetBuilder<K> |
of(Class<K> keyClass) |
ChronicleSetBuilder<E> |
replication(byte identifier) |
ChronicleSetBuilder<E> |
replication(byte identifier,
TcpTransportAndNetworkConfig tcpTransportAndNetwork)
Shortcut for
replication(SimpleReplication.builder() .tcpTransportAndNetwork(tcpTransportAndNetwork).createWithId(identifier)) . |
ChronicleSetBuilder<E> |
replication(SingleChronicleHashReplication replication)
Configures replication of the hash containers, created by this builder.
|
StatelessClientConfig<ChronicleSet<E>> |
statelessClient(InetSocketAddress remoteAddress) |
ChronicleSetBuilder<E> |
timeProvider(TimeProvider timeProvider)
Configures a time provider, used by hash containers, created by this builder, for needs of
replication consensus protocol (conflicting data updates resolution).
|
String |
toString() |
public static <K> ChronicleSetBuilder<K> of(Class<K> keyClass)
public ChronicleSetBuilder<E> clone()
ChronicleHashBuilder
ChronicleHashBuilder
s are mutable and changed on each configuration method call. Original
and cloned builders are independent.clone
in interface ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
clone
in class Object
public ChronicleSetBuilder<E> actualSegments(int actualSegments)
actualSegments
in interface ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
public ChronicleSetBuilder<E> minSegments(int minSegments)
ChronicleHashBuilder
ConcurrentHashMap
.minSegments
in interface ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
minSegments
- the minimum number of segments in containers, constructed by this builderpublic ChronicleSetBuilder<E> actualEntriesPerSegment(long actualEntriesPerSegment)
actualEntriesPerSegment
in interface ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
public ChronicleSetBuilder<E> keySize(int keySize)
ChronicleHashBuilder.constantKeySizeBySample(Object)
method instead of this one.
If key size varies moderately, specify the size higher than average, but lower than the maximum possible, to minimize average memory overuse. If key size varies in a wide range, it's better to use entry size in "chunk" mode and configure it directly.
If key is a boxed primitive type or Byteable
subclass, i. e. if key size is known
statically, it is automatically accounted and shouldn't be specified by user.
Example: if keys in your set(s) are English words in String
form, keys size 10
(a bit more than average English word length) would be a good choice:
ChronicleSet<String> uniqueWords = ChronicleSetBuilder.of(String.class)
.entries(50000)
.keySize(10)
.create();
(Note that 10 is chosen as key size in bytes despite strings in Java are UTF-16 encoded
(and each character takes 2 bytes on-heap), because default off-heap String
encoding
is UTF-8 in ChronicleSet
.)
keySize
in interface ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
keySize
- number of bytes, taken by serialized form of keysconstantKeySizeBySample(Object)
,
entrySize(int)
public ChronicleSetBuilder<E> constantKeySizeBySample(E sampleKey)
sampleKey
, all
keys should take the same number of bytes in serialized form, as this sample object.
If keys are of boxed primitive type or Byteable
subclass, i. e. if key size is
known statically, it is automatically accounted and this method shouldn't be called.
If key size varies, method ChronicleHashBuilder.keySize(int)
or ChronicleHashBuilder.entrySize(int)
should be
called instead of this one.
For example, if your keys are Git commit hashes:
Set<byte[]> gitCommitsOfInterest = ChronicleSetBuilder.of(byte[].class)
.constantKeySizeBySample(new byte[20])
.create();
constantKeySizeBySample
in interface ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
sampleKey
- the sample keykeySize(int)
public ChronicleSetBuilder<E> entrySize(int entrySize)
ChronicleMap
and ChronicleSet
store their data off-heap, so it is required
to serialize key (and values, in ChronicleMap
case) (unless they are direct Byteable
instances). Serialized key bytes (+ serialized value bytes, in ChronicleMap
case) + some metadata bytes comprise "entry space", which ChronicleMap
or ChronicleSet
should allocate. So entry size is a minimum allocation portion in the
hash containers, created by this builder. E. g. if entry size is 100, the created container
could only allocate 100, 200, 300... bytes for an entry. If say 150 bytes of entry space are
required by the entry, 200 bytes will be allocated, 150 used and 50 wasted. To minimize
memory overuse and improve speed, you should pay decent attention to this configuration.
There are three major patterns of this configuration usage:
ChronicleMap
case) sizes are constant. Configure them via ChronicleHashBuilder.constantKeySizeBySample(Object)
and ChronicleMapBuilder.constantValueSizeBySample(Object)
methods, and you will experience no memory waste at all.ChronicleMap
case) varies moderately. Specify them using corresponding methods, or
specify entry size directly by calling this method, by sizes somewhere between average and
maximum possible. The idea is to have most (90% or more) entries to fit a single "entry size"
with moderate memory waste (10-20% on average), rest 10% or less of entries should take 2
"entry sizes", thus with ~50% memory overuse.ChronicleMap
case) varies in a wide range. Then it's best to use entry size configuration in
chunk mode. Specify entry size so that most entries should take from 5 to several
dozens of "chunks". With this approach, average memory waste should be very low.
However, remember that
ChronicleHashBuilder.maxEntryOversizeFactor(int)
, of max
64 chunks. IllegalArgumentException
is thrown on attempt to insert too large entry,
compared to the configured or computed entry size.Example: if values in your ChronicleMap
are adjacency lists of some social graph,
where nodes are represented as long
ids, and adjacency lists are serialized in
efficient manner, for example as long[]
arrays. Typical number of connections is
100-300, maximum is 3000. In this case entry size of
50 * (8 bytes for each id) = 400 bytes would be a good choice:
Map<Long, long[]> socialGraph = ChronicleMapOnHeapUpdatableBuilder
.of(Long.class, long[].class)
// given that graph should have of 1 billion nodes, and 150 average adjacency list size
// => values takes 3 chuncks on average
.entries(1_000_000_000L * (150 / 50))
.entrySize(50 * 8)
.create();
It is minimum possible (because 3000 friends / 50 friends = 60 is close to 64 "max chunks by
single entry" limit, and ensures moderate average memory overuse (not more than 20%). In fully default case you can expect entry size to be about 120-130 bytes. But it is strongly recommended always to configure key size, if they couldn't be derived statically.
If entry size is not configured explicitly by calling this method, it is computed based on meta data bytes, plus key size, plus a few bytes required by implementations.
entrySize
in interface ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
entrySize
- the "chunk size" in bytesChronicleHashBuilder.entries(long)
,
ChronicleHashBuilder.maxEntryOversizeFactor(int)
public ChronicleSetBuilder<E> maxEntryOversizeFactor(int maxEntryOversizeFactor)
ChronicleHashBuilder
maxEntryOversizeFactor
in interface ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
maxEntryOversizeFactor
- number of times the actual entry size could oversize the
configured "entry size" (chunk size, actually)public ChronicleSetBuilder<E> entries(long entries)
ChronicleHashBuilder
IllegalStateException
might be
thrown, because currently ChronicleMap
and ChronicleSet
doesn't support
resizing.
ChronicleMap
case) is constant, this number
is equal to the maximum number of entries (because each entry takes exactly one "entry size"
memory unit).ChronicleMap
case) size varies
moderately, you should pass to this method the maximum number of entries + 5-25%, depending
on your data properties and configured key/value/entry sizes.ChronicleHashBuilder.entrySize(int)
method.You shouldn't put additional margin over the number, computed according the rules above.
This bad practice was popularized by HashMap.HashMap(int)
and HashSet.HashSet(int)
constructors, which accept "capacity", that should be multiplied by
"load factor" to obtain actual maximum expected number of entries. ChronicleMap
and
ChronicleSet
don't have a notion of load factor.
Default value is 2^20 (~ 1 million).
entries
in interface ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
entries
- maximum size of the created maps, in memory allocation units, so-called "entry
size"ChronicleHashBuilder.entrySize(int)
public ChronicleSetBuilder<E> lockTimeOut(long lockTimeOut, TimeUnit unit)
ChronicleHashBuilder
ChronicleHashErrorListener.onLockTimeout(long)
is
called, and then thread tries to obtain the segment lock one more time, and so in a loop,
until thread is interrupted. However, you can configure error listener to throw an exception on the first
(or n-th) lock acquisition fail.
Default lock time out is 2 seconds.
lockTimeOut
in interface ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
lockTimeOut
- new lock timeout for segments of containers created by this builder, in
the given time unitsunit
- time unit of the given lock timeoutpublic ChronicleSetBuilder<E> errorListener(ChronicleHashErrorListener errorListener)
errorListener
in interface ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
public ChronicleSetBuilder<E> metaDataBytes(int metaDataBytes)
metaDataBytes
in interface ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
public ChronicleSetBuilder<E> timeProvider(TimeProvider timeProvider)
ChronicleHashBuilder
Default time provider is TimeProvider.SYSTEM
.
timeProvider
in interface ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
timeProvider
- a new time provider for replication needsChronicleHashBuilder.replication(SingleChronicleHashReplication)
public ChronicleSetBuilder<E> bytesMarshallerFactory(BytesMarshallerFactory bytesMarshallerFactory)
ChronicleHashBuilder
BytesMarshallerFactory
to be used with BytesMarshallableSerializer
, which is a default ObjectSerializer
,
to serialize/deserialize data to/from off-heap memory in hash containers, created by this
builder.
Default BytesMarshallerFactory
is an instance of VanillaBytesMarshallerFactory
. This is a convenience configuration method, it has no effect
on the resulting hash containers, if custom data
marshallers are configured, data types extends one of specific serialization interfaces,
recognized by this builder (e. g. Externalizable
or BytesMarshallable
), or
ObjectSerializer
is configured.
bytesMarshallerFactory
in interface ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
bytesMarshallerFactory
- the marshaller factory to be used with the default ObjectSerializer
, i. e. BytesMarshallableSerializer
ChronicleHashBuilder.objectSerializer(ObjectSerializer)
public ChronicleSetBuilder<E> objectSerializer(ObjectSerializer objectSerializer)
Externalizable
or BytesMarshallable
(for example, if data is loosely typed and just
Object
is specified as the data class), or nullable data, and if custom marshaller is
not configured, in hash containers, created by
this builder. Please read ObjectSerializer
docs for more info and available options.
Default serializer is BytesMarshallableSerializer
, configured with the specified
or default BytesMarshallerFactory
.
Example:
Set<Key> set = ChronicleSetBuilder.of(Key.class)
.entries(1_000_000)
.keySize(100)
// this class hasn't implemented yet, just for example
.objectSerializer(new KryoObjectSerializer())
.create();
objectSerializer
in interface ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
objectSerializer
- the serializer used to serialize loosely typed or nullable data if
custom marshaller is not configuredChronicleHashBuilder.bytesMarshallerFactory(BytesMarshallerFactory)
,
ChronicleHashBuilder.keyMarshaller(BytesMarshaller)
public ChronicleSetBuilder<E> keyMarshaller(@NotNull BytesMarshaller<? super E> keyMarshaller)
ChronicleHashBuilder
BytesMarshaller
used to serialize/deserialize keys to/from off-heap
memory in hash containers, created by this builder. See the
section about serialization in ChronicleMap manual for more information.keyMarshaller
in interface ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
keyMarshaller
- the marshaller used to serialize keysChronicleHashBuilder.keyMarshallers(BytesWriter, BytesReader)
,
ChronicleHashBuilder.objectSerializer(ObjectSerializer)
public ChronicleSetBuilder<E> keyMarshallers(@NotNull BytesWriter<E> keyWriter, @NotNull BytesReader<E> keyReader)
ChronicleHashBuilder
Configuring marshalling this way results to a little bit more compact in-memory layout of
the map, comparing to a single interface configuration: ChronicleHashBuilder.keyMarshaller(BytesMarshaller)
.
Passing BytesInterop
(which is a subinterface of BytesWriter
) as the first
argument is supported, and even more advantageous from performance perspective.
keyMarshallers
in interface ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
keyWriter
- the new key object → Bytes
writer (interop) strategykeyReader
- the new Bytes
→ key object reader strategyChronicleHashBuilder.keyMarshaller(BytesMarshaller)
public ChronicleSetBuilder<E> keySizeMarshaller(@NotNull SizeMarshaller keySizeMarshaller)
ChronicleHashBuilder
Default key size marshaller is so-called stop bit
encoding marshalling. If constant key size is
configured, or defaulted if the key type is always constant and ChronicleHashBuilder
implementation knows about it, this configuration takes no effect, because a special SizeMarshaller
implementation, which doesn't actually do any marshalling, and just returns
the known constant size on SizeMarshaller.readSize(Bytes)
calls, is used instead of
any SizeMarshaller
configured using this method.
keySizeMarshaller
in interface ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
keySizeMarshaller
- the new marshaller, used to serialize actual key sizes to off-heap
memorypublic ChronicleSetBuilder<E> keyDeserializationFactory(@NotNull ObjectFactory<E> keyDeserializationFactory)
Byteable
, BytesMarshallable
or Externalizable
subclass, or key type is
eligible for data value generation, or configured custom key reader implements DeserializationFactoryConfigurableBytesReader
, in maps, created by this builder.
Default key deserialization factory is NewInstanceObjectFactory
, which creates a
new key instance using Class.newInstance()
default constructor. You could provide an
AllocateInstanceObjectFactory
, which uses Unsafe.allocateInstance(Class)
(you
might want to do this for better performance or if you don't want to initialize fields), or a
factory which calls a key class constructor with some arguments, or a factory which
internally delegates to instance pool or ThreadLocal
, to reduce allocations.
Actually this is just a convenience method supporting key marshaller configurations, made
initially during of(Class)
call. Because if you configure own custom key marshaller, this method doesn't
take any effect on the maps constructed by this builder.
keyDeserializationFactory
in interface ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
keyDeserializationFactory
- the key factory used to produce instances to deserialize
data inof(Class)
public ChronicleSetBuilder<E> immutableKeys()
ChronicleHashBuilder
ChronicleMap
or ChronicleSet
are not required
to be immutable, as in ordinary Map
or Set
implementations, because they are
serialized off-heap. However, ChronicleMap
and ChronicleSet
implementations
can benefit from the knowledge that keys are not mutated between queries.
By default, ChronicleHashBuilder
s detects immutability automatically only for very
few standard JDK types (for example, for String
), it is not recommended to rely on
ChronicleHashBuilder
to be smart enough about this.
immutableKeys
in interface ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
public ChronicleSetBuilder<E> replication(SingleChronicleHashReplication replication)
ChronicleHashBuilder
By default, hash containers, created by this builder doesn't replicate their data.
This method call overrides all previous replication configurations of this builder, made
either by this method or ChronicleHashBuilder.replication(byte, TcpTransportAndNetworkConfig)
shortcut
method.
replication
in interface ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
replication
- the replication configChronicleHashInstanceConfig.replicated(SingleChronicleHashReplication)
,
ChronicleHashBuilder.replication(byte, TcpTransportAndNetworkConfig)
public ChronicleSetBuilder<E> replication(byte identifier, TcpTransportAndNetworkConfig tcpTransportAndNetwork)
ChronicleHashBuilder
replication(SimpleReplication.builder() .tcpTransportAndNetwork(tcpTransportAndNetwork).createWithId(identifier))
.replication
in interface ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
identifier
- the network-wide identifier of the containers, created by this
buildertcpTransportAndNetwork
- configuration of tcp connection and networkChronicleHashBuilder.replication(SingleChronicleHashReplication)
,
ChronicleHashInstanceConfig.replicated(byte, TcpTransportAndNetworkConfig)
public ChronicleSetBuilder<E> replication(byte identifier)
replication
in interface ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
public StatelessClientConfig<ChronicleSet<E>> statelessClient(InetSocketAddress remoteAddress)
statelessClient
in interface ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
public ChronicleSet<E> createStatelessClient(InetSocketAddress serverAddress) throws IOException
createStatelessClient
in interface ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
IOException
public ChronicleHashInstanceConfig<ChronicleSet<E>> instance()
instance
in interface ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
public ChronicleSet<E> create()
ChronicleHashBuilder
ChronicleHash.close()
called on the returned container, or after the container
object is collected during GC, or on JVM shutdown the off-heap memory used by the returned
container is freed.
This method is a shortcut for instance().create()
.
create
in interface ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
ChronicleHashBuilder.createPersistedTo(File)
,
ChronicleHashBuilder.instance()
public ChronicleSet<E> createPersistedTo(File file) throws IOException
ChronicleHashBuilder
Multiple containers could give access to the same data simultaneously, either inside a single JVM or across processes. Access is synchronized correctly across all instances, i. e. hash container mapping the data from the first JVM isn't able to modify the data, concurrently accessed from the second JVM by another hash container instance, mapping the same data.
On container's close()
the data isn't removed, it remains on
disk and available to be opened again (given the same file name) or during different JVM
run.
This method is shortcut for instance().persistedTo(file).create()
.
createPersistedTo
in interface ChronicleHashBuilder<E,ChronicleSet<E>,ChronicleSetBuilder<E>>
file
- the file with existing hash container or a desired location of a new off-heap
persisted hash containerIOException
- if any IO error, related to off-heap memory allocation or file mapping,
or establishing replication connections, occursChronicleHash.file()
,
ChronicleHash.close()
,
ChronicleHashBuilder.create()
,
ChronicleHashInstanceConfig.persistedTo(File)
Copyright © 2014. All rights reserved.