com.sleepycat.je.util
Class DbCacheSize

java.lang.Object
  com.sleepycat.je.util.DbCacheSize

public class DbCacheSize
extends Object

Estimating JE in-memory sizes as a function of key and data size is not straightforward for two reasons. There is some fixed overhead for each btree internal node, so tree fanout and degree of node sparseness impacts memory consumption. In addition, JE compresses some of the internal nodes where possible, but compression depends on on-disk layouts. For these reasons, this utility will may output inaccurate values, and most commonly it will output a larger cache size than is actually needed.

DbCacheSize is an aid for estimating cache sizes. To get an estimate of the in-memory footprint for a given database, specify the number of records and record characteristics and DbCacheSize will return a minimum and maximum estimate of the cache size required for holding the database in memory. If the user specifies the record's data size, the utility will return both values for holding just the internal nodes of the btree, and for holding the entire database in cache.

Note that "cache size" is a percentage more than "btree size", to cover general environment resources like log buffers. Each invocation of the utility returns an estimate for a single database in an environment. For an environment with multiple databases, run the utility for each database, add up the btree sizes, and then add 10 percent.

The utility does not yet cover duplicate records and the API is subject to change release to release.

The only required parameters are the number of records and key size. Data size, non-tree cache overhead, btree fanout, and other parameters can also be provided. For example:

You should run DbCacheSize on the same target platform and JVM for which you are sizing the cache, as cache sizes will vary. You should also specify -d32 or -d64 depending on your target as this also figures into the computation.

 $ java DbCacheSize -jar je-X.Y.Z.jar -records 554719 -key 16 -data 100
 Inputs: records=554719 keySize=16 dataSize=100 nodeMax=128 density=80%
 overhead=10%

     Cache Size      Btree Size  Description
 --------------  --------------  -----------
     30,547,440      27,492,696  Minimum, internal nodes only
     41,460,720      37,314,648  Maximum, internal nodes only
    114,371,644     102,934,480  Minimum, internal nodes and leaf nodes
    125,284,924     112,756,432  Maximum, internal nodes and leaf nodes

 Btree levels: 3
 

This says that the minimum cache size to hold only the internal nodes of the btree in cache is approximately 30MB. The maximum size to hold the entire database in cache, both internal nodes and datarecords, is 125Mb. See main(java.lang.String[]) for full parameter descriptions.

Constructor Summary

DbCacheSize(long records, int keySize, int dataSize, int nodeMax, int density, long overhead)
DbCacheSize is an aid for estimating cache sizes.

Method Summary

long	getMaxBtreeSizeInternalNodesOnly()
long	getMaxBtreeSizeWithData()
long	getMaxCacheSizeInternalNodesOnly()
long	getMaxCacheSizeWithData()
long	getMinBtreeSizeInternalNodesOnly()
long	getMinBtreeSizeWithData()
long	getMinCacheSizeInternalNodesOnly()
long	getMinCacheSizeWithData()
int	getNLevels()
static void	main(String[] args) Usage:

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DbCacheSize

public DbCacheSize(long records,
                   int keySize,
                   int dataSize,
                   int nodeMax,
                   int density,
                   long overhead)

DbCacheSize is an aid for estimating cache sizes. To get an estimate of the in-memory footprint for a given database, specify the number of records and record characteristics and DbCacheSize will return a minimum and maximum estimate of the cache size required for holding the database in memory. If the user specifies the record's data size, the utility will return both values for holding just the internal nodes of the btree, and for holding the entire database in cache.

Parameters:

records - Total records (key/data pairs)

keySize - Average key bytes per record

dataSize - Average data bytes per record; if omitted no leaf node sizes are included in the output

nodeMax - Number of entries per Btree node

density - Percentage of node entries occupied

overhead - Overhead of non-Btree objects (log buffers, locks, default: 10% of total cache size)

Method Detail

getMinCacheSizeInternalNodesOnly

public long getMinCacheSizeInternalNodesOnly()

getMaxCacheSizeInternalNodesOnly

public long getMaxCacheSizeInternalNodesOnly()

getMinBtreeSizeInternalNodesOnly

public long getMinBtreeSizeInternalNodesOnly()

getMaxBtreeSizeInternalNodesOnly

public long getMaxBtreeSizeInternalNodesOnly()

getMinCacheSizeWithData

public long getMinCacheSizeWithData()

getMaxCacheSizeWithData

public long getMaxCacheSizeWithData()

getMinBtreeSizeWithData

public long getMinBtreeSizeWithData()

getMaxBtreeSizeWithData

public long getMaxBtreeSizeWithData()

getNLevels

public int getNLevels()

main

public static void main(String[] args)

Usage:

 java { com.sleepycat.je.util.DbCacheSize |
         -jar je-.jar DbCacheSize }
  -records 
    # Total records (key/data pairs); required
  -key 
    # Average key bytes per record; required
  [-data ]
    # Average data bytes per record; if omitted no leaf
    # node sizes are included in the output
  [-nodemax ]
    # Number of entries per Btree node; default: 128
  [-density ]
    # Percentage of node entries occupied; default: 80
  [-overhead ]
    # Overhead of non-Btree objects (log buffers, locks,
    # etc); default: 10% of total cache size
  [-measure ]
    # An empty directory used to write a database to find
    # the actual cache size; default: do not measure;
    # without -data, measures internal nodes only
  [-measurerandom
    # With -measure insert randomly generated keys;
    # default: insert sequential keys