Class HugeGraph

  • All Implemented Interfaces:
    BatchNodeIterable, Degrees, Graph, IdMapping, NodeIterator, NodeMapping, NodePropertyContainer, RelationshipAccess, RelationshipIterator, RelationshipPredicate, RelationshipProperties

    public class HugeGraph
    extends java.lang.Object
    implements Graph
    Huge Graph contains two array like data structures.

    The adjacency data is stored in a ByteArray, which is a byte[] addressable by longs indices and capable of storing about 2^46 (~ 70k bn) bytes – or 64 TiB. The bytes are stored in byte[] pages of 32 KiB size.

    The data is in the format:

    degree ~ targetId1 ~ targetId2 ~ targetIdn
    The degree is stored as a fill-sized 4 byte long int (the neo kernel api returns an int for Nodes.countAll(org.neo4j.internal.kernel.api.NodeCursor, org.neo4j.internal.kernel.api.CursorFactory)). Every target ID is first sorted, then delta encoded, and finally written as variable-length vlongs. The delta encoding does not write the actual value but only the difference to the previous value, which plays very nice with the vlong encoding.

    The seconds data structure is a LongArray, which is a long[] addressable by longs and capable of storing about 2^43 (~9k bn) longs – or 64 TiB worth of 64 bit longs. The data is the offset address into the aforementioned adjacency array, the index is the respective source node id.

    To traverse all nodes, first access to offset from the LongArray, then read 4 bytes into the degree from the ByteArray, starting from the offset, then read degree vlongs as targetId.

    Reading the degree from the offset position not only does not require the offset array to be sorted but also allows the adjacency array to be sparse. This fact is used during the import – each thread pre-allocates a local chunk of some pages (512 KiB) and gives access to this data during import. Synchronization between threads only has to happen when a new chunk has to be pre-allocated. This is similar to what most garbage collectors do with TLAB allocations.

    See Also:
    more abount vlong, more abount TLAB allocation
    • Method Detail

      • nodeCount

        public long nodeCount()
        Description copied from interface: IdMapping
        Number of mapped nodeIds.
        Specified by:
        nodeCount in interface IdMapping
      • idMap

        public IdMap idMap()
      • relationshipCount

        public long relationshipCount()
        Specified by:
        relationshipCount in interface Graph
        Returns:
        returns the total number of relationships in the graph.
      • forEachNode

        public void forEachNode​(java.util.function.LongPredicate consumer)
        Description copied from interface: NodeIterator
        Iterate over each nodeId
        Specified by:
        forEachNode in interface NodeIterator
      • relationshipProperty

        public double relationshipProperty​(long sourceNodeId,
                                           long targetNodeId)
        Description copied from interface: RelationshipProperties
        Returns the property value for a relationship defined by its source and target nodes.
        Specified by:
        relationshipProperty in interface RelationshipProperties
      • relationshipProperty

        public double relationshipProperty​(long sourceId,
                                           long targetId,
                                           double fallbackValue)
        Description copied from interface: RelationshipProperties
        get value of property on relationship between source and target node id
        Specified by:
        relationshipProperty in interface RelationshipProperties
        Parameters:
        sourceId - source node
        targetId - target node
        fallbackValue - value to use if relationship has no property value
        Returns:
        the property value
      • nodeProperties

        public NodeProperties nodeProperties​(java.lang.String propertyKey)
        Description copied from interface: NodePropertyContainer
        Return the property values for a property key
        Specified by:
        nodeProperties in interface NodePropertyContainer
        Parameters:
        propertyKey - the node property key
        Returns:
        the values associated with that key
      • forEachRelationship

        public void forEachRelationship​(long nodeId,
                                        RelationshipConsumer consumer)
        Description copied from interface: RelationshipIterator
        Calls the given consumer function for every relationship of a given node.
        Specified by:
        forEachRelationship in interface RelationshipIterator
        Parameters:
        nodeId - id of the node for which to iterate relationships
        consumer - relationship consumer function
      • forEachRelationship

        public void forEachRelationship​(long nodeId,
                                        double fallbackValue,
                                        RelationshipWithPropertyConsumer consumer)
        Description copied from interface: RelationshipIterator
        Calls the given consumer function for every relationship of a given node. If the graph was loaded with a relationship property, the property value of the relationship will be passed into the consumer. Otherwise the given fallback value will be used.
        Specified by:
        forEachRelationship in interface RelationshipIterator
        Parameters:
        nodeId - id of the node for which to iterate relationships
        fallbackValue - value used as relationship property if no properties were loaded
        consumer - relationship consumer function
      • degree

        public int degree​(long node)
        Specified by:
        degree in interface Degrees
      • toMappedNodeId

        public long toMappedNodeId​(long nodeId)
        Description copied from interface: IdMapping
        Map original nodeId to inner nodeId
        Specified by:
        toMappedNodeId in interface IdMapping
      • toOriginalNodeId

        public long toOriginalNodeId​(long nodeId)
        Description copied from interface: IdMapping
        Map inner nodeId back to original nodeId
        Specified by:
        toOriginalNodeId in interface IdMapping
      • contains

        public boolean contains​(long nodeId)
        Description copied from interface: IdMapping
        Returns true iff the nodeId is mapped, otherwise false.
        Specified by:
        contains in interface IdMapping
      • exists

        public boolean exists​(long sourceNodeId,
                              long targetNodeId)
        O(n) !
        Specified by:
        exists in interface RelationshipPredicate
      • canRelease

        public void canRelease​(boolean canRelease)
        Specified by:
        canRelease in interface Graph
      • releaseTopology

        public void releaseTopology()
        Description copied from interface: Graph
        Release only the topological data associated with that graph.
        Specified by:
        releaseTopology in interface Graph
      • releaseProperties

        public void releaseProperties()
        Description copied from interface: Graph
        Release only the properties associated with that graph.
        Specified by:
        releaseProperties in interface Graph
      • isUndirected

        public boolean isUndirected()
        Specified by:
        isUndirected in interface Graph