Storage

Value Members

1. final def !=(arg0: Any): Boolean

   

   Definition Classes

   AnyRef → Any

2. final def ##(): Int

   

   Definition Classes

   AnyRef → Any

3. final def ==(arg0: Any): Boolean

   

   Definition Classes

   AnyRef → Any

4. final def asInstanceOf[T0]: T0

   

   Definition Classes

   Any

5. def clone(): AnyRef

   

   Attributes

   protected[java.lang]

   Definition Classes

   AnyRef

   Annotations

   @throws( ... )

6. def createFileTable(sparkSession: SparkSession, basePath: Path, tableInfo: AuditTableInfo): Try[AuditTable]

   

   Creates file table with default configurations, region ids will Longs with zeros added to the left to make it 20 chars.

   Creates file table with default configurations, region ids will Longs with zeros added to the left to make it 20 chars.

   basePath

   parent folder which contains folders with table names

   tableInfo

   table metadata

   Exceptions thrown

   StorageException : Storage exceptions when: 1) primary keys are not specified; 2) the folder already exists

7. def createFops(sparkSession: SparkSession, basePath: Path): FileStorageOps

   

   Creates File Operations object that is a bridge between the process actions and actual storage and handle write to temp with move to permanent operations.

   Creates File Operations object that is a bridge between the process actions and actual storage and handle write to temp with move to permanent operations.

   basePath

   parent folder which contains folders with table names, .tmp and .Trash folders will be underneath.

8. final def eq(arg0: AnyRef): Boolean

   

   Definition Classes

   AnyRef

9. def equals(arg0: Any): Boolean

   

   Definition Classes

   AnyRef → Any

10. def finalize(): Unit

    

    Attributes

    protected[java.lang]

    Definition Classes

    AnyRef

    Annotations

    @throws( classOf[java.lang.Throwable] )

11. final def getClass(): Class[_]

    

    Definition Classes

    AnyRef → Any

12. def hashCode(): Int

    

    Definition Classes

    AnyRef → Any

13. final def isInstanceOf[T0]: Boolean

    

    Definition Classes

    Any

14. final def ne(arg0: AnyRef): Boolean

    

    Definition Classes

    AnyRef

15. final def notify(): Unit

    

    Definition Classes

    AnyRef

16. final def notifyAll(): Unit

    

    Definition Classes

    AnyRef

17. def openFileTables(sparkSession: SparkSession, basePath: Path, tableNames: Seq[String], includeHot: Boolean = true): (Map[String, Try[AuditTable]], Seq[String])

    

    Opens multiple tables, reads their metadata and reads the state of the table regions (ids, count of records per region, values of region latest timestamp).

    Opens multiple tables, reads their metadata and reads the state of the table regions (ids, count of records per region, values of region latest timestamp).

    You can specify to only read the cold (compacted) state of the table. This is useful to avoid consistency issues during development on tables into which data is currently flowing. This is useful to avoid consistency issues during development on tables into which data is currently flowing.

    basePath

    parent folder which contains folders with table names

    tableNames

    list of tables to open

    includeHot

    include freshly appended data that has not been compacted yet. Useful for using production data in development.

    returns

    (Map[TABLE NAME, AuditTable], Seq[MISSING TABLES]) - audit table objects that exist and of table names that were not found under the basePath

18. final def synchronized[T0](arg0: ⇒ T0): T0

    

    Definition Classes

    AnyRef

19. def toString(): String

    

    Definition Classes

    AnyRef → Any

20. final def wait(): Unit

    

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

21. final def wait(arg0: Long, arg1: Int): Unit

    

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

22. final def wait(arg0: Long): Unit

    

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )