utils

Type Members

1. trait CliConfig[T] extends AnyRef

   

2. trait CometJacksonModule extends Module with JacksonModule

   

   This module handles some specific type serializers in a central way (so we don't need to pepper the model code with annotations)

3. class CometObjectMapper extends ObjectMapper with ScalaObjectMapper

   

4. class FileLock extends StrictLogging

   

   HDFS does not have a file locking mechanism.

   HDFS does not have a file locking mechanism. We implement it here in the following way - A file is locked when it's modification time is less than 5000ms than the current time - If the modification time is older tahn the current time of more than 5000ms, it is considered unlocked. - The owner of a lock spawn a thread that update the modification every 5s to keep the lock - The process willing to get the lock check every 5s second for the modification time of the file to gain the lock. - When a process gain the lock, it deletes the file first and tries to create a new one, this make sure that of two process gaining the lock, only one will be able to recreate it after deletion.

   To gain the lock, call tryLock, to release it, call release(). If you are going to use the lock and release it within the same call stack frame, please consider calling tryExclusively() (as in exclusive(delay) { action }) instead.

5. trait Formatter extends AnyRef

   

6. trait JobBase extends StrictLogging

   

   All Spark Job extend this trait.

   All Spark Job extend this trait. Build Spark session using spark variables from application.conf.

7. trait JobResult extends AnyRef

   

8. class SinkUtils extends StrictLogging

   

9. trait SparkJob extends JobBase

   

10. case class SparkJobResult(dataframe: Option[DataFrame]) extends JobResult with Product with Serializable

    

11. final class TextSubstitutionEngine extends AnyRef

    

    This class provides a basic text substitution engine, suitable for performing basic variable substitution in a way that is more reliable and faster than applying a sequence of String#replace operations.

    This class provides a basic text substitution engine, suitable for performing basic variable substitution in a way that is more reliable and faster than applying a sequence of String#replace operations.

    The default start and end variable separators are \_\_ and \_\_, respectively, but can be configured to be anything suitable.

12. case class Version(version: String) extends Comparable[Version] with Product with Serializable