Package

com.johnsnowlabs.nlp.util

io

Permalink

package io

Visibility
  1. Public
  2. All

Type Members

  1. case class ExternalResource(path: String, readAs: Format, options: Map[String, String]) extends WritableAnnotatorComponent with Product with Serializable

    Permalink

    This represents an external source which contains information into how an external resource shall be read by Spark-NLP's Resource Helper.

    This represents an external source which contains information into how an external resource shall be read by Spark-NLP's Resource Helper.

    • ReadAs.TEXT will configure the file to be read locally as text
    • ReadAs.BINARY will configure the file to be read locally as binary
    • ReadAs.SPARK will configure the file to be read by Spark. "format" will need to be defined in options.

    Example

    ExternalResource(
      "src/test/resources/regex-matcher/rules.txt",
      ReadAs.TEXT,
      Map("delimiter" -> ",")
    )
    
    ExternalResource(
      "src/test/resources/regex-matcher/rules.txt",
      ReadAs.SPARK,
      Map("format" -> "text", "delimiter" -> ",")
    )
    path

    Path to the resource

    readAs

    How to interpret the resource. Possible values are ReadAs.SPARK, ReadAs.TEXT, ReadAs.BINARY

    options

    Options for Spark. Option format needs to be set if readAs is set to ReadAs.SPARK

Value Members

  1. object ExternalResource extends Serializable

    Permalink
  2. object OutputHelper

    Permalink
  3. object ReadAs extends Enumeration

    Permalink
  4. object ResourceHelper

    Permalink

    Helper one-place for IO management.

    Helper one-place for IO management. Streams, source and external input should be handled from here

Ungrouped