Package com.yahoo.vespa.hadoop.pig

Class VespaStorage

java.lang.Object
org.apache.pig.StoreFunc
com.yahoo.vespa.hadoop.pig.VespaStorage
All Implemented Interfaces:
org.apache.pig.StoreFuncInterface
public class VespaStorage extends org.apache.pig.StoreFunc
A small Pig UDF wrapper around the Vespa http client for feeding data into a Vespa endpoint.
Author:
lesters

  • Constructor Details

    • VespaStorage

      public VespaStorage()

    • VespaStorage

      public VespaStorage(String... params)

  • Method Details

    • getOutputFormat

      public org.apache.hadoop.mapreduce.OutputFormat getOutputFormat() throws IOException
      Specified by:
      getOutputFormat in interface org.apache.pig.StoreFuncInterface
      Specified by:
      getOutputFormat in class org.apache.pig.StoreFunc
      Throws:
      IOException

    • setStoreLocation

      public void setStoreLocation(String endpoint, org.apache.hadoop.mapreduce.Job job) throws IOException
      Specified by:
      setStoreLocation in interface org.apache.pig.StoreFuncInterface
      Specified by:
      setStoreLocation in class org.apache.pig.StoreFunc
      Throws:
      IOException

    • prepareToWrite

      public void prepareToWrite(org.apache.hadoop.mapreduce.RecordWriter recordWriter) throws IOException
      Specified by:
      prepareToWrite in interface org.apache.pig.StoreFuncInterface
      Specified by:
      prepareToWrite in class org.apache.pig.StoreFunc
      Throws:
      IOException

    • putNext

      public void putNext(org.apache.pig.data.Tuple tuple) throws IOException
      Specified by:
      putNext in interface org.apache.pig.StoreFuncInterface
      Specified by:
      putNext in class org.apache.pig.StoreFunc
      Throws:
      IOException

    • checkSchema

      public void checkSchema(org.apache.pig.ResourceSchema resourceSchema) throws IOException
      Specified by:
      checkSchema in interface org.apache.pig.StoreFuncInterface
      Overrides:
      checkSchema in class org.apache.pig.StoreFunc
      Throws:
      IOException

    • relToAbsPathForStoreLocation

      public String relToAbsPathForStoreLocation(String endpoint, org.apache.hadoop.fs.Path path) throws IOException
      Specified by:
      relToAbsPathForStoreLocation in interface org.apache.pig.StoreFuncInterface
      Overrides:
      relToAbsPathForStoreLocation in class org.apache.pig.StoreFunc
      Throws:
      IOException

    • setStoreFuncUDFContextSignature

      public void setStoreFuncUDFContextSignature(String s)
      Specified by:
      setStoreFuncUDFContextSignature in interface org.apache.pig.StoreFuncInterface
      Overrides:
      setStoreFuncUDFContextSignature in class org.apache.pig.StoreFunc

    • cleanupOnFailure

      public void cleanupOnFailure(String s, org.apache.hadoop.mapreduce.Job job) throws IOException
      Specified by:
      cleanupOnFailure in interface org.apache.pig.StoreFuncInterface
      Overrides:
      cleanupOnFailure in class org.apache.pig.StoreFunc
      Throws:
      IOException

    • cleanupOnSuccess

      public void cleanupOnSuccess(String s, org.apache.hadoop.mapreduce.Job job) throws IOException
      Specified by:
      cleanupOnSuccess in interface org.apache.pig.StoreFuncInterface
      Overrides:
      cleanupOnSuccess in class org.apache.pig.StoreFunc
      Throws:
      IOException

    • base64Serialize

      public static String base64Serialize(Object o) throws IOException
      Throws:
      IOException

    • base64Deserialize

      public static <T extends Serializable> T base64Deserialize(String s) throws IOException
      Throws:
      IOException