Class AsyncRdfParserHadoop
java.lang.Object
net.sansa_stack.spark.io.rdf.loader.AsyncRdfParserHadoop
Async parsing RDF on a single node using hadoop
-
Nested Class Summary
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionstatic void
parse
(org.apache.hadoop.fs.Path file, org.apache.jena.riot.RDFFormat rdfFormat, org.apache.hadoop.conf.Configuration conf, org.apache.jena.riot.system.StreamRDF sink) static <T> void
parseRaw
(org.apache.hadoop.fs.Path inputFile, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.mapreduce.InputFormat<?, T> inputFormat, ExecutorService executorService, org.apache.jena.riot.system.StreamRDF sink, BiConsumer<T, org.apache.jena.riot.system.StreamRDF> sendRecordToStreamRDF) The sink must be started beforehand!
-
Constructor Details
-
AsyncRdfParserHadoop
public AsyncRdfParserHadoop()
-
-
Method Details
-
parse
public static void parse(org.apache.hadoop.fs.Path file, org.apache.jena.riot.RDFFormat rdfFormat, org.apache.hadoop.conf.Configuration conf, org.apache.jena.riot.system.StreamRDF sink) throws Exception - Parameters:
file
-rdfFormat
- The language of the file. If it is null then probing will be performed.conf
-sink
-- Throws:
Exception
-
parseRaw
public static <T> void parseRaw(org.apache.hadoop.fs.Path inputFile, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.mapreduce.InputFormat<?, T> inputFormat, ExecutorService executorService, org.apache.jena.riot.system.StreamRDF sink, BiConsumer<T, throws IOException, InterruptedException, ExecutionExceptionorg.apache.jena.riot.system.StreamRDF> sendRecordToStreamRDF) The sink must be started beforehand!- Type Parameters:
T
-- Parameters:
inputFile
-conf
-inputFormat
-executorService
- The Executorservice must be closed externally.sink
-sendRecordToStreamRDF
-- Throws:
IOException
InterruptedException
ExecutionException
-