gate.corpora

Class SerialCorpusImpl

java.lang.Object

gate.util.AbstractFeatureBearer

gate.creole.AbstractResource

gate.creole.AbstractLanguageResource

gate.corpora.SerialCorpusImpl

All Implemented Interfaces:

Corpus, CustomDuplication, IndexedCorpus, CreoleListener, DatastoreListener, LanguageResource, Resource, SimpleCorpus, FeatureBearer, NameBearer, Serializable, Iterable<Document>, Collection<Document>, EventListener, List<Document>

@CreoleResource(name="GATE Serial Corpus",
                isPrivate=true,
                comment="GATE persistent corpus (serialisation)",
                icon="corpus",
                helpURL="http://gate.ac.uk/userguide/sec:developer:datastores")
public class SerialCorpusImpl
extends AbstractLanguageResource
implements Corpus, CreoleListener, DatastoreListener, IndexedCorpus, CustomDuplication

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected List<Document>	addedDocs
protected List<Document>	changedDocs
protected Vector<CorpusListener>	corpusListeners
protected List<DocumentData>	docDataList
protected List<Document>	documents
protected IndexManager	indexManager
protected List<String>	removedDocIDs

Fields inherited from class gate.creole.AbstractLanguageResource

dataStore, lrPersistentId

Fields inherited from class gate.creole.AbstractResource

name

Fields inherited from class gate.util.AbstractFeatureBearer

features

Fields inherited from interface gate.SimpleCorpus

CORPUS_DOCLIST_PARAMETER_NAME, CORPUS_NAME_PARAMETER_NAME

Constructor Summary

Constructors

Modifier	Constructor and Description
	SerialCorpusImpl()
protected	SerialCorpusImpl(Corpus tCorpus) Constructor to create a SerialCorpus from a transient one.

Method Summary

Modifier and Type	Method and Description
boolean	add(Document o)
void	add(int index, Document o)
boolean	addAll(Collection<? extends Document> c)
boolean	addAll(int index, Collection<? extends Document> c)
void	addCorpusListener(CorpusListener l) Registers a new CorpusListener with this corpus.
void	cleanup() Every LR that is a CreoleListener (and other Listeners too) must override this method and make sure it removes itself from the objects which it has been listening to.
void	clear()
boolean	contains(Object o)
boolean	containsAll(Collection<?> c)
void	datastoreClosed(CreoleEvent e) Called when a DataStore has been closed
void	datastoreCreated(CreoleEvent e) Called when a DataStore has been created
void	datastoreOpened(CreoleEvent e) Called when a DataStore has been opened
Resource	duplicate(Factory.DuplicationContext ctx) SerialCorpusImpl does not support duplication.
boolean	equals(Object o)
int	findDocument(Document doc)
protected void	fireDocumentAdded(CorpusEvent e)
protected void	fireDocumentRemoved(CorpusEvent e)
Document	get(int index)
String	getDocumentClassType(int index)
List<String>	getDocumentClassTypes() Gets the persistent IDs of the documents in this corpus.
String	getDocumentName(int index) Gets the name of a document in this corpus.
List<String>	getDocumentNames() Gets the names of the documents in this corpus.
Object	getDocumentPersistentID(int index) Gets the persistent ID of a document in this corpus.
List<Object>	getDocumentPersistentIDs() Gets the persistent IDs of the documents in this corpus.
IndexDefinition	getIndexDefinition()
IndexManager	getIndexManager()
IndexStatistics	getIndexStatistics()
Object	getTransientSource()
int	hashCode()
int	indexOf(Object o)
Resource	init() Initialise this resource, and return it.
boolean	isDocumentLoaded(int index) This method returns true when the document is already loaded in memory
boolean	isEmpty()
boolean	isPersistentDocument(int index) This method returns true when the document is already stored on disk i.e., is not transient
Iterator<Document>	iterator()
int	lastIndexOf(Object o)
ListIterator<Document>	listIterator()
ListIterator<Document>	listIterator(int index)
void	populate(URL directory, FileFilter filter, String encoding, boolean recurseDirectories) Fills this corpus with documents created from files in a directory.
void	populate(URL directory, FileFilter filter, String encoding, String mimeType, boolean recurseDirectories) Fills this corpus with documents created from files in a directory.
long	populate(URL singleConcatenatedFile, String documentRootElement, String encoding, int numberOfFilesToExtract, String documentNamePrefix, String mimeType, boolean includeRootElement) Fills the provided corpus with documents extracted from the provided single concatenated file.
Document	remove(int index)
boolean	remove(Object o)
boolean	removeAll(Collection<?> c)
void	removeCorpusListener(CorpusListener l) Removes one of the listeners registered with this corpus.
void	resourceAdopted(DatastoreEvent evt) Called by a datastore when a new resource has been adopted
void	resourceDeleted(DatastoreEvent evt) Called by a datastore when a resource has been deleted
void	resourceLoaded(CreoleEvent e) Called when a new Resource has been loaded into the system
void	resourceRenamed(Resource resource, String oldName, String newName) Called when the creole register has renamed a resource.1
void	resourceUnloaded(CreoleEvent e) Called when a Resource has been removed from the system
void	resourceWritten(DatastoreEvent evt) Called by a datastore when a resource has been wrote into the datastore
boolean	retainAll(Collection<?> c)
Document	set(int index, Document element)
void	setDataStore(DataStore dataStore) Set the data store that this LR lives in.
void	setDocumentPersistentID(int index, Object persID) This method should only be used by the Serial Datastore to set
void	setIndexDefinition(IndexDefinition definition) Sets the definition to this corpus.
void	setTransientSource(Object source)
int	size()
List<Document>	subList(int fromIndex, int toIndex) persistent Corpus does not support this method as all the documents might no be in memory
Object[]	toArray()
<T> T[]	toArray(T[] a)
String	toString()
void	unloadDocument(Document doc) Unloads a document from memory, calling sync() first, to store the changes.
void	unloadDocument(Document doc, boolean sync) Unloads a document from memory
void	unloadDocument(int index) Unloads the document from memory, calling sync() first, to store the changes.
void	unloadDocument(int index, boolean sync) Unloads a document from memory.

Methods inherited from class gate.creole.AbstractLanguageResource

getDataStore, getLRPersistenceId, getParent, isModified, setLRPersistenceId, setParent, sync

Methods inherited from class gate.creole.AbstractResource

checkParameterValues, flushBeanInfoCache, forgetBeanInfo, getBeanInfo, getInitParameterValues, getInitParameterValues, getName, getParameterValue, getParameterValue, getParameterValues, removeResourceListeners, setName, setParameterValue, setParameterValue, setParameterValues, setParameterValues, setResourceListeners

Methods inherited from class gate.util.AbstractFeatureBearer

getFeatures, setFeatures

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface gate.LanguageResource

getDataStore, getLRPersistenceId, getParent, isModified, setLRPersistenceId, setParent, sync

Methods inherited from interface gate.Resource

getParameterValue, setParameterValue, setParameterValues

Methods inherited from interface gate.util.FeatureBearer

getFeatures, setFeatures

Methods inherited from interface java.util.List

replaceAll, sort, spliterator

Methods inherited from interface java.util.Collection

parallelStream, removeIf, stream

Methods inherited from interface java.lang.Iterable

forEach

Field Detail

corpusListeners

protected transient Vector<CorpusListener> corpusListeners

docDataList

protected List<DocumentData> docDataList

documents

protected transient List<Document> documents

indexManager

protected transient IndexManager indexManager

addedDocs

protected transient List<Document> addedDocs

removedDocIDs

protected transient List<String> removedDocIDs

changedDocs

protected transient List<Document> changedDocs

Constructor Detail

SerialCorpusImpl

public SerialCorpusImpl()

SerialCorpusImpl

protected SerialCorpusImpl(Corpus tCorpus)

Constructor to create a SerialCorpus from a transient one. This is called by adopt() to store the transient corpus and re-route the methods calls to it, until the corpus is sync-ed on disk. After that, the transientCorpus will always be null, so the new functionality will be used instead.

Method Detail

getDocumentNames

public List<String> getDocumentNames()

Gets the names of the documents in this corpus.

Specified by:

getDocumentNames in interface SimpleCorpus

Returns:

a List of Strings representing the names of the documents in this corpus.

getDocumentPersistentIDs

public List<Object> getDocumentPersistentIDs()

Gets the persistent IDs of the documents in this corpus.

Returns:

a List of Objects representing the persistent IDs of the documents in this corpus.

getDocumentClassTypes

public List<String> getDocumentClassTypes()

Gets the persistent IDs of the documents in this corpus.

Returns:

a List of Objects representing the persistent IDs of the documents in this corpus.

setDocumentPersistentID

public void setDocumentPersistentID(int index,
                                    Object persID)

This method should only be used by the Serial Datastore to set

getDocumentName

public String getDocumentName(int index)

Gets the name of a document in this corpus.

Specified by:

getDocumentName in interface SimpleCorpus

Parameters:

index - the index of the document

Returns:

a String value representing the name of the document at index in this corpus.

getDocumentPersistentID

public Object getDocumentPersistentID(int index)

Gets the persistent ID of a document in this corpus.

Parameters:

index - the index of the document

Returns:

a value representing the persistent ID of the document at index in this corpus.

getDocumentClassType

public String getDocumentClassType(int index)

unloadDocument

public void unloadDocument(int index,
                           boolean sync)

Unloads a document from memory.

Parameters:

index - the index of the document to be unloaded.

sync - should the document be sync'ed (i.e. saved) before unloading.

unloadDocument

public void unloadDocument(Document doc,
                           boolean sync)

Unloads a document from memory

Parameters:

doc - the document to be unloaded

sync - should the document be sync'ed (i.e. saved) before unloading.

unloadDocument

public void unloadDocument(Document doc)

Unloads a document from memory, calling sync() first, to store the changes.

Specified by:

unloadDocument in interface Corpus

Parameters:

doc - the document to be unloaded.

unloadDocument

public void unloadDocument(int index)

Unloads the document from memory, calling sync() first, to store the changes.

Parameters:

index - the index of the document to be unloaded.

isDocumentLoaded

public boolean isDocumentLoaded(int index)

This method returns true when the document is already loaded in memory

Specified by:

isDocumentLoaded in interface Corpus

isPersistentDocument

public boolean isPersistentDocument(int index)

This method returns true when the document is already stored on disk i.e., is not transient

cleanup

public void cleanup()

Every LR that is a CreoleListener (and other Listeners too) must override this method and make sure it removes itself from the objects which it has been listening to. Otherwise, the object will not be released from memory (memory leak!).

Specified by:

cleanup in interface Resource

Overrides:

cleanup in class AbstractLanguageResource

populate

public void populate(URL directory,
                     FileFilter filter,
                     String encoding,
                     boolean recurseDirectories)
              throws IOException,
                     ResourceInstantiationException

Fills this corpus with documents created from files in a directory.

Specified by:

populate in interface SimpleCorpus

Parameters:

filter - the file filter used to select files from the target directory. If the filter is null all the files will be accepted.

directory - the directory from which the files will be picked. This parameter is an URL for uniformity. It needs to be a URL of type file otherwise an InvalidArgumentException will be thrown. An implementation for this method is provided as a static method at CorpusImpl.populate(Corpus, URL, FileFilter, String, boolean) .

encoding - the encoding to be used for reading the documents

recurseDirectories - should the directory be parsed recursively?. If true all the files from the provided directory and all its children directories (on as many levels as necessary) will be picked if accepted by the filter otherwise the children directories will be ignored.

Throws:

IOException

ResourceInstantiationException

populate

public void populate(URL directory,
                     FileFilter filter,
                     String encoding,
                     String mimeType,
                     boolean recurseDirectories)
              throws IOException,
                     ResourceInstantiationException

Fills this corpus with documents created from files in a directory.

Specified by:

populate in interface SimpleCorpus

Parameters:

filter - the file filter used to select files from the target directory. If the filter is null all the files will be accepted.

directory - the directory from which the files will be picked. This parameter is an URL for uniformity. It needs to be a URL of type file otherwise an InvalidArgumentException will be thrown. An implementation for this method is provided as a static method at CorpusImpl.populate(Corpus, URL, FileFilter, String, boolean) .

encoding - the encoding to be used for reading the documents

recurseDirectories - should the directory be parsed recursively?. If true all the files from the provided directory and all its children directories (on as many levels as necessary) will be picked if accepted by the filter otherwise the children directories will be ignored.

mimeType - the mime type to be used when loading documents. If null, then the mime type will be automatically determined.

Throws:

IOException

ResourceInstantiationException

populate

public long populate(URL singleConcatenatedFile,
                     String documentRootElement,
                     String encoding,
                     int numberOfFilesToExtract,
                     String documentNamePrefix,
                     String mimeType,
                     boolean includeRootElement)
              throws IOException,
                     ResourceInstantiationException

Fills the provided corpus with documents extracted from the provided single concatenated file.

Specified by:

populate in interface SimpleCorpus

Parameters:

singleConcatenatedFile - the single concatenated file.

documentRootElement - content between the start and end of this element is considered for documents.

encoding - the encoding of the trec file.

numberOfFilesToExtract - indicates the number of files to extract from the trecweb file.

documentNamePrefix - the prefix to use for document names when creating from

mimeType - the mime type which determines how the document is handled

Returns:

total length of populated documents in the corpus in number of bytes

Throws:

IOException

ResourceInstantiationException

removeCorpusListener

public void removeCorpusListener(CorpusListener l)

Description copied from interface: Corpus

Removes one of the listeners registered with this corpus.

Specified by:

removeCorpusListener in interface Corpus

Parameters:

l - the listener to be removed.

addCorpusListener

public void addCorpusListener(CorpusListener l)

Description copied from interface: Corpus

Registers a new CorpusListener with this corpus.

Specified by:

addCorpusListener in interface Corpus

Parameters:

l - the listener to be added.

fireDocumentAdded

protected void fireDocumentAdded(CorpusEvent e)

fireDocumentRemoved

protected void fireDocumentRemoved(CorpusEvent e)

resourceLoaded

public void resourceLoaded(CreoleEvent e)

Description copied from interface: CreoleListener

Called when a new Resource has been loaded into the system

Specified by:

resourceLoaded in interface CreoleListener

resourceRenamed

public void resourceRenamed(Resource resource,
                            String oldName,
                            String newName)

Description copied from interface: CreoleListener

Called when the creole register has renamed a resource.1

Specified by:

resourceRenamed in interface CreoleListener

resourceUnloaded

public void resourceUnloaded(CreoleEvent e)

Description copied from interface: CreoleListener

Called when a Resource has been removed from the system

Specified by:

resourceUnloaded in interface CreoleListener

datastoreOpened

public void datastoreOpened(CreoleEvent e)

Description copied from interface: CreoleListener

Called when a DataStore has been opened

Specified by:

datastoreOpened in interface CreoleListener

datastoreCreated

public void datastoreCreated(CreoleEvent e)

Description copied from interface: CreoleListener

Called when a DataStore has been created

Specified by:

datastoreCreated in interface CreoleListener

datastoreClosed

public void datastoreClosed(CreoleEvent e)

Description copied from interface: CreoleListener

Called when a DataStore has been closed

Specified by:

datastoreClosed in interface CreoleListener

resourceAdopted

public void resourceAdopted(DatastoreEvent evt)

Called by a datastore when a new resource has been adopted

Specified by:

resourceAdopted in interface DatastoreListener

resourceDeleted

public void resourceDeleted(DatastoreEvent evt)

Called by a datastore when a resource has been deleted

Specified by:

resourceDeleted in interface DatastoreListener

resourceWritten

public void resourceWritten(DatastoreEvent evt)

Called by a datastore when a resource has been wrote into the datastore

Specified by:

resourceWritten in interface DatastoreListener

size

public int size()

Specified by:

size in interface Collection<Document>

Specified by:

size in interface List<Document>

isEmpty

public boolean isEmpty()

Specified by:

isEmpty in interface Collection<Document>

Specified by:

isEmpty in interface List<Document>

contains

public boolean contains(Object o)

Specified by:

contains in interface Collection<Document>

Specified by:

contains in interface List<Document>

iterator

public Iterator<Document> iterator()

Specified by:

iterator in interface Iterable<Document>

Specified by:

iterator in interface Collection<Document>

Specified by:

iterator in interface List<Document>

toString

public String toString()

Overrides:

toString in class AbstractResource

toArray

public Object[] toArray()

Specified by:

toArray in interface Collection<Document>

Specified by:

toArray in interface List<Document>

toArray

public <T> T[] toArray(T[] a)

Specified by:

toArray in interface Collection<Document>

Specified by:

toArray in interface List<Document>

add

public boolean add(Document o)

Specified by:

add in interface Collection<Document>

Specified by:

add in interface List<Document>

remove

public boolean remove(Object o)

Specified by:

remove in interface Collection<Document>

Specified by:

remove in interface List<Document>

findDocument

public int findDocument(Document doc)

containsAll

public boolean containsAll(Collection<?> c)

Specified by:

containsAll in interface Collection<Document>

Specified by:

containsAll in interface List<Document>

addAll

public boolean addAll(Collection<? extends Document> c)

Specified by:

addAll in interface Collection<Document>

Specified by:

addAll in interface List<Document>

addAll

public boolean addAll(int index,
                      Collection<? extends Document> c)

Specified by:

addAll in interface List<Document>

removeAll

public boolean removeAll(Collection<?> c)

Specified by:

removeAll in interface Collection<Document>

Specified by:

removeAll in interface List<Document>

retainAll

public boolean retainAll(Collection<?> c)

Specified by:

retainAll in interface Collection<Document>

Specified by:

retainAll in interface List<Document>

clear

public void clear()

Specified by:

clear in interface Collection<Document>

Specified by:

clear in interface List<Document>

equals

public boolean equals(Object o)

Specified by:

equals in interface Collection<Document>

Specified by:

equals in interface List<Document>

Overrides:

equals in class Object

hashCode

public int hashCode()

Specified by:

hashCode in interface Collection<Document>

Specified by:

hashCode in interface List<Document>

Overrides:

hashCode in class Object

get

public Document get(int index)

Specified by:

get in interface List<Document>

set

public Document set(int index,
                    Document element)

Specified by:

set in interface List<Document>

add

public void add(int index,
                Document o)

Specified by:

add in interface List<Document>

remove

public Document remove(int index)

Specified by:

remove in interface List<Document>

indexOf

public int indexOf(Object o)

Specified by:

indexOf in interface List<Document>

lastIndexOf

public int lastIndexOf(Object o)

Specified by:

lastIndexOf in interface List<Document>

listIterator

public ListIterator<Document> listIterator()

Specified by:

listIterator in interface List<Document>

listIterator

public ListIterator<Document> listIterator(int index)

Specified by:

listIterator in interface List<Document>

subList

public List<Document> subList(int fromIndex,
                              int toIndex)

persistent Corpus does not support this method as all the documents might no be in memory

Specified by:

subList in interface List<Document>

setDataStore

public void setDataStore(DataStore dataStore)
                  throws PersistenceException

Description copied from class: AbstractLanguageResource

Set the data store that this LR lives in.

Specified by:

setDataStore in interface LanguageResource

Overrides:

setDataStore in class AbstractLanguageResource

Throws:

PersistenceException

setTransientSource

public void setTransientSource(Object source)

getTransientSource

public Object getTransientSource()

init

public Resource init()
              throws ResourceInstantiationException

Description copied from class: AbstractResource

Initialise this resource, and return it.

Specified by:

init in interface Resource

Overrides:

init in class AbstractResource

Throws:

ResourceInstantiationException

setIndexDefinition

public void setIndexDefinition(IndexDefinition definition)

Description copied from interface: IndexedCorpus

Sets the definition to this corpus.

Specified by:

setIndexDefinition in interface IndexedCorpus

Parameters:

definition - of index for this corpus

getIndexDefinition

public IndexDefinition getIndexDefinition()

Specified by:

getIndexDefinition in interface IndexedCorpus

Returns:

IndexDefinition definition of index for this corpus.

getIndexManager

public IndexManager getIndexManager()

Specified by:

getIndexManager in interface IndexedCorpus

Returns:

IndexManager manager object for this corpus. It creates after seting of IndexDefinition by indexType property.

getIndexStatistics

public IndexStatistics getIndexStatistics()

Specified by:

getIndexStatistics in interface IndexedCorpus

Returns:

IndexStatistics statistics for this index.

duplicate

public Resource duplicate(Factory.DuplicationContext ctx)
                   throws ResourceInstantiationException

SerialCorpusImpl does not support duplication.

Specified by:

duplicate in interface CustomDuplication

Parameters:

ctx - the current duplication context. If an implementation of this method needs to duplicate any other resources as part of the custom duplication process it should pass this context back to the two-argument form of Factory.duplicate rather than using the single-argument form.

Returns:

an independent copy of this resource.

Throws:

ResourceInstantiationException