org.allenai.nlpstack.parse.poly.fsm
SculptureTrainingVectorSource
Related Doc:
package fsm
the data source for the training sculptures
the transition system factory to use (for generating states)
a trained cost function factory to adapt (optional)
a trained cost function factory to adapt (optional)
This generates a list of labeled feature vectors from a gold parse tree (for training).
This generates a list of labeled feature vectors from a gold parse tree (for training). The gold parse tree is reduced to its representation as a list of 2*n transitions, then a TrainingVector is produced for each transition (in order).
Note that this function is implemented using tail-recursion.
the sculpture to generate feature vectors from
a list of training vectors
Returns the set of tasks associated with the training vectors in this source.
Returns the set of tasks associated with the training vectors in this source.
In a perhaps over-careful attempt to avoid having all the non-uniqued tasks being stored in memory simultaneously, this was originally implemented as:
format: OFF lazy val tasks: Iterable[ClassificationTask] = taskHelper(Set(), getVectorIterator) tailrec private def taskHelper( resultSoFar: Set[ClassificationTask], vectorIter: Iterator[FSMTrainingVector] ): Set[ClassificationTask] = {
if (!vectorIter.hasNext) { resultSoFar } else { taskHelper(resultSoFar + vectorIter.next().task, vectorIter) } } format: ON
the data source for the training sculptures
the transition system factory to use (for generating states)
A SculptureTrainingVectorSource reduces a sculpture to a set of feature vectors for classifier training.
Essentially, we derive the transition states that lead to the gold sculpture. Each of these states becomes a feature vector, labeled with the transition executed from that state in the gold sculpture.
One of the constructor arguments is a TaskIdentifer. This will dispatch the feature vectors to train different classifiers. For instance, if taskIdentifier(state) != taskIdentifier(state2), then their respective feature vectors (i.e. feature(state) and feature(state2)) will be used to train different classifiers.
the data source for the training sculptures
the transition system factory to use (for generating states)
a trained cost function factory to adapt (optional)