NOTE: The predictClass, predict and predictImage will call the relevant methods of
object Predictor. Why we do this? Because every these methods uses the ClassTag T. If we do
these jobs in the methods of classPredictor, when we do mapPartition, Spark will find all
used values and do serialization. The T is the argument of constructor, the serialization will
package the whole Predictor class, which contains themodel. It will send a duplicate model
to the workers. So we should move these methods to object Predictor.
Predictor for distributed data
NOTE: The
predictClass,predictandpredictImagewill call the relevant methods of objectPredictor. Why we do this? Because every these methods uses the ClassTagT. If we do these jobs in the methods of classPredictor, when we domapPartition, Spark will find all used values and do serialization. TheTis the argument of constructor, the serialization will package the wholePredictorclass, which contains themodel. It will send a duplicate model to the workers. So we should move these methods to objectPredictor.