BasicFunctions
Related Doc:
package feature
Days in one week (7) (ISO)
Days in one week (7) (ISO)
The suggested default Iterable that should be emitted when data is missing or unexpected in some way.
The suggested default Iterable that should be emitted when data is missing or unexpected in some way. This is provided to the feature generating functions but not doesn't act as a substitute for the framework's responsibility for handling missing data.
Note: The keys (1st field) present in the Iterable's Tuple2s should be prefixed with "=". This is because (as the name suggests, it is mainly for regression models which have a specific format for generated features.
Hours in a typical day (24) (ISO).
Hours in a typical day (24) (ISO).
Hours in a typical week.
Hours in a typical week.
Milliseconds in a typical day (ISO).
Milliseconds in a typical day (ISO).
Milliseconds in one hour (ISO)
Milliseconds in one hour (ISO)
Milliseconds in one minute (ISO)
Milliseconds in one minute (ISO)
Milliseconds in one second (1000) (ISO)
Milliseconds in one second (1000) (ISO)
Milliseconds in a typical week (ISO).
Milliseconds in a typical week (ISO).
Minutes in a typical day (ISO).
Minutes in a typical day (ISO).
Minutes in one hour (ISO)
Minutes in one hour (ISO)
Minutes in a typical week (ISO).
Minutes in a typical week (ISO).
Seconds in a typical day (ISO).
Seconds in a typical day (ISO).
Seconds in one hour (ISO)
Seconds in one hour (ISO)
Seconds in one minute (60) (ISO)
Seconds in one minute (60) (ISO)
Seconds in a typical week (ISO).
Seconds in a typical week (ISO).
Allows conversion from a Boolean to an option of a Boolean.
Allows conversion from a Boolean to an option of a Boolean. It is used to lift the boolean value into the computational context of an Option. This function has many names in many different contexts (point, return, etc). For more info, see http://en.wikipedia.org/wiki/Monad_(functional_programming)
a Boolean to be converted to an Option[Boolean].
the lifted boolean.
should contain at least two knots and knots should be
(default false)
Given a sparse mapping represented as a function (sparseMapping) from the input domain A to a range of optional values (Option[B]), convert to a dense format of the appropriate container type.
Given a sparse mapping represented as a function (sparseMapping) from the input domain A to a range of optional values (Option[B]), convert to a dense format of the appropriate container type. This is done by composing f = g ∘ sparseMapping, where g is defined as:
⎧ y if x = Some(y)
g(x) = ⎨
⎩ whenMissing otherwise
and mapping f over denseDomain.
An example:
val denseDomain = 3 to 6 val sparseKeys = Array(4, 6) val sparseVals = Iterable(1, 2) val sparseMapping = (sparseKeys zip sparseVals).toMap.get _ // Int => Option[Int] val whenMissing = 0 val result = densify(denseDomain, sparseMapping, whenMissing) val expected = Vector(0, 1, 0, 2) // NOTE: The resulting container type is a Vector because of the type of denseDomain. assert(result.getClass.getCanonicalName == "scala.collection.immutable.Vector") assert(result == expected)
type of the dense domain
type of the dense range
the container type of the input. An attempt is made to make the output container type as close as possible to the input container type. While this is a FilterMonadic, it really only needs to be a functor (because we only care about the map function. Flatmap doesn't matter).
the resulting type implementation.
the domain of dense values provided as the preimage to the sparse mapping specified by the parallel iterables.
a mapping from the input domain to an option of the output domain. Once composed with whenMissing this map all values of the domain an appropriate value in the range of the function.
the resulting value when an item from denseDomain isn't contained in sparseKeys.
a CanBuildFrom object
the dense image of the mapping from the dense domain, using sparseMapping and whenMissing.
Given a sparse mapping represented as a map (sparseFeatures), convert to a dense format of the appropriate container type.
Given a sparse mapping represented as a map (sparseFeatures), convert to a dense format of the appropriate container type. This is done by creating a map based on sparseFeatures with a default value specified by whenMissing and mapping the new map's apply function over denseDomain.
An example:
val denseDomain = 3 to 6 val sparseKeys = Array(4, 6) val sparseVals = Iterable(1, 2) val sparseFeatures = (sparseKeys zip sparseVals).toMap val whenMissing = 0 val result = densify(denseDomain, sparseFeatures, whenMissing) val expected = Vector(0, 1, 0, 2) // NOTE: The resulting container type is a Vector because of the type of denseDomain. assert(result.getClass.getCanonicalName == "scala.collection.immutable.Vector") assert(result == expected)
type of the dense domain
type of the dense range
the container type of the input. An attempt is made to make the output container type as close as possible to the input container type. While this is a FilterMonadic, it really only needs to be a functor (because we only care about the map function. Flatmap doesn't matter).
the resulting type implementation.
the domain of dense values provided as the preimage to the sparse mapping specified by the parallel iterables.
a map from the domain to range
the resulting value when an item from denseDomain isn't contained in sparseKeys.
a CanBuildFrom object
the dense image of the mapping from the dense domain, using sparseMapping and whenMissing.
Given a sparse mapping represented as key-value pairs in parallel iterables, convert to a dense format of the appropriate container type.
Given a sparse mapping represented as key-value pairs in parallel iterables, convert to a dense format of the appropriate container type. This is done by mapping over the denseDomain, determining whether a key exists in sparseKeys. If a key exists, the associated value is substituted; otherwise, substitute with the whenMissing value.
In the event of duplicate keys, for each pair of duplicate keys, the key-value pair associated with the second encountered key will be used.
An example:
val denseDomain = 3 to 6 val sparseKeys = Array(4, 6) val sparseVals = Iterable(1, 2) val sparseValOptions = sparseVals.map(Option.apply) val whenMissing = None val result = densify(denseDomain, sparseKeys, sparseValOptions, whenMissing) val expected = Vector(None, Some(1), None, Some(2)) // NOTE: The resulting container type is a Vector because of the type of denseDomain. assert(result.getClass.getCanonicalName == "scala.collection.immutable.Vector") assert(result == expected)
Notice in the example above that when all of the keys are contained in denseDomain, then when sparsifying the results of densify, we get back the original sparseVals:
assert(result.flatten == sparseVals)
type of the dense domain
type of the dense range
the container type of the input. An attempt is made to make the output container type as close as possible to the input container type. While this is a FilterMonadic, it really only needs to be a functor (because we only care about the map function. Flatmap doesn't matter).
the resulting type implementation.
the domain of dense values provided as the preimage to the sparse mapping specified by the parallel iterables.
the keys in the sparse mapping (NOTE: (sparseKeys(i), sparseVals(i)) represents a key-value pair)
the values in the sparse mapping
the resulting value when an item from denseDomain isn't contained in sparseKeys.
a CanBuildFrom object
the dense image of the mapping from the dense domain, using the mapping created by sparseKeys, sparseVals and whenMissing.
Is lb <= x and x <= ub?
Is lb <= x and x <= ub?
type of inputs
type of ordering used for comparison. This must be a super type of the values being compared.
value to be compared against lower bound and upper bound
lower bound (inclusive)
upper bound (inclusive)
an ordering
true if the lb <= x and x <= ub; otherwise, false.
Is lb <= x and x < ub?
Is lb <= x and x < ub?
type of inputs
type of ordering used for comparison. This must be a super type of the values being compared.
value to be compared against lower bound and upper bound
lower bound (inclusive)
upper bound (exclusive)
an ordering
true if the lb <= x and x < ub; otherwise, false.
Is lb < x and x <= ub?
Is lb < x and x <= ub?
type of inputs
type of ordering used for comparison. This must be a super type of the values being compared.
value to be compared against lower bound and upper bound
lower bound (exclusive)
upper bound (inclusive)
an ordering
true if the lb < x and x <= ub; otherwise, false.
Is lb < x and x < ub?
Is lb < x and x < ub?
type of inputs
type of ordering used for comparison. This must be a super type of the values being compared.
value to be compared against lower bound and upper bound
lower bound (exclusive)
upper bound (exclusive)
an ordering
true if the lb < x and x < ub; otherwise, false.
Indicator for any Enum class.
Indicator for any Enum class.
The enum type
a java enum constant
an indicator.
Provides an intercept function that returns a sequence of one key-value pair:
Provides an intercept function that returns a sequence of one key-value pair:
assert(intercept() == Seq(("", 1.0)))
Like sos2U but no underflows are reported.
Like sos2U but no underflows are reported. Instead the values are first clamped to be in range so in the event value < min, return a tuple representing the min.
number to be sos2 binned
minimum bin value
minimum bin value
bin size
sos2 binned value
See com.eharmony.aloha.feature.Sos2.
number to be sos2 binned
minimum bin value
minimum bin value
bin size
When value < min, an underflow key-value pair is emitted. This controls the key that is emitted.
When value is missing, a separate key-value pair is emitted. This controls the key that is emitted.
sos2 binned value
This is the purest form of sos2 binning that clamps the values in the [min, max] interval and then bins.
This is the purest form of sos2 binning that clamps the values in the [min, max] interval and then bins.
scala> (0 to 10).map(_ / 4.0 - 0.25).map(v => s"$v\t${sos2(v, 0, 2, 1)}").foreach(println) -0.25 List((0,1.0)) 0.0 List((0,1.0)) 0.25 List((0,0.75), (1,0.25)) 0.5 List((0,0.5), (1,0.5)) 0.75 List((0,0.25), (1,0.75)) 1.0 List((1,1.0)) 1.25 List((1,0.75), (2,0.25)) 1.5 List((1,0.5), (2,0.5)) 1.75 List((1,0.25), (2,0.75)) 2.0 List((2,1.0)) 2.25 List((2,1.0))
number to be sos2 binned
minimum bin value
minimum bin value
bin size
sos2 binned value
number to be sos2 binned
minimum bin value
minimum bin value
bin size
When value < min, an underflow key-value pair is emitted. This controls the key that is emitted.
When value is missing (None), a separate key-value pair is emitted. This controls the pair(s) that is/are emitted.
scala> (0 to 10).map(_ / 4.0 - 0.25).map(v => s"$v\t${sos2U(v, 0, 2, 1)}").foreach(println) -0.25 List((=UNDERFLOW,1.0)) 0.0 List((=0,1.0)) 0.25 List((=0,0.75), (=1,0.25)) 0.5 List((=0,0.5), (=1,0.5)) 0.75 List((=0,0.25), (=1,0.75)) 1.0 List((=1,1.0)) 1.25 List((=1,0.75), (=2,0.25)) 1.5 List((=1,0.5), (=2,0.5)) 1.75 List((=1,0.25), (=2,0.75)) 2.0 List((=2,1.0)) 2.25 List((=2,1.0))
number to be sos2 binned
minimum bin value
minimum bin value
bin size
sos2 binned value
Provides extension methods to Options via com.eharmony.aloha.models.reg.OptToKv.
Provides extension methods to Options via com.eharmony.aloha.models.reg.OptToKv.
the type of value
an optional value
an OptToKv instance