MapColumnOps
Related Doc:
package MapColumns
Returns value for the given key in value.
Returns value for the given key in value.
Creates a new row for each element in the given map column.
Creates a new row for each element in the given map column.
ORIGINAL SPARK DORIC
+----------------+ +---+-----+ +------+
|col | |key|value| |col |
+----------------+ +---+-----+ +------+
|[a -> b, c -> d]| |a |b | |{a, b}|
|[] | |c |d | |{c, d}|
|null | +---+-----+ +------+
+----------------+WARNING: Unlike spark, doric returns a struct
Creates a new row for each element in the given map column.
Creates a new row for each element in the given map column.
ORIGINAL SPARK DORIC +---+----------------+ +---+----+-----+ +---+------+ |ix |col | |ix |key |value| |ix |col | +---+----------------+ +---+----+-----+ +---+------+ |1 |{a -> b, c -> d}| |1 |a |b | |1 |{a, b}| |2 |{} | |1 |c |d | |1 |{c, d}| |3 |null | |2 |null|null | |2 |null | +---+----------------+ |3 |null|null | |3 |null | +---+----+-----+ +---+------+
WARNING: Unlike spark, doric returns a struct
,Unlike explode, if the array is null or empty then null is produced.
Returns the value if exist of the key
Returns the value if exist of the key
the required key
a DoricColumn of the type of value, can be null if the key doesn't exist.
Returns an unordered array containing the keys of the map.
Returns an unordered array containing the keys of the map.
the DoricColumn of the Array of keys
Transforms the original value to a literal.
Transforms the original value to a literal.
a literal with the same type.
Creates a new row for each element with position in the given map column.
Creates a new row for each element with position in the given map column.
ORIGINAL SPARK DORIC +----------------+ +---+---+-----+ +---------+ |col | |pos|key|value| |col | +----------------+ +---+---+-----+ +---------+ |[a -> b, c -> d]| |1 |a |b | |{1, a, b}| |[] | |2 |c |d | |{2, c, d}| |null | +---+---+-----+ +---------+ +----------------+
WARNING: Unlike spark, doric returns a struct
,Uses the default column name pos for position, and key and value for elements in the map.
Creates a new row for each element with position in the given map column.
Creates a new row for each element with position in the given map column. Unlike posexplode, if the map is null or empty then the row (null, null) is produced.
ORIGINAL SPARK DORIC +----------------+ +---+----+-----+ +---------+ |col | |pos|key |value| |col | +----------------+ +---+----+-----+ +---------+ |[a -> b, c -> d]| |1 |a |b | |{1, a, b}| |[] | |2 |c |d | |{2, c, d}| |null | |2 |null|null | |null | +----------------+ |3 |null|null | |null | +---+----+-----+ +---+-----+
WARNING: Unlike spark, doric returns a struct
,Uses the default column name pos for position, and key and value for elements in the map.
Returns length of map.
Returns length of map.
The function returns null for null input if spark.sql.legacy.sizeOfNull is set to false or spark.sql.ansi.enabled is set to true. Otherwise, the function returns -1 for null input. With the default settings, the function returns -1 for null input.
DORIC EXCLUSIVE! Map to array conversion
Converts a column containing a StructType into a JSON string with the specified schema.
Converts a column containing a StructType into a JSON string with the specified schema.
java.lang.IllegalArgumentException in the case of an unsupported type.
scaladoc link (issue #135)
org.apache.spark.sql.functions.to_json(e:org\.apache\.spark\.sql\.Column,options:scala\.collection\.immutable\.Map\[java\.lang\.String,java\.lang\.String\]):* org.apache.spark.sql.functions.to_csv
Returns an unordered array containing the values of the map.
Returns an unordered array containing the values of the map.
the DoricColumn of the Array of values
Extension methods for Map Columns