FileSetProperties (CDAP API 6.5.1 API)

java.lang.Object
- io.cdap.cdap.api.dataset.lib.FileSetProperties

Direct Known Subclasses:

PartitionedFileSetProperties
```
public class FileSetProperties
extends Object
```
Helper to build properties for files datasets.

Nested Class Summary

Nested Classes
Modifier and Type Class and Description

static class FileSetProperties.Builder
A Builder to construct properties for FileSet datasets.

Nested Classes
Modifier and Type	Class and Description
`static class`	`FileSetProperties.Builder` A Builder to construct properties for FileSet datasets.

Field Summary

Fields
Modifier and Type	Field and Description
`static String`	`BASE_PATH` The base path of the dataset.
`static String`	`DATA_EXTERNAL` Whether the files in the dataset are "external", that is, managed by an external process.
`static String`	`DATA_POSSESS_EXISTING` If true, the file set will use an existing directory as its base path and an existing Explore table, and it will take possession of them.
`static String`	`DATA_USE_EXISTING` If true, the file set will use an existing directory as its base path and an existing Explore table.
`static String`	`INPUT_FORMAT` The name of the input format class.
`static String`	`INPUT_PROPERTIES_PREFIX` Prefix for additional properties for the input format.
`static String`	`OUTPUT_FORMAT` The name of the output format class.
`static String`	`OUTPUT_PROPERTIES_PREFIX` Prefix for additional properties for the output format.
`static String`	`PROPERTY_ENABLE_EXPLORE_ON_CREATE` Whether this dataset should be enabled for explore.
`static String`	`PROPERTY_EXPLORE_FORMAT` The format to use for the explore table.
`static String`	`PROPERTY_EXPLORE_INPUT_FORMAT` The input format to use for the Hive table.
`static String`	`PROPERTY_EXPLORE_OUTPUT_FORMAT` The output format to use for the Hive table.
`static String`	`PROPERTY_EXPLORE_SCHEMA` The schema to use for the explore table.
`static String`	`PROPERTY_EXPLORE_SERDE` The serde to use for the Hive table.
`static String`	`PROPERTY_EXPLORE_TABLE_PROPERTY_PREFIX` Prefix used to store additional table properties for Hive.
`static String`	`PROPERTY_FILES_GROUP` The group name that the permission are assigned to.
`static String`	`PROPERTY_FILES_PERMISSIONS` The permissions for the dataset.

Constructor Summary

Constructors
Constructor and Description

FileSetProperties()

Constructors
Constructor and Description
`FileSetProperties()`

Method Summary

All Methods Static Methods Concrete Methods
Modifier and Type	Method and Description
`static FileSetProperties.Builder`	`builder()`
`static String`	`getBasePath(Map<String,String> properties)`
`static String`	`getExploreFormat(Map<String,String> properties)`
`static Map<String,String>`	`getExploreFormatProperties(Map<String,String> properties)`
`static String`	`getExploreInputFormat(Map<String,String> properties)`
`static String`	`getExploreOutputFormat(Map<String,String> properties)`
`static String`	`getExploreSchema(Map<String,String> properties)`
`static String`	`getFileGroup(Map<String,String> properties)`
`static String`	`getFilePermissions(Map<String,String> properties)`
`static String`	`getInputFormat(Map<String,String> properties)`
`static Map<String,String>`	`getInputProperties(Map<String,String> properties)`
`static String`	`getOutputFormat(Map<String,String> properties)`
`static Map<String,String>`	`getOutputProperties(Map<String,String> properties)`
`static String`	`getSerDe(Map<String,String> properties)`
`static Map<String,String>`	`getTableProperties(Map<String,String> properties)`
`static boolean`	`isDataExternal(Map<String,String> properties)`
`static boolean`	`isExploreEnabled(Map<String,String> properties)`
`static boolean`	`isPossessExisting(Map<String,String> properties)`
`static boolean`	`isUseExisting(Map<String,String> properties)`
`static Map<String,String>`	`propertiesWithPrefix(Map<String,String> properties, String prefix)`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - BASE_PATH
```
public static final String BASE_PATH
```
    The base path of the dataset.
    
    See Also:
    
    Constant Field Values
  - DATA_EXTERNAL
```
public static final String DATA_EXTERNAL
```
    Whether the files in the dataset are "external", that is, managed by an external process.
    
    See Also:
    
    Constant Field Values
  - DATA_USE_EXISTING
```
public static final String DATA_USE_EXISTING
```
    If true, the file set will use an existing directory as its base path and an existing Explore table. When the file set is deleted or truncated, the directory and its contents as well as the Explore table will not be deleted.
    
    See Also:
    
    Constant Field Values
  - DATA_POSSESS_EXISTING
```
public static final String DATA_POSSESS_EXISTING
```
    If true, the file set will use an existing directory as its base path and an existing Explore table, and it will take possession of them. When the file set is deleted or truncated, the directory and its contents as well as the Explore table will be deleted, just as if they had been created by this file set.
    
    See Also:
    
    Constant Field Values
  - INPUT_FORMAT
```
public static final String INPUT_FORMAT
```
    The name of the input format class.
    
    See Also:
    
    Constant Field Values
  - OUTPUT_FORMAT
```
public static final String OUTPUT_FORMAT
```
    The name of the output format class.
    
    See Also:
    
    Constant Field Values
  - INPUT_PROPERTIES_PREFIX
```
public static final String INPUT_PROPERTIES_PREFIX
```
    Prefix for additional properties for the input format. They are added to the Hadoop configuration, with the prefix stripped from the name.
    
    See Also:
    
    Constant Field Values
  - OUTPUT_PROPERTIES_PREFIX
```
public static final String OUTPUT_PROPERTIES_PREFIX
```
    Prefix for additional properties for the output format. They are added to the Hadoop configuration, with the prefix stripped from the name.
    
    See Also:
    
    Constant Field Values
  - PROPERTY_ENABLE_EXPLORE_ON_CREATE
```
public static final String PROPERTY_ENABLE_EXPLORE_ON_CREATE
```
    Whether this dataset should be enabled for explore.
    
    See Also:
    
    Constant Field Values
  - PROPERTY_EXPLORE_FORMAT
```
public static final String PROPERTY_EXPLORE_FORMAT
```
    The format to use for the explore table. Currently, only text is supported.
    
    See Also:
    
    Constant Field Values
  - PROPERTY_EXPLORE_SCHEMA
```
public static final String PROPERTY_EXPLORE_SCHEMA
```
    The schema to use for the explore table. This should have the form: column type, ...
    
    See Also:
    
    Constant Field Values
  - PROPERTY_EXPLORE_SERDE
```
public static final String PROPERTY_EXPLORE_SERDE
```
    The serde to use for the Hive table.
    
    See Also:
    
    Constant Field Values
  - PROPERTY_EXPLORE_INPUT_FORMAT
```
public static final String PROPERTY_EXPLORE_INPUT_FORMAT
```
    The input format to use for the Hive table.
    
    See Also:
    
    Constant Field Values
  - PROPERTY_EXPLORE_OUTPUT_FORMAT
```
public static final String PROPERTY_EXPLORE_OUTPUT_FORMAT
```
    The output format to use for the Hive table.
    
    See Also:
    
    Constant Field Values
  - PROPERTY_EXPLORE_TABLE_PROPERTY_PREFIX
```
public static final String PROPERTY_EXPLORE_TABLE_PROPERTY_PREFIX
```
    Prefix used to store additional table properties for Hive.
    
    See Also:
    
    Constant Field Values
  - PROPERTY_FILES_PERMISSIONS
```
@Beta
public static final String PROPERTY_FILES_PERMISSIONS
```
    The permissions for the dataset. The value for this property must be given either as a 9-character String such as "rwxr-x---" or as an octal-base number such as 750. Permissions will be applied by each dataset depending on the access control paradigm of the storage engine.
    
    See Also:
    
    Constant Field Values
  - PROPERTY_FILES_GROUP
```
@Beta
public static final String PROPERTY_FILES_GROUP
```
    The group name that the permission are assigned to. For file-based datasets, this group name is used as the group for created files and directories; for table-based datasets, group privileges will be granted to this group.
    
    See Also:
    
    Constant Field Values
- Constructor Detail
  - FileSetProperties
```
public FileSetProperties()
```
- Method Detail
  - builder
```
public static FileSetProperties.Builder builder()
```
  - getBasePath
```
public static String getBasePath(Map<String,String> properties)
```
    Returns:
    
    the base path configured in the properties
  - getInputFormat
```
public static String getInputFormat(Map<String,String> properties)
```
    Returns:
    
    the input format configured in the properties
  - isDataExternal
```
public static boolean isDataExternal(Map<String,String> properties)
```
    Returns:
    
    whether the data (the files) in this dataset are considered external
  - isUseExisting
```
public static boolean isUseExisting(Map<String,String> properties)
```
    Returns:
    
    whether the file set is using a pre-existing base location
  - isPossessExisting
```
public static boolean isPossessExisting(Map<String,String> properties)
```
    Returns:
    
    whether the file set is taking possession of a pre-existing base location
  - getOutputFormat
```
public static String getOutputFormat(Map<String,String> properties)
```
    Returns:
    
    the output format configured in the properties
  - getInputProperties
```
public static Map<String,String> getInputProperties(Map<String,String> properties)
```
    Returns:
    
    the input format properties configured in the properties
  - getOutputProperties
```
public static Map<String,String> getOutputProperties(Map<String,String> properties)
```
    Returns:
    
    the output format properties configured in the properties
  - isExploreEnabled
```
public static boolean isExploreEnabled(Map<String,String> properties)
```
    Returns:
    
    whether explore is enabled by the properties
  - getExploreFormat
```
public static String getExploreFormat(Map<String,String> properties)
```
    Returns:
    
    the format of the explore table
  - getExploreSchema
```
public static String getExploreSchema(Map<String,String> properties)
```
    Returns:
    
    the schema of the explore table
  - getExploreFormatProperties
```
public static Map<String,String> getExploreFormatProperties(Map<String,String> properties)
```
    Returns:
    
    the properties for the explore format
  - getSerDe
```
public static String getSerDe(Map<String,String> properties)
```
    Returns:
    
    the class name of the serde configured in the properties
  - getExploreInputFormat
```
public static String getExploreInputFormat(Map<String,String> properties)
```
    Returns:
    
    the class name of the input format to be used in Hive. Note that this can be different than the input format used for the file set itself.
  - getExploreOutputFormat
```
public static String getExploreOutputFormat(Map<String,String> properties)
```
    Returns:
    
    the class name of the output format to be used in Hive. Note that this can be different than the output format used for the file set itself.
  - getFilePermissions
```
@Beta
public static String getFilePermissions(Map<String,String> properties)
```
    Returns:
    
    the default permissions for files and directories
  - getFileGroup
```
@Beta
public static String getFileGroup(Map<String,String> properties)
```
    Returns:
    
    the name of the group for files and directories
  - getTableProperties
```
public static Map<String,String> getTableProperties(Map<String,String> properties)
```
    Returns:
    
    the Hive table properties configured in the properties
  - propertiesWithPrefix
```
public static Map<String,String> propertiesWithPrefix(Map<String,String> properties,
                                                      String prefix)
```
    Returns:
    
    a map of all properties whose key begins with the given prefix, without that prefix

Class FileSetProperties

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

BASE_PATH

DATA_EXTERNAL

DATA_USE_EXISTING

DATA_POSSESS_EXISTING

INPUT_FORMAT

OUTPUT_FORMAT

INPUT_PROPERTIES_PREFIX

OUTPUT_PROPERTIES_PREFIX

PROPERTY_ENABLE_EXPLORE_ON_CREATE

PROPERTY_EXPLORE_FORMAT

PROPERTY_EXPLORE_SCHEMA

PROPERTY_EXPLORE_SERDE

PROPERTY_EXPLORE_INPUT_FORMAT

PROPERTY_EXPLORE_OUTPUT_FORMAT

PROPERTY_EXPLORE_TABLE_PROPERTY_PREFIX

PROPERTY_FILES_PERMISSIONS

PROPERTY_FILES_GROUP

Constructor Detail

FileSetProperties

Method Detail

builder

getBasePath

getInputFormat

isDataExternal

isUseExisting

isPossessExisting

getOutputFormat

getInputProperties

getOutputProperties

isExploreEnabled

getExploreFormat

getExploreSchema

getExploreFormatProperties

getSerDe

getExploreInputFormat

getExploreOutputFormat

getFilePermissions

getFileGroup

getTableProperties

propertiesWithPrefix