co.cask.cdap.data2.dataset2

Interface DatasetFramework

All Known Implementing Classes:

InMemoryDatasetFramework, LineageWriterDatasetFramework, RemoteDatasetFramework, StaticDatasetFramework

public interface DatasetFramework

Provides access to the Datasets System. Typical usage example: DatasetFramework datasetFramework = ...; datasetFramework.addModule("myDatasets", MyDatasetModule.class); datasetFramework.addInstance("myTable", "table", DatasetProperties.EMPTY); TableAdmin admin = datasetFramework.getAdmin("myTable"); admin.create(); Table table = datasetFramework.getDataset("myTable"); try { table.write("key", "value"); } finally { table.close(); }

Method Summary

Methods

Modifier and Type	Method and Description
void	addInstance(String datasetTypeName, Id.DatasetInstance datasetInstanceId, co.cask.cdap.api.dataset.DatasetProperties props) Adds information about dataset instance to the system.
void	addModule(Id.DatasetModule moduleId, co.cask.cdap.api.dataset.module.DatasetModule module) Adds dataset types by adding dataset module to the system.
void	addModule(Id.DatasetModule moduleId, co.cask.cdap.api.dataset.module.DatasetModule module, Location jarLocation) Adds dataset types by adding dataset module to the system with a jar location containing all dataset classes needed by the module.
void	createNamespace(Id.Namespace namespaceId) Creates a namespace in the Storage Providers - HBase/LevelDB, Hive and HDFS/Local File System.
void	deleteAllInstances(Id.Namespace namespaceId) Deletes all dataset instances in the specified namespace.
void	deleteAllModules(Id.Namespace namespaceId) Deletes dataset modules and its types in the specified namespace.
void	deleteInstance(Id.DatasetInstance datasetInstanceId) Deletes dataset instance from the system.
void	deleteModule(Id.DatasetModule moduleId) Deletes dataset module and its types from the system.
void	deleteNamespace(Id.Namespace namespaceId) Deletes a namespace in the Storage Providers - HBase/LevelDB, Hive and HDFS/Local File System.
<T extends co.cask.cdap.api.dataset.DatasetAdmin> T	getAdmin(Id.DatasetInstance datasetInstanceId, ClassLoader classLoader) Gets dataset instance admin to be used to perform administrative operations.
<T extends co.cask.cdap.api.dataset.DatasetAdmin> T	getAdmin(Id.DatasetInstance datasetInstanceId, ClassLoader classLoader, DatasetClassLoaderProvider classLoaderProvider) Gets dataset instance admin to be used to perform administrative operations.
<T extends co.cask.cdap.api.dataset.Dataset> T	getDataset(Id.DatasetInstance datasetInstanceId, Map<String,String> arguments, ClassLoader classLoader) Gets dataset to be used to perform data operations.
<T extends co.cask.cdap.api.dataset.Dataset> T	getDataset(Id.DatasetInstance datasetInstanceId, Map<String,String> arguments, ClassLoader classLoader, DatasetClassLoaderProvider classLoaderProvider, Iterable<? extends Id> owners) Gets dataset to be used to perform data operations.
<T extends co.cask.cdap.api.dataset.Dataset> T	getDataset(Id.DatasetInstance datasetInstanceId, Map<String,String> arguments, ClassLoader classLoader, Iterable<? extends Id> owners) Gets dataset to be used to perform data operations.
co.cask.cdap.api.dataset.DatasetSpecification	getDatasetSpec(Id.DatasetInstance datasetInstanceId) Gets the DatasetSpecification for the specified dataset instance id
Collection<DatasetSpecificationSummary>	getInstances(Id.Namespace namespaceId) Get all dataset instances in the specified namespace
boolean	hasInstance(Id.DatasetInstance datasetInstanceId)
boolean	hasSystemType(String typeName) Checks if the specified type exists in the 'system' namespace
boolean	hasType(Id.DatasetType datasetTypeId) Checks if the specified type exists in the specified namespace
void	updateInstance(Id.DatasetInstance datasetInstanceId, co.cask.cdap.api.dataset.DatasetProperties props) Updates the existing dataset instance in the system.

Method Detail

addModule

void addModule(Id.DatasetModule moduleId,
             co.cask.cdap.api.dataset.module.DatasetModule module)
               throws DatasetManagementException

Adds dataset types by adding dataset module to the system. Calling this method to add DatasetModule may result in tracing class dependencies if the DatasetModule is not a system dataset, which can takes couple seconds for the tracing. If the jar Location containing the DatasetModule is known, it's better to call addModule(Id.DatasetModule, DatasetModule, Location) instead.

Parameters:

moduleId - dataset module id

module - dataset module

Throws:

ModuleConflictException - when module with same name is already registered or this module registers a type with a same name as one of the already registered by another module types

DatasetManagementException - in case of problems

ServiceUnavailableException - when the dataset service is not running

addModule

void addModule(Id.DatasetModule moduleId,
             co.cask.cdap.api.dataset.module.DatasetModule module,
             Location jarLocation)
               throws DatasetManagementException

Adds dataset types by adding dataset module to the system with a jar location containing all dataset classes needed by the module.

Parameters:

moduleId - dataset module id

module - dataset module

jarLocation - location of the jar file that contains the dataset classes needed by the module

Throws:

ModuleConflictException - when module with same name is already registered or this module registers a type with a same name as one of the already registered by another module types

DatasetManagementException - in case of problems

ServiceUnavailableException - when the dataset service is not running

deleteModule

void deleteModule(Id.DatasetModule moduleId)
                  throws DatasetManagementException

Deletes dataset module and its types from the system.

Parameters:

moduleId - dataset module id

Throws:

ModuleConflictException - when module cannot be deleted because of its dependant modules or instances

DatasetManagementException

ServiceUnavailableException - when the dataset service is not running

deleteAllModules

void deleteAllModules(Id.Namespace namespaceId)
                      throws DatasetManagementException

Deletes dataset modules and its types in the specified namespace.

Parameters:

namespaceId - the Id.Namespace to delete all modules from.

Throws:

ModuleConflictException - when some of modules can't be deleted because of its dependant modules or instances

DatasetManagementException

ServiceUnavailableException - when the dataset service is not running

addInstance

void addInstance(String datasetTypeName,
               Id.DatasetInstance datasetInstanceId,
               co.cask.cdap.api.dataset.DatasetProperties props)
                 throws DatasetManagementException,
                        IOException

Adds information about dataset instance to the system. This uses DatasetDefinition.configure(String, DatasetProperties) method to build DatasetSpecification which describes dataset instance and later used to initialize DatasetAdmin and Dataset for the dataset instance.

Parameters:

datasetTypeName - dataset instance type name

datasetInstanceId - dataset instance name

props - dataset instance properties

Throws:

InstanceConflictException - if dataset instance with this name already exists

IOException - when creation of dataset instance using its admin fails

DatasetManagementException

ServiceUnavailableException - when the dataset service is not running

updateInstance

void updateInstance(Id.DatasetInstance datasetInstanceId,
                  co.cask.cdap.api.dataset.DatasetProperties props)
                    throws DatasetManagementException,
                           IOException

Updates the existing dataset instance in the system. This uses DatasetDefinition.configure(String, DatasetProperties) method to build DatasetSpecification with new properties, which describes dataset instance and DatasetAdmin is used to upgrade Dataset for the dataset instance.

Parameters:

datasetInstanceId - dataset instance name

props - dataset instance properties

Throws:

IOException - when creation of dataset instance using its admin fails

DatasetManagementException

ServiceUnavailableException - when the dataset service is not running

getInstances

Collection<DatasetSpecificationSummary> getInstances(Id.Namespace namespaceId)
                                                     throws DatasetManagementException

Get all dataset instances in the specified namespace

Parameters:

namespaceId - the specified namespace id

Returns:

a collection of DatasetSpecifications for all datasets in the specified namespace

Throws:

DatasetManagementException

getDatasetSpec

@Nullable
co.cask.cdap.api.dataset.DatasetSpecification getDatasetSpec(Id.DatasetInstance datasetInstanceId)
                                                             throws DatasetManagementException

Gets the DatasetSpecification for the specified dataset instance id

Parameters:

datasetInstanceId - the Id.DatasetInstance for which the DatasetSpecification is desired

Returns:

DatasetSpecification of the dataset or null if dataset not not exist

Throws:

DatasetManagementException

hasInstance

boolean hasInstance(Id.DatasetInstance datasetInstanceId)
                    throws DatasetManagementException

Parameters:

datasetInstanceId - the Id.DatasetInstance to check for existence

Returns:

true if instance exists, false otherwise

Throws:

DatasetManagementException

ServiceUnavailableException - when the dataset service is not running

hasSystemType

boolean hasSystemType(String typeName)
                      throws DatasetManagementException

Checks if the specified type exists in the 'system' namespace

Returns:

true if type exists in the 'system' namespace, false otherwise

Throws:

DatasetManagementException

ServiceUnavailableException - when the dataset service is not running

hasType

boolean hasType(Id.DatasetType datasetTypeId)
                throws DatasetManagementException

Checks if the specified type exists in the specified namespace

Returns:

true if type exists in the specified namespace, false otherwise

Throws:

DatasetManagementException

ServiceUnavailableException - when the dataset service is not running

deleteInstance

void deleteInstance(Id.DatasetInstance datasetInstanceId)
                    throws DatasetManagementException,
                           IOException

Deletes dataset instance from the system.

Parameters:

datasetInstanceId - dataset instance name

Throws:

InstanceConflictException - if dataset instance cannot be deleted because of its dependencies

IOException - when deletion of dataset instance using its admin fails

DatasetManagementException

ServiceUnavailableException - when the dataset service is not running

deleteAllInstances

void deleteAllInstances(Id.Namespace namespaceId)
                        throws DatasetManagementException,
                               IOException

Deletes all dataset instances in the specified namespace.

Parameters:

namespaceId - the specified namespace id

Throws:

IOException - when deletion of dataset instance using its admin fails

DatasetManagementException

ServiceUnavailableException - when the dataset service is not running

getAdmin

@Nullable
<T extends co.cask.cdap.api.dataset.DatasetAdmin> T getAdmin(Id.DatasetInstance datasetInstanceId,
                                                                    @Nullable
                                                                    ClassLoader classLoader)
                                                         throws DatasetManagementException,
                                                                IOException

Gets dataset instance admin to be used to perform administrative operations. The given classloader must be able to load all classes needed to instantiate the dataset admin. This means if the system classloader is used, only system dataset admins can fetched.

Type Parameters:

T - dataset admin type

Parameters:

datasetInstanceId - dataset instance name

classLoader - classLoader to be used to load classes or null to use system classLoader

Returns:

instance of dataset admin or null if dataset instance of this name doesn't exist.

Throws:

DatasetManagementException - when there's trouble getting dataset meta info

IOException - when there's trouble to instantiate DatasetAdmin

ServiceUnavailableException - when the dataset service is not running

getAdmin

@Nullable
<T extends co.cask.cdap.api.dataset.DatasetAdmin> T getAdmin(Id.DatasetInstance datasetInstanceId,
                                                                    @Nullable
                                                                    ClassLoader classLoader,
                                                                    DatasetClassLoaderProvider classLoaderProvider)
                                                         throws DatasetManagementException,
                                                                IOException

Gets dataset instance admin to be used to perform administrative operations. The class loader provider is used get classloaders for any dataset modules used by the specified dataset admin. This is because the classloader(s) for a dataset admin may create some resources that need to be cleaned up on close.

Type Parameters:

T - dataset admin type

Parameters:

datasetInstanceId - dataset instance name

classLoader - parent classLoader to be used to load classes or null to use system classLoader

classLoaderProvider - provider to get classloaders for different dataset modules

Returns:

instance of dataset admin or null if dataset instance of this name doesn't exist.

Throws:

DatasetManagementException - when there's trouble getting dataset meta info

IOException - when there's trouble to instantiate DatasetAdmin

ServiceUnavailableException - when the dataset service is not running

getDataset

@Nullable
<T extends co.cask.cdap.api.dataset.Dataset> T getDataset(Id.DatasetInstance datasetInstanceId,
                                                                 @Nullable
                                                                 Map<String,String> arguments,
                                                                 @Nullable
                                                                 ClassLoader classLoader,
                                                                 @Nullable
                                                                 Iterable<? extends Id> owners)
                                                      throws DatasetManagementException,
                                                             IOException

Gets dataset to be used to perform data operations.

Type Parameters:

T - dataset type to be returned

Parameters:

datasetInstanceId - dataset instance id

arguments - runtime arguments for the dataset instance

classLoader - classLoader to be used to load classes or null to use system classLoader

owners - owners of the dataset

Returns:

instance of dataset or null if dataset instance of this name doesn't exist.

Throws:

DatasetManagementException - when there's trouble getting dataset meta info

IOException - when there's trouble to instantiate Dataset

ServiceUnavailableException - when the dataset service is not running

getDataset

@Nullable
<T extends co.cask.cdap.api.dataset.Dataset> T getDataset(Id.DatasetInstance datasetInstanceId,
                                                                 @Nullable
                                                                 Map<String,String> arguments,
                                                                 @Nullable
                                                                 ClassLoader classLoader)
                                                      throws DatasetManagementException,
                                                             IOException

Gets dataset to be used to perform data operations.

Type Parameters:

T - dataset type to be returned

Parameters:

datasetInstanceId - dataset instance id

arguments - runtime arguments for the dataset instance

classLoader - classLoader to be used to load classes or null to use system classLoader

Returns:

instance of dataset or null if dataset instance of this name doesn't exist.

Throws:

DatasetManagementException - when there's trouble getting dataset meta info

IOException - when there's trouble to instantiate Dataset

ServiceUnavailableException - when the dataset service is not running

getDataset

@Nullable
<T extends co.cask.cdap.api.dataset.Dataset> T getDataset(Id.DatasetInstance datasetInstanceId,
                                                                 @Nullable
                                                                 Map<String,String> arguments,
                                                                 @Nullable
                                                                 ClassLoader classLoader,
                                                                 DatasetClassLoaderProvider classLoaderProvider,
                                                                 @Nullable
                                                                 Iterable<? extends Id> owners)
                                                      throws DatasetManagementException,
                                                             IOException

Gets dataset to be used to perform data operations. This one is used when the classloader(s) for a dataset may create some resources that need to be cleaned up on close.

Type Parameters:

T - dataset type to be returned

Parameters:

datasetInstanceId - dataset instance id

arguments - runtime arguments for the dataset instance

classLoader - parent classLoader to be used to load classes or null to use system classLoader

classLoaderProvider - provider to get classloaders for different dataset modules

owners - owners of the dataset

Returns:

instance of dataset or null if dataset instance of this name doesn't exist.

Throws:

DatasetManagementException - when there's trouble getting dataset meta info

IOException - when there's trouble to instantiate Dataset

ServiceUnavailableException - when the dataset service is not running

createNamespace

void createNamespace(Id.Namespace namespaceId)
                     throws DatasetManagementException

Creates a namespace in the Storage Providers - HBase/LevelDB, Hive and HDFS/Local File System.

Parameters:

namespaceId - the Id.Namespace to create

Throws:

DatasetManagementException

deleteNamespace

void deleteNamespace(Id.Namespace namespaceId)
                     throws DatasetManagementException

Deletes a namespace in the Storage Providers - HBase/LevelDB, Hive and HDFS/Local File System.

Parameters:

namespaceId - the Id.Namespace to create

Throws:

DatasetManagementException