co.cask.cdap.data2.metadata.lineage

Class LineageDataset

java.lang.Object

co.cask.cdap.api.dataset.lib.AbstractDataset

co.cask.cdap.data2.metadata.lineage.LineageDataset

All Implemented Interfaces:

co.cask.cdap.api.dataset.Dataset, co.cask.cdap.api.dataset.metrics.MeteredDataset, co.cask.tephra.TransactionAware, Closeable, AutoCloseable

public class LineageDataset
extends co.cask.cdap.api.dataset.lib.AbstractDataset

Dataset to store/retrieve Dataset accesses of a Program.

Constructor Summary

Constructors

Constructor and Description
LineageDataset(String instanceName, co.cask.cdap.api.dataset.table.Table accessRegistryTable)

Method Summary

Methods

Modifier and Type	Method and Description
void	addAccess(Id.Run run, Id.DatasetInstance datasetInstance, AccessType accessType, long accessTimeMillis) Add a program-dataset access.
void	addAccess(Id.Run run, Id.DatasetInstance datasetInstance, AccessType accessType, long accessTimeMillis, Id.NamespacedId component) Add a program-dataset access.
void	addAccess(Id.Run run, Id.Stream stream, AccessType accessType, long accessTimeMillis) Add a program-stream access.
void	addAccess(Id.Run run, Id.Stream stream, AccessType accessType, long accessTimeMillis, Id.NamespacedId component) Add a program-stream access.
List<Long>	getAccessTimesForRun(Id.Run run)
Set<Id.NamespacedId>	getEntitiesForRun(Id.Run run)
Set<Relation>	getRelations(Id.DatasetInstance datasetInstance, long start, long end, com.google.common.base.Predicate<Relation> filter) Fetch program-dataset access information for a dataset for a given period.
Set<Relation>	getRelations(Id.Program program, long start, long end, com.google.common.base.Predicate<Relation> filter) Fetch program-dataset access information for a program for a given period.
Set<Relation>	getRelations(Id.Stream stream, long start, long end, com.google.common.base.Predicate<Relation> filter) Fetch program-stream access information for a dataset for a given period.

Methods inherited from class co.cask.cdap.api.dataset.lib.AbstractDataset

close, commitTx, getName, getTransactionAwareName, getTxChanges, postTxCommit, rollbackTx, setMetricsCollector, startTx, toString, updateTx

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

LineageDataset

public LineageDataset(String instanceName,
              co.cask.cdap.api.dataset.table.Table accessRegistryTable)

Method Detail

addAccess

public void addAccess(Id.Run run,
             Id.DatasetInstance datasetInstance,
             AccessType accessType,
             long accessTimeMillis)

Add a program-dataset access.

Parameters:

run - program run information

datasetInstance - dataset accessed by the program

accessType - access type

accessTimeMillis - time of access

addAccess

public void addAccess(Id.Run run,
             Id.DatasetInstance datasetInstance,
             AccessType accessType,
             long accessTimeMillis,
             @Nullable
             Id.NamespacedId component)

Add a program-dataset access.

Parameters:

run - program run information

datasetInstance - dataset accessed by the program

accessType - access type

accessTimeMillis - time of access

component - program component such as flowlet id, etc.

addAccess

public void addAccess(Id.Run run,
             Id.Stream stream,
             AccessType accessType,
             long accessTimeMillis)

Add a program-stream access.

Parameters:

run - program run information

stream - stream accessed by the program

accessType - access type

accessTimeMillis - time of access

addAccess

public void addAccess(Id.Run run,
             Id.Stream stream,
             AccessType accessType,
             long accessTimeMillis,
             @Nullable
             Id.NamespacedId component)

Add a program-stream access.

Parameters:

run - program run information

stream - stream accessed by the program

accessType - access type

accessTimeMillis - time of access

component - program component such as flowlet id, etc.

getEntitiesForRun

public Set<Id.NamespacedId> getEntitiesForRun(Id.Run run)

Returns:

a set of entities (program and data it accesses) associated with a program run.

getRelations

public Set<Relation> getRelations(Id.DatasetInstance datasetInstance,
                         long start,
                         long end,
                         com.google.common.base.Predicate<Relation> filter)

Fetch program-dataset access information for a dataset for a given period.

Parameters:

datasetInstance - dataset for which to fetch access information

start - start time period

end - end time period

filter - filter to be applied on result set

Returns:

program-dataset access information

getRelations

public Set<Relation> getRelations(Id.Stream stream,
                         long start,
                         long end,
                         com.google.common.base.Predicate<Relation> filter)

Fetch program-stream access information for a dataset for a given period.

Parameters:

stream - stream for which to fetch access information

start - start time period

end - end time period

filter - filter to be applied on result set

Returns:

program-dataset access information

getRelations

public Set<Relation> getRelations(Id.Program program,
                         long start,
                         long end,
                         com.google.common.base.Predicate<Relation> filter)

Fetch program-dataset access information for a program for a given period.

Parameters:

program - program for which to fetch access information

start - start time period

end - end time period

filter - filter to be applied on result set

Returns:

program-dataset access information

getAccessTimesForRun

public List<Long> getAccessTimesForRun(Id.Run run)

Returns:

a set of access times (for program and data it accesses) associated with a program run.