org.apache.flink.runtime.executiongraph

Class ExecutionGraph

java.lang.Object

org.apache.flink.runtime.executiongraph.ExecutionGraph

All Implemented Interfaces:

Serializable

public class ExecutionGraph
extends Object
implements Serializable

The execution graph is the central data structure that coordinates the distributed execution of a data flow. It keeps representations of each parallel task, each intermediate result, and the communication between them. The execution graph consists of the following constructs:

• The ExecutionJobVertex represents one vertex from the JobGraph (usually one operation like "map" or "join") during execution. It holds the aggregated state of all parallel subtasks. The ExecutionJobVertex is identified inside the graph by the JobVertexID, which it takes from the JobGraph's corresponding JobVertex.
• The ExecutionVertex represents one parallel subtask. For each ExecutionJobVertex, there are as many ExecutionVertices as the parallelism. The ExecutionVertex is identified by the ExecutionJobVertex and the number of the parallel subtask
• The Execution is one attempt to execute a ExecutionVertex. There may be multiple Executions for the ExecutionVertex, in case of a failure, or in the case where some data needs to be recomputed because it is no longer available when requested by later operations. An Execution is always identified by an ExecutionAttemptID. All messages between the JobManager and the TaskManager about deployment of tasks and updates in the task status always use the ExecutionAttemptID to address the message receiver.

The ExecutionGraph implements Serializable, because it can be archived by sending it to an archive actor via an actor message. The execution graph does contain some non-serializable fields. These fields are not required in the archived form and are cleared in the prepareForArchiving() method.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
ExecutionGraph(org.apache.flink.api.common.JobID jobId, String jobName, org.apache.flink.configuration.Configuration jobConfig, scala.concurrent.duration.FiniteDuration timeout, List<BlobKey> requiredJarFiles, ClassLoader userClassLoader)

Method Summary

Modifier and Type	Method and Description
void	attachJobGraph(List<JobVertex> topologiallySorted)
void	cancel()
void	disableSnaphotCheckpointing()
void	enableSnaphotCheckpointing(long interval, long checkpointTimeout, List<ExecutionJobVertex> verticesToTrigger, List<ExecutionJobVertex> verticesToWaitFor, List<ExecutionJobVertex> verticesToCommitTo, akka.actor.ActorSystem actorSystem)
void	fail(Throwable t)
Iterable<ExecutionVertex>	getAllExecutionVertices()
Map<IntermediateDataSetID,IntermediateResult>	getAllIntermediateResults()
Map<JobVertexID,ExecutionJobVertex>	getAllVertices()
CheckpointCoordinator	getCheckpointCoordinator()
long	getDelayBeforeRetrying()
org.apache.flink.api.common.ExecutionConfig	getExecutionConfig()
Throwable	getFailureCause()
org.apache.flink.configuration.Configuration	getJobConfiguration()
org.apache.flink.api.common.JobID	getJobID()
String	getJobName()
ExecutionJobVertex	getJobVertex(JobVertexID id)
int	getNumberOfRetriesLeft()
Map<ExecutionAttemptID,Execution>	getRegisteredExecutions()
List<BlobKey>	getRequiredJarFiles() Returns a list of BLOB keys referring to the JAR files required to run this job
ScheduleMode	getScheduleMode()
Scheduler	getScheduler()
JobStatus	getState()
long	getStatusTimestamp(JobStatus status)
ClassLoader	getUserClassLoader()
Iterable<ExecutionJobVertex>	getVerticesTopologically()
boolean	isQueuedSchedulingAllowed()
boolean	isSnapshotCheckpointsEnabled()
void	prepareForArchiving() This method cleans fields that are irrelevant for the archived execution attempt.
void	registerExecutionListener(akka.actor.ActorRef listener)
void	registerJobStatusListener(akka.actor.ActorRef listener)
void	restart()
void	scheduleForExecution(Scheduler scheduler)
void	scheduleOrUpdateConsumers(ResultPartitionID partitionId)
void	setDelayBeforeRetrying(long delayBeforeRetrying)
void	setNumberOfRetriesLeft(int numberOfRetriesLeft)
void	setQueuedSchedulingAllowed(boolean allowed)
void	setScheduleMode(ScheduleMode scheduleMode)
boolean	updateState(TaskExecutionState state)
void	waitUntilFinished() For testing: This waits until the job execution has finished.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ExecutionGraph

public ExecutionGraph(org.apache.flink.api.common.JobID jobId,
                      String jobName,
                      org.apache.flink.configuration.Configuration jobConfig,
                      scala.concurrent.duration.FiniteDuration timeout,
                      List<BlobKey> requiredJarFiles,
                      ClassLoader userClassLoader)

Method Detail

setNumberOfRetriesLeft

public void setNumberOfRetriesLeft(int numberOfRetriesLeft)

getNumberOfRetriesLeft

public int getNumberOfRetriesLeft()

setDelayBeforeRetrying

public void setDelayBeforeRetrying(long delayBeforeRetrying)

getDelayBeforeRetrying

public long getDelayBeforeRetrying()

isQueuedSchedulingAllowed

public boolean isQueuedSchedulingAllowed()

setQueuedSchedulingAllowed

public void setQueuedSchedulingAllowed(boolean allowed)

setScheduleMode

public void setScheduleMode(ScheduleMode scheduleMode)

getScheduleMode

public ScheduleMode getScheduleMode()

enableSnaphotCheckpointing

public void enableSnaphotCheckpointing(long interval,
                                       long checkpointTimeout,
                                       List<ExecutionJobVertex> verticesToTrigger,
                                       List<ExecutionJobVertex> verticesToWaitFor,
                                       List<ExecutionJobVertex> verticesToCommitTo,
                                       akka.actor.ActorSystem actorSystem)

disableSnaphotCheckpointing

public void disableSnaphotCheckpointing()

isSnapshotCheckpointsEnabled

public boolean isSnapshotCheckpointsEnabled()

getCheckpointCoordinator

public CheckpointCoordinator getCheckpointCoordinator()

getRequiredJarFiles

public List<BlobKey> getRequiredJarFiles()

Returns a list of BLOB keys referring to the JAR files required to run this job

Returns:

list of BLOB keys referring to the JAR files required to run this job

getScheduler

public Scheduler getScheduler()

getJobID

public org.apache.flink.api.common.JobID getJobID()

getJobName

public String getJobName()

getJobConfiguration

public org.apache.flink.configuration.Configuration getJobConfiguration()

getUserClassLoader

public ClassLoader getUserClassLoader()

getState

public JobStatus getState()

getFailureCause

public Throwable getFailureCause()

getJobVertex

public ExecutionJobVertex getJobVertex(JobVertexID id)

getAllVertices

public Map<JobVertexID,ExecutionJobVertex> getAllVertices()

getVerticesTopologically

public Iterable<ExecutionJobVertex> getVerticesTopologically()

getAllIntermediateResults

public Map<IntermediateDataSetID,IntermediateResult> getAllIntermediateResults()

getAllExecutionVertices

public Iterable<ExecutionVertex> getAllExecutionVertices()

getStatusTimestamp

public long getStatusTimestamp(JobStatus status)

attachJobGraph

public void attachJobGraph(List<JobVertex> topologiallySorted)
                    throws JobException

Throws:

JobException

scheduleForExecution

public void scheduleForExecution(Scheduler scheduler)
                          throws JobException

Throws:

JobException

cancel

public void cancel()

fail

public void fail(Throwable t)

restart

public void restart()

prepareForArchiving

public void prepareForArchiving()

This method cleans fields that are irrelevant for the archived execution attempt.

getExecutionConfig

public org.apache.flink.api.common.ExecutionConfig getExecutionConfig()

waitUntilFinished

public void waitUntilFinished()
                       throws InterruptedException

For testing: This waits until the job execution has finished.

Throws:

InterruptedException

updateState

public boolean updateState(TaskExecutionState state)

scheduleOrUpdateConsumers

public void scheduleOrUpdateConsumers(ResultPartitionID partitionId)

getRegisteredExecutions

public Map<ExecutionAttemptID,Execution> getRegisteredExecutions()

registerJobStatusListener

public void registerJobStatusListener(akka.actor.ActorRef listener)

registerExecutionListener

public void registerExecutionListener(akka.actor.ActorRef listener)