org.apache.flink.runtime.jobgraph

Class JobGraph

java.lang.Object

org.apache.flink.runtime.jobgraph.JobGraph

All Implemented Interfaces:

Serializable

public class JobGraph
extends Object
implements Serializable

The JobGraph represents a Flink dataflow program, at the low level that the JobManager accepts. All programs from higher level APIs are transformed into JobGraphs.

The JobGraph is a graph of vertices and intermediate results that are connected together to form a DAG. Note that iterations (feedback edges) are currently not encoded inside the JobGraph but inside certain special vertices that establish the feedback channel amongst themselves.

The JobGraph defines the job-wide configuration settings, while each vertex and intermediate result define the characteristics of the concrete operation and intermediate data.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
JobGraph() Constructs a new job graph with no name and a random job ID.
JobGraph(org.apache.flink.api.common.JobID jobId, String jobName) Constructs a new job graph with the given name and a random job ID.
JobGraph(org.apache.flink.api.common.JobID jobId, String jobName, JobVertex... vertices) Constructs a new job graph with the given name and a random job ID.
JobGraph(JobVertex... vertices) Constructs a new job graph with no name and a random job ID.
JobGraph(String jobName) Constructs a new job graph with the given name and a random job ID.
JobGraph(String jobName, JobVertex... vertices) Constructs a new job graph with the given name and a random job ID.

Method Summary

Modifier and Type	Method and Description
void	addBlob(BlobKey key) Adds the BLOB referenced by the key to the JobGraph's dependencies.
void	addJar(org.apache.flink.core.fs.Path jar) Adds the path of a JAR file required to run the job on a task manager.
void	addVertex(JobVertex vertex) Adds a new task vertex to the job graph if it is not already included.
JobVertex	findVertexByID(JobVertexID id) Searches for a vertex with a matching ID and returns it.
boolean	getAllowQueuedScheduling()
org.apache.flink.configuration.Configuration	getJobConfiguration() Returns the configuration object for this job if it is set.
org.apache.flink.api.common.JobID	getJobID() Returns the ID of the job.
String	getName() Returns the name assigned to the job graph.
int	getNumberOfExecutionRetries() Gets the number of times the system will try to re-execute failed tasks.
int	getNumberOfVertices() Returns the number of all vertices.
ScheduleMode	getScheduleMode()
JobSnapshottingSettings	getSnapshotSettings() Gets the settings for asynchronous snapshots.
List<BlobKey>	getUserJarBlobKeys() Returns a set of BLOB keys referring to the JAR files required to run this job.
Iterable<JobVertex>	getVertices() Returns an Iterable to iterate all vertices registered with the job graph.
JobVertex[]	getVerticesAsArray() Returns an array of all job vertices that are registered with the job graph.
List<JobVertex>	getVerticesSortedTopologicallyFromSources()
boolean	hasUsercodeJarFiles() Checks whether the JobGraph has user code JAR files attached.
void	setAllowQueuedScheduling(boolean allowQueuedScheduling)
void	setNumberOfExecutionRetries(int numberOfExecutionRetries) Sets the number of times that failed tasks are re-executed.
void	setScheduleMode(ScheduleMode scheduleMode)
void	setSnapshotSettings(JobSnapshottingSettings settings) Sets the settings for asynchronous snapshots.
void	uploadRequiredJarFiles(InetSocketAddress serverAddress) Uploads the previously added user jar file to the job manager through the job manager's BLOB server.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

JobGraph

public JobGraph()

Constructs a new job graph with no name and a random job ID.

JobGraph

public JobGraph(String jobName)

Constructs a new job graph with the given name and a random job ID.

Parameters:

jobName - The name of the job

JobGraph

public JobGraph(org.apache.flink.api.common.JobID jobId,
                String jobName)

Constructs a new job graph with the given name and a random job ID.

Parameters:

jobId - The id of the job

jobName - The name of the job

JobGraph

public JobGraph(JobVertex... vertices)

Constructs a new job graph with no name and a random job ID.

Parameters:

vertices - The vertices to add to the graph.

JobGraph

public JobGraph(String jobName,
                JobVertex... vertices)

Constructs a new job graph with the given name and a random job ID.

Parameters:

jobName - The name of the job.

vertices - The vertices to add to the graph.

JobGraph

public JobGraph(org.apache.flink.api.common.JobID jobId,
                String jobName,
                JobVertex... vertices)

Constructs a new job graph with the given name and a random job ID.

Parameters:

jobId - The id of the job.

jobName - The name of the job.

vertices - The vertices to add to the graph.

Method Detail

getJobID

public org.apache.flink.api.common.JobID getJobID()

Returns the ID of the job.

Returns:

the ID of the job

getName

public String getName()

Returns the name assigned to the job graph.

Returns:

the name assigned to the job graph

getJobConfiguration

public org.apache.flink.configuration.Configuration getJobConfiguration()

Returns the configuration object for this job if it is set.

Returns:

the configuration object for this job, or null if it is not set

setNumberOfExecutionRetries

public void setNumberOfExecutionRetries(int numberOfExecutionRetries)

Sets the number of times that failed tasks are re-executed. A value of zero effectively disables fault tolerance. A value of -1 indicates that the system default value (as defined in the configuration) should be used.

Parameters:

numberOfExecutionRetries - The number of times the system will try to re-execute failed tasks.

getNumberOfExecutionRetries

public int getNumberOfExecutionRetries()

Gets the number of times the system will try to re-execute failed tasks. A value of -1 indicates that the system default value (as defined in the configuration) should be used.

Returns:

The number of times the system will try to re-execute failed tasks.

setAllowQueuedScheduling

public void setAllowQueuedScheduling(boolean allowQueuedScheduling)

getAllowQueuedScheduling

public boolean getAllowQueuedScheduling()

setScheduleMode

public void setScheduleMode(ScheduleMode scheduleMode)

getScheduleMode

public ScheduleMode getScheduleMode()

addVertex

public void addVertex(JobVertex vertex)

Adds a new task vertex to the job graph if it is not already included.

Parameters:

vertex - the new task vertex to be added

getVertices

public Iterable<JobVertex> getVertices()

Returns an Iterable to iterate all vertices registered with the job graph.

Returns:

an Iterable to iterate all vertices registered with the job graph

getVerticesAsArray

public JobVertex[] getVerticesAsArray()

Returns an array of all job vertices that are registered with the job graph. The order in which the vertices appear in the list is not defined.

Returns:

an array of all job vertices that are registered with the job graph

getNumberOfVertices

public int getNumberOfVertices()

Returns the number of all vertices.

Returns:

The number of all vertices.

setSnapshotSettings

public void setSnapshotSettings(JobSnapshottingSettings settings)

Sets the settings for asynchronous snapshots. A value of null means that snapshotting is not enabled.

Parameters:

settings - The snapshot settings, or null, to disable snapshotting.

getSnapshotSettings

public JobSnapshottingSettings getSnapshotSettings()

Gets the settings for asynchronous snapshots. This method returns null, when snapshotting is not enabled.

Returns:

The snapshot settings, or null, if snapshotting is not enabled.

findVertexByID

public JobVertex findVertexByID(JobVertexID id)

Searches for a vertex with a matching ID and returns it.

Parameters:

id - the ID of the vertex to search for

Returns:

the vertex with the matching ID or null if no vertex with such ID could be found

getVerticesSortedTopologicallyFromSources

public List<JobVertex> getVerticesSortedTopologicallyFromSources()
                                                          throws org.apache.flink.api.common.InvalidProgramException

Throws:

org.apache.flink.api.common.InvalidProgramException

addJar

public void addJar(org.apache.flink.core.fs.Path jar)

Adds the path of a JAR file required to run the job on a task manager.

Parameters:

jar - path of the JAR file required to run the job on a task manager

addBlob

public void addBlob(BlobKey key)

Adds the BLOB referenced by the key to the JobGraph's dependencies.

Parameters:

key - path of the JAR file required to run the job on a task manager

hasUsercodeJarFiles

public boolean hasUsercodeJarFiles()

Checks whether the JobGraph has user code JAR files attached.

Returns:

True, if the JobGraph has user code JAR files attached, false otherwise.

getUserJarBlobKeys

public List<BlobKey> getUserJarBlobKeys()

Returns a set of BLOB keys referring to the JAR files required to run this job.

Returns:

set of BLOB keys referring to the JAR files required to run this job

uploadRequiredJarFiles

public void uploadRequiredJarFiles(InetSocketAddress serverAddress)
                            throws IOException

Uploads the previously added user jar file to the job manager through the job manager's BLOB server.

Parameters:

serverAddress - the network address of the BLOB server

Throws:

IOException - thrown if an I/O error occurs during the upload