org.apache.flink.runtime.state

Class AbstractKeyedStateBackend<K>

java.lang.Object

org.apache.flink.runtime.state.AbstractKeyedStateBackend<K>

Type Parameters:

K - Type of the key by which state is keyed.

All Implemented Interfaces:

Closeable, AutoCloseable, CheckpointListener, InternalKeyContext<K>, KeyedStateBackend<K>, KeyedStateFactory, PriorityQueueSetFactory, Snapshotable<SnapshotResult<KeyedStateHandle>,Collection<KeyedStateHandle>>, SnapshotStrategy<SnapshotResult<KeyedStateHandle>>, org.apache.flink.util.Disposable

Direct Known Subclasses:

HeapKeyedStateBackend

public abstract class AbstractKeyedStateBackend<K>
extends Object
implements KeyedStateBackend<K>, Snapshotable<SnapshotResult<KeyedStateHandle>,Collection<KeyedStateHandle>>, Closeable, CheckpointListener

Base implementation of KeyedStateBackend. The state can be checkpointed to streams using SnapshotStrategy.snapshot(long, long, CheckpointStreamFactory, CheckpointOptions).

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.flink.core.fs.CloseableRegistry	cancelStreamRegistry Registry for all opened streams, so they can be closed if the task using this backend is closed.
protected StreamCompressionDecorator	keyGroupCompressionDecorator Decorates the input and output streams to write key-groups compressed.
protected KeyGroupRange	keyGroupRange Range of key-groups for which this backend is responsible.
protected org.apache.flink.api.common.typeutils.TypeSerializer<K>	keySerializer TypeSerializer for our key.
protected TaskKvStateRegistry	kvStateRegistry KvStateRegistry helper for this task.
protected int	numberOfKeyGroups The number of key-groups aka max parallelism.
protected ClassLoader	userCodeClassLoader

Constructor Summary

Constructors

Constructor and Description
AbstractKeyedStateBackend(TaskKvStateRegistry kvStateRegistry, org.apache.flink.api.common.typeutils.TypeSerializer<K> keySerializer, ClassLoader userCodeClassLoader, int numberOfKeyGroups, KeyGroupRange keyGroupRange, org.apache.flink.api.common.ExecutionConfig executionConfig, TtlTimeProvider ttlTimeProvider)

Method Summary

Modifier and Type	Method and Description
<N,S extends org.apache.flink.api.common.state.State,T> void	applyToAllKeys(N namespace, org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer, org.apache.flink.api.common.state.StateDescriptor<S,T> stateDescriptor, KeyedStateFunction<K,S> function) Applies the provided KeyedStateFunction to the state with the provided StateDescriptor of all the currently active keys.
void	close()
void	dispose() Closes the state backend, releasing all internal resources, but does not delete any persistent checkpoint data.
K	getCurrentKey() Used by states to access the current key.
int	getCurrentKeyGroupIndex() Returns the key-group to which the current key belongs.
KeyGroupRange	getKeyGroupRange() Returns the key groups for this backend.
org.apache.flink.api.common.typeutils.TypeSerializer<K>	getKeySerializer() TypeSerializer for the state backend key type.
int	getNumberOfKeyGroups() Returns the number of key-groups aka max parallelism.
<N,S extends org.apache.flink.api.common.state.State,V> S	getOrCreateKeyedState(org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer, org.apache.flink.api.common.state.StateDescriptor<S,V> stateDescriptor) Creates or retrieves a keyed state backed by this state backend.
<N,S extends org.apache.flink.api.common.state.State> S	getPartitionedState(N namespace, org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer, org.apache.flink.api.common.state.StateDescriptor<S,?> stateDescriptor) TODO: NOTE: This method does a lot of work caching / retrieving states just to update the namespace.
abstract int	numKeyValueStateEntries() Returns the total number of state entries across all keys/namespaces.
boolean	requiresLegacySynchronousTimerSnapshots()
void	setCurrentKey(K newKey) Sets the current key that is used for partitioned state.
boolean	supportsAsynchronousSnapshots()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.state.KeyedStateBackend

getKeys

Methods inherited from interface org.apache.flink.runtime.state.KeyedStateFactory

createInternalState, createInternalState

Methods inherited from interface org.apache.flink.runtime.state.PriorityQueueSetFactory

create

Methods inherited from interface org.apache.flink.runtime.state.Snapshotable

restore

Methods inherited from interface org.apache.flink.runtime.state.SnapshotStrategy

snapshot

Methods inherited from interface org.apache.flink.runtime.state.CheckpointListener

notifyCheckpointComplete

Field Detail

keySerializer

protected final org.apache.flink.api.common.typeutils.TypeSerializer<K> keySerializer

TypeSerializer for our key.

numberOfKeyGroups

protected final int numberOfKeyGroups

The number of key-groups aka max parallelism.

keyGroupRange

protected final KeyGroupRange keyGroupRange

Range of key-groups for which this backend is responsible.

kvStateRegistry

protected final TaskKvStateRegistry kvStateRegistry

KvStateRegistry helper for this task.

cancelStreamRegistry

protected org.apache.flink.core.fs.CloseableRegistry cancelStreamRegistry

Registry for all opened streams, so they can be closed if the task using this backend is closed.

userCodeClassLoader

protected final ClassLoader userCodeClassLoader

keyGroupCompressionDecorator

protected final StreamCompressionDecorator keyGroupCompressionDecorator

Decorates the input and output streams to write key-groups compressed.

Constructor Detail

AbstractKeyedStateBackend

public AbstractKeyedStateBackend(TaskKvStateRegistry kvStateRegistry,
                                 org.apache.flink.api.common.typeutils.TypeSerializer<K> keySerializer,
                                 ClassLoader userCodeClassLoader,
                                 int numberOfKeyGroups,
                                 KeyGroupRange keyGroupRange,
                                 org.apache.flink.api.common.ExecutionConfig executionConfig,
                                 TtlTimeProvider ttlTimeProvider)

Method Detail

dispose

public void dispose()

Closes the state backend, releasing all internal resources, but does not delete any persistent checkpoint data.

Specified by:

dispose in interface KeyedStateBackend<K>

Specified by:

dispose in interface org.apache.flink.util.Disposable

setCurrentKey

public void setCurrentKey(K newKey)

Description copied from interface: KeyedStateBackend

Sets the current key that is used for partitioned state.

Specified by:

setCurrentKey in interface KeyedStateBackend<K>

Parameters:

newKey - The new current key.

See Also:

KeyedStateBackend

getKeySerializer

public org.apache.flink.api.common.typeutils.TypeSerializer<K> getKeySerializer()

Description copied from interface: InternalKeyContext

TypeSerializer for the state backend key type.

Specified by:

getKeySerializer in interface InternalKeyContext<K>

See Also:

KeyedStateBackend

getCurrentKey

public K getCurrentKey()

Description copied from interface: InternalKeyContext

Used by states to access the current key.

Specified by:

getCurrentKey in interface InternalKeyContext<K>

See Also:

KeyedStateBackend

getCurrentKeyGroupIndex

public int getCurrentKeyGroupIndex()

Description copied from interface: InternalKeyContext

Returns the key-group to which the current key belongs.

Specified by:

getCurrentKeyGroupIndex in interface InternalKeyContext<K>

See Also:

KeyedStateBackend

getNumberOfKeyGroups

public int getNumberOfKeyGroups()

Description copied from interface: InternalKeyContext

Returns the number of key-groups aka max parallelism.

Specified by:

getNumberOfKeyGroups in interface InternalKeyContext<K>

See Also:

KeyedStateBackend

getKeyGroupRange

public KeyGroupRange getKeyGroupRange()

Description copied from interface: InternalKeyContext

Returns the key groups for this backend.

Specified by:

getKeyGroupRange in interface InternalKeyContext<K>

See Also:

KeyedStateBackend

applyToAllKeys

public <N,S extends org.apache.flink.api.common.state.State,T> void applyToAllKeys(N namespace,
                                                                                   org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer,
                                                                                   org.apache.flink.api.common.state.StateDescriptor<S,T> stateDescriptor,
                                                                                   KeyedStateFunction<K,S> function)
                                                                            throws Exception

Description copied from interface: KeyedStateBackend

Applies the provided KeyedStateFunction to the state with the provided StateDescriptor of all the currently active keys.

Specified by:

applyToAllKeys in interface KeyedStateBackend<K>

Type Parameters:

N - The type of the namespace.

S - The type of the state.

Parameters:

namespace - the namespace of the state.

namespaceSerializer - the serializer for the namespace.

stateDescriptor - the descriptor of the state to which the function is going to be applied.

function - the function to be applied to the keyed state.

Throws:

Exception

See Also:

KeyedStateBackend

getOrCreateKeyedState

public <N,S extends org.apache.flink.api.common.state.State,V> S getOrCreateKeyedState(org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer,
                                                                                       org.apache.flink.api.common.state.StateDescriptor<S,V> stateDescriptor)
                                                                                throws Exception

Description copied from interface: KeyedStateBackend

Creates or retrieves a keyed state backed by this state backend.

Specified by:

getOrCreateKeyedState in interface KeyedStateBackend<K>

Type Parameters:

N - The type of the namespace.

S - The type of the state.

Parameters:

namespaceSerializer - The serializer used for the namespace type of the state

stateDescriptor - The identifier for the state. This contains name and can create a default state value.

Returns:

A new key/value state backed by this backend.

Throws:

Exception - Exceptions may occur during initialization of the state and should be forwarded.

See Also:

KeyedStateBackend

getPartitionedState

public <N,S extends org.apache.flink.api.common.state.State> S getPartitionedState(N namespace,
                                                                                   org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer,
                                                                                   org.apache.flink.api.common.state.StateDescriptor<S,?> stateDescriptor)
                                                                            throws Exception

TODO: NOTE: This method does a lot of work caching / retrieving states just to update the namespace. This method should be removed for the sake of namespaces being lazily fetched from the keyed state backend, or being set on the state directly.

Specified by:

getPartitionedState in interface KeyedStateBackend<K>

Type Parameters:

N - The type of the namespace.

S - The type of the state.

stateDescriptor - The identifier for the state. This contains name and can create a default state value.

Returns:

A new key/value state backed by this backend.

Throws:

Exception - Exceptions may occur during initialization of the state and should be forwarded.

See Also:

KeyedStateBackend

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException

supportsAsynchronousSnapshots

@VisibleForTesting
public boolean supportsAsynchronousSnapshots()

numKeyValueStateEntries

@VisibleForTesting
public abstract int numKeyValueStateEntries()

Returns the total number of state entries across all keys/namespaces.

requiresLegacySynchronousTimerSnapshots

public boolean requiresLegacySynchronousTimerSnapshots()