org.apache.flink.runtime.state.heap

Class CopyOnWriteStateTableSnapshot<K,N,S>

java.lang.Object

org.apache.flink.runtime.state.heap.CopyOnWriteStateTableSnapshot<K,N,S>

Type Parameters:

K - type of key

N - type of namespace

S - type of state

All Implemented Interfaces:

StateSnapshot

@Internal
public class CopyOnWriteStateTableSnapshot<K,N,S>
extends Object

This class represents the snapshot of a CopyOnWriteStateTable and has a role in operator state checkpointing. Besides holding the CopyOnWriteStateTables internal entries at the time of the snapshot, this class is also responsible for preparing and writing the state in the process of checkpointing.

IMPORTANT: Please notice that snapshot integrity of entries in this class rely on proper copy-on-write semantics through the CopyOnWriteStateTable that created the snapshot object, but all objects in this snapshot must be considered as READ-ONLY!. The reason is that the objects held by this class may or may not be deep copies of original objects that may still used in the CopyOnWriteStateTable. This depends for each entry on whether or not it was subject to copy-on-write operations by the CopyOnWriteStateTable. Phrased differently: the CopyOnWriteStateTable provides copy-on-write isolation for this snapshot, but this snapshot does not isolate modifications from the CopyOnWriteStateTable!

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	CopyOnWriteStateTableSnapshot.StateTableKeyGroupPartitioner<K,N,S> This class is the implementation of KeyGroupPartitioner for CopyOnWriteStateTable.
protected static class	CopyOnWriteStateTableSnapshot.TransformingStateTableKeyGroupPartitioner<K,N,S> Extended state snapshot transforming CopyOnWriteStateTableSnapshot.StateTableKeyGroupPartitioner.

Nested classes/interfaces inherited from interface org.apache.flink.runtime.state.StateSnapshot

StateSnapshot.StateKeyGroupWriter

Method Summary

Modifier and Type	Method and Description
StateSnapshot.StateKeyGroupWriter	getKeyGroupWriter() Partitions the snapshot data by key-group.
StateMetaInfoSnapshot	getMetaInfoSnapshot() Returns a snapshot of the state's meta data.
void	release() Optional hook to release resources for this snapshot at the end of its lifecycle.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

getKeyGroupWriter

@Nonnull
public StateSnapshot.StateKeyGroupWriter getKeyGroupWriter()

Partitions the snapshot data by key-group. The algorithm first builds a histogram for the distribution of keys into key-groups. Then, the histogram is accumulated to obtain the boundaries of each key-group in an array. Last, we use the accumulated counts as write position pointers for the key-group's bins when reordering the entries by key-group. This operation is lazily performed before the first writing of a key-group.

As a possible future optimization, we could perform the repartitioning in-place, using a scheme similar to the cuckoo cycles in cuckoo hashing. This can trade some performance for a smaller memory footprint.

getMetaInfoSnapshot

@Nonnull
public StateMetaInfoSnapshot getMetaInfoSnapshot()

Description copied from interface: StateSnapshot

Returns a snapshot of the state's meta data.

release

public void release()

Optional hook to release resources for this snapshot at the end of its lifecycle.

Specified by:

release in interface StateSnapshot