Package org.apache.beam.sdk.io.gcp.bigquery

Class PatchTableSchemaDoFn<DestinationT extends @NonNull java.lang.Object,​ElementT>

  • All Implemented Interfaces:
    java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData
    public class PatchTableSchemaDoFn<DestinationT extends @NonNull java.lang.Object,​ElementT>
extends org.apache.beam.sdk.transforms.DoFn<org.apache.beam.sdk.values.KV<DestinationT,​com.google.cloud.bigquery.storage.v1.TableSchema>,​org.apache.beam.sdk.values.KV<DestinationT,​ElementT>>
    This DoFn is responsible for updating a BigQuery's table schema. The input is a TableSchema containing only the schema delta (new fields, relaxed fields). It outputs elements for all updated tables, which act as notifcations to the buffering stage that the elements can be retried.
    See Also:
    Serialized Form

    • Nested Class Summary

      • Nested classes/interfaces inherited from class org.apache.beam.sdk.transforms.DoFn

        org.apache.beam.sdk.transforms.DoFn.AlwaysFetched, org.apache.beam.sdk.transforms.DoFn.BoundedPerElement, org.apache.beam.sdk.transforms.DoFn.BundleFinalizer, org.apache.beam.sdk.transforms.DoFn.CurrentRecordId, org.apache.beam.sdk.transforms.DoFn.CurrentRecordOffset, org.apache.beam.sdk.transforms.DoFn.Element, org.apache.beam.sdk.transforms.DoFn.FieldAccess, org.apache.beam.sdk.transforms.DoFn.FinishBundle, org.apache.beam.sdk.transforms.DoFn.FinishBundleContext, org.apache.beam.sdk.transforms.DoFn.FireTimestamp, org.apache.beam.sdk.transforms.DoFn.GetInitialRestriction, org.apache.beam.sdk.transforms.DoFn.GetInitialWatermarkEstimatorState, org.apache.beam.sdk.transforms.DoFn.GetRestrictionCoder, org.apache.beam.sdk.transforms.DoFn.GetSize, org.apache.beam.sdk.transforms.DoFn.GetWatermarkEstimatorStateCoder, org.apache.beam.sdk.transforms.DoFn.Key, org.apache.beam.sdk.transforms.DoFn.MultiOutputReceiver, org.apache.beam.sdk.transforms.DoFn.NewTracker, org.apache.beam.sdk.transforms.DoFn.NewWatermarkEstimator, org.apache.beam.sdk.transforms.DoFn.OnTimer, org.apache.beam.sdk.transforms.DoFn.OnTimerContext, org.apache.beam.sdk.transforms.DoFn.OnTimerFamily, org.apache.beam.sdk.transforms.DoFn.OnWindowExpiration, org.apache.beam.sdk.transforms.DoFn.OnWindowExpirationContext, org.apache.beam.sdk.transforms.DoFn.OutputReceiver<T extends java.lang.Object>, org.apache.beam.sdk.transforms.DoFn.ProcessContext, org.apache.beam.sdk.transforms.DoFn.ProcessContinuation, org.apache.beam.sdk.transforms.DoFn.ProcessElement, org.apache.beam.sdk.transforms.DoFn.RequiresStableInput, org.apache.beam.sdk.transforms.DoFn.RequiresTimeSortedInput, org.apache.beam.sdk.transforms.DoFn.Restriction, org.apache.beam.sdk.transforms.DoFn.Setup, org.apache.beam.sdk.transforms.DoFn.SideInput, org.apache.beam.sdk.transforms.DoFn.SplitRestriction, org.apache.beam.sdk.transforms.DoFn.StartBundle, org.apache.beam.sdk.transforms.DoFn.StartBundleContext, org.apache.beam.sdk.transforms.DoFn.StateId, org.apache.beam.sdk.transforms.DoFn.Teardown, org.apache.beam.sdk.transforms.DoFn.TimerFamily, org.apache.beam.sdk.transforms.DoFn.TimerId, org.apache.beam.sdk.transforms.DoFn.Timestamp, org.apache.beam.sdk.transforms.DoFn.TruncateRestriction, org.apache.beam.sdk.transforms.DoFn.UnboundedPerElement, org.apache.beam.sdk.transforms.DoFn.WatermarkEstimatorState, org.apache.beam.sdk.transforms.DoFn.WindowedContext

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void onTeardown()  
      void processElement​(org.apache.beam.sdk.values.KV<DestinationT,​com.google.cloud.bigquery.storage.v1.TableSchema> element, org.apache.beam.sdk.transforms.DoFn.OutputReceiver<org.apache.beam.sdk.values.KV<DestinationT,​@Nullable ElementT>> o, org.apache.beam.sdk.transforms.DoFn.ProcessContext context, org.apache.beam.sdk.options.PipelineOptions pipelineOptions)  

      • Methods inherited from class org.apache.beam.sdk.transforms.DoFn

        getAllowedTimestampSkew, getInputTypeDescriptor, getOutputTypeDescriptor, populateDisplayData, prepareForProcessing

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Method Detail

      • onTeardown

        @Teardown
public void onTeardown()

      • processElement

        @ProcessElement
public void processElement​(@Element
                           org.apache.beam.sdk.values.KV<DestinationT,​com.google.cloud.bigquery.storage.v1.TableSchema> element,
                           org.apache.beam.sdk.transforms.DoFn.OutputReceiver<org.apache.beam.sdk.values.KV<DestinationT,​@Nullable ElementT>> o,
                           org.apache.beam.sdk.transforms.DoFn.ProcessContext context,
                           org.apache.beam.sdk.options.PipelineOptions pipelineOptions)
                    throws java.lang.Exception
        Throws:
        java.lang.Exception