Package org.apache.beam.sdk.io.parquet

Class ParquetIO.Read

  • All Implemented Interfaces:
    java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData
    Enclosing class:
    ParquetIO
    public abstract static class ParquetIO.Read
extends org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PBegin,​org.apache.beam.sdk.values.PCollection<org.apache.avro.generic.GenericRecord>>
    Implementation of ParquetIO.read(Schema).
    See Also:
    Serialized Form

    • Field Summary

      • Fields inherited from class org.apache.beam.sdk.transforms.PTransform

        annotations, displayData, name, resourceHints

    • Constructor Summary

      Constructors 
      Constructor Description
      Read()  

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      org.apache.beam.sdk.values.PCollection<org.apache.avro.generic.GenericRecord> expand​(org.apache.beam.sdk.values.PBegin input)  
      ParquetIO.Read from​(java.lang.String filepattern)
      Like from(ValueProvider).
      ParquetIO.Read from​(org.apache.beam.sdk.options.ValueProvider<java.lang.String> filepattern)
      Reads from the given filename or filepattern.
      void populateDisplayData​(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)  
      ParquetIO.Read withAvroDataModel​(org.apache.avro.generic.GenericData model)
      Define the Avro data model; see AvroParquetReader.Builder.withDataModel(GenericData).
      ParquetIO.Read withBeamSchemas​(boolean inferBeamSchema)  
      ParquetIO.Read withConfiguration​(java.util.Map<java.lang.String,​java.lang.String> configuration)
      Specify Hadoop configuration for ParquetReader.
      ParquetIO.Read withConfiguration​(org.apache.hadoop.conf.Configuration configuration)
      Specify Hadoop configuration for ParquetReader.
      ParquetIO.Read withProjection​(org.apache.avro.Schema projectionSchema, org.apache.avro.Schema encoderSchema)
      Enable the reading with projection.

      • Methods inherited from class org.apache.beam.sdk.transforms.PTransform

        addAnnotation, compose, compose, getAdditionalInputs, getAnnotations, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, setDisplayData, setResourceHints, toString, validate, validate

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

    • Constructor Detail

      • Read

        public Read()

    • Method Detail

      • from

        public ParquetIO.Read from​(org.apache.beam.sdk.options.ValueProvider<java.lang.String> filepattern)
        Reads from the given filename or filepattern.

      • withProjection

        public ParquetIO.Read withProjection​(org.apache.avro.Schema projectionSchema,
                                     org.apache.avro.Schema encoderSchema)
        Enable the reading with projection.

      • withConfiguration

        public ParquetIO.Read withConfiguration​(java.util.Map<java.lang.String,​java.lang.String> configuration)
        Specify Hadoop configuration for ParquetReader.

      • withConfiguration

        public ParquetIO.Read withConfiguration​(org.apache.hadoop.conf.Configuration configuration)
        Specify Hadoop configuration for ParquetReader.

      • withBeamSchemas

        public ParquetIO.Read withBeamSchemas​(boolean inferBeamSchema)

      • withAvroDataModel

        public ParquetIO.Read withAvroDataModel​(org.apache.avro.generic.GenericData model)
        Define the Avro data model; see AvroParquetReader.Builder.withDataModel(GenericData).

      • expand

        public org.apache.beam.sdk.values.PCollection<org.apache.avro.generic.GenericRecord> expand​(org.apache.beam.sdk.values.PBegin input)
        Specified by:
        expand in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PBegin,​org.apache.beam.sdk.values.PCollection<org.apache.avro.generic.GenericRecord>>

      • populateDisplayData

        public void populateDisplayData​(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)
        Specified by:
        populateDisplayData in interface org.apache.beam.sdk.transforms.display.HasDisplayData
        Overrides:
        populateDisplayData in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PBegin,​org.apache.beam.sdk.values.PCollection<org.apache.avro.generic.GenericRecord>>