Package com.google.cloud.hadoop.gcsio

Class GoogleCloudStorageClientImpl

java.lang.Object

com.google.cloud.hadoop.gcsio.ForwardingGoogleCloudStorage

com.google.cloud.hadoop.gcsio.GoogleCloudStorageClientImpl

All Implemented Interfaces:

GoogleCloudStorage

public class GoogleCloudStorageClientImpl
extends ForwardingGoogleCloudStorage

Provides read/write access to Google Cloud Storage (GCS), using Java nio channel semantics. This is a basic implementation of the GoogleCloudStorage interface that mostly delegates through to the appropriate API call(s) google-cloud-storage client.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	GoogleCloudStorageClientImpl.Builder

Nested classes/interfaces inherited from interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

GoogleCloudStorage.ListPage<T>

Field Summary

Fields inherited from interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

LIST_MAX_RESULTS, MAX_COMPOSE_OBJECTS, MAX_RESULTS_UNLIMITED, PATH_DELIMITER

Method Summary

Modifier and Type	Method	Description
static GoogleCloudStorageClientImpl.Builder	builder()
void	close()	Releases resources used by this instance.
void	compose(String bucketName, List<String> sources, String destination, String contentType)	Composes inputs into a single GCS object.
GoogleCloudStorageItemInfo	composeObjects(List<StorageResourceId> sources, StorageResourceId destination, CreateObjectOptions options)	See GoogleCloudStorage.composeObjects(List, StorageResourceId, CreateObjectOptions)} for details about expected behavior.
void	copy(String srcBucketName, List<String> srcObjectNames, String dstBucketName, List<String> dstObjectNames)	See GoogleCloudStorage.copy(String, List, String, List) for details about expected behavior.
void	copy(Map<StorageResourceId,StorageResourceId> sourceToDestinationObjectsMap)	See GoogleCloudStorage.copy(String, List, String, List) for details about expected behavior.
WritableByteChannel	create(StorageResourceId resourceId, CreateObjectOptions options)	Creates and opens an object for writing.
void	createBucket(String bucketName, CreateBucketOptions options)	See GoogleCloudStorage.createBucket(String, CreateBucketOptions) for details about expected behavior.
void	createEmptyObject(StorageResourceId resourceId)	See GoogleCloudStorage.createEmptyObject(StorageResourceId) for details about expected behavior.
void	createEmptyObject(StorageResourceId resourceId, CreateObjectOptions options)	See GoogleCloudStorage.createEmptyObject(StorageResourceId, CreateObjectOptions) for details about expected behavior.
void	createEmptyObjects(List<StorageResourceId> resourceIds)	See GoogleCloudStorage.createEmptyObjects(List) for details about expected behavior.
void	createEmptyObjects(List<StorageResourceId> resourceIds, CreateObjectOptions options)	See GoogleCloudStorage.createEmptyObjects(List, CreateObjectOptions) for details about expected behavior.
void	deleteBuckets(List<String> bucketNames)	See GoogleCloudStorage.deleteBuckets(List) for details about expected behavior.
void	deleteObjects(List<StorageResourceId> fullObjectNames)	See GoogleCloudStorage.deleteObjects(List) for details about the expected behavior.
GoogleCloudStorageItemInfo	getItemInfo(StorageResourceId resourceId)	See GoogleCloudStorage.getItemInfo(StorageResourceId) for details about expected behavior.
List<GoogleCloudStorageItemInfo>	getItemInfos(List<StorageResourceId> resourceIds)	See GoogleCloudStorage.getItemInfos(List) for details about expected behavior.
List<GoogleCloudStorageItemInfo>	listBucketInfo()	See GoogleCloudStorage.listBucketInfo() for details about expected behavior.
List<String>	listBucketNames()	See GoogleCloudStorage.listBucketNames() for details about expected behavior.
void	move(Map<StorageResourceId,StorageResourceId> sourceToDestinationObjectsMap)	See GoogleCloudStorage#move(Map) for details about expected behavior.
SeekableByteChannel	open(GoogleCloudStorageItemInfo itemInfo, GoogleCloudStorageReadOptions readOptions)	Opens an object for reading using GoogleCloudStorageItemInfo.
SeekableByteChannel	open(StorageResourceId resourceId, GoogleCloudStorageReadOptions readOptions)	Opens an object for reading.
List<GoogleCloudStorageItemInfo>	updateItems(List<UpdatableItemInfo> itemInfoList)	Attempt to update metadata of the objects referenced within the passed itemInfo objects.

Methods inherited from class com.google.cloud.hadoop.gcsio.ForwardingGoogleCloudStorage

createFolder, deleteFolders, getDelegate, getFolderInfo, getOptions, getStatistics, isHnBucket, listFolderInfoForPrefixPage, listObjectInfo, listObjectInfoPage, listObjectInfoStartingFrom, renameHnFolder

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

create, createBucket, listObjectInfo, listObjectInfoPage, listObjectInfoStartingFrom, open, open

Method Detail

create

public WritableByteChannel create(StorageResourceId resourceId,
                                  CreateObjectOptions options)
                           throws IOException

Description copied from interface: GoogleCloudStorage

Creates and opens an object for writing. The bucket must already exist. If resourceId contains a known generationId or a generationId of 0, it is used instead of any "overwrite" settings in the provided CreateObjectOptions (overwrite will only succeed if the provided generationId matches the existing object, where a generationId of 0 indicates no existing object expected).

Specified by:

create in interface GoogleCloudStorage

Overrides:

create in class ForwardingGoogleCloudStorage

Parameters:

resourceId - identifies a StorageObject

options - Options to use when creating the object

Returns:

a channel for writing to the given object

Throws:

IOException - on IO error

createBucket

public void createBucket(String bucketName,
                         CreateBucketOptions options)
                  throws IOException

See GoogleCloudStorage.createBucket(String, CreateBucketOptions) for details about expected behavior.

Specified by:

createBucket in interface GoogleCloudStorage

Overrides:

createBucket in class ForwardingGoogleCloudStorage

Parameters:

bucketName - name of the bucket to create

options - options to use when creating bucket

Throws:

IOException - on IO error

createEmptyObject

public void createEmptyObject(StorageResourceId resourceId)
                       throws IOException

See GoogleCloudStorage.createEmptyObject(StorageResourceId) for details about expected behavior.

Specified by:

createEmptyObject in interface GoogleCloudStorage

Overrides:

createEmptyObject in class ForwardingGoogleCloudStorage

Parameters:

resourceId - identifies a StorageObject

Throws:

IOException - on IO error

createEmptyObject

public void createEmptyObject(StorageResourceId resourceId,
                              CreateObjectOptions options)
                       throws IOException

See GoogleCloudStorage.createEmptyObject(StorageResourceId, CreateObjectOptions) for details about expected behavior.

Specified by:

createEmptyObject in interface GoogleCloudStorage

Overrides:

createEmptyObject in class ForwardingGoogleCloudStorage

Parameters:

resourceId - identifies a StorageObject

options - options to use when creating the object

Throws:

IOException - on IO error

createEmptyObjects

public void createEmptyObjects(List<StorageResourceId> resourceIds)
                        throws IOException

See GoogleCloudStorage.createEmptyObjects(List) for details about expected behavior.

Specified by:

createEmptyObjects in interface GoogleCloudStorage

Overrides:

createEmptyObjects in class ForwardingGoogleCloudStorage

Throws:

IOException

createEmptyObjects

public void createEmptyObjects(List<StorageResourceId> resourceIds,
                               CreateObjectOptions options)
                        throws IOException

See GoogleCloudStorage.createEmptyObjects(List, CreateObjectOptions) for details about expected behavior.

Specified by:

createEmptyObjects in interface GoogleCloudStorage

Overrides:

createEmptyObjects in class ForwardingGoogleCloudStorage

Throws:

IOException

copy

public void copy(String srcBucketName,
                 List<String> srcObjectNames,
                 String dstBucketName,
                 List<String> dstObjectNames)
          throws IOException

See GoogleCloudStorage.copy(String, List, String, List) for details about expected behavior.

Specified by:

copy in interface GoogleCloudStorage

Overrides:

copy in class ForwardingGoogleCloudStorage

Parameters:

srcBucketName - name of the bucket containing the objects to copy

srcObjectNames - names of the objects to copy

dstBucketName - name of the bucket to copy to

dstObjectNames - names of the objects after copy

Throws:

FileNotFoundException - if the source object or the destination bucket does not exist

IOException - in all other error cases

move

public void move(Map<StorageResourceId,StorageResourceId> sourceToDestinationObjectsMap)
          throws IOException

See GoogleCloudStorage#move(Map) for details about expected behavior.

Specified by:

move in interface GoogleCloudStorage

Overrides:

move in class ForwardingGoogleCloudStorage

Parameters:

sourceToDestinationObjectsMap - map of destination objects to be moved, keyed by source

Throws:

FileNotFoundException - if the source object or the destination bucket does not exist

IOException - in all other error cases

copy

public void copy(Map<StorageResourceId,StorageResourceId> sourceToDestinationObjectsMap)
          throws IOException

See GoogleCloudStorage.copy(String, List, String, List) for details about expected behavior.

Specified by:

copy in interface GoogleCloudStorage

Overrides:

copy in class ForwardingGoogleCloudStorage

Parameters:

sourceToDestinationObjectsMap - map of destination objects to be copied, keyed by source

Throws:

FileNotFoundException - if the source object or the destination bucket does not exist

IOException - in all other error cases

listBucketNames

public List<String> listBucketNames()
                             throws IOException

See GoogleCloudStorage.listBucketNames() for details about expected behavior.

Specified by:

listBucketNames in interface GoogleCloudStorage

Overrides:

listBucketNames in class ForwardingGoogleCloudStorage

Throws:

IOException

listBucketInfo

public List<GoogleCloudStorageItemInfo> listBucketInfo()
                                                throws IOException

See GoogleCloudStorage.listBucketInfo() for details about expected behavior.

Specified by:

listBucketInfo in interface GoogleCloudStorage

Overrides:

listBucketInfo in class ForwardingGoogleCloudStorage

Throws:

IOException

deleteObjects

public void deleteObjects(List<StorageResourceId> fullObjectNames)
                   throws IOException

See GoogleCloudStorage.deleteObjects(List) for details about the expected behavior.

Specified by:

deleteObjects in interface GoogleCloudStorage

Overrides:

deleteObjects in class ForwardingGoogleCloudStorage

Parameters:

fullObjectNames - names of objects to delete with their respective bucketNames.

Throws:

FileNotFoundException - if the given object does not exist

IOException - if object exists but cannot be deleted

deleteBuckets

public void deleteBuckets(List<String> bucketNames)
                   throws IOException

See GoogleCloudStorage.deleteBuckets(List) for details about expected behavior.

Specified by:

deleteBuckets in interface GoogleCloudStorage

Overrides:

deleteBuckets in class ForwardingGoogleCloudStorage

Parameters:

bucketNames - name of the buckets to delete

Throws:

FileNotFoundException - if the given bucket does not exist

IOException - on IO error

getItemInfos

public List<GoogleCloudStorageItemInfo> getItemInfos(List<StorageResourceId> resourceIds)
                                              throws IOException

See GoogleCloudStorage.getItemInfos(List) for details about expected behavior.

Specified by:

getItemInfos in interface GoogleCloudStorage

Overrides:

getItemInfos in class ForwardingGoogleCloudStorage

Parameters:

resourceIds - names of the GCS StorageObjects or Buckets for which to retrieve info.

Returns:

information about the given resourceIds.

Throws:

IOException - on IO error

getItemInfo

public GoogleCloudStorageItemInfo getItemInfo(StorageResourceId resourceId)
                                       throws IOException

See GoogleCloudStorage.getItemInfo(StorageResourceId) for details about expected behavior.

Specified by:

getItemInfo in interface GoogleCloudStorage

Overrides:

getItemInfo in class ForwardingGoogleCloudStorage

Parameters:

resourceId - identifies either root, a Bucket, or a StorageObject

Returns:

information about the given item

Throws:

IOException - on IO error

open

public SeekableByteChannel open(StorageResourceId resourceId,
                                GoogleCloudStorageReadOptions readOptions)
                         throws IOException

Description copied from interface: GoogleCloudStorage

Opens an object for reading.

Specified by:

open in interface GoogleCloudStorage

Overrides:

open in class ForwardingGoogleCloudStorage

Parameters:

resourceId - identifies a StorageObject

readOptions - Fine-grained options for behaviors of retries, buffering, etc.

Returns:

a channel for reading from the given object

Throws:

FileNotFoundException - if the given object does not exist

IOException - if object exists but cannot be opened

open

public SeekableByteChannel open(GoogleCloudStorageItemInfo itemInfo,
                                GoogleCloudStorageReadOptions readOptions)
                         throws IOException

Description copied from interface: GoogleCloudStorage

Opens an object for reading using GoogleCloudStorageItemInfo.

Specified by:

open in interface GoogleCloudStorage

Overrides:

open in class ForwardingGoogleCloudStorage

Parameters:

itemInfo - identifies a StorageObject

readOptions - Fine-grained options for behaviors of retries, buffering, etc.

Returns:

a channel for reading from the given object

Throws:

FileNotFoundException - if the given object does not exist

IOException - if object exists but cannot be opened

close

public void close()

Description copied from interface: GoogleCloudStorage

Releases resources used by this instance.

Specified by:

close in interface GoogleCloudStorage

Overrides:

close in class ForwardingGoogleCloudStorage

updateItems

public List<GoogleCloudStorageItemInfo> updateItems(List<UpdatableItemInfo> itemInfoList)
                                             throws IOException

Description copied from interface: GoogleCloudStorage

Attempt to update metadata of the objects referenced within the passed itemInfo objects.

Specified by:

updateItems in interface GoogleCloudStorage

Overrides:

updateItems in class ForwardingGoogleCloudStorage

Returns:

Updated GoogleCloudStorageItemInfo objects for the referenced objects.

Throws:

IOException - on IO error

compose

public void compose(String bucketName,
                    List<String> sources,
                    String destination,
                    String contentType)
             throws IOException

Description copied from interface: GoogleCloudStorage

Composes inputs into a single GCS object. This performs a GCS Compose. Objects will be composed according to the order they appear in the input. The destination object, if already present, will be overwritten. Sources and destination are assumed to be in the same bucket.

Specified by:

compose in interface GoogleCloudStorage

Overrides:

compose in class ForwardingGoogleCloudStorage

Parameters:

bucketName - name of the bucket containing the sources and the destination

sources - names of the objects to be composed

destination - name of the resulting object

contentType - content-type of composed object

Throws:

IOException - if the Compose operation was unsuccessful

composeObjects

public GoogleCloudStorageItemInfo composeObjects(List<StorageResourceId> sources,
                                                 StorageResourceId destination,
                                                 CreateObjectOptions options)
                                          throws IOException

See GoogleCloudStorage.composeObjects(List, StorageResourceId, CreateObjectOptions)} for details about expected behavior.

Specified by:

composeObjects in interface GoogleCloudStorage

Overrides:

composeObjects in class ForwardingGoogleCloudStorage

Throws:

IOException

builder

public static GoogleCloudStorageClientImpl.Builder builder()