Package com.google.cloud.hadoop.gcsio

Class PerformanceCachingGoogleCloudStorage

java.lang.Object

com.google.cloud.hadoop.gcsio.ForwardingGoogleCloudStorage

com.google.cloud.hadoop.gcsio.PerformanceCachingGoogleCloudStorage

All Implemented Interfaces:

GoogleCloudStorage

public class PerformanceCachingGoogleCloudStorage
extends ForwardingGoogleCloudStorage

This class adds a caching layer around a GoogleCloudStorage instance, caching calls that create, update, remove, and query for GoogleCloudStorageItemInfo. Those cached copies are returned when requesting data through GoogleCloudStorage.getItemInfo(StorageResourceId) and GoogleCloudStorage.getItemInfo(StorageResourceId). This provides faster access to recently queried data in the scope of this instance. Because the data is cached, modifications made outside of this instance may not be immediately reflected.

Nested Class Summary

Nested classes/interfaces inherited from interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

GoogleCloudStorage.ListPage<T>

Field Summary

Fields inherited from interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

LIST_MAX_RESULTS, MAX_COMPOSE_OBJECTS, MAX_RESULTS_UNLIMITED, PATH_DELIMITER

Constructor Summary

Constructors

Constructor	Description
PerformanceCachingGoogleCloudStorage(GoogleCloudStorage delegate, PerformanceCachingGoogleCloudStorageOptions options)	Creates a wrapper around a GoogleCloudStorage instance, caching calls that create, update, remove, and query for GoogleCloudStorageItemInfo.

Method Summary

Modifier and Type	Method	Description
void	close()	Releases resources used by this instance.
GoogleCloudStorageItemInfo	composeObjects(List<StorageResourceId> sources, StorageResourceId destination, CreateObjectOptions options)	Composes inputs into a single GCS object.
WritableByteChannel	create(StorageResourceId resourceId, CreateObjectOptions options)	Creates and opens an object for writing.
void	deleteBuckets(List<String> bucketNames)	Deletes a list of buckets.
void	deleteObjects(List<StorageResourceId> resourceIds)	Deletes the given objects.
GoogleCloudStorageItemInfo	getItemInfo(StorageResourceId resourceId)	This function may return cached copies of GoogleCloudStorageItemInfo.
List<GoogleCloudStorageItemInfo>	getItemInfos(List<StorageResourceId> resourceIds)	This function may return cached copies of GoogleCloudStorageItemInfo.
void	invalidateCache()
List<GoogleCloudStorageItemInfo>	listBucketInfo()	Gets a list of GoogleCloudStorageItemInfo for all buckets of this project.
List<GoogleCloudStorageItemInfo>	listObjectInfo(String bucketName, String objectNamePrefix, ListObjectOptions listOptions)	This function may return cached copies of GoogleCloudStorageItemInfo.
GoogleCloudStorage.ListPage<GoogleCloudStorageItemInfo>	listObjectInfoPage(String bucketName, String objectNamePrefix, ListObjectOptions listOptions, String pageToken)	The same semantics as GoogleCloudStorage.listObjectInfo(java.lang.String, java.lang.String), but returns only result of single list request (1 page).
void	move(Map<StorageResourceId,StorageResourceId> sourceToDestinationObjectsMap)	Moves objects within the same bucket.
List<GoogleCloudStorageItemInfo>	updateItems(List<UpdatableItemInfo> itemInfoList)	Attempt to update metadata of the objects referenced within the passed itemInfo objects.

Methods inherited from class com.google.cloud.hadoop.gcsio.ForwardingGoogleCloudStorage

compose, copy, copy, createBucket, createEmptyObject, createEmptyObject, createEmptyObjects, createEmptyObjects, createFolder, deleteFolders, getDelegate, getFolderInfo, getOptions, getStatistics, isHnBucket, listBucketNames, listFolderInfoForPrefixPage, listObjectInfoStartingFrom, open, open, renameHnFolder

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

create, createBucket, listObjectInfo, listObjectInfoPage, listObjectInfoStartingFrom, open, open

Constructor Detail

PerformanceCachingGoogleCloudStorage

public PerformanceCachingGoogleCloudStorage(GoogleCloudStorage delegate,
                                            PerformanceCachingGoogleCloudStorageOptions options)

Creates a wrapper around a GoogleCloudStorage instance, caching calls that create, update, remove, and query for GoogleCloudStorageItemInfo. Those cached copies are returned when requesting data through GoogleCloudStorage.getItemInfo(StorageResourceId) and GoogleCloudStorage.getItemInfo(StorageResourceId). This provides faster access to recently queried data in the scope of this instance. Because the data is cached, modifications made outside of this instance may not be immediately reflected.

Parameters:

delegate - the GoogleCloudStorage instance to wrap and delegate calls to.

options - the options to configure this cache with.

Method Detail

create

public WritableByteChannel create(StorageResourceId resourceId,
                                  CreateObjectOptions options)
                           throws IOException

Description copied from interface: GoogleCloudStorage

Creates and opens an object for writing. The bucket must already exist. If resourceId contains a known generationId or a generationId of 0, it is used instead of any "overwrite" settings in the provided CreateObjectOptions (overwrite will only succeed if the provided generationId matches the existing object, where a generationId of 0 indicates no existing object expected).

Specified by:

create in interface GoogleCloudStorage

Overrides:

create in class ForwardingGoogleCloudStorage

Parameters:

resourceId - identifies a StorageObject

options - Options to use when creating the object

Returns:

a channel for writing to the given object

Throws:

IOException - on IO error

deleteBuckets

public void deleteBuckets(List<String> bucketNames)
                   throws IOException

Description copied from interface: GoogleCloudStorage

Deletes a list of buckets. Does not throw any exception for "bucket not found" errors.

Specified by:

deleteBuckets in interface GoogleCloudStorage

Overrides:

deleteBuckets in class ForwardingGoogleCloudStorage

Parameters:

bucketNames - name of the buckets to delete

Throws:

FileNotFoundException - if the given bucket does not exist

IOException - on IO error

deleteObjects

public void deleteObjects(List<StorageResourceId> resourceIds)
                   throws IOException

Description copied from interface: GoogleCloudStorage

Deletes the given objects. Does not throw any exception for "object not found" errors.

Specified by:

deleteObjects in interface GoogleCloudStorage

Overrides:

deleteObjects in class ForwardingGoogleCloudStorage

Parameters:

resourceIds - names of objects to delete with their respective bucketNames.

Throws:

FileNotFoundException - if the given object does not exist

IOException - if object exists but cannot be deleted

move

public void move(Map<StorageResourceId,StorageResourceId> sourceToDestinationObjectsMap)
          throws IOException

Description copied from interface: GoogleCloudStorage

Moves objects within the same bucket. Moving objects between different buckets is not allowed.

Specified by:

move in interface GoogleCloudStorage

Overrides:

move in class ForwardingGoogleCloudStorage

Parameters:

sourceToDestinationObjectsMap - map of destination objects to be moved, keyed by source

Throws:

FileNotFoundException - if the source object or the destination bucket does not exist

IOException - in all other error cases

listBucketInfo

public List<GoogleCloudStorageItemInfo> listBucketInfo()
                                                throws IOException

Description copied from interface: GoogleCloudStorage

Gets a list of GoogleCloudStorageItemInfo for all buckets of this project. This is no more expensive than calling listBucketNames(), since the list API for buckets already retrieves all the relevant bucket metadata.

Specified by:

listBucketInfo in interface GoogleCloudStorage

Overrides:

listBucketInfo in class ForwardingGoogleCloudStorage

Throws:

IOException

listObjectInfo

public List<GoogleCloudStorageItemInfo> listObjectInfo(String bucketName,
                                                       String objectNamePrefix,
                                                       ListObjectOptions listOptions)
                                                throws IOException

This function may return cached copies of GoogleCloudStorageItemInfo.

Specified by:

listObjectInfo in interface GoogleCloudStorage

Overrides:

listObjectInfo in class ForwardingGoogleCloudStorage

Parameters:

bucketName - bucket name

objectNamePrefix - object name prefix or null if all objects in the bucket are desired

listOptions - options to use when listing objects

Returns:

list of objects

Throws:

IOException - on IO error

listObjectInfoPage

public GoogleCloudStorage.ListPage<GoogleCloudStorageItemInfo> listObjectInfoPage(String bucketName,
                                                                                  String objectNamePrefix,
                                                                                  ListObjectOptions listOptions,
                                                                                  String pageToken)
                                                                           throws IOException

Description copied from interface: GoogleCloudStorage

The same semantics as GoogleCloudStorage.listObjectInfo(java.lang.String, java.lang.String), but returns only result of single list request (1 page).

Specified by:

listObjectInfoPage in interface GoogleCloudStorage

Overrides:

listObjectInfoPage in class ForwardingGoogleCloudStorage

Parameters:

bucketName - bucket name

objectNamePrefix - object name prefix or null if all objects in the bucket are desired

listOptions - options to use when listing objects

pageToken - the page token

Returns:

GoogleCloudStorage.ListPage object with listed GoogleCloudStorageItemInfos and next page token if any

Throws:

IOException - on IO error

getItemInfo

public GoogleCloudStorageItemInfo getItemInfo(StorageResourceId resourceId)
                                       throws IOException

This function may return cached copies of GoogleCloudStorageItemInfo.

Specified by:

getItemInfo in interface GoogleCloudStorage

Overrides:

getItemInfo in class ForwardingGoogleCloudStorage

Parameters:

resourceId - identifies either root, a Bucket, or a StorageObject

Returns:

information about the given item

Throws:

IOException - on IO error

getItemInfos

public List<GoogleCloudStorageItemInfo> getItemInfos(List<StorageResourceId> resourceIds)
                                              throws IOException

This function may return cached copies of GoogleCloudStorageItemInfo.

Specified by:

getItemInfos in interface GoogleCloudStorage

Overrides:

getItemInfos in class ForwardingGoogleCloudStorage

Parameters:

resourceIds - names of the GCS StorageObjects or Buckets for which to retrieve info.

Returns:

information about the given resourceIds.

Throws:

IOException - on IO error

updateItems

public List<GoogleCloudStorageItemInfo> updateItems(List<UpdatableItemInfo> itemInfoList)
                                             throws IOException

Description copied from interface: GoogleCloudStorage

Attempt to update metadata of the objects referenced within the passed itemInfo objects.

Specified by:

updateItems in interface GoogleCloudStorage

Overrides:

updateItems in class ForwardingGoogleCloudStorage

Returns:

Updated GoogleCloudStorageItemInfo objects for the referenced objects.

Throws:

IOException - on IO error

composeObjects

public GoogleCloudStorageItemInfo composeObjects(List<StorageResourceId> sources,
                                                 StorageResourceId destination,
                                                 CreateObjectOptions options)
                                          throws IOException

Description copied from interface: GoogleCloudStorage

Composes inputs into a single GCS object. This performs a GCS Compose. Objects will be composed according to the order they appear in the input. The destination object will have metadata set according to options. Overwrite semantics for the destination object will follow the same semantics as GoogleCloudStorage.create(StorageResourceId, CreateObjectOptions). See GoogleCloudStorage.create(StorageResourceId) for the behavior if StorageResourceId.getGenerationId() is explicitly set. The bucket must be the same for all sources and the destination.

Specified by:

composeObjects in interface GoogleCloudStorage

Overrides:

composeObjects in class ForwardingGoogleCloudStorage

Throws:

IOException

close

public void close()

Description copied from interface: GoogleCloudStorage

Releases resources used by this instance.

Specified by:

close in interface GoogleCloudStorage

Overrides:

close in class ForwardingGoogleCloudStorage

invalidateCache

public void invalidateCache()