com.google.cloud.hadoop.gcsio

Class PerformanceCachingGoogleCloudStorage

java.lang.Object

com.google.cloud.hadoop.gcsio.ForwardingGoogleCloudStorage

com.google.cloud.hadoop.gcsio.PerformanceCachingGoogleCloudStorage

All Implemented Interfaces:

GoogleCloudStorage

public class PerformanceCachingGoogleCloudStorage
extends ForwardingGoogleCloudStorage

This class adds a caching layer around a GoogleCloudStorage instance, caching calls that create, update, remove, and query for GoogleCloudStorageItemInfo. Those cached copies are returned when requesting data through GoogleCloudStorage.getItemInfo(StorageResourceId) and GoogleCloudStorage.getItemInfo(StorageResourceId). This provides faster access to recently queried data in the scope of this instance. Because the data is cached, modifications made outside of this instance may not be immediately reflected.

Nested Class Summary

Nested classes/interfaces inherited from interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

GoogleCloudStorage.ListPage<T>

Field Summary

Fields inherited from interface com.google.cloud.hadoop.gcsio.GoogleCloudStorage

MAX_COMPOSE_OBJECTS, MAX_RESULTS_UNLIMITED, PATH_DELIMITER

Constructor Summary

Constructors

Constructor and Description
PerformanceCachingGoogleCloudStorage(GoogleCloudStorage delegate, PerformanceCachingGoogleCloudStorageOptions options) Creates a wrapper around a GoogleCloudStorage instance, caching calls that create, update, remove, and query for GoogleCloudStorageItemInfo.

Method Summary

Modifier and Type	Method and Description
void	close() Releases resources used by this instance.
GoogleCloudStorageItemInfo	composeObjects(java.util.List<StorageResourceId> sources, StorageResourceId destination, CreateObjectOptions options) Composes inputs into a single GCS object.
void	deleteBuckets(java.util.List<java.lang.String> bucketNames) Deletes a list of buckets.
void	deleteObjects(java.util.List<StorageResourceId> fullObjectNames) Deletes the given objects.
GoogleCloudStorageItemInfo	getItemInfo(StorageResourceId resourceId) This function may return cached copies of GoogleCloudStorageItemInfo.
java.util.List<GoogleCloudStorageItemInfo>	getItemInfos(java.util.List<StorageResourceId> resourceIds) This function may return cached copies of GoogleCloudStorageItemInfo.
void	invalidateCache()
java.util.List<GoogleCloudStorageItemInfo>	listBucketInfo() Gets a list of GoogleCloudStorageItemInfo for all buckets of this project.
java.util.List<GoogleCloudStorageItemInfo>	listObjectInfo(java.lang.String bucketName, java.lang.String objectNamePrefix, java.lang.String delimiter) This function may return cached copies of GoogleCloudStorageItemInfo.
java.util.List<GoogleCloudStorageItemInfo>	listObjectInfo(java.lang.String bucketName, java.lang.String objectNamePrefix, java.lang.String delimiter, long maxResults) This function may return cached copies of GoogleCloudStorageItemInfo.
GoogleCloudStorage.ListPage<GoogleCloudStorageItemInfo>	listObjectInfoPage(java.lang.String bucketName, java.lang.String objectNamePrefix, java.lang.String delimiter, java.lang.String pageToken) The same semantics as GoogleCloudStorage.listObjectInfo(java.lang.String, java.lang.String, java.lang.String), but returns only result of single list request (1 page).
java.util.List<GoogleCloudStorageItemInfo>	updateItems(java.util.List<UpdatableItemInfo> itemInfoList) Attempt to update metadata of the objects referenced within the passed itemInfo objects.

Methods inherited from class com.google.cloud.hadoop.gcsio.ForwardingGoogleCloudStorage

compose, copy, create, create, create, create, createEmptyObject, createEmptyObject, createEmptyObjects, createEmptyObjects, getDelegate, getOptions, listBucketNames, listObjectNames, listObjectNames, open, open, waitForBucketEmpty

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PerformanceCachingGoogleCloudStorage

public PerformanceCachingGoogleCloudStorage(GoogleCloudStorage delegate,
                                            PerformanceCachingGoogleCloudStorageOptions options)

Creates a wrapper around a GoogleCloudStorage instance, caching calls that create, update, remove, and query for GoogleCloudStorageItemInfo. Those cached copies are returned when requesting data through GoogleCloudStorage.getItemInfo(StorageResourceId) and GoogleCloudStorage.getItemInfo(StorageResourceId). This provides faster access to recently queried data in the scope of this instance. Because the data is cached, modifications made outside of this instance may not be immediately reflected.

Parameters:

delegate - the GoogleCloudStorage instance to wrap and delegate calls to.

options - the options to configure this cache with.

Method Detail

deleteBuckets

public void deleteBuckets(java.util.List<java.lang.String> bucketNames)
                   throws java.io.IOException

Description copied from interface: GoogleCloudStorage

Deletes a list of buckets. Does not throw any exception for "bucket not found" errors.

Specified by:

deleteBuckets in interface GoogleCloudStorage

Overrides:

deleteBuckets in class ForwardingGoogleCloudStorage

Parameters:

bucketNames - name of the buckets to delete

Throws:

java.io.FileNotFoundException - if the given bucket does not exist

java.io.IOException - on IO error

deleteObjects

public void deleteObjects(java.util.List<StorageResourceId> fullObjectNames)
                   throws java.io.IOException

Description copied from interface: GoogleCloudStorage

Deletes the given objects. Does not throw any exception for "object not found" errors.

Specified by:

deleteObjects in interface GoogleCloudStorage

Overrides:

deleteObjects in class ForwardingGoogleCloudStorage

Parameters:

fullObjectNames - names of objects to delete with their respective bucketNames.

Throws:

java.io.FileNotFoundException - if the given object does not exist

java.io.IOException - if object exists but cannot be deleted

listBucketInfo

public java.util.List<GoogleCloudStorageItemInfo> listBucketInfo()
                                                          throws java.io.IOException

Description copied from interface: GoogleCloudStorage

Gets a list of GoogleCloudStorageItemInfo for all buckets of this project. This is no more expensive than calling listBucketNames(), since the list API for buckets already retrieves all the relevant bucket metadata.

Specified by:

listBucketInfo in interface GoogleCloudStorage

Overrides:

listBucketInfo in class ForwardingGoogleCloudStorage

Throws:

java.io.IOException

listObjectInfo

public java.util.List<GoogleCloudStorageItemInfo> listObjectInfo(java.lang.String bucketName,
                                                                 java.lang.String objectNamePrefix,
                                                                 java.lang.String delimiter)
                                                          throws java.io.IOException

This function may return cached copies of GoogleCloudStorageItemInfo.

Specified by:

listObjectInfo in interface GoogleCloudStorage

Overrides:

listObjectInfo in class ForwardingGoogleCloudStorage

Parameters:

bucketName - bucket name

objectNamePrefix - object name prefix or null if all objects in the bucket are desired

delimiter - delimiter to use (typically "/"), otherwise null

Returns:

list of object info

Throws:

java.io.IOException - on IO error

listObjectInfo

public java.util.List<GoogleCloudStorageItemInfo> listObjectInfo(java.lang.String bucketName,
                                                                 java.lang.String objectNamePrefix,
                                                                 java.lang.String delimiter,
                                                                 long maxResults)
                                                          throws java.io.IOException

This function may return cached copies of GoogleCloudStorageItemInfo.

Specified by:

listObjectInfo in interface GoogleCloudStorage

Overrides:

listObjectInfo in class ForwardingGoogleCloudStorage

Parameters:

bucketName - bucket name

objectNamePrefix - object name prefix or null if all objects in the bucket are desired

delimiter - delimiter to use (typically "/"), otherwise null

maxResults - maximum number of results to return, unlimited if negative or zero

Returns:

list of object info

Throws:

java.io.IOException - on IO error

listObjectInfoPage

public GoogleCloudStorage.ListPage<GoogleCloudStorageItemInfo> listObjectInfoPage(java.lang.String bucketName,
                                                                                  java.lang.String objectNamePrefix,
                                                                                  java.lang.String delimiter,
                                                                                  java.lang.String pageToken)
                                                                           throws java.io.IOException

Description copied from interface: GoogleCloudStorage

The same semantics as GoogleCloudStorage.listObjectInfo(java.lang.String, java.lang.String, java.lang.String), but returns only result of single list request (1 page).

Specified by:

listObjectInfoPage in interface GoogleCloudStorage

Overrides:

listObjectInfoPage in class ForwardingGoogleCloudStorage

Parameters:

bucketName - bucket name

objectNamePrefix - object name prefix or null if all objects in the bucket are desired

delimiter - delimiter to use (typically "/"), otherwise null

pageToken - the page token

Returns:

GoogleCloudStorage.ListPage object with listed GoogleCloudStorageItemInfos and next page token if any

Throws:

java.io.IOException - on IO error

getItemInfo

public GoogleCloudStorageItemInfo getItemInfo(StorageResourceId resourceId)
                                       throws java.io.IOException

This function may return cached copies of GoogleCloudStorageItemInfo.

Specified by:

getItemInfo in interface GoogleCloudStorage

Overrides:

getItemInfo in class ForwardingGoogleCloudStorage

Parameters:

resourceId - identifies either root, a Bucket, or a StorageObject

Returns:

information about the given item

Throws:

java.io.IOException - on IO error

getItemInfos

public java.util.List<GoogleCloudStorageItemInfo> getItemInfos(java.util.List<StorageResourceId> resourceIds)
                                                        throws java.io.IOException

This function may return cached copies of GoogleCloudStorageItemInfo.

Specified by:

getItemInfos in interface GoogleCloudStorage

Overrides:

getItemInfos in class ForwardingGoogleCloudStorage

Parameters:

resourceIds - names of the GCS StorageObjects or Buckets for which to retrieve info.

Returns:

information about the given resourceIds.

Throws:

java.io.IOException - on IO error

updateItems

public java.util.List<GoogleCloudStorageItemInfo> updateItems(java.util.List<UpdatableItemInfo> itemInfoList)
                                                       throws java.io.IOException

Description copied from interface: GoogleCloudStorage

Attempt to update metadata of the objects referenced within the passed itemInfo objects.

Specified by:

updateItems in interface GoogleCloudStorage

Overrides:

updateItems in class ForwardingGoogleCloudStorage

Returns:

Updated GoogleCloudStorageItemInfo objects for the referenced objects.

Throws:

java.io.IOException - on IO error

close

public void close()

Description copied from interface: GoogleCloudStorage

Releases resources used by this instance.

Specified by:

close in interface GoogleCloudStorage

Overrides:

close in class ForwardingGoogleCloudStorage

composeObjects

public GoogleCloudStorageItemInfo composeObjects(java.util.List<StorageResourceId> sources,
                                                 StorageResourceId destination,
                                                 CreateObjectOptions options)
                                          throws java.io.IOException

Description copied from interface: GoogleCloudStorage

Composes inputs into a single GCS object. This performs a GCS Compose. Objects will be composed according to the order they appear in the input. The destination object will have metadata set according to options. Overwrite semantics for the destination object will follow the same semantics as GoogleCloudStorage.create(StorageResourceId, CreateObjectOptions). See GoogleCloudStorage.create(StorageResourceId) for the behavior if StorageResourceId.getGenerationId() is explicitly set. The bucket must be the same for all sources and the destination.

Specified by:

composeObjects in interface GoogleCloudStorage

Overrides:

composeObjects in class ForwardingGoogleCloudStorage

Throws:

java.io.IOException

invalidateCache

public void invalidateCache()