org.kuali.maven.wagon

Class S3Wagon

java.lang.Object

org.kuali.maven.wagon.AbstractWagon

org.kuali.maven.wagon.S3Wagon

All Implemented Interfaces:

org.apache.maven.wagon.Wagon, RequestFactory

public class S3Wagon
extends AbstractWagon
implements RequestFactory

An implementation of the Maven Wagon interface that is integrated with the Amazon S3 service.

URLs that reference the S3 service should be in the form of s3://bucket.name. As an example s3://maven.kuali.org puts files into the maven.kuali.org bucket on the S3 service.

This implementation uses the username and password portions of the server authentication metadata for credentials.

Author:

Ben Hale, Jeff Caddel

Field Summary

Fields

Modifier and Type	Field and Description
static int	DEFAULT_DIVISOR
static int	DEFAULT_MAX_THREAD_COUNT
static int	DEFAULT_MIN_THREAD_COUNT
static int	DEFAULT_READ_TIMEOUT
static String	DIVISOR_KEY
static String	HTTP
static String	HTTP_ENDPOINT_VALUE
static String	HTTPS
static String	MAX_THREADS_KEY
static String	MIN_THREADS_KEY
static String	PROTOCOL_KEY Set the system property maven.wagon.protocol to http to force the wagon to communicate over http.

Fields inherited from interface org.apache.maven.wagon.Wagon

DEFAULT_CONNECTION_TIMEOUT, ROLE

Constructor Summary

Constructors

Constructor and Description
S3Wagon()

Method Summary

Modifier and Type	Method and Description
protected void	connectToRepository(org.apache.maven.wagon.repository.Repository source, org.apache.maven.wagon.authentication.AuthenticationInfo auth, org.apache.maven.wagon.proxy.ProxyInfo proxy) Subclass must implement with specific connection behavior
protected void	disconnectFromRepository() Subclasses must implement with specific disconnection behavior
protected boolean	doesRemoteResourceExist(String resourceName) Subclass must implement with specific detection behavior
protected com.amazonaws.services.s3.model.CannedAccessControlList	getAclFromRepository(org.apache.maven.wagon.repository.Repository repository)
protected com.amazonaws.services.s3.AmazonS3Client	getAmazonS3Client(com.amazonaws.auth.AWSCredentials credentials)
protected String	getBaseDir(org.apache.maven.wagon.repository.Repository source) Convert "/" -> "" Convert "/snapshot/" > "snapshot/" Convert "/snapshot" -> "snapshot/"
protected static File	getCanonicalFile(File file)
protected static File	getCanonicalFile(String path)
protected String	getCanonicalKey(String key) Normalize the key to our S3 object: Convert ./css/style.css into /css/style.css Convert /foo/bar/../../css/style.css into /css/style.css
protected com.amazonaws.ClientConfiguration	getClientConfiguration()
protected com.amazonaws.auth.AWSCredentials	getCredentials(org.apache.maven.wagon.authentication.AuthenticationInfo authenticationInfo) Create AWSCredentionals from the information in system properties, environment variables, settings.xml, or EC2 instance metadata (only applicable when running the wagon on an Amazon EC2 instance)
protected String	getDestinationPath(String destination)
protected int	getDivisor()
protected InputStream	getInputStream(File source, org.kuali.maven.wagon.TransferProgress progress)
protected int	getMaxThreads()
protected int	getMinThreads()
protected com.amazonaws.services.s3.model.ObjectMetadata	getObjectMetadata(File source, String destination)
protected PutFileContext	getPutFileContext(File source, String destination)
protected com.amazonaws.services.s3.model.PutObjectRequest	getPutObjectRequest(File source, String destination, org.kuali.maven.wagon.TransferProgress progress) Create a PutObjectRequest based on the source file and destination passed in.
com.amazonaws.services.s3.model.PutObjectRequest	getPutObjectRequest(PutFileContext context) Create a PutObjectRequest based on the PutContext
int	getReadTimeout()
protected int	getRequestsPerThread(int threads, int requests)
protected void	getResource(String resourceName, File destination, org.kuali.maven.wagon.TransferProgress progress) Pull an object out of an S3 bucket and write it to a file
protected String	getUploadCompleteMsg(long millis, long bytes, long count)
protected String	getUploadStartMsg(int fileCount, long bytes)
protected int	getValue(String key, int defaultValue)
protected String	getValue(String key, String defaultValue)
protected void	info(String msg)
protected boolean	isRemoteResourceNewer(String resourceName, long timestamp) Is the S3 object newer than the timestamp passed in?
protected List<String>	listDirectory(String directory) List all of the objects in a given directory
void	putDirectory(File sourceDir, String destinationDir) On S3 there are no true "directories".
protected void	putResource(File source, String destination, org.kuali.maven.wagon.TransferProgress progress) Store a resource into S3
void	setReadTimeout(int readTimeout)
protected long	sum(List<PutFileContext> contexts)
protected void	validateBucket(com.amazonaws.services.s3.AmazonS3Client client, String bucketName)
protected void	validatePermissions(com.amazonaws.services.s3.AmazonS3Client client, String bucketName) Establish that we have enough permissions on this bucket to do what we need to do

Methods inherited from class org.kuali.maven.wagon.AbstractWagon

addSessionListener, addTransferListener, connect, connect, connect, connect, connect, connect, disconnect, doConnect, get, getFileList, getIfNewer, getPutFileContexts, getRepository, getSessionListeners, getTimeout, getTransferListeners, handleException, hasSessionListener, hasTransferListener, isInteractive, openConnection, put, removeSessionListener, removeTransferListener, resourceExists, setInteractive, setTimeout, supportsDirectoryCopy

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PROTOCOL_KEY

public static final String PROTOCOL_KEY

Set the system property maven.wagon.protocol to http to force the wagon to communicate over http. Default is https.

See Also:

Constant Field Values

HTTP

public static final String HTTP

See Also:

Constant Field Values

HTTP_ENDPOINT_VALUE

public static final String HTTP_ENDPOINT_VALUE

See Also:

Constant Field Values

HTTPS

public static final String HTTPS

See Also:

Constant Field Values

MIN_THREADS_KEY

public static final String MIN_THREADS_KEY

See Also:

Constant Field Values

MAX_THREADS_KEY

public static final String MAX_THREADS_KEY

See Also:

Constant Field Values

DIVISOR_KEY

public static final String DIVISOR_KEY

See Also:

Constant Field Values

DEFAULT_MIN_THREAD_COUNT

public static final int DEFAULT_MIN_THREAD_COUNT

See Also:

Constant Field Values

DEFAULT_MAX_THREAD_COUNT

public static final int DEFAULT_MAX_THREAD_COUNT

See Also:

Constant Field Values

DEFAULT_DIVISOR

public static final int DEFAULT_DIVISOR

See Also:

Constant Field Values

DEFAULT_READ_TIMEOUT

public static final int DEFAULT_READ_TIMEOUT

See Also:

Constant Field Values

Constructor Detail

S3Wagon

public S3Wagon()

Method Detail

validateBucket

protected void validateBucket(com.amazonaws.services.s3.AmazonS3Client client,
                              String bucketName)

validatePermissions

protected void validatePermissions(com.amazonaws.services.s3.AmazonS3Client client,
                                   String bucketName)

Establish that we have enough permissions on this bucket to do what we need to do

Parameters:

client - S3 Client

bucketName - AWS S3 Bucket Name.

getAclFromRepository

protected com.amazonaws.services.s3.model.CannedAccessControlList getAclFromRepository(org.apache.maven.wagon.repository.Repository repository)

getClientConfiguration

protected com.amazonaws.ClientConfiguration getClientConfiguration()

getAmazonS3Client

protected com.amazonaws.services.s3.AmazonS3Client getAmazonS3Client(com.amazonaws.auth.AWSCredentials credentials)

connectToRepository

protected void connectToRepository(org.apache.maven.wagon.repository.Repository source,
                                   org.apache.maven.wagon.authentication.AuthenticationInfo auth,
                                   org.apache.maven.wagon.proxy.ProxyInfo proxy)

Description copied from class: AbstractWagon

Subclass must implement with specific connection behavior

Specified by:

connectToRepository in class AbstractWagon

Parameters:

source - The repository connection information

auth - Authentication information, if any

proxy - Proxy information, if any

doesRemoteResourceExist

protected boolean doesRemoteResourceExist(String resourceName)

Description copied from class: AbstractWagon

Subclass must implement with specific detection behavior

Specified by:

doesRemoteResourceExist in class AbstractWagon

Parameters:

resourceName - The remote resource to detect

Returns:

true if the remote resource exists

disconnectFromRepository

protected void disconnectFromRepository()

Description copied from class: AbstractWagon

Subclasses must implement with specific disconnection behavior

Specified by:

disconnectFromRepository in class AbstractWagon

getResource

protected void getResource(String resourceName,
                           File destination,
                           org.kuali.maven.wagon.TransferProgress progress)
                    throws org.apache.maven.wagon.ResourceDoesNotExistException,
                           IOException

Pull an object out of an S3 bucket and write it to a file

Specified by:

getResource in class AbstractWagon

Parameters:

resourceName - The name of the remote resource to read

destination - The local file to write to

progress - A progress notifier for the upload. It must be used or hashes will not be calculated correctly

Throws:

org.apache.maven.wagon.ResourceDoesNotExistException

IOException

isRemoteResourceNewer

protected boolean isRemoteResourceNewer(String resourceName,
                                        long timestamp)

Is the S3 object newer than the timestamp passed in?

Specified by:

isRemoteResourceNewer in class AbstractWagon

Parameters:

resourceName - The name of the resource being compared

timestamp - The timestamp to compare against

Returns:

true if the current version of the resource is newer than the timestamp

listDirectory

protected List<String> listDirectory(String directory)
                              throws Exception

List all of the objects in a given directory

Specified by:

listDirectory in class AbstractWagon

Parameters:

directory - The directory to list files in

Returns:

A collection of file names

Throws:

Exception - Implementations can throw any exception and it will be handled by the base class

info

protected void info(String msg)

getCanonicalKey

protected String getCanonicalKey(String key)

Normalize the key to our S3 object:
Convert ./css/style.css into /css/style.css
Convert /foo/bar/../../css/style.css into /css/style.css

Parameters:

key - S3 Key string.

Returns:

Normalized version of key.

getCanonicalFile

protected static File getCanonicalFile(String path)

getCanonicalFile

protected static File getCanonicalFile(File file)

getObjectMetadata

protected com.amazonaws.services.s3.model.ObjectMetadata getObjectMetadata(File source,
                                                                           String destination)

getPutObjectRequest

public com.amazonaws.services.s3.model.PutObjectRequest getPutObjectRequest(PutFileContext context)

Create a PutObjectRequest based on the PutContext

Specified by:

getPutObjectRequest in interface RequestFactory

getInputStream

protected InputStream getInputStream(File source,
                                     org.kuali.maven.wagon.TransferProgress progress)
                              throws IOException

Throws:

IOException

getPutObjectRequest

protected com.amazonaws.services.s3.model.PutObjectRequest getPutObjectRequest(File source,
                                                                               String destination,
                                                                               org.kuali.maven.wagon.TransferProgress progress)

Create a PutObjectRequest based on the source file and destination passed in.

Parameters:

source - Local file to upload.

destination - Destination S3 key.

progress - Transfer listener.

Returns:

PutObjectRequest instance.

putDirectory

public final void putDirectory(File sourceDir,
                               String destinationDir)
                        throws org.apache.maven.wagon.TransferFailedException

On S3 there are no true "directories". An S3 bucket is essentially a Hashtable of files stored by key. The integration between a traditional file system and an S3 bucket is to use the path of the file on the local file system as the key to the file in the bucket. The S3 bucket does not contain a separate key for the directory itself.

Specified by:

putDirectory in interface org.apache.maven.wagon.Wagon

Throws:

org.apache.maven.wagon.TransferFailedException

getUploadCompleteMsg

protected String getUploadCompleteMsg(long millis,
                                      long bytes,
                                      long count)

getUploadStartMsg

protected String getUploadStartMsg(int fileCount,
                                   long bytes)

getRequestsPerThread

protected int getRequestsPerThread(int threads,
                                   int requests)

sum

protected long sum(List<PutFileContext> contexts)

putResource

protected void putResource(File source,
                           String destination,
                           org.kuali.maven.wagon.TransferProgress progress)
                    throws IOException

Store a resource into S3

Specified by:

putResource in class AbstractWagon

Parameters:

source - The local source file to read from

destination - The name of the remote resource to write to

progress - A progress notifier for the upload. It must be used or hashes will not be calculated correctly

Throws:

IOException

getDestinationPath

protected String getDestinationPath(String destination)

getBaseDir

protected String getBaseDir(org.apache.maven.wagon.repository.Repository source)

Convert "/" -> ""
Convert "/snapshot/" > "snapshot/"
Convert "/snapshot" -> "snapshot/"

Parameters:

source - Repository info.

Returns:

Normalized repository base dir.

getCredentials

protected com.amazonaws.auth.AWSCredentials getCredentials(org.apache.maven.wagon.authentication.AuthenticationInfo authenticationInfo)

Create AWSCredentionals from the information in system properties, environment variables, settings.xml, or EC2 instance metadata (only applicable when running the wagon on an Amazon EC2 instance)

Parameters:

authenticationInfo - Authentication credentials from Maven settings.

Returns:

Resolved AWS Credentials.

getPutFileContext

protected PutFileContext getPutFileContext(File source,
                                           String destination)

Overrides:

getPutFileContext in class AbstractWagon

getMinThreads

protected int getMinThreads()

getMaxThreads

protected int getMaxThreads()

getDivisor

protected int getDivisor()

getValue

protected int getValue(String key,
                       int defaultValue)

getValue

protected String getValue(String key,
                          String defaultValue)

getReadTimeout

public int getReadTimeout()

Specified by:

getReadTimeout in interface org.apache.maven.wagon.Wagon

setReadTimeout

public void setReadTimeout(int readTimeout)

Specified by:

setReadTimeout in interface org.apache.maven.wagon.Wagon