Java S3 Example

In this tutorial I will explain how to use Amazon’s S3 storage with the Java API provided by Amazon. The example shows you how to create a bucket, list it’s content, create a folder into a bucket, upload a file, give the file a public access and finally how to delete all this items.

amazon s3 java example

Setting Up Your Project

  1. You will need the AWS SDK for Java for this example to work. If you didn’t downloaded the SDK by now, please download it here. You will also need to integrate the .JAR files from the archive into your project. Alternately you can use Maven with following dependency
    <dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>aws-java-sdk</artifactId>
        <version>1.9.2</version>
    </dependency>
  2. Create a user in Amazon IAM (https://console.aws.amazon.com/iam) if you don’t have one. Here you will get a “Access key” and “Secret Access Key”. You will need this credentials to connect to S3.
  3. Add the “AWSConnector” and “AmazonS3FullAccess” permissions in IAM to your new user. Without this you want be able to authenticate with the server.

Authenticate with Amazon S3

There are 4 different methods to authenticate your requests against Amazon S3

1. Use the default credential profiles file – this is the recommend way by Amazon. Create a file with following structure and fill in the access keys:

# Move this credentials file to (~/.aws/credentials)
# after you fill in your access and secret keys in the default profile

# WARNING: To avoid accidental leakage of your credentials,
#          DO NOT keep this file in your source directory.
[default]
aws_access_key_id=
aws_secret_access_key=

save this file under the file-name “credentials” in your .aws folder by default C:\Users\user\.aws\credentials for Windows users or your home directory in Linux

If you use this method you can create a Credentials object in your code like this:

AWSCredentials credentials = new ProfileCredentialsProvider().getCredentials();

2. Use Environment Variables – set the values of following environment variables in your system AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY

3. Java System Propertiesaws.accessKeyId and aws.secretKey. Use SystemPropertiesCredentialsProvider to load the variables in your program

4. Programmatically set credentials – in this example I will use this method, because it is easier to follow

Use following in your code:

AWSCredentials credentials = new BasicAWSCredentials("YourAccessKeyID", "YourSecretAccessKey");

Create S3 Client

To be able to communicate with S3 you have to use an implementation of AmazonS3. You will use the instance to address requests to the server

AmazonS3 s3client = new AmazonS3Client(credentials);

Create Bucket

Buckets must have unique names within the whole S3 realm

String bucketName = "javatutorial-net-example-bucket";
s3client.createBucket(bucketName);

List Buckets

You can get the list off all buckets like this

for (Bucket bucket : s3client.listBuckets()) {
	System.out.println(" - " + bucket.getName());
}

Create Folder in S3 Bucket

Use this code to create an empty folder in your bucket

public static void createFolder(String bucketName, String folderName, AmazonS3 client) {
	// create meta-data for your folder and set content-length to 0
	ObjectMetadata metadata = new ObjectMetadata();
	metadata.setContentLength(0);

	// create empty content
	InputStream emptyContent = new ByteArrayInputStream(new byte[0]);

	// create a PutObjectRequest passing the folder name suffixed by /
	PutObjectRequest putObjectRequest = new PutObjectRequest(bucketName,
				folderName + SUFFIX, emptyContent, metadata);

	// send request to S3 to create folder
	client.putObject(putObjectRequest);
}

Upload file in S3

If you want to upload a file into a folder use this

String fileName = folderName + SUFFIX + "testvideo.mp4";
s3client.putObject(new PutObjectRequest(bucketName, fileName, 
		new File("C:\\Users\\user\\Desktop\\testvideo.mp4")));

Just remove the folder and the suffix in the file-name to upload directly to the bucket. If you want to make your file public (files are private by default in Amazon S3) set this to your PutObjectRequest (see the complete example below for more details)

.withCannedAcl(CannedAccessControlList.PublicRead)

Deleting Files, Folders and Buckets

To delete a bucket use this. Buckets must be empty or you can’t delete them

s3client.deleteBucket(bucketName);

To delete files use:

s3client.deleteObject(bucketName, fileName);

To delete a folder you have to delete all files in it first. Please look at the complete example below for more info.




Complete Example

Here you will find all the snippets above in one working program. Exception handling is eexcluded for readability, please don’t forget to add it in your code

import java.io.ByteArrayInputStream;
import java.io.File;
import java.io.InputStream;
import java.util.List;

import com.amazonaws.auth.AWSCredentials;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3Client;
import com.amazonaws.services.s3.model.Bucket;
import com.amazonaws.services.s3.model.CannedAccessControlList;
import com.amazonaws.services.s3.model.ObjectMetadata;
import com.amazonaws.services.s3.model.PutObjectRequest;
import com.amazonaws.services.s3.model.S3ObjectSummary;

public class AmazonS3Example {
	
	private static final String SUFFIX = "/";
	
	public static void main(String[] args) {
		// credentials object identifying user for authentication
		// user must have AWSConnector and AmazonS3FullAccess for 
		// this example to work
		AWSCredentials credentials = new BasicAWSCredentials(
				"YourAccessKeyID", 
				"YourSecretAccessKey");
		
		// create a client connection based on credentials
		AmazonS3 s3client = new AmazonS3Client(credentials);
		
		// create bucket - name must be unique for all S3 users
		String bucketName = "javatutorial-net-example-bucket";
		s3client.createBucket(bucketName);
		
		// list buckets
		for (Bucket bucket : s3client.listBuckets()) {
			System.out.println(" - " + bucket.getName());
		}
		
		// create folder into bucket
		String folderName = "testfolder";
		createFolder(bucketName, folderName, s3client);
		
		// upload file to folder and set it to public
		String fileName = folderName + SUFFIX + "testvideo.mp4";
		s3client.putObject(new PutObjectRequest(bucketName, fileName, 
				new File("C:\\Users\\user\\Desktop\\testvideo.mp4"))
				.withCannedAcl(CannedAccessControlList.PublicRead));
		
		deleteFolder(bucketName, folderName, s3client);
		
		// deletes bucket
		s3client.deleteBucket(bucketName);
	}
	
	public static void createFolder(String bucketName, String folderName, AmazonS3 client) {
		// create meta-data for your folder and set content-length to 0
		ObjectMetadata metadata = new ObjectMetadata();
		metadata.setContentLength(0);

		// create empty content
		InputStream emptyContent = new ByteArrayInputStream(new byte[0]);

		// create a PutObjectRequest passing the folder name suffixed by /
		PutObjectRequest putObjectRequest = new PutObjectRequest(bucketName,
				folderName + SUFFIX, emptyContent, metadata);

		// send request to S3 to create folder
		client.putObject(putObjectRequest);
	}

	/**
	 * This method first deletes all the files in given folder and than the
	 * folder itself
	 */
	public static void deleteFolder(String bucketName, String folderName, AmazonS3 client) {
		List fileList = 
				client.listObjects(bucketName, folderName).getObjectSummaries();
		for (S3ObjectSummary file : fileList) {
			client.deleteObject(bucketName, file.getKey());
		}
		client.deleteObject(bucketName, folderName);
	}
}

If you find this tutorial helpful, please check out our other tutorials – we are sure you will find other interesting stuff on this page. Feel free to ask questions in “comments” section or just show us some love by sharing this tutorial 🙂

4.7 9 votes
Article Rating
guest
21 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
abhay.ku
abhay.ku
7 years ago

I have added aws-java-sdk-1.9.7.jar jar in my project but still its not working. Can you please list out the jar file require to setting up the project.

bhavani
bhavani
5 years ago

I get java.lang.ClassNotFoundException: org.apache.commons.logging.LogFactory exception on line AmazonS3 s3client = new AmazonS3Client(credentials);

Any ideas please?

I am struggling to export a file to local system in my java web application. Please help!

janiparthiv
janiparthiv
5 years ago

Hi,

This is really helpful. was stuck with creating credentials for s3client.

Thanks

kumar.vimal0503
kumar.vimal0503
5 years ago

I have implemented above code and also have added all mentioned jars in my classpath.
I am getting following error: Please help

Exception in thread “main” java.lang.IllegalArgumentException
at com.amazonaws.auth.SignerFactory.createSigner(SignerFactory.java:118)
at com.amazonaws.auth.SignerFactory.lookupAndCreateSigner(SignerFactory.java:107)
at com.amazonaws.auth.SignerFactory.getSigner(SignerFactory.java:80)
at com.amazonaws.AmazonWebServiceClient.computeSignerByServiceRegion(AmazonWebServiceClient.java:311)
at com.amazonaws.AmazonWebServiceClient.computeSignerByURI(AmazonWebServiceClient.java:284)
at com.amazonaws.AmazonWebServiceClient.setEndpoint(AmazonWebServiceClient.java:160)
at com.amazonaws.services.s3.AmazonS3Client.(AmazonS3Client.java:200)
at com.amazonaws.services.s3.AmazonS3Client.(AmazonS3Client.java:182)
at com.aspireiten.util.UploadObjectSingleOperation.main(UploadObjectSingleOperation.java:26)

kumar.vimal0503
kumar.vimal0503
5 years ago

Exception is occurring at line

AmazonS3 s3client = new AmazonS3Client(credentials);

Yogesh
Yogesh
4 years ago

Hi,
very useful and well defined steps.

Thanks

Alok Kumar
Alok Kumar
4 years ago

I am getting below error and i am using aws-java-sdk-1.11.299,
error on AmazonS3 s3client = new AmazonS3Client(credentials);
java.lang.NoClassDefFoundError: Could not initialize class com.amazonaws.http.conn.ssl.SdkTLSSocketFactory

java.lang.NoClassDefFoundError: Could not initialize class com.amazonaws.http.conn.ssl.SdkTLSSocketFactory
at com.amazonaws.http.apache.client.impl.ApacheConnectionManagerFactory.getPreferredSocketFactory(ApacheConnectionManagerFactory.java:88)
at com.amazonaws.http.apache.client.impl.ApacheConnectionManagerFactory.create(ApacheConnectionManagerFactory.java:65)
at com.amazonaws.http.apache.client.impl.ApacheConnectionManagerFactory.create(ApacheConnectionManagerFactory.java:58)
at com.amazonaws.http.apache.client.impl.ApacheHttpClientFactory.create(ApacheHttpClientFactory.java:50)
at com.amazonaws.http.apache.client.impl.ApacheHttpClientFactory.create(ApacheHttpClientFactory.java:38)
at com.amazonaws.http.AmazonHttpClient.(AmazonHttpClient.java:314)
at com.amazonaws.http.AmazonHttpClient.(AmazonHttpClient.java:298)
at com.amazonaws.AmazonWebServiceClient.(AmazonWebServiceClient.java:165)
at com.amazonaws.services.s3.AmazonS3Client.(AmazonS3Client.java:562)
at com.amazonaws.services.s3.AmazonS3Client.(AmazonS3Client.java:542)
at com.amazonaws.services.s3.AmazonS3Client.(AmazonS3Client.java:520)
at com.amazonaws.services.s3.AmazonS3Client.(AmazonS3Client.java:487)
at com.amazonaws.services.s3.AmazonS3Client.(AmazonS3Client.java:466)
at _jsp._clzone._ImageTextDetect__jsp._jspService(clzone/ImageTextDetect.jsp:10)
at _jsp._clzone._ImageTextDetect__jsp._jspService(_ImageTextDetect__jsp.java:53)
at com.caucho.jsp.JavaPage.service(JavaPage.java:64)
at com.caucho.jsp.Page.pageservice(Page.java:548)
at com.caucho.server.dispatch.PageFilterChain.doFilter(PageFilterChain.java:194)
at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:156)
at com.caucho.server.webapp.AccessLogFilterChain.doFilter(AccessLogFilterChain.java:95)
at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:290)
at com.caucho.server.hmux.HmuxRequest.handleInvocation(HmuxRequest.java:476)
at com.caucho.server.hmux.HmuxRequest.handleRequestImpl(HmuxRequest.java:374)
at com.caucho.server.hmux.HmuxRequest.handleRequest(HmuxRequest.java:341)
at com.caucho.network.listen.TcpSocketLink.dispatchRequest(TcpSocketLink.java:1353)
at com.caucho.network.listen.TcpSocketLink.handleRequest(TcpSocketLink.java:1309)
at com.caucho.network.listen.TcpSocketLink.handleRequestsImpl(TcpSocketLink.java:1293)
at com.caucho.network.listen.TcpSocketLink.handleRequests(TcpSocketLink.java:1201)
at com.caucho.network.listen.TcpSocketLink.handleAcceptTaskImpl(TcpSocketLink.java:997)
at com.caucho.network.listen.ConnectionTask.runThread(ConnectionTask.java:117)
at com.caucho.network.listen.ConnectionTask.run(ConnectionTask.java:93)
at com.caucho.network.listen.SocketLinkThreadLauncher.handleTasks(SocketLinkThreadLauncher.java:169)
at com.caucho.network.listen.TcpSocketAcceptThread.run(TcpSocketAcceptThread.java:61)
at com.caucho.env.thread2.ResinThread2.runTasks(ResinThread2.java:173)
at com.caucho.env.thread2.ResinThread2.run(ResinThread2.java:118)

Deepak rawal
Deepak rawal
3 years ago

I am getting this exception :- DEBUG com.amazonaws.internal.config.InternalConfig – Configuration override awssdk_config_override.json not found.
When running this code.

Manu narang
Manu narang
3 years ago

the listing of the buckets doesn’t show the bucket that has the zip files , need to download the zip objects and then modify it and then upload to different bucket

Bhavesh
Bhavesh
3 years ago
Reply to  Manu narang

After putting above code to create a folder and upload a file, I am able to run the program successfully but when I logged into aws console I am able to see only the folder structure and not the file for ex. I see only A/B/C and there is no file in it.

AbandonedGod
AbandonedGod
3 years ago

Couldn’t stop myself from commenting. Thanks a ton :). It worked great. Much needed support for starting.

Shahjade Alam
Shahjade Alam
3 years ago

hi, i am looking java code to worked with amazon s3 and amazon cloudfront CDN to deliver data to viewers.
can anyone give me idea how can i code over them.

vatsalya
vatsalya
3 years ago

It was comming errors like: INFO: Unable to execute HTTP request: Connection reset java.net.SocketException: Connection reset at java.net.SocketInputStream.read(Unknown Source) at java.net.SocketInputStream.read(Unknown Source) at sun.security.ssl.InputRecord.readFully(Unknown Source) at sun.security.ssl.InputRecord.read(Unknown Source) at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source) at sun.security.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source) at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:534) at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:402) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:178) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:610) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:445) at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57) at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:679) at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:454) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:294) at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3658) at com.amazonaws.services.s3.AmazonS3Client.createBucket(AmazonS3Client.java:786) at com.amazonaws.services.s3.AmazonS3Client.createBucket(AmazonS3Client.java:711) at com.amazonaws.samples.bucketcreation.main(bucketcreation.java:28) Dec 31, 2018 3:10:52 PM com.amazonaws.http.AmazonHttpClient executeHelper INFO: Unable to execute HTTP request: Connection reset java.net.SocketException: Connection reset at java.net.SocketInputStream.read(Unknown Source) at java.net.SocketInputStream.read(Unknown Source) at sun.security.ssl.InputRecord.readFully(Unknown Source) at… Read more »

Abdul
Abdul
1 year ago

If i want to move file from one folder(Source) to another folder(destination). While doing this operation, i am getting NPE. No, idea on fixing.

code snipet
==========
AmazonS3URI source = new AmazonS3URI(filePath);
AmazonS3URI destination = new AmazonS3URI(destinationPath);

 CopyObjectRequest copyRequest = new CopyObjectRequest(source.getBucket(),
 source.getKey(), destination.getBucket(), destination.getKey());
  
 
 CopyObjectResult copyResult = s3Client.copyObject(copyRequest);//Getting null on this line.

Please add response asap

Rahul
Rahul
1 year ago

I dont think you need all these dependencies you only need aws-java-sdk-s3 dependency in maven,

srik
srik
1 year ago

Hi I had one query regarding this.after creating the s3 client using AmazonS3 s3client = new AmazonS3Client(credentials);

how do i know if the connection is successful to aws s3.without performing actions like creating a bucket/deleting etc .i want to know if the client object we created is actually connecting to aws s3.please help me with this.are there any methods to know whether the connection is successful or not?
Thank you.

Collin
Collin
1 year ago

Thanks for this

Arit
Arit
10 months ago

Suppose my client is not allowing me to have aws secret access key and key. Then how It will change the code. We are working on SSO login where profile switching is happening. Can you please help me its urgent.