Android and Amazon S3: Secure file storage in the cloud!

Android and Amazon integration

android S3

Amazon S3 is a cloud storage service where you can store any number of objects from anywhere on the internet. We’ll use S3 in this tutorial to store an image uploaded from our Android app and then download and display the image in our app.

Cognito is a sign-up, sign-in, and access control service. We’ll use it in this tutorial to control access to our S3 images. We’ll only allow signed-in users to upload images to our S3 bucket while both signed-in and guest users will be able to download images.

Check out our accompanying youtube version of this tutorial.

You’ll be able to download the code for this tutorial here. – COMING SOON!

I’ll make the following assumptions for the purpose of this tutorial:

  • You have an S3 bucket, if not, it’s pretty easy to set one up. Check out the documentation
  • You have a Cognito User Pool set up, if not, it’s pretty easy to set one up. Check out the documentation

Also check out our User Pool tutorial on youtube

You could also use any authentication provider recognized by Cognito’s Identity Pool:

    • Cognito
    • Amazon
    • Facebook
    • Google+
    • Twitter
    • OpenID
    • SAML
    • Custom

All we need is the authenticated users’ ID token which we will swop for credentials. Check out the documentation

  • You have a Cognito Identity Pool set up, if not, it’s pretty easy to set one up. Check out documentation

Also check out our Identity Pool tutorial on youtube

  • I’ve used Android Studio to create my Android app but you could probably follow along with any IDE

Setting up the app

The manifest file

Include the following permissions in your manifest file:

<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />

You also need to include the TransferService which does the file upload and download in the background. The service stops itself when finished.

<service
android:name="com.amazonaws.mobileconnectors.s3.transferutility.TransferService"
android:enabled="true" />

The build.gradle (Module:app) file

Include the following dependencies:

implementation 'com.amazonaws:aws-android-sdk-cognitoidentityprovider:2.6.29'
implementation 'com.amazonaws:aws-android-sdk-core:2.6.29'

implementation 'com.amazonaws:aws-android-sdk-s3:2.6.29'
implementation ('com.amazonaws:aws-android-sdk-mobile-client:2.6.29@aar'>) { transitive = true }

A custom class

Create a class where you can centralize your user pool id, client id, client secret, Cognito region and identity pool id. You can then create a getUserPool(getter as well as a getCredentialsProvider() getter.

The AWS Mobile configuration file

Create an AWS Mobile configuration file that links your app to your S3 bucket. See how to do that here.

{
"Version": "1.0",
"CredentialsProvider": {
"CognitoIdentity": {
"Default": {
"PoolId": "your identity pool id",
"Region": "your pool region"
}
}
},
"IdentityManager": {
"Default": {}
},
"S3TransferUtility": {
"Default": {
"Bucket": "your bucket",
"Region": "your bucket region"
}
}
}

Here’s a bird’s eye view of our app

We have a single activity, the ShowMeActivity activity which has two buttons and an ImageView. Clicking a button either uploads an image from our apps’ Pictures folder to our S3 bucket or downloads an image from our S3 bucket, saves it in a cache file and displays it in an ImageView.

We use Cognito to allow users to sign-in, and use Cognito’s Identity Pools to issue credentials to control access to the S3 bucket. Only authenticated users can insert images into our S3 bucket while both authenticated and unauthenticated users can retrieve images from our S3 bucket.

It’s all in the code

In our onCreate() method, we call:

AWSMobileClient.getInstance().initialize(this).execute();

This establishes a connection with AWS Mobile and enables our app to interface with other AWS services such as Cognito and S3.

Then we have two buttons, one for upload and the other for download. Pressing a button calls the proceed() method.

The proceed() method

The proceed() method essentially checks whether the user is logged in and updates the credentials, setting the id token for that user if they are. If the user is not logged in, then the credentials default to those for an unauthenticated user.

We then call the upload or download methods which use the given credentials to access the S3 bucket. The credentials have an IAM Role with policies which determine whether or not the given user has permission to save an image in our S3 bucket or to get an image out of our S3 bucket.

Here’s the basic code we used:

Our custom CognitoSettings class

Our CognitoSettings class centralizes our user pool id, client id, client secret, region, and identity pool id.

We’ve also created a few getters in our CognitoSettings class, one of which returns a CognitoCachingCredentialsProvider object, credentialsProvider. This is the default credentials provider, providing the credentials that are cached (if any) else it gets new credentials as per our constructor, which would be for an unauthenticated user.

Something to know about the CognitoCachingCredentialsProvider

The CognitoCachingCredentialsProvider, to reduce the number of network calls, saves the Cognito identity in a Shared Preferences file and also caches the session credentials. Once you have constructed this credentials provider object, it will use the identity saved in the Shared Preferences file and the cached credentials while they are still valid. If not, then it will construct a new identity and set of credentials using the parameters in our constructor.

public CognitoCachingCredentialsProvider getCredentialsProvider() {
return new CognitoCachingCredentialsProvider(
context.getApplicationContext(),
identityPoolId, // Identity pool ID
cognitoRegion// Region;
);
}

We call clear() on the credentials provider in our activity’s onDestroy() method to delete the saved identity in the Shared Preferences file and to destroy the cached credentials. This will force the activity to get a new set of credentials when next the activity is started.

Another getter returns a CognitoUserPool object, which we use to get the currentUser, a CognitoUser object.

public CognitoUserPool getUserPool() {
return new CognitoUserPool(context, userPoolId, clientId
, clientSecret, cognitoRegion);
}

This is the last authenticated user whose tokens are cached on the device. When a user signs out, these cached tokens are removed. Note that these tokens are not the same as the credentials.

We then call getSessionInBackground() to get the valid tokens for the current user. Its’ callback, AuthenticationHandler has an onSuccess() method which has a CognitoUserSession parameter, userSession which we use to get the tokens:

if (userSession.isValid()) {
Log.i(TAG, "user session valid, getting token...");
// Get id token from CognitoUserSession.
String idToken = userSession.getIdToken().getJWTToken();
if (idToken.length() > 0) {
// Set up as a credentials provider.
Log.i(TAG, "got id token - setting credentials using token");
Map<String, String> logins = new HashMap<>();
logins.put("cognito-idp.eu-west-1.amazonaws.com/eu-west-1_2n6uKeWCd", idToken);
credentialsProvider.setLogins(logins);

Log.i(TAG, "using credentials for the logged in user");

/*refresh provider off main thread*/
Log.i(TAG, "refreshing credentials provider in asynctask..");
new RefreshAsyncTask().execute(action);

 
} else {
Log.i(TAG, "no token...");
}
} else {
Log.i(TAG, "user session not valid - using identity pool credentials - guest user");
}
 

We need the ID token to be able to get credentials for this authenticated user from the credentials provider. If we have a valid token then we need to pass it to the credentials provider by calling setLogins(), passing our Identity Pool id and id token as parameters. We then call refresh() in an AsyncTask to update the credentials provider. We now have credentials for the authenticated user.

If the users’ id token is not valid, then we use the default credentials. Remember that the credentials are cached so they would be whatever is in the cache.

Once we have the credentials, we call the performAction() method passing an integer parameter indicating whether we are to upload or download an image. The performAction() method simply calls either uploadWithTransferUtility() or downloadWithTransferUtility(), depending on its parameter.

So here’s what we need to know about Roles and Policies

Users of our app can either log in, which means they are authenticated users. Or if they are not logged in, then they are unauthenticated users. Both users can request credentials from our Identity Pool. These credentials are associated with a specific IAM Role. You can learn more about IAM or Identity and Access Management in the documentation.

When we created our Cognito Identity Pool, we created two IAM Roles:

  • An Authenticated role which is associated with signed-in or authenticated users
  • And an Unauthenticated role which is associated with guests or unauthenticated users

Both these Roles have permission policies attached to them. These policies control what these Roles can or cannot do.

We have two policies in our tutorial:

  • A policy permitting the Role to insert an image into our S3 bucket

{

   "Version": "2012-10-17",

   "Statement": [

       {

           "Sid": "VisualEditor0",

           "Effect": "Allow",

           "Action": "s3:PutObject",

           "Resource": "arn:aws:s3:::xxxx-bucket/*"

       }

   ]

}

  • And a policy permitting a Role to retrieve an image from our S3 bucket

{

   "Version": "2012-10-17",

   "Statement": [

       {

           "Sid": "VisualEditor0",

           "Effect": "Allow",

           "Action": "s3:GetObject",

           "Resource": "arn:aws:s3:::xxxx-bucket/*"

       }

   ]

}

We attached these policies to our Roles:

  • Our authenticated role has both policies attached so users associated with this role can upload and download images
  • Our unauthenticated role has only the policy permitting retrieval attached so these users can only download images

Users and credentials

It will be a good idea to clarify that we are dealing with two different parts of Cognito here.

  • Users that sign-in via Cognito User Pools. They are given an ID token. The ID token does not give them access to any AWS service, including our S3 bucket. We need credentials for that
  • Credentials. Credentials are needed to access any AWS service, including our S3 bucket. Credentials are issued by the Identity Pool in exchange for the users’ id token. If the user is unauthenticated then the credentials provider issues a unique Cognito identity as well as credentials for the unauthenticated user

Uploading and Downloading

The TransferUtility class

We use the TransferUtility class to upload and download the image. The transfer utility class enables applications to upload and download files. It inserts upload and download records into the database and starts a Service to execute the tasks in the background.

TransferUtility transferUtility =
TransferUtility.builder()
.context(getApplicationContext())
.awsConfiguration(AWSMobileClient.getInstance().getConfiguration())
.s3Client(s3Client)
.build();

The TransferUtility object has these three parameters:

  • the application context
  • an awsConfiguration object which sets the region of the S3 client and the default bucket used for uploads and downloads. You therefore don’t have to supply the bucket name to the upload and download methods. These values are retrieved from your res/raw/awsconfiguration.json file
  • the AmazonS3Client

The AmazonS3Client

This is a client enabling us to access our S3 service. We instantiate an AmazonS3Client using the current credentials. The credentials are linked to an IAM Role which has a policy attached to it. The policy determines whether this user is permitted to insert and retrieve images in and out our S3 bucket.

You will get an Access Denied error if the user does not have permission to perform that action.

The TransferObserver

The TransferObserver is used to track the state and progress of a transfer. We set a listener to it to get notified of the progress or when the state changes. In our case, when the upload or download is complete.

The downloadWithTransferUtility() method

We get our S3 client and create a temporary cache file to save the downloaded image.

Then we build our TransferUtility object and create a TransferObserver to track the state and progress of our download.

TransferObserver downloadObserver =
transferUtility.download(
"advert.png", tempCacheFile);

We call download() to start the download, passing the image filename as the key and the tempCacheFile as the file where the downloaded image will be saved. The default S3 bucket is used.

We set a listener on the observer and use its onStateChanged() method to notify us when the download is complete and the file is saved in our temporary cached file. We display the image in an ImageView when the image download is complete.

You can also use its onProgressChanged() method to monitor the progress of the download.

The uploadWithTransferUtility()

We get the path to our Android device’s Pictures directory and then create a new file for the given image in the Pictures directory.

Next we get our S3 client and build a TransferUtility object as well as a TransferObserver object.

Then we start the upload, passing the image filename as the key and the file where the image exists as the parameters.

We use the onStateChanged() method to notify us when the upload is complete.

You can also use the onProgressChanged() method to notify you of the upload progress.

References

Check out our accompanying youtube version of this tutorial.

You can download the code here download35

Check out Amazon's tutorial for uploading and downloading a file.