S3 URL sharing: simply available
As S3 is a web-based file share, rather than a locally attached block device such as a SCSI disk or thumb drive. Because it is (only) accessible via HTTP, we can choose to direct web clients directly to it, instead of serving it from our web server, thereby offloading the load from the web servers and enjoying the built-in redundancy of S3.
The beauty of it is that it requires absolutely no change from either the web server or the client browser - just be sure to generate your HTML code with absolute paths to the relevant files in S3 and we are good to go:
<img src="http://s3.amazonaws.com/MyBucket/MyPicture.jpeg"/>
Simple has its own limitations
The classic use-case for this feature is where we have a public website serving equally public multimedia content (such as pictures) for anonymous internet clients.
But what happens when we want to implement access-control and authenticate users in our application before we allow them direct access to the content stored on S3?
The bad news is that S3 supports setting file permission ACLs, but it only works with Amazon user accounts (the same ones used for AWS and the Amazon bookstore) - which isn’t really practical to control from inside our application and doesn’t integrate with any existing user database.
The solution is to use an S3 feature called URL Expiration.
URL Expiration to the rescue
S3 allows us to generate a unique URL alias to individual files on S3. This URL can be set to automatically expire, say, after one minute from it’s generation.
And now if we mix together all of the ingredients: it is entirely possible to have a web application that authenticates a user for access to this particular image, an at that point generate a unique URL for that user, set to auto expire one hour later (just like the user session timeout in our web application) with the confidence that we are blocking out unauthenticated users as well as tightly controlling the distribution of these URL (as they auto-expire, users are forced to re-authenticate in order to access the same resources).
Not bulletproof, but pretty close
Some purists will point out that the generated unique URL does open a window of opportunity for a would-be attacker to brute-force his way into that resource, or for the legitimate user to pass-on the URL to a third party.
While these are valid concerns, the reality is that due to the very short lifespan of each URL makes brute-forcing statistically impossible, and as for passing along or intercepting/hijacking the URL from the user - this exposure is no greater than the widely used existing practice of using an HTTP session token.
Conclusion
Although S3 URL Expiration doesn’t bring any new functionality to the table, compared to the existing method of just serving the files directly from the web server - it still holds much value in reducing cost and administrative overhead while greatly improving the scalability and reliability of our generic HTTP application.