CloudStacking.com

Introduction to Amazon Simple Queue Service (SQS)

The beauty of electronic mail (or plain old snail mail for that matter) is that as the sender, you didn’t need to bother yourself with the the details of how your message reaches it’s destination. After composing the message, all you need to do is just to hand it off to your trusted mailman who’s entire job is to handle the logistics and challenges of mail delivery for you.

This mechanism offers such level of abstraction, that you as a sender are not only not responsible for the actual delivery of the message - you don’t even need (or want) to know how it got to it’s destination (by plane, by truck, by carriage etc.).

Message Queuing (MQ) takes this same concept of relieving the sender from the responsibility of actually delivering the message to it’s destination by offloading it to a dedicated component that specializes in doing exactly that - reliably delivering messages to their destination. It is also the technological cornerstone of SOA (Service Oriented Architecture), but that’s a story for another post.

My applications already communicate just fine, what’s in it for me?

TCP/IP Sockets are natively available in every programming language, and using them in applications is both common practice and common knowledge.

With both the technology and the know-how already in place - it is so very tempting to just use it. After all, why bother to invest in bring something new into the mix (together with the associated overhead of skills and other costs) when we already have a viable way of doing business “for free”?

But, just like many other things in life - “free” often times comes with a hidden price tag:

It places the burden of validating successful delivery (network-wise as well as logically in an application-specific manner) on the sending applications base code, requiring greater programmatic effort to successfully implement the program as a whole.
More code = more bugs. Plain and simple.
That’s not to say that an “off the shelf” middleware is bug-free, but leveraging existing and proven technologies is always safer and easier than writing and maintaining custom code produced in-house, period.
Sockets are opened between two computers: meaning that transferring a message to multiple recipients requires additional coding effort and more importantly - requires the sender to know where to send the message - a requirement that doesn’t always co-exists well with the dynamic nature of Cloud Computing.
Sockets are immediate - meaning that in order to pass on a given message the recipient must be ready and willing to receive it right now.
If the recipient suddenly goes offline or simply unable to immediately process the messages (like an electronic trading system outside trade hours) it is the responsibility of the application to take necessary actions such as buffering the messages until the right time to process them.

Message Queuing is designed to address all of the above and much more - it is a proven concept widely used in the most demanding environments and enjoys numerous implementations such as the stand-alone IBM Websphere MQ, Microsoft Message Queuing (MSMQ) which is bundled with Windows Server, as well as open source implementations like Apache Active MQ.

So how does it work?

Message Queuing revolves around the concept of Queue: a logical FIFO list of messages that share the same logical destination (such as incoming messages from clients to application X, or outgoing messages from application X to application Y). In essence, the Queue’s logical function as a pipeline, enabling messages (read information) to freely and easily flow from one application to another.

Note that a Queue doesn’t explicitly specify servers, as it is a detached and independent entity from any single server - more on that in just a bit.

All we need to do from our application(s) is to invoke the Queue’s “send” method in the sending application (called “Producer Application” in SOA terminology) and similarly invoke the “Receive” method in the receiving application (again, in SOA terminology this application will be called “Consumer Application”) - and that’s literally all there is to it (application-wise).

Message Queuing Diagram

This two-step process in which one application’s send followed by other application’s receive has two very important properties:

It is asynchronous: meaning that any period of time can pass between the send and the receive - in which time the message will exist inside of the queue, waiting for delivery.
This is a very convenient and “free” way of implementing resiliency into our applications as messages aren’t lost when the receiver is down, only queued until the receiver finally does return to normal operation (think of messages in your mailbox while you are away for vacation or sick-leave).
It is also loosely coupled: meaning that the sender does have to be specifically address the messages to any particular recipient - instead they could be independently fetched by the receiver, or load balanced across multiple receivers (think of call-centers, where your call is placed in a queue and will be answered by the first representative who becomes available).

Together, these two properties are well tailored to Cloud-Computing environments where there is a constantly shifting number of senders and receivers that using this method don’t need to actually know of each other - rather they just pivot around this one central Queue who handles both the load balancing and fault tolerance.

So how does AWS do Message Queuing?

Similarly to S3, SQS is an independent HTTP-based service happily living in the AWS cloud, not attached to any instance in particular.

True to it’s name SQS is very simple to set-up and use (at the expense of being feature-poor compared to other MQ implementations), and can be configured and used by any Internet connected computer (inside and outside of EC2).

Below is a working example, taken from the SQS API Reference for sending a message to a Queue via invoking the following HTTP URL:

http://queue.amazonaws.com/123456789012/testQueue/
?Action=SendMessage
&MessageBody=This+is+a+test+message
&Version=2009-02-01
&SignatureMethod=HmacSHA256
&Expires=2009-04-18T22%3A52%3A43PST
&AWSAccessKeyId=0GS7553JW74RRM612K02EXAMPLE
&SignatureVersion=2
&Signature=Dqlp3Sd6ljTUA9Uf6SGtEExwUQEXAMPLE

Similarly, in order to receive my message from from my queue will invoke the following HTTP URL:

http://queue.amazonaws.com/123456789012/testQueue/
?Action=ReceiveMessage
&MaxNumberOfMessages=5
&VisibilityTimeout=15
&AttributeName=All;
&Version=2009-02-01
&SignatureMethod=HmacSHA256
&Expires=2009-04-18T22%3A52%3A43PST
&AWSAccessKeyId=0GS7553JW74RRM612K02EXAMPLE
&SignatureVersion=2
&Signature=Dqlp3Sd6ljTUA9Uf6SGtEExwUQEXAMPLE

And that’s all there is to it, no servers needed to support the Queue and no dedicated client needed needed to tap into it.

Simple.

Conclusion and Further Reading.

Message Queuing is a proven technological concept that is heavily leveraged both independently and as the foundation of larger concepts (such as SOA, or J2EE) to great success. Oftentimes, Message Queuing is the perfect bridge in the gap between the need to keep applications simple (read: “Single Minded”) and yet sophisticated enough to keep track of what’s going on in highly dynamic Cloud-Computing environments.

Stay tuned, in future posts we will drill down the pros, and cons, of SQS.

Introduction

As the goal of Application Virtualization is to decouple the application from the underlying Operating System, we often find ourselves lacking this lost integration in unexpected places. One issue I frequently face with customers is how to configure Windows Explorer’s Sendto shortcut (the one available when you right-click on a file) to actually work with a ThinApped copy of Microsoft Outlook.

To thinreg or not to thinreg?

ThinApp’s method of integrating itself into the OS is via a tool called thinreg.exe - but before we discuss why isn’t it suitable for this particular task we need to understand what it does do: When we create a ThinApp package, we are essentially creating a self-contained application “bubble” which can have any number of “Entry Points” - these Entry Points usually represent the individual sub-components of the application that we want to expose to the user (and as they share the same bubble, they share files, registry and configuration with each other).

In the case of Microsoft Office, we typically want to create an Entry Point for each of the suite’s programs such as: Word, Outlook, Excel and Powerpoint - and the configuration of the Entry Points is captured by the ThinApp composer and placed into the package.ini file.

Let’s go over the output of the outlook.exe entry point as captured in a default Office installation, with the sections that interest us bolded:

[Microsoft Office Outlook 2007.exe]
Source=%ProgramFilesDir%\Microsoft Office\Office12\OUTLOOK.EXE
Shortcut=Microsoft Office Enterprise 2007.dat
Icon=%SystemRoot%\Installer{90120000-0030-0000-0000-0000000FF1CE}\outicon.exe
FileTypes=.hol.ibc.ics.msg.oft.vcf.vcs
Protocols=feed;feeds;mailto;Outlook.URL.feed;Outlook.URL.mailto;Outlook.URL.stssync;Outlook.URL.webcal;outlookfeed;outlookfeeds;stssync;webcal;webcals ObjectTypes=DOCSITE.DocSiteControl.1;MailMsgAtt;Outlook.Application;Outlook.Application.12;Outlook.FileAttach;Outlook.MsgAttach;Outlook.OlkBusinessCardControl;Outlook.OlkBusinessCardControl.1;Outlook.OlkCategoryStrip;Outlook.OlkCategoryStrip.1;Outlook.OlkCheckBox;Outlook.OlkCheckBox.1;Outlook.OlkComboBox;Outlook.OlkComboBox.1;Outlook.OlkCommandButton;Outlook.OlkCommandButton.1;Outlook.OlkContactPhoto;Outlook.OlkContactPhoto.1;Outlook.OlkDateControl;Outlook.OlkDateControl.1;Outlook.OlkFrameHeader;Outlook.OlkFrameHeader.1;Outlook.OlkInfoBar;Outlook.OlkInfoBar.1;Outlook.OlkLabel;Outlook.OlkLabel.1;Outlook.OlkListBox;Outlook.OlkListBox.1;Outlook.OlkOptionButton;Outlook.OlkOptionButton.1;Outlook.OlkPageControl;Outlook.OlkPageControl.1;Outlook.OlkSenderPhoto;Outlook.OlkSenderPhoto.1;Outlook.OlkTextBox;Outlook.OlkTextBox.1;Outlook.OlkTimeControl;Outlook.OlkTimeControl.1;Outlook.OlkTimeZone;Outlook.OlkTimeZone.1;RECIP.RecipCtl.1
Shortcuts=%Programs%\Microsoft Office

These parameters interest us because they define the specifics of how to register with the underlying physical OS.
“FileTypes” indicates what file suffixes will be associated with this Entry Point.
“Protocols” is the association with Internet Explorer URL path suffixes (using http and https has become such a second nature to us, that we tend to forget that we can also browse to other types of addresses such using the same Explorer such as ftp).
“ObjectTypes” registers COM objects from ThinApp in the underlying COM Provider.
“Shortcuts” is the list of the locations in which to place a shortcut to start the Entry Point (such as the user’s Desktop and Programs).

So far so good, but what about the Sendto?

As we’ve learned in the previous section, the functionality to enable Sendto context menu simply doesn’t exist in thinreg - but hope is not lost! We can still manually (or via script) configure the host’s “Send to mail recipient” short cut so that instead of trying to locate a physical installation of a mail client - it will immediately launch our ThinApped Outlook instead.

First, we will need to locate and delete the existing Sendto link (aptly named “mail recipient.MAPIMail”), located under %userprofile%\sendto and create a new shortcut to the path shortcut invoking the following command:

Outlook 2007:

“C:\Path to ThinApp\Outlook.exe” /c ipm.note /a

Outlook 2003:

“C:\Path to ThinApp\Outlook.exe” /c ipm.note

Enough talking - just hand over the answer!

For the sake of practicality, below is a VBScript which will replace the original “mail recipient” with one invoking your ThinApped Outlook. Just copy the text and paste it into a .vbs file and execute the script in the user’s behalf via logon script.

' First, we set the path for the ThinApped Outlook
OutlookPath = "C:\ThinApp\Outlook 2007.exe /c ipm.note /a"

' Don't forget to use these arguments for Outlook 2003
'set OutlookPath = "C:\ThinApp\Outlook 2003.exe /c ipm.note"

' Initialize the necessary objects
set WshShell = WScript.CreateObject("WScript.Shell")
Set objEnv = WshShell.Environment("Process")
SendTo = objEnv("userprofile") & "\sendto"

' Delete the original "mail recipient" as well as any previous "mail recipient" created by a previous run of this script
Set filesys = CreateObject("Scripting.FileSystemObject")
If filesys.FileExists(SendTo & "\Mail Recipient.MAPIMail") Then
 filesys.DeleteFile SendTo & "\Mail Recipient.MAPIMail"
End If 

If filesys.FileExists(SendTo & "\Mail Recipient.lnk") Then <br>
 filesys.DeleteFile SendTo & "\Mail Recipient.lnk" <br>
End If 

set oShortCutLink = WshShell.CreateShortcut(SendTo & "\Mail Recipient.lnk")
oShortCutLink.TargetPath = OutlookPath
oShortCutLink.WindowStyle = 1
oShortCutLink.Hotkey = "CTRL+SHIFT+N"
oShortCutLink.Description = "Send mail via ThinApped Outlook"
oShortCutLink.Save