Background
I work for a Microsoft Managed Partner during the day, so I spend a LOT of time working with their cloud, Azure; so to try and stop my self from becoming boxed in to that ecosystem I try to branch out when I am doing stuff outside work. Recently I wrote a small .net core application, that I run on a regular basis on a Linode Linux Server I run in the cloud. It’s great, it only costs a few bucks a month, and is easy to monitor/control.
Recently I thought that instead of logging into the website or SSH-ing into the server itself, it would be really nice if I could just look at a website/app to see
- Is it running?
- How far through it’s job is it?
- How long did the last job take?
- Are there any problems?
Really simple stuff. I didn’t want to bother with a lot of overhead, and I am no expert on nginx and couldn’t be bothered adding a web server to my little server, as that brings issues with firewalls, and security that I didn’t have time for.
So I decided to go for a simple “reactive” design and just dump out little messages onto a queue and have something grab those messages and update a status based on the last message it processed. Great, I can separate the “Status Updater” from the main application, avoiding a lot of security issues, and I can play around with various technologies for the “Status Client” without having to make any more changes to the main application itself. Just have it post messages, and it doesn’t need to know what is happening with them.
AWS SQS (Simple Queue Service)
I decided to try out Amazon’s queue offering, SQS. My biggest issue with AWS is that it is fairly unintuitive to get started with if you just want to jump in for one little thing.. I had a lot of trouble getting a basic Elastic Beanstalk site up and running and pointing at a custom domain (I still haven’t got it pointing to the apex domain address).
Basically when you create an application using their AWS SDK for .NET they want you to add a section to an app config with a bunch of details, that it can just read from… well I figured it would be easier to just skip by this, provide creds manually and get something working, then I can always go for better practice when I move it into my application. WRONG. It was a bit of a bitch, as the AWS documentation sends you all over the place for authorising stuff, sending stuff, different clients (which don’t all follow the same design patterns) and most of the tutorials seem to be in Java/Python. All of these things are fine, if you are sitting down for best practice stuff, and know AWS well. But I just wanted to hack together a hello-world console app to see how stuff worked.
I will go on record saying that you probably SHOULD do things their way, there’s other issues with hacking credentials in at compile time… but that’s what I wanted to do.
Creating a Queue and IAM user
This post is really focussed on the code side of things, I followed these steps to create an IAM user for authentication in SQS and then create a queue with that user having full access.
Creating .NET Core Project
You can download and install .NET Core from the official site here. Simply select your operating system of choice, and follow the instructions.
Create a folder for your project using your UI or a mkdir command, then using command prompt/terminal, init a new project using the command
> $ dotnet new
This will create you a basic console application shell, including a project.json and program.cs file. There are you basic building blocks for the sample application.
I use Visual Studio Code for my .NET Core development (as there is great C# support in the form of extensions, which should be offered to you if you open the folder you created and initialised earlier). Its a great cross platform text editor aimed at being used for development, it’s got built in git support and is great for debugging with. You can get it here.
In your editor of choice, open up project.json and add the AWS SQS Library as a dependancy. It should look like this:
You can see that I’ve added version 3.* to mine, meaning when you restore the packages in the project, it will get the highest version with a major version of 3 (i.e. 3.5).
Visual studio code will prompt you to restore dependencies when you edit this file, or you can download the required resources using restore at the command line
> $ dotnet restore
Now your project is all setup and ready to be used.
Posting and reading from the queue
This is a very simple application that posts a message to a queue, and receives one, just by passing the credentials in the code. I have noticed that AWS nicely handles duplicates (Probably handled by the MD5 Hash of the content) so if you send the same message multiple times, it only seems to show up once.
You can get this working for yourself by cloning the repo on GitHub here or simply copy from the sample below.
| //the url for our queue | |
| var queueUrl = "https://sqs.eu-west-1.amazonaws.com/%5BUSERID%5D/%5BQUEUENAME%5D"; | |
| Console.WriteLine("Queue Test Starting!"); | |
| Console.WriteLine("Creating Client and request"); | |
| //Create some Credentials with our IAM user | |
| var awsCreds = new BasicAWSCredentials("[ACCESSKEY]", "[SECRETKEY]"); | |
| //Create a client to talk to SQS | |
| var amazonSQSClient = new AmazonSQSClient(awsCreds,Amazon.RegionEndpoint.EUWest1); | |
| //Create the request to send | |
| var sendRequest = new SendMessageRequest(); | |
| sendRequest.QueueUrl = queueUrl; | |
| sendRequest.MessageBody = "{ 'message' : 'hello world' }"; | |
| //Send the message to the queue and wait for the result | |
| Console.WriteLine("Sending Message"); | |
| var sendMessageResponse = amazonSQSClient.SendMessageAsync(sendRequest).Result; | |
| Console.WriteLine("Receiving Message"); | |
| //Create a receive requesdt to see if there are any messages on the queue | |
| var receiveMessageRequest = new ReceiveMessageRequest(); | |
| receiveMessageRequest.QueueUrl = queueUrl; | |
| //Send the receive request and wait for the response | |
| var response = amazonSQSClient.ReceiveMessageAsync(receiveMessageRequest).Result; | |
| //If we have any messages available | |
| if(response.Messages.Any()) | |
| { | |
| foreach(var message in response.Messages) | |
| { | |
| //Spit it out | |
| Console.WriteLine(message.Body); | |
| //Remove it from the queue as we don't want to see it again | |
| var deleteMessageRequest = new DeleteMessageRequest(); | |
| deleteMessageRequest.QueueUrl = queueUrl; | |
| deleteMessageRequest.ReceiptHandle = message.ReceiptHandle; | |
| var result = amazonSQSClient.DeleteMessageAsync(deleteMessageRequest).Result; | |
| } | |
| } |
Checking the status of your queue
If you login to your AWS console, then navigate to SQS, you will be able to select your queue and check the number of messages in it (Don’t be alarmed if you used the sample, saw “Hello world” output but your queue reads 0, as after we have output the message, that sample code removes the message from the queue). If you add more messages, or comment out the removal portion of the code, you will see messages sitting in your queue shown like this
Mark McGookin 
Leave a Reply