Posting to Amazon SQS with .NET Core

Background

I work for a Microsoft Managed Partner during the day, so I spend a LOT of time working with their cloud, Azure; so to try and stop my self from becoming boxed in to that ecosystem I try to branch out when I am doing stuff outside work. Recently I wrote a small .net core application, that I run on a regular basis on a Linode Linux Server I run in the cloud. It’s great, it only costs a few bucks a month, and is easy to monitor/control.

Recently I thought that instead of logging into the website or SSH-ing into the server itself, it would be really nice if I could just look at a website/app to see

  1. Is it running?
  2. How far through it’s job is it?
  3. How long did the last job take?
  4. Are there any problems?

Really simple stuff. I didn’t want to bother with a lot of overhead, and I am no expert on nginx and couldn’t be bothered adding a web server to my little server, as that brings issues with firewalls, and security that I didn’t have time for.

So I decided to go for a simple “reactive” design and just dump out little messages onto a queue and have something grab those messages and update a status based on the last message it processed. Great, I can separate the “Status Updater” from the main application, avoiding a lot of security issues, and I can play around with various technologies for the “Status Client” without having to make any more changes to the main application itself. Just have it post messages, and it doesn’t need to know what is happening with them.

AWS SQS (Simple Queue Service)

I decided to try out Amazon’s queue offering, SQS. My biggest issue with AWS is that it is fairly unintuitive to get started with if you just want to jump in for one little thing.. I had a lot of trouble getting a basic Elastic Beanstalk site up and running and pointing at a custom domain (I still haven’t got it pointing to the apex domain address).

Basically when you create an application using their AWS SDK for .NET they want you to add a section to an app config with a bunch of details, that it can just read from… well I figured it would be easier to just skip by this, provide creds manually and get something working, then I can always go for better practice when I move it into my application. WRONG. It was a bit of a bitch, as the AWS documentation sends you all over the place for authorising stuff, sending stuff, different clients (which don’t all follow the same design patterns) and most of the tutorials seem to be in Java/Python. All of these things are fine, if you are sitting down for best practice stuff, and know AWS well. But I just wanted to hack together a hello-world console app to see how stuff worked.

I will go on record saying that you probably SHOULD do things their way, there’s other issues with hacking credentials in at compile time… but that’s what I wanted to do.

Creating a Queue and IAM user

This post is really focussed on the code side of things, I followed these steps to create an IAM user for authentication in SQS and then create a queue with that user having full access.

SQS Getting Setup

Creating .NET Core Project

You can download and install .NET Core from the official site here. Simply select your operating system of choice, and follow the instructions.

Create a folder for your project using your UI or a mkdir command, then using command prompt/terminal, init a new project using the command

> $ dotnet new

This will create you a basic console application shell, including a project.json and program.cs file. There are you basic building blocks for the sample application.

I use Visual Studio Code for my .NET Core development (as there is great C# support in the form of extensions, which should be offered to you if you open the folder you created and initialised earlier). Its a great cross platform text editor aimed at being used for development, it’s got built in git support and is great for debugging with. You can get it here.

In your editor of choice, open up project.json and add the AWS SQS Library as a dependancy. It should look like this:

project

You can see that I’ve added version 3.* to mine, meaning when you restore the packages in the project, it will get the highest version with a major version of 3 (i.e. 3.5).

Visual studio code will prompt you to restore dependencies when you edit this file, or you can download the required resources using restore at the command line

> $ dotnet restore

Now your project is all setup and ready to be used.

Posting and reading from the queue

This is a very simple application that posts a message to a queue, and receives one, just by passing the credentials in the code. I have noticed that AWS nicely handles duplicates (Probably handled by the MD5 Hash of the content) so if you send the same message multiple times, it only seems to show up once.

You can get this working for yourself by cloning the repo on GitHub here or simply copy from the sample below.

Checking the status of your queue

If you login to your AWS console, then navigate to SQS, you will be able to select your queue and check the number of messages in it (Don’t be alarmed if you used the sample, saw “Hello world” output but your queue reads 0, as after we have output the message, that sample code removes the message from the queue). If you add more messages, or comment out the removal portion of the code, you will see messages sitting in your queue shown like this

queue

And it’s as simple as that! I would recommend using the AWS best practices and use a credentials file to properly secure your login information, but for a quick test, this will get you up and running and posting to the queue on a cross platform language in no time.

Using webstorm 2016.2.3 with Node JS

I recently ran into a little problem setting up a new Node.js project on a Linux (Ubuntu 16 with Cinnamon) VM.

Trying to create a new application I was asked to pick a ‘Node interpreter’ … no problem I thought, and ran a which command for nodes.

$which nodejs
/usr/bin/nodejs

Awesome… that’s us done right… nope.

untitled
Oh, ok, surely that’s /usr/bin/npm right?

 

untitled
Nope, and don’t call me Shirley.

I will come clean here, I am not a node expert, and this is a bit new to me. I think at some point there has been a change, and the old ‘node’ paths and keywords have changed to ‘nodejs’ and things have moved, thus all my google searches have come up wanting. Either that or I got apt-get happy and installed some weird version of node. When I tried to run ‘node -v’ I got told it wasn’t installed, and that I should could install it by using apt-get and looking for something called ‘node-legacy’ so I figured I had the latest version.

I found a stack overflow post that suggested running ‘npm root -g’

$npm root -g
/usr/local/lib/node_modules

Great right? … only that doesn’t exist on my machine. However, the error said that it couldn’t find ‘npm-cli.js’ under that directory… A quick file search later and I have found it.

/usr/share/npm/bin/npm-cli.js

Bazinga! It worked.. I put in the ‘/usr/share/npm’ setting, and it seems to be happy with that. I am not sure what has happened, if I’ve installed something wrong, or this is just where stuff goes, or I’ve installed it with some weird config. Maybe someone smarter, or more knowledgable than me will come across this and let me know in the comments.

 

Blog at WordPress.com.

Up ↑