SQS Compatible Message Queue Service in Open Source Cloud

The open source project SmoothMQ offers a drop-in-replacement for SQS and is now available as a service in Open Source Cloud. In this post we will look at how you can setup a message queue and using the Python boto3 client to push messages on the queue. Service is available in the Free plan and if you don’t have any other active services or instances running you can try it out for free.

Create a message queue service with the Open Source Cloud command line tool osc

% osc create poundifdef-smoothmq blog -o AccessKey=blog SecretKey=blog

It is recommended that instead of providing the access key and secret in clear text you create a service secret and refer to these instead. For example:

% osc create poundifdef-smoothmq blog \
  -o AccessKey="{{secrets.accesskey}}" \ 
  -o SecretKey="{{secrets.secretkey}}"

The message queue is now created and available atthe endpoint https://demo-blog.poundifdef-smoothmq.auto.prod.osaas.io/

Let us try it out by writing a small Python application to create a queue and place a message on the queue.

Setup a Python virtual environment

% python3 -m venv .venv

Activate the virtual environment

% . .venv/bin/activate

Install the Boto3 client

% python3 -m pip install boto3

Create a python script we call demo.py containing:

import boto3

sqs = boto3.client("sqs", endpoint_url="https://demo-blog.poundifdef-smoothmq.auto.prod.osaas.io")
print ("Queues:")
response = sqs.list_queues()
for q in response['QueueUrls']:
  print(q)

Run the script with the access key and secret that you provided when creating the message queue stored in the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY

% export AWS_ACCESS_KEY_ID=blog
% export AWS_SECRET_ACCESS_KEY=blog
% python3 demo.py
Queues:

As expected the list of queues are empty. Let us create a new script create.py to create a queue called blog

import boto3

sqs = boto3.client("sqs", endpoint_url="https://demo-blog.poundifdef-smoothmq.auto.prod.osaas.io");
response = sqs.create_queue(QueueName='blog')

And run it

% python3 create.py

If we then run the blog.py script again we get

% python3 test.py
Queues:
https://sqs.us-east-1.amazonaws.com/1/blog

Let us now create a script send.pyto send a simple message

import boto3

sqs = boto3.client("sqs", endpoint_url="https://demo-blog.poundifdef-smoothmq.auto.prod.osaas.io");
sqs.send_message(
  QueueUrl="https://sqs.us-east-1.amazonaws.com/1/blog",
  MessageBody="Hello world"
)

And we can run it

% python3 send.py

To read the message from the queue we can write this script recv.py

import boto3

sqs = boto3.client("sqs", endpoint_url="https://demo-blog.poundifdef-smoothmq.auto.prod.osaas.io");
response = sqs.receive_message(QueueUrl="https://sqs.us-east-1.amazonaws.com/1/blog")
print (response['Messages'][0]['Body'])

And when we run it we get

% python recv.py 
Hello world

Now we have created an SQS compatible message queue in Open Source Cloud.

What is Open Source Cloud

A software as a service based on open source with a unique transparent model where revenue is shared with the open source authors. Open Source Cloud offers media industry companies a quick way to incorporate open source in their solutions and the option to run the same software in-house as the source code is publicly available.

• API and CLI documentation: https://api.osaas.io
• Javascript Client Library reference documentation: https://js.docs.osaas.io/