dramatiq_sqs

Warning

The project is under active development. There will be breaking changes before v1.0.0. Please make sure to pin the version when using this for anything important.

A Dramatiq broker that can be used with Amazon SQS.

This backend has a number of limitations compared to the built-in Redis and RMQ backends:

• the max amount of time messages can be delayed by is 15 minutes,
• messages can be at most 1MiB large and
• messages must be processed within 12 hours of being pulled, otherwise they will be redelivered.

The backend uses boto3 under the hood. For details on how authorization works, check out its docs.

Installation

pip install dramatiq_sqs

Usage

import dramatiq

from dramatiq.middleware import AgeLimit, TimeLimit, Callbacks, Pipelines, Prometheus, Retries
from dramatiq_sqs import SQSBroker

broker = SQSBroker(
    namespace="dramatiq_sqs_tests",
    middleware=[
        Prometheus(),
        AgeLimit(),
        TimeLimit(),
        Callbacks(),
        Pipelines(),
        Retries(min_backoff=1000, max_backoff=900000, max_retries=96),
    ],
)
dramatiq.set_broker(broker)

Usage with ElasticMQ

broker = SQSBroker(
    # ...
    endpoint_url="http://127.0.0.1:9324",
)

Example IAM Policy

Here are the IAM permissions needed by Dramatiq:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "sqs:CreateQueue",
                "sqs:ReceiveMessage",
                "sqs:DeleteMessage",
                "sqs:DeleteMessageBatch",
                "sqs:SendMessage",
                "sqs:SendMessageBatch"
            ],
            "Resource": ["*"]
        }
    ]
}

License

dramatiq_sqs is licensed under Apache 2.0.