Micro services

Intro

And we are back.

After 10 years, I'm still working in software development.

What I'm working on.

Well, after what I believe to be a failed contract, my company decided that I should be researching microservices, AWS (ECS, EC2, SES, SQS, SNS), Elastic (soon to be replaced by mongo).

What is each acronym that I'm using there?

AWS: Amazon Web Services. Yeah, I'm still on AWS. Been using it on my previous employer (a company in Argentina, that had a contract with someone from the States), where I had some sort of devops / developer position.

ECS: This is an Amazon service that allows you to use Docker inside amazon EC2 containers.

Before giving you my explanation, without reading this article by Yevgeniy Brikman, I could have never ever set up my docker tasks in ECS.

Give it a try to that link first, is the best explanation I found, way better than the amazon documentation.

The main problem that I see with the amazon guide is that you don't have an ordered tutorial.

The article by Yevgeniy Brikman is excellent.

You have the documentation, but it doesn't follow a proper order.

The first thing you will need to have if you are going to use this is an EC2 instance of the type that is optimized for ECS, for example (amzn-ami-2016.03.d-amazon-ecs-optimized)

Amazon covers the containers on the documentation, but is not the first thing.

They start the documentation with creating private docker registries that you will need way later on.

SES: Is used to send emails using the SES amazon service. SES stands for Simple Email Service.

SQS: Amazon Simple Queue Service. A simple queue, self-explanatory

SNS: Amazon Simple Notification Service. This is used to deliver notifications.

Together, SQS + SNS = Like rabbitmq. Amazon does all the heavy lifting for you. Though it has some peculiar things. Due to high replication, there is a chance that depending on how you configure your SQS, one or more workers may receive the same message published by the SNS. They state that in the documentation, your workers must be able to handle properly if a message was already processed. You can handle the visibility of a message once is consumed with a timeframe that you can configure.

The other part of my highlight is going to be the SES part.

SES is like mandril (or MailChimp now). The main thing is when you start the service, you are in the sandbox mode. You will need to verify an email, and you will be only able to send emails to that verified domain.

You need to fulfill a request, so they allow you to deliver emails.

The most important part here is that they ask you how you handle bounces and complaints.

You can create an SNS topic that will receive the bounces/complaints/ deliveries, and you will hook it up with an SQS consumer.

All that, by clicking with the mouse.

Seems pretty straight forward, but they didn't put that on the page.

The main idea is that if you receive a complaint or a bounce, you just take it out of circulation (i.e., don't try to deliver emails to that recipient).

With which language?

Python 3.5.1 (yeah, I decided to use something newer)

I'm using uwsgi / flask, with the following extensions (used across some of the microservices)

• flask-jwt
• alembic
• flask-sqlalchemy
• greenlet
• boto3
• flasgger
• swagger-ui

Flask-JWT: Is an extension to create JWT tokens, that will be consumed by a JS client in my case.

The extension has some gotchas, you need to read the code if you need to change things.

With some monkey-patching and inheritance, you can alter the functionality of JWT completely, but you need to spend some time reading.

flasgger: I love flasgger. I was using flask-swagger, but being honest. The heredocs under the function names make the whole thing illegible. Flasgger lets you point to a file. If you don't know what is this, this is used to write documentation for swagger-UI.

Swagger-ui: The only complex thing that I needed to do here, was to add support for the JWT token, since some endpoints are protected by JWT tokens, I didn't had a way to push the token there.

I found the way.

Authentication.

I was able to handle this by using JWT. When you are developing microservices, and you have to manage authentication, there are multiple things to take into account.

Disclaimer: this is not a definitive guide. At the moment I'm experimenting. So far is my best solution, but I believe that there are other solutions out there.

If you split a user service and a permission service, moving that token will become tricky, and it's difficult to think sometimes.

The idea is that you pass around that token until it expires.

No token, no service. Simple.

Obtaining the permissions of the entity after logging is a must, and I think that a caching strategy will be needed there.

A simple example, I log in, I obtain my permissions and put some key, like user_id: permissions in a Redis storage.

On the next request, I will ask the Redis for that user permissions.

When for some reason I decide to revoke a permission, I will delete the Redis record, and well, they will need to fetch the permissions again.

This is a work in progress, which I may extend much further later on.