What is the blog about?
It’s about my experiences on a commercial project which uses micro-services*. I hope to cover some of the challenges faced, the approaches we took and my learnings from them.
Ok, What are micro-services again?
Sam Newman defines micro-services as
Small, autonomous services that work together.
On similar lines, I think of micro-services as an extension of the Single-Responsibility Principle, where in a service does only one thing but one thing well.
Sounds interesting, how was this used in your project?
Our project is an E-Commerce application where one can buy or subscribe to various products. Some of the services we had were orders_service, customer_service, payment_service, products_catalog_service, communication_service. These were consumed by a website and a call center application.
Cool, the first thing one hears when talking about micro-services is “Independently deployable services”. How was that done?
A more important question is how to test if the service being deployed integrates properly with the other services in production. That is to say it doesn’t break any existing functionality. Let us understand this better with an example. Say the version in production of all the services is 1. Now we want to deploy order service with version 2. But is order_service-v2 compatible with other v1 services? Is there any dependency of order_service-v2 on say customer_service-v2 ? Here is how we answered these questions.
Every service has a separate repository, for which contract and unit tests are run on every checkin for that service. It is then converted to an artifact. This artifact is versioned. To deploy only order_service-v2 we need the v2 artifact of orders_service and the production artifacts (v1) for all other services. The v1 artifacts can be fetched by referring to the previous run of the deployment pipeline. V2 of order service can be fetched from the artifact generation stage. This ‘mix’ will then be deployed on an environment. Now we can run automated and manual tests against this ‘mix’ and verify if everything is working as expected. Once verified, we can deploy orders_service-v2 artifact to production without touching the other services.
Because of this ability, releases were very frequent, sometimes 2-3 times a week.
Right, but what’s the catch ?
There are a few caveats here,
- Deploying different services from the same pipeline means changing the deployment command in our CI tool (GO) each time. Typically this involves changing the list of apps to be deployed.
- Releasing services with dependants meant halting the system because most inter-service communication was synchronous and we did not have blue-green deployments. This sufficed for us as there were off-hours where we could do deployments. One way to remove this dependency is by having all communication asynchronous, but more on that in part 2.
Hmm, pretty heavy stuff, where else did you leverage this separation of services?
Software upgrades can be done easily and incrementally. We upgraded ruby from 1.9.3 to 2.1 by starting with the least risk product_catalog_service and then moving to the other services. So the services were running different versions of ruby at the same time, but that didn’t affect the system because they communicate through HTTP and are deployed on different VMs.
We also have the ability to update configuration files in production (essentially in any environment) by just re-deploying the same code with new configuration files. This is possible because configurations are extracted to a separate repository and hence they do not have to go through the entire pipeline. Further, this re-deployed only those services which are affected by the change in configuration. This made actions like turning on feature toggles very trivial and fast.
That is the end of part one. Till now we have briefly covered micro-services, how it is applied in our project and aspects of separation of services with respect to deployments and software upgrades. Part two will cover deciding on service boundaries, its implications and more. Stay tuned.