Microservices for Startups: Avoiding Common Microservice Mistakes

When I first mentioned the concept of microservices to my engineering team, I was surprised by how quickly everyone researched the idea and then jumped to the conclusion that our monolith should be split up into tiny APIs that were each responsible for a model in our existing Rails application.

From the research I'd done, I knew that it was dangerous to build a bunch of microservices without careful consideration about size, boundaries, and other tradeoffs. But no one on my team seemed concerned about it.

So I probed my team with questions. Why so many services? Why not less? What were the tradeoffs?

What I found was that people on my team were jumping to conclusions based on shallow but dangerously firm notions of what a microservice was. They held knowledge that microservices were small API's pieced together to create whole systems. But they weren't aware of the intricate tradeoffs and design considerations that can mean the difference between success and failure. They were pitching architectures with little ability to justify or reason about them. They had simply jumped to the conclusion of creating a bunch of API's because that's what they thought microservices were supposed to be.

I wish I could say this was a unique occurrence, but I've worked on several teams where this has happened. If you're reading this, you probably have too.

Why does this happen? And what is a microservice, anyway?

Poorly named, ambiguously defined

For a profession that stresses the importance of naming things well, we've done ourselves a disservice with microservices. The problem is that that there is nothing inherently "micro" about microservices. Microservices do not have to be small. Some are, but size is relative and there's no standard of unit of measure across organizations. A "small" service at one company might be one million lines of code while far less at another.

Some argue that microservices aren’t a new thing at all and are rather a rebranding of Service Oriented Architectures, while others advocate for viewing microservices as an implementation of SOA similar to how Scrum is an implementation of Agile.
This poor naming is one of the reasons behind misconceptions and misapplication of microservices. Lots of developers, even those in 3-man teams, are jumping on the bandwagon of splitting up apps into far too many services without understanding what microservices really are.

When a popular, attractive-sounding term like “microservices” flies around, it's easy for the the concepts behind it to get lost and distorted. Martin Fowler calls this Semantic Diffusion.

Semantic diffusion is essentially a succession of Chinese whispers where a different group of people to the originators of a term start talking about it without being careful about following the original definition. These people are listened to by a further group which then goes on to add their own distortions. After a few of these hand-offs it's easy to lose a lot of the key meaning of the term unless you make the point of going back to the originators.

The misconceptions don't just affect people who want to use microservices, it also stokes the fires of those that dismayed by the industry hopping on the microservices bandwagon without deep understanding of its concepts.

I'll keep saying this ... if people can't build monoliths properly, microservices won't help. #qconlondon #DesignThinking #Modularity

— Simon Brown (@simonbrown) March 4, 2015

Most critique is not necessarily a critique of microservices themselves, but rather criticism of failed application of microservices coupled to (and likely caused by) a misunderstanding of microservices.

Without this knowledge, failures are attributed to the entire concept of microservices rather than the failure to use them successfully.

The Ideal Definition

There’s a lot of ambiguity around what microservices are in part because no precise definition exists. Like Agile, microservices are a collection of broad concepts rather than concrete practices.

How exactly the term “microservice” was invented has been documented by Martin Fowler:

The term "microservice" was discussed at a workshop of software architects near Venice in May, 2011 to describe what the participants saw as a common architectural style that many of them had been recently exploring. In May 2012, the same group decided on "microservices" as the most appropriate name.

Jame’s Lewis’s 2012 presentation at 33rd Degree in Krakow titled “Microservices - Java, the Unix Way” presented these new ideas and described microservices as a way of develop software faster by dividing and conquering using Conway's law to structure teams.

Since then many others have continued to pioneer and advance the concepts around microservices including Fred George, Adrian Cockcroft (Netflix), Martin Fowler (ThoughtWorks), and Sam Newman (ThoughtWorks).

Today’s leading definitions are fairly well aligned:

Microservices are small, autonomous services that work together.

- Sam Newman

Fine grained SOA architecture done the UNIX way.

- James Lewis

Loosely coupled service oriented architecture with bounded contexts.

- Adrian Crockford

In short, the microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies.
Martin Fowler and James Lewis https://martinfowler.com/microservices/#what

The problem with these definitions is that while they are helpful for introducing the idea of microservices, but when it comes to time put them into practice, they are not very helpful. Using these definitions, how would you determine whether it makes more sense to have 10 tiny services versus five medium-sized ones?

In his book Building Microservices, Sam Newman explains that the right size is “Small enough and no smaller… We seem to have a very good sense of what is too big, and so it could be argued that once a piece of code no longer feels too big, it’s probably small enough." Again, this doesn’t help you make a clear decision.

Ambiguous descriptions of microservices aren’t useful other than as an introduction. When it comes time to put microservices into practice, you need to find other ways to align your team.

Getting alignment on your team

How do you align your team when no precise definitions of microservices exist? The most important thing when talking about microservices on a team is to ensure that you are grounded in a common starting point.

But ambiguous definitions don’t help with this. It would be like trying to put Agile into practice without context for what you are trying to achieve, or an understanding of precise methodologies like Scrum.

On one team I worked on, we tried to not get hung up on definitions and instead, we first focused on defining the benefits we were trying to achieve by adopting microservices:

Shipping software faster

Our main application was a large codebase with several small teams of developers trying to build features for different purposes. This meant that every change had to try to satisfy the different groups. For example, a database change that was only serving one group would have to be reviewed and accepted by others that didn’t have as much context. This was tedious and slowed us down.

Having different groups of developers sharing the same codebase also meant that the code continually grew more complex in undeliberate ways. As the codebase grew larger, no one on the team could own it and make sure all the parts were organized and fit together optimally. This also made deploying a scary ordeal. A one-line change to our application still required the whole thing to be deployed in order to push out the change. Because deploying our large application was high-risk, our QA process grew and we deployed less.

With a microservices architecture, we hoped to be able to divide our code up so different teams of developers could fully own them. This would enable teams to innovate much more quickly without tedious design, review, and deployment processes. We also hoped that by having smaller codebases worked on by fewer developers, our codebases would be easier to develop, test, and keep well organized.

Flexibly with technology choices

Our main large application was built with Ruby on Rails and a custom JavaScript framework with complex build processes.

Several parts of our application were hitting major performance issues that were difficult to fix and bringing down the rest of the application with it. We saw an opportunity to rewrite these parts of our application using a better approach, but inter-tangled our codebase was with the affected areas, this felt too big and costly to do.

At the same time, one of our front-end teams was wanting to pull away from our custom JavaScript framework and instead build product features with a newer framework like React. But mixing React into our existing application and complex front-end build process seemed expensive to configure.

As time went on, our teams grew frustrated with the feeling of being trapped in a big codebase that was too big and expensive to fix or replace.

By adopting microservices architecture, we hoped that keeping individual services smaller would mean that the cost to replace them with a better implementation would be much easier to manage. We also hoped to be able to pick the right tool for each job rather than having to go with a one-size-fits-all approach. We’d have the flexibility to use multiple technologies across our different applications as we saw fit. If a team wanted to use something other than Ruby for better performance, or switch from our custom JavaScript framework React, they could do so.

Microservices are not a free lunch

In addition to outlining the benefits we hoped to achieve, we also made sure we were being realistic about the costs and challenges associated with building and managing microservices.

Developing, hosting, and managing numerous services requires substantial overhead. A single monolith running on a few processes can easily translate to a couple dozen processes across a handful of services, requiring load balancers, messaging layers, and clustering for resiliency. Managing all this requires substantial skill and tooling.

Furthermore, microservices involve distributed systems which introduce a whole host of concerns such as network latency, fault tolerance, transactions, unreliable networks, and asynchronicity.

Setting your own path

Once we defined the benefits and costs of microservices, we could talk about architecture without falling into counterproductive debates about who was doing microservices right or wrong. Instead of trying to find our way using others’ descriptions or examples of microservices, we instead focused on the core problems we were trying to solve.

• How would having more services help us ship software faster in the next 6-12 mos?
• Were there strong technical advantages to using a specific tool for a portion of our system?
• Did we foresee wanting to replace one of the systems with a more appropriate one down the line?
• How did we want to structure our teams around services as we hired more people?
• Was the productivity gain from having more services worth the foreseeable costs?

In summary, here are the recommended steps for aligning your team that is jumping into microservices:

1. Learn about microservices while agreeing that there is no “right” definition
   Discuss and memorialize your anticipated benefits and costs of adopting microservices
2. Avoid too eagerly hopping on the microservices bandwagon–be open to creative ideas and spirited debate about how best to architect your systems. Stay rooted in the benefits and costs you have previously identified.
3. Focus on making sure the team has a concretely defined set of common goals to work off of. It’s more valuable to discuss and define what you’d like to achieve with microservices than it is to try and pin down what a microservice actually is.

Thanks to Darby Frey for providing feedback on this chapter.