Separation of Concerns Made Easy in Elixir+Phoenix

This is a story about how I start my first professional work with Elixir and Phoenix Framework

(Image from Google image search, seems have no copyright)

I have been dwelling with Elixir since its 1.0 inception, started by dissatisfaction of my RoR application throughput. The suggested solution usually are to do horizontal and/or vertical scaling and then load-balance it to improve its throughput, but I felt something was not right about Ruby’s under the baseline performance. Relating it to my experience as an ex-game developer, I know that asking gamers to upgrade their system is not even an option most of the time and also bad for business because of market size trimming.

By following my guts, I started a quest to find an ideal platform which will keep my mind sane, have good baseline performance, and won’t giving me another sleepless nights caused by its own deficiency. I narrowed down to three platforms that were the safest bet and have a good concurrency model: Java (including Scala, Clojure, and friends), Go, and Elixir. In terms of raw computing power, Elixir is no contest against Go and Java. But Elixir has something in its sleeve, a foundation that built and designed for horizontal and vertical scalability and are part of the VM and language itself. We don’t need to tinker the internals and do (much more) trial and errors to code for scalability.

At this point I feel the hesitation to make this post lean towards to technical or project experience aspect. I’ll let the reader decide whether the title is correct and represent the content.

Let’s jump to the problem! We are creating a discovery, promotional, activity information, and feedback platform; with read only data source from our client for the user’s activity information. The system consists of Android & iOS mobile client, simple CMS, reporting system, and REST API backend. All looks easy and will be fast to make ;)

Our tech stack of choice are:

• AWS Elastic Beanstalk with Docker
• AWS Lambda to run scheduled jobs
• Nginx to route URLs to corresponding Phoenix server
• Elixir + Phoenix
• Mysql (needed for system integration)
• Swift and Java for the mobile client
• Slack and Datadog for backend monitoring
• Crashlytics for mobile crash reporting
• Facebook Account Kit for phone number verification
• Firebase Cloud Messaging for push notification

As of 2017, I believe those are the safest bet for quick and small project as many are free to use and keeping us sane when non-trivial scaling needs to happen. Sure you might argue about the design, but it’s my opinionated view as the designer.

Our reported ELB average response time is under 10 ms and only have problem once we reach 50k artificially concurrent accesses with four t2.micro instances. I believe we are not doing something right there, hence that limitation. But 50k is already more than enough for now, also most of the time it only needs one instance to serve all requests. I know, high availability stuffs need at least 2 instances to be available, but.. cost. Cost is another factor where an Elixir project could shine, in our case.

Separating concerns in Elixir feels natural for us since it has umbrella project support (with caveat, only use this if required). In our design, using umbrella is a good fits since we have two databases, CMS, and API that can be split into four OTP applications. That design allowed us to reduce AWS cost, why? Because we are putting our four independent OTP applications into a single EC2 instance in ELB environment instead of creating another ELB env just for splitting API and CMS endpoint. Nginx here acts as a router to our two independent API and CMS web server, for example /cms will route to port 8080 and /api will route to port 8081.

Sharp readers will take a look back at our tech stack and ask why we don’t have cache layer? This is a form of AWS cost reduction too. And how we overcome that? BEAM VM provides! We are using ETS (Erlang Term Storage), Erlang’s in-memory storage. No redis, no mongodb, no memcache, and no elastic cache. The downside is when we are at load, the request might tossed to random instance by ELB and cache miss happened. We will revisit this again once it becomes a real problem.

AWS Lambda is chosen since we have no centralized lock (usually redis) to run our scheduled job. The job is to send push notifications to our mobile clients to remind a certain activity. So far it’s doing well, though in a long run it will be our slowest process, we still need to figure out how to convert this process as streams. Making it as stream will need a major change in our current data-flow, because the data come from our client’s side, and they are rather hesitant to changes.

I hope you can start to see the benefit of using Elixir and OTP as your next sensible choice of platform. I know proving this to upper management might be hard if you already have working solution with Go or Java (Scala, Clojure, and friends) as those will be faster in the synthetics benchmarks. I recommend to start implementing Elixir in you side projects and start to learn how it behaves in production. Also, notice how much sane words in this article, that’s my main mind concept.

This article is my own personal opinion and does not represent my current employers and partners at all.