Posts tagged #elixir

Open your text editor with the migration file when you run mix ecto.gen.migration

There is a neat little trick which I found while browsing Ecto’s code Adding the following line to your ~/.bashrc will open up your new migration file with the text editor of your choice

ECTO_EDITOR="code" # put the name of your editor here

Now, when you run mix ecto.gen.migration, it will open up your editor and you can modify your migration.

I actually use neovim to edit my code. However, it doesn’t open up from erlang. I tried running :os.cmd 'nvim /tmp/a' but it fails with an error about stdin not being found.

A simple way to automatically set the semantic version of your Elixir app

There is a neat trick which I bumped into while doing Rails development which I’ve been using to set the semver value of my Elixir Apps.

This works if you use Git for your version control. The basic idea is to use git tags, and the number of commits since the git tag to generate your version number. Elixir allows you to use a version string like below (You can read more about this at


When I want to bump the major or the minor version, I create a tag with the version information e.g. v1.4 using the command

git tag v1.4 --annotate --message 'Version 1.4'
git push --tags --all

I use the git describe command to get the major, minor and the patch info. A part of the describe output also goes into the build information

git describe
# => v1.4-270-fa78ab71e
# => major.minor-patch-git_commit_id

Putting all of this together, I have the following in my mix config, it also uses the BUILD_NUMBER passed by Jenkins (the build server that we use)

defmodule Dan.Mixfile do
  use Mix.Project

  def project do
    [app: :dan,
     version: app_version(), # call out to another function which generates the version
     # ...

  # ...

  def app_version do
    # get suffix
    build_number = System.get_env("BUILD_NUMBER")
    suffix = if build_number, do: ".build-#{build_number}", else: build_number # => .build-443

    # get git version
    {git_desc, 0} = System.cmd("git", ~w[describe])
    ["v" <> major_minor, patch, git_commit_id] = git_desc |> String.trim |> String.split("-") # => ["v1.4", "270", "fa78ab71e"]
    "#{major_minor}.#{patch}+ref-#{git_commit_id}#{suffix}" # =>


Creating such a beautiful version number without showing it anywhere wouldn’t be very useful :) I usually put the version information of the app in a footer and the head inside a meta tag (if it is a phoenix app)

defmodule Dan do

  # cache the app_version during build time
  @app_version Mix.Project.config[:version]
  def app_version, do: @app_version


Inside the app.html

<!doctype html>
<meta name="version" content="<%= Dan.app_version %>">

So, now when something goes wrong I can take a look at the current version of the app by visiting a page, and know which precise git commit reproduces the problem. Our QA team too uses this information when filing bug reports.

I also send this version info to my error monitoring and metric services like Rollbar and AppSignal

Hope you find this technique useful :)

Annotating variables with underscore variables to make code more readable

We should always strive to make our code as readable as possible. Underscore variables aid is in making our code more readable by annotating literal values with meanings.

Look at the following variations

without any underscore variable annotation

Tentacat.Contents.find(@owner, @repo, "", client)

While reading the code, it is difficult to know what the third parameter is. In most these cases, I navigate to the actual function definition and read the variable name.

with underscore annotations

Here is an improved version of the same code with and underscore variable annotation. It makes it crystal clear that the third argument is a path.

Tentacat.Contents.find(@owner, @repo, _path = "", client)

What are the techniques you use in your code to make it more readable?

How to create a Cartesian product of two sets

Elixir has a very rich set of functions to work with collections in the Enum module. The name Enum however, is a bit unfortunate. Enum makes me think of (enumerated data)[] more than the enumerable.

Anyway, the other day I needed to compute the Cartesian product between to lists in my elixir app. And, I knew there would be something for that in the Enum module. I went through all the documentation and couldn’t find anything useful. That is when it hit me that comprehensions are best suited for this and this is a piece of cake for comprehensions.

So, without further ado, here is the code to get a Cartesian product in elixir:

a = [1, 2, 3, 4]
b = [1, 2, 3, 4]

cp = for x <- a, y <- b, do: {x, y}
# =>
# [{1, 1}, {1, 2}, {1, 3}, {2, 1}, {2, 2}, {2, 3}, {3, 1}, {3, 2}, {3, 3}]

How to keep mix tasks private to your hex packages

A few days ago I created a mix task for one of my hex packages. This package was just a wrapper around the awesome shipit package. Now, this task is specific to my hex package and isn’t going to be used by apps which depend on my hex package. However, today I found my mix task happily listed in the host apps’ list of tasks. This is not a good thing :)

I initially tried changing the extension of the mix task to .exs but that didn’t help. The mix task was not even listed in my hex package now.

After a lot of digging I found two useful pieces of info which helped me get around this.

  1. Mix deps have an :env key which defaults to :prod. This means, when mix compiles a dependency, the value of Mix.env inside that dependency is set to :prod.

  2. Mix compiler has an :elixirc_paths flag which tells the compiler where to look for elixir source code.

Knowing this, all I had to do was:

  1. Move the mix tasks from ./lib/mix/tasks to ./mix/tasks so that it is not picked up by default.
  2. Change mix.exs by adding the following to the project, so that in the :dev env of the hex package our mix task is loaded/compiled.
# mix.exs
def project do
  [app: :honeybadger,
  # ...
   elixirc_paths: elixirc_paths(Mix.env),
  # ...

defp elixirc_paths(:dev), do: ["lib", "mix"]
defp elixirc_paths(_), do: ["lib"]

Now my hex specific mix tasks are loaded only when I am working on my hex package. And the users of my hex package don’t see my mix utility tasks.

How to model an enumerated list of values

When dealing with things like statuses you may need to store and access a fixed set of values. The following is one of way of doing it

defmodule ProductStatus do
  def active, do: :active
  def inactive, do: :inactive
  def cancelled, do: :cancelled

If you have a lot of values like these, the above code may become tedious. Metaprogramming to the rescue :)

defmodule ProductStatus do
  @statuses ~w[

  for event <- @events do
    def unquote(event)(), do: unquote(event)

This allows you to use statuses as

the difference between the for comprehension and Enum.each

Look at the code below and try to guess what happens when you run the following lines of code.


for {:resp, f} <- [{:resp, 3}, :b, {:resp, 4}] do
  IO.inspect f


Enum.each([{:resp, 3}, :b, {:resp, 4}], fn {:resp, f} ->
  IO.inspect f

The for version chugs along just fine :) which might surprise you. The version blows up as expected.

Be careful about using for in your code, especially your tests. I had a small test which was the following lines:

assert length(stats) == 2
for {:resp, stat} <- stats do
  assert stat.meta == %{a: 3}
  assert stat.time_ms in 10..20

And this was passing every time even when I changed the assert to the code below.

for {:resp, stat} <- stats do
  assert stat.meta == nil
  assert stat.time_ms in 10..20

All because I had an incorrect pattern match. I had {:resp, stat} instead of a { {:resp, _id}, stat }. So, the for was filtering out all the stats and the inner block was not being executed even once.

Treat your elixir warnings as errors in your non dev environments

One of the bittersweet things about Go is its relentlessness with warnings. You cannot compile your Go code which has warnings in it. This is great for the long term health of the project. However, it is not so pleasant while writing the code. Elixir takes an opposite approach to this where even errors which should break your build are treated as warnings. One such instance are Behaviours. If you have a module which implements a behaviour and does not implement a non optional callback, all elixir does is emit a warning! It should really throw an error and stop the compilation. However, all hope is not lost! You can use an elixir compiler flag to treat warnings as errors. All you need to do is add the following to your mix.exs:

  def project do
    [app: :awesome_possum,
     # treat warnings as errors in non dev environments
     elixirc_options: [warnings_as_errors: Mix.env != :dev],

You can even hard code it to true and it will always treat warnings as errors. However, non dev is the sweet spot for me, as I may be testing incomplete code with warnings in dev.

You can also pass these options to via the cli like so:

# using elixirc
elixirc --warnings-as-errors awesome_possum.ex
# using mix
mix compile --warnings-as-errors

Using mnesia with distillery

If you are using mnesia with distillery. You may run into an error which likes below:

09:46:26.974 [info]  Application dynamic_store exited: DS.Application.start(:normal, []) returned an error: shutdown: failed to start child: DB
    ** (EXIT) an exception was raised:
        ** (UndefinedFunctionError) function :mnesia.create_schema/1 is undefined (module :mnesia is not available)

This is because distillery doesn’t export mnesia by default. You need to tell distillery to export :mnesia by adding it to the extra_applications option in your mix application.

  # Run "mix help" to learn about applications.
  def application do
      extra_applications: [:logger, :mnesia],
      mod: {DS.Application, []}

When not to use apply for dynamically calling functions in Elixir

Elixir has a nice apply function which allows you to call any module’s function with a list of arguments normally called the mfa. However, I see apply being used in places where it shouldn’t be. Let us take the below example.

defmodule WorkerBehaviour do
  @callback perform(job :: Job.t)

defmodule EmailWorker do
  @behaviour WorkerBehaviour
  def perform(job) do
  # ...

defmodule ScreenshotWorker do
  @behaviour WorkerBehaviour
  def perform(job) do
  # ...

defmodule Processor do
  def process_job(worker_module, job) do
    apply(worker_module, :perform, [job])

In this example the Processor.process_job uses the apply function to send the job to the right worker. However, there is a more readable version of this code. Just use the following:

defmodule Processor do
  def process_job(worker_module, job) do

Since, in this particular scenario we know beforehand what the name of the function is and the number of arguments is the same for all modules, we can directly invoke the required function on the module using the above syntax. This makes your code more readable overall.

how to process an ets table in chunks or batches

We have an etl module which stores a lot of data in an ets table. This is loaded from a file using :ets.file2tab when the app starts and is written back to the disk using :ets.tab2file when the processing is done.

However, we have been running out of memory in one particular path of the module (this is the one which creates an export of this data). Our current ets table on disk is at 8GB and with a server having 32GB of memory and the fact that elixir/erlang structures are immutable, we are running out of memory when we do a few transformations on the ets data before the export.

Our code looked something like below:

@db_path '....'
|> ....)
|> Enum.filter(fn ....)
|> Enum.into(!("urls"))

Fortunately for us, :ets provides a way to process a batch of records at a time without processing all the data in one go. For batch processing needs :ets.match/3 function is your friend. With this knowledge in hand we changed our code to look like below:

defmodule BatchProc do
  @batch_size 100

  @db_path '....'
  def run do
    process(:ets.match(:db, {:"$1", :"$2"}, @batch_size))

  def process({els, cnt}) do
    |> Enum.filter(&filter/1)
    |> Enum.into(!("./urls.#{:erlang.unique_integer([:monotonic, :positive])}"))

    |> Enum.each(&:erlang.garbage_collect/1)


  def process(:"$end_of_table") do

  defp transform(el), do: el
  defp filter(el), do: true

The key takeaway here is that the :ets.match takes a third argument which is an integer and limits the number of rows returned. This is similar to the LIMIT clause in sql. And, when you use :ets.match/3, it returns a list of elements and a special structure called a continuation in a 2 element tuple {els, continuation}. This continuation can be thought of like a database cursor. It can be used with :ets.match(continuation) to get the next batch of elements. Once it reaches the end, you’ll get :"$end_of_table" as the continuation value. The above code uses this fact to terminate the recursion.