An Introduction to Releases with Erlybank

Posted by on September 17, 2008

This is the sixth article in the otp introduction series. If you haven’t yet, I recommend you start with the first article which talks about gen_server and lays the foundation for our bank system. If you’re a quick learner, you can download the ErlyBank sources as of the beginning of this post.

This article will introduce release management of an Erlang application. This is not necessarily restricted to Erlang/OTP applications, and is a general tool that may be used for all your Erlang applications.

Release Management refers to consolidating an application and its dependencies into a single package which can be easily started and stopped as a whole. Releases can be “installed” into an Erlang runtime system. Releases can also be upgraded in real-time, so when a new version of the application is created, it can be seamlessly installed into the system without bringing it down. (Note that real-time system upgrades are covered in the next article, not this one!)

After bundling the ErlyBank system as an application, restructuring it to be an Erlang release is very easy. But first, we’ll need to do some housecleaning to the directory structure.

Release Directory Structure

In general, all application releases follow a certain directory structure. This structure is not required but it is a common practice which increases in readability of application files:

Application-Vsn/ebin
               /include
               /priv
               /src

 

The src directory contains all the *.erl files, the sources for an application. The ebin directory contains all the compiled *.beam files for the sources and the application specification file. The include directory contains all the *.hrl files which may be included by the sources. And the priv directory contains any executables that are used by the application, such as port applications and Erlang drivers.

All these directories are under a parent directory named Application-Vsn where Application is the name of the application and Vsn is the version.

Reorganizing ErlyBank to follow this structure is quite easy and I will do this before moving on. Simply move all the *.erl files to the src/ directory, and move erlybank.app to the ebin/ directory.

Another important thing to remember is that when compiling Erlang sources, the resulting beam file is usually placed in the same directory as the erl file. To make it so that our sources are compiled into the ebin/ directory, compilation should be done with the following command, which you may easily turn into a make command or whatever you want:

erlc -o ../ebin eb_app.erl eb_sup.erl eb_event_manager.erl eb_atm.erl eb_withdrawal_handler.erl eb_server.erl

 

The above command should be run from the src/ directory and will place all beam files in the corresponding ebin/ directory.

Release Specification File

The one thing you need to create a release is a release specification file. This file is used by the systools module to figure out how to package the Erlang application. This file format is documented very well at the release resource documentation, which I recommend you glance over now. The release specification I’ve created for the ErlyBank release is:

{release, {"eb_rel", "1"},
          {erts, "5.6.3"},
          [{kernel, "2.12.3"},
           {stdlib, "1.15.3"},
           {sasl, "2.1.5.3"},
           {erlybank, "1.0"}]
}.

 

The contents of the file are saved as eb_rel-1.rel. eb_rel is the name of the release with “1″ being the version number. The tuple {erts, “5.6.3″} specifies the version of the Erlang runtime system that the release is meant for. This can be obtained by opening the Erlang shell and seeing what version you have. Following these tuples, there is a list of more tuples which specify the applications for this release. By convention they are listed in order of dependency but systools is smart enough to figure this out on its own when you create the boot script later.

The versions of dependencies can be retrieved by using the application:loaded_applications/0 method.

One thing you may notice that was added was the dependency on sasl, which stands for “System Architecture Support Libraries.” The sasl application is required for the release handler module which is used to install releases into a system.

Creating a Boot Script

The purpose of the release specification file is create a boot script that we will use to boot up the entire release quickly and easily. To create the boot script, run the following commands:

Chip ~/.../6_release_management: erl -pa ./ebin
Erlang (BEAM) emulator version 5.6.3 [source] [async-threads:0] [kernel-poll:false]                                     

Eshell V5.6.3  (abort with ^G)
1> systools:make_script("eb_rel-1", [local]).
ok

 

The important thing is that the Erlang shell is started by appending ebin to the code path. This is what the pa flag does. This is so that the systools module can find the erlybank.app file.

Once the shell is open, the systools:make_script/2 method is used to create the boot script. The make_script method accepts as its first argument the name of the release spec file without the extension, and it is expected to be in the current working directory. The second argument is a list of options. The option specified here, local, uses the complete path of where the dependency applications are found instead of using variables such as $ROOT. This is useful for testing in a development environment. When you’re ready to package it for true release, omit this option.

If the result of calling the function is ok, then you should find a eb_rel-1.boot file in the current working directory. This file is used to start the Erlang system.

Starting the System with the Boot File

Remember how easy it was to start the ErlyBank system using application:start/1? Well now its even easier by typing the following into the shell:

erl -boot eb_rel-1

 

The Erlang system will locate the boot file specified and will follow it to startup the system. You can test that it worked by using the ErlyBank server to create accounts, make deposits, etc.

To shut down the system, use the normal q() command in the shell. All processes will be gracefully stopped.

Packaging the Release

The final step is to make the system a nice single package that can be carried around different systems to ease in setup and installation. This can all be done in a single step now that we already have the correct directory structure and a compiled boot script.

4> systools:make_tar("eb_rel-1").
ok

 

In the working directory, there should now be an eb_rel-1.tar.gz. That’s all there is to it!

Installing a Packaged Release

To install the packaged release, we first need to copy the tar.gz package we created earlier into the $ROOT/releases directory. $ROOT can be determined by running code:root_dir().. Next, start up and Erlang shell and make sure the sasl application is started so we can use the release_handler module. Also, make sure the shell is running with valid permissions to make changes to the $ROOT directory. This may require running erl with sudo.

3> release_handler:unpack_release("eb_rel-1").
{ok,"1"} 

 

The unpack_release method does just what it says. It unpacks the package file, copying the libs to the code:lib_dir() and created a proper release directory. Since this is the initial release, there is no need to “install” it, as its already done.

Now restart the Erlang shell, start sasl, and you should now be able to start erlybank with application:start(erlybank) anywhere! The system has been installed!

Alternatively you can start the ErlyBank application with a single command:

erl -boot $ROOT/releases/1/start

 

$ROOT must be replaced with the root directory of the Erlang install!

Final Notes

In this article I introduced creating a basic release, covering topics such as creating boot scripts, packaging a release, and installing a release. The real power of releases, however, comes at the ability to perform system upgrades to a running system, and this will be covered in the next, and final, article in the series.

If you wish to download all the files as of the end of this post, I have zipped them up here.

Trackbacks

Use this link to trackback from your own site.

Comments

Leave a response

  1. Zamous Sep 17, 2008 11:37

    What about test files? Should they still be in your src and compiled to ebin with your application?

    Also, mochiweb seems to be a bit different. They keep the .hrl and .app files inside the src directory.

    Lastly, what is best way to run erlang apps and detach from them, so they can run like a server without console open? I know we could possibly use nohup or screen, however what is typical “erlang” way?

  2. alex Sep 17, 2008 13:01

    First of all, thanks a lot for the series of articles about Erlang. It helped me a lot to understand few “fuzzy” things in Erlang.

    I’m also curious to run an application in a “daemon” state. It would be excellent if you could elaborate on that and update the articles.

    Thanks again!

  3. Mitchell Sep 17, 2008 14:02

    Generally test files I put in a separate test/ directory. But there is no real standard way I have found yet (not very much open source Erlang projects relative to other languages). And sometimes the test() method is directly on the module itself which means that its just in the src/ directory anyways. All matters how you test.

    Some projects do put .hrl files in the src directory, I’m not sure why this is. I’m guessing, but I haven’t tested, that the src/ directory isn’t exposed when doing an include in other projects, while the include/ directory is in the include path. Again, this is something I’ll have to check out and this paragraph of the comment is all complete bullshit until I do :)

    As for .app in the src directory, this is okay but you’ll have to add the src/ directory to the code path when you start the Erlang shell so you can make the script. Additionally when I put the app in the src directory I had problems starting the application once the package was installed. Maybe I was doing something wrong… But I’ve always found things to be a lot easier and smoother for me when I put it in the ebin directory.

    And to answer both your and Alex’s question: There is a -detached flag you can pass on the erl command. This will detach it from the shell. Be warned that to reattach to the shell, you’ll need to set a node name (with sname or otherwise) so you can use the “remsh” flag later to reattach to a node.

    I’ll look into the src/ and include/ for hrls when I have time, my client work at CitrusByte has been crazy! So busy lately.

  4. Harish Mallipeddi Sep 17, 2008 19:02

    The link to the zip file at the end of the post leads to a 404 :(

  5. Mitchell Sep 17, 2008 19:31

    Harish,

    Sorry for that, I had a bad symbolic link on a folder! It was pointing to a non-existent location. I fixed it now :)

  6. [...] An Introduction to Releases with ErlyBank (Mitchell Hashimoto) [...]

  7. Dmitriy Kopylenko Sep 18, 2008 11:35

    Mitchell. Very inforamtive series of articles you have going here. Thanks for that.

    I would really love to see some pointers, examples in the future installments, on how to integrate Java (using JInterface) with Erlybank e.g. write Java client.

    Thanks.

  8. Zamous Sep 19, 2008 21:18

    I know your company does a lot of Ruby on Rails, so assuming you use Textmate? I have a really simple command to compile Erlang in erl shell from Textmate if interested:

    http://www.citizencult.org/2008/09/18/compiling-erlang-within-textmate/

  9. Mitchell Sep 19, 2008 21:22

    Zamous,

    I saw this blog post already (I’m a troll, really :( haha) and its great, but unfortunately I use emacs almost exclusively now :) But thanks, its a great contribution to the textmate community.

  10. [...] An Introduction to Releases with Erlybank (tags: erlang programming spawnlink) [...]

  11. Stephan Oct 03, 2008 05:38

    Hi,

    thanks for this article series, it really brings light to my darkness ;)

    But I have a question: At the end you say to start SASL. Is this an application? Can you give an example?

    Regards,
    Stephan

  12. Mitchell Oct 04, 2008 08:39

    Stephan,

    Yes this is an application. To start it from an Erlang shell you can simply write:

    application:start(sasl).

     

    SASL is the “System Architecture Support Layer” or something like that (I could be wrong on that second S). It gives applications logging support, alarm handlers, release handling, etc. It is usually started with every OTP application.

    Mitchell

  13. David Weldon Oct 04, 2008 13:28

    Let’s say I have a static resource file I needed to read in whenever I start my server - for example a list of US city names. I’m unclear about how paths get resolved so I’m not sure where I should put my file. During development I put it in something like $DEV_DIR/cities/citynames. When I do a release, do I put it in src? If so, when I run file:open(”cities/citynames”, [read]) will it know where to look?

  14. Dan Milstein Jan 15, 2009 22:01

    First off, have to join with everyone in saying just how extremely useful this series of articles is.

    Second off, about .app files in the src/ directory. The comment above mentions that MochiWeb does this, but, in fact, the Makefile with MW copies it to ebin/. So that does seem to be the right place.

    Also, my understanding of .hrl files and include/ is that it’s only important to put them in include/ if code outside of your application will need to include them (which is more or less what you say, above, I think). Otherwise, src/ is fine.

  15. Wakhanu Abiud Jun 08, 2009 23:36

    Hi Mitchell,

    You are a great man. Your tutorials on OTP behaviours and release handling have really broadened my understanding of the erlang language.

    Now there is one thing I would like to know. I have followed your step by step explanation of how to create an application up to installing a release and things have worked out pretty well. However, in my “Myapplication.erl” file I have specified something like:

    start(_Type, _Args) ->
    erlangsmppgw:start_link().
    So when I first ran the function : erlangsmppgw_app:start(normal,[]), I was so pleased that this automatically invoked my application to run as expected .i.e the server process “erlangsmppgw” was spawned and called the rest of the program.

    Next, I package the application into a release and finally test as instructed by typing:
    export ROOT=/usr/local/lib/erlang
    erl -boot $ROOT/releases/1/start

    The end result is that myapplication “erlangsmppgw_app” is actually started” as shown
    by :
    =PROGRESS REPORT==== 9-Jun-2009::10:15:35 ===
    application: erlangsmppgw_app
    started_at: nonode@nohost

    - However, I dont see an invocation to the function “erlangsmppgw:start_link()” which I earlier on specified in the “erlangsmppgw_app.erl”. In other words “erlangsmppgw:start_link()” should start my server, but It doesnt. So how do I go around this?

    Any assistance accorded will be highly appreciated

    Thanks

  16. [...] Exit the console. You should find FIRST-1.0.tar.gz in your current directory. Ideally, this would be the last step, but more likely, you’ll need to do the customizations covered below. Unpack the tarball into your target directory and cd into it. For a different take on these first steps, check out An Introduction to Releases with Erlybank. [...]

Comments

Comments: