How to Run Python and Ruby on Heroku with Multiple Buildpacks

One of my mentees here in codementor.io encountered a problem while running Python and Ruby together on Heroku. That session inspired me to write this tutorial—how to run more than one environment on Heroku.

PROBLEM

The project was working in Cloud9 without any problems. However, after deploying to Heroku, the log was displaying error messages about Python problems. Steve had written a scraper in Python, and it was not working. Steve, my mentee, had created an old test environment on Cloud9 for me to work on his problem.

SOLUTION

This solution uses this guideline from Heroku.

I solved the problem by using Heroku multi-buildpacks for Python and Ruby.

Heroku multi-buildpacks

Buildpacks are responsible for configuring your application. Heroku explains buildpacks in its documentation like this: "Buildpacks are composed of a set of scripts and depending on the programming language, the scripts will retrieve the dependencies, and output generated assets or compiled code, and more. This output is assembled into a slug by the slug compiler."

There are many Buildpacks (many can be seen here and here). There are distinct buildpacks for Ruby and Python.

So, If someone wants to deploy Ruby and Python together on Heroku, he needs to use “multiple buildpacks". So, Heroku would use different buildpacks to run your application. Each buildpack creates a distinct environment.

Running apps with Procfile

Steve’s application needs to run on Puma Server (the default web server of Ruby on Rails). So, I needed to create a Procfile. Procfile runs commands automatically after the deployment. Procfiles are responsible for running shellcode on Heroku after deployment. For example, you can start web servers and long-running tasks. See more from here.

So, I created a Procfile file like this on his project's root:

# Procfile
web: bundle exec rails server -p $PORT
worker: python scraper.py

In multiple buildpacks, the order of the buildpacks is critical.

I needed to add Ruby buildpack as the last one because Steve would be running a Puma web server for Ruby. If I had not done it like that, Steve would not be able to run his Ruby server on Heroku. So, I added Python buildpack first and then added Ruby buildpack later. Together, they become "the mighty multiple buildpack," ready to use on Heroku.

# command line commands
heroku buildpacks:clear  # remove all of the buildpacks first
heroku buildpacks:add heroku/python  # add the python buildpack first
heroku buildpacks:add heroku/ruby    # add the ruby buildpack second

heroku buildpacks  # to see what I have done
# === golden-ratio-1618 Buildpack
# 1. heroku/python
# 2. heroku/ruby

Internals of a Buildpack

To know more about the internals of a Buildpack, you can review the Github source of a Buildpack, for example, Ruby. bin/detect file is essential. You can see the source code of this file to see what is needed for Heroku to deploy your app. For example, for Ruby, you need to create a Gemfile in the root directory of your app, or it will fail.

Required packages were missing

However, after adding all of the buildpacks, the project was not functioning on Heroku, yet. That was because of Python requirements.txt file. The file is responsible for adding the required Python packages to your app to be installed on Heroku automatically after each deployment (like Gemfile in Ruby). Every time you deploy, Heroku launches a new instance and initializes it by running those files. They are the required files for the environments, like Python and Ruby.

I also needed to verify the behavior in the local, so I ran pip install -r requirements.txt as a command to install the required packages to the local.

Moreover, I tested the setup with: heroku local command. See more about it here.

I put the required packages by looking to Python scraper’s header code. One of them was lxml. I found its latest version number by looking at its github page (to the tags).

So, we created a requirements.txt file like this:

# requirements.txt file
lxml==3.6.4

So, after delivering the training session, Steve responded like this:

Conclusion

With this article, my aim was that trying to show how I handled a mentee's request and let you know about how to run multiple-buildpacks on Heroku. Of course, there are many details that I skipped over. Maybe, those can be the next article's topics. Similarly, you can also read more tutorials about Heroku, like: how to parse dashboard Heroku or how to get push notifications when your Heroku app is deployed. See you next time.