Deploying Rails apps on CentOS SELinux

In the course of our work we have come across a variety to linux flavors; RedHat Enterprise Linux, Ubuntu, and CentOS seem to be the most common. Lately I had to get a new CentOS box set up with a typical Rails stack, ruby, rvm, Passenger, mysql and so on.

There are plenty of articles that describe how to install the stack, and since the commands change constantly I won't bother with rehashing this. What I want to describe is how to make sure your Passenger app can be run on a locked down SELinux box safely.

The first thing to consider is what SELinux does. It is designed to combine all the best practices on server security and turn them ON by default. In fact, the only port that will respond to traffic is port 22, so that you can ssh in. Everything else is shut down.

Opening the Firewall

So the first thing we need to do is open things up a little bit more, specifically, we'll want to make sure outside people can access our web site, so lets open port 80 for http traffic. On CentOS, the system firewall is handled by the service iptables and configured in the file /etc/sysconfig/iptables. You can enable traffic on port 80 by adding this line to your iptables config file:

-A INPUT -m state –state NEW -m tcp -p tcp –dport 80 -j ACCEPT

However, you need to make sure to add this line before any REJECT lines, as the rules are read in order and any REJECT directives will override later ACCEPT directives. Once you have saved the file, you need to restart iptables

$ sudo service iptables restart 

Directory Permissions

Another difference between Ubuntu and CentOS is that CentOS requires that your application directory, and all parent directories, have absolutely the right permissions. To accomplish this without chmod 777 -R * (please don't do that). I like to set up my groups and users carefully. First, off, I have a user that is in charge of logging in and managing the apps. Lets call her releng (release engineer). On the server, I use the /srv dir as it is at root, and has nothing inside. I add an apps dir, then set up my app inside of that, so I have a path like /srv/apps/appsauce (my new app is called appsauce). I then create a symlink ln -s /srv/apps /apps so that in all scripts and config files, I can simply use /apps/appname

Now, we want to make sure releng has access to all these files, so we run chown -R releng /srv. But the apache user needs access to these files as well! So I like to add apache to releng's group: usermod -G releng apache. Now that means we need to make sure the group releng has access to all these files. So we can run: chgrp -R releng /srv. At this point, we have a human user and a process user in the same group, accessing the same directory tree under /srv

Starting Passenger

If you are used to Ubuntu and simply install passenger right away without taking a breath, you probably will run into some trouble. If you check the httpd error logs (/var/log/httpd/error_log) you'll see that passenger is complaining about permissions. It seems that passenger is not allowed to do something it really needs to do. In this case, create files in the tmp directory.

In SELinux, not only are the ports shuttered, but the file permission structure is very strict. The apache user that is running passenger is not allowed to access files or directories it does not create itself, unless specifically given permission at the system level. Giving the apache user this access is a process I won't get into here. Luckily enough, there is a way to create permissions based on how an application is used. This article explains how to properly add permissions for passenger. The idea is to turn off SElinux, run the app, then turn the log of what the app asked for permissions-wise into a permissions policy for that app. Once the correct policy is in place, you can turn SELinux back on and run your app securely.

The article is missing an important piece of information though. If audit2allow has not been installed on your system, you won't find it in the yum repositories. You'll need to install the package policycoreutils-python

Finally

At this point, we should be able to run our app with SElinux turned on, and only port 22 and port 80 open for connections. This is a rock solid platform for your product or your clients applications. Leave a comment if you have any improvement suggestions.

Posted by adevadeh 14 Jun 2012 at 08:05AM


rails 3.2: cap deploy with assets

In rails 3.2, you need to precompile your assets (javascripts, stylesheets & images) when doing your deployment to server. There are couple ways to do this:


Add public/assets into your git branch before deployment.

You need to run the rake assets:precompile task locally, and then commit it into your staging or production branch. Note that you cannot commit into your development branch, otherwise, you have problem in development mode, e.g. javascript function got included twice.

The problem is obvious, you end up with extra files in your git repository, and your staging or production branch cannot be merged back into development branch.


Use Capistrano deploy/assets to precompile automatically on server.

This is quite easy to do. You just need to modify your Capfile:


# Uncomment if you are using Rails' asset pipeline
# load 'deploy/assets'

I use this method if the assets is quite tiny. However, it's not something super smart that will detect if your assets content really changes. It just simple precompile every time when you do cap deploy. So, it your assets is pretty large, your every deployment will be very slow. And if your server is quite cheap, you will notice that the server CPU is very high for a while.

I end up to write a cap task, cap deploy:assets, to do with it.

My idea is we don't deal with any assets precompile if just cap deploy, since it might be quite expensive. We wrote another deploy task deploy:assets for it, similar to the deploy:migrations.

  • first, we precompile locally, to avoid high cpu on server, and I actually found that my MacBook Pro is much powerful than some of our servers.
  • zip the whole public/assets directory as public/assets.tar.gz
  • upload the public/assets.tar.gz to shared directory
  • unzip the assets.tar.gz to shared/assets
  • remove the public/assets & public/assets.tar.gz locally, so that we won't have problem in the development mode below is the task I wrote for this deployment behavior.


namespace :deploy do
  task :ln_assets do
    run <<-CMD
      rm -rf #{latest_release}/public/assets &&
      mkdir -p #{shared_path}/assets &&
      ln -s #{shared_path}/assets #{latest_release}/public/assets
    CMD
  end

  task :assets do
    update_code
    ln_assets
    
    run_locally "rake assets:precompile"
    run_locally "cd public; tar -zcvf assets.tar.gz assets"
    top.upload "public/assets.tar.gz", "#{shared_path}", :via => :scp
    run "cd #{shared_path}; tar -zxvf assets.tar.gz"
    run_locally "rm public/assets.tar.gz"
    run_locally "rm -rf public/assets"
    
    create_symlink
    restart
  end

end


Updated: I saw a simple trick to speed up by checking if the assets directories changed or not before doing a assets:precompile task


namespace :deploy do
  # http://stackoverflow.com/questions/9016002/speed-up-assetsprecompile-with-rails-3-1-3-2-capistrano-deployment
  namespace :assets do
    task :precompile, :roles => :web, :except => { :no_release => true } do
      from = source.next_revision(current_revision) rescue nil
      
      if from.nil? || capture("cd #{latest_release} && #{source.local.log(from)} vendor/assets/ app/assets/ | wc -l").to_i > 0
        run %Q{cd #{latest_release} && #{rake} RAILS_ENV=#{rails_env} #{asset_env} assets:precompile}
      else
        logger.info "Skipping asset pre-compilation because there were no asset changes"
      end
    end
  end
end

Posted by Shaokun 28 Mar 2012 at 07:11AM