Setting up rubygems without root access

Recently we had to install our rails app on a box, but we didn't have root access. The machine had ruby 1.8.6 and rubygems 0.9.6 (ouch). We can live with Ruby 1.8.6; but to run modern rails, we needed modern rubygems.

The rubygems install page has a helpful section on how to install locally without root access. Note: be sure to add export GEM_HOME=~/rubygems/gems to your .bashrc or your .bash_profile or else the next time you log in, your system gets confused again.

$ mkdir rubygems
$ cd rubygems-a.b.c
$ export GEM_HOME=~/rubygems/gems
$ ruby setup.rb --prefix=~/rubygems

Unfortunately, this didn't go as smoothly as I hoped. The result:

" `gem_original_require': no such file to load -- rubygems/exceptions"

and many other similar error messages.

The issue was that since the server installation already had an old version of rubygems installed, my local gem script was calling up the old rubygems files, but using my new GEM_HOME. A big mess.

What needs to happen is that your new gem executable needs to load your new rubygems files, and work with your new GEM_HOME. There may be a better way to do this, but I have found a workable solution for my case. It involves messing with the executable files that rubygems installs.

If you take a look at your gem executable, you'll see it is just a text file, a ruby program that executes itself.

$ cat  ~/rubygems/bin/gem

#!/usr/local/bin/ruby
#--
# Copyright 2006 by Chad Fowler, Rich Kilmer, Jim Weirich and others.
# All rights reserved.
# See LICENSE.txt for permissions.
#++

require 'rubygems'
require 'rubygems/gem_runner'
require 'rubygems/exceptions'

...

If you are familiar with shell scripts, you'll know that the first line (the shebang) specifies the binary to use to process the file. In this case, its #!/usr/local/bin/ruby. In general this is great. Ruby loads up the file and runs the script and magic happens. But what if the require 'rubygems' line is loading up the wrong rubygems?

We need to tell ruby where to find our custom rubygems libraries. Luckily, ruby has an option for this, -I path/to/lib. So, all we need to do is open up our gem executable and modify the shebang line to include this option.

$ vim  ~/rubygems/bin/gem

#!/usr/local/bin/ruby -I ~/rubygems/lib
#--
# Copyright 2006 by Chad Fowler, Rich Kilmer, Jim Weirich and others.
# All rights reserved.
# See LICENSE.txt for permissions.
#++

require 'rubygems'
require 'rubygems/gem_runner'
require 'rubygems/exceptions'

This tells the ruby interpreter to start the process with our custom path in the include path. If we have a newer version of the rubygems library there, they get required instead of the old versions. I can now install gems locally into my GEM_HOME without needing root access like so:

$ gem install mongrel

But that's not all. mongrel also installs its own executable, and when you try to run it, it will require rubygems. And yes, you guessed it, it will find the old version of rubygems and get completely confused. So again, we need to go into the executable file ~/rubygems/gems/bin/mongrel_rails and modify the shebang line.

As far as I can tell, you would need to do this for each and every gem that installs its own executable.

Update: after further consideration, using the $LOAD_PATH environment variable would probably be more robust, but my app is already installed. Perhaps you will let me know how it goes with $LOAD_PATH

Posted by Adeh Tue, 20 Jul 2010 03:52:00 GMT


Automated testing tools:Selenium & Watir

When should use automation?

What is advantageous for automated testing? When should one decide to automate test cases?

For the short term project,manual testing may be more effective. If an application has a very tight deadline, there is currently no test automation available,then manual testing is the best solution.

However, automation has specific advantages for improving the long-term efficiency of a software team’s testing processes.As we can save the time special for the regression test.What is more,it is trivial for the tester if repeat the same test case.

Selenium

  • Start from Selenium IDE:
  • Selenium IDE is a plug-in for FireFox,and isn't available for other browser.

    It provides an easy-to-use interface for developing and running individual test cases or entire test suites.

    Selenium-IDE has a recording feature,then can export the code in several kinds of language.

    Right now,give a example for how to use it.

    Suppose we want to login Gmail:

    Open the FireFox browser,click the "Tools" tab,then you can see the "Selenium IDE" is in the list,click it and will pop up a window as following:

    Selenium IDE 1.0.7

    Then back to the FireFox browser,open google home page and click the "Gmail" link,then fill info and press "Sign in" button.

    Turn to the Selenium IDE again,all the actions are recorded,see the following screenshot:

    Record_login_gmail

    Click the red round button in the upper right corner to stop the record.The browser will execute the action what you did just now when hit the green triangle button.

  • Selenium RC:
  • Selenium IDE can only run the test in FireFox.Obviously,this is not enough.So we must mention Selenium RC.

    We can export the code and apply the test in others browser through Selenium RC.

    It is simple to export the code in Selenium IDE,view the picture:

    export_code

    Usually,the exported code isn't perfect.Luckily,Selenium-RC allows the test automation developer to use a programming language for maximum flexibility and extensibility in developing test logic.

    It provides an API and library for each of its supported languages: HTML, Java, C#, Perl, PHP, Python, and Ruby. This ability to use Selenium-RC with a high-level programming language to develop test cases also allows the automated testing to be integrated with a project’s automated build environment.

  • Selenium Commands:
  • Contain three parts:Command,Target and Value

    Just give the simple explanation here

    Command:What you want to do for the browser.Such as,"click" a link,"fill" the textarea and so on.

    Target:Tell the browser which element you want to execute the command.

    Value:The value that you want to fill or expect.Sometimes it is blank.It depends what kind of command is used.

    Watir:

  • Introdution
  • Watir is an open-source library for automating web browsers.

    It is a family of Ruby libraries but it supports your app no matter what technology it is developed in.

    It drives browsers the same way people do. Watir also checks results, such as whether expected text appears on the page.

    Watir supports multiple browsers on different platforms.

    Windows 2000/2003/XP/Vista/7 : IE6,IE7,IE8 and FireFox

    Mac:FireFox and Safari

    Ubuntu:FireFox

  • Install Watir
  • Install watir on different system is not the same.I just describe how to install it on Mac for Safari.

    You have to install Ruby first,we recommend using Ruby 1.8.6 on all platforms.

    The type the following commands in Terminal:

    $ sudo gem update --system
    
    $ sudo gem install rb-appscript
    
    $ sudo gem install safariwatir
    

    A moment later,the following info appears and indicates the Safariwatir is installed successfully.

    $ sudo gem install safariwatir
    Successfully installed safariwatir-0.3.7
    1 gem installed
    Installing ri documentation for safariwatir-0.3.7...
    Installing RDoc documentation for safariwatir-0.3.7...
    
  • Use Safariwatir step by step:
  • We also choose the Login Gmail as example so that it is easy to compare what is different between Watir and Selenium.

    Open your Terminal and start irb.

    $ irb
    

    Before going to the Google,you have to let Ruby know you want to use SafariWatir,so please type:

    irb(main):001:0> require "rubygems"
    => true
    irb(main):002:0> require "safariwatir"
    => true
    

    Then start the browser

    irb(main):003:0> safari=Watir::Sfari.new
    => #, @app=app("/Applications/Safari.app"), @document=app("/Applications/Safari.app").documents[1]>>
    

    Go to Google:

    irb(main):004:0> safari.goto "http://www.google.com"
    => nil
    

    Click the "Gmail" link in the upper left corner:

    irb(main):005:0> safari.link(:text,"Gmail").click
    => nil
    

    After flowing to the login page,you can fill personal info and hit "Sign in" button to login.

    To do this, we have to find out some attribute of text field, like id or name, so Safariwatir can find it on the page. We will use Firebug for the first time.

    irb(main):006:0> safari.text_field(:id,"Email").set("ruby@gmail.com")
    => :missing_value
    irb(main):007:0> safari.text_field(:id,"Passwd").set("ruby")
    => :missing_value
    irb(main):008:0> safari.button(:value,"Sign in").click
    => nil
    

    If the username and password are correct,you should login successfully.

    Compare

  • Convenience
  • Selenium IED can record the acitons and export the code,so can save lots of time for the script developer.Watir doesn't have this tools,all the codes have to be typed by hand.

  • On different system
  • Mac OS X:

    Developers still need to strive for Safariwatir and Firewatir,as lots of commands that works fine in Windows cannot apply in Mac.What is more,performance isn’t stable.

    However,Selenium can resolve all the issue that watir is unable to deal with.

    So watir is not good as Selenium on Mac.

    Windows:

    Use selenium to test IE is not the same as FireFox in Mac.Especially for the locator,running the script will be very slow if the locator isn't suitable.

    Originally developed watir is to apply on IE.Until now,it is perfect.

    Posted by 钟宇平 Thu, 10 Jun 2010 09:29:00 GMT


    Start a new project and push to remote git repo

    Like many folks out there, we have been making the switch to git from svn. Although we struggled with git early on, after using both for some time I have come to appreciate the benefits of git. So here is our way of starting new projects with git.

    Using gitosis on our server, we create an empty remote repo first. Then, one guy gets the project going and pushes the code into the empty repo. Here are the commands to get that going.

    $ cd path/to/new/project
    $ git init
    $ git add .
    $ git commit -m 'initial commit'
    $ git remote add origin git@devserver.com:newproject.git
    $ git push origin master
    

    Posted by Adeh Tue, 25 May 2010 08:09:00 GMT


    Problems I've met after upgrading to Snow Leopard

    I upgraded my Mac OS X Leopard to Snow Leopard a couple days ago. Everything seems to be fine, system runs faster than before in general. But a couple of softwares doesnt work compatible with SL.

    The 1st one is mysql, after upgrading, it's completely gone, I just cannt find it. Then I try to reinstall it with macport, but zlib doesnt work with SL too! So, install mysql manually may be the best way.

    Solution: Compiling MySQL on Snow Leopard

    The 2nd one is mysql-gem. When I wanna reload db, it shows "uninitialized constant MysqlCompat::MysqlRes" error. To fix this, specify ARCHFLAGS when you install the 'mysql' gem:

    sudo env ARCHFLAGS="-arch x86_64" gem install mysql -- --with-mysql-config=/usr/local/mysql/bin/mysql_config

    refer to: Error: uninitialized constant MysqlCompat::MysqlRes

    The 3rd one is thin-gem. Well, easy to fix:

    sudo gem update thin

    Posted by Billy Mon, 07 Sep 2009 05:05:00 GMT


    functional tests with machinist and authlogic

    Authlogic and Machinist are two libraries that are a great help in getting a Rails app up and running quickly. But once we had a prototype running and some solid unit tests going, I ran into a snag. How do I create Users and UserSessions in a way that agrees with AuthLogic so that my requires_user before filter is working properly within a functional test?

    The Machinist test_case.rb has some good explanation, but they focus on fixtures and leave out some easy steps.

    Here's how I setup my functional tests: In test_helper.rb I added a login() method :
      def login(u = nil) 
        user = User.make(u)
        assert UserSession.create(user)
        user
      end
    
    In blueprints.rb, i added the fields that AuthLogic needs to authenticate a user:
    User.blueprint do
      email   { "testuser@company.biz" }
      password    { "test" }
      password_confirmation    { password }
      
      #this is all needed so we can log in for functional tests
      password_salt {Authlogic::Random.hex_token }
      crypted_password { Authlogic::CryptoProviders::Sha512.encrypt(password + password_salt) }
      persistence_token { Authlogic::Random.hex_token }
      single_access_token { Authlogic::Random.friendly_token }
      perishable_token { Authlogic::Random.friendly_token }
      active { true }
    end
    
    Finally, in the functional tests, you can simply use login() to simulate a session within the functional test:
      test "should force login" do
        get :index
        assert_redirects_to login_path
      end
      
      test "should get index" do
        login
        get :index
        assert_response :success
        assert_not_nil assigns(:items)
      end
    
    I hope this helps. If you have any suggestions or improvements, please leave a comment.

    Posted by Adeh Tue, 18 Aug 2009 01:05:00 GMT


    AJAX loading notice based on Prototype

    Loading Notice is a javascript plugin, built on Prototype, which automatically shows an indictor when a slow AJAX is under process. Basically, it’s like the “loading” div appears in gmail when busy. And when the users scroll or resize the window, it will still keep showing on the top of the page.

    Check it out here: http://github.com/shaokun/loading_notice/tree/master

    HOWTO

    1. Install the plugin:

    $ ruby script/plugin install git://github.com/shaokun/loading-notice.git
    $ rake loading_notice:install
    

    2. Include the library into your project:

    <%= javascript_include_tag 'prototype', 'loading_notice' %>
    <%= stylesheet_link_tag 'loading_notice' %>
    

    3. Add a “loading” div and initialize a javascript object:

    <div id="loading" style="display: none">Loading...</div>
    
    <script type="text/javascript">
      window.onload = function() {
        window.loading_notice = new LoadingNotice("loading");
      }
    </script>
    

    4. That’s all!

    Posted by Shaokun Mon, 20 Jul 2009 05:55:00 GMT


    Simple SVNHelper for Rails Applications

    原创:黄文斌
    来自:http://github.com/kudelabs/SVNHelper/tree/master 这个小程序是专门写给使用SVN版本控制的Rails应用程序。
    有了它,我们可以在某个特定的网页上面直接看到我们的程序revision版本,还有最后的修改日期。
    它的实现原理很简单,就是查看.svn/entries文件里面的修改记录。
    一般用法,把这个文件复制到root/lib目录下面,然后在environment.rb里面用全局变量记录它:
    $current_version = SVNHelper.version(RAILS_ROOT)
    这样子,你可以在erb页面里面调用这个变量,比如:
    程序版本: $current_version.rev_with_date
    同时,它会在每次服务器重启的时候重新记录最新修改的svn版本。
    This little library is specialized for those Rails Applications which are using SVN as version control. It allows us to view the revision and the last modified date of our application on the website. It’s rather simple to implement such function, we just look into‘.svn/enries’file in RAILS_ROOT and record the things we want inside.
    How to use it?
    Please copy the file into root/lib, and set up a global variable in config/environment.rb:
     $current_version = SVNHelper.version(RAILS_ROOT) 
    And then you can use this variable everywhere you wish to see the revision and last modified date. Also, its value may be changed each time you restart the server if any changes to the SVN found. If you are interested in this, you can see the source code from here: http://github.com/kudelabs/SVNHelper/tree/master

    Posted by Mysen Sun, 12 Jul 2009 04:05:00 GMT


    Rails add_index to optimize your Database

    When you create your tables through Rails migration, did you ever think about adding a index? I bet that most of you just ignore it, especially when startup and everything might change a lot later. I ignored index too, but when our projects come to production mode, I start to realized that we really really need to optimize our database.

    It’s super easy to add and remove a index through migration:

    
    def self.up
      add_index :students, :name
    end
    
    def self.down
      remove_index :students, :name
    end
    
    

    But the difficulty is that you need to find the right fields to index and after that, you need a way to check if it helps. The difficulty increases as your projects grows, i.e. when there are many of fields and relationships between tables. You wouldn’t know what’s the real neck lock through a glance. Fortunately, there are tools that could be used to analyze your existing system and locate what could be improved.

    I use jeberly’s query-analyzer to help. After install query-analyzer, you could tail log/development.log, and then start to use your website. “Each SQL select query will be ‘EXPLAIN’ed and added to the log files right below the original query.”

    Student Load (0.3ms)   SELECT * FROM `students` WHERE (name = 'mike') LIMIT 1
      ############ FIXME - UNOPTIMIZED QUERY for Student Load ############ 
       select_type | key_len | type | Extra       | id |
       ---------------------------------------------------------... ...
       SIMPLE      |         | ALL  | Using where | 1  |
    

    I suggest you go through the Example Use in query-analyzer. It covers usage, results analysis and adding/removing indexes. Here, what I would like to add is how to speed up adding index and reviewing the improvement. It’s pain that you have to write one migration file, then migrate it before you could see the improvement, especially when you have tens of FIXME warnings in the log. My tricks is to do experiment part through script/console, and after you get enough information, you could simply add a migration file with a batch of add_index. That would save a lot of time. Below is how I did in one of our projects:

    ruby script/console
    ActiveRecord::Base.connection.add_index :students, :name
    # You could do this through the model class too
    Student.connection.add_index :students, :name
    

    From the log, you could see something like this: CREATE INDEX `index_students_on_name` ON `students` (`name`)

    And if you want to remove the index:

    ruby script/console
    ActiveRecord::Base.connection.remove_index :students, :name
    # You could do this through the model class too
    Student.connection.remove_index :students, :name
    

    From the log, you could see something like this: DROP INDEX `index_students_on_name` ON `students`

    After adding the index, you could refresh your web page, and check the log, see if there are still warnings from query-analyzer.

    Note that there are 4 types of index:

    1. “Normal” Indexes
    2. Unique Indexes
    3. Primary keys
    4. Full-text indexes

    The one we talked above is the “Normal” Indexes. I suggest you check this article out for more about indexes.

    Posted by Shaokun Tue, 09 Jun 2009 18:08:00 GMT


    Railscasts in text form - ASCIIcasts.com

    This is an update to my earlier post about Rails Screencasts .

    British Rails developer Eifion Bedford is putting together a collection of Ryan Bates’ free Railscasts series in a text form. This means he is painstakingly transcribing Ryan’s words, putting together screenshots that you can learn from, and code examples that you can copy and paste. This is a great way to consume the material, especially if English is not your first language. The ASCII Casts, as he is calling them, are a great addition to the growing volume of free learning material for Ruby and Rails. I recommend the security series and the performance tips. Thanks to Eifon and his efforts to illuminate the wild world of web development!

    Posted by Adeh Tue, 24 Feb 2009 17:42:00 GMT


    ruby-on-rails-with-mysql-on-ubuntu-8-10

    Hi guys, I’m writing this to share with you the joy of using ROR with Mysql on Ubuntu 8.10 Desktop Edition.

    First, we can get a latest free version of Ubuntu from here. Please choose a desktop edition, because Server Edition is not containing a GUI for us which specializing in host side.

    Well, after we have installed the newest Ubuntu, let’s try our ROR with Mysql development enviroment ASAP. The official installing ROR site goes here, or you may need only type several commands as following:

    1. Installing RUBY:

     sudo apt-get install ruby-full 
    #’ruby-full’ package will make sure you have all the packages that add up to a full Ruby installation.
    2. Installing RUBYGEMS:
    wget http://rubyforge.org/frs/download.php/45905/rubygems-1.3.1.tgz
    tar xzvf rubygems-1.3.1.tgz
    cd rubygems-1.3.1
    sudo ruby setup.rb
    sudo ln -s /usr/bin/gem1.8 /usr/bin/gem 
    
    3. Installing RAILS and Mongrel:
      sudo gem install rails mongrel  
    # yeah, ROR has been done here!

    4. Installing MYSQL:

      sudo apt-get install mysql-server mysql-client   

    It seemingly worked and I could start my rails projects, but when I tried to rake task, the system was just crashed and came out a result:” !!! The bundled mysql.rb driver has been removed from Rails 2.2. Please install the mysql gem and try again: gem install mysql. rake aborted!”

    Oops, let’s try to listen to it:
    sudo gem install mysql  
    it failed:
    "Building native extensions.  This could take a while...
    ERROR:  Error installing mysql:
    ERROR: Failed to build gem native extension.
    /usr/bin/ruby1.8 extconf.rb install mysql 
    *** extconf.rb failed ***
    Could not create Makefile due to some reason, probably lack of
    necessary libraries and/or headers.  Check the mkmf.log file for more
    details.  You may need configuration options."
    It made me feel quite irritating after I tried many times without efforts, but finally, the problem was solved by installing a package named libmysql-ruby1.8. This package is an API module that allows to access MySQL database from programs in Ruby programming language.

    So let’s correct our step 4 to:

      sudo apt-get install mysql-server mysql-client   libmysql-ruby1.8

    Cheers, we can really start our RAILS application now.

    Last but not least, to make our default gnome-editor more useful as a programming IDE, we can try to install some plugins into gedit to make it like Textmate On Mac. e.g. there is an excellent project named gmate which included many plugins that we need indeed.

    Please let me know when you still have any installing problems with ROR & MYSQL on Ubuntu.

    Posted by Mysen Tue, 24 Feb 2009 00:00:00 GMT