Biodegradable Geek » Ruby

How to Maintain Static Sites with Git & Jekyll

Isam — Wed, 01 Apr 2009 04:10:00 +0000

Static sites in this context just means non-database driven sites. Your static site can be an elaborate PHP script or just a few markup and image files. For this I am using Jekyll – A neat Ruby gem that makes your static sites dynamic. It lets you create layouts and embed custom variables in your HTML (this is a “prototype” of the site).

Jekyll tackles all the nuisances involved in creating static pages (I used to add just enough PHP to make a layout). It works by running your prototype through some parsers and outputs plain static HTML/XML (RSS feeds) etc. It’s perfect for lightweight sites that would be impractical on Wordpress, like a few static pages of information, landing pages, portfolio/resume pages, and parked domains.

Git takes care of keeping your development (local) and production (remote) environments synced. Git might be a little confusing if you’re learning it with the mindset that it works like Subversion.

I’ll update this post when the guide is done. For now, the following will assume you’re familiar with Jekyll (or at least have an empty file in the prototype directory) and git. This Bash script simplifies creating the remote git repository:

** please read through the code and make sure you know what this does, and what you’re doing. As of now, this is bias towards my own Apache/vhost setup. It’s trivial to edit for your specific needs. You’re using this at your own risk.

(direct link – repogen.sh)

#!/bin/sh
# 
# 04/01/2009 | http://biodegradablegeek.com | GPL 
# 
# You should be in site (NOT public) root (be in same dir as public/ log/ etc)
# proto/ is created and will house the jekyll prototype
# public/ will be the generated static site
# the public/ folder will be REMOVED and regenerated on every push
# 
 
if [ -z "$1" ]; then
  echo "Usage: ./repogen.sh domain.comn"
  exit
fi
 
# optional. will make it easier to copy/paste cmd to clone repo 
SSHURL="ssh.domain.com"
URL="$1"
 
echo "** creating tmp repo"
mkdir proto
cd proto
git init 
touch INITIAL
git add INITIAL
git commit -a -m "Initial Commit"
 
echo "** creating bare repo"
cd ..
git clone --bare proto proto.git
mv proto proto.old
git clone proto.git
rm -rf proto.old
 
echo "** generating hook"
HOOK=proto.git/hooks/post-update
 
mv $HOOK /tmp
echo '#!/bin/sh' >> $HOOK
echo '# To enable this hook, make this file executable by "chmod +x post-update".' >> $HOOK
echo '#exec git-update-server-info' >> $HOOK
echo '' >> $HOOK
echo '' >> $HOOK
echo 'URL='"$URL" >> $HOOK
echo 'PROTO="/home/$USER/www/$URL/proto"' >> $HOOK
echo 'PUBLIC="/home/$USER/www/$URL/public"' >> $HOOK
echo  '' >> $HOOK
echo 'export GIT_DIR="$PROTO/.git"' >> $HOOK
echo 'pushd $PROTO > /dev/null' >> $HOOK
echo 'git pull' >> $HOOK
echo 'popd > /dev/null' >> $HOOK
echo '' >> $HOOK
echo "echo -----------------------------" >> $HOOK
echo "echo '** Pushing changes to '$URL" >> $HOOK
echo "echo '** Moving current public to /tmp'" >> $HOOK
echo 'mv "$PUBLIC" "/tmp/'$URL'public-`date '+%m%d%Y'`"' >> $HOOK
echo 'echo "** Generating new public"' >> $HOOK
echo 'jekyll "$PROTO" "$PUBLIC"' >> $HOOK
 
echo "** enabling hook"
chmod a+x $HOOK 
 
echo "** clone repo on local machina. example:"
echo "git clone ssh://$USER@$SSHURL/~$USER/www/$SSHURL/proto.git"

Usage

Your site structure might be different. repogen.sh is made by pasting the above code in a new file, and then chmod a+x to make it executable. This should be done on the remote server.

cd www/domain.com/
 
ls
public/ private/ log/ cgi-bin/
 
./repogen.sh domain.com

Now on your local machine, clone the new repo, move your files in, and push:

git clone ssh://[username]@ssh.domain.com/~[username]/www/domain.com/proto.git
cd proto/
cat "hello, world" > index.htm
git add index.htm
git commit -a -m 'first local commit'
git push

After you push your changes, the post-update hook will delete the public/ directory (the root of the site). This dir and its contents are automatically generated and will get wiped out on EVERY push. Keep this in mind. All your changes and content should reside in proto/.

The proto/ repo will pull in the new changes, and then Jekyll will be invoked to generate the updated site in public/ from the prototype.

Should you need to edit it, the post-update hook is in the bare git repo (proto.git/hooks/)

Thanks to the authors in the posts below for sharing ideas. I first read this git method on dmiessler’s site.

Resources:
dmiessler.com – using git to maintain static pages
toroid.org – using git to manage a web site
Jekyll @ GitHub
git info
more git info

Scraping Google Trends with Mechanize and Hpricot

Isam — Sat, 24 Jan 2009 06:53:06 +0000

This is a small Ruby script that fetches the 100 trends of the day for a specific date. If multiple dates are searched, one can find out how many times a keyword occurred between two dates, or just find out what keywords are constantly appearing on the top 100 list. Very profitable info! but alas, the script is incomplete and one must implement the “implement me!” methods to get full functionality. This, in its current state, should serve as a good starting point for scraping Google Trends.

On a technical note, it’s using mechanize, hpricot, tempfile (for the cache). A lot of this is just copy & paste programming from the earlier anime scraper.

To grab the gems (rdoc takes 10x as long as the gem to fetch and install):

sudo gem install mechanize --no-rdoc
sudo gem install hpricot --no-rdoc

#!/usr/bin/env ruby
# biodegradablegeek.com
# public domain
# 
 
require 'rubygems'
require 'hpricot'
require 'tempfile'
require 'mechanize'
#require 'highline/import'
#HighLine.track_eof = false
 
$mech = WWW::Mechanize.new
$mech.user_agent_alias = 'Mac Safari'
$master = []
 
 
def puts2(txt=''); puts "*** #{txt}"; end
 
 
class Cache
  def initialize
    # Setup physical cache location 
    @path = 'cache'
    Dir.mkdir @path unless File.exists? @path
 
    # key/val = url/filename (of fetched data)
    @datafile = "#{@path}/cache.data"
    @cache = load @datafile
  end
 
  def put key, val
    tf = Tempfile.new('googletrends', @path)
    path = tf.path
    tf.close! # important!
 
    puts2 "Saving to cache (#{path})"
    open(path, 'w') { |f|
      f.write(val)
      @cache[key] = path
    }
 
    save @datafile
  end
 
  def get key
    return nil unless exists?(key) && File.exists?(@cache[key])
    open(@cache[key], 'r') { |f| f.read }
  end
 
  def files
    @cache.values
  end
 
  def first
    @cache.first
  end
 
  def exists? key
    @cache.has_key? key
  end
 
private
  # Load saved cache 
  def load file
    return File.exists?(file) ? YAML.load(open(file).read) : {}
  end
 
  # Save cache 
  def save path
    open(path, 'w') { |f|
      f.write @cache.to_yaml
    }
  end
end
 
$cache = Cache.new
 
 
def fetch(url)
  body = $mech.get(url).body()
  $cache.put(url, body)
  body
end
 
 
def getPage(url)
  body = $cache.get(url) 
 
  if body.nil?
    puts "Not cached. Fetching from site..."
    body = fetch url 
  end
  body
end
 
 
def loadState
  mf = 'cache/master.data'
  $master = File.exists?(mf) ? YAML.load(open(mf).read) : {}
  $master = {} if $master==false
end
 
 
def saveState
  open('cache/master.data', 'w+') { |f|
    f.write $master.to_yaml
  }
end
 
 
def main
  #loadState
 
  # Grab top 100 Google Trends (today)
  #date = Time.now.strftime '%Y-%m-%d'
  date = '2009-01-21'
 
  puts2 "Getting Google's top 100 search trends for #{date}"
  url = "http://www.google.com/trends/hottrends?sa=X&date=#{date}"
  puts2 url
 
  begin
    body = getPage(url)
  rescue WWW::Mechanize::ResponseCodeError
    puts2 "Couldn't fetch URL. Invalid date..?"
    exit 5
  end
 
  puts2 "Fetched page (#{body.size} bytes)"
 
  if body['There is no data on date']
    puts2 'No data available for this date.'
    puts2 'Date might be too old or too early for report, or just invalid'
    exit 3
  end
 
  doc = Hpricot(body)
 
  (doc/"td[@class='hotColumn']/table[@class='Z2_list']//tr").each do |tr|
    td = (tr/:td)
    num = td[0].inner_text.sub('.','').strip
    kw = td[1].inner_text
    url = (td[1]/:a).first[:href]
    Keyword.find_or_new(kw) << Occurance.new(num, date, url)
  end
  puts "Got info on #{$master.size} keywords for #{date}"
  puts "keyword '#{$master.first.name}' occured #{$master.first.occurances} times"
end
 
class Occurance
  attr_accessor :pos, :date, :url
  def initialize(pos, date, url)
    @pos = pos
    @date = date
    @url = url
  end
end
 
class Keyword
  attr_accessor :name, :occurances
  def initialize(name)
    @name = name
    @occurances = []
    @position_average = nil
    @count = nil
    $master << self
  end
 
  def self.find_or_new(name)
    x = $master.find { |m| name==m.name }
    x || Keyword.new(name)
  end
 
  def << occurance
    @occurances << occurance
  end
 
  def occured_on? datetime
    raise 'implement me'
  end
 
  def occured_between? datetime
    raise 'implement me'
  end
 
  def occurances datetime=nil
    raise 'implement me' if datetime
    @occurances.size 
  end
 
  def occurances_between datetime
    raise 'implement me'
  end
 
  def pos_latest
    @occurances.last.date
  end
 
  def pos_average
    @position_average
  end
 
  def pos_average_between datetime
    raise 'implement me'
  end
end
 
#   Instance= [num, date, url]
#   Keyword=[Instance, Intance, Instance]
#   Methods for keywords: 
#   KW.occured_on? date 
#   KW.occured_between? d1, d2 
#   KW.occurances
#   KW.occurances_between? d1, d2
#   KW.pos_latest
#   KW.pos_average
#   KW.pos_average_between
 
#   KW has been on the top 100 list KW.occurances.size times
#   The #1 keywords for the month of January: Master.sort_by KW.occurances_between? Jan1,Jan31.pos_average_between Jan1,Jan31 
#
#   Top keywords: sort by KW.occurances.size = N keyword was listed the most.
#   Top keywords for date D: Master.sort_by KW.occured_on (x).num
 
main

AnimeCrazy Scraper Example Using Hpricot & Mechanize

Isam — Sun, 11 Jan 2009 20:44:40 +0000

This is a little (as of now incomplete) scraper I wrote to grab all the anime video code off of AnimeCrazy (dot) net. This site doesn’t host any videos on its own server, but just embeds ones that have been uploaded to other sites (Megavideo, YouTube, Vimeo, etc). I don’t know who the original uploaders of the videos are, but I’ve seen this same collection of anime links being used on some other sites. This site has about 10,000 episodes/parts (1 movie may have 6+ parts). The scraper below was only tested with “completed anime shows” and got around 6300 episodes. The remaining content (anime movies and running anime shows) should work as-is, but I personally held off on getting those because I want to examine them closely to try cleaning up the inconsistencies as much as possible.

This scraper needs some initial setup and won’t work out of the box, but I’m including it here in the hopes that it will serve as a decent example of a small real world scraper, if you’re looking to learn the basics of scraping with Hpricot and Mechanize. Let me know if you find any use for it. I will update the posted code later this week when I have time to complete it and add some more features.

There’s one major problem with the organization of episodes on AnimeCrazy, and it’s the fact that some episodes are glued together into one post. Right now the scraper stops and asks you how to proceed when it comes across such a post. You basically need to tell the scraper if a post (page) contains 1 episode (video) or multiple. If there’s 1, it proceeds on its own, but if there’s two, it requires that you give it the names and links of each individual episode (part1 and part2 usually). Sometimes 2 episodes are together in 1 video. Sorta like those music albums on KaZaA or LimeWire that are basically ripped as one huge mp3 instead of individual songs.

This only accounts for maybe 30-40 out of 6000 videos, and it’s not that big of a deal because the amount of work needed to proceed with the scraping is small, but it IS work, and is a bitch slap to the entire concept of automation, but coding around the issue is a major hassle and there would still be a high chance that some inconsistencies will still come through. It would be far less work to just find another anime site which is far more consistent, though the reason animecrazy is good is because it’s active, and the site IS updated manually these days, as far as I can tell.

BTW, Why The Lucky Stiff rocks, and Hpricot is amazing. But the serious scrapologist should consider scrAPI or sCRUBYt (uses Hpricot) for big projects.

#!/usr/bin/env ruby
# License: Public domain. Go sell it to newbs on DigitalPoint.
 
require 'rubygems'
require 'hpricot'
require 'mechanize'
require 'tempfile'
require 'highline/import'
HighLine.track_eof = false
 
$mech = WWW::Mechanize.new
$mech.user_agent_alias = 'Mac Safari'
 
###############################
$skip_until = false
DEBUG=false
###############################
 
def debug?
  DEBUG
end
 
def puts2(txt='')
  puts "*** #{txt}"
end
 
#  Anime has: title, type (series, movie), series
#  Episode has name/#, description, parts (video code)
 
class Episode
  attr_accessor :name, :src, :desc, :cover
  def initialize(title, page)
    @src = page # parts (megavideo, youtube etc)
    @name = title
    @desc = nil # episode description
    @cover = nil # file path
  end
end
 
class Anime
  attr_accessor :name, :page, :completed, :anime_type, :episodes
  def initialize(title, page)
    @name = title
    @page = page
    @episodes = []
    @anime_type = 'series'
    @completed = false
  end
 
  def complete!
    @completed = true
  end
 
  def episode! episode
    @episodes &lt;&lt; episode
  end
end
 
class Cache
  def initialize
    # Setup physical cache location
    @path = 'cache'
    Dir.mkdir @path unless File.exists? @path
 
    # key/val = url/filename (of fetched data)
    @datafile = "#{@path}/cache.data"
    @cache = load @datafile
    #puts @cache.inspect
  end
 
  def put key, val
    tf = Tempfile.new('animecrazy', @path)
    path = tf.path
    tf.close! # important!
 
    puts2 "Saving to cache (#{path})"
    open(path, 'w') { |f|
      f.write(val)
      @cache[key] = path
    }
 
    save @datafile
  end
 
  def get key
    return nil unless exists?(key) &amp;&amp; File.exists?(@cache[key])
    open(@cache[key], 'r') { |f| f.read }
  end
 
  def exists? key
    @cache.has_key? key
  end
 
private
  # Load saved cache
  def load file
    return File.exists?(file) ? YAML.load(open(file).read) : {}
  end
 
  # Save cache
  def save path
    open(path, 'w') { |f|
      f.write @cache.to_yaml
    }
  end
end
 
$cache = Cache.new
 
def fetch(url)
  body = $mech.get(url).body()
  $cache.put(url, body)
  body
end
 
def getPage(url)
  # First let's see if this is cached already.
  body = $cache.get(url) 
 
  if body.nil?
    puts "Not cached. Fetching from site..."
    body = fetch url
  end
  body
end
 
def main
  # Open anime list (anime_list = saved HTML of
<ul>...ul>
sidebar from animecrazy.net)
  anime_list = Hpricot(open('anime_list', 'r') { |f| f.read })
  puts2 "Anime list open"
 
  # Read in the URL to every series
  masterlist = []
 
  (anime_list/:li/:a).each do |series|
    anime = Anime.new(series.inner_text, series[:href])
    masterlist &lt;&lt; anime
    puts2 "Built structure for #{anime.name}..."
  end
 
  puts2
 
  puts2 "Fetched #{masterlist.size} animes. Now fetching episodes..."
  masterlist.each do |anime|
    puts2 "Fetching body (#{anime.name})"
    body = getPage(anime.page)
    puts2 "Snatched that bitch (#{body.size} bytes of Goku Goodness)"
    puts2
 
    doc = Hpricot(body)
    (doc/"h1/a[@rel='bookmark']").each do |episode|
      name = clean(episode.inner_text)
 
      if $skip_until
        #$skip_until = !inUrl(episode[:href], 'basilisk-episode-2')
        #$skip_until = nil == name['Tsubasa Chronicles']
        puts2 "Resuming from #{episode[:href]}" if !$skip_until
        next
      end
 
      # Here it gets tricky. This is a major source of inconsistencies in the site.
      # They group episodes into 1 post sometimes, and the only way to find
      # out from the title of the post is by checking for the following patterns
      # (7 and 8 are example episode #s)
      # X = 7+8, 7 + 8, 7 and 8, 7and8, 7 & 8, 7&8
 
      # If an episode has no X then it is 1 episode.
      # If it has multiple parts, they are mirrors.
      if single_episode? name
        begin
 
          puts2 "Adding episode #{name}..."
          ep = Episode.new(name, episode[:href])
          ep.src = getPage(episode[:href])
          anime.episode! ep
        rescue WWW::Mechanize::ResponseCodeError
          puts2 "ERROR: Page not found? Skipping..."
          puts name
          puts2 episode[:href]
        end
      else
        # If an episode DOES have X, it *may* have 2 episodes (but may have mirrors, going up to 4 parts/vids per page).
        # Multiple parts will be the individual episodes in chronological order.
        puts2 "Help me! I'm confused @ '#{name}'"
        puts2 "This post might contain multiple episodes..."
 
        puts2 "Please visit this URL and verify the following:"
        puts episode[:href]
 
        if agree("Is this 1 episode? yes/no ")
          begin
            puts2 "Adding episode #{name}..."
            ep = Episode.new(name, episode[:href])
            ep.src = getPage(episode[:href])
            anime.episode! ep
          rescue WWW::Mechanize::ResponseCodeError
            puts2 "ERROR: Page not found? Skipping..."
            puts name
            puts2 episode[:href]
          end
        else
          more = true
          while more
            ename = ask("Enter the name of an episode: ")
            eurl =  ask("Enter the URL of an episode: ")
 
            begin
              puts2 "Adding episode #{ename}..."
              ep = Episode.new(name, episode[:href])
              ep.src = getPage(episode[:href])
              anime.episode! ep
            rescue WWW::Mechanize::ResponseCodeError
              puts2 "ERROR: Page not found? Skipping..."
              puts name
              puts2 episode[:href]
            end
            more = agree("Add another episode? Y/N")
          end
          puts2 "Added episodes manually... moving on"
        end
      end
    end
    anime.complete!
    # XXX save the entire anime object, instead of just cache
  end
end
 
def inTitle(document, title)
  return (document/:title).inner_text[title]
end
 
def inUrl(url, part)
  return url[part]
end
 
def single_episode?(name)
  !(name =~ /[0-9] ?([+&amp;]|and) ?[0-9]/)
end
 
def clean(txt)
  # This picks up most of them, but some are missing. Like *Final* and just plain "Final"
  txt[' (Final)']='' if txt[' (Final)']
  txt[' (Final Episode)']='' if txt[' (Final Episode)']
  txt[' (FINAL)']='' if txt[' (FINAL)']
  txt[' (FINAL EPISODE)']='' if txt[' (FINAL EPISODE)']
 
  txt['(Final)']='' if txt['(Final)']
  txt['(Final Episode)']='' if txt['(Final Episode)']
  txt['(FINAL)']='' if txt[' (FINAL)']
  txt['(FINAL EPISODE)']='' if txt[' (FINAL EPISODE)']
 
  txt
end
 
main

If you’re writing your own scraper and would like to use the minimal caching functionality present below, you can gut everything in main() out and put in your own code. Feel free to contact me for assistance.

Here is some sample output:

*** Adding episode Initial D: Episode 1 (Stage 2)...
Not cached. Fetching from site...
*** Saving to cache (cache/animecrazy20090111-12300-mbdpcl-0)
*** Fetching body (Initial D: Third Stage)
*** Snatched that bitch (77695 bytes of Goku Goodness)
***
*** Adding episode Initial D: Third Stage...
Not cached. Fetching from site...
*** Saving to cache (cache/animecrazy20090111-12300-ea69nr-0)
*** Fetching body (Kaiji)
*** Snatched that bitch (87553 bytes of Goku Goodness)
***
*** Adding episode Basilisk Episode 4...
Not cached. Fetching from site...
*** Saving to cache (cache/animecrazy20090110-14992-fomoh0-0)
*** Adding episode Basilisk Episode 3...
Not cached. Fetching from site...
*** Saving to cache (cache/animecrazy20090110-14992-1dx9xm-0)
*** Adding episode Basilisk Episode 2...
Not cached. Fetching from site...
*** Saving to cache (cache/animecrazy20090110-14992-5xt774-0)
*** Adding episode Basilisk Episode 1...
Not cached. Fetching from site...
*** Saving to cache (cache/animecrazy20090110-14992-br5fxd-0)
*** Adding episode Tsubasa Chronicles: Tokyo Revelations Episode 3...
Not cached. Fetching from site...
*** Saving to cache (cache/animecrazy20090110-14992-zmuwix-0)
*** Adding episode Tsubasa Chronicles: Tokyo Revelations Episode 2...
Not cached. Fetching from site...
*** Saving to cache (cache/animecrazy20090110-14992-1ah20eg-0)
*** Adding episode Tsubasa Chronicles: Tokyo Revelations Episode 1...
Not cached. Fetching from site...

This was written for fun, but primarily profit, and not for my own viewing pleasure. The only anime I’ve seen was Akira a decade or so ago, and only because the cover looked cool, but feel free to recommend your favorites.

Script to Quickly Setup WebApp Environment and Domain

Isam — Sun, 12 Oct 2008 04:10:32 +0000

Just sharing a script I wrote to quickly deploy Wordpress (and eventually a few other webapps) sites, which somebody might find useful. This uses Linode’s API* to add the domain name to the DNS server along with some subdomains. If you’re using another server, (Slicehost, your own, etc), you can alter the dns class to use that API, or just ignore the DNS stuff completely; Its optional.

This will be updated periodically as I refactor and add support for more apps (notably Joomla and Clipshare – though this would violate their terms unless you have the unlimited license). This was written primarily because I couldn’t stand setting up another vhost and Wordpress installation. There are plenty of existing deployers but I plan on adding very specific features and tweaking this for in-house work. I also wanted to try Rio (Ruby-IO). GPL license. Go nuts.

* As of 10/11, the apicore.rb file on the site has some syntactic errors in the domainResourceSave method. I sent an email out to the author about it. Problems aren’t major. You can get my apicore.rb here.

This won’t run unless you create the appropriate folder structure in /etc/mksite/. I’ll get going on this in a bit. See the code below:

#!/usr/bin/env ruby 
 
########################################################
# Isam M. 
# http://biodegradablegeek.com
# MKSITE.RB (0.2) Last updated Oct 19th, 2008
#
# mksite makes it quicker to setup sites and web
# apps by doing most of the tedious work.
#
#
#                     UNSTABLE
#           Run it in your imagination,
#              not on your system!
#
#
######################################################## 
 
require 'rubygems'
require 'rio'
require 'yaml'
require 'mysql'
require 'highline/import'
HighLine.track_eof = false
 
begin
  require 'apicore'
rescue LoadError
  puts "NOTICE: Unable to load apicore.rb - domains will not be added automatically"
end
 
Apache_sites = '/etc/apache2/sites-available/'
Subdomains = ['', 'www', 'mail', 'blog', "dev#{(rand*10).floor}"]
Testing = false
Homedir = ENV['HOME']
Username = ENV['USER']
Applications = [:Skeleton, :Clipshare, :Joomla, :PHPMotion, 'PmWiki (N/A)', :Wordpress]
$config = nil
$log = nil
 
def say(txt)
  super txt
#  $log << txt if $log
end
 
# flash('message', :notice || :error) to output msg
def flash(msg, message_type = :notice)
 if message_type.eql? :notice
    say("INFORMATION: #{msg}")
  elsif message_type.eql? :emphasize
    say("***************************************************")
    say("NOTICE: #{msg}n")
    say("***************************************************")
  elsif message_type.eql? :error
    STDERR.puts("FATAL ERROR: #{msg}n")
  end
end
 
# DZone 2111
def genAlpha(size=64)
  s=''
  size.times {
    s &lt;&lt; (i = Kernel.rand(62); i += ((i &lt; 10) ? 48 : ((i &lt; 36) ? 55 : 61 ))).chr
  }
  s
end
 
if defined? ApiCore
  class DNSAPI &lt; ApiCore
    # This depends on the Linode API Ruby bindings (apicore.rb)
    def initialize(key, debug=false, batching=false)
      super
      @batching=false
    end
 
    # Add domain. Return ID on success, nil on failure
    def addDomain domain
      return nil if !domain
      params = {}
      params[:DomainID] = 0
      params[:Domain] = domain
      params[:Type] = 'master'
      params[:Status] = 1
      params[:SOA_Email] = getVal('email') || ask("Enter SOA email for the domain: ")
      domainSave params
    end
 
    def addDomainResource(domain, resource, target, record_type = 'A')
      return nil if (did = getDomainIdByName domain).nil?
      params = {}
      params[:ResourceID] = 0
      params[:DomainID] = did
      params[:Name] = resource
      params[:Type] = record_type
      params[:Target] = target
      domainResourceSave params
    end
 
    def getDomainIdByName domain
      domainList.find do |dom|
        return dom["DOMAINID"] if dom["DOMAIN"].downcase == domain.downcase
      end
    end
  end
else
  flash('ApiCore not loaded. Skipping DNS stuff', :notice)
end
 
class App
  class &lt;&lt; self; attr_reader :message, :name, :version, :description, :vhost; end
  @message = nil
  @name = nil
  @version = nil
  @description = 'Just Another Web App'
  @vhost = 'generic'
 
  def initialize(rootdir, domain, db)
    flash("Initializing application (dir=#{rootdir}, domain=#{domain})")
    @rootdir = (rootdir[-1].chr.eql? '/') ? rootdir.chop : rootdir
    @domain = domain
    @db = db # hash of database info 
 
    # System stuff
    #if Testing
      #@templates = '/home/kiwi/Code/mksite/templates'
      #@configs = '/home/kiwi/Code/mksite/configs'
      #@vhosts = '/home/kiwi/Code/mksite/vhosts'
    #else
      @templates = '/etc/mksite/templates'
      @configs = '/etc/mksite/configs'
      @vhosts = '/etc/mksite/vhosts'
    #end
  end
 
  # This generally does not need to be overriden.
  # It does 'generic shit' like creating the rootdir
  # and setting permissions
  def envSetup
    if !rio(@rootdir).exist?
      flash("Creating directory #{@rootdir}")
      rio(@rootdir).mkpath
    end
 
    rio(@rootdir).chdir do |root|
      # Copy the generic public/private/log apache
      # structure to rootdir
      flash("Changing working dir to #{@rootdir}")
      flash("Working inside '#{root.to_s}'")
      rio(@templates,'skeleton.www').each { |df|
        # Overwrite existing files? .. yes.
        #while rio(root, df).exist? do
        df &gt; root
        #flash("Copied #{df.to_s} to #{root.to_s}")
      } 
 
      # Set permissions (a+w on logs, etc)
      flash('Setting permissions...')
      flash('666 ./log/*.log')
      rio('./log/access.log').chmod(00666)
      rio('./log/error.log').chmod(00666)
      flash('700 ./private')
      rio('./private/').chmod(00700)
    end
  end
 
  def setup
    # envSetup()
    # databaseSetup()
    raise 'OVERRIDE ME'
    #flash("setup() template function invoked. 'OVERRIDE ME'", :log)
  end
 
  def databaseSetup
    # Create database if it doesn't already exist
    # This usually doesn't need to be overriden
    #Mysql.server_connect(@db['name'])
    #  flash('Checking database connection... ')
    begin
      dbo = Mysql.real_connect(@db['host'], @db['user'], @db['pass'])
      flash("Creating database '#{@db['name']}'");
      res = dbo.query("CREATE DATABASE IF NOT EXISTS #{@db['name']};")
      flash("Database server returned #{res}") if res
    rescue Mysql::Error =&gt; err
      flash('Unable to connect to access/create database', :error)
      flash("Error returned (#{err.errno}) = '#{err.error}'", :error)
      exit 1
    ensure
      dbo.close if dbo
    end
  end
 
  def dnsSetup
    return if !defined? ApiCore
 
    # Setup DNS - currently uses Linode API
    flash("Setting up DNS for '#{@domain}'")
    flash("This requires a Linode API keynLogin to linode.com and find it under 'My Profile'", :emphasize)
    dns = nil
    api_key = getVal('apikey')
    loop do
      api_key = ask('Paste your Linode API key (or 'skip'): ') if api_key.nil? || api_key.empty?
      break if api_key.downcase.eql? 'skip'
 
      # Check API key
      begin
        dns = DNSAPI.new api_key
        dns.domainList
        break
      rescue RuntimeError
        flash("API key invalid (or service down?). Learn to paste and try again.n")
        api_key = nil
      end
    end
 
    unless api_key.nil? or api_key.downcase.eql? 'skip'
      flash("Adding master domain '#{@domain}'")
      begin
        begin
          dns.addDomain(@domain)
        rescue RuntimeError
          flash('Unable to add domain (exists already?). Attempting to add subdomains...', :error)
        end
 
        server = getVal('server') ||
                   ask("Enter IP subdomains should point to (or 'skip'): ") { |q| q.default = 'skip' } 
 
        unless server.downcase.eql? 'skip'
          Subdomains.each do |sub|
            flash("Adding subdomain '#{sub}.#{@domain}' (points to #{server})")
            flash('Could not add subdomain', :error) if !dns.addDomainResource(@domain, sub, server)
          end
        end
     rescue
       raise
       flash("Unable to add domain/subdomain. Skipping", :error)
     end
    end
  end
 
  def serverSetup
    # Set Apache vhost
    vhost_file = (defined? self.class.vhost) ? self.class.vhost : 'generic'
    flash("Generating vhost file (#{@vhosts}/#{vhost_file}) for Apache 2.x")
    vhost = ''
    email = getVal('email') || ask("Enter a valid email for tech support: ")
    rio(@vhosts, vhost_file) &gt; vhost
    vhost.gsub!('_MKS_DOMAIN_', @domain)
    vhost.gsub!('_MKS_EMAIL_', email)
    vhost.gsub!('_MKS_ROOT_', "#{@rootdir}")
    vhost.gsub!('_MKS_PUBLIC_', "#{@rootdir}/public")
    #rio(Testing ? '/tmp/' : Apache_sites, @domain).puts(vhost)
    rio('/tmp/', @domain).puts(vhost) 
 
    # Your enemies should not read this
    rio('/tmp/', @domain).chmod(00600)
    flash("vhost file has been generated as /tmp/#{@domain}n
          It is YOUR responsibility to move this to #{Apache_sites}n
          Site will not work until you 'a2ensite && apache2ctl restart'", :emphasize)
  end
 
  def postInstall()
    flash("Finished!", :emphasize)
  end
end
 
class Skeleton &lt; App
  @name = 'Skeleton'
  @description = 'Generic WWW directory structure'
  @message = ''
  @vhost = 'generic'
 
  def setup
    self.envSetup()
    self.dnsSetup()
    self.serverSetup()
  end
end
 
class Wordpress &lt; App
  @name = 'Wordpress'
  @version = '2.6.2'
  @description = 'A popular blogging platform'
  @message = 'Trying to make monies on the Internets?'
  @vhost = 'generic'
 
  def setup
    # This sets up the initial environment / permissions
    self.envSetup() 
 
    # Copy the wordpress skeleton directory to the new dir
    wproot = rio(@rootdir,'public')
    flash("Wordpress root will be #{wproot}")
    flash('Copying Wordpress data over... (may take awhile)')
    rio(@templates,'wordpress').each { |df| df &gt; wproot } 
 
    rio(wproot).chdir do
      rio('wp-config-sample.php').rm()
      # Generate and output wp config
      flash('Generating wp-config.php based on your DB settings...')
      wpcfg = rio(@configs, 'wordpress.cfg')
      if !wpcfg.exist? || !wpcfg.readable?
        flash("Wordpress config template missing or unreadable, quitting", :error)
        exit 2
      end
 
      # Copy the config into a string, do things with it and then write it to disk
      config = ''
      rio(@configs, 'wordpress.cfg') &gt; config
      flash("Warning: config file '#{@configs}/wordpress.cfg' is empty") if config.empty?
      config['_MKS_DB_HOST_'] = @db['host']
      config['_MKS_DB_USER_'] = @db['user']
      config['_MKS_DB_PASS_'] = @db['pass']
      config['_MKS_DB_NAME_'] = @db['name']
      config.gsub!('_MKS_SECRET_', genAlpha())
      flash("Writing #{wpcfg} data")
      rio('wp-config.php').w!.puts(config)
    end
 
    # Setup the database
    self.databaseSetup() 
 
    # Add DNS info
    self.dnsSetup() 
 
    # Setup the server/vhost
    self.serverSetup()
  end
end
 
class Clipshare &lt; App
  def setup
    raise 'Clipshare support is currently not available. sowwie'
  end
end
 
class Joomla &lt; App
  @name = 'Joomla'
  @version = '1.5.7'
  @description = 'A widely used Content Management System'
  @message = ''
  @vhost = 'generic'
 
  def setup
    # This sets up the initial environment / permissions
    self.envSetup() 
 
    # Copy the wordpress skeleton directory to the new dir
    flash('Copying Joomla data over... (may take awhile)')
    jooroot = rio(@rootdir,'public')
    flash("Joomla root will be #{jooroot}")
    rio(@templates,'joomla-1.5.7').each { |df| df &gt; jooroot } 
 
    rio(jooroot).chdir do
      flash('Removing stock Joomla config...')
      rio('configuration.php-dist').rm()
 
      # Generate and output joomla config
      flash('Generating configuration.php based on provided settings...')
      joocfg = rio(@configs, 'joomla.cfg')
      if !joocfg.exist? || !joocfg.readable?
        flash("Joomla config template missing or unreadable, quitting", :error)
        exit 2
      end
 
      # Copy the config into a string, do things with it and then write it to disk
      config = ''
      rio(@configs, 'joomla.cfg') &gt; config
      flash("Warning: config file '#{@configs}/joomla.cfg' is empty") if config.empty?
      config['_MKS_DB_HOST_'] = @db['host']
      config['_MKS_DB_USER_'] = @db['user']
      config['_MKS_DB_PASS_'] = @db['pass']
      config['_MKS_DB_NAME_'] = @db['name']
      config.gsub!('_MKS_DOMAIN_', @domain)
      config.gsub!('_MKS_SECRET_', genAlpha()) 
 
      flash("Writing ./configuration.php")
      rio('configuration.php').w!.puts(config) 
 
      flash("Making configuration.php world writable (0666)")
      rio('configuration.php').chmod(0666)
    end
 
    # Setup the database
    self.databaseSetup() 
 
    # Add DNS info
    self.dnsSetup() 
 
    # Setup the server/vhost
    self.serverSetup()
  end
 
  def postInstall()
    flash('Joomla is ready to be setup using the web interface', :emphasize)
    flash("Go to #{@domain} where the Joomla! web based installer will
           guide you through the rest of the installation")
    flash("Here's the database information:n
           USERNAME: #{@db['user']}n
           DB HOST : #{@db['host']}n
           DB NAME : #{@db['name']}") 
 
    flash("nAdmin panel is located @ #{@domain}/administrator ")
    flash("You can log into Admin using the username 'admin' along with the
    password that was generated or you chose during the web based install.")
    super
  end
end
 
class PHPMotion &lt; App
  def setup
    raise 'PHPMotion support is currently not available. sowwie'
  end
end
 
class Log
  def initialize(filename="/tmp/#{Username}_mksite.log")
    @log = rio(filename)
    n=0; @log = rio("#{filename}.#{n+=1}") while @log.exist?
    @log.puts("# GENERATED BY MKS - BEGAN @ #{Time.now.to_i}")
    @log.chmod(00600)
    flash("Log file generated (important): #{@log.to_s} (no worries, set to 600)", :emphasize)
    return @log
  end
 
  # append to log
  def &lt;&lt;(data)
    #super <<(data)
    @log.puts(data)
    @log.puts("n")
    #@log << data << "n"
  end
end
 
def loadConfig()
  cfgpath = "#{ENV['HOME']}/.mksite"
  return YAML.load(rio(cfgpath).read()) if rio(cfgpath).exist?
  nil
end
 
def getVal(key, default=nil)
  return $config[key] if $config and defined? $config[key]
  default
end
 
def main
  exit 1 if 'root'==ENV['USER']
  if ($config = loadConfig())
    flash('~/.mksite config loaded')
  else
    flash('~/.mksite not found. It's fine, I'll annoy you with questions.')
  end
 
  puts $config.inspect if Testing
  flash('Press ^C (CTRL+C) at any time quit', :emphasize)
  # Let's ask neutral questions about the new site.
  if Testing
    rootdir = '/tmp/sandbox9/'
    domain = 'domain.cxm'
  else
    domain = ask('Enter site's domain name (no http:// or www): ') #{ |d| d.validate = !! /^www.|^http:/// }
    rootdir = ask('Root site directory (leave blank for default): ') { |q|
      q.default = "/home/#{Username}/www/#{domain}/"
      q.validate = /^/home/#{Username}//
    }
  end
 
  flash("Domain has been set to "#{domain}"")
  #$log = Log.new("/tmp/mks_#{domain}.log") 
 
  flash("Site will reside in "#{rootdir}", and the index/script")
  flash(" files (index.php, .htaccess etc) will go in "#{rootdir}/public/"nn")
 
  # Ask the user what app she wants to install and specific questions about that app
  app = nil
  choose do |menu|
    menu.prompt = 'Choose the software for your new site: '
    Applications.each do |app|
      menu.choice app do |a|
        app = a
      end
    end
  end
 
  flash("You chose #{app}. #{Kernel.const_get(app).message}n") 
 
  # Fetch DB info if this app needs a database
  # XXX Should just have a db flag in the App's class
  db = nil
  unless app.eql? :Skeleton
    flash('This app requires a database (only MySQL supported)')
    flash('It will be created if it doesn't exist')
 
    db = {}
    db['host'] = 'localhost'
    loop do
      if Testing
        db['user'] = 'kiwi'
        db['pass'] = ''
        db['name'] = 'kiwi_sandbox9'
      else
        db['user'] = getVal('db_user') || ask('Enter MySQL username: ')
        db['pass'] = getVal('db_pass') || ask('Enter MySQL password: ') { |q| q.echo = '*'}
        db['name'] = ask('Enter database name (should begin w/ your 'username_')') { |dbn| dbn.validate = /^#{Username}_/}
        puts db.inspect
      end
      flash("Please double check: mysql://#{db['user']}:#{'*' * (db['pass']).size}@#{db['host']}/#{db['name']}")
      break if Testing || agree('Is this correct?')
      flash("Please re-enter database infon")
    end
  end
 
  # Check permissions and database login
  flash('Doing a preliminary check', :emphasize)
  flash('Checking installation environment')
  # Quit if the directory exists and the user does not want to go ahead
  unless Testing
    exit 1 if rio(rootdir).exist? && !agree("Directory '#{rootdir}' exists! Continue? (Y/N)")
  end
 
  # Is the parent directory available and writable?
  parentdir = rio(rootdir).dirname
  unless parentdir.exist? &amp;&amp; parentdir.writable?
    flash("Directory '#{parentdir.to_s}' either doesn't exist or is not writable. Check permissions", :error)
    exit 1
  end
 
  unless db.nil? || false==getVal('db_confirm')
    # Check DB connection - but this does not check if user has any privileges
    flash('Checking database connection')
    begin
      dbo = Mysql.real_connect(db['host'], db['user'], db['pass'])
      flash("Successfully connected to '#{dbo.get_server_info}'")
    rescue Mysql::Error =&gt; err
      flash('Unable to connect to database server', :error)
      flash("Error returned (#{err.errno}) = '#{err.error}'", :error)
      exit 1
    ensure
      dbo.close if dbo
    end
  end
 
  # Begin installation
  flash("Word on the server racks is... you're good to go", :emphasize)
  klass = Kernel.const_get(app)
  webapp = klass.new(rootdir, domain, db)
  webapp.setup()
  webapp.postInstall()
end
 
main()

Download the ASCII mksite.rb file here.

Got API? Instantly Search API Documentation

Isam — Sun, 01 Jun 2008 07:21:04 +0000

gotAPI.com does an excellent job congregating API documentation for numerous programming languages under an AJAX interface. No more bulging neck veins or fulmination when you can’t remember the order of those pesky arguments.

No support for your favorite language? Contribute.

You can add a gotAPI Search Widget to your site: http://www.gotapi.com/widgets/index.html
See Ruby/Rails widget below (requires Javascript). Still in beta and might have UI issues, but it’s functional. Try typing map or validates and hitting enter.

Quick Ruby / Rails lookup

How to POST Form Data Using Ruby

Isam — Thu, 24 Apr 2008 15:43:09 +0000

POSTing data on web forms is essential for writing tools and services that interact with resources already available on the web. You can grab information from your Gmail account, add a new thread to a forum from your own app, etc.

The following is a brief example on how this can be done in Ruby using Net::HTTPand this POST form example.

Looking at the source (interlacken.com/webdbdev/ch05/formpost.asp):




#!/usr/bin/ruby

require "uri"
require "net/http"

params = {'box1′ => 'Nothing is less important than which fork you use. Etiquette is the science of living. It embraces everything. It is ethics. It is honor. -Emily Post',
'button1′ => 'Submit'
}
x = Net::HTTP.post_form(URI.parse('http://www.interlacken.com/webdbdev/ch05/formpost.asp'), params)
puts x.body

# Uncomment this if you want output in a file
# File.open('out.htm', 'w') { |f| f.write x.body }

Sending the value of button1 is optional in this case, but sometimes this value is checked in the server side script. One example is when the coder wants to find out if the form has been submitted – as opposed to it being the user’s first visit to the form – without creating a hidden attribute to send along w/ the other form fields. Besides, there’s no harm in sending a few more bytes.

If you’re curious about URI.parse, it simply makes the URI easier to work with by separating and classifying each of its attributes, effectively letting the methods in Net::HTTP do their sole job only, instead of having to analyze and parse the URL. More info on this in the Ruby doc.

Assuming no errors, running this example (ruby postpost or chmod a+x postpost.rb; ./postpost.rb) yields:

Mechanize and Rest-client.

Expanding List of Ruby on Rails Sites, Editors, Tutorials, Cheat Sheets and More

Isam — Wed, 02 Jan 2008 18:24:29 +0000

Most of these Ruby on Rails related links are right out of my bookmarks. I checked for 404s and added more recent entries, but this list is far from complete. New blogs, sites, tutorials and tools are released on a daily basis. Just leave a comment or contact me if you want your link(s) or anything related added, or have spotted any broken links.

(Updated 02/22/2008. Keep the suggestions coming!)

Installation Guides

These are to help you get Ruby and Rails installed and configured, ready for Hello World.

Freeware and Commercial Ruby on Rails IDEs and Editors

Freeware IDEs
NetBeans IDE (Windows/Linux/Mac OS X/Solaris)
Aptana Studio (Community Edition)
Mondrian IDE (Windows/Linux)
RAD Rails (now part of Aptana Studio Suite, still free)
RoRED (Windows only at the moment)
FreeRIDE – IDE for Ruby
Commercial IDEs
Aptana Studio (Professional Edition)
Komodo IDE (Windows/Mac/Linux)
Arachno Ruby (Windows/Linux, OS X release pending)
TextMate (OS X only)
Editors
JEdit – Programmer’s Text Editor
GEdit (Official Gnome text editor)
Textmate-like Gedit in few steps
SciTE – A free source code editor for Win32 and X
Using Vim as a Complete Ruby on Rails IDE (Shameless plug)
rails.vim – Ruby on Rails IDE
Ruby on Rails on Emacs (video)
How To Use Emacs With Rails
Extensions and everything else
Ruby Development Tool (RDT) Requires Eclipse
Rubyweaver – Dreamweaver extension (Compatible with MX through CS3)
Rails.vim (Plugin for vi/vim/gvim/cream)
snippetsEmuAn attempt to emulate TextMate’s snippet expansion
intellij IDEA Ruby Plugn Plugin for the intellijIDEA Java IDE

Hello World!

Rails Screencasts

Code Examples and Snippets

Ruby / Rails Blogs

These are blogs dedicated primarily to Ruby and/or Rails.

Communities / Wikis / Forums / Mailing lists / IRC

Gems / Plugins / Tools

These are some miscellaneous Rails stuff I had bookmarked. I know there are hundreds of great things I’m missing here. Don’t hesitate to recommend your favorites.

Rails 2.0

Free Ruby / Rails Books

These are books that you can read online or download for free.

References / Cheatsheets

Handy references and cheat sheets to bookmark or print out.

Using Vim as a Complete Ruby on Rails IDE

Isam — Thu, 13 Dec 2007 23:11:46 +0000

NOTE: If you are experiencing segmentation faults with vim and rails.vim, see this post.

When coding in Ruby on Rails, you’ll usually be switching between files and running scripts a lot. It can be time-consuming and frustrating coding Rails using a traditional text editor designed for working on big files individually. Vim lets you hop around within a file with enough speed to activate the cosmic treadmill – but without a plethora of hacks and custom key mappings, it’s weak as a Rails IDE. Fortunately, for those of us who are reluctant to kick the vim habit, Tim Pope comes to the rescue with rails.vim; A plugin that makes working with Rails in vim painless and efficient. In this guide, I will explain how to install and use rails.vim, along with a few other plugins you’ll find useful when writing Rails applications.

Vim would certainly fail the mom test, and it’s not something you can learn in one sitting. Learning vim is a lifelong process, and there are many sites and books that go over it extensively, therefore I won’t do that here. This guide assumes that you at least know what vim is and can open, edit and save files. Rails.vim works for vim, gvim and Cream, so vim in this article is referring to any of these three, unless otherwise stated.

For those who prefer Emacs, check out Effective Emacs by Steve Yegge and the official Wiki’s HowTouseEmacsWithRails.

TOC
Configuring vim/gvim
Rails.vim and Other Helpful Plugins
Usage and Examples
    Starting a New Project and Setting Up The Database
        Database.yml Setup
    Generating Models
    Migrations and :Rake
        :Rmigration pattern
        Related Files and The Command :R and its Mapping ]f
    A Glimpse of the Vim Database Extension Plugin
    Controllers, Views and Interfile Navigation
        :R or ]f
        :Rview [[controller/]view]
        :Rfind [name]
        gf ("goto file")
    Views and Partials
    We've Got You Surrounded
        Caveats
        Adding Surroundings
        Replacing Surroundings
        Removing Surroundings
        Rails.vim and Custom Surroundings
But Wait, There's More

Configuring Vim/gvim

Open your vim configuration file or create one if it doesn’t exist (i.e., new installation):

$HOME/.vimrc in Linux and Mac OS X (might be ~/.gvimrc instead)
$HOME\_vimrc (or $HOME\_gvimrc) in Windows

If you’re having trouble finding out where the config file goes, type “:version” and hit enter in vim for a list of compile flags, options and directories vim is using.

Add the following to the file and save it (if you must worry about what they do, see the vimdoc)

filetype on  " Automatically detect file types.
set nocompatible  " We don't want vi compatibility.
 
" Add recently accessed projects menu (project plugin)
set viminfo^=!
 
" Minibuffer Explorer Settings
let g:miniBufExplMapWindowNavVim = 1
let g:miniBufExplMapWindowNavArrows = 1
let g:miniBufExplMapCTabSwitchBufs = 1
let g:miniBufExplModSelTarget = 1
 
" alt+n or alt+p to navigate between entries in QuickFix
map   :cp 
map   :cn 
 
" Change which file opens after executing :Rails command
let g:rails_default_file='config/database.yml'
 
syntax enable

Before installing rails.vim, you may want to glance over some additional (g)vim options I use. Otherwise, skip this optional step and jump directly to the installation.

The following settings in my config are worth looking over. These can be appended to the same vimrc file as above:

set cf  " Enable error files & error jumping.
set clipboard+=unnamed  " Yanks go on clipboard instead.
set history=256  " Number of things to remember in history.
set autowrite  " Writes on make/shell commands
set ruler  " Ruler on
set nu  " Line numbers on
set nowrap  " Line wrapping off
set timeoutlen=250  " Time to wait after ESC (default causes an annoying delay)
" colorscheme vividchalk  " Uncomment this to set a default theme
 
" Formatting (some of these are for coding in C and C++)
set ts=2  " Tabs are 2 spaces
set bs=2  " Backspace over everything in insert mode
set shiftwidth=2  " Tabs under smart indent
set nocp incsearch
set cinoptions=:0,p0,t0
set cinwords=if,else,while,do,for,switch,case
set formatoptions=tcqr
set cindent
set autoindent
set smarttab
set expandtab
 
" Visual
set showmatch  " Show matching brackets.
set mat=5  " Bracket blinking.
set list
" Show $ at end of line and trailing space as ~
set lcs=tab:\ \ ,eol:$,trail:~,extends:>,precedes:<
set novisualbell  " No blinking .
set noerrorbells  " No noise.
set laststatus=2  " Always show status line.
 
" gvim specific
set mousehide  " Hide mouse after chars typed
set mouse=a  " Mouse in all modes

The following optional commands are helpful but require explicit creation of directories and files:

" Backups & Files
set backup                     " Enable creation of backup file.
set backupdir=~/.vim/backups " Where backups will go.
set directory=~/.vim/tmp     " Where temporary files will go.

Run vim to make sure it doesn’t halt to display any warnings or error messages. If it does, check that the specific folders (~/.vim/skeletons and ~/.vim/backups etc) exist and vim has write permission on them.

Rails.vim and Other Helpful Plugins

Installing vim plugins just requires putting files where they belong in vim’s root directory. We will be installing rails.vim along with some dependency plugins and a few others that make coding Rails in vim much simpler.

Below is a list of all the plugins you need to download and install. If they are .vim files, move them into your plugin folder, probably $HOME/.vim/plugin or $VIM\vimfiles\plugin (probably C:\vim\vimfiles\plugin). Doc files (.txt) go in $HOME/.vim/doc or vimfiles\doc\. Do the same with the contents of compressed files after you’ve extracted them. Again, if you’re having trouble locating where these files should go, type “:version” in vim and hit enter.

rails.vim: This is the magic. The rails.zip file contains plugin/rails.vim and doc/rails.txt. Both should be moved into their proper location in your root vim directory ($VIM).
Project: This adds a list of frequently used files to the lefthand side of vim. Rails.vim utilizes this script to create a directory tree of your Rails application. You’re probably familiar with this feature already, as is a basic feature of most modern IDEs.
genutils 1.20 (NOT 2.x): Adds many useful functions
multvals.vim: genutils depends on this.
minibufexpl.vim: This adds (ugly, but functional) tabs to vim, making it much easier to keep track of the many files you’ll be working on simultaneously.
dbext.vim: Adds database support to vim. This integrates seemlessly with rails.vim. More on it below.
surround: Surround.vim adds mappings to let you easily work with “surroundings” in pairs. Such as { }, ” ” and < >. More on it below.

To get the help files working, Rails needs to generate help tags from the txt files you placed in the doc directory. Open vim and in normal (command) mode type the following and hit enter:
Linux: :helptags $HOME/.vim/doc
Windows: :helptags $VIM\vimfiles\doc

Your directory might differ from the above. In which case, again, check “:version”

Make sure it worked:

1	:help rails

For more information on vim’s help feature, use :help help or read about it in your browser. For help about a specific command, use :help command. Command can be a vim, rails.vim or other plugin command. For example :help rails or :help rgenerate.

Usage and Examples

Usage might seem intimidating at first because the plugins have-on top of vim’s already rich set of functions-added so much more functionality and commands to memorize, but you’ll quickly realize that getting used to rails.vim is easy due to the natural command names (and tab completion). Let’s begin by creating a dummy rails project.

Starting a New Project and Setting Up The Database

In Cream or gvim, you can create projects using the menu Plugin -> Rails -> Projects -> New. Alternatively, you can use the :Rails command in normal mode. Both the menu option and the :Rails command are wrappers for the native rails application, making it possible to pass them what you would normally pass rails in the console.

If you do not specify an absolute path when passing :Rails your project name, the rails application will be generated in the current working directory, usually the directory you were in when you started vim.

Let’s create a new framework named dummy. This command is typed in vim while in command mode (not the mode you type content in) and followed with the Enter key. Change /tmp/ to where ever you want this to reside (I.e., C:\Windows\Temp\):

1	:Rails /tmp/dummy

Database.yml Setup
The usual rails output will scroll across the screen and you will be dumped at the default file you specified in the vimrc file (g:rails_default_file). This will be README by default. If you haven’t changed it to database.yml like in my config above, you can jump to the database config file by typing :Rfind database.yml.

Put your database information under dummy_development and save as usual (:w). Now creating our dummy_development database is as easy as one short command:

:Rdbext!

(You can use a database frontend (like the excellent PHPMyAdmin) or the mysql command in a console to create your dummy_development database if you did not install dbext.)

Note: This issues a CREATE DATABASE command on the Rails environment in $RAILS_ENV, which is development by default, but you can have it create the test and production environments as well:

1 2	:Rdbext! test :Rdbext! production

The last loaded environment configuration is what will be used throughout the Rails application. To change the environment, just provide the desired one as a parameter to the :Rdbext command (notice lack of bang: !). If you’ve created three environments and want to switch to the test environment:

1	:Rdbext test

This commands opens up a Results tab and displays the results on the bottom of the screen in a new window. Sometimes, this window can be intrusive and downright annoying. To close it, click it and hit ctrl+w, let go of both and quickly hit c.

For more information, check :help Rdbext. More on DBExt later.

Generating Models

If you wish to run something in the application’s script directory, you can use :Rscript:

:Rscript about
:Rscript console
:Rscript breakpointer
etc...

but you will probably never need to, as Rails.vim provides wrappers for most of Rails’ scripts and functions. The names of these will usually be :R followed by the name of the script. For example, :Rgenerate, :Rserver, :Rconsole, etcetra…

:Rgenerate takes the same parameters as Rails’ generate script. Let’s generate a model named Thing:

1	:Rgenerate model Thing

The model file is automatically loaded and given focus after its creation. Notice that you can still access database.yml via its tab on top of the vim window. Double click a tab to open it, and click it once to give it focus, then hit d to close it. Try that now on database.yml (and Results if it’s still open). More on tabs and the minibufexpl later.

Migrations and :Rake

Let’s edit the model’s migration file, which is created automatically for each generated model. You have a number of choices for jumping to this file:

:Rmigration pattern
The optional pattern can be any part of the migration’s filename (in this case, 001_create_things.rb). You can use things (notice it’s plural) because it matches the filename:

1	:Rmigration things

The migration number with or without the zeros. All three examples open the same migration:

1
2
3

:Rmigration 1
:Rmigration 01
:Rmigration 001

or any string of characters that match part of that migration’s filename without causing an ambiguity, like 001_create_things.rb:

1	:Rmigration te_t

Rmigration is context aware. It will default to opening the related migration file if executed without a parameter while the model file is open:

1	:Rmigration

A Glimpse of the Vim Database Extension Plugin

DBExt is powerful and deserves its own paper, but before getting derailed from our main topic, here are some useful DBExt commands:

:DBDescribeTable table outputs a table’s columns and their data types. Run it now on the things table:

:DBDescribeTable things
 
Connection: T(MYSQL)  D(bic_development)  U(username)
Field    Type    Null    Key        Default        Extra
id    int(11)    NO        PRI        NULL        auto_increment
description    text    YES        NULL
weight    int(11)    YES        NULL

:DBExecSQL query executes low level SQL queries that are either under the cursor, or passed as parameters. Let’s use it to populate our table:

1 2	:DBExecSQL INSERT INTO things VALUES (1, 'Worlds fattest man', 1200); :DBExecSQL INSERT INTO things VALUES (2, 'Four quarter pounders', 1);

Let’s view the contents of the things table:

1	:DBSelectFromTable things

If the results window on the bottom gets stuck, click it and then hit ctrl+w the c to close it.

Read the extensive help file using :help DBExt.

Let’s create a controller and a skeleton for a method named index:

1	:Rgenerate controller Things index

We want to display all the records from our Thing table in our index view. Let’s add some code this controller’s index method:

1
2
3

def index
    @people = Thing.find(:all)
end

For test purposes later, add the following method to ThingsController, under the index method:

1
2
3

def viewless
    @cast = "Cloud Strife"
end

Since we passed a method name (index) to :Rgenerate, Rails has automatically generated a view for that method. We have a number of ways to navigate to that view:

:R or ]f
The Related command was discussed above under Migrations and :Rake. A controller can have many views, so :R looks at the current method the cursor resides in to look up what view to open. Place the cursor anywhere in the index method and type :R (or hit ]f) to open up index.rhtml.

This command only works if the related file exists. If the view doesn’t exist, you’ll get an “E345: Can’t find file … in path” error. Try this by putting the cursor on the viewless method before running the :R command again.

You can use :RS to split (what the S stands for) the current window into two, allowing you to view and edit both the controller and view simultaneously.

:Rview [[controller/]view]
Typed with no arguments, this command behaves the same as :R. If that view doesn’t exist, you can specify a name, either by the full path or just the view name and extension, and that view will be created.

For example, let’s create a view on the fly for the viewless method (extension is mandatory):

1	:Rview viewless.rhtml

We haven’t defined a controller above, so by default, :Rview assumes it is for the current controller. It will go into app/views/things/. We could specify a controller by using :Rview controller/view_name.extension.

Note that the file is only generated in memory and must be saved to disk explicitly (:w).

Alternating between files

A lot of times, you’ll be working exclusively on two files, back and forth. To alternate between the current file and the last file you were editing, use CTRL+^ (although in practice, you’re actually just hitting CTRL+6).

Rails.vim also has an alternate command (:A or the [f mapping), similar in function to related. It usually takes you to the current file's test file.

From :help rails-alternate, this tells you what the alternate file will be for the Current File open:

Current file      Alternate file
model                               unit test
controller (in method)    functional test
template (view)              helper
migration                         previous migration
config/routes.rb               config/database.yml

:Rfind [name]
Like vim’s :find command, this one also opens a file it finds either by the name you pass it or by whatever is under the cursor (behaves similarly to gf, more on that below). Rails.vim enhances the native command by giving it the ability to find files by controller and model name.

This opens the Thing model:

1	:Rfind thing

This opens the Things controller (case sensitive):

1	:Rfind ThingsController

This opens the index.rhtml view (you must be in the ThingsController):

1	:Rfind index

gf (“goto file”)
In rails.vim, the gf mapping is used to jump to a file under the cursor taking context into account. The following is from :help rails-gf:
—
Example uses of gf, and where they might lead. (* indicates cursor position)
Pos*t.find(:first)
app/models/post.rb

has_many :c*omments
app/models/comment.rb

link_to “Home”, :controller => :bl*og
app/controllers/blog_controller.rb

<%= render :partial => ’sh*ared/sidebar’ %>
app/views/shared/_sidebar.rhtml

<%= stylesheet_link_tag :scaf*fold %>
public/stylesheets/scaffold.css

class BlogController < Applica*tionController
app/controllers/application.rb

class ApplicationController < ActionCont*roller::Base
.../action_controller/base.rb

fixtures :pos*ts
test/fixtures/posts.yml

layout :pri*nt
app/views/layouts/print.rhtml

# In the Blog controller
def li*st
app/views/blog/list.rhtml

<%= link_to "New", new_comme*nt_path %>
app/controllers/comments_controller.rb (jumps to def new)

In the last example, the controller and action for the named route are determined by evaluating routes.rb as Ruby and doing some introspection. This means code from the application is executed. Keep this in mind when navigating unfamiliar applications.
—

Add the following to the index.rhtml view file, overwriting the default code already in there:

All the things in our Thing table
<h1>All the things in our Thing table</h1>

  <% @stuff.each do |x| %>
    
    <%= x.description %> (weight: <%= x.weight %> pound(s))
    
  <% end %>

Views and Partials

The :Rextract command (:Rpartial is deprecated) makes creating partials a breeze. You choose a range of lines and supply the command a name you want for the new partial. The lines get replaced by a call to render the newly created partial, which now holds the lines that were extracted.

Let’s try using this by delegating output code between the

and

tags in our index.rhtml view into a partial named “ppp” by explicitly defining the line numbers (your line numbers might not match. Alter it accordingly):

1	:5,5Rextract ppp

A new partial with the filename _ppp.rhtml is created containing line #5 from our index.rhtml view. Line #5 now is a call to render the newly created ppp view: <%= render :partial => ‘ppp’ %> – The new partial only exists in memory at this point, and must be saved to disk explicitly.

This could also have been done in visual mode, selecting the lines we want extracted visually instead of by line numbers. Place the cursor at the beginning of the line, press v to enter visual mode and hit ‘$’ (shift+4) to jump to the end of the line, selecting the entire line along the way. Now hit : and type :Rextract ppp (it will come out as :’<,'>Rextract ppp) and hit enter. Same effect as above.

We’ve Got You Surrounded

Whether you’re coding, writing, or editing, Tim Pope’s surround plugin is very useful. It adds vim mappings to add, remove or edit “surroundings.” A surrounding is a pair of parentheses, brackets, quotes, tags, etc.

Some mappings require that you press the first few keystrokes within a second. For full functionality, make sure you’re using vim 7 or greater. Let’s begin by making sure the plugin is installed and works properly by testing a command:

Type (hello world) into vim, and with your cursor anywhere in that phrase, hit ds(. If (hello world) turns to hello world, the plugin is working fine. If not, then make sure you’re installed it correctly and are using Vim 7.0.

Caveats
* The keystrokes need to be typed in succession very fast. The timeoutlen setting in the vim config file can impair the ys* functions. The smaller the value for timeoutlen, the less time you have in between keystrokes to ensure vim catches the command. I set timeoutlen to 200, but put 250 as the value in the example vimrc at the beginning of this tutorial. Delete this setting or use a higher value (300+) if you’re having trouble using commands like yssa or yss.

* b, B, r, and a are aliases for ), }, ], and > respectively, and t represents a pair of HTML or XML tags.

* In the examples below, the cursor position is represented by an asterisk (*).

Adding Surroundings
You can add a pair of surroundings around a word, line, sentence, paragraph or a specific number of characters.

Adding quotes around a word:
Old text | Command | Result
H*eya world | ysiw” | “Heya” world

Adding a pair of tags around a word:
Old text | Command | Result
I*talic text | ysiw | Italic text

Adding arrows around a sentence:
Old text | Command | Result
Clandestin*e operation | yssa |

An angry Kenny face:
Old text | Command | Result
*>< | yss)yss)yss( | ( ((><)) )
(notice usage of parantheses in the command. Using the opening bracket adds spaces between the target being surrounded.)

Replacing Surroundings
Replacing parentheses with quotes in a word:
Old text | Command | Result
(qu*ack) quack | cs(” | “quack” quack

Changing those double quotes to a pair of tags:
Old text | Command | Result
“q*uack” quack | cs” | quack quack

Changing those tags to brackets:
Old text | Command | Result
q*uack quack | cstr | [quack] quack

Removing Surroundings
Remove quotes around a word:
Old text | Command | Result
Hash “eater*” | ds” | Hash eater

Remove tags around a word:
Old text | Command | Result

h*ead
| dst | head

Removing multiple surroundings around a sentence:
Old text | Command | Result
{“(iiight)”} | ds(ds”dsB | iiight

Rails.vim and Custom Surroundings
Rails.vim makes use of the surround plugin by adding a few useful surroundings. These only work when rails.vim is enabled (when RoR projects are open):

Old text | Command | Result
*end | yss= | <%= end %>

Old text | Command | Result
*end | yss- | <%= end -%>

Old text | Command | Result
*cat | yss# | <%# cat %>

You can create custom surroundings. Here’s one example:
:let g:surround_65 = “ \r ”

65 is the ASCII decimal for the ‘A’ character. You can use an ascii table to find the decimal corresponding to the character you want to use.

Usage:
Old text | Command | Result
I*Shot The Sheriff | yssA | I Shot The Sheriff

Be sure to read the extensive help files :help surround and :help rails-surround. They provide details and examples.

But Wait, There’s More

We’ve merely scratched the surface of rails.vim here. Be sure to check out the Official rails.vim Website, the author’s site, and the external links I provided below. For those looking for a more user-friendly, “real” GUI-based (as opposed to a TUI) IDE based around vim, try out VimMate.

External links

How To Use Vim With Rails (official RoR wiki)

Rails.vim RoR IDE (About.com)

NERD tree – A great plugin that adds a nested file explorer to vim, much like the Project plugin

rcodetools

Thanks to _vikram for the vi photo. It was done using an optical mouse and a long exposure. He was actually attempting to spell out “vik”