<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Biodegradable Geek &#187; Automation</title>
	<atom:link href="http://biodegradablegeek.com/category/automation/feed/" rel="self" type="application/rss+xml" />
	<link>http://biodegradablegeek.com</link>
	<description></description>
	<lastBuildDate>Tue, 22 Jun 2010 21:52:41 +0000</lastBuildDate>
	<generator>http://wordpress.org/?v=abc</generator>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
			<item>
		<title>I Can&#8217;t Live Without My vim Config</title>
		<link>http://biodegradablegeek.com/2009/04/i-cant-live-without-my-vim-config/</link>
		<comments>http://biodegradablegeek.com/2009/04/i-cant-live-without-my-vim-config/#comments</comments>
		<pubDate>Sat, 11 Apr 2009 17:43:18 +0000</pubDate>
		<dc:creator>Isam</dc:creator>
				<category><![CDATA[Automation]]></category>
		<category><![CDATA[Linux]]></category>
		<category><![CDATA[Productivity]]></category>
		<category><![CDATA[Tools]]></category>
		<category><![CDATA[code]]></category>
		<category><![CDATA[editors]]></category>
		<category><![CDATA[gvim]]></category>
		<category><![CDATA[ide]]></category>
		<category><![CDATA[vi]]></category>
		<category><![CDATA[vim]]></category>

		<guid isPermaLink="false">http://biodegradablegeek.com/?p=384</guid>
		<description><![CDATA[I have updated the vim page with my vimrc/gvimrc configs. Instead of repeating myself, I will quote some parts of the page ..
More details and the vim config itself here
I recommend turning backups on if you have them off. I personally hate having the ~ files all over my OS, so I keep them along [...]]]></description>
			<content:encoded><![CDATA[<p>I have updated the <a href="/vim">vim</a> page with my vimrc/gvimrc configs. Instead of repeating myself, I will quote some parts of the page ..</p>
<p><strong><a href="/vim">More details and the vim config itself here</a></strong></p>
<p>I recommend turning backups on if you have them off. I personally hate having the ~ files all over my OS, so I keep them along with the .swp files in 1 backup dir in ~/.vim/</p>
<p>The programming language skeleton stuff will detect what files you are editing and change options in vim by inheriting the specified files which I put in ~/.vim/skeletons and ~/.vim/inherit.</p>
<p>The skeletons are automatically inserted in new files that vim is aware of. For example, in my own config, I have ~/.vim/inherit/c which has all the usual includes and int main() code. When I make a new C file (&#8220;gvim hello.c&#8221;), the new file begins with the skeleton code already present. Neat huh?</p>
<p>The inherit files can be used to set specific options for each language. This can mean different bindings, whitespace options, themes, etc depending on what language you&#8217;re working with, automatically.</p>
<p><a href="/vim">See the vim page</a></p>
<p><strong>What options have helped you the most?</strong></p>
]]></content:encoded>
			<wfw:commentRss>http://biodegradablegeek.com/2009/04/i-cant-live-without-my-vim-config/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Bash Script to Force an Empty Git Push</title>
		<link>http://biodegradablegeek.com/2009/04/bash-script-to-force-an-empty-git-push/</link>
		<comments>http://biodegradablegeek.com/2009/04/bash-script-to-force-an-empty-git-push/#comments</comments>
		<pubDate>Thu, 02 Apr 2009 01:57:50 +0000</pubDate>
		<dc:creator>Isam</dc:creator>
				<category><![CDATA[Automation]]></category>
		<category><![CDATA[Scripts]]></category>
		<category><![CDATA[Tips]]></category>
		<category><![CDATA[bash]]></category>
		<category><![CDATA[git]]></category>
		<category><![CDATA[scripting]]></category>

		<guid isPermaLink="false">http://biodegradablegeek.com/?p=377</guid>
		<description><![CDATA[Sometimes, like when you&#8217;re testing hooks or trying to create synced remote and local repos, you&#8217;ll find yourself touching empty files just to get a git push going. This script automates this task by creating a unique temporary file, committing it, pushing, and then removing the file.

#!/bin/sh
TMP=tmp-`date +'%m%s'`
touch $TMP
git add $TMP
git commit $TMP -m '(forced [...]]]></description>
			<content:encoded><![CDATA[<p>Sometimes, like when you&#8217;re testing hooks or trying to create synced remote and local repos, you&#8217;ll find yourself <strong>touch</strong>ing empty files just to get a git push going. This script automates this task by creating a unique temporary file, committing it, pushing, and then removing the file.</p>
<pre class="brush: bash;">
#!/bin/sh
TMP=tmp-`date +'%m%s'`
touch $TMP
git add $TMP
git commit $TMP -m '(forced push)'
git push
git rm $TMP
&lt;/pre&gt;
Usage, assuming you named it git-force and made it executable (chmod)
&lt;pre lang=&quot;bash&quot;&gt;cd git-repo/
./git-force
</pre>
<p>I place this in ~/bin/ which is in my $PATH. You might want to if you use this a lot.</p>
]]></content:encoded>
			<wfw:commentRss>http://biodegradablegeek.com/2009/04/bash-script-to-force-an-empty-git-push/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>How to Maintain Static Sites with Git &amp; Jekyll</title>
		<link>http://biodegradablegeek.com/2009/03/how-to-maintain-static-sites-with-git-jekyll/</link>
		<comments>http://biodegradablegeek.com/2009/03/how-to-maintain-static-sites-with-git-jekyll/#comments</comments>
		<pubDate>Wed, 01 Apr 2009 04:10:00 +0000</pubDate>
		<dc:creator>Isam</dc:creator>
				<category><![CDATA[Automation]]></category>
		<category><![CDATA[Code]]></category>
		<category><![CDATA[Linux]]></category>
		<category><![CDATA[Productivity]]></category>
		<category><![CDATA[Ruby]]></category>
		<category><![CDATA[Snippets]]></category>
		<category><![CDATA[Tips]]></category>
		<category><![CDATA[Tools]]></category>
		<category><![CDATA[bash]]></category>
		<category><![CDATA[code]]></category>
		<category><![CDATA[scripting]]></category>
		<category><![CDATA[web development]]></category>

		<guid isPermaLink="false">http://biodegradablegeek.com/?p=371</guid>
		<description><![CDATA[Static sites in this context just means non-database driven sites. Your static site can be an elaborate PHP script or just a few markup and image files. For this I am using Jekyll &#8211; A neat Ruby gem that makes your static sites dynamic. It lets you create layouts and embed custom variables in your [...]]]></description>
			<content:encoded><![CDATA[<p>Static sites in this context just means non-database driven sites. Your static site can be an elaborate PHP script or just a few markup and image files. For this I am using <strong><a href="http://github.com/mojombo/jekyll/tree/master">Jekyll</a> &#8211; A neat Ruby gem that makes your static sites dynamic.</strong> It lets you create layouts and embed custom variables in your HTML (this is a &#8220;prototype&#8221; of the site). </p>
<p>Jekyll tackles all the nuisances involved in creating static pages (I used to add just enough PHP to make a layout). It works by running your prototype through some parsers and outputs plain static HTML/XML (RSS feeds) etc. It&#8217;s perfect for lightweight sites that would be impractical on Wordpress, like a few static pages of information, landing pages, portfolio/resume pages, and parked domains. </p>
<p>Git takes care of keeping your development (local) and production (remote) environments synced. Git might be a little confusing if you&#8217;re learning it with the mindset that it works like Subversion. </p>
<p><strong>I&#8217;ll update this post when the guide is done. For now, the following will assume you&#8217;re familiar with Jekyll (or at least have an empty file in the prototype directory) and git. This Bash script simplifies creating the remote git repository:</strong></p>
<p>** please read through the code and make sure you know what this does, and what you&#8217;re doing. As of now, this is bias towards my own Apache/vhost setup. It&#8217;s trivial to edit for your specific needs. <strong>You&#8217;re using this at your own risk</strong>.</p>
<p>(<a href="http://code.biodegradablegeek.com/repogen.sh" target="_blank">direct link &#8211; repogen.sh</a>)</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #666666; font-style: italic;">#!/bin/sh</span>
<span style="color: #666666; font-style: italic;"># </span>
<span style="color: #666666; font-style: italic;"># 04/01/2009 | http://biodegradablegeek.com | GPL </span>
<span style="color: #666666; font-style: italic;"># </span>
<span style="color: #666666; font-style: italic;"># You should be in site (NOT public) root (be in same dir as public/ log/ etc)</span>
<span style="color: #666666; font-style: italic;"># proto/ is created and will house the jekyll prototype</span>
<span style="color: #666666; font-style: italic;"># public/ will be the generated static site</span>
<span style="color: #666666; font-style: italic;"># the public/ folder will be REMOVED and regenerated on every push</span>
<span style="color: #666666; font-style: italic;"># </span>
&nbsp;
<span style="color: #000000; font-weight: bold;">if</span> <span style="color: #7a0874; font-weight: bold;">&#91;</span> <span style="color: #660033;">-z</span> <span style="color: #ff0000;">&quot;$1&quot;</span> <span style="color: #7a0874; font-weight: bold;">&#93;</span>; <span style="color: #000000; font-weight: bold;">then</span>
  <span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;Usage: ./repogen.sh domain.comn&quot;</span>
  <span style="color: #7a0874; font-weight: bold;">exit</span>
<span style="color: #000000; font-weight: bold;">fi</span>
&nbsp;
<span style="color: #666666; font-style: italic;"># optional. will make it easier to copy/paste cmd to clone repo </span>
<span style="color: #007800;">SSHURL</span>=<span style="color: #ff0000;">&quot;ssh.domain.com&quot;</span>
<span style="color: #007800;">URL</span>=<span style="color: #ff0000;">&quot;$1&quot;</span>
&nbsp;
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;** creating tmp repo&quot;</span>
<span style="color: #c20cb9; font-weight: bold;">mkdir</span> proto
<span style="color: #7a0874; font-weight: bold;">cd</span> proto
git init 
<span style="color: #c20cb9; font-weight: bold;">touch</span> INITIAL
git add INITIAL
git commit <span style="color: #660033;">-a</span> <span style="color: #660033;">-m</span> <span style="color: #ff0000;">&quot;Initial Commit&quot;</span>
&nbsp;
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;** creating bare repo&quot;</span>
<span style="color: #7a0874; font-weight: bold;">cd</span> ..
git clone <span style="color: #660033;">--bare</span> proto proto.git
<span style="color: #c20cb9; font-weight: bold;">mv</span> proto proto.old
git clone proto.git
<span style="color: #c20cb9; font-weight: bold;">rm</span> <span style="color: #660033;">-rf</span> proto.old
&nbsp;
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;** generating hook&quot;</span>
<span style="color: #007800;">HOOK</span>=proto.git<span style="color: #000000; font-weight: bold;">/</span>hooks<span style="color: #000000; font-weight: bold;">/</span>post-update
&nbsp;
<span style="color: #c20cb9; font-weight: bold;">mv</span> <span style="color: #007800;">$HOOK</span> <span style="color: #000000; font-weight: bold;">/</span>tmp
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">'#!/bin/sh'</span> <span style="color: #000000; font-weight: bold;">&gt;&gt;</span> <span style="color: #007800;">$HOOK</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">'# To enable this hook, make this file executable by &quot;chmod +x post-update&quot;.'</span> <span style="color: #000000; font-weight: bold;">&gt;&gt;</span> <span style="color: #007800;">$HOOK</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">'#exec git-update-server-info'</span> <span style="color: #000000; font-weight: bold;">&gt;&gt;</span> <span style="color: #007800;">$HOOK</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">''</span> <span style="color: #000000; font-weight: bold;">&gt;&gt;</span> <span style="color: #007800;">$HOOK</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">''</span> <span style="color: #000000; font-weight: bold;">&gt;&gt;</span> <span style="color: #007800;">$HOOK</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">'URL='</span><span style="color: #ff0000;">&quot;<span style="color: #007800;">$URL</span>&quot;</span> <span style="color: #000000; font-weight: bold;">&gt;&gt;</span> <span style="color: #007800;">$HOOK</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">'PROTO=&quot;/home/$USER/www/$URL/proto&quot;'</span> <span style="color: #000000; font-weight: bold;">&gt;&gt;</span> <span style="color: #007800;">$HOOK</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">'PUBLIC=&quot;/home/$USER/www/$URL/public&quot;'</span> <span style="color: #000000; font-weight: bold;">&gt;&gt;</span> <span style="color: #007800;">$HOOK</span>
<span style="color: #7a0874; font-weight: bold;">echo</span>  <span style="color: #ff0000;">''</span> <span style="color: #000000; font-weight: bold;">&gt;&gt;</span> <span style="color: #007800;">$HOOK</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">'export GIT_DIR=&quot;$PROTO/.git&quot;'</span> <span style="color: #000000; font-weight: bold;">&gt;&gt;</span> <span style="color: #007800;">$HOOK</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">'pushd $PROTO &gt; /dev/null'</span> <span style="color: #000000; font-weight: bold;">&gt;&gt;</span> <span style="color: #007800;">$HOOK</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">'git pull'</span> <span style="color: #000000; font-weight: bold;">&gt;&gt;</span> <span style="color: #007800;">$HOOK</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">'popd &gt; /dev/null'</span> <span style="color: #000000; font-weight: bold;">&gt;&gt;</span> <span style="color: #007800;">$HOOK</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">''</span> <span style="color: #000000; font-weight: bold;">&gt;&gt;</span> <span style="color: #007800;">$HOOK</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;echo -----------------------------&quot;</span> <span style="color: #000000; font-weight: bold;">&gt;&gt;</span> <span style="color: #007800;">$HOOK</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;echo '** Pushing changes to '<span style="color: #007800;">$URL</span>&quot;</span> <span style="color: #000000; font-weight: bold;">&gt;&gt;</span> <span style="color: #007800;">$HOOK</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;echo '** Moving current public to /tmp'&quot;</span> <span style="color: #000000; font-weight: bold;">&gt;&gt;</span> <span style="color: #007800;">$HOOK</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">'mv &quot;$PUBLIC&quot; &quot;/tmp/'</span><span style="color: #007800;">$URL</span><span style="color: #ff0000;">'public-`date '</span>+<span style="color: #000000; font-weight: bold;">%</span>m<span style="color: #000000; font-weight: bold;">%</span>d<span style="color: #000000; font-weight: bold;">%</span>Y<span style="color: #ff0000;">'`&quot;'</span> <span style="color: #000000; font-weight: bold;">&gt;&gt;</span> <span style="color: #007800;">$HOOK</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">'echo &quot;** Generating new public&quot;'</span> <span style="color: #000000; font-weight: bold;">&gt;&gt;</span> <span style="color: #007800;">$HOOK</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">'jekyll &quot;$PROTO&quot; &quot;$PUBLIC&quot;'</span> <span style="color: #000000; font-weight: bold;">&gt;&gt;</span> <span style="color: #007800;">$HOOK</span>
&nbsp;
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;** enabling hook&quot;</span>
<span style="color: #c20cb9; font-weight: bold;">chmod</span> a+x <span style="color: #007800;">$HOOK</span> 
&nbsp;
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;** clone repo on local machina. example:&quot;</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;git clone ssh://<span style="color: #007800;">$USER</span>@<span style="color: #007800;">$SSHURL</span>/~<span style="color: #007800;">$USER</span>/www/<span style="color: #007800;">$SSHURL</span>/proto.git&quot;</span></pre></div></div>

<p><strong>Usage</strong></p>
<p>Your site structure might be different. <strong>repogen.sh</strong> is made by pasting the above code in a new file, and then chmod a+x to make it executable. This should be done on the remote server.</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #7a0874; font-weight: bold;">cd</span> www<span style="color: #000000; font-weight: bold;">/</span>domain.com<span style="color: #000000; font-weight: bold;">/</span>
&nbsp;
<span style="color: #c20cb9; font-weight: bold;">ls</span>
public<span style="color: #000000; font-weight: bold;">/</span> private<span style="color: #000000; font-weight: bold;">/</span> log<span style="color: #000000; font-weight: bold;">/</span> cgi-bin<span style="color: #000000; font-weight: bold;">/</span>
&nbsp;
.<span style="color: #000000; font-weight: bold;">/</span>repogen.sh domain.com</pre></div></div>

<p>Now on your local machine, clone the new repo, move your files in, and push:</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;">git clone <span style="color: #c20cb9; font-weight: bold;">ssh</span>:<span style="color: #000000; font-weight: bold;">//</span><span style="color: #7a0874; font-weight: bold;">&#91;</span>username<span style="color: #7a0874; font-weight: bold;">&#93;</span><span style="color: #000000; font-weight: bold;">@</span>ssh.domain.com<span style="color: #000000; font-weight: bold;">/</span>~<span style="color: #7a0874; font-weight: bold;">&#91;</span>username<span style="color: #7a0874; font-weight: bold;">&#93;</span><span style="color: #000000; font-weight: bold;">/</span>www<span style="color: #000000; font-weight: bold;">/</span>domain.com<span style="color: #000000; font-weight: bold;">/</span>proto.git
<span style="color: #7a0874; font-weight: bold;">cd</span> proto<span style="color: #000000; font-weight: bold;">/</span>
<span style="color: #c20cb9; font-weight: bold;">cat</span> <span style="color: #ff0000;">&quot;hello, world&quot;</span> <span style="color: #000000; font-weight: bold;">&gt;</span> index.htm
git add index.htm
git commit <span style="color: #660033;">-a</span> <span style="color: #660033;">-m</span> <span style="color: #ff0000;">'first local commit'</span>
git push</pre></div></div>

<p>After you push your changes, the post-update hook will delete the public/ directory (the root of the site). This dir and its contents are automatically generated and will get wiped out on EVERY push. Keep this in mind. All your changes and content should reside in proto/. </p>
<p>The proto/ repo will pull in the new changes, and then Jekyll will be invoked to generate the updated site in public/ from the prototype.</p>
<p>Should you need to edit it, the <strong>post-update hook</strong> is in the bare git repo (proto.git/hooks/)</p>
<p>Thanks to the authors in the posts below for sharing ideas. I first read this git method on dmiessler&#8217;s site. </p>
<p><strong>Resources:</strong><br />
<a href="http://dmiessler.com/blog/using-git-to-maintain-your-website">dmiessler.com &#8211; using git to maintain static pages</a><br />
<a href="http://toroid.org/ams/git-website-howto">toroid.org &#8211; using git to manage a web site</a><br />
<a href="http://github.com/mojombo/jekyll/tree/master">Jekyll @ GitHub</a><br />
<a href="http://media.pragprog.com/titles/tsgit/chap-005-extract.html">git info</a><br />
<a href="http://www.nardol.org/2009/2/19/git-basics-reversing-the-git-sucks-effect">more git info</a></p>
]]></content:encoded>
			<wfw:commentRss>http://biodegradablegeek.com/2009/03/how-to-maintain-static-sites-with-git-jekyll/feed/</wfw:commentRss>
		<slash:comments>4</slash:comments>
		</item>
		<item>
		<title>Scraping Google Trends with Mechanize and Hpricot</title>
		<link>http://biodegradablegeek.com/2009/01/scraping-google-trends-with-mechanize-and-hpricot/</link>
		<comments>http://biodegradablegeek.com/2009/01/scraping-google-trends-with-mechanize-and-hpricot/#comments</comments>
		<pubDate>Sat, 24 Jan 2009 06:53:06 +0000</pubDate>
		<dc:creator>Isam</dc:creator>
				<category><![CDATA[Automation]]></category>
		<category><![CDATA[Code]]></category>
		<category><![CDATA[Ruby]]></category>
		<category><![CDATA[Scraping]]></category>
		<category><![CDATA[Scripts]]></category>
		<category><![CDATA[Code example]]></category>
		<category><![CDATA[making monies]]></category>
		<category><![CDATA[programming]]></category>
		<category><![CDATA[public domain]]></category>

		<guid isPermaLink="false">http://biodegradablegeek.com/?p=313</guid>
		<description><![CDATA[This is a small Ruby script that fetches the 100 trends of the day for a specific date. If multiple dates are searched, one can find out how many times a keyword occurred between two dates, or just find out what keywords are constantly appearing on the top 100 list. Very profitable info! but alas, [...]]]></description>
			<content:encoded><![CDATA[<p>This is a small Ruby script that fetches the 100 trends of the day for a specific date. If multiple dates are searched, one can find out how many times a keyword occurred between two dates, or just find out what keywords are constantly appearing on the top 100 list. <strong>Very profitable info!</strong> but alas, the script is incomplete and one must implement the &#8220;implement me!&#8221; methods to get full functionality. This, in its current state, should serve as a good starting point for scraping Google Trends.</p>
<p>On a technical note, it&#8217;s using mechanize, hpricot, tempfile (for the cache). A lot of this is just <a href="http://en.wikipedia.org/wiki/Copy_and_paste_programming">copy &amp; paste programming</a> from the <a href="http://biodegradablegeek.com/2009/01/animecrazy-scraper-example-using-hpricot-mechanize/">earlier anime scraper</a>. </p>
<p>To grab the gems <em>(rdoc takes 10x as long as the gem to fetch and install)</em>:</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #c20cb9; font-weight: bold;">sudo</span> gem <span style="color: #c20cb9; font-weight: bold;">install</span> mechanize <span style="color: #660033;">--no-rdoc</span>
<span style="color: #c20cb9; font-weight: bold;">sudo</span> gem <span style="color: #c20cb9; font-weight: bold;">install</span> hpricot <span style="color: #660033;">--no-rdoc</span></pre></div></div>


<div class="wp_syntax"><div class="code"><pre class="ruby" style="font-family:monospace;"><span style="color:#008000; font-style:italic;">#!/usr/bin/env ruby</span>
<span style="color:#008000; font-style:italic;"># biodegradablegeek.com</span>
<span style="color:#008000; font-style:italic;"># public domain</span>
<span style="color:#008000; font-style:italic;"># </span>
&nbsp;
<span style="color:#CC0066; font-weight:bold;">require</span> <span style="color:#996600;">'rubygems'</span>
<span style="color:#CC0066; font-weight:bold;">require</span> <span style="color:#996600;">'hpricot'</span>
<span style="color:#CC0066; font-weight:bold;">require</span> <span style="color:#996600;">'tempfile'</span>
<span style="color:#CC0066; font-weight:bold;">require</span> <span style="color:#996600;">'mechanize'</span>
<span style="color:#008000; font-style:italic;">#require 'highline/import'</span>
<span style="color:#008000; font-style:italic;">#HighLine.track_eof = false</span>
&nbsp;
<span style="color:#ff6633; font-weight:bold;">$mech</span> = <span style="color:#6666ff; font-weight:bold;">WWW::Mechanize</span>.<span style="color:#9900CC;">new</span>
<span style="color:#ff6633; font-weight:bold;">$mech</span>.<span style="color:#9900CC;">user_agent_alias</span> = <span style="color:#996600;">'Mac Safari'</span>
<span style="color:#ff6633; font-weight:bold;">$master</span> = <span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#006600; font-weight:bold;">&#93;</span>
&nbsp;
&nbsp;
<span style="color:#9966CC; font-weight:bold;">def</span> puts2<span style="color:#006600; font-weight:bold;">&#40;</span>txt=<span style="color:#996600;">''</span><span style="color:#006600; font-weight:bold;">&#41;</span>; <span style="color:#CC0066; font-weight:bold;">puts</span> <span style="color:#996600;">&quot;*** #{txt}&quot;</span>; <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
&nbsp;
<span style="color:#9966CC; font-weight:bold;">class</span> Cache
  <span style="color:#9966CC; font-weight:bold;">def</span> initialize
    <span style="color:#008000; font-style:italic;"># Setup physical cache location </span>
    <span style="color:#0066ff; font-weight:bold;">@path</span> = <span style="color:#996600;">'cache'</span>
    <span style="color:#CC00FF; font-weight:bold;">Dir</span>.<span style="color:#9900CC;">mkdir</span> <span style="color:#0066ff; font-weight:bold;">@path</span> <span style="color:#9966CC; font-weight:bold;">unless</span> <span style="color:#CC00FF; font-weight:bold;">File</span>.<span style="color:#9900CC;">exists</span>? <span style="color:#0066ff; font-weight:bold;">@path</span>
&nbsp;
    <span style="color:#008000; font-style:italic;"># key/val = url/filename (of fetched data)</span>
    <span style="color:#0066ff; font-weight:bold;">@datafile</span> = <span style="color:#996600;">&quot;#{@path}/cache.data&quot;</span>
    <span style="color:#0066ff; font-weight:bold;">@cache</span> = <span style="color:#CC0066; font-weight:bold;">load</span> <span style="color:#0066ff; font-weight:bold;">@datafile</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> put key, val
    tf = <span style="color:#CC00FF; font-weight:bold;">Tempfile</span>.<span style="color:#9900CC;">new</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'googletrends'</span>, <span style="color:#0066ff; font-weight:bold;">@path</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    path = tf.<span style="color:#9900CC;">path</span>
    tf.<span style="color:#9900CC;">close</span>! <span style="color:#008000; font-style:italic;"># important!</span>
&nbsp;
    puts2 <span style="color:#996600;">&quot;Saving to cache (#{path})&quot;</span>
    <span style="color:#CC0066; font-weight:bold;">open</span><span style="color:#006600; font-weight:bold;">&#40;</span>path, <span style="color:#996600;">'w'</span><span style="color:#006600; font-weight:bold;">&#41;</span> <span style="color:#006600; font-weight:bold;">&#123;</span> <span style="color:#006600; font-weight:bold;">|</span>f<span style="color:#006600; font-weight:bold;">|</span>
      f.<span style="color:#9900CC;">write</span><span style="color:#006600; font-weight:bold;">&#40;</span>val<span style="color:#006600; font-weight:bold;">&#41;</span>
      <span style="color:#0066ff; font-weight:bold;">@cache</span><span style="color:#006600; font-weight:bold;">&#91;</span>key<span style="color:#006600; font-weight:bold;">&#93;</span> = path
    <span style="color:#006600; font-weight:bold;">&#125;</span>
&nbsp;
    save <span style="color:#0066ff; font-weight:bold;">@datafile</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> get key
    <span style="color:#0000FF; font-weight:bold;">return</span> <span style="color:#0000FF; font-weight:bold;">nil</span> <span style="color:#9966CC; font-weight:bold;">unless</span> exists?<span style="color:#006600; font-weight:bold;">&#40;</span>key<span style="color:#006600; font-weight:bold;">&#41;</span> <span style="color:#006600; font-weight:bold;">&amp;&amp;</span> <span style="color:#CC00FF; font-weight:bold;">File</span>.<span style="color:#9900CC;">exists</span>?<span style="color:#006600; font-weight:bold;">&#40;</span>@cache<span style="color:#006600; font-weight:bold;">&#91;</span>key<span style="color:#006600; font-weight:bold;">&#93;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    <span style="color:#CC0066; font-weight:bold;">open</span><span style="color:#006600; font-weight:bold;">&#40;</span>@cache<span style="color:#006600; font-weight:bold;">&#91;</span>key<span style="color:#006600; font-weight:bold;">&#93;</span>, <span style="color:#996600;">'r'</span><span style="color:#006600; font-weight:bold;">&#41;</span> <span style="color:#006600; font-weight:bold;">&#123;</span> <span style="color:#006600; font-weight:bold;">|</span>f<span style="color:#006600; font-weight:bold;">|</span> f.<span style="color:#9900CC;">read</span> <span style="color:#006600; font-weight:bold;">&#125;</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> files
    <span style="color:#0066ff; font-weight:bold;">@cache</span>.<span style="color:#9900CC;">values</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> first
    <span style="color:#0066ff; font-weight:bold;">@cache</span>.<span style="color:#9900CC;">first</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> exists? key
    <span style="color:#0066ff; font-weight:bold;">@cache</span>.<span style="color:#9900CC;">has_key</span>? key
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
private
  <span style="color:#008000; font-style:italic;"># Load saved cache </span>
  <span style="color:#9966CC; font-weight:bold;">def</span> <span style="color:#CC0066; font-weight:bold;">load</span> file
    <span style="color:#0000FF; font-weight:bold;">return</span> <span style="color:#CC00FF; font-weight:bold;">File</span>.<span style="color:#9900CC;">exists</span>?<span style="color:#006600; font-weight:bold;">&#40;</span>file<span style="color:#006600; font-weight:bold;">&#41;</span> ? <span style="color:#CC00FF; font-weight:bold;">YAML</span>.<span style="color:#CC0066; font-weight:bold;">load</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#CC0066; font-weight:bold;">open</span><span style="color:#006600; font-weight:bold;">&#40;</span>file<span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">read</span><span style="color:#006600; font-weight:bold;">&#41;</span> : <span style="color:#006600; font-weight:bold;">&#123;</span><span style="color:#006600; font-weight:bold;">&#125;</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#008000; font-style:italic;"># Save cache </span>
  <span style="color:#9966CC; font-weight:bold;">def</span> save path
    <span style="color:#CC0066; font-weight:bold;">open</span><span style="color:#006600; font-weight:bold;">&#40;</span>path, <span style="color:#996600;">'w'</span><span style="color:#006600; font-weight:bold;">&#41;</span> <span style="color:#006600; font-weight:bold;">&#123;</span> <span style="color:#006600; font-weight:bold;">|</span>f<span style="color:#006600; font-weight:bold;">|</span>
      f.<span style="color:#9900CC;">write</span> <span style="color:#0066ff; font-weight:bold;">@cache</span>.<span style="color:#9900CC;">to_yaml</span>
    <span style="color:#006600; font-weight:bold;">&#125;</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
<span style="color:#ff6633; font-weight:bold;">$cache</span> = Cache.<span style="color:#9900CC;">new</span>
&nbsp;
&nbsp;
<span style="color:#9966CC; font-weight:bold;">def</span> fetch<span style="color:#006600; font-weight:bold;">&#40;</span>url<span style="color:#006600; font-weight:bold;">&#41;</span>
  body = <span style="color:#ff6633; font-weight:bold;">$mech</span>.<span style="color:#9900CC;">get</span><span style="color:#006600; font-weight:bold;">&#40;</span>url<span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">body</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
  <span style="color:#ff6633; font-weight:bold;">$cache</span>.<span style="color:#9900CC;">put</span><span style="color:#006600; font-weight:bold;">&#40;</span>url, body<span style="color:#006600; font-weight:bold;">&#41;</span>
  body
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
&nbsp;
<span style="color:#9966CC; font-weight:bold;">def</span> getPage<span style="color:#006600; font-weight:bold;">&#40;</span>url<span style="color:#006600; font-weight:bold;">&#41;</span>
  body = <span style="color:#ff6633; font-weight:bold;">$cache</span>.<span style="color:#9900CC;">get</span><span style="color:#006600; font-weight:bold;">&#40;</span>url<span style="color:#006600; font-weight:bold;">&#41;</span> 
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">if</span> body.<span style="color:#0000FF; font-weight:bold;">nil</span>?
    <span style="color:#CC0066; font-weight:bold;">puts</span> <span style="color:#996600;">&quot;Not cached. Fetching from site...&quot;</span>
    body = fetch url 
  <span style="color:#9966CC; font-weight:bold;">end</span>
  body
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
&nbsp;
<span style="color:#9966CC; font-weight:bold;">def</span> loadState
  mf = <span style="color:#996600;">'cache/master.data'</span>
  <span style="color:#ff6633; font-weight:bold;">$master</span> = <span style="color:#CC00FF; font-weight:bold;">File</span>.<span style="color:#9900CC;">exists</span>?<span style="color:#006600; font-weight:bold;">&#40;</span>mf<span style="color:#006600; font-weight:bold;">&#41;</span> ? <span style="color:#CC00FF; font-weight:bold;">YAML</span>.<span style="color:#CC0066; font-weight:bold;">load</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#CC0066; font-weight:bold;">open</span><span style="color:#006600; font-weight:bold;">&#40;</span>mf<span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">read</span><span style="color:#006600; font-weight:bold;">&#41;</span> : <span style="color:#006600; font-weight:bold;">&#123;</span><span style="color:#006600; font-weight:bold;">&#125;</span>
  <span style="color:#ff6633; font-weight:bold;">$master</span> = <span style="color:#006600; font-weight:bold;">&#123;</span><span style="color:#006600; font-weight:bold;">&#125;</span> <span style="color:#9966CC; font-weight:bold;">if</span> <span style="color:#ff6633; font-weight:bold;">$master</span>==<span style="color:#0000FF; font-weight:bold;">false</span>
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
&nbsp;
<span style="color:#9966CC; font-weight:bold;">def</span> saveState
  <span style="color:#CC0066; font-weight:bold;">open</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'cache/master.data'</span>, <span style="color:#996600;">'w+'</span><span style="color:#006600; font-weight:bold;">&#41;</span> <span style="color:#006600; font-weight:bold;">&#123;</span> <span style="color:#006600; font-weight:bold;">|</span>f<span style="color:#006600; font-weight:bold;">|</span>
    f.<span style="color:#9900CC;">write</span> <span style="color:#ff6633; font-weight:bold;">$master</span>.<span style="color:#9900CC;">to_yaml</span>
  <span style="color:#006600; font-weight:bold;">&#125;</span>
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
&nbsp;
<span style="color:#9966CC; font-weight:bold;">def</span> main
  <span style="color:#008000; font-style:italic;">#loadState</span>
&nbsp;
  <span style="color:#008000; font-style:italic;"># Grab top 100 Google Trends (today)</span>
  <span style="color:#008000; font-style:italic;">#date = Time.now.strftime '%Y-%m-%d'</span>
  date = <span style="color:#996600;">'2009-01-21'</span>
&nbsp;
  puts2 <span style="color:#996600;">&quot;Getting Google's top 100 search trends for #{date}&quot;</span>
  url = <span style="color:#996600;">&quot;http://www.google.com/trends/hottrends?sa=X&amp;date=#{date}&quot;</span>
  puts2 url
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">begin</span>
    body = getPage<span style="color:#006600; font-weight:bold;">&#40;</span>url<span style="color:#006600; font-weight:bold;">&#41;</span>
  <span style="color:#9966CC; font-weight:bold;">rescue</span> <span style="color:#6666ff; font-weight:bold;">WWW::Mechanize::ResponseCodeError</span>
    puts2 <span style="color:#996600;">&quot;Couldn't fetch URL. Invalid date..?&quot;</span>
    <span style="color:#CC0066; font-weight:bold;">exit</span> <span style="color:#006666;">5</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  puts2 <span style="color:#996600;">&quot;Fetched page (#{body.size} bytes)&quot;</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">if</span> body<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'There is no data on date'</span><span style="color:#006600; font-weight:bold;">&#93;</span>
    puts2 <span style="color:#996600;">'No data available for this date.'</span>
    puts2 <span style="color:#996600;">'Date might be too old or too early for report, or just invalid'</span>
    <span style="color:#CC0066; font-weight:bold;">exit</span> <span style="color:#006666;">3</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  doc = Hpricot<span style="color:#006600; font-weight:bold;">&#40;</span>body<span style="color:#006600; font-weight:bold;">&#41;</span>
&nbsp;
  <span style="color:#006600; font-weight:bold;">&#40;</span>doc<span style="color:#006600; font-weight:bold;">/</span><span style="color:#996600;">&quot;td[@class='hotColumn']/table[@class='Z2_list']//tr&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">each</span> <span style="color:#9966CC; font-weight:bold;">do</span> <span style="color:#006600; font-weight:bold;">|</span>tr<span style="color:#006600; font-weight:bold;">|</span>
    td = <span style="color:#006600; font-weight:bold;">&#40;</span>tr<span style="color:#006600; font-weight:bold;">/</span>:td<span style="color:#006600; font-weight:bold;">&#41;</span>
    num = td<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#006666;">0</span><span style="color:#006600; font-weight:bold;">&#93;</span>.<span style="color:#9900CC;">inner_text</span>.<span style="color:#CC0066; font-weight:bold;">sub</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'.'</span>,<span style="color:#996600;">''</span><span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">strip</span>
    kw = td<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#006666;">1</span><span style="color:#006600; font-weight:bold;">&#93;</span>.<span style="color:#9900CC;">inner_text</span>
    url = <span style="color:#006600; font-weight:bold;">&#40;</span>td<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#006666;">1</span><span style="color:#006600; font-weight:bold;">&#93;</span><span style="color:#006600; font-weight:bold;">/</span>:a<span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">first</span><span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#ff3333; font-weight:bold;">:href</span><span style="color:#006600; font-weight:bold;">&#93;</span>
    Keyword.<span style="color:#9900CC;">find_or_new</span><span style="color:#006600; font-weight:bold;">&#40;</span>kw<span style="color:#006600; font-weight:bold;">&#41;</span> <span style="color:#006600; font-weight:bold;">&lt;&lt;</span> Occurance.<span style="color:#9900CC;">new</span><span style="color:#006600; font-weight:bold;">&#40;</span>num, date, url<span style="color:#006600; font-weight:bold;">&#41;</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
  <span style="color:#CC0066; font-weight:bold;">puts</span> <span style="color:#996600;">&quot;Got info on #{$master.size} keywords for #{date}&quot;</span>
  <span style="color:#CC0066; font-weight:bold;">puts</span> <span style="color:#996600;">&quot;keyword '#{$master.first.name}' occured #{$master.first.occurances} times&quot;</span>
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
<span style="color:#9966CC; font-weight:bold;">class</span> Occurance
  attr_accessor <span style="color:#ff3333; font-weight:bold;">:pos</span>, <span style="color:#ff3333; font-weight:bold;">:date</span>, <span style="color:#ff3333; font-weight:bold;">:url</span>
  <span style="color:#9966CC; font-weight:bold;">def</span> initialize<span style="color:#006600; font-weight:bold;">&#40;</span>pos, date, url<span style="color:#006600; font-weight:bold;">&#41;</span>
    <span style="color:#0066ff; font-weight:bold;">@pos</span> = pos
    <span style="color:#0066ff; font-weight:bold;">@date</span> = date
    <span style="color:#0066ff; font-weight:bold;">@url</span> = url
  <span style="color:#9966CC; font-weight:bold;">end</span>
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
<span style="color:#9966CC; font-weight:bold;">class</span> Keyword
  attr_accessor <span style="color:#ff3333; font-weight:bold;">:name</span>, <span style="color:#ff3333; font-weight:bold;">:occurances</span>
  <span style="color:#9966CC; font-weight:bold;">def</span> initialize<span style="color:#006600; font-weight:bold;">&#40;</span>name<span style="color:#006600; font-weight:bold;">&#41;</span>
    <span style="color:#0066ff; font-weight:bold;">@name</span> = name
    <span style="color:#0066ff; font-weight:bold;">@occurances</span> = <span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#006600; font-weight:bold;">&#93;</span>
    <span style="color:#0066ff; font-weight:bold;">@position_average</span> = <span style="color:#0000FF; font-weight:bold;">nil</span>
    <span style="color:#0066ff; font-weight:bold;">@count</span> = <span style="color:#0000FF; font-weight:bold;">nil</span>
    <span style="color:#ff6633; font-weight:bold;">$master</span> <span style="color:#006600; font-weight:bold;">&lt;&lt;</span> <span style="color:#0000FF; font-weight:bold;">self</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> <span style="color:#0000FF; font-weight:bold;">self</span>.<span style="color:#9900CC;">find_or_new</span><span style="color:#006600; font-weight:bold;">&#40;</span>name<span style="color:#006600; font-weight:bold;">&#41;</span>
    x = <span style="color:#ff6633; font-weight:bold;">$master</span>.<span style="color:#9900CC;">find</span> <span style="color:#006600; font-weight:bold;">&#123;</span> <span style="color:#006600; font-weight:bold;">|</span>m<span style="color:#006600; font-weight:bold;">|</span> name==m.<span style="color:#9900CC;">name</span> <span style="color:#006600; font-weight:bold;">&#125;</span>
    x <span style="color:#006600; font-weight:bold;">||</span> Keyword.<span style="color:#9900CC;">new</span><span style="color:#006600; font-weight:bold;">&#40;</span>name<span style="color:#006600; font-weight:bold;">&#41;</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> <span style="color:#006600; font-weight:bold;">&lt;&lt;</span> occurance
    <span style="color:#0066ff; font-weight:bold;">@occurances</span> <span style="color:#006600; font-weight:bold;">&lt;&lt;</span> occurance
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> occured_on? datetime
    <span style="color:#CC0066; font-weight:bold;">raise</span> <span style="color:#996600;">'implement me'</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> occured_between? datetime
    <span style="color:#CC0066; font-weight:bold;">raise</span> <span style="color:#996600;">'implement me'</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> occurances datetime=<span style="color:#0000FF; font-weight:bold;">nil</span>
    <span style="color:#CC0066; font-weight:bold;">raise</span> <span style="color:#996600;">'implement me'</span> <span style="color:#9966CC; font-weight:bold;">if</span> datetime
    <span style="color:#0066ff; font-weight:bold;">@occurances</span>.<span style="color:#9900CC;">size</span> 
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> occurances_between datetime
    <span style="color:#CC0066; font-weight:bold;">raise</span> <span style="color:#996600;">'implement me'</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> pos_latest
    <span style="color:#0066ff; font-weight:bold;">@occurances</span>.<span style="color:#9900CC;">last</span>.<span style="color:#9900CC;">date</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> pos_average
    <span style="color:#0066ff; font-weight:bold;">@position_average</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> pos_average_between datetime
    <span style="color:#CC0066; font-weight:bold;">raise</span> <span style="color:#996600;">'implement me'</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
<span style="color:#008000; font-style:italic;">#   Instance= [num, date, url]</span>
<span style="color:#008000; font-style:italic;">#   Keyword=[Instance, Intance, Instance]</span>
<span style="color:#008000; font-style:italic;">#   Methods for keywords: </span>
<span style="color:#008000; font-style:italic;">#   KW.occured_on? date </span>
<span style="color:#008000; font-style:italic;">#   KW.occured_between? d1, d2 </span>
<span style="color:#008000; font-style:italic;">#   KW.occurances</span>
<span style="color:#008000; font-style:italic;">#   KW.occurances_between? d1, d2</span>
<span style="color:#008000; font-style:italic;">#   KW.pos_latest</span>
<span style="color:#008000; font-style:italic;">#   KW.pos_average</span>
<span style="color:#008000; font-style:italic;">#   KW.pos_average_between</span>
&nbsp;
<span style="color:#008000; font-style:italic;">#   KW has been on the top 100 list KW.occurances.size times</span>
<span style="color:#008000; font-style:italic;">#   The #1 keywords for the month of January: Master.sort_by KW.occurances_between? Jan1,Jan31.pos_average_between Jan1,Jan31 </span>
<span style="color:#008000; font-style:italic;">#</span>
<span style="color:#008000; font-style:italic;">#   Top keywords: sort by KW.occurances.size = N keyword was listed the most.</span>
<span style="color:#008000; font-style:italic;">#   Top keywords for date D: Master.sort_by KW.occured_on (x).num</span>
&nbsp;
main</pre></div></div>

]]></content:encoded>
			<wfw:commentRss>http://biodegradablegeek.com/2009/01/scraping-google-trends-with-mechanize-and-hpricot/feed/</wfw:commentRss>
		<slash:comments>3</slash:comments>
		</item>
		<item>
		<title>AnimeCrazy Scraper Example Using Hpricot &amp; Mechanize</title>
		<link>http://biodegradablegeek.com/2009/01/animecrazy-scraper-example-using-hpricot-mechanize/</link>
		<comments>http://biodegradablegeek.com/2009/01/animecrazy-scraper-example-using-hpricot-mechanize/#comments</comments>
		<pubDate>Sun, 11 Jan 2009 20:44:40 +0000</pubDate>
		<dc:creator>Isam</dc:creator>
				<category><![CDATA[Automation]]></category>
		<category><![CDATA[Code]]></category>
		<category><![CDATA[Ruby]]></category>
		<category><![CDATA[Scraping]]></category>
		<category><![CDATA[Scripts]]></category>
		<category><![CDATA[Code example]]></category>
		<category><![CDATA[hpricot]]></category>
		<category><![CDATA[mechanize]]></category>
		<category><![CDATA[tuts]]></category>

		<guid isPermaLink="false">http://biodegradablegeek.com/?p=306</guid>
		<description><![CDATA[This is a little (as of now incomplete) scraper I wrote to grab all the anime video code off of AnimeCrazy (dot) net. This site doesn&#8217;t host any videos on its own server, but just embeds ones that have been uploaded to other sites (Megavideo, YouTube, Vimeo, etc). I don&#8217;t know who the original uploaders [...]]]></description>
			<content:encoded><![CDATA[<p>This is a little <em>(as of now incomplete)</em> scraper I wrote to grab all the anime video code off of AnimeCrazy (dot) net. This site doesn&#8217;t host any videos on its own server, but just embeds ones that have been uploaded to other sites (Megavideo, YouTube, Vimeo, etc). I don&#8217;t know who the original uploaders of the videos are, but I&#8217;ve seen this same collection of anime links being used on some other sites. This site has about 10,000 episodes/parts (1 movie may have 6+ parts). The scraper below was only tested with &#8220;completed anime shows&#8221; and got around 6300 episodes. The remaining content (anime movies and running anime shows) should work as-is, but I personally held off on getting those because I want to examine them closely to try cleaning up the inconsistencies as much as possible.</p>
<p>This scraper needs some initial setup and <strong>won&#8217;t work out of the box</strong>, but I&#8217;m including it here in the hopes that it will serve as a decent example of a small real world scraper, if you&#8217;re looking to learn the basics of scraping with <a href="http://redhanded.hobix.com/inspect/hpricot01.html">Hpricot</a> and Mechanize. Let me know if you find any use for it. I will update the posted code later this week when I have time to complete it and add some more features.</p>
<p>There&#8217;s one major problem with the organization of episodes on AnimeCrazy, and it&#8217;s the fact that some episodes are glued together into one post. Right now the scraper stops and asks you how to proceed when it comes across such a post. You basically need to tell the scraper if a post (page) contains 1 episode (video) or multiple. If there&#8217;s 1, it proceeds on its own, but if there&#8217;s two, it requires that you give it the names and links of each individual episode (part1 and part2 usually). Sometimes 2 episodes are together in 1 video. Sorta like those music albums on KaZaA or LimeWire that are basically ripped as one huge mp3 instead of individual songs.</p>
<p>This only accounts for maybe 30-40 out of 6000 videos, and it&#8217;s not that big of a deal because the amount of work needed to proceed with the scraping is small, but it IS work, and is a bitch slap to the entire concept of automation, but coding around the issue is a major hassle and there would still be a high chance that some inconsistencies will still come through. It would be far less work to just find another anime site which is far more consistent, though the reason animecrazy is good is because it&#8217;s active, and the site IS updated manually these days, as far as I can tell.</p>
<p>BTW, <strong><a href="http://whytheluckystiff.net/">Why The Lucky Stiff rocks</a>, and Hpricot is amazing.</strong> But the serious scrapologist should consider <a href="http://blog.labnotes.org/2006/07/11/scraping-with-style-scrapi-toolkit-for-ruby/">scrAPI</a> or <a href="http://scrubyt.org/">sCRUBYt</a> (uses Hpricot) for big projects.</p>

<div class="wp_syntax"><div class="code"><pre class="ruby" style="font-family:monospace;"><span style="color:#008000; font-style:italic;">#!/usr/bin/env ruby</span>
<span style="color:#008000; font-style:italic;"># License: Public domain. Go sell it to newbs on DigitalPoint.</span>
&nbsp;
<span style="color:#CC0066; font-weight:bold;">require</span> <span style="color:#996600;">'rubygems'</span>
<span style="color:#CC0066; font-weight:bold;">require</span> <span style="color:#996600;">'hpricot'</span>
<span style="color:#CC0066; font-weight:bold;">require</span> <span style="color:#996600;">'mechanize'</span>
<span style="color:#CC0066; font-weight:bold;">require</span> <span style="color:#996600;">'tempfile'</span>
<span style="color:#CC0066; font-weight:bold;">require</span> <span style="color:#996600;">'highline/import'</span>
HighLine.<span style="color:#9900CC;">track_eof</span> = <span style="color:#0000FF; font-weight:bold;">false</span>
&nbsp;
<span style="color:#ff6633; font-weight:bold;">$mech</span> = <span style="color:#6666ff; font-weight:bold;">WWW::Mechanize</span>.<span style="color:#9900CC;">new</span>
<span style="color:#ff6633; font-weight:bold;">$mech</span>.<span style="color:#9900CC;">user_agent_alias</span> = <span style="color:#996600;">'Mac Safari'</span>
&nbsp;
<span style="color:#008000; font-style:italic;">###############################</span>
<span style="color:#ff6633; font-weight:bold;">$skip_until</span> = <span style="color:#0000FF; font-weight:bold;">false</span>
DEBUG=<span style="color:#0000FF; font-weight:bold;">false</span>
<span style="color:#008000; font-style:italic;">###############################</span>
&nbsp;
<span style="color:#9966CC; font-weight:bold;">def</span> debug?
  DEBUG
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
<span style="color:#9966CC; font-weight:bold;">def</span> puts2<span style="color:#006600; font-weight:bold;">&#40;</span>txt=<span style="color:#996600;">''</span><span style="color:#006600; font-weight:bold;">&#41;</span>
  <span style="color:#CC0066; font-weight:bold;">puts</span> <span style="color:#996600;">&quot;*** #{txt}&quot;</span>
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
<span style="color:#008000; font-style:italic;">#  Anime has: title, type (series, movie), series</span>
<span style="color:#008000; font-style:italic;">#  Episode has name/#, description, parts (video code)</span>
&nbsp;
<span style="color:#9966CC; font-weight:bold;">class</span> Episode
  attr_accessor <span style="color:#ff3333; font-weight:bold;">:name</span>, <span style="color:#ff3333; font-weight:bold;">:src</span>, <span style="color:#ff3333; font-weight:bold;">:desc</span>, <span style="color:#ff3333; font-weight:bold;">:cover</span>
  <span style="color:#9966CC; font-weight:bold;">def</span> initialize<span style="color:#006600; font-weight:bold;">&#40;</span>title, page<span style="color:#006600; font-weight:bold;">&#41;</span>
    <span style="color:#0066ff; font-weight:bold;">@src</span> = page <span style="color:#008000; font-style:italic;"># parts (megavideo, youtube etc)</span>
    <span style="color:#0066ff; font-weight:bold;">@name</span> = title
    <span style="color:#0066ff; font-weight:bold;">@desc</span> = <span style="color:#0000FF; font-weight:bold;">nil</span> <span style="color:#008000; font-style:italic;"># episode description</span>
    <span style="color:#0066ff; font-weight:bold;">@cover</span> = <span style="color:#0000FF; font-weight:bold;">nil</span> <span style="color:#008000; font-style:italic;"># file path</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
<span style="color:#9966CC; font-weight:bold;">class</span> Anime
  attr_accessor <span style="color:#ff3333; font-weight:bold;">:name</span>, <span style="color:#ff3333; font-weight:bold;">:page</span>, <span style="color:#ff3333; font-weight:bold;">:completed</span>, <span style="color:#ff3333; font-weight:bold;">:anime_type</span>, <span style="color:#ff3333; font-weight:bold;">:episodes</span>
  <span style="color:#9966CC; font-weight:bold;">def</span> initialize<span style="color:#006600; font-weight:bold;">&#40;</span>title, page<span style="color:#006600; font-weight:bold;">&#41;</span>
    <span style="color:#0066ff; font-weight:bold;">@name</span> = title
    <span style="color:#0066ff; font-weight:bold;">@page</span> = page
    <span style="color:#0066ff; font-weight:bold;">@episodes</span> = <span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#006600; font-weight:bold;">&#93;</span>
    <span style="color:#0066ff; font-weight:bold;">@anime_type</span> = <span style="color:#996600;">'series'</span>
    <span style="color:#0066ff; font-weight:bold;">@completed</span> = <span style="color:#0000FF; font-weight:bold;">false</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> complete!
    <span style="color:#0066ff; font-weight:bold;">@completed</span> = <span style="color:#0000FF; font-weight:bold;">true</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> episode! episode
    <span style="color:#0066ff; font-weight:bold;">@episodes</span> <span style="color:#006600; font-weight:bold;">&amp;</span>lt;<span style="color:#006600; font-weight:bold;">&amp;</span>lt; episode
  <span style="color:#9966CC; font-weight:bold;">end</span>
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
<span style="color:#9966CC; font-weight:bold;">class</span> Cache
  <span style="color:#9966CC; font-weight:bold;">def</span> initialize
    <span style="color:#008000; font-style:italic;"># Setup physical cache location</span>
    <span style="color:#0066ff; font-weight:bold;">@path</span> = <span style="color:#996600;">'cache'</span>
    <span style="color:#CC00FF; font-weight:bold;">Dir</span>.<span style="color:#9900CC;">mkdir</span> <span style="color:#0066ff; font-weight:bold;">@path</span> <span style="color:#9966CC; font-weight:bold;">unless</span> <span style="color:#CC00FF; font-weight:bold;">File</span>.<span style="color:#9900CC;">exists</span>? <span style="color:#0066ff; font-weight:bold;">@path</span>
&nbsp;
    <span style="color:#008000; font-style:italic;"># key/val = url/filename (of fetched data)</span>
    <span style="color:#0066ff; font-weight:bold;">@datafile</span> = <span style="color:#996600;">&quot;#{@path}/cache.data&quot;</span>
    <span style="color:#0066ff; font-weight:bold;">@cache</span> = <span style="color:#CC0066; font-weight:bold;">load</span> <span style="color:#0066ff; font-weight:bold;">@datafile</span>
    <span style="color:#008000; font-style:italic;">#puts @cache.inspect</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> put key, val
    tf = <span style="color:#CC00FF; font-weight:bold;">Tempfile</span>.<span style="color:#9900CC;">new</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'animecrazy'</span>, <span style="color:#0066ff; font-weight:bold;">@path</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    path = tf.<span style="color:#9900CC;">path</span>
    tf.<span style="color:#9900CC;">close</span>! <span style="color:#008000; font-style:italic;"># important!</span>
&nbsp;
    puts2 <span style="color:#996600;">&quot;Saving to cache (#{path})&quot;</span>
    <span style="color:#CC0066; font-weight:bold;">open</span><span style="color:#006600; font-weight:bold;">&#40;</span>path, <span style="color:#996600;">'w'</span><span style="color:#006600; font-weight:bold;">&#41;</span> <span style="color:#006600; font-weight:bold;">&#123;</span> <span style="color:#006600; font-weight:bold;">|</span>f<span style="color:#006600; font-weight:bold;">|</span>
      f.<span style="color:#9900CC;">write</span><span style="color:#006600; font-weight:bold;">&#40;</span>val<span style="color:#006600; font-weight:bold;">&#41;</span>
      <span style="color:#0066ff; font-weight:bold;">@cache</span><span style="color:#006600; font-weight:bold;">&#91;</span>key<span style="color:#006600; font-weight:bold;">&#93;</span> = path
    <span style="color:#006600; font-weight:bold;">&#125;</span>
&nbsp;
    save <span style="color:#0066ff; font-weight:bold;">@datafile</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> get key
    <span style="color:#0000FF; font-weight:bold;">return</span> <span style="color:#0000FF; font-weight:bold;">nil</span> <span style="color:#9966CC; font-weight:bold;">unless</span> exists?<span style="color:#006600; font-weight:bold;">&#40;</span>key<span style="color:#006600; font-weight:bold;">&#41;</span> <span style="color:#006600; font-weight:bold;">&amp;</span>amp;<span style="color:#006600; font-weight:bold;">&amp;</span>amp; <span style="color:#CC00FF; font-weight:bold;">File</span>.<span style="color:#9900CC;">exists</span>?<span style="color:#006600; font-weight:bold;">&#40;</span>@cache<span style="color:#006600; font-weight:bold;">&#91;</span>key<span style="color:#006600; font-weight:bold;">&#93;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    <span style="color:#CC0066; font-weight:bold;">open</span><span style="color:#006600; font-weight:bold;">&#40;</span>@cache<span style="color:#006600; font-weight:bold;">&#91;</span>key<span style="color:#006600; font-weight:bold;">&#93;</span>, <span style="color:#996600;">'r'</span><span style="color:#006600; font-weight:bold;">&#41;</span> <span style="color:#006600; font-weight:bold;">&#123;</span> <span style="color:#006600; font-weight:bold;">|</span>f<span style="color:#006600; font-weight:bold;">|</span> f.<span style="color:#9900CC;">read</span> <span style="color:#006600; font-weight:bold;">&#125;</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> exists? key
    <span style="color:#0066ff; font-weight:bold;">@cache</span>.<span style="color:#9900CC;">has_key</span>? key
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
private
  <span style="color:#008000; font-style:italic;"># Load saved cache</span>
  <span style="color:#9966CC; font-weight:bold;">def</span> <span style="color:#CC0066; font-weight:bold;">load</span> file
    <span style="color:#0000FF; font-weight:bold;">return</span> <span style="color:#CC00FF; font-weight:bold;">File</span>.<span style="color:#9900CC;">exists</span>?<span style="color:#006600; font-weight:bold;">&#40;</span>file<span style="color:#006600; font-weight:bold;">&#41;</span> ? <span style="color:#CC00FF; font-weight:bold;">YAML</span>.<span style="color:#CC0066; font-weight:bold;">load</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#CC0066; font-weight:bold;">open</span><span style="color:#006600; font-weight:bold;">&#40;</span>file<span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">read</span><span style="color:#006600; font-weight:bold;">&#41;</span> : <span style="color:#006600; font-weight:bold;">&#123;</span><span style="color:#006600; font-weight:bold;">&#125;</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#008000; font-style:italic;"># Save cache</span>
  <span style="color:#9966CC; font-weight:bold;">def</span> save path
    <span style="color:#CC0066; font-weight:bold;">open</span><span style="color:#006600; font-weight:bold;">&#40;</span>path, <span style="color:#996600;">'w'</span><span style="color:#006600; font-weight:bold;">&#41;</span> <span style="color:#006600; font-weight:bold;">&#123;</span> <span style="color:#006600; font-weight:bold;">|</span>f<span style="color:#006600; font-weight:bold;">|</span>
      f.<span style="color:#9900CC;">write</span> <span style="color:#0066ff; font-weight:bold;">@cache</span>.<span style="color:#9900CC;">to_yaml</span>
    <span style="color:#006600; font-weight:bold;">&#125;</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
<span style="color:#ff6633; font-weight:bold;">$cache</span> = Cache.<span style="color:#9900CC;">new</span>
&nbsp;
<span style="color:#9966CC; font-weight:bold;">def</span> fetch<span style="color:#006600; font-weight:bold;">&#40;</span>url<span style="color:#006600; font-weight:bold;">&#41;</span>
  body = <span style="color:#ff6633; font-weight:bold;">$mech</span>.<span style="color:#9900CC;">get</span><span style="color:#006600; font-weight:bold;">&#40;</span>url<span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">body</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
  <span style="color:#ff6633; font-weight:bold;">$cache</span>.<span style="color:#9900CC;">put</span><span style="color:#006600; font-weight:bold;">&#40;</span>url, body<span style="color:#006600; font-weight:bold;">&#41;</span>
  body
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
<span style="color:#9966CC; font-weight:bold;">def</span> getPage<span style="color:#006600; font-weight:bold;">&#40;</span>url<span style="color:#006600; font-weight:bold;">&#41;</span>
  <span style="color:#008000; font-style:italic;"># First let's see if this is cached already.</span>
  body = <span style="color:#ff6633; font-weight:bold;">$cache</span>.<span style="color:#9900CC;">get</span><span style="color:#006600; font-weight:bold;">&#40;</span>url<span style="color:#006600; font-weight:bold;">&#41;</span> 
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">if</span> body.<span style="color:#0000FF; font-weight:bold;">nil</span>?
    <span style="color:#CC0066; font-weight:bold;">puts</span> <span style="color:#996600;">&quot;Not cached. Fetching from site...&quot;</span>
    body = fetch url
  <span style="color:#9966CC; font-weight:bold;">end</span>
  body
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
<span style="color:#9966CC; font-weight:bold;">def</span> main
  <span style="color:#008000; font-style:italic;"># Open anime list (anime_list = saved HTML of</span>
<span style="color:#006600; font-weight:bold;">&lt;</span>ul<span style="color:#006600; font-weight:bold;">&gt;</span>...<span style="color:#006600; font-weight:bold;">&lt;/</span>ul<span style="color:#006600; font-weight:bold;">&gt;</span>
sidebar from animecrazy.<span style="color:#9900CC;">net</span><span style="color:#006600; font-weight:bold;">&#41;</span>
  anime_list = Hpricot<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#CC0066; font-weight:bold;">open</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'anime_list'</span>, <span style="color:#996600;">'r'</span><span style="color:#006600; font-weight:bold;">&#41;</span> <span style="color:#006600; font-weight:bold;">&#123;</span> <span style="color:#006600; font-weight:bold;">|</span>f<span style="color:#006600; font-weight:bold;">|</span> f.<span style="color:#9900CC;">read</span> <span style="color:#006600; font-weight:bold;">&#125;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
  puts2 <span style="color:#996600;">&quot;Anime list open&quot;</span>
&nbsp;
  <span style="color:#008000; font-style:italic;"># Read in the URL to every series</span>
  masterlist = <span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#006600; font-weight:bold;">&#93;</span>
&nbsp;
  <span style="color:#006600; font-weight:bold;">&#40;</span>anime_list<span style="color:#006600; font-weight:bold;">/</span>:li<span style="color:#006600; font-weight:bold;">/</span>:a<span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">each</span> <span style="color:#9966CC; font-weight:bold;">do</span> <span style="color:#006600; font-weight:bold;">|</span>series<span style="color:#006600; font-weight:bold;">|</span>
    anime = Anime.<span style="color:#9900CC;">new</span><span style="color:#006600; font-weight:bold;">&#40;</span>series.<span style="color:#9900CC;">inner_text</span>, series<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#ff3333; font-weight:bold;">:href</span><span style="color:#006600; font-weight:bold;">&#93;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    masterlist <span style="color:#006600; font-weight:bold;">&amp;</span>lt;<span style="color:#006600; font-weight:bold;">&amp;</span>lt; anime
    puts2 <span style="color:#996600;">&quot;Built structure for #{anime.name}...&quot;</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  puts2
&nbsp;
  puts2 <span style="color:#996600;">&quot;Fetched #{masterlist.size} animes. Now fetching episodes...&quot;</span>
  masterlist.<span style="color:#9900CC;">each</span> <span style="color:#9966CC; font-weight:bold;">do</span> <span style="color:#006600; font-weight:bold;">|</span>anime<span style="color:#006600; font-weight:bold;">|</span>
    puts2 <span style="color:#996600;">&quot;Fetching body (#{anime.name})&quot;</span>
    body = getPage<span style="color:#006600; font-weight:bold;">&#40;</span>anime.<span style="color:#9900CC;">page</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    puts2 <span style="color:#996600;">&quot;Snatched that bitch (#{body.size} bytes of Goku Goodness)&quot;</span>
    puts2
&nbsp;
    doc = Hpricot<span style="color:#006600; font-weight:bold;">&#40;</span>body<span style="color:#006600; font-weight:bold;">&#41;</span>
    <span style="color:#006600; font-weight:bold;">&#40;</span>doc<span style="color:#006600; font-weight:bold;">/</span><span style="color:#996600;">&quot;h1/a[@rel='bookmark']&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">each</span> <span style="color:#9966CC; font-weight:bold;">do</span> <span style="color:#006600; font-weight:bold;">|</span>episode<span style="color:#006600; font-weight:bold;">|</span>
      name = clean<span style="color:#006600; font-weight:bold;">&#40;</span>episode.<span style="color:#9900CC;">inner_text</span><span style="color:#006600; font-weight:bold;">&#41;</span>
&nbsp;
      <span style="color:#9966CC; font-weight:bold;">if</span> <span style="color:#ff6633; font-weight:bold;">$skip_until</span>
        <span style="color:#008000; font-style:italic;">#$skip_until = !inUrl(episode[:href], 'basilisk-episode-2')</span>
        <span style="color:#008000; font-style:italic;">#$skip_until = nil == name['Tsubasa Chronicles']</span>
        puts2 <span style="color:#996600;">&quot;Resuming from #{episode[:href]}&quot;</span> <span style="color:#9966CC; font-weight:bold;">if</span> !$skip_until
        <span style="color:#9966CC; font-weight:bold;">next</span>
      <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
      <span style="color:#008000; font-style:italic;"># Here it gets tricky. This is a major source of inconsistencies in the site.</span>
      <span style="color:#008000; font-style:italic;"># They group episodes into 1 post sometimes, and the only way to find</span>
      <span style="color:#008000; font-style:italic;"># out from the title of the post is by checking for the following patterns</span>
      <span style="color:#008000; font-style:italic;"># (7 and 8 are example episode #s)</span>
      <span style="color:#008000; font-style:italic;"># X = 7+8, 7 + 8, 7 and 8, 7and8, 7 &amp;amp; 8, 7&amp;amp;8</span>
&nbsp;
      <span style="color:#008000; font-style:italic;"># If an episode has no X then it is 1 episode.</span>
      <span style="color:#008000; font-style:italic;"># If it has multiple parts, they are mirrors.</span>
      <span style="color:#9966CC; font-weight:bold;">if</span> single_episode? name
        <span style="color:#9966CC; font-weight:bold;">begin</span>
&nbsp;
          puts2 <span style="color:#996600;">&quot;Adding episode #{name}...&quot;</span>
          ep = Episode.<span style="color:#9900CC;">new</span><span style="color:#006600; font-weight:bold;">&#40;</span>name, episode<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#ff3333; font-weight:bold;">:href</span><span style="color:#006600; font-weight:bold;">&#93;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
          ep.<span style="color:#9900CC;">src</span> = getPage<span style="color:#006600; font-weight:bold;">&#40;</span>episode<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#ff3333; font-weight:bold;">:href</span><span style="color:#006600; font-weight:bold;">&#93;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
          anime.<span style="color:#9900CC;">episode</span>! ep
        <span style="color:#9966CC; font-weight:bold;">rescue</span> <span style="color:#6666ff; font-weight:bold;">WWW::Mechanize::ResponseCodeError</span>
          puts2 <span style="color:#996600;">&quot;ERROR: Page not found? Skipping...&quot;</span>
          <span style="color:#CC0066; font-weight:bold;">puts</span> name
          puts2 episode<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#ff3333; font-weight:bold;">:href</span><span style="color:#006600; font-weight:bold;">&#93;</span>
        <span style="color:#9966CC; font-weight:bold;">end</span>
      <span style="color:#9966CC; font-weight:bold;">else</span>
        <span style="color:#008000; font-style:italic;"># If an episode DOES have X, it *may* have 2 episodes (but may have mirrors, going up to 4 parts/vids per page).</span>
        <span style="color:#008000; font-style:italic;"># Multiple parts will be the individual episodes in chronological order.</span>
        puts2 <span style="color:#996600;">&quot;Help me! I'm confused @ '#{name}'&quot;</span>
        puts2 <span style="color:#996600;">&quot;This post might contain multiple episodes...&quot;</span>
&nbsp;
        puts2 <span style="color:#996600;">&quot;Please visit this URL and verify the following:&quot;</span>
        <span style="color:#CC0066; font-weight:bold;">puts</span> episode<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#ff3333; font-weight:bold;">:href</span><span style="color:#006600; font-weight:bold;">&#93;</span>
&nbsp;
        <span style="color:#9966CC; font-weight:bold;">if</span> agree<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Is this 1 episode? yes/no &quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
          <span style="color:#9966CC; font-weight:bold;">begin</span>
            puts2 <span style="color:#996600;">&quot;Adding episode #{name}...&quot;</span>
            ep = Episode.<span style="color:#9900CC;">new</span><span style="color:#006600; font-weight:bold;">&#40;</span>name, episode<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#ff3333; font-weight:bold;">:href</span><span style="color:#006600; font-weight:bold;">&#93;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
            ep.<span style="color:#9900CC;">src</span> = getPage<span style="color:#006600; font-weight:bold;">&#40;</span>episode<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#ff3333; font-weight:bold;">:href</span><span style="color:#006600; font-weight:bold;">&#93;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
            anime.<span style="color:#9900CC;">episode</span>! ep
          <span style="color:#9966CC; font-weight:bold;">rescue</span> <span style="color:#6666ff; font-weight:bold;">WWW::Mechanize::ResponseCodeError</span>
            puts2 <span style="color:#996600;">&quot;ERROR: Page not found? Skipping...&quot;</span>
            <span style="color:#CC0066; font-weight:bold;">puts</span> name
            puts2 episode<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#ff3333; font-weight:bold;">:href</span><span style="color:#006600; font-weight:bold;">&#93;</span>
          <span style="color:#9966CC; font-weight:bold;">end</span>
        <span style="color:#9966CC; font-weight:bold;">else</span>
          more = <span style="color:#0000FF; font-weight:bold;">true</span>
          <span style="color:#9966CC; font-weight:bold;">while</span> more
            ename = ask<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Enter the name of an episode: &quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
            eurl =  ask<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Enter the URL of an episode: &quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
&nbsp;
            <span style="color:#9966CC; font-weight:bold;">begin</span>
              puts2 <span style="color:#996600;">&quot;Adding episode #{ename}...&quot;</span>
              ep = Episode.<span style="color:#9900CC;">new</span><span style="color:#006600; font-weight:bold;">&#40;</span>name, episode<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#ff3333; font-weight:bold;">:href</span><span style="color:#006600; font-weight:bold;">&#93;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
              ep.<span style="color:#9900CC;">src</span> = getPage<span style="color:#006600; font-weight:bold;">&#40;</span>episode<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#ff3333; font-weight:bold;">:href</span><span style="color:#006600; font-weight:bold;">&#93;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
              anime.<span style="color:#9900CC;">episode</span>! ep
            <span style="color:#9966CC; font-weight:bold;">rescue</span> <span style="color:#6666ff; font-weight:bold;">WWW::Mechanize::ResponseCodeError</span>
              puts2 <span style="color:#996600;">&quot;ERROR: Page not found? Skipping...&quot;</span>
              <span style="color:#CC0066; font-weight:bold;">puts</span> name
              puts2 episode<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#ff3333; font-weight:bold;">:href</span><span style="color:#006600; font-weight:bold;">&#93;</span>
            <span style="color:#9966CC; font-weight:bold;">end</span>
            more = agree<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Add another episode? Y/N&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
          <span style="color:#9966CC; font-weight:bold;">end</span>
          puts2 <span style="color:#996600;">&quot;Added episodes manually... moving on&quot;</span>
        <span style="color:#9966CC; font-weight:bold;">end</span>
      <span style="color:#9966CC; font-weight:bold;">end</span>
    <span style="color:#9966CC; font-weight:bold;">end</span>
    anime.<span style="color:#9900CC;">complete</span>!
    <span style="color:#008000; font-style:italic;"># XXX save the entire anime object, instead of just cache</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
<span style="color:#9966CC; font-weight:bold;">def</span> inTitle<span style="color:#006600; font-weight:bold;">&#40;</span>document, title<span style="color:#006600; font-weight:bold;">&#41;</span>
  <span style="color:#0000FF; font-weight:bold;">return</span> <span style="color:#006600; font-weight:bold;">&#40;</span>document<span style="color:#006600; font-weight:bold;">/</span>:title<span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">inner_text</span><span style="color:#006600; font-weight:bold;">&#91;</span>title<span style="color:#006600; font-weight:bold;">&#93;</span>
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
<span style="color:#9966CC; font-weight:bold;">def</span> inUrl<span style="color:#006600; font-weight:bold;">&#40;</span>url, part<span style="color:#006600; font-weight:bold;">&#41;</span>
  <span style="color:#0000FF; font-weight:bold;">return</span> url<span style="color:#006600; font-weight:bold;">&#91;</span>part<span style="color:#006600; font-weight:bold;">&#93;</span>
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
<span style="color:#9966CC; font-weight:bold;">def</span> single_episode?<span style="color:#006600; font-weight:bold;">&#40;</span>name<span style="color:#006600; font-weight:bold;">&#41;</span>
  !<span style="color:#006600; font-weight:bold;">&#40;</span>name =~ <span style="color:#006600; font-weight:bold;">/</span><span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#006666;">0</span><span style="color:#006600; font-weight:bold;">-</span><span style="color:#006666;">9</span><span style="color:#006600; font-weight:bold;">&#93;</span> ?<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#006600; font-weight:bold;">+&amp;</span>amp;<span style="color:#006600; font-weight:bold;">&#93;</span><span style="color:#006600; font-weight:bold;">|</span>and<span style="color:#006600; font-weight:bold;">&#41;</span> ?<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#006666;">0</span><span style="color:#006600; font-weight:bold;">-</span><span style="color:#006666;">9</span><span style="color:#006600; font-weight:bold;">&#93;</span><span style="color:#006600; font-weight:bold;">/</span><span style="color:#006600; font-weight:bold;">&#41;</span>
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
<span style="color:#9966CC; font-weight:bold;">def</span> clean<span style="color:#006600; font-weight:bold;">&#40;</span>txt<span style="color:#006600; font-weight:bold;">&#41;</span>
  <span style="color:#008000; font-style:italic;"># This picks up most of them, but some are missing. Like *Final* and just plain &quot;Final&quot;</span>
  txt<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">' (Final)'</span><span style="color:#006600; font-weight:bold;">&#93;</span>=<span style="color:#996600;">''</span> <span style="color:#9966CC; font-weight:bold;">if</span> txt<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">' (Final)'</span><span style="color:#006600; font-weight:bold;">&#93;</span>
  txt<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">' (Final Episode)'</span><span style="color:#006600; font-weight:bold;">&#93;</span>=<span style="color:#996600;">''</span> <span style="color:#9966CC; font-weight:bold;">if</span> txt<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">' (Final Episode)'</span><span style="color:#006600; font-weight:bold;">&#93;</span>
  txt<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">' (FINAL)'</span><span style="color:#006600; font-weight:bold;">&#93;</span>=<span style="color:#996600;">''</span> <span style="color:#9966CC; font-weight:bold;">if</span> txt<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">' (FINAL)'</span><span style="color:#006600; font-weight:bold;">&#93;</span>
  txt<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">' (FINAL EPISODE)'</span><span style="color:#006600; font-weight:bold;">&#93;</span>=<span style="color:#996600;">''</span> <span style="color:#9966CC; font-weight:bold;">if</span> txt<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">' (FINAL EPISODE)'</span><span style="color:#006600; font-weight:bold;">&#93;</span>
&nbsp;
  txt<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'(Final)'</span><span style="color:#006600; font-weight:bold;">&#93;</span>=<span style="color:#996600;">''</span> <span style="color:#9966CC; font-weight:bold;">if</span> txt<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'(Final)'</span><span style="color:#006600; font-weight:bold;">&#93;</span>
  txt<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'(Final Episode)'</span><span style="color:#006600; font-weight:bold;">&#93;</span>=<span style="color:#996600;">''</span> <span style="color:#9966CC; font-weight:bold;">if</span> txt<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'(Final Episode)'</span><span style="color:#006600; font-weight:bold;">&#93;</span>
  txt<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'(FINAL)'</span><span style="color:#006600; font-weight:bold;">&#93;</span>=<span style="color:#996600;">''</span> <span style="color:#9966CC; font-weight:bold;">if</span> txt<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">' (FINAL)'</span><span style="color:#006600; font-weight:bold;">&#93;</span>
  txt<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'(FINAL EPISODE)'</span><span style="color:#006600; font-weight:bold;">&#93;</span>=<span style="color:#996600;">''</span> <span style="color:#9966CC; font-weight:bold;">if</span> txt<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">' (FINAL EPISODE)'</span><span style="color:#006600; font-weight:bold;">&#93;</span>
&nbsp;
  txt
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
main</pre></div></div>

<p>If you&#8217;re writing your own scraper and would like to use the minimal caching functionality present below, you can gut everything in main() out and put in your own code. Feel free to <a href="/contact">contact me for assistance</a>.</p>
<p>Here is some sample output:<br />
<span id="more-306"></span></p>

<div class="wp_syntax"><div class="code"><pre class="text" style="font-family:monospace;">*** Adding episode Initial D: Episode 1 (Stage 2)...
Not cached. Fetching from site...
*** Saving to cache (cache/animecrazy20090111-12300-mbdpcl-0)
*** Fetching body (Initial D: Third Stage)
*** Snatched that bitch (77695 bytes of Goku Goodness)
***
*** Adding episode Initial D: Third Stage...
Not cached. Fetching from site...
*** Saving to cache (cache/animecrazy20090111-12300-ea69nr-0)
*** Fetching body (Kaiji)
*** Snatched that bitch (87553 bytes of Goku Goodness)
***
*** Adding episode Basilisk Episode 4...
Not cached. Fetching from site...
*** Saving to cache (cache/animecrazy20090110-14992-fomoh0-0)
*** Adding episode Basilisk Episode 3...
Not cached. Fetching from site...
*** Saving to cache (cache/animecrazy20090110-14992-1dx9xm-0)
*** Adding episode Basilisk Episode 2...
Not cached. Fetching from site...
*** Saving to cache (cache/animecrazy20090110-14992-5xt774-0)
*** Adding episode Basilisk Episode 1...
Not cached. Fetching from site...
*** Saving to cache (cache/animecrazy20090110-14992-br5fxd-0)
*** Adding episode Tsubasa Chronicles: Tokyo Revelations Episode 3...
Not cached. Fetching from site...
*** Saving to cache (cache/animecrazy20090110-14992-zmuwix-0)
*** Adding episode Tsubasa Chronicles: Tokyo Revelations Episode 2...
Not cached. Fetching from site...
*** Saving to cache (cache/animecrazy20090110-14992-1ah20eg-0)
*** Adding episode Tsubasa Chronicles: Tokyo Revelations Episode 1...
Not cached. Fetching from site...</pre></div></div>

<p>This was written for fun, but primarily profit, and not for my own viewing pleasure. The only anime I&#8217;ve seen was Akira a decade or so ago, and only because the cover looked cool, but feel free to recommend your favorites.</p>
]]></content:encoded>
			<wfw:commentRss>http://biodegradablegeek.com/2009/01/animecrazy-scraper-example-using-hpricot-mechanize/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
		<item>
		<title>Quick BASH Script to Dump &amp; Compress a MySQL Database</title>
		<link>http://biodegradablegeek.com/2008/12/quick-bash-script-to-dump-compress-a-mysql-database/</link>
		<comments>http://biodegradablegeek.com/2008/12/quick-bash-script-to-dump-compress-a-mysql-database/#comments</comments>
		<pubDate>Sat, 27 Dec 2008 18:01:19 +0000</pubDate>
		<dc:creator>Isam</dc:creator>
				<category><![CDATA[Automation]]></category>
		<category><![CDATA[Code]]></category>
		<category><![CDATA[Linux]]></category>
		<category><![CDATA[Projects]]></category>
		<category><![CDATA[SQL]]></category>
		<category><![CDATA[Scripts]]></category>
		<category><![CDATA[backups]]></category>
		<category><![CDATA[mysql]]></category>
		<category><![CDATA[sql]]></category>

		<guid isPermaLink="false">http://biodegradablegeek.com/?p=299</guid>
		<description><![CDATA[A quick script I whipped up to dump my MySQL database.
Usage: sh backthatsqlup.sh
(be warned that it dumps ALL databases. This can get huge uncompressed)

#!/bin/sh
# Isam (Biodegradablegeek.com) public domain 12/28/2008
# Basic BASH script to dump and compress a MySQL dump
&#160;
out=sequel_`date +'%m%d%Y_%M%S'`.sql
dest=/bx/
&#160;
function e &#123;
  echo -e &#34;n** $1&#34;
&#125;
&#160;
e &#34;Dumping SQL file ($out). May take awhile...&#34;
#echo &#34;oh [...]]]></description>
			<content:encoded><![CDATA[<p>A quick script I whipped up to dump my MySQL database.<br />
<strong>Usage: sh backthatsqlup.sh</strong></p>
<p><em>(be warned that it dumps ALL databases. This can get huge uncompressed)</em></p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #666666; font-style: italic;">#!/bin/sh</span>
<span style="color: #666666; font-style: italic;"># Isam (Biodegradablegeek.com) public domain 12/28/2008</span>
<span style="color: #666666; font-style: italic;"># Basic BASH script to dump and compress a MySQL dump</span>
&nbsp;
<span style="color: #007800;">out</span>=sequel_<span style="color: #000000; font-weight: bold;">`</span><span style="color: #c20cb9; font-weight: bold;">date</span> +<span style="color: #ff0000;">'%m%d%Y_%M%S'</span><span style="color: #000000; font-weight: bold;">`</span>.sql
<span style="color: #007800;">dest</span>=<span style="color: #000000; font-weight: bold;">/</span>bx<span style="color: #000000; font-weight: bold;">/</span>
&nbsp;
<span style="color: #000000; font-weight: bold;">function</span> e <span style="color: #7a0874; font-weight: bold;">&#123;</span>
  <span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #660033;">-e</span> <span style="color: #ff0000;">&quot;n** $1&quot;</span>
<span style="color: #7a0874; font-weight: bold;">&#125;</span>
&nbsp;
e <span style="color: #ff0000;">&quot;Dumping SQL file (<span style="color: #007800;">$out</span>). May take awhile...&quot;</span>
<span style="color: #666666; font-style: italic;">#echo &quot;oh snap&quot; &amp;gt; $out</span>
<span style="color: #c20cb9; font-weight: bold;">sudo</span> mysqldump <span style="color: #660033;">-u</span> root <span style="color: #660033;">-p</span> <span style="color: #660033;">--all-databases</span> <span style="color: #000000; font-weight: bold;">&amp;</span>gt; <span style="color: #007800;">$out</span>
<span style="color: #000000; font-weight: bold;">if</span> <span style="color: #7a0874; font-weight: bold;">&#91;</span> <span style="color: #007800;">$?</span> <span style="color: #660033;">-ne</span> <span style="color: #000000;">0</span> <span style="color: #7a0874; font-weight: bold;">&#93;</span>; <span style="color: #000000; font-weight: bold;">then</span>
  e <span style="color: #ff0000;">&quot;MySQL dump failed. Check that server is up and your username/pass&quot;</span>
  <span style="color: #7a0874; font-weight: bold;">exit</span> <span style="color: #000000;">7</span>
<span style="color: #000000; font-weight: bold;">fi</span>
&nbsp;
e <span style="color: #ff0000;">&quot;Uncompressed SQL file size&quot;</span>
<span style="color: #c20cb9; font-weight: bold;">du</span> <span style="color: #660033;">-hs</span> <span style="color: #007800;">$out</span>
&nbsp;
e <span style="color: #ff0000;">&quot;Compressing SQL file&quot;</span>
<span style="color: #007800;">gz</span>=<span style="color: #007800;">$out</span>.tar.gz
<span style="color: #c20cb9; font-weight: bold;">tar</span> <span style="color: #660033;">-zvvcf</span> <span style="color: #007800;">$gz</span> <span style="color: #007800;">$out</span>
<span style="color: #007800;">rt</span>=<span style="color: #007800;">$?</span>
&nbsp;
<span style="color: #000000; font-weight: bold;">if</span> <span style="color: #7a0874; font-weight: bold;">&#91;</span> <span style="color: #007800;">$rt</span> <span style="color: #660033;">-ne</span> <span style="color: #000000;">0</span> <span style="color: #7a0874; font-weight: bold;">&#93;</span>; <span style="color: #000000; font-weight: bold;">then</span>
  e <span style="color: #ff0000;">&quot;tar failed (error=<span style="color: #007800;">$rt</span>). Will NOT remove uncompressed SQL file&quot;</span>
<span style="color: #000000; font-weight: bold;">else</span>
  e <span style="color: #ff0000;">&quot;Removing uncompressed SQL file&quot;</span>
  <span style="color: #c20cb9; font-weight: bold;">rm</span> <span style="color: #660033;">-f</span> <span style="color: #007800;">$out</span>
  <span style="color: #007800;">out</span>=<span style="color: #007800;">$gz</span>
&nbsp;
  e <span style="color: #ff0000;">&quot;Compressed SQL file size&quot;</span>
  <span style="color: #c20cb9; font-weight: bold;">du</span> <span style="color: #660033;">-hs</span> <span style="color: #007800;">$out</span>
<span style="color: #000000; font-weight: bold;">fi</span>
&nbsp;
e <span style="color: #ff0000;">&quot;Moving shit to '<span style="color: #007800;">$dest</span>'&quot;</span>
<span style="color: #c20cb9; font-weight: bold;">sudo</span> <span style="color: #c20cb9; font-weight: bold;">mv</span> <span style="color: #007800;">$out</span> <span style="color: #007800;">$dest</span></pre></div></div>

<p><a href="http://code.biodegradablegeek.com/backthatsqlup.sh">BackThatSqlUp.sh</a></p>
]]></content:encoded>
			<wfw:commentRss>http://biodegradablegeek.com/2008/12/quick-bash-script-to-dump-compress-a-mysql-database/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>Using Javascript to Populate Forms During Development</title>
		<link>http://biodegradablegeek.com/2008/12/using-javascript-to-populate-forms-during-development/</link>
		<comments>http://biodegradablegeek.com/2008/12/using-javascript-to-populate-forms-during-development/#comments</comments>
		<pubDate>Sun, 07 Dec 2008 15:43:41 +0000</pubDate>
		<dc:creator>Isam</dc:creator>
				<category><![CDATA[Automation]]></category>
		<category><![CDATA[Code]]></category>
		<category><![CDATA[Ruby on Rails]]></category>
		<category><![CDATA[Snippets]]></category>
		<category><![CDATA[javascript]]></category>
		<category><![CDATA[jquery]]></category>
		<category><![CDATA[rails]]></category>

		<guid isPermaLink="false">http://biodegradablegeek.com/?p=291</guid>
		<description><![CDATA[During development, working with forms quickly gets annoying because you have to constantly fill in each field, sometimes with unique info. One way around this is to write a little Javascript code that just populates the fields. I use something like this on the bottom of the form. I had jQuery no-conflict mode on in [...]]]></description>
			<content:encoded><![CDATA[<p>During development, working with forms quickly gets annoying because you have to constantly fill in each field, sometimes with unique info. One way around this is to write a little Javascript code that just populates the fields. I use something like this on the bottom of the form. I had jQuery no-conflict mode on in this case. In your app you might be able to get away replacing _j() with $():</p>

<div class="wp_syntax"><div class="code"><pre class="rails" style="font-family:monospace;"><span style="color:#006600; font-weight:bold;">&lt;%</span> <span style="color:#9966CC; font-weight:bold;">if</span> ENV<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'RAILS_ENV'</span><span style="color:#006600; font-weight:bold;">&#93;</span>==<span style="color:#996600;">'development'</span> <span style="color:#006600; font-weight:bold;">-%&gt;</span>
&lt;!-- Generate random field data --&gt;
&lt;script lang=&quot;text/javascript&quot;&gt;
  jQuery(function() {
    if (typeof(_j)=='undefined') _j = jQuery;
    function randstr() {
      return Math.floor(Math.random()*99999)
    }
&nbsp;
    _j('#name').val('Chuck Norris');
    _j('#company_name').val('Roundhouse LLC');
    _j('#email').val('moo'+randstr()+'@yawhoo.com');
    _j('#login').val('user'+randstr());
    _j('#password').val('admin');
    _j('#password_confirmation').val('admin');
  })
&lt;/script&gt;
<span style="color:#006600; font-weight:bold;">&lt;%</span> <span style="color:#9966CC; font-weight:bold;">end</span> <span style="color:#006600; font-weight:bold;">-%&gt;</span></pre></div></div>

]]></content:encoded>
			<wfw:commentRss>http://biodegradablegeek.com/2008/12/using-javascript-to-populate-forms-during-development/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>Script to Quickly Setup WebApp Environment and Domain</title>
		<link>http://biodegradablegeek.com/2008/10/script-to-quickly-setup-webapp-environment-and-domain/</link>
		<comments>http://biodegradablegeek.com/2008/10/script-to-quickly-setup-webapp-environment-and-domain/#comments</comments>
		<pubDate>Sun, 12 Oct 2008 04:10:32 +0000</pubDate>
		<dc:creator>Isam</dc:creator>
				<category><![CDATA[Automation]]></category>
		<category><![CDATA[Code]]></category>
		<category><![CDATA[Linux]]></category>
		<category><![CDATA[Ruby]]></category>
		<category><![CDATA[code]]></category>
		<category><![CDATA[deployment]]></category>
		<category><![CDATA[joomla]]></category>
		<category><![CDATA[phpmotion]]></category>
		<category><![CDATA[rio]]></category>
		<category><![CDATA[scripting]]></category>
		<category><![CDATA[Scripts]]></category>
		<category><![CDATA[warez]]></category>

		<guid isPermaLink="false">http://biodegradablegeek.com/?p=240</guid>
		<description><![CDATA[Just sharing a script I wrote to quickly deploy Wordpress (and eventually a few other webapps) sites, which somebody might find useful. This uses Linode&#8217;s API* to add the domain name to the DNS server along with some subdomains. If you&#8217;re using another server, (Slicehost, your own, etc), you can alter the dns class to [...]]]></description>
			<content:encoded><![CDATA[<p>Just sharing a script I wrote to quickly deploy Wordpress (and eventually a few other webapps) sites, which somebody might find useful. This uses <a href="http://linode.com">Linode</a>&#8217;s API* to add the domain name to the DNS server along with some subdomains. If you&#8217;re using another server, (Slicehost, your own, etc), you can alter the dns class to use that API, or just ignore the DNS stuff completely; Its optional.</p>
<p>This will be updated periodically as I refactor and add support for more apps (notably Joomla and Clipshare &#8211; though this would violate their terms unless you have the unlimited license). This was written primarily because I couldn&#8217;t stand setting up another vhost and Wordpress installation. There are plenty of existing deployers but I plan on adding very specific features and tweaking this for in-house work. I also wanted to try Rio (Ruby-IO). GPL license. Go nuts.</p>
<p><em>* As of 10/11, the apicore.rb file on the site has some syntactic errors in the domainResourceSave method. I sent an email out to the author about it. Problems aren&#8217;t major. You can get <a href="http://biodegradablegeek.com/wp-content/uploads/2008/10/apicore.rb" target="_new">my apicore.rb here</a>.</em></p>
<p>This won&#8217;t run unless you create the appropriate folder structure in /etc/mksite/. I&#8217;ll get going on this in a bit. See the code below:</p>
<p><span id="more-240"></span></p>

<div class="wp_syntax"><div class="code"><pre class="ruby" style="font-family:monospace;"><span style="color:#008000; font-style:italic;">#!/usr/bin/env ruby </span>
&nbsp;
<span style="color:#008000; font-style:italic;">########################################################</span>
<span style="color:#008000; font-style:italic;"># Isam M. </span>
<span style="color:#008000; font-style:italic;"># http://biodegradablegeek.com</span>
<span style="color:#008000; font-style:italic;"># MKSITE.RB (0.2) Last updated Oct 19th, 2008</span>
<span style="color:#008000; font-style:italic;">#</span>
<span style="color:#008000; font-style:italic;"># mksite makes it quicker to setup sites and web</span>
<span style="color:#008000; font-style:italic;"># apps by doing most of the tedious work.</span>
<span style="color:#008000; font-style:italic;">#</span>
<span style="color:#008000; font-style:italic;">#</span>
<span style="color:#008000; font-style:italic;">#                     UNSTABLE</span>
<span style="color:#008000; font-style:italic;">#           Run it in your imagination,</span>
<span style="color:#008000; font-style:italic;">#              not on your system!</span>
<span style="color:#008000; font-style:italic;">#</span>
<span style="color:#008000; font-style:italic;">#</span>
<span style="color:#008000; font-style:italic;">######################################################## </span>
&nbsp;
<span style="color:#CC0066; font-weight:bold;">require</span> <span style="color:#996600;">'rubygems'</span>
<span style="color:#CC0066; font-weight:bold;">require</span> <span style="color:#996600;">'rio'</span>
<span style="color:#CC0066; font-weight:bold;">require</span> <span style="color:#996600;">'yaml'</span>
<span style="color:#CC0066; font-weight:bold;">require</span> <span style="color:#996600;">'mysql'</span>
<span style="color:#CC0066; font-weight:bold;">require</span> <span style="color:#996600;">'highline/import'</span>
HighLine.<span style="color:#9900CC;">track_eof</span> = <span style="color:#0000FF; font-weight:bold;">false</span>
&nbsp;
<span style="color:#9966CC; font-weight:bold;">begin</span>
  <span style="color:#CC0066; font-weight:bold;">require</span> <span style="color:#996600;">'apicore'</span>
<span style="color:#9966CC; font-weight:bold;">rescue</span> <span style="color:#CC00FF; font-weight:bold;">LoadError</span>
  <span style="color:#CC0066; font-weight:bold;">puts</span> <span style="color:#996600;">&quot;NOTICE: Unable to load apicore.rb - domains will not be added automatically&quot;</span>
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
Apache_sites = <span style="color:#996600;">'/etc/apache2/sites-available/'</span>
Subdomains = <span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">''</span>, <span style="color:#996600;">'www'</span>, <span style="color:#996600;">'mail'</span>, <span style="color:#996600;">'blog'</span>, <span style="color:#996600;">&quot;dev#{(rand*10).floor}&quot;</span><span style="color:#006600; font-weight:bold;">&#93;</span>
Testing = <span style="color:#0000FF; font-weight:bold;">false</span>
Homedir = ENV<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'HOME'</span><span style="color:#006600; font-weight:bold;">&#93;</span>
Username = ENV<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'USER'</span><span style="color:#006600; font-weight:bold;">&#93;</span>
Applications = <span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#ff3333; font-weight:bold;">:Skeleton</span>, <span style="color:#ff3333; font-weight:bold;">:Clipshare</span>, <span style="color:#ff3333; font-weight:bold;">:Joomla</span>, <span style="color:#ff3333; font-weight:bold;">:PHPMotion</span>, <span style="color:#996600;">'PmWiki (N/A)'</span>, <span style="color:#ff3333; font-weight:bold;">:Wordpress</span><span style="color:#006600; font-weight:bold;">&#93;</span>
<span style="color:#ff6633; font-weight:bold;">$config</span> = <span style="color:#0000FF; font-weight:bold;">nil</span>
<span style="color:#ff6633; font-weight:bold;">$log</span> = <span style="color:#0000FF; font-weight:bold;">nil</span>
&nbsp;
<span style="color:#9966CC; font-weight:bold;">def</span> say<span style="color:#006600; font-weight:bold;">&#40;</span>txt<span style="color:#006600; font-weight:bold;">&#41;</span>
  <span style="color:#9966CC; font-weight:bold;">super</span> txt
<span style="color:#008000; font-style:italic;">#  $log &amp;lt;&amp;lt; txt if $log</span>
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
<span style="color:#008000; font-style:italic;"># flash('message', :notice || :error) to output msg</span>
<span style="color:#9966CC; font-weight:bold;">def</span> flash<span style="color:#006600; font-weight:bold;">&#40;</span>msg, message_type = <span style="color:#ff3333; font-weight:bold;">:notice</span><span style="color:#006600; font-weight:bold;">&#41;</span>
 <span style="color:#9966CC; font-weight:bold;">if</span> message_type.<span style="color:#9900CC;">eql</span>? <span style="color:#ff3333; font-weight:bold;">:notice</span>
    say<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;INFORMATION: #{msg}&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
  <span style="color:#9966CC; font-weight:bold;">elsif</span> message_type.<span style="color:#9900CC;">eql</span>? <span style="color:#ff3333; font-weight:bold;">:emphasize</span>
    say<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;***************************************************&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    say<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;NOTICE: #{msg}n&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    say<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;***************************************************&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
  <span style="color:#9966CC; font-weight:bold;">elsif</span> message_type.<span style="color:#9900CC;">eql</span>? <span style="color:#ff3333; font-weight:bold;">:error</span>
    STDERR.<span style="color:#CC0066; font-weight:bold;">puts</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;FATAL ERROR: #{msg}n&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
<span style="color:#008000; font-style:italic;"># DZone 2111</span>
<span style="color:#9966CC; font-weight:bold;">def</span> genAlpha<span style="color:#006600; font-weight:bold;">&#40;</span>size=<span style="color:#006666;">64</span><span style="color:#006600; font-weight:bold;">&#41;</span>
  s=<span style="color:#996600;">''</span>
  size.<span style="color:#9900CC;">times</span> <span style="color:#006600; font-weight:bold;">&#123;</span>
    s <span style="color:#006600; font-weight:bold;">&amp;</span>lt;<span style="color:#006600; font-weight:bold;">&amp;</span>lt; <span style="color:#006600; font-weight:bold;">&#40;</span>i = <span style="color:#CC00FF; font-weight:bold;">Kernel</span>.<span style="color:#CC0066; font-weight:bold;">rand</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006666;">62</span><span style="color:#006600; font-weight:bold;">&#41;</span>; i <span style="color:#006600; font-weight:bold;">+</span>= <span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006600; font-weight:bold;">&#40;</span>i <span style="color:#006600; font-weight:bold;">&amp;</span>lt; <span style="color:#006666;">10</span><span style="color:#006600; font-weight:bold;">&#41;</span> ? <span style="color:#006666;">48</span> : <span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006600; font-weight:bold;">&#40;</span>i <span style="color:#006600; font-weight:bold;">&amp;</span>lt; <span style="color:#006666;">36</span><span style="color:#006600; font-weight:bold;">&#41;</span> ? <span style="color:#006666;">55</span> : <span style="color:#006666;">61</span> <span style="color:#006600; font-weight:bold;">&#41;</span><span style="color:#006600; font-weight:bold;">&#41;</span><span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">chr</span>
  <span style="color:#006600; font-weight:bold;">&#125;</span>
  s
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
<span style="color:#9966CC; font-weight:bold;">if</span> <span style="color:#9966CC; font-weight:bold;">defined</span>? ApiCore
  <span style="color:#9966CC; font-weight:bold;">class</span> DNSAPI <span style="color:#006600; font-weight:bold;">&amp;</span>lt; ApiCore
    <span style="color:#008000; font-style:italic;"># This depends on the Linode API Ruby bindings (apicore.rb)</span>
    <span style="color:#9966CC; font-weight:bold;">def</span> initialize<span style="color:#006600; font-weight:bold;">&#40;</span>key, debug=<span style="color:#0000FF; font-weight:bold;">false</span>, batching=<span style="color:#0000FF; font-weight:bold;">false</span><span style="color:#006600; font-weight:bold;">&#41;</span>
      <span style="color:#9966CC; font-weight:bold;">super</span>
      <span style="color:#0066ff; font-weight:bold;">@batching</span>=<span style="color:#0000FF; font-weight:bold;">false</span>
    <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
    <span style="color:#008000; font-style:italic;"># Add domain. Return ID on success, nil on failure</span>
    <span style="color:#9966CC; font-weight:bold;">def</span> addDomain domain
      <span style="color:#0000FF; font-weight:bold;">return</span> <span style="color:#0000FF; font-weight:bold;">nil</span> <span style="color:#9966CC; font-weight:bold;">if</span> !domain
      params = <span style="color:#006600; font-weight:bold;">&#123;</span><span style="color:#006600; font-weight:bold;">&#125;</span>
      params<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#ff3333; font-weight:bold;">:DomainID</span><span style="color:#006600; font-weight:bold;">&#93;</span> = <span style="color:#006666;">0</span>
      params<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#ff3333; font-weight:bold;">:Domain</span><span style="color:#006600; font-weight:bold;">&#93;</span> = domain
      params<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#ff3333; font-weight:bold;">:Type</span><span style="color:#006600; font-weight:bold;">&#93;</span> = <span style="color:#996600;">'master'</span>
      params<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#ff3333; font-weight:bold;">:Status</span><span style="color:#006600; font-weight:bold;">&#93;</span> = <span style="color:#006666;">1</span>
      params<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#ff3333; font-weight:bold;">:SOA_Email</span><span style="color:#006600; font-weight:bold;">&#93;</span> = getVal<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'email'</span><span style="color:#006600; font-weight:bold;">&#41;</span> <span style="color:#006600; font-weight:bold;">||</span> ask<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Enter SOA email for the domain: &quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
      domainSave params
    <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
    <span style="color:#9966CC; font-weight:bold;">def</span> addDomainResource<span style="color:#006600; font-weight:bold;">&#40;</span>domain, resource, target, record_type = <span style="color:#996600;">'A'</span><span style="color:#006600; font-weight:bold;">&#41;</span>
      <span style="color:#0000FF; font-weight:bold;">return</span> <span style="color:#0000FF; font-weight:bold;">nil</span> <span style="color:#9966CC; font-weight:bold;">if</span> <span style="color:#006600; font-weight:bold;">&#40;</span>did = getDomainIdByName domain<span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#0000FF; font-weight:bold;">nil</span>?
      params = <span style="color:#006600; font-weight:bold;">&#123;</span><span style="color:#006600; font-weight:bold;">&#125;</span>
      params<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#ff3333; font-weight:bold;">:ResourceID</span><span style="color:#006600; font-weight:bold;">&#93;</span> = <span style="color:#006666;">0</span>
      params<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#ff3333; font-weight:bold;">:DomainID</span><span style="color:#006600; font-weight:bold;">&#93;</span> = did
      params<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#ff3333; font-weight:bold;">:Name</span><span style="color:#006600; font-weight:bold;">&#93;</span> = resource
      params<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#ff3333; font-weight:bold;">:Type</span><span style="color:#006600; font-weight:bold;">&#93;</span> = record_type
      params<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#ff3333; font-weight:bold;">:Target</span><span style="color:#006600; font-weight:bold;">&#93;</span> = target
      domainResourceSave params
    <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
    <span style="color:#9966CC; font-weight:bold;">def</span> getDomainIdByName domain
      domainList.<span style="color:#9900CC;">find</span> <span style="color:#9966CC; font-weight:bold;">do</span> <span style="color:#006600; font-weight:bold;">|</span>dom<span style="color:#006600; font-weight:bold;">|</span>
        <span style="color:#0000FF; font-weight:bold;">return</span> dom<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">&quot;DOMAINID&quot;</span><span style="color:#006600; font-weight:bold;">&#93;</span> <span style="color:#9966CC; font-weight:bold;">if</span> dom<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">&quot;DOMAIN&quot;</span><span style="color:#006600; font-weight:bold;">&#93;</span>.<span style="color:#9900CC;">downcase</span> == domain.<span style="color:#9900CC;">downcase</span>
      <span style="color:#9966CC; font-weight:bold;">end</span>
    <span style="color:#9966CC; font-weight:bold;">end</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
<span style="color:#9966CC; font-weight:bold;">else</span>
  flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'ApiCore not loaded. Skipping DNS stuff'</span>, <span style="color:#ff3333; font-weight:bold;">:notice</span><span style="color:#006600; font-weight:bold;">&#41;</span>
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
<span style="color:#9966CC; font-weight:bold;">class</span> App
  <span style="color:#9966CC; font-weight:bold;">class</span> <span style="color:#006600; font-weight:bold;">&amp;</span>lt;<span style="color:#006600; font-weight:bold;">&amp;</span>lt; <span style="color:#0000FF; font-weight:bold;">self</span>; attr_reader <span style="color:#ff3333; font-weight:bold;">:message</span>, <span style="color:#ff3333; font-weight:bold;">:name</span>, <span style="color:#ff3333; font-weight:bold;">:version</span>, <span style="color:#ff3333; font-weight:bold;">:description</span>, <span style="color:#ff3333; font-weight:bold;">:vhost</span>; <span style="color:#9966CC; font-weight:bold;">end</span>
  <span style="color:#0066ff; font-weight:bold;">@message</span> = <span style="color:#0000FF; font-weight:bold;">nil</span>
  <span style="color:#0066ff; font-weight:bold;">@name</span> = <span style="color:#0000FF; font-weight:bold;">nil</span>
  <span style="color:#0066ff; font-weight:bold;">@version</span> = <span style="color:#0000FF; font-weight:bold;">nil</span>
  <span style="color:#0066ff; font-weight:bold;">@description</span> = <span style="color:#996600;">'Just Another Web App'</span>
  <span style="color:#0066ff; font-weight:bold;">@vhost</span> = <span style="color:#996600;">'generic'</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> initialize<span style="color:#006600; font-weight:bold;">&#40;</span>rootdir, domain, db<span style="color:#006600; font-weight:bold;">&#41;</span>
    flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Initializing application (dir=#{rootdir}, domain=#{domain})&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    <span style="color:#0066ff; font-weight:bold;">@rootdir</span> = <span style="color:#006600; font-weight:bold;">&#40;</span>rootdir<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#006600; font-weight:bold;">-</span><span style="color:#006666;">1</span><span style="color:#006600; font-weight:bold;">&#93;</span>.<span style="color:#9900CC;">chr</span>.<span style="color:#9900CC;">eql</span>? <span style="color:#996600;">'/'</span><span style="color:#006600; font-weight:bold;">&#41;</span> ? rootdir.<span style="color:#CC0066; font-weight:bold;">chop</span> : rootdir
    <span style="color:#0066ff; font-weight:bold;">@domain</span> = domain
    <span style="color:#0066ff; font-weight:bold;">@db</span> = db <span style="color:#008000; font-style:italic;"># hash of database info </span>
&nbsp;
    <span style="color:#008000; font-style:italic;"># System stuff</span>
    <span style="color:#008000; font-style:italic;">#if Testing</span>
      <span style="color:#008000; font-style:italic;">#@templates = '/home/kiwi/Code/mksite/templates'</span>
      <span style="color:#008000; font-style:italic;">#@configs = '/home/kiwi/Code/mksite/configs'</span>
      <span style="color:#008000; font-style:italic;">#@vhosts = '/home/kiwi/Code/mksite/vhosts'</span>
    <span style="color:#008000; font-style:italic;">#else</span>
      <span style="color:#0066ff; font-weight:bold;">@templates</span> = <span style="color:#996600;">'/etc/mksite/templates'</span>
      <span style="color:#0066ff; font-weight:bold;">@configs</span> = <span style="color:#996600;">'/etc/mksite/configs'</span>
      <span style="color:#0066ff; font-weight:bold;">@vhosts</span> = <span style="color:#996600;">'/etc/mksite/vhosts'</span>
    <span style="color:#008000; font-style:italic;">#end</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#008000; font-style:italic;"># This generally does not need to be overriden.</span>
  <span style="color:#008000; font-style:italic;"># It does 'generic shit' like creating the rootdir</span>
  <span style="color:#008000; font-style:italic;"># and setting permissions</span>
  <span style="color:#9966CC; font-weight:bold;">def</span> envSetup
    <span style="color:#9966CC; font-weight:bold;">if</span> !rio<span style="color:#006600; font-weight:bold;">&#40;</span>@rootdir<span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">exist</span>?
      flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Creating directory #{@rootdir}&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
      rio<span style="color:#006600; font-weight:bold;">&#40;</span>@rootdir<span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">mkpath</span>
    <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
    rio<span style="color:#006600; font-weight:bold;">&#40;</span>@rootdir<span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">chdir</span> <span style="color:#9966CC; font-weight:bold;">do</span> <span style="color:#006600; font-weight:bold;">|</span>root<span style="color:#006600; font-weight:bold;">|</span>
      <span style="color:#008000; font-style:italic;"># Copy the generic public/private/log apache</span>
      <span style="color:#008000; font-style:italic;"># structure to rootdir</span>
      flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Changing working dir to #{@rootdir}&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
      flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Working inside '#{root.to_s}'&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
      rio<span style="color:#006600; font-weight:bold;">&#40;</span>@templates,<span style="color:#996600;">'skeleton.www'</span><span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">each</span> <span style="color:#006600; font-weight:bold;">&#123;</span> <span style="color:#006600; font-weight:bold;">|</span>df<span style="color:#006600; font-weight:bold;">|</span>
        <span style="color:#008000; font-style:italic;"># Overwrite existing files? .. yes.</span>
        <span style="color:#008000; font-style:italic;">#while rio(root, df).exist? do</span>
        df <span style="color:#006600; font-weight:bold;">&amp;</span>gt; root
        <span style="color:#008000; font-style:italic;">#flash(&quot;Copied #{df.to_s} to #{root.to_s}&quot;)</span>
      <span style="color:#006600; font-weight:bold;">&#125;</span> 
&nbsp;
      <span style="color:#008000; font-style:italic;"># Set permissions (a+w on logs, etc)</span>
      flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'Setting permissions...'</span><span style="color:#006600; font-weight:bold;">&#41;</span>
      flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'666 ./log/*.log'</span><span style="color:#006600; font-weight:bold;">&#41;</span>
      rio<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'./log/access.log'</span><span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">chmod</span><span style="color:#006600; font-weight:bold;">&#40;</span>00666<span style="color:#006600; font-weight:bold;">&#41;</span>
      rio<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'./log/error.log'</span><span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">chmod</span><span style="color:#006600; font-weight:bold;">&#40;</span>00666<span style="color:#006600; font-weight:bold;">&#41;</span>
      flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'700 ./private'</span><span style="color:#006600; font-weight:bold;">&#41;</span>
      rio<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'./private/'</span><span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">chmod</span><span style="color:#006600; font-weight:bold;">&#40;</span>00700<span style="color:#006600; font-weight:bold;">&#41;</span>
    <span style="color:#9966CC; font-weight:bold;">end</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> setup
    <span style="color:#008000; font-style:italic;"># envSetup()</span>
    <span style="color:#008000; font-style:italic;"># databaseSetup()</span>
    <span style="color:#CC0066; font-weight:bold;">raise</span> <span style="color:#996600;">'OVERRIDE ME'</span>
    <span style="color:#008000; font-style:italic;">#flash(&quot;setup() template function invoked. 'OVERRIDE ME'&quot;, :log)</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> databaseSetup
    <span style="color:#008000; font-style:italic;"># Create database if it doesn't already exist</span>
    <span style="color:#008000; font-style:italic;"># This usually doesn't need to be overriden</span>
    <span style="color:#008000; font-style:italic;">#Mysql.server_connect(@db['name'])</span>
    <span style="color:#008000; font-style:italic;">#  flash('Checking database connection... ')</span>
    <span style="color:#9966CC; font-weight:bold;">begin</span>
      dbo = Mysql.<span style="color:#9900CC;">real_connect</span><span style="color:#006600; font-weight:bold;">&#40;</span>@db<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'host'</span><span style="color:#006600; font-weight:bold;">&#93;</span>, <span style="color:#0066ff; font-weight:bold;">@db</span><span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'user'</span><span style="color:#006600; font-weight:bold;">&#93;</span>, <span style="color:#0066ff; font-weight:bold;">@db</span><span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'pass'</span><span style="color:#006600; font-weight:bold;">&#93;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
      flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Creating database '#{@db['name']}'&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>;
      res = dbo.<span style="color:#9900CC;">query</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;CREATE DATABASE IF NOT EXISTS #{@db['name']};&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
      flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Database server returned #{res}&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span> <span style="color:#9966CC; font-weight:bold;">if</span> res
    <span style="color:#9966CC; font-weight:bold;">rescue</span> <span style="color:#6666ff; font-weight:bold;">Mysql::Error</span> =<span style="color:#006600; font-weight:bold;">&amp;</span>gt; err
      flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'Unable to connect to access/create database'</span>, <span style="color:#ff3333; font-weight:bold;">:error</span><span style="color:#006600; font-weight:bold;">&#41;</span>
      flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Error returned (#{err.errno}) = '#{err.error}'&quot;</span>, <span style="color:#ff3333; font-weight:bold;">:error</span><span style="color:#006600; font-weight:bold;">&#41;</span>
      <span style="color:#CC0066; font-weight:bold;">exit</span> <span style="color:#006666;">1</span>
    <span style="color:#9966CC; font-weight:bold;">ensure</span>
      dbo.<span style="color:#9900CC;">close</span> <span style="color:#9966CC; font-weight:bold;">if</span> dbo
    <span style="color:#9966CC; font-weight:bold;">end</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> dnsSetup
    <span style="color:#0000FF; font-weight:bold;">return</span> <span style="color:#9966CC; font-weight:bold;">if</span> !<span style="color:#9966CC; font-weight:bold;">defined</span>? ApiCore
&nbsp;
    <span style="color:#008000; font-style:italic;"># Setup DNS - currently uses Linode API</span>
    flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Setting up DNS for '#{@domain}'&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;This requires a Linode API keynLogin to linode.com and find it under 'My Profile'&quot;</span>, <span style="color:#ff3333; font-weight:bold;">:emphasize</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    dns = <span style="color:#0000FF; font-weight:bold;">nil</span>
    api_key = getVal<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'apikey'</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    <span style="color:#CC0066; font-weight:bold;">loop</span> <span style="color:#9966CC; font-weight:bold;">do</span>
      api_key = ask<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'Paste your Linode API key (or '</span>skip<span style="color:#996600;">'): '</span><span style="color:#006600; font-weight:bold;">&#41;</span> <span style="color:#9966CC; font-weight:bold;">if</span> api_key.<span style="color:#0000FF; font-weight:bold;">nil</span>? <span style="color:#006600; font-weight:bold;">||</span> api_key.<span style="color:#9900CC;">empty</span>?
      <span style="color:#9966CC; font-weight:bold;">break</span> <span style="color:#9966CC; font-weight:bold;">if</span> api_key.<span style="color:#9900CC;">downcase</span>.<span style="color:#9900CC;">eql</span>? <span style="color:#996600;">'skip'</span>
&nbsp;
      <span style="color:#008000; font-style:italic;"># Check API key</span>
      <span style="color:#9966CC; font-weight:bold;">begin</span>
        dns = DNSAPI.<span style="color:#9900CC;">new</span> api_key
        dns.<span style="color:#9900CC;">domainList</span>
        <span style="color:#9966CC; font-weight:bold;">break</span>
      <span style="color:#9966CC; font-weight:bold;">rescue</span> <span style="color:#CC00FF; font-weight:bold;">RuntimeError</span>
        flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;API key invalid (or service down?). Learn to paste and try again.n&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
        api_key = <span style="color:#0000FF; font-weight:bold;">nil</span>
      <span style="color:#9966CC; font-weight:bold;">end</span>
    <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
    <span style="color:#9966CC; font-weight:bold;">unless</span> api_key.<span style="color:#0000FF; font-weight:bold;">nil</span>? <span style="color:#9966CC; font-weight:bold;">or</span> api_key.<span style="color:#9900CC;">downcase</span>.<span style="color:#9900CC;">eql</span>? <span style="color:#996600;">'skip'</span>
      flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Adding master domain '#{@domain}'&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
      <span style="color:#9966CC; font-weight:bold;">begin</span>
        <span style="color:#9966CC; font-weight:bold;">begin</span>
          dns.<span style="color:#9900CC;">addDomain</span><span style="color:#006600; font-weight:bold;">&#40;</span>@domain<span style="color:#006600; font-weight:bold;">&#41;</span>
        <span style="color:#9966CC; font-weight:bold;">rescue</span> <span style="color:#CC00FF; font-weight:bold;">RuntimeError</span>
          flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'Unable to add domain (exists already?). Attempting to add subdomains...'</span>, <span style="color:#ff3333; font-weight:bold;">:error</span><span style="color:#006600; font-weight:bold;">&#41;</span>
        <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
        server = getVal<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'server'</span><span style="color:#006600; font-weight:bold;">&#41;</span> <span style="color:#006600; font-weight:bold;">||</span>
                   ask<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Enter IP subdomains should point to (or 'skip'): &quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span> <span style="color:#006600; font-weight:bold;">&#123;</span> <span style="color:#006600; font-weight:bold;">|</span>q<span style="color:#006600; font-weight:bold;">|</span> q.<span style="color:#9900CC;">default</span> = <span style="color:#996600;">'skip'</span> <span style="color:#006600; font-weight:bold;">&#125;</span> 
&nbsp;
        <span style="color:#9966CC; font-weight:bold;">unless</span> server.<span style="color:#9900CC;">downcase</span>.<span style="color:#9900CC;">eql</span>? <span style="color:#996600;">'skip'</span>
          Subdomains.<span style="color:#9900CC;">each</span> <span style="color:#9966CC; font-weight:bold;">do</span> <span style="color:#006600; font-weight:bold;">|</span>sub<span style="color:#006600; font-weight:bold;">|</span>
            flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Adding subdomain '#{sub}.#{@domain}' (points to #{server})&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
            flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'Could not add subdomain'</span>, <span style="color:#ff3333; font-weight:bold;">:error</span><span style="color:#006600; font-weight:bold;">&#41;</span> <span style="color:#9966CC; font-weight:bold;">if</span> !dns.<span style="color:#9900CC;">addDomainResource</span><span style="color:#006600; font-weight:bold;">&#40;</span>@domain, <span style="color:#CC0066; font-weight:bold;">sub</span>, server<span style="color:#006600; font-weight:bold;">&#41;</span>
          <span style="color:#9966CC; font-weight:bold;">end</span>
        <span style="color:#9966CC; font-weight:bold;">end</span>
     <span style="color:#9966CC; font-weight:bold;">rescue</span>
       <span style="color:#CC0066; font-weight:bold;">raise</span>
       flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Unable to add domain/subdomain. Skipping&quot;</span>, <span style="color:#ff3333; font-weight:bold;">:error</span><span style="color:#006600; font-weight:bold;">&#41;</span>
     <span style="color:#9966CC; font-weight:bold;">end</span>
    <span style="color:#9966CC; font-weight:bold;">end</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> serverSetup
    <span style="color:#008000; font-style:italic;"># Set Apache vhost</span>
    vhost_file = <span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#9966CC; font-weight:bold;">defined</span>? <span style="color:#0000FF; font-weight:bold;">self</span>.<span style="color:#9966CC; font-weight:bold;">class</span>.<span style="color:#9900CC;">vhost</span><span style="color:#006600; font-weight:bold;">&#41;</span> ? <span style="color:#0000FF; font-weight:bold;">self</span>.<span style="color:#9966CC; font-weight:bold;">class</span>.<span style="color:#9900CC;">vhost</span> : <span style="color:#996600;">'generic'</span>
    flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Generating vhost file (#{@vhosts}/#{vhost_file}) for Apache 2.x&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    vhost = <span style="color:#996600;">''</span>
    email = getVal<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'email'</span><span style="color:#006600; font-weight:bold;">&#41;</span> <span style="color:#006600; font-weight:bold;">||</span> ask<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Enter a valid email for tech support: &quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    rio<span style="color:#006600; font-weight:bold;">&#40;</span>@vhosts, vhost_file<span style="color:#006600; font-weight:bold;">&#41;</span> <span style="color:#006600; font-weight:bold;">&amp;</span>gt; vhost
    vhost.<span style="color:#CC0066; font-weight:bold;">gsub!</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'_MKS_DOMAIN_'</span>, <span style="color:#0066ff; font-weight:bold;">@domain</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    vhost.<span style="color:#CC0066; font-weight:bold;">gsub!</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'_MKS_EMAIL_'</span>, email<span style="color:#006600; font-weight:bold;">&#41;</span>
    vhost.<span style="color:#CC0066; font-weight:bold;">gsub!</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'_MKS_ROOT_'</span>, <span style="color:#996600;">&quot;#{@rootdir}&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    vhost.<span style="color:#CC0066; font-weight:bold;">gsub!</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'_MKS_PUBLIC_'</span>, <span style="color:#996600;">&quot;#{@rootdir}/public&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    <span style="color:#008000; font-style:italic;">#rio(Testing ? '/tmp/' : Apache_sites, @domain).puts(vhost)</span>
    rio<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'/tmp/'</span>, <span style="color:#0066ff; font-weight:bold;">@domain</span><span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#CC0066; font-weight:bold;">puts</span><span style="color:#006600; font-weight:bold;">&#40;</span>vhost<span style="color:#006600; font-weight:bold;">&#41;</span> 
&nbsp;
    <span style="color:#008000; font-style:italic;"># Your enemies should not read this</span>
    rio<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'/tmp/'</span>, <span style="color:#0066ff; font-weight:bold;">@domain</span><span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">chmod</span><span style="color:#006600; font-weight:bold;">&#40;</span>00600<span style="color:#006600; font-weight:bold;">&#41;</span>
    flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;vhost file has been generated as /tmp/#{@domain}n
          It is YOUR responsibility to move this to #{Apache_sites}n
          Site will not work until you 'a2ensite &amp;amp;&amp;amp; apache2ctl restart'&quot;</span>, <span style="color:#ff3333; font-weight:bold;">:emphasize</span><span style="color:#006600; font-weight:bold;">&#41;</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> postInstall<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Finished!&quot;</span>, <span style="color:#ff3333; font-weight:bold;">:emphasize</span><span style="color:#006600; font-weight:bold;">&#41;</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
<span style="color:#9966CC; font-weight:bold;">class</span> Skeleton <span style="color:#006600; font-weight:bold;">&amp;</span>lt; App
  <span style="color:#0066ff; font-weight:bold;">@name</span> = <span style="color:#996600;">'Skeleton'</span>
  <span style="color:#0066ff; font-weight:bold;">@description</span> = <span style="color:#996600;">'Generic WWW directory structure'</span>
  <span style="color:#0066ff; font-weight:bold;">@message</span> = <span style="color:#996600;">''</span>
  <span style="color:#0066ff; font-weight:bold;">@vhost</span> = <span style="color:#996600;">'generic'</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> setup
    <span style="color:#0000FF; font-weight:bold;">self</span>.<span style="color:#9900CC;">envSetup</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    <span style="color:#0000FF; font-weight:bold;">self</span>.<span style="color:#9900CC;">dnsSetup</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    <span style="color:#0000FF; font-weight:bold;">self</span>.<span style="color:#9900CC;">serverSetup</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
<span style="color:#9966CC; font-weight:bold;">class</span> Wordpress <span style="color:#006600; font-weight:bold;">&amp;</span>lt; App
  <span style="color:#0066ff; font-weight:bold;">@name</span> = <span style="color:#996600;">'Wordpress'</span>
  <span style="color:#0066ff; font-weight:bold;">@version</span> = <span style="color:#996600;">'2.6.2'</span>
  <span style="color:#0066ff; font-weight:bold;">@description</span> = <span style="color:#996600;">'A popular blogging platform'</span>
  <span style="color:#0066ff; font-weight:bold;">@message</span> = <span style="color:#996600;">'Trying to make monies on the Internets?'</span>
  <span style="color:#0066ff; font-weight:bold;">@vhost</span> = <span style="color:#996600;">'generic'</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> setup
    <span style="color:#008000; font-style:italic;"># This sets up the initial environment / permissions</span>
    <span style="color:#0000FF; font-weight:bold;">self</span>.<span style="color:#9900CC;">envSetup</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006600; font-weight:bold;">&#41;</span> 
&nbsp;
    <span style="color:#008000; font-style:italic;"># Copy the wordpress skeleton directory to the new dir</span>
    wproot = rio<span style="color:#006600; font-weight:bold;">&#40;</span>@rootdir,<span style="color:#996600;">'public'</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Wordpress root will be #{wproot}&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'Copying Wordpress data over... (may take awhile)'</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    rio<span style="color:#006600; font-weight:bold;">&#40;</span>@templates,<span style="color:#996600;">'wordpress'</span><span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">each</span> <span style="color:#006600; font-weight:bold;">&#123;</span> <span style="color:#006600; font-weight:bold;">|</span>df<span style="color:#006600; font-weight:bold;">|</span> df <span style="color:#006600; font-weight:bold;">&amp;</span>gt; wproot <span style="color:#006600; font-weight:bold;">&#125;</span> 
&nbsp;
    rio<span style="color:#006600; font-weight:bold;">&#40;</span>wproot<span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">chdir</span> <span style="color:#9966CC; font-weight:bold;">do</span>
      rio<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'wp-config-sample.php'</span><span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">rm</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
      <span style="color:#008000; font-style:italic;"># Generate and output wp config</span>
      flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'Generating wp-config.php based on your DB settings...'</span><span style="color:#006600; font-weight:bold;">&#41;</span>
      wpcfg = rio<span style="color:#006600; font-weight:bold;">&#40;</span>@configs, <span style="color:#996600;">'wordpress.cfg'</span><span style="color:#006600; font-weight:bold;">&#41;</span>
      <span style="color:#9966CC; font-weight:bold;">if</span> !wpcfg.<span style="color:#9900CC;">exist</span>? <span style="color:#006600; font-weight:bold;">||</span> !wpcfg.<span style="color:#9900CC;">readable</span>?
        flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Wordpress config template missing or unreadable, quitting&quot;</span>, <span style="color:#ff3333; font-weight:bold;">:error</span><span style="color:#006600; font-weight:bold;">&#41;</span>
        <span style="color:#CC0066; font-weight:bold;">exit</span> <span style="color:#006666;">2</span>
      <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
      <span style="color:#008000; font-style:italic;"># Copy the config into a string, do things with it and then write it to disk</span>
      config = <span style="color:#996600;">''</span>
      rio<span style="color:#006600; font-weight:bold;">&#40;</span>@configs, <span style="color:#996600;">'wordpress.cfg'</span><span style="color:#006600; font-weight:bold;">&#41;</span> <span style="color:#006600; font-weight:bold;">&amp;</span>gt; config
      flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Warning: config file '#{@configs}/wordpress.cfg' is empty&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span> <span style="color:#9966CC; font-weight:bold;">if</span> config.<span style="color:#9900CC;">empty</span>?
      config<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'_MKS_DB_HOST_'</span><span style="color:#006600; font-weight:bold;">&#93;</span> = <span style="color:#0066ff; font-weight:bold;">@db</span><span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'host'</span><span style="color:#006600; font-weight:bold;">&#93;</span>
      config<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'_MKS_DB_USER_'</span><span style="color:#006600; font-weight:bold;">&#93;</span> = <span style="color:#0066ff; font-weight:bold;">@db</span><span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'user'</span><span style="color:#006600; font-weight:bold;">&#93;</span>
      config<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'_MKS_DB_PASS_'</span><span style="color:#006600; font-weight:bold;">&#93;</span> = <span style="color:#0066ff; font-weight:bold;">@db</span><span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'pass'</span><span style="color:#006600; font-weight:bold;">&#93;</span>
      config<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'_MKS_DB_NAME_'</span><span style="color:#006600; font-weight:bold;">&#93;</span> = <span style="color:#0066ff; font-weight:bold;">@db</span><span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'name'</span><span style="color:#006600; font-weight:bold;">&#93;</span>
      config.<span style="color:#CC0066; font-weight:bold;">gsub!</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'_MKS_SECRET_'</span>, genAlpha<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006600; font-weight:bold;">&#41;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
      flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Writing #{wpcfg} data&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
      rio<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'wp-config.php'</span><span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">w</span>!.<span style="color:#CC0066; font-weight:bold;">puts</span><span style="color:#006600; font-weight:bold;">&#40;</span>config<span style="color:#006600; font-weight:bold;">&#41;</span>
    <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
    <span style="color:#008000; font-style:italic;"># Setup the database</span>
    <span style="color:#0000FF; font-weight:bold;">self</span>.<span style="color:#9900CC;">databaseSetup</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006600; font-weight:bold;">&#41;</span> 
&nbsp;
    <span style="color:#008000; font-style:italic;"># Add DNS info</span>
    <span style="color:#0000FF; font-weight:bold;">self</span>.<span style="color:#9900CC;">dnsSetup</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006600; font-weight:bold;">&#41;</span> 
&nbsp;
    <span style="color:#008000; font-style:italic;"># Setup the server/vhost</span>
    <span style="color:#0000FF; font-weight:bold;">self</span>.<span style="color:#9900CC;">serverSetup</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
<span style="color:#9966CC; font-weight:bold;">class</span> Clipshare <span style="color:#006600; font-weight:bold;">&amp;</span>lt; App
  <span style="color:#9966CC; font-weight:bold;">def</span> setup
    <span style="color:#CC0066; font-weight:bold;">raise</span> <span style="color:#996600;">'Clipshare support is currently not available. sowwie'</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
<span style="color:#9966CC; font-weight:bold;">class</span> Joomla <span style="color:#006600; font-weight:bold;">&amp;</span>lt; App
  <span style="color:#0066ff; font-weight:bold;">@name</span> = <span style="color:#996600;">'Joomla'</span>
  <span style="color:#0066ff; font-weight:bold;">@version</span> = <span style="color:#996600;">'1.5.7'</span>
  <span style="color:#0066ff; font-weight:bold;">@description</span> = <span style="color:#996600;">'A widely used Content Management System'</span>
  <span style="color:#0066ff; font-weight:bold;">@message</span> = <span style="color:#996600;">''</span>
  <span style="color:#0066ff; font-weight:bold;">@vhost</span> = <span style="color:#996600;">'generic'</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> setup
    <span style="color:#008000; font-style:italic;"># This sets up the initial environment / permissions</span>
    <span style="color:#0000FF; font-weight:bold;">self</span>.<span style="color:#9900CC;">envSetup</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006600; font-weight:bold;">&#41;</span> 
&nbsp;
    <span style="color:#008000; font-style:italic;"># Copy the wordpress skeleton directory to the new dir</span>
    flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'Copying Joomla data over... (may take awhile)'</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    jooroot = rio<span style="color:#006600; font-weight:bold;">&#40;</span>@rootdir,<span style="color:#996600;">'public'</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Joomla root will be #{jooroot}&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    rio<span style="color:#006600; font-weight:bold;">&#40;</span>@templates,<span style="color:#996600;">'joomla-1.5.7'</span><span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">each</span> <span style="color:#006600; font-weight:bold;">&#123;</span> <span style="color:#006600; font-weight:bold;">|</span>df<span style="color:#006600; font-weight:bold;">|</span> df <span style="color:#006600; font-weight:bold;">&amp;</span>gt; jooroot <span style="color:#006600; font-weight:bold;">&#125;</span> 
&nbsp;
    rio<span style="color:#006600; font-weight:bold;">&#40;</span>jooroot<span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">chdir</span> <span style="color:#9966CC; font-weight:bold;">do</span>
      flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'Removing stock Joomla config...'</span><span style="color:#006600; font-weight:bold;">&#41;</span>
      rio<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'configuration.php-dist'</span><span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">rm</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
&nbsp;
      <span style="color:#008000; font-style:italic;"># Generate and output joomla config</span>
      flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'Generating configuration.php based on provided settings...'</span><span style="color:#006600; font-weight:bold;">&#41;</span>
      joocfg = rio<span style="color:#006600; font-weight:bold;">&#40;</span>@configs, <span style="color:#996600;">'joomla.cfg'</span><span style="color:#006600; font-weight:bold;">&#41;</span>
      <span style="color:#9966CC; font-weight:bold;">if</span> !joocfg.<span style="color:#9900CC;">exist</span>? <span style="color:#006600; font-weight:bold;">||</span> !joocfg.<span style="color:#9900CC;">readable</span>?
        flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Joomla config template missing or unreadable, quitting&quot;</span>, <span style="color:#ff3333; font-weight:bold;">:error</span><span style="color:#006600; font-weight:bold;">&#41;</span>
        <span style="color:#CC0066; font-weight:bold;">exit</span> <span style="color:#006666;">2</span>
      <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
      <span style="color:#008000; font-style:italic;"># Copy the config into a string, do things with it and then write it to disk</span>
      config = <span style="color:#996600;">''</span>
      rio<span style="color:#006600; font-weight:bold;">&#40;</span>@configs, <span style="color:#996600;">'joomla.cfg'</span><span style="color:#006600; font-weight:bold;">&#41;</span> <span style="color:#006600; font-weight:bold;">&amp;</span>gt; config
      flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Warning: config file '#{@configs}/joomla.cfg' is empty&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span> <span style="color:#9966CC; font-weight:bold;">if</span> config.<span style="color:#9900CC;">empty</span>?
      config<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'_MKS_DB_HOST_'</span><span style="color:#006600; font-weight:bold;">&#93;</span> = <span style="color:#0066ff; font-weight:bold;">@db</span><span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'host'</span><span style="color:#006600; font-weight:bold;">&#93;</span>
      config<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'_MKS_DB_USER_'</span><span style="color:#006600; font-weight:bold;">&#93;</span> = <span style="color:#0066ff; font-weight:bold;">@db</span><span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'user'</span><span style="color:#006600; font-weight:bold;">&#93;</span>
      config<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'_MKS_DB_PASS_'</span><span style="color:#006600; font-weight:bold;">&#93;</span> = <span style="color:#0066ff; font-weight:bold;">@db</span><span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'pass'</span><span style="color:#006600; font-weight:bold;">&#93;</span>
      config<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'_MKS_DB_NAME_'</span><span style="color:#006600; font-weight:bold;">&#93;</span> = <span style="color:#0066ff; font-weight:bold;">@db</span><span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'name'</span><span style="color:#006600; font-weight:bold;">&#93;</span>
      config.<span style="color:#CC0066; font-weight:bold;">gsub!</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'_MKS_DOMAIN_'</span>, <span style="color:#0066ff; font-weight:bold;">@domain</span><span style="color:#006600; font-weight:bold;">&#41;</span>
      config.<span style="color:#CC0066; font-weight:bold;">gsub!</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'_MKS_SECRET_'</span>, genAlpha<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006600; font-weight:bold;">&#41;</span><span style="color:#006600; font-weight:bold;">&#41;</span> 
&nbsp;
      flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Writing ./configuration.php&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
      rio<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'configuration.php'</span><span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">w</span>!.<span style="color:#CC0066; font-weight:bold;">puts</span><span style="color:#006600; font-weight:bold;">&#40;</span>config<span style="color:#006600; font-weight:bold;">&#41;</span> 
&nbsp;
      flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Making configuration.php world writable (0666)&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
      rio<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'configuration.php'</span><span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">chmod</span><span style="color:#006600; font-weight:bold;">&#40;</span>0666<span style="color:#006600; font-weight:bold;">&#41;</span>
    <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
    <span style="color:#008000; font-style:italic;"># Setup the database</span>
    <span style="color:#0000FF; font-weight:bold;">self</span>.<span style="color:#9900CC;">databaseSetup</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006600; font-weight:bold;">&#41;</span> 
&nbsp;
    <span style="color:#008000; font-style:italic;"># Add DNS info</span>
    <span style="color:#0000FF; font-weight:bold;">self</span>.<span style="color:#9900CC;">dnsSetup</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006600; font-weight:bold;">&#41;</span> 
&nbsp;
    <span style="color:#008000; font-style:italic;"># Setup the server/vhost</span>
    <span style="color:#0000FF; font-weight:bold;">self</span>.<span style="color:#9900CC;">serverSetup</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">def</span> postInstall<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'Joomla is ready to be setup using the web interface'</span>, <span style="color:#ff3333; font-weight:bold;">:emphasize</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Go to #{@domain} where the Joomla! web based installer will
           guide you through the rest of the installation&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Here's the database information:n
           USERNAME: #{@db['user']}n
           DB HOST : #{@db['host']}n
           DB NAME : #{@db['name']}&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span> 
&nbsp;
    flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;nAdmin panel is located @ #{@domain}/administrator &quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;You can log into Admin using the username 'admin' along with the
    password that was generated or you chose during the web based install.&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    <span style="color:#9966CC; font-weight:bold;">super</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
<span style="color:#9966CC; font-weight:bold;">class</span> PHPMotion <span style="color:#006600; font-weight:bold;">&amp;</span>lt; App
  <span style="color:#9966CC; font-weight:bold;">def</span> setup
    <span style="color:#CC0066; font-weight:bold;">raise</span> <span style="color:#996600;">'PHPMotion support is currently not available. sowwie'</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
<span style="color:#9966CC; font-weight:bold;">class</span> Log
  <span style="color:#9966CC; font-weight:bold;">def</span> initialize<span style="color:#006600; font-weight:bold;">&#40;</span>filename=<span style="color:#996600;">&quot;/tmp/#{Username}_mksite.log&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    <span style="color:#0066ff; font-weight:bold;">@log</span> = rio<span style="color:#006600; font-weight:bold;">&#40;</span>filename<span style="color:#006600; font-weight:bold;">&#41;</span>
    n=<span style="color:#006666;">0</span>; <span style="color:#0066ff; font-weight:bold;">@log</span> = rio<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;#{filename}.#{n+=1}&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span> <span style="color:#9966CC; font-weight:bold;">while</span> <span style="color:#0066ff; font-weight:bold;">@log</span>.<span style="color:#9900CC;">exist</span>?
    <span style="color:#0066ff; font-weight:bold;">@log</span>.<span style="color:#CC0066; font-weight:bold;">puts</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;# GENERATED BY MKS - BEGAN @ #{Time.now.to_i}&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    <span style="color:#0066ff; font-weight:bold;">@log</span>.<span style="color:#9900CC;">chmod</span><span style="color:#006600; font-weight:bold;">&#40;</span>00600<span style="color:#006600; font-weight:bold;">&#41;</span>
    flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Log file generated (important): #{@log.to_s} (no worries, set to 600)&quot;</span>, <span style="color:#ff3333; font-weight:bold;">:emphasize</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    <span style="color:#0000FF; font-weight:bold;">return</span> <span style="color:#0066ff; font-weight:bold;">@log</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#008000; font-style:italic;"># append to log</span>
  <span style="color:#9966CC; font-weight:bold;">def</span> <span style="color:#006600; font-weight:bold;">&amp;</span>lt;<span style="color:#006600; font-weight:bold;">&amp;</span>lt;<span style="color:#006600; font-weight:bold;">&#40;</span>data<span style="color:#006600; font-weight:bold;">&#41;</span>
    <span style="color:#008000; font-style:italic;">#super &amp;lt;&amp;lt;(data)</span>
    <span style="color:#0066ff; font-weight:bold;">@log</span>.<span style="color:#CC0066; font-weight:bold;">puts</span><span style="color:#006600; font-weight:bold;">&#40;</span>data<span style="color:#006600; font-weight:bold;">&#41;</span>
    <span style="color:#0066ff; font-weight:bold;">@log</span>.<span style="color:#CC0066; font-weight:bold;">puts</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;n&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    <span style="color:#008000; font-style:italic;">#@log &amp;lt;&amp;lt; data &amp;lt;&amp;lt; &quot;n&quot;</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
<span style="color:#9966CC; font-weight:bold;">def</span> loadConfig<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
  cfgpath = <span style="color:#996600;">&quot;#{ENV['HOME']}/.mksite&quot;</span>
  <span style="color:#0000FF; font-weight:bold;">return</span> <span style="color:#CC00FF; font-weight:bold;">YAML</span>.<span style="color:#CC0066; font-weight:bold;">load</span><span style="color:#006600; font-weight:bold;">&#40;</span>rio<span style="color:#006600; font-weight:bold;">&#40;</span>cfgpath<span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">read</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006600; font-weight:bold;">&#41;</span><span style="color:#006600; font-weight:bold;">&#41;</span> <span style="color:#9966CC; font-weight:bold;">if</span> rio<span style="color:#006600; font-weight:bold;">&#40;</span>cfgpath<span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">exist</span>?
  <span style="color:#0000FF; font-weight:bold;">nil</span>
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
<span style="color:#9966CC; font-weight:bold;">def</span> getVal<span style="color:#006600; font-weight:bold;">&#40;</span>key, default=<span style="color:#0000FF; font-weight:bold;">nil</span><span style="color:#006600; font-weight:bold;">&#41;</span>
  <span style="color:#0000FF; font-weight:bold;">return</span> <span style="color:#ff6633; font-weight:bold;">$config</span><span style="color:#006600; font-weight:bold;">&#91;</span>key<span style="color:#006600; font-weight:bold;">&#93;</span> <span style="color:#9966CC; font-weight:bold;">if</span> <span style="color:#ff6633; font-weight:bold;">$config</span> <span style="color:#9966CC; font-weight:bold;">and</span> <span style="color:#9966CC; font-weight:bold;">defined</span>? <span style="color:#ff6633; font-weight:bold;">$config</span><span style="color:#006600; font-weight:bold;">&#91;</span>key<span style="color:#006600; font-weight:bold;">&#93;</span>
  default
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
<span style="color:#9966CC; font-weight:bold;">def</span> main
  <span style="color:#CC0066; font-weight:bold;">exit</span> <span style="color:#006666;">1</span> <span style="color:#9966CC; font-weight:bold;">if</span> <span style="color:#996600;">'root'</span>==ENV<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'USER'</span><span style="color:#006600; font-weight:bold;">&#93;</span>
  <span style="color:#9966CC; font-weight:bold;">if</span> <span style="color:#006600; font-weight:bold;">&#40;</span>$config = loadConfig<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006600; font-weight:bold;">&#41;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'~/.mksite config loaded'</span><span style="color:#006600; font-weight:bold;">&#41;</span>
  <span style="color:#9966CC; font-weight:bold;">else</span>
    flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'~/.mksite not found. It'</span>s fine, I<span style="color:#996600;">'ll annoy you with questions.'</span><span style="color:#006600; font-weight:bold;">&#41;</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#CC0066; font-weight:bold;">puts</span> <span style="color:#ff6633; font-weight:bold;">$config</span>.<span style="color:#9900CC;">inspect</span> <span style="color:#9966CC; font-weight:bold;">if</span> Testing
  flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'Press ^C (CTRL+C) at any time quit'</span>, <span style="color:#ff3333; font-weight:bold;">:emphasize</span><span style="color:#006600; font-weight:bold;">&#41;</span>
  <span style="color:#008000; font-style:italic;"># Let's ask neutral questions about the new site.</span>
  <span style="color:#9966CC; font-weight:bold;">if</span> Testing
    rootdir = <span style="color:#996600;">'/tmp/sandbox9/'</span>
    domain = <span style="color:#996600;">'domain.cxm'</span>
  <span style="color:#9966CC; font-weight:bold;">else</span>
    domain = ask<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'Enter site'</span>s domain name <span style="color:#006600; font-weight:bold;">&#40;</span>no http:<span style="color:#006600; font-weight:bold;">//</span> <span style="color:#9966CC; font-weight:bold;">or</span> www<span style="color:#006600; font-weight:bold;">&#41;</span>: <span style="color:#996600;">') #{ |d| d.validate = !! /^www.|^http:/// }
    rootdir = ask('</span>Root site directory <span style="color:#006600; font-weight:bold;">&#40;</span>leave blank <span style="color:#9966CC; font-weight:bold;">for</span> default<span style="color:#006600; font-weight:bold;">&#41;</span>: <span style="color:#996600;">') { |q|
      q.default = &quot;/home/#{Username}/www/#{domain}/&quot;
      q.validate = /^/home/#{Username}//
    }
  end
&nbsp;
  flash(&quot;Domain has been set to &quot;#{domain}&quot;&quot;)
  #$log = Log.new(&quot;/tmp/mks_#{domain}.log&quot;) 
&nbsp;
  flash(&quot;Site will reside in &quot;#{rootdir}&quot;, and the index/script&quot;)
  flash(&quot; files (index.php, .htaccess etc) will go in &quot;#{rootdir}/public/&quot;nn&quot;)
&nbsp;
  # Ask the user what app she wants to install and specific questions about that app
  app = nil
  choose do |menu|
    menu.prompt = '</span>Choose the software <span style="color:#9966CC; font-weight:bold;">for</span> your new site: <span style="color:#996600;">'
    Applications.each do |app|
      menu.choice app do |a|
        app = a
      end
    end
  end
&nbsp;
  flash(&quot;You chose #{app}. #{Kernel.const_get(app).message}n&quot;) 
&nbsp;
  # Fetch DB info if this app needs a database
  # XXX Should just have a db flag in the App'</span>s <span style="color:#9966CC; font-weight:bold;">class</span>
  db = <span style="color:#0000FF; font-weight:bold;">nil</span>
  <span style="color:#9966CC; font-weight:bold;">unless</span> app.<span style="color:#9900CC;">eql</span>? <span style="color:#ff3333; font-weight:bold;">:Skeleton</span>
    flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'This app requires a database (only MySQL supported)'</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'It will be created if it doesn'</span>t exist<span style="color:#996600;">')
&nbsp;
    db = {}
    db['</span>host<span style="color:#996600;">'] = '</span>localhost<span style="color:#996600;">'
    loop do
      if Testing
        db['</span>user<span style="color:#996600;">'] = '</span>kiwi<span style="color:#996600;">'
        db['</span>pass<span style="color:#996600;">'] = '</span><span style="color:#996600;">'
        db['</span>name<span style="color:#996600;">'] = '</span>kiwi_sandbox9<span style="color:#996600;">'
      else
        db['</span>user<span style="color:#996600;">'] = getVal('</span>db_user<span style="color:#996600;">') || ask('</span>Enter MySQL username: <span style="color:#996600;">')
        db['</span>pass<span style="color:#996600;">'] = getVal('</span>db_pass<span style="color:#996600;">') || ask('</span>Enter MySQL password: <span style="color:#996600;">') { |q| q.echo = '</span><span style="color:#006600; font-weight:bold;">*</span><span style="color:#996600;">'}
        db['</span>name<span style="color:#996600;">'] = ask('</span>Enter database name <span style="color:#006600; font-weight:bold;">&#40;</span>should <span style="color:#9966CC; font-weight:bold;">begin</span> w<span style="color:#006600; font-weight:bold;">/</span> your <span style="color:#996600;">'username_'</span><span style="color:#006600; font-weight:bold;">&#41;</span><span style="color:#996600;">') { |dbn| dbn.validate = /^#{Username}_/}
        puts db.inspect
      end
      flash(&quot;Please double check: mysql://#{db['</span>user<span style="color:#996600;">']}:#{'</span><span style="color:#006600; font-weight:bold;">*</span><span style="color:#996600;">' * (db['</span>pass<span style="color:#996600;">']).size}@#{db['</span>host<span style="color:#996600;">']}/#{db['</span>name<span style="color:#996600;">']}&quot;)
      break if Testing || agree('</span>Is this correct?<span style="color:#996600;">')
      flash(&quot;Please re-enter database infon&quot;)
    end
  end
&nbsp;
  # Check permissions and database login
  flash('</span>Doing a preliminary check<span style="color:#996600;">', :emphasize)
  flash('</span>Checking installation environment<span style="color:#996600;">')
  # Quit if the directory exists and the user does not want to go ahead
  unless Testing
    exit 1 if rio(rootdir).exist? &amp;amp;&amp;amp; !agree(&quot;Directory '</span><span style="color:#008000; font-style:italic;">#{rootdir}' exists! Continue? (Y/N)&quot;)</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#008000; font-style:italic;"># Is the parent directory available and writable?</span>
  parentdir = rio<span style="color:#006600; font-weight:bold;">&#40;</span>rootdir<span style="color:#006600; font-weight:bold;">&#41;</span>.<span style="color:#9900CC;">dirname</span>
  <span style="color:#9966CC; font-weight:bold;">unless</span> parentdir.<span style="color:#9900CC;">exist</span>? <span style="color:#006600; font-weight:bold;">&amp;</span>amp;<span style="color:#006600; font-weight:bold;">&amp;</span>amp; parentdir.<span style="color:#9900CC;">writable</span>?
    flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Directory '#{parentdir.to_s}' either doesn't exist or is not writable. Check permissions&quot;</span>, <span style="color:#ff3333; font-weight:bold;">:error</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    <span style="color:#CC0066; font-weight:bold;">exit</span> <span style="color:#006666;">1</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#9966CC; font-weight:bold;">unless</span> db.<span style="color:#0000FF; font-weight:bold;">nil</span>? <span style="color:#006600; font-weight:bold;">||</span> <span style="color:#0000FF; font-weight:bold;">false</span>==getVal<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'db_confirm'</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    <span style="color:#008000; font-style:italic;"># Check DB connection - but this does not check if user has any privileges</span>
    flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'Checking database connection'</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    <span style="color:#9966CC; font-weight:bold;">begin</span>
      dbo = Mysql.<span style="color:#9900CC;">real_connect</span><span style="color:#006600; font-weight:bold;">&#40;</span>db<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'host'</span><span style="color:#006600; font-weight:bold;">&#93;</span>, db<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'user'</span><span style="color:#006600; font-weight:bold;">&#93;</span>, db<span style="color:#006600; font-weight:bold;">&#91;</span><span style="color:#996600;">'pass'</span><span style="color:#006600; font-weight:bold;">&#93;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
      flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Successfully connected to '#{dbo.get_server_info}'&quot;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
    <span style="color:#9966CC; font-weight:bold;">rescue</span> <span style="color:#6666ff; font-weight:bold;">Mysql::Error</span> =<span style="color:#006600; font-weight:bold;">&amp;</span>gt; err
      flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">'Unable to connect to database server'</span>, <span style="color:#ff3333; font-weight:bold;">:error</span><span style="color:#006600; font-weight:bold;">&#41;</span>
      flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Error returned (#{err.errno}) = '#{err.error}'&quot;</span>, <span style="color:#ff3333; font-weight:bold;">:error</span><span style="color:#006600; font-weight:bold;">&#41;</span>
      <span style="color:#CC0066; font-weight:bold;">exit</span> <span style="color:#006666;">1</span>
    <span style="color:#9966CC; font-weight:bold;">ensure</span>
      dbo.<span style="color:#9900CC;">close</span> <span style="color:#9966CC; font-weight:bold;">if</span> dbo
    <span style="color:#9966CC; font-weight:bold;">end</span>
  <span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
  <span style="color:#008000; font-style:italic;"># Begin installation</span>
  flash<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#996600;">&quot;Word on the server racks is... you're good to go&quot;</span>, <span style="color:#ff3333; font-weight:bold;">:emphasize</span><span style="color:#006600; font-weight:bold;">&#41;</span>
  klass = <span style="color:#CC00FF; font-weight:bold;">Kernel</span>.<span style="color:#9900CC;">const_get</span><span style="color:#006600; font-weight:bold;">&#40;</span>app<span style="color:#006600; font-weight:bold;">&#41;</span>
  webapp = klass.<span style="color:#9900CC;">new</span><span style="color:#006600; font-weight:bold;">&#40;</span>rootdir, domain, db<span style="color:#006600; font-weight:bold;">&#41;</span>
  webapp.<span style="color:#9900CC;">setup</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
  webapp.<span style="color:#9900CC;">postInstall</span><span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006600; font-weight:bold;">&#41;</span>
<span style="color:#9966CC; font-weight:bold;">end</span>
&nbsp;
main<span style="color:#006600; font-weight:bold;">&#40;</span><span style="color:#006600; font-weight:bold;">&#41;</span></pre></div></div>

<p><a href="http://biodegradablegeek.com/wp-content/uploads/2008/10/mksite.rb" target="_new"><strong>Download</strong> the ASCII mksite.rb file here.</a></p>
]]></content:encoded>
			<wfw:commentRss>http://biodegradablegeek.com/2008/10/script-to-quickly-setup-webapp-environment-and-domain/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>How to POST Form Data Using Ruby</title>
		<link>http://biodegradablegeek.com/2008/04/how-to-post-form-data-using-ruby/</link>
		<comments>http://biodegradablegeek.com/2008/04/how-to-post-form-data-using-ruby/#comments</comments>
		<pubDate>Thu, 24 Apr 2008 15:43:09 +0000</pubDate>
		<dc:creator>Isam</dc:creator>
				<category><![CDATA[Automation]]></category>
		<category><![CDATA[Code]]></category>
		<category><![CDATA[Ruby]]></category>
		<category><![CDATA[Scraping]]></category>
		<category><![CDATA[Snippets]]></category>
		<category><![CDATA[code]]></category>
		<category><![CDATA[http]]></category>
		<category><![CDATA[scripting]]></category>
		<category><![CDATA[Scripts]]></category>
		<category><![CDATA[Tips]]></category>

		<guid isPermaLink="false">http://biodegradablegeek.com/2008/04/24/how-to-post-form-data-using-ruby/</guid>
		<description><![CDATA[POSTing data on web forms is essential for writing tools and services that interact with resources already available on the web. You can grab information from your Gmail account, add a new thread to a forum from your own app, etc. 
The following is a brief example on how this can be done in Ruby [...]]]></description>
			<content:encoded><![CDATA[<p><strong>POST</strong>ing data on web forms is essential for writing tools and services that interact with resources already available on the web. You can grab information from your Gmail account, add a new thread to a forum from your own app, etc. </p>
<p>The following is a brief example on how this can be done in Ruby using <a href="http://ruby-doc.org/stdlib/libdoc/net/http/rdoc/index.html" target="_blank">Net::HTTP</a>and <a href="http://www.interlacken.com/webdbdev/ch05/formpost.asp" target="_blank">this POST form example</a>.</p>
<p>Looking at the source (interlacken.com/webdbdev/ch05/formpost.asp):</p>
<pre class="brush: xml;">
&lt;form method=&quot;POST&quot; action=&quot;formpost.asp&quot;&gt;
&lt;p&gt;&lt;input type=&quot;text&quot; name=&quot;box1″ size=&quot;20″ value=&quot;&quot;&gt;
&lt;input type=&quot;submit&quot; value=&quot;Submit&quot; name=&quot;button1″&gt;&lt;/p&gt;
&lt;/form&gt;
</pre>
<p>We see two attributes are sent to the formpost.asp script when the user hits the submit button: A textbox named <strong>box1</strong> and the value of the submit button, named <strong>Submit</strong>. If this form used a GET method, we would just fetch the URL postfixed with (for example) <strong>?box1=our+text+here</strong>. Fortunately, Ruby&#8217;s Net::HTTP makes posting data just as easy.</p>
<p>The Ruby code: </p>
<pre class="brush: ruby;">
#!/usr/bin/ruby

require &quot;uri&quot;
require &quot;net/http&quot;

params = {'box1′ =&gt; 'Nothing is less important than which fork you use. Etiquette is the science of living. It embraces everything. It is ethics. It is honor. -Emily Post',
'button1′ =&gt; 'Submit'
}
x = Net::HTTP.post_form(URI.parse('http://www.interlacken.com/webdbdev/ch05/formpost.asp'), params)
puts x.body

# Uncomment this if you want output in a file
# File.open('out.htm', 'w') { |f| f.write x.body }
</pre>
<p>Sending the value of button1 is optional in this case, but sometimes this value is checked in the server side script. One example is when the coder wants to find out if the form has been submitted &#8211; as opposed to it being the user&#8217;s first visit to the form &#8211; without creating a hidden attribute to send along w/ the other form fields. Besides, there&#8217;s no harm in sending a few more bytes.</p>
<p>If you&#8217;re curious about URI.parse, it simply makes the URI easier to work with by separating and classifying each of its attributes, effectively letting the methods in Net::HTTP do their sole job only, instead of having to analyze and parse the URL. More info on this in the <a href="http://www.ruby-doc.org/stdlib/libdoc/uri/rdoc/classes/URI.html#M009241" target="_blank">Ruby doc</a>.</p>
<p>Assuming no errors, running this example (<em>ruby postpost</em> or <em>chmod a+x postpost.rb; ./postpost.rb</em>) yields:</p>
<pre class="brush: xml;">
&lt;form method=&quot;POST&quot; action=&quot;formpost.asp&quot;&gt;
&lt;p&gt;&lt;input type=&quot;text&quot; name=&quot;box1″ size=&quot;20″ value=&quot;NOTHING IS LESS
IMPORTANT THAN WHICH FORK YOU USE. ETIQUETTE IS THE
SCIENCE OF LIVING. IT EMBRACES EVERYTHING. IT IS ETHICS.
IT IS HONOR. -EMILY POST&quot;&gt;
&lt;input type=&quot;submit&quot; value=&quot;Submit&quot; name=&quot;button1″&gt;&lt;/p&gt;
&lt;/form&gt;
</pre>
<p>In practice, you might want to use a more specialized library to handle what you&#8217;re doing. Be sure to check out <a href="http://mechanize.rubyforge.org/mechanize/" target="_blank">Mechanize</a> and <a href="http://github.com/adamwiggins/rest-client/tree/master" target="_blank">Rest-client</a>.</p>
]]></content:encoded>
			<wfw:commentRss>http://biodegradablegeek.com/2008/04/how-to-post-form-data-using-ruby/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
	</channel>
</rss>
