Tag Archives: Shell

Bash Tips for Power Users

Every Geek site needs an obligatory Bash Tips post

Copy Files Securely Between Two Machines

I used to always forget the syntax for this, until I realized that the syntax is exactly like the standard cp command. In fact, you can copy files like you normally would using scp, on your local machine. The following are equivalent:

$ cp file file.orig
$ scp file file.orig

Where they differ is, scp lets you copy files over a network, through SSH. Here’s an example:

$ scp contents.txt silver@ssh.domain.com:/tmp

This will copy local file contents.txt to /tmp on the remote machine ssh.domain.com, as user silver. Here are some more examples:

$ scp draft.pdf ssh.domain.com:

(copy draft.pdf to my home dir on remote machine. username is implied to be the same locally and remotely.)

$ scp swine.jpg rex@ssh.domain.com

(read: This will copy swine.jpg to local machine as a file named rex@ssh.domain.com. To make it go remote, append a : to the address, like above)

scp supports, among other things, compression (-C) and recursive copying of directories (-r).

$ scp -rC code/ ssh.domain.com:/archive/code_02032009

Trying to copy to a directory you don’t have permission to (/usr etc) will fail.

Don’t Get Lost Jumping To and Fro Between Directories

You can use cd - to jump to the previous (NOT parent) dir. For example:

kiwi@localhost: ~ $ cd /usr/local/share
kiwi@localhost: /usr/local/share $ cd -
/home/kiwi
kiwi@localhost: ~ $ cd -
/usr/local/share
kiwi@localhost: /usr/local/share $

Another way is using pushd/popd – A Last In First Out (LIFO) stack of dirs.

kiwi@localhost: ~ $ pushd /usr/local/share/
/usr/local/share ~

pushd is like cd but keeps note of the current dir before cd’ing into a new one. The stack of dirs is listed every time you invoke pushd (the “/usr/local/share ~” output you see above.)

kiwi@localhost: /usr/local/share $ pushd /
/ /usr/local/share ~

Stack is ordered left to right, latest push first. If we pop the first dir off:

kiwi@localhost: / $ popd
/usr/local/share /tmp ~
kiwi@localhost: /usr/local/share $

We’re back in the share dir. We can keep popping until there’s nothing left (throws an error):

kiwi@localhost: /usr/local/share $ popd
/tmp ~
kiwi@localhost: /tmp $ pushd /lib
/lib /tmp ~
kiwi@localhost: /lib $ popd
/tmp ~
kiwi@localhost: /tmp $ popd
~
kiwi@localhost: ~ $ popd
bash: popd: directory stack empty

Working with Long Lines

No need for more Bash shortcut cheat sheets, but here are some useful ones to help you work with long lines.

You can jump to the start & end of a line using CTRL+a & CTRL+e respectively. Example (* is the cursor):

kiwi@localhost: ~ $ echo al the ducks are swimming in the w*

and you want to fix the first word. You can hop to the beginning of the line with CTRL+a:

kiwi@localhost: ~ $ *echo al the ducks are swimming in the w

and now you can jump to the end of the misspelled word “al” using CTRL+Right twice to correct it:

kiwi@localhost: ~ $ echo all*the ducks are swimming in the w

Now ctrl+e to jump to the end of line:

kiwi@localhost: ~ $ echo all the ducks are swimming in the w*

Instead of backspacing every character, use ALT+Backspace to backspace entire words. You can also delete all or part of a line using CTRL+u combo. It deletes everything before the cursor. Likewise, CTRL+k wipes out everything after the cursor. I’ve developed a habit of using CTRL+e CTRL+k to delete lines.

Bash has a lot of ALT commands that let you move and manipulate words. ALT+l and ALT+u will make a word in front of the cursor lowercase or uppercase, for example. A neat one I don’t think I ever used is ALT+\ It pulls everything after the cursor left to the first non-whitespace character. Here’s an example, * is the cursor:

BEFORE:

$ my     spacebar is    *sticky

AFTER (ALT+\):

$ my     spacebar issticky

Avoid Retyping Commands & Arguments

ESC + . is very useful. Escape followed by a period will output the argument you sent to your last Bash command. Command calls themselves are outputted if they were invoked without any arguments (popd, ls, etc).

Example, unzipping a file and moving the archive to /tmp:

$ unzip archive-with-a-long-ambiguous-name-03092009-5960-1.2.5.zip
$ mv archive-with-a-long-ambiguous-name-03092009-5960-1.2.5.zip /tmp

In the mv command, the archive name was outputted by pressing ESC+. (full command being mv (ESC+.) /tmp) There was no need to type the long archive name twice.

The argument is taken from your bash history. You can keep invoking ESC+. to cycle back through all your recent command arguments. (history -c to clear)

Try not to forget this; You’ll naturally find plenty of uses for it.

Another way to avoid re-typing commands is CTRL+R. It will initiate a search of your command history. Begin typing, and watch Bash try to complete your command from previous ones you entered.

Command Getting Too Big? Send it to your Editor

Sometimes you begin writing what you think will be a simple command, only to realize that it has grown too complex for the command line, and you wish you were in your text editor.

First make sure your default editor is set. This is either in $EDITOR (export EDITOR=/usr/local/bin/vim) or elsewhere depending on the distro.

Use “fc” to open the last executed command in your editor:

ls -paul --sort=size
... ls output ...
fc

Now the ls line will be open in your editor. But what if you hadn’t executed the command yet? No problem. You’re sending off an email, but quickly realize that the command line isn’t ideal for everything:

echo -e "Dear Santa, \n\n\tIt has become evident that your fat ass is contributing to Global Warming, primarily due to the large quantity of coal you distribute annually. We hereby

No matter where you are on the line, hit CTRL+x, CTRL+e to invoke your editor, which now contains what you were typing on the cmd line.

I always find myself wanting to finish a command in vim, but unwilling to type the first few lines over, especially when I’m trying to write a for loop or any ugly multiline Bash code.

IMPORTANT: Whatever you type in your editor is executed automatically after you quit the editor.
Continue reading Bash Tips for Power Users

Calculate Your GPA Using this Bash Script

This Bash script is used to calculate your Grade Point Average (GPA) on the command line. Usage might not be intuitive. Please see the usage function or just run the script without passing it any arguments.

The gval function should be edited to reflect your own region or university. It has been written and tested on Bash 3.2.48.

#!/bin/sh
#
# Bash GPA calculator
#
# Isam | r0cketjump@yahoo.com | biodegradablegeek.com
# 05/21/2009 - Just another 4 AM project

function usage {
  echo -e "\nBASH GPA Calculator"
  echo
  echo -e "\tAccepts an even # of arguments in the form of C G C G C G ..."
  echo -e "\t (C = number of credits, G = grade for the course)"
  echo
  echo -e "\tExample: You got a B+ in a 4 credit course, "
  echo -e "\t         an A in a 3 credit course, etc.."
  echo
  echo -e "\tUSAGE: $0 4 B+ 3 A 3 F 3 B-"
  echo
  echo "Acceptable grades are A B C D F WU (eq to F)"
  echo
}

function calc {
  echo `echo "scale=3; $1" | bc`
}

function gval {
  grade=`echo "$1" | tr [a-z] [A-Z]`
  case $grade in
    A+ ) echo '4.3';;
    A ) echo '4';;
    A- ) echo '3.7';;

    B+ ) echo '3.3';;
    B ) echo '3.00';;
    B- ) echo '2.7';;

    C+ ) echo '2.3';;
    C ) echo '2.0';;
    C- ) echo '1.7';;

    D+ ) echo '1.3';;
    D ) echo '1.0';;
    D- ) echo '0.7';;

    F ) echo '0';;
    WF ) echo '0';;
    WU ) echo '0';;
  esac
}

# check # of arguments. is it even?
let MOD=$#%2
if [ ! $MOD -eq 0 ]; then
  usage
  exit
elif [ $# -eq 0 ]; then
  usage
  exit
fi

args=($@)
n=${#args[@]}

points=0
credits=0

for ((i=0;i<$n-1;i+=2)); do
  k=${i}

  creds=${args[$k]}
  cgrade=${args[$k+1]}

  # convert cgrade (C-) to a number
  grade=`gval $cgrade`
  pts=`calc $grade*$creds`

  echo "$creds * $cgrade ($grade) = $pts"

  points=`calc $points+$pts`
  credits=`calc $credits+$creds`
done

gpa=`calc $points/$credits`
echo "------------"
echo "Total points  = $points"
echo "Total credits = $credits"
echo "------------"
echo "** GPA (pts/crd) = $gpa"
echo "------------"

(Script uses bc as the calculator. Change that in the calc function if you need to.)

I’ll never get used to Bash’s ugly ass syntax. … esac?

Bash Script to Force an Empty Git Push

Sometimes, like when you’re testing hooks or trying to create synced remote and local repos, you’ll find yourself touching empty files just to get a git push going. This script automates this task by creating a unique temporary file, committing it, pushing, and then removing the file.

#!/bin/sh
TMP=tmp-`date +'%m%s'`
touch $TMP
git add $TMP
git commit $TMP -m '(forced push)'
git push
git rm $TMP
</pre>
Usage, assuming you named it git-force and made it executable (chmod)
<pre lang="bash">cd git-repo/
./git-force

I place this in ~/bin/ which is in my $PATH. You might want to if you use this a lot.

How to Maintain Static Sites with Git & Jekyll

Static sites in this context just means non-database driven sites. Your static site can be an elaborate PHP script or just a few markup and image files. For this I am using Jekyll – A neat Ruby gem that makes your static sites dynamic. It lets you create layouts and embed custom variables in your HTML (this is a “prototype” of the site).

Jekyll tackles all the nuisances involved in creating static pages (I used to add just enough PHP to make a layout). It works by running your prototype through some parsers and outputs plain static HTML/XML (RSS feeds) etc. It’s perfect for lightweight sites that would be impractical on WordPress, like a few static pages of information, landing pages, portfolio/resume pages, and parked domains.

Git takes care of keeping your development (local) and production (remote) environments synced. Git might be a little confusing if you’re learning it with the mindset that it works like Subversion.

I’ll update this post when the guide is done. For now, the following will assume you’re familiar with Jekyll (or at least have an empty file in the prototype directory) and git. This Bash script simplifies creating the remote git repository:

** please read through the code and make sure you know what this does, and what you’re doing. As of now, this is bias towards my own Apache/vhost setup. It’s trivial to edit for your specific needs. You’re using this at your own risk.

(direct link – repogen.sh)

#!/bin/sh
# 
# 04/01/2009 | http://biodegradablegeek.com | GPL 
# 
# You should be in site (NOT public) root (be in same dir as public/ log/ etc)
# proto/ is created and will house the jekyll prototype
# public/ will be the generated static site
# the public/ folder will be REMOVED and regenerated on every push
# 

if [ -z "$1" ]; then
  echo "Usage: ./repogen.sh domain.comn"
  exit
fi

# optional. will make it easier to copy/paste cmd to clone repo 
SSHURL="ssh.domain.com"
URL="$1"

echo "** creating tmp repo"
mkdir proto
cd proto
git init 
touch INITIAL
git add INITIAL
git commit -a -m "Initial Commit"

echo "** creating bare repo"
cd ..
git clone --bare proto proto.git
mv proto proto.old
git clone proto.git
rm -rf proto.old

echo "** generating hook"
HOOK=proto.git/hooks/post-update

mv $HOOK /tmp
echo '#!/bin/sh' &gt;&gt; $HOOK
echo '# To enable this hook, make this file executable by "chmod +x post-update".' &gt;&gt; $HOOK
echo '#exec git-update-server-info' &gt;&gt; $HOOK
echo '' &gt;&gt; $HOOK
echo '' &gt;&gt; $HOOK
echo 'URL='"$URL" &gt;&gt; $HOOK
echo 'PROTO="/home/$USER/www/$URL/proto"' &gt;&gt; $HOOK
echo 'PUBLIC="/home/$USER/www/$URL/public"' &gt;&gt; $HOOK
echo  '' &gt;&gt; $HOOK
echo 'export GIT_DIR="$PROTO/.git"' &gt;&gt; $HOOK
echo 'pushd $PROTO &gt; /dev/null' &gt;&gt; $HOOK
echo 'git pull' &gt;&gt; $HOOK
echo 'popd &gt; /dev/null' &gt;&gt; $HOOK
echo '' &gt;&gt; $HOOK
echo "echo -----------------------------" &gt;&gt; $HOOK
echo "echo '** Pushing changes to '$URL" &gt;&gt; $HOOK
echo "echo '** Moving current public to /tmp'" &gt;&gt; $HOOK
echo 'mv "$PUBLIC" "/tmp/'$URL'public-`date '+%m%d%Y'`"' &gt;&gt; $HOOK
echo 'echo "** Generating new public"' &gt;&gt; $HOOK
echo 'jekyll "$PROTO" "$PUBLIC"' &gt;&gt; $HOOK

echo "** enabling hook"
chmod a+x $HOOK 

echo "** clone repo on local machina. example:"
echo "git clone ssh://$USER@$SSHURL/~$USER/www/$SSHURL/proto.git"

Usage

Your site structure might be different. repogen.sh is made by pasting the above code in a new file, and then chmod a+x to make it executable. This should be done on the remote server.

cd www/domain.com/
ls
public/ private/ log/ cgi-bin/

./repogen.sh domain.com

Now on your local machine, clone the new repo, move your files in, and push:

git clone ssh://[username]@ssh.domain.com/~[username]/www/domain.com/proto.git
cd proto/
cat "hello, world" &gt; index.htm
git add index.htm
git commit -a -m 'first local commit'
git push

After you push your changes, the post-update hook will delete the public/ directory (the root of the site). This dir and its contents are automatically generated and will get wiped out on EVERY push. Keep this in mind. All your changes and content should reside in proto/.

The proto/ repo will pull in the new changes, and then Jekyll will be invoked to generate the updated site in public/ from the prototype.

Should you need to edit it, the post-update hook is in the bare git repo (proto.git/hooks/)

Thanks to the authors in the posts below for sharing ideas. I first read this git method on dmiessler’s site.

Resources:
dmiessler.com – using git to maintain static pages
toroid.org – using git to manage a web site
Jekyll @ GitHub
git info
more git info