A A

Bash Tips for Power Users

Wed, Jun 10, 2009

Linux, List, Misc, Productivity, Tips

Every Geek site needs an obligatory Bash Tips post

Copy Files Securely Between Two Machines

I used to always forget the syntax for this, until I realized that the syntax is exactly like the standard cp command. In fact, you can copy files like you normally would using scp, on your local machine. The following are equivalent:

$ cp file file.orig
$ scp file file.orig

Where they differ is, scp lets you copy files over a network, through SSH. Here’s an example:

$ scp contents.txt silver@ssh.domain.com:/tmp

This will copy local file contents.txt to /tmp on the remote machine ssh.domain.com, as user silver. Here are some more examples:

$ scp draft.pdf ssh.domain.com:

(copy draft.pdf to my home dir on remote machine. username is implied to be the same locally and remotely.)

$ scp swine.jpg rex@ssh.domain.com

(read: This will copy swine.jpg to local machine as a file named rex@ssh.domain.com. To make it go remote, append a : to the address, like above)

scp supports, among other things, compression (-C) and recursive copying of directories (-r).

$ scp -rC code/ ssh.domain.com:/archive/code_02032009

Trying to copy to a directory you don’t have permission to (/usr etc) will fail.

Don’t Get Lost Jumping To and Fro Between Directories

You can use cd - to jump to the previous (NOT parent) dir. For example:

kiwi@localhost: ~ $ cd /usr/local/share
kiwi@localhost: /usr/local/share $ cd -
/home/kiwi
kiwi@localhost: ~ $ cd -
/usr/local/share
kiwi@localhost: /usr/local/share $

Another way is using pushd/popd – A Last In First Out (LIFO) stack of dirs.

kiwi@localhost: ~ $ pushd /usr/local/share/
/usr/local/share ~

pushd is like cd but keeps note of the current dir before cd’ing into a new one. The stack of dirs is listed every time you invoke pushd (the “/usr/local/share ~” output you see above.)

kiwi@localhost: /usr/local/share $ pushd /
/ /usr/local/share ~

Stack is ordered left to right, latest push first. If we pop the first dir off:

kiwi@localhost: / $ popd
/usr/local/share /tmp ~
kiwi@localhost: /usr/local/share $

We’re back in the share dir. We can keep popping until there’s nothing left (throws an error):

kiwi@localhost: /usr/local/share $ popd
/tmp ~
kiwi@localhost: /tmp $ pushd /lib
/lib /tmp ~
kiwi@localhost: /lib $ popd
/tmp ~
kiwi@localhost: /tmp $ popd
~
kiwi@localhost: ~ $ popd
bash: popd: directory stack empty

Working with Long Lines

No need for more Bash shortcut cheat sheets, but here are some useful ones to help you work with long lines.

You can jump to the start & end of a line using CTRL+a & CTRL+e respectively. Example (* is the cursor):

kiwi@localhost: ~ $ echo al the ducks are swimming in the w*

and you want to fix the first word. You can hop to the beginning of the line with CTRL+a:

kiwi@localhost: ~ $ *echo al the ducks are swimming in the w

and now you can jump to the end of the misspelled word “al” using CTRL+Right twice to correct it:

kiwi@localhost: ~ $ echo all*the ducks are swimming in the w

Now ctrl+e to jump to the end of line:

kiwi@localhost: ~ $ echo all the ducks are swimming in the w*

Instead of backspacing every character, use ALT+Backspace to backspace entire words. You can also delete all or part of a line using CTRL+u combo. It deletes everything before the cursor. Likewise, CTRL+k wipes out everything after the cursor. I’ve developed a habit of using CTRL+e CTRL+k to delete lines.

Bash has a lot of ALT commands that let you move and manipulate words. ALT+l and ALT+u will make a word in front of the cursor lowercase or uppercase, for example. A neat one I don’t think I ever used is ALT+\ It pulls everything after the cursor left to the first non-whitespace character. Here’s an example, * is the cursor:

BEFORE:

$ my     spacebar is    *sticky

AFTER (ALT+\):

$ my     spacebar issticky

Avoid Retyping Commands & Arguments

ESC + . is very useful. Escape followed by a period will output the argument you sent to your last Bash command. Command calls themselves are outputted if they were invoked without any arguments (popd, ls, etc).

Example, unzipping a file and moving the archive to /tmp:

$ unzip archive-with-a-long-ambiguous-name-03092009-5960-1.2.5.zip
$ mv archive-with-a-long-ambiguous-name-03092009-5960-1.2.5.zip /tmp

In the mv command, the archive name was outputted by pressing ESC+. (full command being mv (ESC+.) /tmp) There was no need to type the long archive name twice.

The argument is taken from your bash history. You can keep invoking ESC+. to cycle back through all your recent command arguments. (history -c to clear)

Try not to forget this; You’ll naturally find plenty of uses for it.

Another way to avoid re-typing commands is CTRL+R. It will initiate a search of your command history. Begin typing, and watch Bash try to complete your command from previous ones you entered.

Command Getting Too Big? Send it to your Editor

Sometimes you begin writing what you think will be a simple command, only to realize that it has grown too complex for the command line, and you wish you were in your text editor.

First make sure your default editor is set. This is either in $EDITOR (export EDITOR=/usr/local/bin/vim) or elsewhere depending on the distro.

Use “fc” to open the last executed command in your editor:

ls -paul --sort=size
... ls output ...
fc

Now the ls line will be open in your editor. But what if you hadn’t executed the command yet? No problem. You’re sending off an email, but quickly realize that the command line isn’t ideal for everything:

echo -e "Dear Santa, \n\n\tIt has become evident that your fat ass is contributing to Global Warming, primarily due to the large quantity of coal you distribute annually. We hereby

No matter where you are on the line, hit CTRL+x, CTRL+e to invoke your editor, which now contains what you were typing on the cmd line.

I always find myself wanting to finish a command in vim, but unwilling to type the first few lines over, especially when I’m trying to write a for loop or any ugly multiline Bash code.

IMPORTANT: Whatever you type in your editor is executed automatically after you quit the editor.

Multiple Commands on a Single Line

There are a number of ways to piece together commands (||, pipes, etc), depending on your need, but sometimes, you just commands executed consecutively. You can use ; or &&.

semicolon (;) vs AND (&&): The semicolon will run through each command consecutively, whereas && is a little smarter, and will not continue if a command does not end successfully (return 0 – you can check the return value of the last app ran with echo $?).

&& is generally safer. i.e., ./configure && make (&& sudo make install)

AND

$ cp bogus && echo "** copied"
cp: missing destination file operand after `bogus'
Try `cp --help' for more information.

SEMICOLON;

cp bogus; echo "** copied... or did I? tun tun tunnn!"
cp: missing destination file operand after `bogus'
Try `cp --help' for more information.
** copied... or did I? tun tun tunnn!

Convert between DOS and UNIX ASCII files

Sometimes you get a text file that has weird ^M characters in it. These are due to a difference in how Unix and Windows systems end lines. You can convert between these formats using unix2dos or todos and dos2unix or fromdos.

$ mkdir /tmp/rcfl
$ cd /tmp/rcfl
$ echo -e "Justa Lonely\nASCII File" > out
$ file out
out: ASCII text
$ todos out
$ file out
out: ASCII text, with CRLF line terminators
$ vim out # notice [dos] flag in status bar, quit :q!
$ fromdos out
$ file out
out: ASCII text

Background Processes

Have a little more control over your apps.

Stop Right Thurr
When a program is running in the foreground, you no longer have access to the command line. An example is ‘tail -f’ or ‘ruby script/server’

You can have a running process pause for a sec with CTRL+z.
Do your dirty work and then bring the app back to the foreground with fg.
To list the processes you have paused, use jobs

$ tail -f useful.log
00:00:50 User did something that was log-worthy
00:00:56 User did something that was log-worthy
00:00:57 User did something that was log-worthy

# (press CTRL+z)
[1]+  Stopped                 tail -f useful.log

$ echo "look ma, I can type"
look ma, I can type

$ fg
tail -f useful.log

# (press CTRL+z)
[1]+  Stopped                 tail -f useful.log

$ tail -f blah.tmp

# (press CTRL+z)
[2]+  Stopped                 tail -f blah.tmp

$ jobs
[1]-  Stopped                 tail -f useful.log
[2]+  Stopped                 tail -f blah.tmp
$ fg 1
(process [2] continues)

In the Background

You can have a process start in the background by appending to it a &.
and bring this to the foreground using fg [#].
As before, jobs will list background processes, but with status Running instead of Stopped.

Programs running in the background will still output to stdout, which means they’ll make the shell ugly. So if you plan on using them, think about redirecting the output.

Bash Redirection

Some things are mentioned on nearly every ‘bash tips’ page — like redirecting output. Here are the basics. We’re concerned with 2 I/O streams: STDOUT and STDERR. STDOUT has a value of 1, and it is the screen. If a program writes to stdout, that text is shown in the console. Errors are sent through a different stream, stderr, which has a value of 2. Value 0 is stdin, used for user input. The technical details aren’t important. Just remember that 1 is screen and 2 is error.

There’s a number of ways to redirect output:

$ echo "asdfasdf" 1> /tmp/asdf.txt # overwrite existing file
$ echo "asdfasdf" > /tmp/asdf.txt # (same, 1 is default, optional)
$
$ echo "32452345" >> /tmp/asdf.txt # append to end of existing file*
$ echo "wash the dishes" > /dev/null # just ignore output
$ echo "wash the dishes" 2> /dev/null # just ignore errors
$
$ more 1> /dev/null # error still shown
$ more 2> /dev/null # nothing shown
$ more &> /dev/null # nothing shown
$
$ more 2>| /dev/null # silence errors
$ more >| /tmp/more.txt # save output to file
$
$ more 1> /tmp/more.txt 2>&1 # redirect stdout to file and redirect stderr to stdout (same file)
$ more 2> /tmp/more.txt 1>&2 # have stdout follow stderr to file

To redirect output to a file & screen, use tee

$ echo "dont forget the milk" | tee /tmp/toforget.txt

More a more extensive guide on redirection, see Bash IO Redirection or some of the External Links below.

Art of teh Alias

I use a lot of aliases. Here are a few:

# I use these a lot. Can also have aptupdate, aptremove etc...
alias aptinstall='sudo apt-get install'
alias aptsearch='apt-cache search'
alias suvim='sudo vim'

# Aliasing command names. To use original commands, you'd need to specify absolute path.
alias a2restart='sudo apache2ctl restart'
alias gem='sudo gem'
alias checkinstall='sudo checkinstall'

# I used these to workaround the infamous FF memory leak (ugly)
# alias swapoff='sudo swapoff'
# alias swapon='sudo swapon'

# Going places. This + ssh keypair
alias macbookshell='ssh 192.168.1.17'
alias workshell='ssh meh@ssh.domain.com'

# Pretty output. (--group-directories-first might not work on your system).
alias lsf='ls -hAlF --group-directories-first --color=always --time-style=+" %m/%d/%y %I:%M %p "'

# List only directories
alias lsd='ls -d */'

# I'm a measly human!
alias free="free -m"

# 'gimme x' is equivalent to 'sudo chown me.me x'
# alias gimme="ME6=`whoami` && sudo \"chown $ME6.$ME6\""

# Usage: nullminate bloated-file.log
alias nullminate="cat /dev/null > "

# Search contents of an entire dir. Usage: scan "PESKY_VARIABLE ?=" project-123/
alias scan="grep -Rin --color"

These should go in a ~/.bash_aliases file and invoked from within your user conf (.bashrc?)

External links (related):

Tags: , , ,

2 Comments For This Post

  1. chisiyuan Says:

    Very helpful! Thanks!

  2. Uncle Bob Says:

    Another useful one when working with bash is CTRL+w

    CTRL+w will delete the word left of the cursor.

    e.g. $ I love bash!*

    Since the cursor is at the end of ‘bash!’, after pressing CTRL+w you will have

    $ I love *

Leave a Reply