Articles May 25, 2007 at 2:17 pm

Terminal Productivity Tips

There’s no doubt that the Command Line Interface can be a powerful way to control any machine. Many also call it fast, which IMHO isn’t always the case. Mundane tasks like changing directories and performing repetitive commands (as opposed to using keyboard shortcuts) and the lack of Undo/Redo can sometimes be anything but quick. Luckily with a few tricks, we can make the whole experience much more enjoyable…

Introduction

The following article is a look at what I’ve found to be the most useful hints for working on the command line in the context of OS X server. It’s probably aimed at people who work with Terminal every day (none of the totally obvious stuff is explained… hopefully) but have never had the time to look around for tips to boost their productivity.

Practically all of this is bash-specific, which seems to be the most popular modern shell out there anyway and has also been the default OS X shell since Jaguar, IIRC.

Getting around

Since paths can get really long when navigating the depths of OS X, the first thing I like to do is change the default prompt to something more compact. Typically I’m most concerned about which machine I’m on and the current directory:

$ export PS1="\H [\W] > "

We probably spend the most time typing path-names navigating different directories – oftentimes going back and forth between two – configuration and logs, application and data, home directories and backups. That’s why I was thrilled to find this extremely useful shortcut:

$ cd -

… which simply toggles you back and forth between your present and previous directory (the $PWD and $OLDPWD shell variables, to be exact). Very often I find myself working between /Library/WebServer/Documents/something and /etc/httpd/sites and so this becomes a very welcome shortcut indeed.

Just like the $PATH variable that helps us run programs without specifying their exact location, $CDPATH can really speed up getting to those “hard to reach” folders on our systems (that sounded almost like an infomercial):

$ export CDPATH=.:/Library/WebServer/Documents:/etc/httpd:/var/log/httpd:/Volumes/2TBRAID/Shared\ Items

Now if I wanted to jump to the site “webstore” (inside the default web server content directory), I can do so by simply typing:

$ cd webstore

… while located anywhere on the system. It’s a good idea to keep the current directory (“.”) as the first option in that list.

There are a lot of useful environment variables to help us working with paths. For example. to convert the current relative location to an absolute path, use the $PWD variable. Many scripts require absolute paths to work properly so the next time you want to say “directory ‘bin’ inside the current directory”, use $PWD/bin instead of “/where/ever/the/heck/i/might/be/bin”.

To have bash consider typical typos of path-names, enable this handy option:

$ shopt -s cdspell

And finally, if you ever feel the need to warp into GUI-land, don’t forget how easy it is to open the current directory in the Finder:

$ open .

Time-travel Is Easy

So you piped together some amazing command that can disable global warming and turn sea-water into fine wine and 6 hours later you find yourself needing it again. You could probably find it by pressing the up arrow enough times, but there’s a much better way. Just hit Ctrl-R and some letters that you remember the command contained. To edit the search result, press the left or right arrows. To execute the currently selected command, just hit Enter.

A really annoying situation I used to face quite often is when you’re editing a file deep in the bowels of OS X and try to save it, only to discover you don’t have write permission. So you quit nano, hit the up arrow, Option-click in the beginning of the line (you do have “Option click to position cursor” turned on in Terminal > Window Settings, right?), prefix it with sudo and try again. But there’s a much quicker way:

$ sudo !!

No, I’m not mad at sudo. Here, the two exclamation points simply mean “the previous command”. It can even be used selectively:

$ sudo !apache

… will run sudo with the previous command that started with “apache”. Simple and extremely useful. All the tokens of the previous command are available from !:0 (the program name) to !:n (the n’th argument) quite similarly to shell scripting arguments ($0 … $n). Use “!$” to reuse all the parameters of the previous command.

By default, the bash_history file is updated only when a shell exits or a Terminal window is closed. That’s why you don’t get the history of another running shell when you open a new window. To override this we have to tell bash to append to, rather than overwrite, the history list:

$ shopt -s histappend; PROMPT_COMMAND='history -a'

An important thing to note about history, especially when using a more publicly accessible terminal, is that it’s contents are always preserved so to clear it you should always do:

$ clear; history -c; echo "" > ~/.bash_history

Commands and Editing

Heredoc (or “herestring”) syntax comes in handy if you just want to quickly write a few lines to a file (or pipe them to any other program). It looks a bit weird at first, but is easily rememberable if you already understand the basic IO redirection principles. For example when writing a directive for httpd:

$ cat > mysite.conf << EOF
> <IfModule mod_alias.c>
> Redirect permanent "/" "http://backup.knownissue.org"
> </IfModule>
> Ctrl-D

How many times have you had some service tell you “syntax error in service.conf on line 114”? The next time you dive in to fix it, remember to give nano the line number as the first parameter, as in:

$ nano +114 service.conf

… which will jump you right to that line. No more frantic Ctrl-V! ๐Ÿ˜‰

While on the subject of nano – if you’ve never felt great affection for vi or emacs and like your editor to be simple, then don’t forget to let your server know that too:

$ export EDITOR=nano

My final point concerning commands is by no means a trick. It’s the bash for-loop structure, on a single line, which is just one of those things you never remember when you need it:

$ for file in *.woa ; do cp $file $file.old; done

Many programs will already accept wild-cards as arguments and thus don’t need loop structures to iterate through folders, but to all of those that don’t this can be a massive timesaver.

The Tab Is Key

The tab-key is probably the greatest thing to happen to the CLI. Here’s a few pointers to make it even more useful.

By default, you have to hit the tab key twice if auto completion hits an ambiguous path, good thing there’s a way to change that:

$ echo "set show-all-if-ambiguous on" >> ~/.inputrc

Shell variables can also be auto-completed, to see all the set variables just hit “$[Tab]”.

Finally I’d like to quickly touch on bash’s programmable completion. In addition to all the possibilities mentioned earlier, bash also supports creating completion for arbitrary commands, prime examples of this being completing arguments for any program. For example, SSH:

$ ssh [email protected][Tab]
[email protected] [email protected] [email protected] [email protected] [email protected]

There’s a project that’s adding completion features to a great number of popular programs already. The best way to understand all of this is to simply install and play with it. To do that, just copy “bash_completion” to /etc (you’ll have to change lines 35 and 36 if you want to keep it somewhere else) and then source it (. /etc/bash_completion).

The above example works by bash_completion looking into ~/.known_hosts. The bash_completion project doesn’t (yet) handle any Darwin-only commands, but it’s still well worth to install and play with.

Passwords

As sysadmins, we obviously have to type more passwords than the average user. Here’s a couple of ways to save you the trouble of quite a few keystrokes.

In addition to passwords, OpenSSH can use DSA or RSA keys to authenticate and encrypt traffic. All we need to do is add our public key to a remote machine’s list of authorised keys. We start by generating a key-pair and then copying over the public part:

$ ssh-keygen -t rsa -b 2048 -f ~/.ssh/id_rsa -P "" && scp ~/.ssh/id_rsa.pub [email protected]:

Then we log in to the remote machine using our password (for the last time!) and “notify the authorities”:

$ cat id_rsa.pub >> ~/.ssh/authorized_keys2
$ chmod 0600 !:3; rm id_rsa.pub

Now just log out and try to SSH back in. You should be welcomed without being bothered for a password.

This 2-step solution is extremely easy and significantly cuts down on keystrokes, but it’s also less than ideal because, as you noticed, we didn’t provide a pass-phrase for the key-pair. Now anyone with access to our private key file can also log into any machine it’s public counterpart has been authorised on.

To cut a long story shorter, we want to use the same approach as before, except with a password which the system will “type” for us. To make this work, we need ssh-agent (bundled with OpenSSH) and a wonderful bash script called keychain from the folks over at Gentoo Linux.

After installing keychain (I just copied it to /usr/bin), we use the same steps as before, but also provide a password for the key-pair (just leave out the -P “”), start keychain and source the generated SSH variables:

$ keychain id_rsa; source ~/.keychain/$(hostname)-sh

Now try SSH’ing to a system that has been setup to use your encrypted key. You should be able to log in without a password. As you can see, with a little bit of effort, we can finally have a system that’s both secure and convenient to use.

Aliases Galore

Command aliases are perfect for commands and arguments that you have to use over and over again. For example, a typical admin task is tailing various log files, system.log probably more than any other. So instead of typing the same command over and over again, let’s define an alias:

$ alias systail='tail -f /var/log/system.log'

Another favourite of mine are SSH aliases, for example:

$ alias ssh-kif='ssh [email protected]'

Combining that with the previous password tip makes moving around between machines a real snap. We can also use aliases to re-define existing commands. Like top which by default, lists processes by PID. Since we’re typically more interested in CPU usage, it makes sense to change the default behaviour by creating an alias:

$ alias top='top -o cpu'

There’s a ton on really useful shortcuts you can create with aliases, I just hope I’ve given a few useful examples to start creating your own.

Terminal.app Tips

OS X’s bundled terminal application provides us with some welcome shortcuts that a surprisingly large number of users are unaware of or just don’t bother remembering.

  • Connect to server (Shift-Cmd-K) allows you to browse and connect to Bonjour-enabled SSH and (S)FTP servers and is a fantastic timesaver for quickly accessing Macs on your LAN.
  • Split window – that little box in the upper right hand corner of Terminal creates a separate scroll-back buffer which is perfect for those lengthy nano sessions or stringing together configure script parameters. Occasionally, this drives nano insane (as does sometimes resizing the Terminal window) in which case the only way to get out of it seems a Ctrl-Z followed by a kill.
  • Clear (Cmd – K) is a nice alternative to “clear”. Works also inside tail.
  • Edit > Paste Escaped Text (Ctrl – Cmd – V) does exactly what it says.

And don’t forget that you can always drag files or folders from virtually any window’s document icon onto a Terminal window to insert the properly escaped absolute path of that item.

Other Tools and Closing Remarks

screen is a virtual terminal application. It allows you to run several terminal sessions (as opposed to just more copies of bash) within the same terminal window. It can save you the trouble of another SSH connection when you want to run several interactive processes at a time or just run interactive applications in the background without having to stay logged in… IRC and other tasks being the prime example, of course. ;-). Ctrl-A+D to detach from a screen (leaving all the applications in it running) and screen -r to jump back into it. More info, as always, in the man-page.

Visor is a nifty SIMBL hack that gives you instant access to Terminal.app from within any application with the push of a keyboard shortcut. It’s novelty may wear off quite quickly for some, but I thought it would be nice to mention it here nonetheless.

The next time you have to quickly FTP that encrypted DMG of SQL dumps to another server, don’t bother with ftp or some GUI FTP client. Just remember the T-switch for curl (the user – and hostname stuff is pretty obvious anyway):

$ curl -u username:password -T mydumps.dmg 'ftp://myserver.com/dumps/'

… just remember that bit about clearing the command history.

Finally, all of the bash-related tips can be saved for later use into ~/.bash_profile or, to be available to any shell account on the system, in /etc/bashrc. Since these are just regular files, they can easily be shared between several machines with the help of scp:

$ scp ~/.bash_profile [email protected]:

… would copy over the bash shortcuts, settings and other goodies I’ve set up on my local machine, to the remote server.

It’s not always easy to learn new tricks, no matter how cool they may be, however I hope that some of the pointers presented here will intrigue you enough to still try them out and maybe incorporate into your own “command-lining”. There’s probably more bash tips out there than there are Mac Pro configuration combinations at the Apple Store, and so one of the reasons for writing this article was to raise a discussion about everyone else’s tips as well. So if you have any, don’t hesitate to leave them in the comments! ๐Ÿ˜‰

A big thanks to Simon Myers for his brilliant presentation and to all other sites I learned much of this stuff from.

No Comments

  • Two other good terminal commands for navigating folders are pushd and popd. They push a path name onto, and pop a path name off of, a stack of paths you create by invoking the command.


    http://www.aaronadams.net
    My two-bit, half-assed opinion concerning whatever it is I fee like writing about.

  • I recommend Adrian Mayo’s excellent “Mac OS X Unix 101 Byte-Sized Projects”. It has all these tips mentioned here and more. It never hurts to go back to basics, and I learned something new everytime I opened this book up. It’s the best book I’ve read since starting out with UNIX for dummies 15 years ago.

    My fav tip is “control – r” in BASH shell for searching through all your previous commands. A real time saver.


    Mat X – Mac VFX SysAdmin

  • Great article, thank you. Anyone know w better way then esc-f to go forward a word?

    Adam

  • This was a great idea for an article, Here are a few other tips, top of my head but in the same vein.

    I use this bit of code in my .bashrc file to color each host I ssh in to and also to remind that I am still logged in to a sudo -s shell ( Mainly made for my classes, to “Highlight” for students when I am running commands as root ) .

    First off though I personally dont like to maintain the .bash_profile I just have it source ~/.bashrc so that additions will also be used for sub-shells ( like screen ). So you can create a file like so, which is pretty standard with *nix skeleton homes.

    # ~/.bash_profile

    
    if [ -f ~/.bashrc ]; then
        source ~/.bashrc
    fi
    

    When copied out (.bash*) to the remote servers you can maintain just one rc file but have a different color prompt at each one.

    I find this really nice as I normally have quite a few terminal windows open and dont want to reboot the wrong machine acidently.

    You can see that I am using the upper and lower case versions as I have some machines that are windows boxes (cygwin i.e. NETBIOSNAME ) and the “star” as they could be .local or .mydomain.com if they happen to be mobile or whatever.

    # ~/.bashrc

    
    declare -x HOST=$(/bin/hostname)
    case $HOST in
            MAX* | max* ) declare PS1="[@][33][33[1;32m]h[33[0m]:W]$ ";;
            MUTE* | mute* ) declare PS1="[@][33][33[1;34m]h[33[0m]:W]$ ";;
            MINI* | mini* ) declare PS1="[@][33][33[1;36m]h[33[0m]:W]$ ";;
            MONA* | mona* ) declare PS1="[@][33][33[1;33m]h[33[0m]:W]$ ";;
            IDURO* | iduro* ) declare PS1="[@][33][33[1;30m]h[33[0m]:W]$ ";;
            ICE* | ice* ) declare PS1="[@][33][33[1;35m]h[33[0m]:W]$ ";;
            * ) declare PS1="[@][33][33[1;37m]h[33[0m]:W]$ ";;
    esac
    if [  -n "$SUDO_USER" ] ; then
    declare PS1="[@][33[41m][33[1;30m]h[33[0m]:W]# "
    fi
    

    For help with decoding the colors and other options google is your friend ( worth the read ),
    but just to get the colors rocking, notice the differences between the strings, “1;30m” or “1;35m” and for the root (sudo -s) string background color “41m” and compare them to the output of the script below (author unknown) :

    
    #!/bin/bash
    #
    #   This file echoes a bunch of color codes to the 
    #   terminal to demonstrate what's available.  Each 
    #   line is the color code of one forground color,
    #   out of 17 (default + 16 escapes), followed by a 
    #   test use of that color on all nine background 
    #   colors (default + 8 escapes).
    #
    
    T='gYw'   # The test text
    
    echo -e "n                 40m     41m     42m     43m
         44m     45m     46m     47m";
    
    for FGs in '    m' '   1m' '  30m' '1;30m' '  31m' '1;31m' '  32m' 
               '1;32m' '  33m' '1;33m' '  34m' '1;34m' '  35m' '1;35m' 
               '  36m' '1;36m' '  37m' '1;37m';
      do FG=${FGs// /}
      echo -en " $FGs 33[$FG  $T  "
      for BG in 40m 41m 42m 43m 44m 45m 46m 47m;
        do echo -en "$EINS 33[$FG33[$BG  $T  33[0m";
      done
      echo;
    done
    echo
    

    If you would like to have ls show color as well you can use:

    
    echo "declare -x CLICOLOR=true" >> ~/.bashrc
    

    To get around Tab completions default dehaviour to not complete ( and correct )

    the “wrong” case you can also use the following option. show “cd /lib[TAB]” becomes “cd /Library”

    
    echo "set completion-ignore-case On" >> ~/.inputrc
    

    screen is defiantly a handy utility to get comfortable with for instance to run softwareupdate (-i Install -a All) with, incase you get disconnected during download or install.

    
    screen sudo softwareupdate -i -a
    

    My favorite feature through is splitting screens for like log viewing and troubleshooting

    Or building a command in one and reading the man in the other.
    Open up a new screen, just by typing

    
    screen 
    

    if you want to in the future get rid of the startup message just run

    
    echo "startup_message off" >>~/.screenrc
    

    but anyway.. Now that you are use the keyboard command “Control + a” then immediately type “Shift + s” (split)

    You should see a bar appear between in the middle of your terminal window, now hit “Control +a” again then “Tab” and you should see the cursor move the bottom have of the split, now again “Control +a” then “c” (create). You should see two prompts now, you can start a command like

     top -du 

    and use the sequence “Control + a” then “Tab” to switch back and forth. You can make many splits by using “Control + a” then “shift + s” and again use the ctrl+a->tab and ctrl+a->c to make new shells.

    Thats it for right now, I am sure I will think of a few other later too, and will post to this thread
    Happy typing
    -Zack

  • Awesome comments! Thank you muchly!


    “The future is here. It’s just not evenly distributed yet.” –William Gibson

Leave a reply

You must be logged in to post a comment.