AFP548

Terminal Productivity Tips

filipp

There’s no doubt that the Command Line Interface can be a powerful way to control any machine. Many also call it fast, which IMHO isn’t always the case. Mundane tasks like changing directories and performing repetitive commands (as opposed to using keyboard shortcuts) and the lack of Undo/Redo can sometimes be anything but quick. Luckily with a few tricks, we can make the whole experience much more enjoyable…

Introduction

The following article is a look at what I’ve found to be the most useful hints for working on the command line in the context of OS X server. It’s probably aimed at people who work with Terminal every day (none of the totally obvious stuff is explained… hopefully) but have never had the time to look around for tips to boost their productivity.

Practically all of this is bash-specific, which seems to be the most popular modern shell out there anyway and has also been the default OS X shell since Jaguar, IIRC.

Getting around

Since paths can get really long when navigating the depths of OS X, the first thing I like to do is change the default prompt to something more compact. Typically I’m most concerned about which machine I’m on and the current directory:

$ export PS1="\H [\W] > "

We probably spend the most time typing path-names navigating different directories – oftentimes going back and forth between two – configuration and logs, application and data, home directories and backups. That’s why I was thrilled to find this extremely useful shortcut:

$ cd -

… which simply toggles you back and forth between your present and previous directory (the $PWD and $OLDPWD shell variables, to be exact). Very often I find myself working between /Library/WebServer/Documents/something and /etc/httpd/sites and so this becomes a very welcome shortcut indeed.

Just like the $PATH variable that helps us run programs without specifying their exact location, $CDPATH can really speed up getting to those “hard to reach” folders on our systems (that sounded almost like an infomercial):

$ export CDPATH=.:/Library/WebServer/Documents:/etc/httpd:/var/log/httpd:/Volumes/2TBRAID/Shared\ Items

Now if I wanted to jump to the site “webstore” (inside the default web server content directory), I can do so by simply typing:

$ cd webstore

… while located anywhere on the system. It’s a good idea to keep the current directory (“.”) as the first option in that list.

There are a lot of useful environment variables to help us working with paths. For example. to convert the current relative location to an absolute path, use the $PWD variable. Many scripts require absolute paths to work properly so the next time you want to say “directory ‘bin’ inside the current directory”, use $PWD/bin instead of “/where/ever/the/heck/i/might/be/bin”.

To have bash consider typical typos of path-names, enable this handy option:

$ shopt -s cdspell

And finally, if you ever feel the need to warp into GUI-land, don’t forget how easy it is to open the current directory in the Finder:

$ open .

Time-travel Is Easy

So you piped together some amazing command that can disable global warming and turn sea-water into fine wine and 6 hours later you find yourself needing it again. You could probably find it by pressing the up arrow enough times, but there’s a much better way. Just hit Ctrl-R and some letters that you remember the command contained. To edit the search result, press the left or right arrows. To execute the currently selected command, just hit Enter.

A really annoying situation I used to face quite often is when you’re editing a file deep in the bowels of OS X and try to save it, only to discover you don’t have write permission. So you quit nano, hit the up arrow, Option-click in the beginning of the line (you do have “Option click to position cursor” turned on in Terminal > Window Settings, right?), prefix it with sudo and try again. But there’s a much quicker way:

$ sudo !!

No, I’m not mad at sudo. Here, the two exclamation points simply mean “the previous command”. It can even be used selectively:

$ sudo !apache

… will run sudo with the previous command that started with “apache”. Simple and extremely useful. All the tokens of the previous command are available from !:0 (the program name) to !:n (the n’th argument) quite similarly to shell scripting arguments ($0 … $n). Use “!$” to reuse all the parameters of the previous command.

By default, the bash_history file is updated only when a shell exits or a Terminal window is closed. That’s why you don’t get the history of another running shell when you open a new window. To override this we have to tell bash to append to, rather than overwrite, the history list:

$ shopt -s histappend; PROMPT_COMMAND='history -a'

An important thing to note about history, especially when using a more publicly accessible terminal, is that it’s contents are always preserved so to clear it you should always do:

$ clear; history -c; echo "" > ~/.bash_history

Commands and Editing

Heredoc (or “herestring”) syntax comes in handy if you just want to quickly write a few lines to a file (or pipe them to any other program). It looks a bit weird at first, but is easily rememberable if you already understand the basic IO redirection principles. For example when writing a directive for httpd:

$ cat > mysite.conf << EOF

<IfModule mod_alias.c>
Redirect permanent "/" "http://backup.knownissue.org"
</IfModule>
Ctrl-D

How many times have you had some service tell you “syntax error in service.conf on line 114”? The next time you dive in to fix it, remember to give nano the line number as the first parameter, as in:

$ nano +114 service.conf

… which will jump you right to that line. No more frantic Ctrl-V! 😉

While on the subject of nano – if you’ve never felt great affection for vi or emacs and like your editor to be simple, then don’t forget to let your server know that too:

$ export EDITOR=nano

My final point concerning commands is by no means a trick. It’s the bash for-loop structure, on a single line, which is just one of those things you never remember when you need it:

$ for file in *.woa ; do cp $file $file.old; done

Many programs will already accept wild-cards as arguments and thus don’t need loop structures to iterate through folders, but to all of those that don’t this can be a massive timesaver.

The Tab Is Key

The tab-key is probably the greatest thing to happen to the CLI. Here’s a few pointers to make it even more useful.

By default, you have to hit the tab key twice if auto completion hits an ambiguous path, good thing there’s a way to change that:

$ echo "set show-all-if-ambiguous on" >> ~/.inputrc

Shell variables can also be auto-completed, to see all the set variables just hit “$[Tab]”.

Finally I’d like to quickly touch on bash’s programmable completion. In addition to all the possibilities mentioned earlier, bash also supports creating completion for arbitrary commands, prime examples of this being completing arguments for any program. For example, SSH:

$ ssh filipp@192[Tab]
filipp@192.168.0.10 filipp@192.168.0.106 filipp@192.168.0.11 filipp@192.168.0.12 filipp@192.168.1.111

There’s a project that’s adding completion features to a great number of popular programs already. The best way to understand all of this is to simply install and play with it. To do that, just copy “bash_completion” to /etc (you’ll have to change lines 35 and 36 if you want to keep it somewhere else) and then source it (. /etc/bash_completion).

The above example works by bash_completion looking into ~/.known_hosts. The bash_completion project doesn’t (yet) handle any Darwin-only commands, but it’s still well worth to install and play with.

Passwords

As sysadmins, we obviously have to type more passwords than the average user. Here’s a couple of ways to save you the trouble of quite a few keystrokes.

In addition to passwords, OpenSSH can use DSA or RSA keys to authenticate and encrypt traffic. All we need to do is add our public key to a remote machine’s list of authorised keys. We start by generating a key-pair and then copying over the public part:

$ ssh-keygen -t rsa -b 2048 -f ~/.ssh/id_rsa -P "" && scp ~/.ssh/id_rsa.pub filipp@fry.knownissue.org:

Then we log in to the remote machine using our password (for the last time!) and “notify the authorities”:

$ cat id_rsa.pub >> ~/.ssh/authorized_keys2
$ chmod 0600 !:3; rm id_rsa.pub

Now just log out and try to SSH back in. You should be welcomed without being bothered for a password.

This 2-step solution is extremely easy and significantly cuts down on keystrokes, but it’s also less than ideal because, as you noticed, we didn’t provide a pass-phrase for the key-pair. Now anyone with access to our private key file can also log into any machine it’s public counterpart has been authorised on.

To cut a long story shorter, we want to use the same approach as before, except with a password which the system will “type” for us. To make this work, we need ssh-agent (bundled with OpenSSH) and a wonderful bash script called keychain from the folks over at Gentoo Linux.

After installing keychain (I just copied it to /usr/bin), we use the same steps as before, but also provide a password for the key-pair (just leave out the -P “”), start keychain and source the generated SSH variables:

$ keychain id_rsa; source ~/.keychain/$(hostname)-sh

Now try SSH’ing to a system that has been setup to use your encrypted key. You should be able to log in without a password. As you can see, with a little bit of effort, we can finally have a system that’s both secure and convenient to use.

Aliases Galore

Command aliases are perfect for commands and arguments that you have to use over and over again. For example, a typical admin task is tailing various log files, system.log probably more than any other. So instead of typing the same command over and over again, let’s define an alias:

$ alias systail='tail -f /var/log/system.log'

Another favourite of mine are SSH aliases, for example:

$ alias ssh-kif='ssh administrator@kif.knownissue.org'

Combining that with the previous password tip makes moving around between machines a real snap. We can also use aliases to re-define existing commands. Like top which by default, lists processes by PID. Since we’re typically more interested in CPU usage, it makes sense to change the default behaviour by creating an alias:

$ alias top='top -o cpu'

There’s a ton on really useful shortcuts you can create with aliases, I just hope I’ve given a few useful examples to start creating your own.

Terminal.app Tips

OS X’s bundled terminal application provides us with some welcome shortcuts that a surprisingly large number of users are unaware of or just don’t bother remembering.

And don’t forget that you can always drag files or folders from virtually any window’s document icon onto a Terminal window to insert the properly escaped absolute path of that item.

Other Tools and Closing Remarks

screen is a virtual terminal application. It allows you to run several terminal sessions (as opposed to just more copies of bash) within the same terminal window. It can save you the trouble of another SSH connection when you want to run several interactive processes at a time or just run interactive applications in the background without having to stay logged in… IRC and other tasks being the prime example, of course. ;-). Ctrl-A+D to detach from a screen (leaving all the applications in it running) and screen -r to jump back into it. More info, as always, in the man-page.

Visor is a nifty SIMBL hack that gives you instant access to Terminal.app from within any application with the push of a keyboard shortcut. It’s novelty may wear off quite quickly for some, but I thought it would be nice to mention it here nonetheless.

The next time you have to quickly FTP that encrypted DMG of SQL dumps to another server, don’t bother with ftp or some GUI FTP client. Just remember the T-switch for curl (the user – and hostname stuff is pretty obvious anyway):

$ curl -u username:password -T mydumps.dmg 'ftp://myserver.com/dumps/'

… just remember that bit about clearing the command history.

Finally, all of the bash-related tips can be saved for later use into ~/.bash_profile or, to be available to any shell account on the system, in /etc/bashrc. Since these are just regular files, they can easily be shared between several machines with the help of scp:

$ scp ~/.bash_profile filipp@fry.knownissue.org:

… would copy over the bash shortcuts, settings and other goodies I’ve set up on my local machine, to the remote server.

It’s not always easy to learn new tricks, no matter how cool they may be, however I hope that some of the pointers presented here will intrigue you enough to still try them out and maybe incorporate into your own “command-lining”. There’s probably more bash tips out there than there are Mac Pro configuration combinations at the Apple Store, and so one of the reasons for writing this article was to raise a discussion about everyone else’s tips as well. So if you have any, don’t hesitate to leave them in the comments! 😉

A big thanks to Simon Myers for his brilliant presentation and to all other sites I learned much of this stuff from.

filipp

More Posts - Website

Exit mobile version