Tuesday, November 17, 2009

one .profile to rule them all

UPDATED POST/VERSION HERE (11/2012)

================ORIGINAL POST BELOW================

I got sick of having to try to copy and make major modifications for each environment that I was in. As a consultant I sometimes have to deal with ridiculous policy driven constraints as to what software I can or cannot use. Sometimes it's simply ignorance (like when the CTO of a company said that I could only script in a "...standard shell like BASH..." nothing obscure like KSH) but whatever the case it's a pain when you've grown accustom to something as personal as your shell settings. Now I have a solution.

The shell is often taken for granted by many a unix user. Some (dare I say most) don't even think about the shell they're using until they try a feature that they've seen somewhere and realize that it doesn't work. Others have extreme bias toward a family of shells basically C(sh) derived shells (Csh, TCsh, Zsh??, etc) vs bourne-like shells (Ksh, BAsh, Zsh??, Ash, DAsh, etc). Still others want a specific version of a particular shell. Because so much functionality has cross-pollenated the shell landscape it can be rather vexing to discover that one feature you've come to expect form your shell isn't available in the shell you're currently stuck using, or maybe has a different syntax or is otherwise mysteriously incongruent with your habits.

Well I can't solve all of your problems with shells. What I can do however is make shell initialization easier (for myself and hopefully for you as well) with a new hefty .profile/.shrc file. Each shell has it's own rules on how it uses profile and rc files. In the past there were things that you'd absolutely want to have only in one and not the other. These days things are different and in many cases it's okay to have the same file meet both needs. That is, unless your shell double invokes your initialization script(s); then you're just wasting resources.

My solution came about because I wanted uniform functionality across the various platforms I use (Darwin/OS X, FreeBSD, OpenBSD, Solaris, HP-UX, and various flavors of Linux) as well as what ever shells I may be forced to use . I prefer ZSH and/or KSH93, but am often stuck with BASH, DASH, ASH, or PDKSH. Rarely am I lucky enough to have MKSH. So I created the following:
#Title: N/A
#Author:    G. Clifford Williams
#Purpose:   used as a .shrc or .profile this script initializes interactive
#           shell sessions with the specified configuration determined by
#           Operating System and/or shell implementation. In particular this
#           script is for shells that implement the POSIX
#           This Part

#-----------------------------------------------------------------------------#
#------------------------------------INIT-------------------------------------#
#-----------------------------------------------------------------------------#
    #This section checks to see whether the file has been read already. If so
    #it exits (using the return command) with a nice message. It works by
    #setting a variable named for the pid of the current shell ($$). If the
    #variable is not empty the the file has been processed before.
    #this makes one file suitable for both .provile and .shrc use
[ "$(eval "echo \${my_${$}}")" = "processed" ] && \
        { echo "already read for my_${$}"; return 1;}

eval "my_${$}=processed"

#-----------------------------------------------------------------------------#
#-------------------------------RUN MODE CHECK--------------------------------#
#-----------------------------------------------------------------------------#
case "$-" in
    #This section checks for the 'interactive mode' flag in the '$-' variable
    #By checking the my_INTERACTIVE variable you can set chunks of code to
    #be conditional on the manner in which the shell was invoked

    *i* )
            my_INTERACTIVE="yes"
            ;;
    * )
            my_INTERACTIVE="no"
            ;;
esac

#-----------------------------------------------------------------------------#
#------------------------------------FUNCTIONS--------------------------------#
#-----------------------------------------------------------------------------#
my_lecho(){
    [ -n my_SILENT ] || echo "$(date +%H:%M:%S)|$@"
}

my_pathadd(){
    my_tmp1=$1
    shift
    case $my_tmp1 in
        LUA_PATH)
            my_OFS=";"
            ;;
        *)
            my_OFS=":"
            ;;
    esac

    for PATH_add in $@; do
        if eval "[ -n \"\${$my_tmp1}\" ]" ; then
            if [ -d ${PATH_add} ] ; then
                eval "$my_tmp1=\"\${$my_tmp1}${my_OFS}${PATH_add}\""
            else
                echo "path ($PATH_add) not valid"
            fi
        else
            eval "$my_tmp1=$PATH_add"
        fi
    done
}

my_cleanpath(){
    #function to set a very basic PATH
    PATH=/bin:/usr/bin:/sbin:/usr/sbin
}

my_getshell(){
    ps -p $$ |\
        awk '/sh/ {
            for (i=1;i<=NF;i++){                 if (match($i,/[[:alnum:]]*sh/)){                     sub(/\)/,"",$i);                     print substr($i,RSTART);                     exit;                 }             }         }' } #-----------------------------------------------------------------------------# #-------------------Universal/Generic Settings--------------------------------# #-----------------------------------------------------------------------------# #------Get our SHELL-----# [ -n $my_SHELL ] && my_SHELL=$(my_getshell) #-----Get our OS-----# my_OS=$(uname)  #----Get our username----# #my_usrname=$(whoami) my_USERNAME=$(id -u) #------Set EDITOR(s)-----# if { which vim 2> /dev/null  1> /dev/null ;}; then
    EDITOR=vim
else
    EDITOR=vi
fi
FCEDIT=$EDITOR
HISTEDIT=$EDITOR
export EDITOR
export FCEDIT
export HISTEDIT
#------Set EDITOR-----#
if { which less 2> /dev/null 1> /dev/null;}; then
    PAGER=less
else
    PAGER=more
fi
export PAGER

my_FULLHOSTNAME=$(hostname)
my_HOST=${my_FULLHOSTNAME%%.*}
my_DOMAIN=${my_FULLHOSTNAME#*.}
#HISTFILE=${HOME}/.sh_history
my_NEWLINE="
"
#SSH AGENT STUFF
#sagent.sh -T && . ~/.sagent_info || { sagent.sh -s && . ~/.sagent_info ; }

case ${my_OS:-unset} in
    Darwin )
        #-------OS X Specifics-------#
        my_pathadd PATH /usr/pkg/bin
        # MacPorts Installer addition on 2009-01-19_at_01:36:04: adding an appropriate PATH variable for use with MacPorts.
        export PATH=/opt/local/bin:/opt/local/sbin:$PATH
        # Finished adapting your PATH environment variable for use with MacPorts.
        # MacPorts Installer addition on 2009-01-19_at_01:36:04: adding an appropriate MANPATH variable for use with MacPorts.
        export MANPATH=/opt/local/share/man:$MANPATH:/usr/pkg/man
        # Finished adapting your MANPATH environment variable for use with MacPorts.
        export LC_CTYPE=en_US.UTF-8
        ;;
    FreeBSD )
        #-------FreeBSD Specifics-------#
        BLOCKSIZE=K
        export BLOCKSIZE
        my_cleanpath
        my_pathadd PATH /usr/local/bin /usr/local/sbin
        ;;
    Linux )
        #-------Linux Specifics-------#
        my_cleanpath
        my_pathadd PATH /usr/local/bin /usr/local/sbin
        ;;
    CYGWIN_NT-5.1 )
        #-------CygWin Specifics-------#
        CYGWIN=tty ; export CYGWIN
        ;;
esac

case ${my_SHELL:-unset} in
    zsh )
        #--------Z SHELL--------#
        my_lecho "initializing ZSH"
        PATH=$PATH:${HOME}/scripts
        # Lines configured by zsh-newuser-install
        HISTFILE=~/.zsh_history
        HISTSIZE=2500
        SAVEHIST=1000000
        setopt appendhistory notify
        bindkey -v
        # End of lines configured by zsh-newuser-install
        # The following lines were added by compinstall
        zstyle :compinstall filename '/cygdrive/c/.zshrc'
        autoload -Uz compinit
        compinit
        # End of lines added by compinstall
        PS1="[%n@%m:%/>${my_NEWLINE}%# "
        ENV=${HOME}/.zshrc
        export ENV
        ;;
    bash )
        #--------Bourne Again SHELL--------#
        my_lecho "initializing BASH"
        set -o vi #vi mode editing
        set -b #immediate background job reporting
        set -B #brace expansion
        BASH_ENV=${HOME}/.bashrc
        export BASH_ENV
        #source the BASH_ENV if it's readable
        [ -r ${BASH_ENV} ] && . ${BASH_ENV}
        HISTFILE=${HOME}/.bash_history
        HISTSIZE=2500
        HISTFILESIZE=100000
        PS1="[\u@\h:\w>\n\$ "
        ;;
    *ksh )
        #--------Korn SHELL--------#
        my_lecho "initializing KSH (or something pretending to be it)"
        set -o vi #vi mode
        set -o bgnice #nice background processes
        set -b #immediate background job reporting
        ENV=${HOME}/.kshrc
        export ENV
        HISTFILE=${HOME}/.ksh_history
        HISTSIZE=2500
        HISTFILESIZE=100000
        PS1='$(whoami)@$(hostname -s):$(pwd)> '
        case $(id -u) in
            0 ) PS1="${PS1}${my_NEWLINE}# ";;
            * ) PS1="${PS1}${my_NEWLINE}$ ";;
        esac
        ;;
    * )
        #--------GENERIC SHELL--------#
        my_lecho "initializing unknown shell"
        set -o vi
        HISTFILE=${HOME}/.sh_history
        HISTSIZE=2500
        HISTFILESIZE=100000
        ENV=${HOME}/.shrc
        export ENV
        #PS1='$(whoami)@$(hostname -s):$(pwd)>'
        PS1="$my_USERNAME@$my_HOST> "
        ;;
esac

#-------After all is said and done-------#
my_pathadd PATH ~/bin ~/scripts ~/.bin

#-------Domain specific RC-------#
[ -r ${HOME}/.shrc_${my_DOMAIN} ]  && .  ${HOME}/.shrc_${my_DOMAIN}
#-------HOST specific RC-------#
[ -r ${HOME}/.shrc_${my_HOST} ]  && .  ${HOME}/.shrc_${my_HOST}

I tried to remove most of the customizations that are specific to what I do and might not be of use to other users.

Some of the cool things it does:

  • You might notice is that there is built-in invocation checking to help prevent multiple sourcing of the file. In other words if your shell automatically reads both .profile and .shrc when it starts, there is a facility in the above to make sure that it's read only once in cases where it's used as both your .shrc and .profile.

  • There's a handy function to add paths to *PATH variables. It actually checks to see whether the given path(s) exist and will only add valid directories to the path variable given.

  • There's a function to find out what shell you're running ($SHELL won't always give you what you expect)

  • There's an interactivity check to determine whether the shell was invoked interactively or via a script. This is handy for many reasons.

  • There's no non-standard code here. I've only used POSIX facilities here. The only non-shell piece of the whole thing is where i call awk to parse the output of ps -p $$ in my_getshell(). The use of AWK there is generic enough that it should work on all platforms.

I'll probably put together a todo list as I think of other features to add. I'm currently using this as my .profile/.bash_profile/.zsh_profile and .shrc/.kshrc/.bashrc/.zshrc on several machines.

UPDATE: I've had the above profile/shrc available online for a while now... you can track my changes to it and other "dot-files" at http://git.secution.com/cgit/dotfiles

Saturday, November 7, 2009

Shell tricks: eval helps make dynamic scripting rock.

If you write shell scripts of any significant size, eval is a very good command to understand (well). It can make some code simpler, other code obsolete, and yet other code it makes possible that would not be otherwise.

What is eval?


According to the 2004 IEEE Std. 1003.1:

  • eval constructs commands by concatenating arguments.


The example given on the site is pretty simple but not very revealing.

What can I do with eval?


In practice eval lets you (better) write programs that write programs, or at least parts of programs. One common use of eval is to create variables with names relative to their execution environment at runtime. I won't go into all the reasons that this might be beneficial but some good ones are: tying the process to a user id it prevent file clobbering, process accounting based on parent process id, to avoid creating arrays or other complex data structures  (good if you want your scripts to be portable and conform to the POSIX standard).

Enough theory! How do I use eval?


Okay here's an example of how to use eval. Sometimes in a script you want to be able to name a variable dynamically. The following function does just that:
tackon(){
#simple function to tack arg2 ($2) on to the end of
#variable X (arg1)
error_msg="usage: tackon "
eval "${1:?${error_msg}}=\"\${$1} ${2:?${error_msg}}\""
}

This function is named tackon because it takes two arguments and appends arg2 to the end of (the value) of arg1. Here's an example of how it works:
gcw@gcwmbp:/Users/gcw>
$ students="Alex Mark Mike"
gcw@gcwmbp:/Users/gcw>
$ tackon students "Al Tim Steve"
gcw@gcwmbp:/Users/gcw>
$ echo $students
Alex Mark Mike Al Tim Steve

To understand what's going on you first have to know that arguments to the function are assigned to two variables named $1 and $2. When tackon() is called it takes whatever string is passed in as $1 and turns it into a variable. The diagnostics in the above version may be somewhat confusing so here's the function again stripped down to the bare essentials:
tackon(){
eval "${1}=\"\${$1} ${2}\""
}

Now we can more easily see how this works. Everywhere that we have $1 and ${1} it gets evaluated to the first argument. If we called our example, where we called tackon with "students" as the first argument it becomes "students=\"${students} ${2}\"". The quotes and other special characters may neeed to be escaped (based on what behavior you want).

From this point on there's no magic. Eval executes the interpolated command and you benefit from the heavy lifting of a very nifty command. There are many other cool ways to use eval and maybe I'll go over some of them in the future but for now I'll leave you to fiddle around with this.