These pages about different languages / apis / best practices were mostly jotted down quckily and rarely corrected afterwards.
The languages / apis / best practices may have changed over time (e.g. the facebook api being a prime example), so what was documented as a good way to do something at the time might be outdated when you read it (some pages here are over 15 years old).
Just as a reminder.

Shell Scripts for linux / unix / macosx

Useful linux shell scripts

Multiple Search & Replace (recursive)

 if [ $# -lt 3 ] ; then
   echo -e "Wrong number of parameters."
   echo -e "Usage:"
   echo -e " $0 'filepattern' 'search' 'replace'"
   exit 1
 find ./ -type f -name $1 -exec sed -i "s/$2/$3/" {} \;

Backup of MySql database, suitable for cron job

This will email the database to the person specified. Suitable to throw into a cron job and let it run once a week.

echo "Subject: MySql backup: `date`" > meckfil ; /bin/mysqldump --add-drop-table -h `/bin/mysqlbackups database | head -1` -uusername -ppassword database >> meckfil ; mail rocky@somedomainatsomeplacesomwhere.com < meckfil; rm meckfil

Prepend an ls entry with something (e.g. the dirname)

meck='theDirectory';ls -1 $meck| while read line; do echo "$meck/$line"; done

Random things in KSH-scripts

Send a mail

subject="the output from adams.txt"
mailx -s "${subject}" dilbert@somewhereelsereally.com wally@somewhereelsereally.com < adams.txt 2> /dev/null

Sub routines




_sendFile arg1 arg2

Command line arguments

while getopts htf:r:m: arguments
   case $arguments in
        h) echo $USAGE;exit;;
        f) CALVIN=$OPTARG;;
        m) ADAMS=$OPTARG;;
        r) DILBERT=$OPTARG;;
        t) WALLY='adams';;

Paranoia check return code for each command

cleanup() {
    echo "Cleanup: Removing $TMPDIR"
    rm -r $TMPDIR

checkreturn() {
    if [ "0" -ne "$1" ]; then
        echo "Failed: $1"
        exit $1

zip -r shellscripts_collection.zip
checkreturn $?

Find out which operating system

dilbert@comic >uname -s
dilbert@comic >rlogin adams
dilbert@adams >uname -s
dilbert@adams >

storeme - push directories to file

Script I used to jump to different directories and add them to a list (cannot recall why I needed to do this, but whatever):
dilbert@wally >cat ~/bin/storeme
echo $PWD >> /tmp/storeme_list.txt
dilbert@wally >pwd
dilbert@wally >storeme
dilbert@wally >cd /tmp
dilbert@wally >storeme
dilbert@wally >cat /tmp/storeme_list.txt
dilbert@wally >

logging script calls

When first trying to sort out the mess with all hundreds of ksh-scripts at a company that I were, there were loads of calls from script to other scripts that called other scripts and so on. To help out in which order and how the scripts called each other I added these lines to the scripts.
It was very helpful for tracking down how the scripts were called, and also a nice way to make sure that I had replaced all scripts afterwards (i.e. when no log files were generated all scripts were obsolete and could be replaced)
# ...

filename=/tmp/theFiles_stat_`date +"%Y_%m_%d_%H"`.log
echo "$0 $1" >> $filename

# ....

redirect output to file without buffering

script -q /dev/null python somescript.py | tee -a mylog.log

killing process after a certain amount of time

Based on cpu time only

Make a wrapper and call
ulomit -t XX
before calling your command, with XX being the time before kill in seconds.
Note: this is just counting cpu time!

timeout from coreutils

This is the way to go rather than your own hack, see some alternatives below if you cannot install timeout for some reasons
timeout 5m mycommand myarg

making sleep and then killing it

make a wrapper that kills it after 15 second (note that the sleep will still go on even if the command finish earlier)
( mycommand $@ ) & sleep 15 ; kill $!

making sleep and then killing it, and kill sleep process if command finishes

not ideal, note that the pid for the sleep is written to a file to keep track of it.
( mycommand $@ ) &

( (sleep 900 & echo $! >&3 )  3>/tmp/vspid; kill $PID_MYCMD) &

# just so the pid get a chance to be written
sleep 2



This one starts for example an application called "dilbert", and will kill it again after 8 seconds.
  $2 &
  sleep $TIMEOUT | ( read nothing; kill $cmd_pid 2>/dev/null;);
  wait $cmd_pid
  kill -ALRM $sleep_pid 2>/dev/null


_exec $TIMEOUT dilbert

echo "I just killed Dilbert."

Makefile for different OS

        @make -f Makefile.`uname` all

        @make -f Makefile.`uname` clean

        @make -f Makefile.`uname` install
and create corresponding Makefiles, e.g. Makefile.HP-UX

A sample .profile for KSH

This is the .profile I use on KSH, the interesting parts are the function mycd and the mapping of the UP arrow.

The .profile

# history
alias h="fc -l"

alias mroe=more
alias moer=more
alias "cd.."="cd .."

# to have the path echoed instead of in the prompt
function mycd {         # function to change working directory
        cd $* > /dev/null
	echo $PWD
        return 0;
alias cd=mycd

alias e="emacs -nw"

alias ls="ls -F"
alias sl=ls
alias "ls-l"="ls -l"
alias "ls-al"="ls -al"
alias ll='ls -lp|more'
alias la='ls -al|more'
alias lt='ls -lt|more'

#create files that are readable but not writeable
umask 022

export PATH=$PATH:/home/userid/bin

#Set the prompt

set -o emacs

# History on the Up arrow
alias __A=`echo ""`

# paths for ProC
export ORACLE_HOME=/oracle806
export PATH="$PATH:/oracle806/bin"

# To make sure that @ is not treated as a kill signal (that would cause problems when using SqlPlus)
stty kill ^U

Calculate in bash

echo "scale=2;1000 * ( 2 * 60 + 52) + 96 * 10"|bc

Delete files older than three days find /some/dir/filematch* -type f -mtime +3 -exec rm {} \;

Sed search replace beginning of line, example where filename contains itemids, and we want to create sql statements deleting those items in the db

sed 's/^\(.*\)/delete from my_table where objectid=;/' filename Move random files from one dir to another shuf -n [Number of files to move] -e [PATH to the files to be moved] | xargs -i mv {} [PATH to the dest] Random lines from a file shuf -n N input > output Split file into two parts with random lines #First randomize the file shuf -n 1000000 "{trainfile}" > tmp wc -l tmp Repalce newline to comma and remove last char cat somefile | tr " " "," |sed 's/.$//'

More programming related pages

Workflow: Release process
Workflow: Bug tracking
Teambox (Redbooth) - Mantis connector
Design Patterns
Git & Github
Go / Golang
CVS backup script
Distribution process
Installation script
Java Server Faces
Perl tips
Perl links
Python / py2app
Shell scripts
Facebook / Opengraph
PHP developer notes
Redbooth API through php
Website optimization
jqTableKit demo
Javascript / html / css links