Hints and tips on bash

These are some helpful hints on various topics.

apache bash cmd database java javascript linux mod_rewrite mssql mysql oracle os perl php regex rsync ssh svn unix web_design windows

 

Search and replace in bash scripting

To get the value of variable X with all occurances of 'yy' replaced by 'zz' use the following expression:

${X//yy/zz}

For example:

ORIG="foo to the bar"
REPLACED=${ORIG//bar/baz}
echo $REPLACED

The result is:
foo to the baz

Argument list too long, or how to use xargs

Suppose you want to remove all jpeg files from several directories. You use
rm -f `find . -name *.jpg -type f`
and get "Argument list too long". What you have to do is to use xargs to automatically split the long list to several shorter and pass them to rm:
find . -name *.jpg -type f | xargs rm -f
This way several rm commands are executed, but your files are finally gone.

Creating files with fixed length and random content

Handy command to create file with fixed length and random content is  dd. Some examples:

File test1.bin, 10k size, containing only zero bytes: 
 dd bs=1024 count=10 if=/dev/zero of=test1.bin

File test2.bin, 100k size and random content:
 dd bs=1024 count=100 if=/dev/random of=test2.bin

Above example is rather slow, especially for large files, so here's another way to generate pseudo-random file:
 dd bs=1024 count=100 if=/dev/hda of=test2.bin skip=1000
This will copy 100k from the hard drive 1М after the beginning.

Getting the last word of bash string

To get the last word of a bash string use awk '{print $NF}':

echo "one two three" | awk '{print $NF}'

returns

three

One-liner for transfering SSH public key to remote host

cat ~/.ssh/id_rsa.pub | ssh hostname "cat >> ~/.ssh/authorized_keys"

Rsync over ssh with non-default port

If you use rsync over ssh to a server running on port different from 22, for example 555, use the following format:

rsync -a -e "ssh -p 555" rsyncuser@remoteserver:/data/to/sync /archive/

More on rsync

Prefix each output line with timestamp

Imagine the case: you were writing a program or a script that outputs some information in the console. Just when everything is almost complete you realize that you actually need timestamp on each message output. For example you need to know how long it takes to backup each of your users' home dirs. The solution is to go through all output commands and wrap them in a function that appends timestamps. Believe it or not, there's a lazier way to do this - it's a simple bash one-liner:
 
> ./your_program | while read line; do stamp=`date`; echo "$stamp $line"; done
 
You can try it with
 
> du -sc /home | while read line; do stamp=`date`; echo "$stamp $line"; done

Iterating filenames with spaces in bash


Usually when you write some bash script and need to iterate through some files, you use constructions like those:

for file in /dir/*.c
do
    echo $file
done

or

for file in `find /dir -name *.c`
do
    echo $file
done

Both approaches fail when it comes to files with spaces, like "file with space.c". In this case you iterate over all three parts "file", "with" and "space.c" instead the file itself. To cope with this use:

find /dir -name *.c | while read file
do
    echo $file
done

Add easily "sleep" functionality to your Linux box with installing zero new packages

If you want to be able to turn of your computer after a number of minutes you have a plenty of programs to choose from. Here's an easy way to do this with just using bash and at. Add the following to your .bashrc:
 
sleeper() {
    echo 'init 0' | sudo at now + $1 minutes
}
 
The just use it - set you desktop to poweroff in 50 minutes:
 
> sleeper 50

Use pm-suspend or pm-hibernate instead of init 0 to suspend or hibernate.

Getting full directory path in bash and cmd prompt

To get the absolute path for the current or any other realtively addressed folder, for example the parent, use the following constructs:

bash

parent_dir=`cd .. && pwd`

cmd

set current_dir=%CD%
cd ..
set parent_dir=%CD%
cd %current_dir%

Recursively rename all files in a directory to another extension using bash


The following script renames all files in the current folder and its subfolders with extension .jad to .java

for file in `find . -type f -name *.jad`
do
    mv $file `dirname $file`/`basename $file .jad`.java
done

Find files not owned by user

Here's how to find files in user foo home folder that are not owned by foo

find /home/foo ! -user foo

Find the jar that contains given Java class

If you have a folder containing several jars and you want to quickly find which one contains given class, use the following script:

findjar.sh:

dir=$1
class=$2
if [ $# -ne 2 ]
then
  echo "Usage: `basename $0` <directory> <class>"
  exit 1
fi

for file in `find $dir -name *.jar`
do
unzip -t $file | grep $class && echo $file
done

Example: find the jar containing MyClass in the folder /my/jars/folder:

findjar.sh /my/jars/folder MyClass