How to find empty files in UNIX?

There are multiple ways to find whether a file is empty or not. Please see few examples below,

volcano@volcano-laptop:~/shellscript$ ls -ltr | grep '\<0\>'
-rw-r--r-- 1 volcano volcano    0 2011-03-12 15:41 sujay
-rw-r--r-- 1 volcano volcano    0 2011-03-12 15:42 jsk
-rw-r--r-- 1 volcano volcano    0 2011-03-12 15:45 a
-rw-r--r-- 1 volcano volcano    0 2011-03-12 15:46 b
-rw-r--r-- 1 volcano volcano    0 2011-04-10 22:30 emptyfile

volcano@volcano-laptop:~/shellscript$ ls -ltr | awk ' $5==0 {print}'
-rw-r--r-- 1 volcano volcano    0 2011-03-12 15:41 sujay
-rw-r--r-- 1 volcano volcano    0 2011-03-12 15:42 jsk
-rw-r--r-- 1 volcano volcano    0 2011-03-12 15:45 a
-rw-r--r-- 1 volcano volcano    0 2011-03-12 15:46 b
-rw-r--r-- 1 volcano volcano    0 2011-04-10 22:30 emptyfile

volcano@volcano-laptop:~/shellscript$ for fname in `ls`;do if [ ! -s $fname ];then ls -l $fname;fi;done
-rw-r--r-- 1 volcano volcano 0 2011-03-12 15:45 a
-rw-r--r-- 1 volcano volcano 0 2011-03-12 15:46 b
-rw-r--r-- 1 volcano volcano 0 2011-04-10 22:30 emptyfile
-rw-r--r-- 1 volcano volcano 0 2011-03-12 15:42 jsk
-rw-r--r-- 1 volcano volcano 0 2011-03-12 15:41 sujay

volcano@volcano-laptop:~/shellscript$ find . -maxdepth 1 -size 0 -ls
 58438    0 -rw-r--r--   1 volcano  volcano         0 Mar 12 15:42 ./jsk
 58434    0 -rw-r--r--   1 volcano  volcano         0 Mar 12 15:46 ./b
 58437    0 -rw-r--r--   1 volcano  volcano         0 Mar 12 15:41 ./sujay
 58211    0 -rw-r--r--   1 volcano  volcano         0 Mar 12 15:45 ./a
 58234    0 -rw-r--r--   1 volcano  volcano         0 Apr 10 22:30 ./emptyfile

How to remove empty lines from a file in UNIX or Linx

volcano@volcano-laptop:~/shellscript/empty$ cat testempty
hi first line

above line is empty
hi dude

above line is empty
volcano@volcano-laptop:~/shellscript/empty$ sed '/^$/d' testempty > testwithoutempty
volcano@volcano-laptop:~/shellscript/empty$ cat testwithoutempty
hi first line
above line is empty
hi dude
above line is empty
volcano@volcano-laptop:~/shellscript/empty$

volcano@volcano-laptop:~/shellscript/empty$ grep -v '^$' testempty > without
volcano@volcano-laptop:~/shellscript/empty$ cat without
hi first line
above line is empty
hi dude
above line is empty

One exception case is available for empty line removal. That is line will be empty but it will have space or tab characters. To handle that, we can use the below command format.


grep -v '^[]*]$'  oldfile > newfile

volcano@volcano-laptop:~/shellscript/empty$ cat test
hi first line

above line is empty
hi dude

above line is empty
   
volcano@volcano-laptop:~/shellscript/empty$ grep -v '^[         ]*$' test > newtest
volcano@volcano-laptop:~/shellscript/empty$ cat newtest
hi first line
above line is empty
hi dude
above line is empty
volcano@volcano-laptop:~/shellscript/empty$

volcano@volcano-laptop:~/shellscript/empty$ sed '/^$/d' test
hi first line
above line is empty
hi dude
above line is empty
   
volcano@volcano-laptop:~/shellscript/empty$ sed '/^[    ]*$/d' test
hi first line
above line is empty
hi dude
above line is empty
volcano@volcano-laptop:~/shellscript/empty$

How to remove CONTROL-M characters in UNIX or Linux

Control-m characters will get appended to a file when a file is transferred from windows to UNIX machine. There are multiple ways it can be removed.


Using vi editor:
:%s/^M//g

Using col command:
cat filename | col -b > newfilename

Using sed command:
sed 's/^M//g' filename > newfilename

Using dos2unix comand:
dos2unix filename newfilename

Examples:

volcano@volcano-laptop:~/shellscript/ctrl$ cat -v jsk
hi^M
jsk^M
volcano@volcano-laptop:~/shellscript/ctrl$ cat jsk | col -b > jsk.new
volcano@volcano-laptop:~/shellscript/ctrl$ cat -v jsk.new
hi
jsk
volcano@volcano-laptop:~/shellscript/ctrl$ sed 's/^M//g' jsk > jsk.new2
volcano@volcano-laptop:~/shellscript/ctrl$ cat -v jsk.new2
hi
jsk

Note: Hold the control key and then press v and m to get the control-m character