CRON
There are four common causes for cron job commands to behave differently compared to commands typed directly into an interactive shell:
- Cron provides a limited environment, e.g., a minimal
$PATH, and other expected variables missing. - Cron invokes
/bin/shby default, whereas you may be using some other shell interactively. - Cron treats the
%character specially (it is turned into a newline in the command). precede with "\" - The command may behave differently because it doesn't have a terminal available.
FIND
- find + multiple exec + pipe + redirect:
find . -name *.AET -exec ksh -c 'grep ^"\-\- [0-9]\{2\}:[0-9]\{2\}:[0-9]\{2\}.*(PeopleCode)" {} |uniq|wc -l >> xxx' \; -exec ksh -c 'grep ^"\-\- [0-9]\{2\}:[0-9]\{2\}:[0-9]\{2\}.*(PeopleCode)" {} | wc -l >> yyy' \;
source: https://stackoverflow.com/questions/5119946/find-exec-with-multiple-commands
find accepts multiple -exec portions to the command. For example:
find . -name "*.txt" -exec echo {} \; -exec grep banana {} \;
Note that in this case the second command will only run if the first one returns successfully, as mentioned by @Caleb. If you want both commands to run regardless of their success or failure, you could use this construct:
find . -name "*.txt" \( -exec echo {} \; -o -exec true \; \) -exec grep banana {} \;- find -modified > 3 days : find . -mtime +3
find only in current dir, exclude sub-dir: find . \( ! -name . -prune \) \( -name "xyz.dat*" -o -name "abc*" \) find in all sub-dirs: find . -type d -print
find only in 1st level sub-dirs: find . \( ! -name . -prune \) -type d -print find only in 1st level sub-dirs, folder names matching: find . \( ! -name . -prune \) \( -type d -a \( -name "5AA00" -o -name "5X[0789]00" \) \)
find files > or <: data-blogger-escaped-font="">
find . -type f -size +(-)100000c
more - http://www.unix.com/answers-frequently-asked-questions/13863-advanced-complex-uses-find-command.html
On RH Linux 5:
- find . -type f -cmin -60 # creation time
- find . -type f -amin -60 # access time
- find . -type f -mmin -60 # modify time
- find . -maxdepth 5 -mindepth 3 # sub-folders
find . -newermt "2011-01-01" ! -newermt "2011-12-31" # between dates
-daystart -mtime 0 means today -daystart -mtime +0 means before today.* Executing awk on each file found with find, then redirecting the result to a new filename
find -type f -iname "*.txt" -exec awk '{print $1, $2 >(FILENAME "-new")}' {} +
1. grep -P "\t" foo.txt
2. grep "$(printf '\t')" foo.txt
3. awk '/\t/'
4. sed -n '/\t/p'
5. grep $'\t' foo.txt
grep - exlude multiple patterns
1. grep -v PSORA| grep -v "Store Field:"
2. grep -Fv -e PSORA -e "Store Field:"
3. grep -Ev "PSORA|Store Field:"
Grab file records by line #
awk '{if (NR>=11895 && NR<=11905) print $0}' xxx > yyy
awk 'NR==11895, NR==11905' xxx > yyy
awk '{if (NR==11895 || NR==11905) print $0}' xxx > yyy
Get E
64 ways to get Environment variables: http://rosettacode.org/wiki/Environment_variables
awk 'BEGIN{print "HOME:"ENVIRON["HOME"],"USER:"ENVIRON["USER"]}'
print N records after some pattern:
groups - find user groupsfind groups a user belongs to: groups user1 user2...
find users in a group: lsgroup -f group
(file: /etc/group)
IFS - change field delimiter
see current value:
$echo "$IFS" | od -b
0000000 040 011 012 012
0000004
$echo "$IFS" | od -c
0000000 \t \n \n
0000004
changing IFS to ","
IFS_SAV=$IFS
IFS=,
line='a,,b,c,d'
for x in $line
do
echo $x
done
ls - get file date time stamp
ls -l = modification time
ls -lu = access timels -l = creation time
ls -ld *dirname* = show folder info, no sub-folder, no files
ls -ld */ = show folder info, no sub-folder, no files
du -sh <folder>= total space used by folder
2. grep "$(printf '\t')" foo.txt
3. awk '/\t/'
4. sed -n '/\t/p'
5. grep $'\t' foo.txt
grep - exlude multiple patterns
1. grep -v PSORA| grep -v "Store Field:"
2. grep -Fv -e PSORA -e "Store Field:"
3. grep -Ev "PSORA|Store Field:"
Grab file records by line #
awk '{if (NR>=11895 && NR<=11905) print $0}' xxx > yyy
awk 'NR==11895, NR==11905' xxx > yyy
awk '{if (NR==11895 || NR==11905) print $0}' xxx > yyy
Get E
64 ways to get Environment variables: http://rosettacode.org/wiki/Environment_variables
awk 'BEGIN{print "HOME:"ENVIRON["HOME"],"USER:"ENVIRON["USER"]}'
print N records after some pattern:
awk 'c&&c--;/pattern/{c=N}' file
groups - find user groupsfind groups a user belongs to: groups user1 user2...
find users in a group: lsgroup -f group
(file: /etc/group)
IFS - change field delimiter
see current value:
$echo "$IFS" | od -b
0000000 040 011 012 012
0000004
$echo "$IFS" | od -c
0000000 \t \n \n
0000004
changing IFS to ","
IFS_SAV=$IFS
IFS=,
line='a,,b,c,d'
for x in $line
do
echo $x
done
ls - get file date time stamp
ls -l = modification time
ls -lu = access timels -l = creation time
ls -ld *dirname* = show folder info, no sub-folder, no files
ls -ld */ = show folder info, no sub-folder, no files
du -sh <folder>= total space used by folder
case insensitive matching: https://unix.stackexchange.com/questions/48770/how-to-match-case-insensitive-patterns-with-ls
ksh93: ls -l ~(i:a*)
port -
AIX - This file shows the usage of ports -
netstat -plunt
netstat -tap
(port # may be replaced by its symbolic name in
if so, use
nc -z ; echo $?
proctree -
proctree (-a) PIDprocfiles -
procfiles (-n) PID
ps -
for user xxx
>> ps -fuxxx
>> ps -fl -uxxx
all jobs by xxx sort by cpu
>> ps -fuxxx | sort +3
all jobs related to PID ??????
>> ps -fL ??????
display environment of another process:
ps eww PID | tr ' ' '\n' | grep ORACLE_SID
check process priority:
ps -lef
change priority:
nice/renice
***** look at process ENV *****
==>ps -ef | grep db2sysc
instv9 24060 24059 0 16:33 pts/10 00:00:00 db2sysc 0
instv9 24114 23951 0 16:43 pts/10 00:00:00 grep db2sysc
==> ps ewww 24060
PID TTY STAT TIME COMMAND
24060 pts/10 S 0:00 db2sysc 0 HOME=/home/instv9 KRB5CCNAME=
FILE:/tmp/krb5cc_p23950 PWD=/home/instv9 DB2INSTANCE=instv9 DB2INSTDEF=instv9 DB2LPORT=0 DB2NODE=0
HOSTNAME=xxxxxxx.com LANG=en_US.UTF-8 USER=instv9 CLASSPATH=/home/instv9/sqllib/java/db2java.zip:
/home/instv9/sqllib/java/db2jcc.jar:/home/instv9/sqllib/java/sqlj.zip:/home/instv9/sqllib/function:/home/instv9/
ps wwee
read -
(break fields on a record/line into variables)
>>move 1st field of each line to end of line (xx=1st field, yy=rest of fields)
while read -r xx yy
do
print printf "%s %s/n" $yy $xx
done <>> To read a line and split it into fields, and use "enter ur name:" as a prompt
read word1?"enter ur name: " word2 word3 word4....
>> read more than 1 values into a variable (reak up the list by using for)
$ read x y z; echo $x; echo $y; echo $z
one two three four
$one
$two
$three four
>> show the oldest file
: ls -tr| read xxx; echo $xxx
: ls -tr |&
read -p $xxx
echo $xxx
>> read from a file into variables
exec 3<>
sed -
sed -n '
/APPSTVCH.BI120-3/ {
N
/\n.*1/ p
}' APPSTVCH_703295.log
(great site with a # of one-liners-explained http://www.catonmat.net/)
sort -
>> sort by column 4-14 of 1st field:
sort -k 1.4,1.14
>> sort by numeric, 3rd field, delimited by ':'
sort -n -t ':' +2 /etc/passwd
>> second field as the sort key
sort -k 2,2 infile # new style
sort +1 -2 infile # old style
>> reverse sort, contents of infile1 and infile2, placing the output in outfile and using the second character of the second field as the sort key (assuming that the first character of the second field is the field separator):
sort -r -o outfile -k 2.2,2.2 infile1 infile2 # new style key definition used
sort -r -o outfile +1.1 -1.2 infile1 infile2 # old style key definition used
>> sorts the contents of infile1 and infile2 using the second non-blank character of the second field as the sort key:
sort -k 2.2b,2.2b infile1 infile2
sort +1.1b -1.2b infile1 infile2
>> prints the passwd(4) file (user database) sorted by the numeric user ID (the third colon-separated field):
sort -t ':' -k 3,3n /etc/passwd
sort -t ':' +2 -3n /etc/passwd
sort -n -t ':' -k 3,3 /etc/passwd
>> imitate uniq using sort: either of the following commands prints the lines of the already sorted file infile, suppressing all but one occurrence of lines having the same third field:
sort -um -k 3.1,3.0 infile
sort -um +2.0 -3.0 infile
The -n option informs sort to compare the specified field as numbers, not ASCII characters. The r option reverses the order of the sort.
sort -t: +5 -6 +0 -1 /etc/passwd # The output is now sorted by field 6, then by field 1 if necessary.
touch -
- change DTTM stamp
03:04:55 a.m. on Jan 2, 1985
>>touch -t 198501020304.55 program.c
use the time stamp of another file
>>touch -r file1 program.c
avoid creating a new file, enter
>>touch -c program
touch: cannot touch xxx: No space left on device
check:
* spaces - df
* inodes - df -i
* deleted - lsof -nP | grep 'deleted'
viminfo - https://stackoverflow.com/questions/2816719/clear-certain-criteria-from-viminfo-file
to manually edit entries -
port -
AIX - This file shows the usage of ports -
lsof -i :port#
(port # may be replaced by its symbolic name in
/etc/services, for ex:commplex-link 5001/tcp #
commplex-link 5001/udp # if so, use
lsof -i :<#> or netstat -an | grep # )nc -z
proctree -
proctree (-a) PIDprocfiles -
procfiles (-n) PID
ps -
for user xxx
>> ps -fuxxx
>> ps -fl -uxxx
all jobs by xxx sort by cpu
>> ps -fuxxx | sort +3
all jobs related to PID ??????
>> ps -fL ??????
display environment of another process:
ps eww PID | tr ' ' '\n' | grep ORACLE_SID
check process priority:
ps -eo pid,state,nice,args | less -S
ps -o user,cpu,pcpu,pid,etime,time,comm -p 19234892 ps -lef
change priority:
nice/renice
***** look at process ENV *****
| e | Displays the environment as well as the parameters to the command, up to a limit of 80 characters. |
| ew | Wraps the display from the e flag one extra line. |
| eww | Wraps the display from the e flag and displays the ENV list until the flag reaches the LINE_MAX value. |
| ewww | Wraps the display from the e flag and displays the ENV list until the flag reaches the INT_MAX value. |
==>ps -ef | grep db2sysc
instv9 24060 24059 0 16:33 pts/10 00:00:00 db2sysc 0
instv9 24114 23951 0 16:43 pts/10 00:00:00 grep db2sysc
==> ps ewww 24060
PID TTY STAT TIME COMMAND
24060 pts/10 S 0:00 db2sysc 0 HOME=/home/instv9 KRB5CCNAME=
FILE:/tmp/krb5cc_p23950 PWD=/home/instv9 DB2INSTANCE=instv9 DB2INSTDEF=instv9 DB2LPORT=0 DB2NODE=0
HOSTNAME=xxxxxxx.com LANG=en_US.UTF-8 USER=instv9 CLASSPATH=/home/instv9/sqllib/java/db2java.zip:
/home/instv9/sqllib/java/db2jcc.jar:/home/instv9/sqllib/java/sqlj.zip:/home/instv9/sqllib/function:/home/instv9/
ps wwee
read -
(break fields on a record/line into variables)
>>move 1st field of each line to end of line (xx=1st field, yy=rest of fields)
while read -r xx yy
do
print printf "%s %s/n" $yy $xx
done <>> To read a line and split it into fields, and use "enter ur name:" as a prompt
read word1?"enter ur name: " word2 word3 word4....
>> read more than 1 values into a variable (reak up the list by using for)
$ read x y z; echo $x; echo $y; echo $z
one two three four
$one
$two
$three four
>> show the oldest file
: ls -tr| read xxx; echo $xxx
: ls -tr |&
read -p $xxx
echo $xxx
>> read from a file into variables
exec 3<>
sed -
- print a line with a pattern & the next line
sed -n '
/APPSTVCH.BI120-3/ {
N
/\n.*1/ p
}' APPSTVCH_703295.log
- Handy one-liners for SED: http://www.catonmat.net/blog/sed-one-liners-explained-part-two/
(great site with a # of one-liners-explained http://www.catonmat.net/)
# print 1 line of context before and after regexp, with line number
# indicating where the regexp occurred
sed -n -e '/regexp/{=;x;1!p;g;$!N;p;D;}' -e h
- Find text after tag/pattern:
- Find WORD after pattern: (https://unix.stackexchange.com/questions/207171/get-value-after-specific-word)
- awk '{for (I=1;I<=NF;I++) if ($I == "FROM") {print $(I+1)};}' file
- grep -oP "FROM\s+\K\w+" file
-o ==> option for printing only the matching part of the line
-P ==> use perl-regexp
\K ==> do not print that comes before \K (zero-width look-behind assertion)
\w ==> match word characters
3. awk -F 'FROM' '{print $2}' file | awk '{print $1}' -- works if only 1 WORD per line
sort -
>> sort by column 4-14 of 1st field:
sort -k 1.4,1.14
>> sort by numeric, 3rd field, delimited by ':'
sort -n -t ':' +2 /etc/passwd
>> second field as the sort key
sort -k 2,2 infile # new style
sort +1 -2 infile # old style
>> reverse sort, contents of infile1 and infile2, placing the output in outfile and using the second character of the second field as the sort key (assuming that the first character of the second field is the field separator):
sort -r -o outfile -k 2.2,2.2 infile1 infile2 # new style key definition used
sort -r -o outfile +1.1 -1.2 infile1 infile2 # old style key definition used
>> sorts the contents of infile1 and infile2 using the second non-blank character of the second field as the sort key:
sort -k 2.2b,2.2b infile1 infile2
sort +1.1b -1.2b infile1 infile2
>> prints the passwd(4) file (user database) sorted by the numeric user ID (the third colon-separated field):
sort -t ':' -k 3,3n /etc/passwd
sort -t ':' +2 -3n /etc/passwd
sort -n -t ':' -k 3,3 /etc/passwd
>> imitate uniq using sort: either of the following commands prints the lines of the already sorted file infile, suppressing all but one occurrence of lines having the same third field:
sort -um -k 3.1,3.0 infile
sort -um +2.0 -3.0 infile
The -n option informs sort to compare the specified field as numbers, not ASCII characters. The r option reverses the order of the sort.
sort -t: +5 -6 +0 -1 /etc/passwd # The output is now sorted by field 6, then by field 1 if necessary.
- using TAB as delimiter, sort on 5th field: sort -t "`/bin/echo '\t'" +6 xx
sort -t"," -k1,1 -k2n,2 file
>> use a range of hexdecimal escapes in sequare brackets?
LC_ALL=C sed 's/[\xE0-\xEF]/_/g'touch -
- change DTTM stamp
03:04:55 a.m. on Jan 2, 1985
>>touch -t 198501020304.55 program.c
use the time stamp of another file
>>touch -r file1 program.c
avoid creating a new file, enter
>>touch -c program
touch: cannot touch xxx: No space left on device
check:
* spaces - df
* inodes - df -i
* deleted - lsof -nP | grep 'deleted'
viminfo - https://stackoverflow.com/questions/2816719/clear-certain-criteria-from-viminfo-file
to manually edit entries -
:set viminfo= to turn off the auto-saving of info to the .viminfo file. If you don't do this, Vim will overwrite all your changes when you quit,