Unix Integer & Float calculations

Source: https://unix.stackexchange.com/questions/40786/how-to-do-integer-float-calculations-in-bash-or-other-languages-frameworks

Q:  How to do integer & float calculations, in bash or other languages/frameworks?

$ printf %.10f\\n "$((10**9 * 20/7))e-9"   # many shells. Not mksh.
$ echo "$((20.0/7))"                       # (ksh93/zsh/yash, not bash)
$ awk "BEGIN {print (20+5)/2}"
$ zcalc
$ bc <<< 20+5/2
$ bc <<< "scale=4; (20+5)/2"
$ dc <<< "4 k 20 5 + 2 / p"
$ expr 20 + 5
$ calc 2 + 4
$ node -pe 20+5/2  # Uses the power of JavaScript, e.g. : node -pe 20+5/Math.PI
$ echo 20 5 2 / + p | dc 
$ echo 4 k 20 5 2 / + p | dc 
$ perl -E "say 20+5/2"
$ python -c "print 20+5/2"
$ python -c "print 20+5/2.0"
$ clisp -x "(+ 2 2)"
$ lua -e "print(20+5/2)"
$ php -r 'echo 20+5/2;'
$ ruby -e 'p 20+5/2'
$ ruby -e 'p 20+5/2.0'
$ guile -c '(display (+ 20 (/ 5 2)))'
$ guile -c '(display (+ 20 (/ 5 2.0)))'
$ slsh -e 'printf("%f",20+5/2)'
$ slsh -e 'printf("%f",20+5/2.0)'
$ tclsh <<< 'puts [expr 20+5/2]'
$ tclsh <<< 'puts [expr 20+5/2.0]'
$ sqlite3 <<< 'select 20+5/2;'
$ sqlite3 <<< 'select 20+5/2.0;'
$ echo 'select 1 + 1;' | sqlite3 
$ psql -tAc 'select 1+1'
$ R -q -e 'print(sd(rnorm(1000)))'
$ r -e 'cat(pi^2, "\n")'
$ r -e 'print(sum(1:100))'
$ smjs
$ jspl

Send HTML email in Llnux

A lot of Q&A out on the web, the best one I can find.

On my system , 2 successful implementations:

1. sendmail
(echo "From: sender@aaa.com}";
echo "To: receiver@bbb.com";
echo "Reply-To: all@ccc.com";
echo "Subject: HTML test";
echo "Content-Type: text/html";
echo "MIME-Version: 1.0";
echo "";
echo "<b>line1</b>";
echo "line2";
echo "<b>line3</b>";
) | sendmail -t


2. mutt  man
export REPLYTO="replyTo"

mutt -e 'set content_type="text/html"' -s "test"  xxx@yyy.com <$FILE


* cannot get mailx to include mail header info
* REPLYTO can store very long text, but mutt only takes around 225 chars, so if the reply list long switch to sendmail.

Access Env of anothe process

• Read env of another process: http://unix.stackexchange.com/questions/29128/how-to-read-environment-variables-of-a-process

1. xargs --null --max-args=1 echo < /proc/<pid>/environ
2.  ps wwwe <pid> | tr  ' ' '\n'

• Orphan vs Zombie vs Daemon processes: http://gmarik.info/blog/2012/08/15/orphan-vs-zombie-vs-daemon-processes

•  How To Kill Defunct Or Zombie Process: source: http://www.linuxnix.com/how-to-kill-defunct-or-zombie-process/

 

Display Tivoli Job Name next to logs

This ksh function lt (list tivoli) display additional Job Stream+ Job names next to log file list so we can see which job created the logs. It based on "ls -ltr" with Job names displayed next to log file name.

Tivoli logs are stored in ~maestroX/stdlist/YYYY.MM.DD. They keeps the job execution details but is hard to tell which job it is based on.  This function makes it easy to investigate the log files.

In a stdlist/YYYY.MM.DD folder and run lt, Output:

-rw-r--r-- 1 user1 uGroup 1543 Jun 24 00:05 O33964244.0004 (JOBSTREAM1.JOB1:0/1)
-rw-r--r-- 1 user1 uGroup 1060 Jun 24 00:05 O33964246.0005 (JOBSTREAM1.JOB2:1/0)
-rw-r--r-- 1 user1 uGroup 1578 Jun 24 00:10 O31059994.0010 (JOBSTREAM2.JOB1:0/4)
-rw-r--r-- 1 user1 uGroup 1543 Jun 24 00:10 O34291814.0009 (JOBSTREAM3.JOB1:0/1)
-rw-r--r-- 1 user1 uGroup 1060 Jun 24 00:15 O35487936.0015 (JOBSTREAM3.JOB2:1/0)
-rw-r--r-- 1 user1 uGroup 1060 Jun 24 00:15 O35389680.0015 (JOBSTREAM2.JOB2:1/10)
-rw-r--r-- 1 user1 uGroup 1058 Jun 24 00:15 O34934840.0015 (JOBSTREAM4.JOB1:1/12)
-rw-r--r-- 1 user1 uGroup 1059 Jun 24 00:15 O34775842.0015 (JOBSTREAM4.JOB1:0/36)    
-rw-r--r-- 1 user1 uGroup 1059 Jun 24 00:15 O34775841.0015 (JOBSTREAM5.JOB1:-/-)

• First # after Job Name is the Exit Status. '-' if still running
• 2nd # after Job Name is  Elapsed Time.  '-' if still running
• grep on a Job Stream to see all Jobs within and processing time
• grep on a Job across dates to see overall performance

linux/ksh:

lt(){ for x in `ls -tr $* | grep  -E "O[0-9]*.[0-9]{4}$"`; do; echo -e `ls -l $x` "\c" ; grep ^"= JOB" $x | sed "s/[:#\\[,.]/ /g" | awk '{printf "(%s.%s", $4,$8}'; echo `grep '^= Exit Status' $x`|sed 's/ : /:/g'| awk -F: '{printf ": %s/", (length($0) == 0)?"-":$2}'; echo `grep 'Elapsed' $x`|sed 's/ : /:/g'| awk -F: '{printf "%s)\n", (length($0) == 0)?"-":$3}';done; }


linux/bash:
lt(){ for x in `ls -tr $* | grep  -E "O[0-9]*.[0-9]{4}$"`; do
echo -e `ls -l $x` "\c" ;grep ^"= JOB" $x | sed "s/[:#\\[,.]/ /g" | awk '{printf "(%s.%s", $4,$8}';
echo `grep '^= Exit Status' $x`|sed 's/ : /:/g'| awk -F: '{printf ": %s/", (length($0) == 0)?"-":$2}'; echo `grep 'Elapsed' $x`|sed 's/ : /:/g'| awk -F: '{printf "%s)\n", (length($0) == 0)?"-":$3}';done;}


aix/ksh:
lt{for x in `ls -tr $* | grep  -E "O[0-9]*.[0-9]{4}$"`; do
   echo  `ls -l $x` "\c" ;
   grep ^"= JOB" $x | sed "s/[:#\\[,.]/ /g" | awk '{printf "(%s.%s", $4,$8}'; echo `grep '^= Exit Status' $x`|sed 's/ : /:/g'| awk -F: '{printf ": %s/", (length($0) == 0)?"-":$2}'; echo `grep 'Elapsed' $x`|sed 's/ : /:/g'| awk -F: '{printf "%s)\n", (length($0) == 0)?"-":$3}';
done; }

Misc Linux/Shell

Get rid of ^M

• cat xxx | tr -d "\r"
• :%s/^M//g (in vi, ^M = Ctrl-V + Ctrl-M)
• sed -e 's/^M//g' 
• tr -d $'\r' < xx > yy

 Process ID

http://unix.stackexchange.com/questions/32576/how-to-get-subshells-pid-in-korn-shell-equivalent-of-bashpid

• sub-shell id: 

1. readlink /proc/self
2. p=$(exec sh -c 'echo $PPID')

• current shell id:

1. $$
2. read -r xxx _ </proc/self/stat (xxx=1st argv, _=rest of line)

• parent shell id:

1. $PPID  
2. ps -p 12345678 -o ppid=

How to tell Korn shell version

https://community.oracle.com/thread/2251850

1. [ "${ERRNO}" ] && echo ksh88 || echo ksh99

How to tell if login or interactive shell

• [[ $- == *i* ]] && echo 'Interactive' || echo 'Not interactive'
• case $- in *i*) echo "This shell is interactive";; *) echo "This is a script";; esac
• http://unix.stackexchange.com/questions/38175/difference-between-login-shell-and-non-login-shell: interactive+login,  non-interactive+login(rare),  non-interactive+non-login,  interactive+non-login,  
• to see all options: echo $- (http://www.tutorialspoint.com/unix_commands/ksh.htm)

Ksh startup

• /etc/profile
•  $HOME/.profile
•  ksh's startup file(any file pointed to by ENV variable whether or not in login shell)

Print lines between patterns

Source: www.shellhacks.com/en/Using-SED-and-AWK-to-Print-Lines-Between-Two-Patterns+&cd=5&hl=en&ct=clnk&gl=us

File:

I Love Linux 
***** BEGIN ***** 
BASH is awesome 
BASH is awesome 
***** END ***** 
I Love Linux

• sed 

Syntax :

sed -n '/StartPattern/,/EndPattern/p' FileName  <<<---inclusive both

Option	Description
-n, --quiet, --silent	Suppress automatic printing of pattern space
p	Print the current pattern space

Example :

sed -n '/BEGIN/,/END/p' info.txt 

***** BEGIN *****
BASH is awesome
BASH is awesome
***** END *****

•  awk

Syntax :

awk '/StartPattern/,/EndPattern/' FileName

Example :

awk '/BEGIN/,/END/' info.txt

***** BEGIN *****
BASH is awesome
BASH is awesome
***** END ***** 

Another VERY Cool snipet:

source: 
http://stackoverflow.com/questions/9476018/split-text-file-into-parts-based-on-a-pattern-taken-from-the-text-file 

BEGIN { fn=0 }
NR==1 { next }
NR==2 { delim=$1 }
$1 == delim {
    f=sprintf("test%02d.txt",fn++);
    print "Creating " f
}

{ print $0 > f }

1. initialize output file number
2. ignore the first line
3. extract the delimiter from the second line
4. for every input line whose first token matches the delimiter, set up the output file name
5. for all lines, write to the current output file

---------------------------------------- another ex:

* file: TEST_TAF PREF: RAC1 RAC2 RAC3 ...... AVAIL: RAC4  (PREF, AVAIL)

   >  sed 's/.*PREF//;s/AVAIL.*//' yourfile

   >  sed 's/.*PREF: //;s/ AVAIL.*//;s/  */,/g' yourfile

* https://nixtip.wordpress.com/2010/10/12/print-lines-between-two-patterns-the-awk-way/

file: 

test -3

test -2

test -1

OUTPUT

top 2

bottom 1

left 0

right 0

page 66

END

test 1

test 2

test 3

> awk '/OUTPUT/ {flag=1;next} /END/{flag=0} flag {print}'

-------------------------- Another Ex: starting with multiple patterns

sed -n '/\(12:05:43.376\|Begin FS_BP.Init.*Init.OnExecute\)/,/12:05:43.605/p'  AE_FS_BP_2740745.trc 

More examples ------ https://stackoverflow.com/questions/18185771/extract-nth-line-after-matching-pattern

To extract the Nth line after a matching pattern you want:

awk 'c&&!--c;/pattern/{c=N}' file

e.g.

awk 'c&&!--c;/Revision:/{c=5}' file

would print the 5th line after the text "Revision:"/.

FYI the following idioms describe how to select a range of records given a specific pattern to match:

a) Print all records from some pattern:

awk '/pattern/{f=1}f' file

b) Print all records after some pattern:

awk 'f;/pattern/{f=1}' file

c) Print the Nth record after some pattern:

awk 'c&&!--c;/pattern/{c=N}' file

d) Print every record except the Nth record after some pattern:

awk 'c&&!--c{next}/pattern/{c=N}1' file

e) Print the N records after some pattern:

awk 'c&&c--;/pattern/{c=N}' file

f) Print every record except the N records after some pattern:

awk 'c&&c--{next}/pattern/{c=N}1' file

g) Print the N records from some pattern:

awk '/pattern/{c=N}c&&c--' file

I changed the variable name from "f" for "found" to "c" for "count" where appropriate as that's more expressive of what the variable actually IS.

SSH

3 Steps to Perform SSH Login Without Password Using ssh-keygen & ssh-copy-id

http://www.thegeekstuff.com/2008/11/3-steps-to-perform-ssh-login-without-password-using-ssh-keygen-ssh-copy-id/

* Step 1: Create public and private keys using ssh-key-gen on local-host
* Step 2: Copy the public key to remote-host using ssh-copy-id
* Step 3: Login to remote-host without entering the password

IP address

nslookup host/ip
host host/ip

netstat -in 

netstat  -npl - list port & pid/process

Files related....

fuser: Identifies processes using a file or file structure
whodo: Lists the jobs being performed by users on the system.
procfiles:list symbols from object files
lsof -p
ls -l /proc//fd

vmstat, mpstat, iostat, sar

ar: list library contents (.a)
nm: list symbols from object files (.a, .so, .o)
ldd: Lists dynamic dependencies (XCOFF) ($LD_LIBRARY_PATH)
what: Displays identifying information in files. (.a, .so, .o, XCOFF), work with GET/SCCS

Network Monitoring, packet analysis tool

http://www.wireshark.org/download.html

Korn Shell Snippets

Numeric Test --

1.
if [[ -z $(echo $2 | sed 's/[0-9]//g') ]]
then
echo "integer only"
fi

if [[ -z $(echo $2 | awk '/^[0-9]+$/') ]]
then
echo "integer only"
fi

if [[ ! -z $(awk -v x=$a 'END{if(x==x+0) print "integer"}' /dev/null) ]]
then
echo "integer only"
fi

Select * From table_name Where Regexp_Like(column_name, '^[0-9]+
Display Shell line number --

typeset -x PS4='[$LINENO] '
ksh -x script)

Alias to display File access/mod/create time --
alias ft='_(){ echo "modify time: \c";ls -l $1; echo "create time: \c"; ls -lc $1; echo "access time: \c";ls -lu $1; }; _'

Display Date Time at prompt:
export SECONDS="$(date '+3600*%H+60*%M+%S')";typeset -Z2 _h; typeset -Z2 _m ; typeset -Z2 _s;_hh="(SECONDS/3600)%24";_mm="(SECONDS/60)%60";_ss="(SECONDS)%60";_time='${_x[(_m=_mm)==(_h=_hh)==(_s=_ss)]}$_h:$_m:$_s';
export PS1=$(echo "${_time}")':$PWD>';

Date math & TZ (AIX): /etc/environment (https://www-304.ibm.com/support/docview.wss?uid=isg3T1000252)

TZ=CST6CDT,M3.2.0/2:00:00,M11.1.0/2:00:00 (DST enabled)

• CST6CDT is the time zone you are in;
• M3 is the third month;
• .2 is the second occurrence of the day in the month;
• .0 is Sunday;
• /2:00:00 is the time.

# get current TZ value
tz=`echo $TZ | tr -s '[:upper:]' '[\0*]' | cut -d, -f"1"`

# 2 hrs back
((tz_2hr_back=$tz+2))

# date string format = MMDDHHMIYY
TZ=GMT+$tz_2hr_back date +%m%d%H%M%y;


ksh93 Associative Array:
unsert color
set -A color
color=([Application Engine]=1 \
[COBOL SQL]=2 \
[Crystal]=3 \
[SQR Process]=4 \
[SQR Report]=5)


${color[@]} = all elements of the array
${!color[@]} = subscripts of an array
${#color[@]} = # of elements within the array.
${color[@]:offset:length} = Elements within a numeric subscript range


echo ${color[@]}
echo ${!color[@]}
echo ${#color[@]}
echo ${color[@]:1:length}

idx=0
while [ $idx -lt ${#color[@]} ]
do
echo ${color[@]:$idx:1}
((idx=idx+1))
done

My bin

dd -

#!/bin/csh -f
#-------------------------------------------------------------------------------
set parm = 'ls -l'

while (1)
if ($1 == '/od') then
if ($?bysize) then
echo 'Error: cannot sort by both date & size'
exit(1)
else
set bydate = '-t -r'
endif
else
if ($1 == '/os') then
if ($?bydate) then
echo 'Error: cannot sort by both date & size'
exit(1)
else
set bysize = 'sort -n -k 5'
endif
else
if ($1 == '/r') then
set revse = '-r'
else
if ($1 == '/a') then
set hidden = '-a'
else
break
endif
endif
endif
endif
shift
end

#---------------------------------------------------------------------------
if ($?hidden) then
set parm = "$parm $hidden"
endif

if ($?bydate) then
set parm = "$parm $bydate"
endif

if ($?bysize) then
$parm $* | $bysize | more -e
else
$parm $* | more -e
endif

exit(0);

---

gr -

#!/bin/ksh


# this script greps Process

if [ $1'x' = 'x' ]; then
echo 'Nothing to grep'
exit 1
fi

flag=ef

while getopts :c:l args
do
case $args in

c) clear
shift
;;

l) flag=fL
shift
;;

esac
done

if [ $flag == "ef" ]; then
echo "tracing process by name >>>> ps -ef | grep $* "

ps -ef | grep $* | grep -v ~djen/bin/gr | grep -v grep
else

echo "tracing single process tree >>>> ps -fL "

ps -fL $*
fi

---

path -

#!/bin/ksh
#-------------------------------------------------------------------------------

if [ $# -eq 0 ]; then

echo $PATH | tr ':' '\n'

else

set -A arPATH `echo $PATH | tr ':' ' '`

idx=0
while [ $idx -lt ${#arPATH[*]} ]
do
if [ -f ${arPATH[idx]}/$1 ]; then
ls -l ${arPATH[idx]}/$1
break
else
((idx=idx+1))
fi
done

fi

my quick ksh

Numeric test -

1.
nonums="$(echo $answer | sed 's/[0-9]//g')"

if [ ! -z "$nonums" ] ; then
echo "Not an integer value."
fi

2.
echo -n "Pick a number between 1 and 20: " read answer

nonums="$(echo $answer | sed 's/[0-9]//g')"

if [ ! -z "$nonums" ] ; then
echo "Not an integer value." exit 0
fi

if [ $answer -lt 10 ] ; then
echo "Your answer is less than ten"
else
echo "Your answer is not less than ten"
fi

3.case xxx in
+([0-9]))
echo 1
;;
* )
echo 0
;;
esac

4. stackflow:

re='^[0-9]+$'
if ! [[ $yournumber =~ $re ]] ; then
   echo "error: Not a number" >&2; exit 1
fi

---

Parse file Name & Ext -

file=xxxx.yyy

nam=`echo ${file%.*}`
ext=`echo ${file#*.}`

---

remove CR from EOF -

• tr '\012' ' '; echo

• awk '{printf "%s ", $0} END { print "" }'

---

Ksh Version -

at a ksh prompt - print xxx

$ set|egrep -i ver

$ what /bin/ksh | egrep Version

---

Email -

simple text body:

• cat file1 | mailx -s "subject" user@host.com
• mailx -s "subject" user@host.com <>

attachments:

• uuencode file1 file1 | mailx -s "subject" user@host.com
• (uuencode file1 file1; uuencode file2 file2) | mailx -s "subject" user@host.com

both:

• (cat file1; uuencode file1 file1;uuencode file2 file2 ) | mailx -s "subject" user@host.com

save attachment in mbox then uuencode to retrieve the file.

sendmail:

#!/bin/ksh
set -xv
# sendmail header function
fn_sendmail()
{
# Build sendmail header.
echo "From: user1@test.com"
echo "To: user2@test2.com"
echo "Reply-to: user3@test3.com, user4@user4"
echo "Subject: ${SUBJECT}"
echo "X-MSMail-Priority: High"
echo "importance: 1"
#echo "Mime-Version: 1.0"
#echo "Content-Type: text/plain"
#echo "Content-Type: multipart/mixed"
echo "

# Run the data function.
Hi XXX,

This is a test email

YYY
"
# uuencode the attachments, add checks if required file extension,
for k in file1 file2
do
uuencode $k $k
done

}

SUBJECT=test_subject

# Call sendmail header and pipe to sendmail.
(fn_sendmail)| sendmail -t -oi
##