Saturday, April 6, 2024

Use .profile in Linux to customize the shell prompt

When you have many EBS instances in a multi-nodes environment, it will be very useful to let the Linux prompt display current user ID, server name and the path location. A custom .profile saved under $HOME works for me very well. Its colors tell if you are in a Admin node or not, and if you are in a production environment or not (assume the last character of production server's name is "p").

For a Linux account, environment variable $HOME is defined by file /etc/passwd. But, if the account was created by AD (Active Directory), the default value of $HOME is defined in "Home Directory" section of AD. The value of $HOME could be set by "export" in file ~applMgr/.profile, in case some confusion.

Our EBS applMgr accounts use Korn shell which uses two startup files under $HOME, the .profile and the .kshrc. During a session start, .profile is first read once, then .kshrc (if it exists) is read by each new ksh. e.g. :

$ echo $SHELL
/bin/ksh
$ echo $0
-ksh
$ which ksh
/usr/bin/ksh
$ more .kshrc
alias ftp="print 'Reminder: Use sftp instead of \\\ftp'"
echo "This is .kshrc"
$ ksh
This is .kshrc
$ ftp
Reminder: Use sftp instead of \ftp

============= $HOME/.profile =============
PATH=/bin:/usr/bin:/usr/local/bin
export PATH
MANPATH=/usr/share/man:/usr/local/share/man
export MANPATH      # for man manual 
EDITOR=/bin/vi
export EDITOR
# ENV=$HOME/.kshrc
# export ENV
. /u02/app/EBSPROD/EBSapps.env RUN     # R12.2 env file
. /u02/app/xxx_scripts/.EBSpassenv              # password file (custom)
isMaster="no"
if [ ! -z $APPS_VERSION ] && [ ${APPS_VERSION:0:4} == "12.2" ]
then
s_status=cat $CONTEXT_FILE | grep -i s_adminserverstatus
isMaster="${s_status:60:7}"
fi
if [ $isMaster == "enabled" ]   # on admin/primary node
then
if [ echo -n ${HOSTNAME%%.*} | tail -c -1 != "p" ]   
             # last character of server name is not "p" => non-production server
then
PS1=$'
\e[0;31m$USER@${HOSTNAME%%.}[$TWO_TASK]\e[m$PWD
-->$ '  
else       # on production server: Red, and Green color on PWD
PS1=$'
\e[0;31m$USER@${HOSTNAME%%.}[$TWO_TASK]\e[m\E[32m$PWD \E[0m
-->$ '
fi
else                                          # on other node(s)
if [ echo -n ${HOSTNAME%%.*} | tail -c -1 != "p" ]   
            # on non-production server
then
PS1='
$USER@${HOSTNAME%%.}[$TWO_TASK]$PWD
-->$ '
else      # on production server
PS1=$'
$USER@${HOSTNAME%%.}[$TWO_TASK]\E[32m$PWD \E[0m
-->$ '
fi
fi

alias rm='rm -i'
stty erase ^?
umask u=rwx,g=rwx,o=rx
================ end =================

On an Admin node in production env, the prompt looks like this:
applMgr@server_1p[EBSPROD]/u02/app
-->$

applMgr@server_1p[EBSPROD]/u02/app
-->$ echo $USER
applMgr
applMgr@server_1p[EBSPROD]/u02/app
-->$ echo $TWO_TASK
EBSPROD
applMgr@server_1p[EBSPROD]/u02/app
-->$ cd $TWO_TASK
applMgr@server_1p[EBSPROD]/u02/app/EBSPROD
-->$ echo $HOME
/u02/app
applMgr@server_1p[EBSPROD]/u02/app/EBSPROD
-->$ ls
EBSapps.env   fs1   fs2   fs_ne

Wednesday, March 6, 2024

script to check if a password is expiring

The environment variable $HOME for a Linux account is defined by file /etc/passwd if the account was not created in AD (Active Directory). Each account has an entry line in file /etc/passwd. For example, I can get my account's password expiration date by: 

$ echo $HOME
/u02/app
$ whoami
applmgr
$ grep applmgr /etc/passwd
applmgr:x:50378:102:Oracle EBS ID - J Y:/u02/app:/bin/ksh
$ expstr=$( chage -l $(whoami) | grep "^Password expires" | awk -F: '{ print $(NF) }' | sed -e 's/^ *//g; s/ *$//g;' )
$ echo "password for account `whoami` will expire on $expstr"
password for account applmgr will expire on Jul 30, 2025

But, if the account was created by Windows AD (Active Directory), the variable $HOME is defined in AD by "Home Directory" (Note: an entry in .profile or such could change $HOME to a different path immediately after login). ADHelp search page may show info:
    Unix Account
Home Directory:   /users/applmgr
Login Shell:          /bin/ksh

In that case, "chage" will give a different result:
$ echo $HOME
/users/applmgr
$ expstr=$( chage -l $(whoami) | grep "^Password expires" | awk -F: '{ print $(NF) }' | sed -e 's/^ *//g; s/ *$//g;' )
chage: user 'applmgr' does not exist in /etc/passwd

For an important account created in Linux (vs. an AD account), I wrote a script to email warning out before its password expires. It can be run by a cron job, such as
30 12 * * * /path/to/xxxx_scripts/checkPWDexpire.sh 2>&1

============= script checkPWDexpire.sh =============
let secs_per_day=60*60*24
nowtime=$( date +%s )
expstr=$( chage -l $(whoami) | grep "^Password expires" | awk -F: '{ print $(NF) }' | sed -e 's/^ *//g; s/ *$//g;' )
echo "DEBUG: expstr is $expstr"
if [ "$expstr" == "never" ]; then
echo "Password never expires.";
exit 0;
fi
exptime=$( date --date "$expstr" +%s )
if [ "$exptime" -lt 1 ];        then
echo "Something is wrong.";
exit 255;   # Or, email a message out
fi
if [ "$exptime" -lt "$nowtime" ]; then
echo "Password already expired.";
exit 1;      # Or, email a message out
fi
secs_til_exp=$(expr $exptime - $nowtime)
days_til_exp=$(expr $secs_til_exp / $secs_per_day)
echo "Password expires in $days_til_exp days."
if [ "$days_til_exp" -lt 6 ]; then
# send email out
echo "Please reset password manually and update 3rd party environments." | mailx -s "`whoami` on `uname -n` will expire in $days_til_exp days" me@email.com
# or 
# mailx -s "`whoami` on `uname -n` will expire in $days_til_exp days" -a aFile.log me@email.com < aFile.log
else
echo "All is fine.";
exit ;
fi
============== end =====================

"chage" Linux command:
If OS user applmgr is granted sudo, it can act as root to check another account's status or change password status.

$ sudo su -
[sudo] password for applmgr:
Last login: Mon Mar 28 03:22:57 EDT xxxx
Hostname:  server_name.domain.com
OS:  Red Hat Enterprise Linux release 8.10 (Ootpa)
Arch:  x86_64

[root@server_name ~]# chage -E -1 batch_mgr   # -1 <== number
Notes: passing the number -1 to Expire Date (-E) only never expires the account, but not unexpire the password.   

[root@server_name ~]# chage -l batch_mgr       # -l <== --list
Last password change                                 : Feb 14, 2023
Password expires                                        : May 15, 2023
Password inactive                                       : Jun 14, 2023
Account expires                                          : never
Minimum number of days between password change      : 7
Maximum number of days between password change     : 90
Number of days of warning before password expires       : 7

[root@server_name ~]# chage -M -1 batch_mgr  

Notes: passing the number -1 as MAX DAYS (-M) will remove checking a password validity, which turns off the various password aging properties. Now batch_mgr can use its existing password to login.

[root@server_name ~]# chage -l batch_mgr
Last password change                              : Feb 14, 2023
Password expires                                      : never
Password inactive                                     : never     <= never be deactivated due to inactivity
Account expires                                        : never
Minimum number of days between password change      : 7
Maximum number of days between password change     : -1
Number of days of warning before password expires       : 7

[root@server_name ~]# chage -l applmgr      # b/c applmgr was originally created in AD 
chage: user 'applmgr' does not exist in /etc/passwd

To change a user's password as root:
[root@server_name ~]# passwd batch_mgr
Enter new UNIX password:
... ...

Saturday, February 17, 2024

Shell script for renewing ssl certificate

My post Re-new R12.2 ssl certificate has details on how to renew a certificate. A shell script helps a lot when there are many EBS instances waiting for renewal. I wrote below script which takes only one minute to renew the cert on each node after the certificate is renewed on Venafi website and downloaded/copied to Linux server. 

As of today, we still have difficulties using .yaml file to extract certificate from Venafi server to Linux server automatically. We tried to set up a "push" way on Venafi website to do the automation. But if the password is changed on the Linux account, the push will fail. 

============= Script renew_cert.sh ============
# Script for renewing ssl certificate after new cert file is saved to Linux server

walletpwd='putPWDhere'
# walletpwd='tttest'
walletloc=$HOME/xxx/Certs_Renew   # path where the Venafi cert file is saved
walletname='ewallet.p12'                # Must name Venafi cert file to this name
certname='cwallet.sso'

echo "cert at: $walletloc"
echo "cert name: $walletname"
echo $walletpwd

cd $walletloc

errorC=`env| grep RUN_BASE | wc -l`
if [ $errorC -lt 1 ]; then
  echo "No R12.2 environment"
  exit 1
  # . $HOME/EBSQA/EBSapps.env RUN
fi

alias orapki=$FMW_HOME/oracle_common/bin/orapki

orapki wallet display -wallet $walletloc/$walletname -pwd $walletpwd > viewCert.log
errorC=`egrep -i 'PKI-' viewCert.log | wc -l`
echo "Error: $errorC"

if [ $errorC -gt 0 ]; then
   echo "The password is incorrect or the Venafi cert file is incorrect."
   exit 2
fi

DT=`date +"%h_%d_%y_%H%M"`
if [ -f $certname ]; then
   mv $certname ${certname}_${DT}
fi

orapki wallet create -wallet $walletloc/$walletname -pwd $walletpwd -auto_login

if [ ! -f $certname ]; then
   echo "Failure in getting new cert file. Exiting."
   exit 3
fi

echo " "
echo "Copy cert file to directories ..."

cd $NE_BASE/inst/$CONTEXT_NAME/certs    # save a copy in this folder
if [ -d Apache ]; then
mv Apache Apache_${DT}
fi
mkdir Apache
cd Apache
pwd
cp -p $walletloc/$walletname ${walletname}
cp -p $walletloc/$certname ${certname}

iName=$(tr < $CONTEXT_FILE '<>' '  ' | awk '/"s_ohs_instance"/ {print $(NF-1)}' )
SUBiName=${iName%?????}
cd $FMW_HOME/webtier/instances/$iName/config/OPMN/opmn/wallet
pwd
if [ -f $certname ]; then
   mv $certname ${certname}_${DT}
fi
cp -p $walletloc/$certname ${certname}

cd $FMW_HOME/webtier/instances/$iName/config/OHS/$SUBiName/keystores/default
pwd
if [ -f $certname ]; then
   mv $certname ${certname}_${DT}
fi
cp -p $walletloc/$certname ${certname}

cd $FMW_HOME/webtier/instances/$iName/config/OHS/$SUBiName/proxy-wallet
pwd
if [ -f $certname ]; then
   mv $certname ${certname}_${DT}
fi
cp -p $walletloc/$certname ${certname}

echo " "
echo "Recycle Apache service..."
cd $ADMIN_SCRIPTS_HOME
./adopmnctl.sh stop
sleep 10
./adopmnctl.sh status

./adapcctl.sh start
./adopmnctl.sh status

echo "Paths for log files:"
echo $FMW_HOME/webtier/instances/$iName/diagnostics/logs/OHS/$SUBiName
echo $FMW_HOME/webtier/instances/$iName/diagnostics/logs/OPMN/opmn
cd
============ End ==========

Run the script to renew certificate on each node:
$ ./renew_cert.sh
cert at: $HOME/temp/Certs_Renew
cert name: ewallet.p12
putPWDhere
Error: 0
Oracle PKI Tool : Version 11.1.1.9.0
Copyright (c) 2004, 2015, Oracle and/or its affiliates. All rights reserved.

Copy cert file to directories ...

/u04/app/EBSQA/fs_ne/inst/EBSQA_nodeName/certs/Apache
$FMW_HOME/webtier/instances/EBS_web_EBSQA_OHS1/config/OPMN/opmn/wallet
$FMW_HOME/webtier/instances/EBS_web_EBSQA_OHS1/config/OHS/EBS_web_EBSQA/keystores/default
$FMW_HOME/webtier/instances/EBS_web_EBSQA_OHS1/config/OHS/EBS_web_EBSQA/proxy-wallet

Recycle Apache service ...

You are running adopmnctl.sh version 120.0.12020000.2

Stopping Oracle Process Manager (OPMN)  and the managed processes ...
opmnctl stopall: stopping opmn and all managed processes...

adopmnctl.sh: exiting with status 0

adopmnctl.sh: check the logfile $LOG_HOME/appl/admin/log/adopmnctl.txt for more information ...  

You are running adopmnctl.sh version 120.0.12020000.2

Checking status of OPMN managed processes...
opmnctl status: opmn is not running.

adopmnctl.sh: exiting with status 0

adopmnctl.sh: check the logfile $LOG_HOME/appl/admin/log/adopmnctl.txt for more information ...  

You are running adapcctl.sh version 120.0.12020000.6

Starting OPMN managed Oracle HTTP Server (OHS) instance ...

adapcctl.sh: exiting with status 0

adapcctl.sh: check the logfile $LOG_HOME/appl/admin/log/adapcctl.txt for more information ...  

You are running adopmnctl.sh version 120.0.12020000.2

Checking status of OPMN managed processes...

Processes in Instance: EBS_web_ARQA_OHS1
--------------------------------+--------------------+---------+---------
ias-component                    | process-type       |     pid    | status  
--------------------------------+--------------------+---------+---------
EBS_web_EBSQA             | OHS                   |   14542 | Alive   

adopmnctl.sh: exiting with status 0

adopmnctl.sh: check the logfile $LOG_HOME/appl/admin/log/adopmnctl.txt for more information ...  

Paths for log files:
$FMW_HOME/webtier/instances/EBS_web_EBSQA_OHS1/diagnostics/logs/OHS/EBS_web_EBSQA
$FMW_HOME/webtier/instances/EBS_web_EBSQA_OHS1/diagnostics/logs/OPMN/opmn

Check files in folder $HOME/xxx/Certs_Renew:
$ ls 
renew_cert.sh
ewallet.p12
ewallet.p12.lck
cwallet.sso.lck
viewCert.log
cwallet.sso

NOTES: there is a cert file in $EBS_DOMAIN_HOME/opmn/EBS_web_EBSQA_OHS1/wallet and $EBS_DOMAIN_HOME/opmn/EBS_web_EBSQA_OHS1/EBS_web/wallet. But I do not know what uses them.

Wednesday, November 22, 2023

How to run AutoConfig on PATCH file system in R12.2

 Steps for running AutoConfig on PATCH file system:

1) Disable a trigger
SQL> conn system/systemPWD
Connected.
SQL> alter trigger ebs_logon disable;
Trigger altered.

If EBS_SYSTEM account exists in db, disable same trigger owned by it to avoid errors:
$ sqlplus apps/appsPwd
ERROR:
ORA-04088: error during execution of trigger 'EBS_SYSTEM.EBS_LOGON'
ORA-00604: error occurred at recursive SQL level 1
ORA-20099: E-Business Suite Patch Edition does not exist.
ORA-06512: at line 48

2) Set PATCH env and connect to database
$ . /<EBS_HOME_BASE>/EBSapps.env patch

$ echo $TWO_TASK
EBSDEV_patch
$ sqlplus apps/appsPWD  
    -- If got error, compare tnsnames.ora in both RUN and PATCH file systems &
    -- modifying tnsnames.ora in PATCH file system may be needed in Oracle 19c db. 
SQL> show user
USER is "APPS"
SQL> exit

3) Run autoconfig in PATCH file system
$ cd $ADMIN_SCRIPTS_HOME
$ echo $FILE_EDITION
patch
$ ./adautocfg.sh
Enter the APPS user password:

The log file for this session is located at: $INST_TOP/admin/log/MMDDHHMI/adconfig.log
... ...
AutoConfig completed successfully.

4). Enable the trigger
SQL> conn system/systemPWD
Connected.
SQL> alter trigger ebs_logon enable;
Trigger altered.

SQL> conn ebs_system/systemPWD
Connected.
SQL> alter trigger ebs_logon enable;
Trigger altered.

Friday, November 10, 2023

Port conflict during R12.2 apps clone

If Target mid-tier and Source mid-tier are on the same server, EBS clone script adcfgclone.pl may fail with port conflict error. 

Prots used for an EBS instance is listed in file $INST_TOP/admin/out/portpool.lst. Depending on which port has conflict on the server, the cloning error message will be different.  

- If the port for httpd is busy and used by another environment on the server, the cloning script will stop on Target instance with errors:

ERROR: Failed to configure the target system,
please check the logfile in : $RUN_BASE/inst/apps/$CONTEXT_NAME/admin/log/clone

CLONE-26003   Error in validating listen host and port. 
CLONE-26176  In config group httpd.conf , the value of "Listen" config property was xxxx. xxxx was not free

Log message shows the problem is from file httpd.conf. When I checked the port number and compare it in file $CONTEXT_FILE and in file $FMW_HOME/webtier/instances/EBS_web_<SID>_OHS1/config/OHS/EBS_web_<SID>/httpd.conf. 
$ grep Listen httpd.conf
$ grep s_http_listen_parameter $CONTEXT_FILE

I saw they are different in Source instance. Even they are not the same, Apache still works fine on Source instance and httpd.worker processes occupy the port number in httpd.conf. After the clone script copied file httpd.conf to Target instance, Apache failed to start because OHS was running on the same port as the other instance

Apparently httpd.conf of Source instance was edited by picking up randomly a port number. httpd.conf can be edited and updated manually as AutoConfig in R12.2 does not update it. For more details on modifying port values for OHS, see Doc ID 1905593.1 (Managing Configuration of Oracle HTTP Server and Web Application Services in Oracle E-Business Suite Release 12.2).

- While cloning script uses a temporary port but it was not available, the error could be 
CLONE-20372 Server port validation failed.
The fix could be just shutdown apps services of Source instance to let the cloning complete. See Doc ID 2002613.1

- Doc ID 2437111.1 gives a fix on port for s_ohs_adminport. 

On Linux server, use command to find if a port is used or not:

$ netstat -tuanp | grep 6230
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp    0 0 167.69.109.82:6230 0.0.0.0:*   LISTEN 

Then the process owner (or root) can use below line to find what process is using the port: 

$ lsof -i :6230 
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
opmn 31876 ebsdev 9u IPv4 42886593 0t0 TCP server3d.domain.com:6230 (LISTEN)

After adcfgclone.pl failed, you have to remove/rename folders BEFORE re-run clone script adcfgclone.pl 
- Remove two new folders under $RUN_BASE (or only folder $RUN_BASE/FMW_Home, depending on the failure stage) to avoid error "Exiting cloning as FMW Home already exists".
- Also remove the folder defined by inventory_loc in file /etc/oraInst.loc to avoid error "Oracle Homes are already registered in the inventory". If the inventory is just for one environment, do not try to detach a home because it may give error:
$ ./runInstaller -detachhome $FMW_HOMEe/oracle_common
Starting Oracle Universal Installer...
Checking swap space: must be greater than 500 MB.   Actual 13519 MB    Passed
The inventory pointer is located at /etc/oraInst.loc
The inventory is located at ... ...
The operation failed as it was called without name of the Oracle Home being attached.
- Since the script failed before the completion, there is not need to remove info in the database.