I followed Doc ID 1080973.1 in below steps to apply patch 9239090 for upgrading R12.1.1 to R12.1.3 on an Oracle 11.2.0.3.0 database.
Current system status:
SQL> select * from ad_bugs where bug_number in ('9239089', '9239090', '9239095', '9817770', '9966055');
only 9239089 was applied.
$ $IAS_ORACLE_HOME/Apache/Apache/bin/httpd -v
Server version: Oracle-Application-Server-10g/10.1.3.5.0 Oracle-HTTP-Server
Server built: Jul 21 2009 11:12:22
$ $ORACLE_HOME/bin/frmcmp_batch|grep Forms| grep Version
Forms 10.1 (Form Compiler) Version 10.1.2.3.0 (Production)
$ cat $FORMS_WEB_CONFIG_FILE|grep sun_plugin_version| cut -c 1-35
sun_plugin_version=1.6.0_06
Pre steps in database (to avoid headaches during the upgrading):
1. Setup two database parameters:
_disable_fast_validate=TRUE <-- this parameter shall be removed after upgrade completed.
pga_aggregate_target=2G
2. Follow Doc ID 1112223.1 to grant XDBADMIN the right role to avoid ORA-31000 error with csrrsreg.sql.
SQL> show user
apps
SQL> select * from USER_ROLE_PRIVS;
USERNAME GRANTED_ROLE ADM DEF OS_
----------------------------------- ------------------------------------------------- ------- ------ ---
APPS AQ_ADMINISTRATOR_ROLE YES YES NO
APPS CTXAPP NO YES NO
APPS JAVASYSPRIV NO YES NO
APPS RESOURCE NO YES NO
APPS XDBADMIN NO YES NO
3. Follow Doc ID 1357760.1 & 1081227.1 to avoid autoconfig error from jtfictx.sh
DECLARE
*
ERROR at line 1:
ORA-29874: warning in the execution of ODCIINDEXALTER routine
ORA-29960: line 1,
DRG-10595: ALTER INDEX JTF_AMV_ITEMS_URL_CTX failed
DRG-10758: index owner does not have the privilege to use file or URL datastore
ORA-06512: at "SYS.DBMS_SQL", line 1199
ORA-06512: at line 96
Fixes:
SQL> create role apps_ctx_role;
SQL> grant apps_ctx_role to apps;
SQL> exec ctxsys.ctx_adm.set_parameter('file_access_role', 'apps_ctx_role');
SQL> grant apps_ctx_role to JTF; -- Grant it to the index owners
SQL> grant apps_ctx_role to AMV;
SQL> grant apps_ctx_role to APPLSYS;
SQL> grant apps_ctx_role to HR;
SQL> grant apps_ctx_role to IBC;
SQL> grant apps_ctx_role to OKC;
4. Follow Doc ID 1551854.1 & 847687.1 to avoid error from EGOSILDU.sql
DECLARE
*
ERROR at line 1:
ORA-02149: Specified partition does not exist
ORA-06512: at line 408
SQL> SELECT ATTR_GROUP_ID
FROM EGO_FND_DSC_FLX_CTX_EXT WHERE APPLICATION_ID = 431
AND DESCRIPTIVE_FLEXFIELD_NAME = 'EGO_ITEMMGMT_GROUP'
AND DESCRIPTIVE_FLEX_CONTEXT_CODE = 'ItemDetailDesc';
ATTR_GROUP_ID
-------------------------
2
SQL> SELECT 1 FROM EGO_MTL_SY_ITEMS_EXT_TL WHERE ATTR_GROUP_ID = 2
AND ROWNUM = 1;
no rows selected
SQL> SELECT 1 FROM EGO_MTL_SY_ITEMS_EXT_TL WHERE ATTR_GROUP_ID = 2
AND ROWNUM = 1;
no rows selected
Fixes: Open a SR with Oracle Support to get a datafix script from them. And run it as sysdba:
SQL> @Partition_Fix_Script_Ver2.sql
Notes: After I used $AD_TOP/sql/adutconf.sql to confirm module EGO is inactive, I let the failed worker to quit and then skipped EGOSILDU.sql in my Dev instance. Seems ignoring the error worked fine.
5. Make sure account OLAPSYS is not locked.
SQL> select username, account_status from dba_users like '%OLAP%';
If locked, it will hit errors in Doc ID 1054417.1:
ERROR at line 1:
ORA-33292: Insufficient permissions to access analytic workspace APPS.ODPCODE
using the specified access mode.
ORA-06512: at "SYS.DBMS_AW", line 67
ORA-06512: at "SYS.DBMS_AW", line 131
ORA-06512: at "SYS.DBMS_AW", line 984
ORA-06512: at "SYS.DBMS_AW", line 937
ORA-33262: Analytic workspace ODPCODE does not exist.
ORA-06512: at "SYS.DBMS_AW", line 908
ORA-06512: at "APPS.MSD_AW_LOADER", line 10
Pre steps on Apps side:
1. If any schema owner password (such as HR) was changed by Sql*plus abnormally (not by FNDCPASS), the upgrade will fail because adpatch would not have the right password for that module. I had to tell all workers to quit and then "kill" adpatch process on OS level because adpatch kept remembering the bad password. After I used FNDCPASS to change the wrong password, I re-ran adpatch from the beginning and it worked.
2. Apply patch 10257394 (Doc ID 1286779.1) to avoid error with IEX_SCORE_COMP_DET_U1
(or, copy iexscore.lct from patch .zip file and manually replace the old one)
Apply the patch:
$ unzip 6 .zip files (for patch 9239090) to same folder /path/to/R1213CD
($ unzip xxxX.zip -d /path/to/R1213CD)
$ adadmin to enable maintenance mode
$ cd /path/to/R1213CD/9239090 <== make folder 9239090 10GB free and rw permissions
$ adpatch <== it takes 3 or more hours
Then, optional step: start apps to do a quick check out.
Post steps:
$ cd 9239095
$ adpatch
$ cd 9817770
$ adpatch
$ cd 9966055
$ adpatch
$ cd $AD_TOP/bin
$ perl admkappsutil.pl ==> then copy appsutil.zip file to DB server.
$ adconfig.sh on DB tier
$ adautocfg.sh on each apps tiers
disable maintenance mode
start apps services
remove _disable_fast_validate=TRUE
NOTES: In a R12.1.1 instance, a lot of database sessions kept in INACTIVE for a long time (15+ days) and never got closed. So, the number of db sessions increased daily and exceeded easily database parameter PROCESSES (3000). We had to recycle Apps services each two weeks. Modules AR and GL are used most in this instance and most idle sessions are from "e::bes:oracle.apps.icx.security.session.created" by JDBC Thin Client. We tried and did not find a fix to this problem. But, after the instance was upgraded to R12.1.3 and its database was upgraded to 12c, the idle session problem went away surprisingly.
Saturday, December 19, 2015
Sunday, December 13, 2015
Configure printer to work with EBS
This summary is not available. Please
click here to view the post.
Saturday, December 12, 2015
EBS Refresh
Refresh from EBSPROD (under /u06/app/EBSPROD) to EBSQA (under /u01/app/EBSQA) on a different host in Linux OS.
1. On source nodes, run PERL script.
$ cd $ADMIN_SCRIPTS_HOME
$ perl adpreclone.pl appsTier <-- No password is needed
NOTE: Pre-clone log file located at $INST_TOP/admin/log/StageAppsTier_MMDDHHMM.log
$ cd $COMMON_TOP/clone <-- to verify new folders are created
$ ls -altr $COMMON_TOP/clone
total 40
drwxr-xr-x 2 applmgr appsuser 4096 Jan 22 2008 html
drwxr-xr-x 3 applmgr appsuser 4096 Jan 22 2008 context
drwxr-xr-x 2 applmgr appsuser 4096 Jan 22 2008 bin
drwxr-xr-x 9 applmgr appsuser 4096 Sep 15 2013 ..
drwxr-xr-x 5 applmgr appsuser 4096 Dec 1 10:06 appsts
drwx------ 7 applmgr appsuser 4096 Dec 1 10:07 jre
drwxr-xr-x 3 applmgr appsuser 4096 Dec 1 10:07 oui
drwxr-xr-x 4 applmgr appsuser 4096 Dec 1 10:07 jlib
drwxr-xr-x 5 applmgr appsuser 4096 Dec 1 10:07 appl
drwxr-xr-x 10 applmgr appsuser 4096 Dec 1 10:07 .
Do the same on other source hosts.
2. On target nodes, backup filess
Create a folder, such as /u01/app/admin_scripts/backups, to hold backed-up files. Then run the script backup_4refresh.sh to back up <SID_HOST> specific files.
$ cd admin_scripts/backups
$ ./backup_4refresh.sh
Confirm necessary files are just backed up:
$ ls -al /u01/app/admin_scripts/backups
$ ls -al /u01/app/admin_scripts/backups/admin
$ ls -al /u01/app/admin_scripts/backups/Apache
Optional: Backup the entire folder to aother partition
$ cd /u01/app
$ tar -zhcvf /u02/app/$CONTEXT_NAME_backup_MMDDYY.tar.gz EBSQA >> backup_MMDD.log
Do the same on other hosts.
3. Make sure all apps services were stopped and database refresh was completed.
Now, remove directories
$ cd /u01/app
$ rm -rf EBSQA
$ rm -rf oraEBSQAinventory; mkdir oraEBSQAinventory
4. Copy directories from source node.
NOTE: copy CM host files to CM host, web/forms host to web/forms host.
$ cd /u01/app
$ rsync --progress -avze ssh applmgr@source_host:/u06/app/EBSPROD /u01/app
enter password
sent 12619428 bytes received 9649138852 bytes 1312649.72 bytes/sec
total size is 33018009660 speedup is 3.42
It may give warning message:
rsync warning: some files vanished before they could be transferred (code 24) at main.c(1532) [generator=3.0.6]
5. Rename the folder and verify the size
$ cd /u01/app
$ mv EBSPROD EBSQA
$ ls -al EBSQAtotal 16
drwxr-xr-x 4 applmgr appsuser 4096 May 7 2013 .
drwxr-xr-x 16 applmgr appsuser 4096 Apr 15 11:27 ..
drwxr-xr-x 4 applmgr appsuser 4096 May 2 2013 apps
drwxr-xr-x 3 applmgr appsuser 4096 May 2 2013 inst
$cd apps
$ du -ks
31640672 .
$ cd ../inst
$ du -ks
2502420 .
6. Edit /etc/oraInst.loc file to make sure below line is used:
inventory_loc=/u01/app/oraEBSQAInventory
7. Start a new OS session (without running the .env file)
If you do not modify the .profile, re-login will get message:
$APPL_TOP/${CONTEXT_NAME}.env: cannot open [No such file or directory]
Note: by this time, the .env file was deleted when old folder EBSQA was removed in Step 5.
So env vairables do not exist
8. Go to /u01/app/EBSQA/apps/apps_st/comn/clone/bin ($COMMON_TOP/clone/bin)
option: delete old files in /u01/app/EBSQA/apps/apps_st/comn/clone
$ cd /u01/app/EBS/apps/apps_st/comn/clone
$ rm -rf jre_BAK_*
Now, get below answers ready -
Database Server Node => dbhost1q.domain.com
database SID => EBSQA
apps => appsPWD
Base Directory => /u01/app/EBSQA
PORT => 1542
port pool: 21 (1542 -1521)
9. Run clone perl script and answer questions
$ perl adcfgclone.pl appsTier <<-- run this in CM node first!!
++++++++++++++++ answers on CM node ++++++++++++++++
Target System Root Service [disabled] : <-- for CM node. WEB/Form node will be opposite
Target System Web Entry Point Services [disabled] :
Target System Web Application Services [disabled] :
Target System Batch Processing Services [enabled] :
Target System Other Services [enabled] : <-- for CM node
Do you want to preserve the Display [y] (y/n) : n
Target System Display [host1q:0.0] : host1q.domain.com:21.0 <-- OAM will use it for display
Do you want the the target system to have the same port values as the source system (y/n) [y] ? : n
Target System Port Pool [0-99] : 21
++++++++++++++++ answers on Form/Web node ++++++++++++++++
Target System Root Service [enabled] : <= for Form node
Target System Web Entry Point Services [enabled] :
Target System Web Application Services [enabled] :
Target System Batch Processing Services [enabled] : disabled <= for Form node
Target System Other Services [enabled] : disabled
Do you want to preserve the Display [xxx_name:0.0] (y/n) : n
Target System Display [host2q:0.0] : host2q.domain.com:21.0 <= OAM will use it for display
Do you want the the target system to have the same port values as the source system (y/n) [y] ? : n
Target System Port Pool [0-99] : 21
The script may remain silent for a few minutes (or even 40 minutes) on executing $INST_TOP/admin/install/txkWfClone.sql !
(only statement SELECT * from v$session where upper(machine) like 'HOSTNAME%'; can tell if the sql script is running).
Answer "no" - Dot not start apps services after autoconfig completed.
Check the log file. Any more lines than below need your attention or fix:
$ egrep -i 'fail|error|ora-' $INST_TOP/admin/log/ApplyAppsTier_04151536.log
Error while running adlnktools.sh.
ERRORCODE = 0 ERRORCODE_END
ERRORCODE = 0 ERRORCODE_END
AC-50480: Internal error occurred: java.lang.Exception: Error while generating listener.ora.
Error generating tnsnames.ora from the database, temporary tnsnames.ora will be generated using templates
ERRORCODE = 2 ERRORCODE_END
Result : FAILED
ERRORCODE = 0 ERRORCODE_END
Unique constraint error (00001) is OK if key already exists
ERRORCODE = 0 ERRORCODE_END
ERRORCODE = 0 ERRORCODE_END
. . . . . . . . .
10. Start a new OS session. If necessary, modify the .profile to run .env
$ . ./.profile
11. Remove obsolete files
$ cd $APPLPTMP <-- make sure $APPLPTMP points to the right folder!
$ rm -f *.tmp
Note: seems the clone script cleans the folder $APPLTMP automatically.
12. Generate autoconfig file for DBA (only do this on ONE node)
$ cd $AD_TOP/bin
$ ls -al ad*mk*
$ perl admkappsutil.pl
Starting the generation of appsutil.zip
Log file located at $INST_TOP/admin/log/MakeAppsUtil_04151607.log
output located at $INST_TOP/admin/out/appsutil.zip
MakeAppsUtil completed successfully.
$ cp -p $INST_TOP/admin/out/appsutil.zip $APPLPTMP
13. Verify apps password before email DBA with the password. And clean CM tables
$ sqlplus apps/passwd
SQL> @cmclean.sql
SQL> EXEC FND_CONC_CLONE.SETUP_CLEAN;
SQL> select * from fnd_nodes;
(This SELECT shell get 0 row returned. After DBA runs adconfig.sh below, it shall have one row.)
Notes: cmclean.sql is an old file but it still works for me in my R12.1.3 instances
REM $Id: cmclean.sql,v 1.4 2001/04/07 15:55:07 pferguso Exp $
14. Ask DBA to run ADCONFIG
$ adconfig.sh contextfile=$ORACLE_HOME/db_ebsqa/appsutil/$CONTEXT_NAME.xml
NOTE: if .xml does not exist on a new instance, generate it first. here is how to generate the .xml file after unzip appsutil.zip on database server :
$ export PERL5LIB=$ORACLE_HOME/appsutil/perl
$ export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$OARCLE_HOME/ctx/lib
$ perl $ORACLE_HOME/appsutil/bin/adbldxml.pl
<enter apps password when prompted>
<enter hostname when prompted>
<enter listener port when prompted>
<enter SID name when prompted>
Output should indicate:
The context file has been created at: $ORACLE_HOME/db_ebsdev/appsutil/EBSDEV_ebsdb1d.xml
(or $ perl adbldxml.pl tier=db appsuser=<APPSuser> )
15. Run script to restore files from backup
$ cd admin_scripts/backups
$ ./resotore_refresh.sh
15. Verify files and make sure two files for site custom
$FND_TOP/fndenv.env
$INST_TOP/ora/10.1.2/forms/server/default.env
Note: two files shall have a section in below format (Otherwise it will be overwritten by autoconfig). Or, modify them after adautocfg.
#Begin Customizations
custom tops
#End Customizations
$ ls -al $INST_TOP/certs/Apache
cwallet.sso ewallet.p12
17. Update $CONTEXT_FILE
- APPLCSF & APPLPTMP on ALL nodes !
- Below ones are ONLY on Form/Web nodes:
webport
activewebport
login
external_url
webentryhost
webentryurlprotocol
Notes: For real port (not as 443 in F5) when SSL is enabled, make sure the port (such as 4472) for s_active_webport match the port in file ssl.conf and s_webssl_port in $CONTEXT_FILE .
18. Run AD autocfg : always make CM node work first (after DBA is done with adconfig!)
$ cd $ADMIN_SCRIPTS_HOME
$ ./adautocfg.sh
Optional: change APPS password using FNDCPASS here, and then run adautocfg.sh again on all nodes.
19. Source .profile to take new env variables after autocfg (or re-login)
$ . ./.profile
20. Start services
21. Login as XML Publisher Admin => Administration => General => replace the path with $APPLPTMP.
22. Update profile "Site%name%"
Check profile option: BNE Debug Log Directory, BNE Upload Staging Directory (then, bounce services)
RRA: Service Prefix (this should be BLANK. Not hard coded !!)
RRA: Enabled (someones say this must be set to YES. Seems not necessary)
Viewer: Text (this should be blank or browser)
23. Modify files for customization. Maybe only on the web node. Such as:
- TEMPLATE FILE : $FND_TOP/admin/template/httpd_conf_1013.tmp
add a section to it
- TEMPLATE FILE : $FND_TOP/admin/template/oracle_apache_conf_1013.tmp
#include "%s_ora_config_home%/10.1.3/Apache/modplsql/conf/plsql.conf"
include "%s_weboh_oh%/Apache/modplsql/conf/plsql.conf" <<-- add this line
- Add APPS password to $IAS_ORACLE_HOME/Apache/modplsql/conf/dads.conf
24. Re-create database links, check DBA_DIRECTORIES (DBA task)
25. Re-link some folders for $CUSTOM_TOP:
$ cd $CUSTOM_TOP/ftp
$ rm in
$ rm out
$ ln -s /path/to/ftp/in in
$ ln -s /path/to/ftp/out out
$ chmod 777 in
$ chmod 777 out
26. Optional: disable production users
-- Doc. ID: 1064798.1. Disable EBS users
DECLARE
-- get user list which should be disabled
cc number :=0;
cursor c1 is
select user_name
from fnd_user
where (end_date is null or end_date > sysdate)
-- access all users that has no end-date or end-date is future
and (user_name like 'B%' or user_name like 'M%')
and user_name not in ('SYSADMIN', 'USERVIP1', 'USERVIP2', 'A999999'); -- excluding VIPs
BEGIN
for c in c1 loop
-- disable user
fnd_user_pkg.disableuser(c.user_name);
cc := cc+1;
If ( MOD(cc, 50) = 0) then
commit;
end if;
end loop;
dbms_output.put_line('Totla disabled ' || cc);
commit; --commit changes
END;
/
++++++++ Troubleshooting +++++++
1. After CM node is up in the first time, run a CM job got error when opening the log
An error occurred while attempting to establish an Applications File Server connection with the
node FNDFS_CMHOST. There may be a network configuration problem, or the TNS listener
on node FNDFS_CMHOST may not be running. Please contact your system administrator.
you can confirm the problem by running below command on Web Host:
$ tnsping FNDFS_CMHOST
TNS-12547: TNS:lost contact
after host names were deleted by "EXEC FND_CONC_CLONE.SETUP_CLEAN" from database tables and when autoconfig was executed first on Concurrent Host CMhost, web host names are not saved to database tables yet and so tnsnames.ora file on CMhost does not have entries for web hosts.
Fix: two ways to fix it
1). on CM host
(a). Replace tnsnames.ora and sqlnet.ora in $TNS_ADMIN with files from the backup before refresh.
(b). adalnctl.sh stop
(c). adalnctl.sh start
Now, on the Web Host server, "tnsping FNDFS_CMHOST" returns "OK"
2). Run autoconfig AGAIN on all hosts, starting with CM host first.
2. When stopping CM services (to run autoconfig again for the new APPS password), 5 processes on
OS level did not stop. See all five of them in the Concurrent Manager:
CONCURRENT MANAGER Actual Target
Output Post Processor 2 0
Workflow Agent Listener Service 1 0
Workflow Mailer Service 1 0
Workflow Document Web Services Service 1 0
I clicked "Terminate" on them, but that did not kill the OS processes. I had to kill them by OS command:
$ kill -12 15524 15529 15544 15548 15555
After I ran autoconfig and started CM services, they showed "Terminted" in GUI Concurrent Manager.
I had to clicked each of them to "Activate" them. They did get started after the clicks !!
Should not try to terminate them in the GUI !!
3. When I ran the autoconfig last on CM node, the login page did not work!!
So, run autoconfig on web/forms node last.
4. After clone script adcfgclone.pl worked, Autoconfig ended with below error in adconfig.log & ApplyAppsTier_11011132.log:
[CVM Error Report]
The following report lists errors encountered during CVM Phase
<filename> <return code where appropriate>
/u03/app/EBSUAT/apps/apps_st/appl/ad/12.0.0/bin/adgentns.pl 2
Updating s_tnsmode to 'generateTNS'
UpdateContext exited with status: 0
AC-50480: Internal error occurred: java.lang.Exception: Error while generating listener.ora.
Error generating tnsnames.ora from the database, temporary tnsnames.ora will be generated using templates
Instantiating Tools tnsnames.ora
Tools tnsnames.ora instantiated
Web tnsnames.ora instantiated
adgentns.pl exiting with status 2
Fix: run below line AGAIN
SQL> EXEC FND_CONC_CLONE.SETUP_CLEAN;
Then, run adconfig.sh on DB host. After that, make sure the DB host name shows up in below query before run adautocfg.sh on Apps tiers:
SQL> select * from fnd_nodes;
~~~~~~~~~~~~~~~~~~ script: backup_4refresh.sh ~~~~~~~~~~~~~~~~~~
#!/bin/ksh
# backup files before the refresh Dec. 2015
cd $HOME/admin_scripts/backups
DT=`date +"%h_%d_%y"` # backup date
echo It will delete old backup files. Continue ? yes or no
read answer
case $answer in
[yY][eE][sS]|[yY])
if [ -n "$CONTEXT_FILE" ]; then
echo 'running'
else
echo 'environment variable $CONTEXT_FILE does not exit. Exit ...'
exit 1;
fi;
if [ -f $CONTEXT_FILE ]; then
echo 'starting copy ...'
else
echo 'CONTEXT file $CONTEXT_FILE does not exist. Exit ...'
exit 1;
fi
file1=`basename $CONTEXT_FILE`
if [ -f $file1 ]; then
mv $file1 $file1'_'$DT # backup date
fi
file2=custom`basename $CONTEXT_FILE .xml`'.env'
if [ -f $APPL_TOP/$file2 ]; then
cp -p $APPL_TOP/$file2 .
fi
rm -rf Apache admin
cp -p $CONTEXT_FILE .
cp -rp $INST_TOP/certs/Apache .
cp -rp $TNS_ADMIN .
cp -p $FND_TOP/fndenv.env .
cp -p $INST_TOP/ora/*2/forms/server/default.env .
cp -p $APPL_TOP/admin/adkeystore.dat .
cp -p $APPL_TOP/admin/adsign.txt .
## more optional files. Need to manually copy them back if needed
cp -p $FND_TOP/admin/template/custom/rwbuilder_conf_1012.tmp .
cp -p $FND_TOP/admin/template/ssl_conf_1013.tmp .
cp -p $FND_TOP/admin/template/httpd_conf_1013.tmp .
cp -p $FND_TOP/admin/template/oracle_apache_conf_1013.tmp .
cp -p $IAS_ORACLE_HOME/Apache/modplsql/conf/dads.conf . echo 'Done with backing up files'
ls -altr $HOME/admin_scripts/backups
exit 0
;;
[nN][oO]|[nN])
echo "No"
;;
*)
echo "Invalid input..."
exit 1
;;
esac
~~~~~~~~~~~~~~~~~~ scripts: restore_refresh.sh ~~~~~~~~~~~~~~~~~~~
#!/bin/ksh
# restore files AFTER clone script during the refresh Dec. 2015
cd $HOME/admin_scripts/backups
DT=`date +"%h_%d_%y"`
echo It will overwrite some files. Continue? yes or no
read answer
case $answer in
[yY][eE][sS]|[yY])
if [ -n "$CONTEXT_FILE" ]; then
echo 'running'
else
echo 'environment variable $CONTEXT_FILE does not exit. Exit ...'
exit 1;
fi;
file2=custom`basename $CONTEXT_FILE .xml`'.env'
if [ -f $file2 ]; then
cp -p $file2 $APPL_TOP/.
fi
mv $FND_TOP/fndenv.env $FND_TOP/fndenv.env_clone
cp -p fndenv.env $FND_TOP/.
mv $INST_TOP/ora/10.1.2/forms/server/default.env $INST_TOP/ora/10.1.2/forms/server/default.env_clone
cp -p default.env $INST_TOP/ora/10.1.2/forms/server/.
mv $INST_TOP/certs/Apache $INST_TOP/certs/Apache'_'$DT
cp -pr Apache $INST_TOP/certs/.
## files for JRE 1.8. They will be used if Java signing is re-ran.
mv $APPL_TOP/admin/adkeystore.dat $APPL_TOP/admin/adkeystore.dat_clone
cp -p adkeystore.dat $APPL_TOP/admin/.
mv $APPL_TOP/admin/adsign.txt $APPL_TOP/admin/adsign.txt_clone
cp -p adsign.txt $APPL_TOP/admin/.
echo 'Done with restoring files.'
;;
[nN][oO]|[nN])
echo "No"
;;
*)
echo "Invalid input..."
exit 1
;;
esac
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The first task of EBS refresh is to refresh the database. Here are the steps:
0. Stop EBS apps
1. In the source database EBSPROD (in 10G and 11G)
1.1 Identify the SCN nmuber for the RMAN script. Check source database backup log to find the sequence.
1.2 SQL> select first_change#, next_change#, first_time from v$log_history
where sequence# = 11047;
2. In the target database EBSQA
2.1 Shutdown the target database
SQL> alter database backup controlfile to trace;
SQL> shutdown immediate;
2.2 Drop the databae
SQL> startup mount restrict;
SQL> drop database;
2.3 Get ready for the RMAN restore
SQL> startup nomount;
2.4 Make sure host has enough disk space
2.5 Assume the source database was backed up by RMAN. Run the script
$ cat db_refresh.sh
ORACLE_SID=targetDB_ID
export ORACLE_SID
export TNS_ADMIN=$ORACLE_HOME/network/admin
ORACLE_USER=oracle
NLS_LANG=AMERICAN_AMERICA.AL32UTF8
NLS_DATE_FORMAT='YYYY-MM-DD hh24:mi:ss'
export NLS_LANG
export NLS_DATE_FORMAT
NB_ORA_CLASS=sourceDBserver
export NB_ORA_CLASS
DATE=`date +"%h%d%y_%H%M"`
rman catalog rman/rmanPWD@rmanDB_ID target sys/PSWD@sourceDB_ID auxiliary / msglog db_refresh.${DATE}.log << EOF
run {
ALLOCATE AUXILIARY CHANNEL ch00 TYPE 'SBT_TAPE';
ALLOCATE AUXILIARY CHANNEL ch01 TYPE 'SBT_TAPE';
ALLOCATE AUXILIARY CHANNEL ch02 TYPE 'SBT_TAPE';
SEND 'NB_ORA_CLIENT=sourceDBserver';
SET NEWNAME FOR TEMPFILE 1 to '/path/to/temp01.dbf';
SET NEWNAME FOR TEMPFILE 2 to '/path/to/temp02.dbf';
duplicate target database to targetDB_ID
UNTIL scn = 5968220384735;
}
EOF
$ ./db_refresh.sh &
3. Bring the database up.
Do NOT change any password (except SYSTEM).
1. On source nodes, run PERL script.
$ cd $ADMIN_SCRIPTS_HOME
$ perl adpreclone.pl appsTier <-- No password is needed
NOTE: Pre-clone log file located at $INST_TOP/admin/log/StageAppsTier_MMDDHHMM.log
$ cd $COMMON_TOP/clone <-- to verify new folders are created
$ ls -altr $COMMON_TOP/clone
total 40
drwxr-xr-x 2 applmgr appsuser 4096 Jan 22 2008 html
drwxr-xr-x 3 applmgr appsuser 4096 Jan 22 2008 context
drwxr-xr-x 2 applmgr appsuser 4096 Jan 22 2008 bin
drwxr-xr-x 9 applmgr appsuser 4096 Sep 15 2013 ..
drwxr-xr-x 5 applmgr appsuser 4096 Dec 1 10:06 appsts
drwx------ 7 applmgr appsuser 4096 Dec 1 10:07 jre
drwxr-xr-x 3 applmgr appsuser 4096 Dec 1 10:07 oui
drwxr-xr-x 4 applmgr appsuser 4096 Dec 1 10:07 jlib
drwxr-xr-x 5 applmgr appsuser 4096 Dec 1 10:07 appl
drwxr-xr-x 10 applmgr appsuser 4096 Dec 1 10:07 .
Do the same on other source hosts.
2. On target nodes, backup filess
Create a folder, such as /u01/app/admin_scripts/backups, to hold backed-up files. Then run the script backup_4refresh.sh to back up <SID_HOST> specific files.
$ cd admin_scripts/backups
$ ./backup_4refresh.sh
Confirm necessary files are just backed up:
$ ls -al /u01/app/admin_scripts/backups
$ ls -al /u01/app/admin_scripts/backups/admin
$ ls -al /u01/app/admin_scripts/backups/Apache
Optional: Backup the entire folder to aother partition
$ cd /u01/app
$ tar -zhcvf /u02/app/$CONTEXT_NAME_backup_MMDDYY.tar.gz EBSQA >> backup_MMDD.log
Do the same on other hosts.
3. Make sure all apps services were stopped and database refresh was completed.
Now, remove directories
$ cd /u01/app
$ rm -rf EBSQA
$ rm -rf oraEBSQAinventory; mkdir oraEBSQAinventory
4. Copy directories from source node.
NOTE: copy CM host files to CM host, web/forms host to web/forms host.
$ cd /u01/app
$ rsync --progress -avze ssh applmgr@source_host:/u06/app/EBSPROD /u01/app
enter password
sent 12619428 bytes received 9649138852 bytes 1312649.72 bytes/sec
total size is 33018009660 speedup is 3.42
It may give warning message:
rsync warning: some files vanished before they could be transferred (code 24) at main.c(1532) [generator=3.0.6]
5. Rename the folder and verify the size
$ cd /u01/app
$ mv EBSPROD EBSQA
$ ls -al EBSQAtotal 16
drwxr-xr-x 4 applmgr appsuser 4096 May 7 2013 .
drwxr-xr-x 16 applmgr appsuser 4096 Apr 15 11:27 ..
drwxr-xr-x 4 applmgr appsuser 4096 May 2 2013 apps
drwxr-xr-x 3 applmgr appsuser 4096 May 2 2013 inst
$cd apps
$ du -ks
31640672 .
$ cd ../inst
$ du -ks
2502420 .
6. Edit /etc/oraInst.loc file to make sure below line is used:
inventory_loc=/u01/app/oraEBSQAInventory
7. Start a new OS session (without running the .env file)
If you do not modify the .profile, re-login will get message:
$APPL_TOP/${CONTEXT_NAME}.env: cannot open [No such file or directory]
Note: by this time, the .env file was deleted when old folder EBSQA was removed in Step 5.
So env vairables do not exist
8. Go to /u01/app/EBSQA/apps/apps_st/comn/clone/bin ($COMMON_TOP/clone/bin)
option: delete old files in /u01/app/EBSQA/apps/apps_st/comn/clone
$ cd /u01/app/EBS/apps/apps_st/comn/clone
$ rm -rf jre_BAK_*
Now, get below answers ready -
Database Server Node => dbhost1q.domain.com
database SID => EBSQA
apps => appsPWD
Base Directory => /u01/app/EBSQA
PORT => 1542
port pool: 21 (1542 -1521)
9. Run clone perl script and answer questions
$ perl adcfgclone.pl appsTier <<-- run this in CM node first!!
++++++++++++++++ answers on CM node ++++++++++++++++
Target System Root Service [disabled] : <-- for CM node. WEB/Form node will be opposite
Target System Web Entry Point Services [disabled] :
Target System Web Application Services [disabled] :
Target System Batch Processing Services [enabled] :
Target System Other Services [enabled] : <-- for CM node
Do you want to preserve the Display [y] (y/n) : n
Target System Display [host1q:0.0] : host1q.domain.com:21.0 <-- OAM will use it for display
Do you want the the target system to have the same port values as the source system (y/n) [y] ? : n
Target System Port Pool [0-99] : 21
++++++++++++++++ answers on Form/Web node ++++++++++++++++
Target System Root Service [enabled] : <= for Form node
Target System Web Entry Point Services [enabled] :
Target System Web Application Services [enabled] :
Target System Batch Processing Services [enabled] : disabled <= for Form node
Target System Other Services [enabled] : disabled
Do you want to preserve the Display [xxx_name:0.0] (y/n) : n
Target System Display [host2q:0.0] : host2q.domain.com:21.0 <= OAM will use it for display
Do you want the the target system to have the same port values as the source system (y/n) [y] ? : n
Target System Port Pool [0-99] : 21
The script may remain silent for a few minutes (or even 40 minutes) on executing $INST_TOP/admin/install/txkWfClone.sql !
(only statement SELECT * from v$session where upper(machine) like 'HOSTNAME%'; can tell if the sql script is running).
Answer "no" - Dot not start apps services after autoconfig completed.
Check the log file. Any more lines than below need your attention or fix:
$ egrep -i 'fail|error|ora-' $INST_TOP/admin/log/ApplyAppsTier_04151536.log
Error while running adlnktools.sh.
ERRORCODE = 0 ERRORCODE_END
ERRORCODE = 0 ERRORCODE_END
AC-50480: Internal error occurred: java.lang.Exception: Error while generating listener.ora.
Error generating tnsnames.ora from the database, temporary tnsnames.ora will be generated using templates
ERRORCODE = 2 ERRORCODE_END
Result : FAILED
ERRORCODE = 0 ERRORCODE_END
Unique constraint error (00001) is OK if key already exists
ERRORCODE = 0 ERRORCODE_END
ERRORCODE = 0 ERRORCODE_END
. . . . . . . . .
10. Start a new OS session. If necessary, modify the .profile to run .env
$ . ./.profile
11. Remove obsolete files
$ cd $APPLPTMP <-- make sure $APPLPTMP points to the right folder!
$ rm -f *.tmp
Note: seems the clone script cleans the folder $APPLTMP automatically.
12. Generate autoconfig file for DBA (only do this on ONE node)
$ cd $AD_TOP/bin
$ ls -al ad*mk*
$ perl admkappsutil.pl
Starting the generation of appsutil.zip
Log file located at $INST_TOP/admin/log/MakeAppsUtil_04151607.log
output located at $INST_TOP/admin/out/appsutil.zip
MakeAppsUtil completed successfully.
$ cp -p $INST_TOP/admin/out/appsutil.zip $APPLPTMP
13. Verify apps password before email DBA with the password. And clean CM tables
$ sqlplus apps/passwd
SQL> @cmclean.sql
SQL> EXEC FND_CONC_CLONE.SETUP_CLEAN;
SQL> select * from fnd_nodes;
(This SELECT shell get 0 row returned. After DBA runs adconfig.sh below, it shall have one row.)
Notes: cmclean.sql is an old file but it still works for me in my R12.1.3 instances
REM $Id: cmclean.sql,v 1.4 2001/04/07 15:55:07 pferguso Exp $
14. Ask DBA to run ADCONFIG
$ adconfig.sh contextfile=$ORACLE_HOME/db_ebsqa/appsutil/$CONTEXT_NAME.xml
NOTE: if .xml does not exist on a new instance, generate it first. here is how to generate the .xml file after unzip appsutil.zip on database server :
$ export PERL5LIB=$ORACLE_HOME/appsutil/perl
$ export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$OARCLE_HOME/ctx/lib
$ perl $ORACLE_HOME/appsutil/bin/adbldxml.pl
<enter apps password when prompted>
<enter hostname when prompted>
<enter listener port when prompted>
<enter SID name when prompted>
Output should indicate:
The context file has been created at: $ORACLE_HOME/db_ebsdev/appsutil/EBSDEV_ebsdb1d.xml
(or $ perl adbldxml.pl tier=db appsuser=<APPSuser> )
15. Run script to restore files from backup
$ cd admin_scripts/backups
$ ./resotore_refresh.sh
15. Verify files and make sure two files for site custom
$FND_TOP/fndenv.env
$INST_TOP/ora/10.1.2/forms/server/default.env
Note: two files shall have a section in below format (Otherwise it will be overwritten by autoconfig). Or, modify them after adautocfg.
#Begin Customizations
custom tops
#End Customizations
$ ls -al $INST_TOP/certs/Apache
cwallet.sso ewallet.p12
17. Update $CONTEXT_FILE
- APPLCSF & APPLPTMP on ALL nodes !
- Below ones are ONLY on Form/Web nodes:
webport
activewebport
login
external_url
webentryhost
webentryurlprotocol
Notes: For real port (not as 443 in F5) when SSL is enabled, make sure the port (such as 4472) for s_active_webport match the port in file ssl.conf and s_webssl_port in $CONTEXT_FILE .
18. Run AD autocfg : always make CM node work first (after DBA is done with adconfig!)
$ cd $ADMIN_SCRIPTS_HOME
$ ./adautocfg.sh
Optional: change APPS password using FNDCPASS here, and then run adautocfg.sh again on all nodes.
19. Source .profile to take new env variables after autocfg (or re-login)
$ . ./.profile
20. Start services
21. Login as XML Publisher Admin => Administration => General => replace the path with $APPLPTMP.
22. Update profile "Site%name%"
Check profile option: BNE Debug Log Directory, BNE Upload Staging Directory (then, bounce services)
RRA: Service Prefix (this should be BLANK. Not hard coded !!)
RRA: Enabled (someones say this must be set to YES. Seems not necessary)
Viewer: Text (this should be blank or browser)
23. Modify files for customization. Maybe only on the web node. Such as:
- TEMPLATE FILE : $FND_TOP/admin/template/httpd_conf_1013.tmp
add a section to it
- TEMPLATE FILE : $FND_TOP/admin/template/oracle_apache_conf_1013.tmp
#include "%s_ora_config_home%/10.1.3/Apache/modplsql/conf/plsql.conf"
include "%s_weboh_oh%/Apache/modplsql/conf/plsql.conf" <<-- add this line
- Add APPS password to $IAS_ORACLE_HOME/Apache/modplsql/conf/dads.conf
24. Re-create database links, check DBA_DIRECTORIES (DBA task)
25. Re-link some folders for $CUSTOM_TOP:
$ cd $CUSTOM_TOP/ftp
$ rm in
$ rm out
$ ln -s /path/to/ftp/in in
$ ln -s /path/to/ftp/out out
$ chmod 777 in
$ chmod 777 out
26. Optional: disable production users
-- Doc. ID: 1064798.1. Disable EBS users
DECLARE
-- get user list which should be disabled
cc number :=0;
cursor c1 is
select user_name
from fnd_user
where (end_date is null or end_date > sysdate)
-- access all users that has no end-date or end-date is future
and (user_name like 'B%' or user_name like 'M%')
and user_name not in ('SYSADMIN', 'USERVIP1', 'USERVIP2', 'A999999'); -- excluding VIPs
BEGIN
for c in c1 loop
-- disable user
fnd_user_pkg.disableuser(c.user_name);
cc := cc+1;
If ( MOD(cc, 50) = 0) then
commit;
end if;
end loop;
dbms_output.put_line('Totla disabled ' || cc);
commit; --commit changes
END;
/
++++++++ Troubleshooting +++++++
1. After CM node is up in the first time, run a CM job got error when opening the log
An error occurred while attempting to establish an Applications File Server connection with the
node FNDFS_CMHOST. There may be a network configuration problem, or the TNS listener
on node FNDFS_CMHOST may not be running. Please contact your system administrator.
you can confirm the problem by running below command on Web Host:
$ tnsping FNDFS_CMHOST
TNS-12547: TNS:lost contact
after host names were deleted by "EXEC FND_CONC_CLONE.SETUP_CLEAN" from database tables and when autoconfig was executed first on Concurrent Host CMhost, web host names are not saved to database tables yet and so tnsnames.ora file on CMhost does not have entries for web hosts.
Fix: two ways to fix it
1). on CM host
(a). Replace tnsnames.ora and sqlnet.ora in $TNS_ADMIN with files from the backup before refresh.
(b). adalnctl.sh stop
(c). adalnctl.sh start
Now, on the Web Host server, "tnsping FNDFS_CMHOST" returns "OK"
2). Run autoconfig AGAIN on all hosts, starting with CM host first.
2. When stopping CM services (to run autoconfig again for the new APPS password), 5 processes on
OS level did not stop. See all five of them in the Concurrent Manager:
CONCURRENT MANAGER Actual Target
Output Post Processor 2 0
Workflow Agent Listener Service 1 0
Workflow Mailer Service 1 0
Workflow Document Web Services Service 1 0
I clicked "Terminate" on them, but that did not kill the OS processes. I had to kill them by OS command:
$ kill -12 15524 15529 15544 15548 15555
After I ran autoconfig and started CM services, they showed "Terminted" in GUI Concurrent Manager.
I had to clicked each of them to "Activate" them. They did get started after the clicks !!
Should not try to terminate them in the GUI !!
3. When I ran the autoconfig last on CM node, the login page did not work!!
So, run autoconfig on web/forms node last.
4. After clone script adcfgclone.pl worked, Autoconfig ended with below error in adconfig.log & ApplyAppsTier_11011132.log:
[CVM Error Report]
The following report lists errors encountered during CVM Phase
<filename> <return code where appropriate>
/u03/app/EBSUAT/apps/apps_st/appl/ad/12.0.0/bin/adgentns.pl 2
Updating s_tnsmode to 'generateTNS'
UpdateContext exited with status: 0
AC-50480: Internal error occurred: java.lang.Exception: Error while generating listener.ora.
Error generating tnsnames.ora from the database, temporary tnsnames.ora will be generated using templates
Instantiating Tools tnsnames.ora
Tools tnsnames.ora instantiated
Web tnsnames.ora instantiated
adgentns.pl exiting with status 2
Fix: run below line AGAIN
SQL> EXEC FND_CONC_CLONE.SETUP_CLEAN;
Then, run adconfig.sh on DB host. After that, make sure the DB host name shows up in below query before run adautocfg.sh on Apps tiers:
SQL> select * from fnd_nodes;
~~~~~~~~~~~~~~~~~~ script: backup_4refresh.sh ~~~~~~~~~~~~~~~~~~
#!/bin/ksh
# backup files before the refresh Dec. 2015
cd $HOME/admin_scripts/backups
DT=`date +"%h_%d_%y"` # backup date
echo It will delete old backup files. Continue ? yes or no
read answer
case $answer in
[yY][eE][sS]|[yY])
if [ -n "$CONTEXT_FILE" ]; then
echo 'running'
else
echo 'environment variable $CONTEXT_FILE does not exit. Exit ...'
exit 1;
fi;
if [ -f $CONTEXT_FILE ]; then
echo 'starting copy ...'
else
echo 'CONTEXT file $CONTEXT_FILE does not exist. Exit ...'
exit 1;
fi
file1=`basename $CONTEXT_FILE`
if [ -f $file1 ]; then
mv $file1 $file1'_'$DT # backup date
fi
file2=custom`basename $CONTEXT_FILE .xml`'.env'
if [ -f $APPL_TOP/$file2 ]; then
cp -p $APPL_TOP/$file2 .
fi
rm -rf Apache admin
cp -p $CONTEXT_FILE .
cp -rp $INST_TOP/certs/Apache .
cp -rp $TNS_ADMIN .
cp -p $FND_TOP/fndenv.env .
cp -p $INST_TOP/ora/*2/forms/server/default.env .
cp -p $APPL_TOP/admin/adkeystore.dat .
cp -p $APPL_TOP/admin/adsign.txt .
## more optional files. Need to manually copy them back if needed
cp -p $FND_TOP/admin/template/custom/rwbuilder_conf_1012.tmp .
cp -p $FND_TOP/admin/template/ssl_conf_1013.tmp .
cp -p $FND_TOP/admin/template/httpd_conf_1013.tmp .
cp -p $FND_TOP/admin/template/oracle_apache_conf_1013.tmp .
cp -p $IAS_ORACLE_HOME/Apache/modplsql/conf/dads.conf . echo 'Done with backing up files'
ls -altr $HOME/admin_scripts/backups
exit 0
;;
[nN][oO]|[nN])
echo "No"
;;
*)
echo "Invalid input..."
exit 1
;;
esac
~~~~~~~~~~~~~~~~~~ scripts: restore_refresh.sh ~~~~~~~~~~~~~~~~~~~
#!/bin/ksh
# restore files AFTER clone script during the refresh Dec. 2015
cd $HOME/admin_scripts/backups
DT=`date +"%h_%d_%y"`
echo It will overwrite some files. Continue? yes or no
read answer
case $answer in
[yY][eE][sS]|[yY])
if [ -n "$CONTEXT_FILE" ]; then
echo 'running'
else
echo 'environment variable $CONTEXT_FILE does not exit. Exit ...'
exit 1;
fi;
file2=custom`basename $CONTEXT_FILE .xml`'.env'
if [ -f $file2 ]; then
cp -p $file2 $APPL_TOP/.
fi
mv $FND_TOP/fndenv.env $FND_TOP/fndenv.env_clone
cp -p fndenv.env $FND_TOP/.
mv $INST_TOP/ora/10.1.2/forms/server/default.env $INST_TOP/ora/10.1.2/forms/server/default.env_clone
cp -p default.env $INST_TOP/ora/10.1.2/forms/server/.
mv $INST_TOP/certs/Apache $INST_TOP/certs/Apache'_'$DT
cp -pr Apache $INST_TOP/certs/.
## files for JRE 1.8. They will be used if Java signing is re-ran.
mv $APPL_TOP/admin/adkeystore.dat $APPL_TOP/admin/adkeystore.dat_clone
cp -p adkeystore.dat $APPL_TOP/admin/.
mv $APPL_TOP/admin/adsign.txt $APPL_TOP/admin/adsign.txt_clone
cp -p adsign.txt $APPL_TOP/admin/.
echo 'Done with restoring files.'
;;
[nN][oO]|[nN])
echo "No"
;;
*)
echo "Invalid input..."
exit 1
;;
esac
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The first task of EBS refresh is to refresh the database. Here are the steps:
0. Stop EBS apps
1. In the source database EBSPROD (in 10G and 11G)
1.1 Identify the SCN nmuber for the RMAN script. Check source database backup log to find the sequence.
1.2 SQL> select first_change#, next_change#, first_time from v$log_history
where sequence# = 11047;
2. In the target database EBSQA
2.1 Shutdown the target database
SQL> alter database backup controlfile to trace;
SQL> shutdown immediate;
2.2 Drop the databae
SQL> startup mount restrict;
SQL> drop database;
2.3 Get ready for the RMAN restore
SQL> startup nomount;
2.4 Make sure host has enough disk space
2.5 Assume the source database was backed up by RMAN. Run the script
$ cat db_refresh.sh
ORACLE_SID=targetDB_ID
export ORACLE_SID
export TNS_ADMIN=$ORACLE_HOME/network/admin
ORACLE_USER=oracle
NLS_LANG=AMERICAN_AMERICA.AL32UTF8
NLS_DATE_FORMAT='YYYY-MM-DD hh24:mi:ss'
export NLS_LANG
export NLS_DATE_FORMAT
NB_ORA_CLASS=sourceDBserver
export NB_ORA_CLASS
DATE=`date +"%h%d%y_%H%M"`
rman catalog rman/rmanPWD@rmanDB_ID target sys/PSWD@sourceDB_ID auxiliary / msglog db_refresh.${DATE}.log << EOF
run {
ALLOCATE AUXILIARY CHANNEL ch00 TYPE 'SBT_TAPE';
ALLOCATE AUXILIARY CHANNEL ch01 TYPE 'SBT_TAPE';
ALLOCATE AUXILIARY CHANNEL ch02 TYPE 'SBT_TAPE';
SEND 'NB_ORA_CLIENT=sourceDBserver';
SET NEWNAME FOR TEMPFILE 1 to '/path/to/temp01.dbf';
SET NEWNAME FOR TEMPFILE 2 to '/path/to/temp02.dbf';
duplicate target database to targetDB_ID
UNTIL scn = 5968220384735;
}
EOF
$ ./db_refresh.sh &
3. Bring the database up.
Do NOT change any password (except SYSTEM).
EBS apps RDA
Two types of RDA on Oracle EBS apps.
1. Data collection test
- Run it for R12.1 from browser (Doc ID 732091.1)
$ cd $IZU_TOP/bin
$ ./rda.sh -vdCRP -e APPL _SHORT='AR' ACT
If getting error "RDA-00001: Cannot change to the output directory", rename the setup.cfg file in the directory and try it again.
2. Download file p20603454_8715310_Linux-x86-64.zip from Oracle support site and install it from unzip. Then
$ cd rda
$ ./rda.sh -T hcve
1. Data collection test
- Run it for R12.1 from browser (Doc ID 732091.1)
- Log onto Oracle E-Business Suite
- Click on responsibility Application Diagnostics
- Select the Diagnose menu option
- Click button Select Application and select an appropriate Application, such as payables (SQLAP), receivables (AR)
- Scroll down to group "System Snapshot"
- Select test name "RDA"
- Input Parameters (* required)
- Responsibility Id (LOV) * (seems System Administrator works)
- Application Shortname (LOV) * (such as AR, XLA)
- Mask sensitive data (LOV) * (seems optional)
- APPS Schema Username
- APPS Schema Password
$ cd $IZU_TOP/bin
$ ./rda.sh -vdCRP -e APPL _SHORT='AR' ACT
If getting error "RDA-00001: Cannot change to the output directory", rename the setup.cfg file in the directory and try it again.
2. Download file p20603454_8715310_Linux-x86-64.zip from Oracle support site and install it from unzip. Then
$ cd rda
$ ./rda.sh -T hcve
Monday, November 23, 2015
Run scp or sftp Without a Password
To automate a job for exchanging data files between two servers, we want to run it without entering a password by human being. Assume there are two servers local2d (with users batchdev and batchmgr) and remote1d (with user user01).
A pair of key files is required: id_rsa resides on the local server, id_rsa.pub is imported to remote server.
A. if a passphase was not entered for "ssh-keygen"
1. Generate the key files on the local server local2d
batchdev@local2d: /u06/app
$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/u06/app/.ssh/id_rsa): <Hit enter> <== option: another path/name
Enter passphrase (empty for no passphrase): <Hit enter> <== leave it empty
Enter same passphrase again: <Hit enter>
Your identification has been saved in /u06/app/.ssh/id_rsa.
Your public key has been saved in /u06/app/.ssh/id_rsa.pub.
The key fingerprint is:
66:38:f8:65:74:5d:10:88:f9:9a:3e:89:4a:d5:0c:49 batchdev@local2d.domain.com
The key's randomart image is:
+--[ RSA 2048]----+
| E o .oo. |
| . .o .. . |
| o ... . |
| . * .. |
| . + So |
| o *o |
| . .o . |
| . . + |
| .. . |
+-----------------+
NOTES: If run "$ ssh-keygen -t dsa", it will generate two files with names id_dsa and id_dsa.phb. More options, such as:
$ ssh-keygen -f $HOME/.ssh/ppm -t rsa -b 4096 -P "" -C "PPM Key"
batchdev@locald2d: /u06/app
$ cd .ssh
$ ls -al
-rw------- 1 batchdev users 1675 Oct 13 11:46 id_rsa
-rw-r--r-- 1 batchdev users 410 Oct 13 11:46 id_rsa.pub
-rw-r--r-- 1 batchdev users 1622 Aug 6 10:50 known_hosts
2. On the remote server remote1d
$ hostname
remote1d.domain.com
$ whoami
user01
## copy the pub key to remote1d (from server local2d).
$ cd /u04/apps/.ssh
$ scp -p batchdev@local2d:/u06/app/.ssh/id_rsa.pub local2d.key
batchdev@local2d's password: xxxxx
$ mv local2d.key authorized_keys <== do not overwrite the file!
## Note if the file exists, run "$ cat local2d.key >> authorized_keys"
$ chmod 600 authorized_keys
$ ls -al /u04/app/.ssh
-rw------- 1 user01 users 410 Oct 13 11:46 authorized_keys
-rw-r--r-- 1 user01 users 2087 Mar 30 2015 known_hosts
A. if a passphase was not entered for "ssh-keygen"
1. Generate the key files on the local server local2d
batchdev@local2d: /u06/app
$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/u06/app/.ssh/id_rsa): <Hit enter> <== option: another path/name
Enter passphrase (empty for no passphrase): <Hit enter> <== leave it empty
Enter same passphrase again: <Hit enter>
Your identification has been saved in /u06/app/.ssh/id_rsa.
Your public key has been saved in /u06/app/.ssh/id_rsa.pub.
The key fingerprint is:
66:38:f8:65:74:5d:10:88:f9:9a:3e:89:4a:d5:0c:49 batchdev@local2d.domain.com
The key's randomart image is:
+--[ RSA 2048]----+
| E o .oo. |
| . .o .. . |
| o ... . |
| . * .. |
| . + So |
| o *o |
| . .o . |
| . . + |
| .. . |
+-----------------+
NOTES: If run "$ ssh-keygen -t dsa", it will generate two files with names id_dsa and id_dsa.phb. More options, such as:
$ ssh-keygen -f $HOME/.ssh/ppm -t rsa -b 4096 -P "" -C "PPM Key"
batchdev@locald2d: /u06/app
$ cd .ssh
$ ls -al
-rw------- 1 batchdev users 1675 Oct 13 11:46 id_rsa
-rw-r--r-- 1 batchdev users 410 Oct 13 11:46 id_rsa.pub
-rw-r--r-- 1 batchdev users 1622 Aug 6 10:50 known_hosts
2. On the remote server remote1d
Task: Import the key from file id_rsa.pub of local machine to file authorized_keys of the remove machine.
$ hostname
remote1d.domain.com
$ whoami
user01
## copy the pub key to remote1d (from server local2d).
$ cd /u04/apps/.ssh
$ scp -p batchdev@local2d:/u06/app/.ssh/id_rsa.pub local2d.key
batchdev@local2d's password: xxxxx
$ mv local2d.key authorized_keys <== do not overwrite the file!
## Note if the file exists, run "$ cat local2d.key >> authorized_keys"
$ chmod 600 authorized_keys
$ ls -al /u04/app/.ssh
-rw------- 1 user01 users 410 Oct 13 11:46 authorized_keys
-rw-r--r-- 1 user01 users 2087 Mar 30 2015 known_hosts
3. Test SFTP and SSH on local server local2d
batchdev@local2d: /u06/app/
$ sftp remote1d ## atchdev can not get onto it. sftp asks the password
Connecting to remote1d...
Password:
$ sftp user01@remote1d ## But user01 can login to remote1d w/o a password!
Connecting to remote1d...
Password:
$ sftp user01@remote1d ## But user01 can login to remote1d w/o a password!
Connecting to remote1d...
sftp> dir -all
-rw-r--r-- 1 user01 users 481 Jun 22 2010 .profile
-rw------- 1 user01 users 933 Oct 13 12:35 .sh_history
drwx------ 2 user01 users 096 Oct 13 12:27 .ssh
sftp> quit
In a script, use SFTP syntax to connect to remote server without a password after file id_rsa is created (on local server):
sftp> dir -all
-rw-r--r-- 1 user01 users 481 Jun 22 2010 .profile
-rw------- 1 user01 users 933 Oct 13 12:35 .sh_history
drwx------ 2 user01 users 096 Oct 13 12:27 .ssh
sftp> quit
In a script, use SFTP syntax to connect to remote server without a password after file id_rsa is created (on local server):
$ sftp -oidentityFile=/path/to/id_rsa userID@remoteServer.domain.com
Use "ssh -i /path/to/id_rsa ... " and "sftp -oidentityFile=/path/to/id_rsa ... " on local server to make connect to a remote server if private key id_rsa is saved in folder /path/to/ (vs. the default location $HOME/.ssh) of the local server. Key in id_rsa.pub is still needed by the remote server.
SSH to remote1s as user01 also works:
batchdev@local2d: /u06/app
$ ssh user01@remote1d
$ hostname
remote1d.domain.com
$ exit
batchdev@local2d: /u06/app
$ scp user01@remote1d:/u04/app/cert.txt . ## even SCP does not ask for the pwd
cert.txt 100% 1383 1.4KB/s 00:00
TROUBLESHOOTING: permission on /u06/app and /u04/app should be 755 or 700. 777 will make passwordless authentication not work. Needs more restrictive.
If it gives below warnings:
B. If there is a 2nd user batchMgr on server local2d wants to run sftp into remote1d, additional steps are necessary:
1. Login to local2d server with batchMgr account
2. Create a new directory apps_sftp_key and do a chmod 700 on this directory
3. Copy files id_rsa and id_rsa.pub from ~batchdev/.ssh to apps_sftp_key directory and do a chmod 600 on id_rsa
$ hostname
local2d
$ whoami
batchmgr
$ pwd
/home/batchmgr/apps_sftp_key
$ ls -al
total 16
drwx------ 2 batchmgr 1211 4096 Sep 16 14:52 .
drwxrwxr-x 12 batchmgr 1211 4096 Sep 19 12:23 ..
-rw------- 1 batchmgr users 1675 Oct 13 11:46 id_rsa
-rw-r--r-- 1 batchmgr users 410 Oct 13 11:46 id_rsa.pub
One line to run SFTP without interactive:
$ ssh user01@remote1d
$ hostname
remote1d.domain.com
$ exit
batchdev@local2d: /u06/app
$ scp user01@remote1d:/u04/app/cert.txt . ## even SCP does not ask for the pwd
cert.txt 100% 1383 1.4KB/s 00:00
TROUBLESHOOTING: permission on /u06/app and /u04/app should be 755 or 700. 777 will make passwordless authentication not work. Needs more restrictive.
If it gives below warnings:
$ ssh user01@remote1d
The authenticity of host 'remote1d (167.xx.xx.xxx)' can't be established.
ECDSA key fingerprint is SHA256:r4ofQUP1F8ebW5hWCcCsK7ah...A.
ECDSA key fingerprint is MD5:49:86:cd:21:a3:a4:22:05:68:4a:0e:...:35.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'remote1d' (ECDSA) to the list of known hosts.
Warning: the ECDSA host key for 'remote1d' differs from the key for the IP address '167.xx.xx.xxx'
Offending key for IP in /u06/app/.ssh/known_hosts:20
Are you sure you want to continue connecting (yes/no)? yes
First, answer Yes, and delete line 20 from file /u06/app/.ssh/known_hosts (after doing a backup) on local2d.
Then, try ssh to login twice. The warning will go away in the 2nd try.
$ ssh user01@remote1d
Warning: Permanently added the ECDSA host key for IP address '167.xx.xx.xxx' to the list of known hosts.
$ ssh user01@remote1d
Now, no more warnings
B. If there is a 2nd user batchMgr on server local2d wants to run sftp into remote1d, additional steps are necessary:
1. Login to local2d server with batchMgr account
2. Create a new directory apps_sftp_key and do a chmod 700 on this directory
3. Copy files id_rsa and id_rsa.pub from ~batchdev/.ssh to apps_sftp_key directory and do a chmod 600 on id_rsa
$ hostname
local2d
$ whoami
batchmgr
$ pwd
/home/batchmgr/apps_sftp_key
$ ls -al
total 16
drwx------ 2 batchmgr 1211 4096 Sep 16 14:52 .
drwxrwxr-x 12 batchmgr 1211 4096 Sep 19 12:23 ..
-rw------- 1 batchmgr users 1675 Oct 13 11:46 id_rsa
-rw-r--r-- 1 batchmgr users 410 Oct 13 11:46 id_rsa.pub
One line to run SFTP without interactive:
batchmgr@locald2d: $ sftp -oidentityFile=~batchmgr/apps_sftp_key/id_rsa user01@remote1d
Or, more actions in one line:
batchmgr@locald2d: $ sftp -oport=22 -b sftp.ctl -oidentityFile=~batchmgr/apps_sftp_key/id_rsa user01@remote1d:/apps/local/ftp/in
Changing to: /apps/local/ftp/in
sftp> put /home/batchmgr/temp/test.del test.del
Uploading /home/batchmgr/temp/test.del to /apps/local/ftp/in/test.del
sftp> version
SFTP protocol 3
sftp> quit
Here, sftp.ctl is a command file in the current directory:
$ more sftp.ctl
put /home/batchmgr/temp/test.del test.del
version
quit
C. If a passphrase was entered for "ssh-keygen"
$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/u06/app/.ssh/id_rsa): <Hit enter>
Enter passphrase (empty for no passphrase): <== "welcome01" works, but "welcome" not
Enter same passphrase again:<Enter your passphrase again> <== welcome01
After complete steps in part A, logging onto remote machine by ssh will have to enter the passphrase:
batchmgr@locald2d: $ sftp -oport=22 -b sftp.ctl -oidentityFile=~batchmgr/apps_sftp_key/id_rsa user01@remote1d:/apps/local/ftp/in
Changing to: /apps/local/ftp/in
sftp> put /home/batchmgr/temp/test.del test.del
Uploading /home/batchmgr/temp/test.del to /apps/local/ftp/in/test.del
sftp> version
SFTP protocol 3
sftp> quit
Here, sftp.ctl is a command file in the current directory:
$ more sftp.ctl
put /home/batchmgr/temp/test.del test.del
version
quit
C. If a passphrase was entered for "ssh-keygen"
$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/u06/app/.ssh/id_rsa): <Hit enter>
Enter passphrase (empty for no passphrase): <== "welcome01" works, but "welcome" not
Enter same passphrase again:<Enter your passphrase again> <== welcome01
After complete steps in part A, logging onto remote machine by ssh will have to enter the passphrase:
$ ssh user01@remote1d
Enter passphrase for key "/u06/app/.ssh/id_rsa': welcome01
<= Use file id_rsa on local machine
To suppress that, additional steps on ssh agent are needed on the local2d:
batchdev@local2d: /u06/app
$ echo $SHELL
/usr/bin/ksh
$ ssh-agent $SHELL
$ ps -ef | grep ssh-agent
batchdev 118621 174327 0 17:32:17 - 0:00 ssh-agent /usr/bin/ksh
batchdev 161219 174327 0 17:32:28 pts/0 0:00 grep ssh-agent
$ ssh-add
Enter passphrase for /u06/app/.ssh/id_rsa: welcome01
Identity added: /u06/app/.ssh/id_rsa (/u06/app/.ssh/id_rsa)
Now, sftp, ssh and scp shall work without a password or a passphrase.
To suppress that, additional steps on ssh agent are needed on the local2d:
batchdev@local2d: /u06/app
$ echo $SHELL
/usr/bin/ksh
$ ssh-agent $SHELL
$ ps -ef | grep ssh-agent
batchdev 118621 174327 0 17:32:17 - 0:00 ssh-agent /usr/bin/ksh
batchdev 161219 174327 0 17:32:28 pts/0 0:00 grep ssh-agent
$ ssh-add
Enter passphrase for /u06/app/.ssh/id_rsa: welcome01
Identity added: /u06/app/.ssh/id_rsa (/u06/app/.ssh/id_rsa)
Now, sftp, ssh and scp shall work without a password or a passphrase.
UPDATES in 2024:
Linux RHEL8, "sestatus" (or file /etc/selinux/config) can be used to check if SELinux label is enabled or not. If it is enabled, above passwordless setups may become not working. The cause is the labels on file authorized_keys in RHEL 8. Use " ls -alZ " to check it. Or, check my new post on this topic.
Subscribe to:
Comments (Atom)