Apps CPU (Critical Patch Update) patches involve different areas. Normally CPU patches are applied at the same time to EBS apps and two ORACLE_HOMEs. Doc ID 2051000.1 has the instruction for October 2015 CPU patches.
1. Database patches
In my 12.1.0.2 database, database October 2015 PSU
and other patches were applied. But most of them are for other purposes, and are not pre-requisites for EBS Oct 2015 CPU.
Patch Conflict patch
21520444 Combo of 12.1.0.2.5 OJVM PSU and 12.1.0.2.5 DB PSU (Oct 2015)
6880880 Latest OPatch
X 19382851 Already included in 12.1.0.2.5 PSU
X 19393542 Already included in 12.1.0.2.5 PSU
X 19627012 Already included in 12.1.0.2.5 PSU
X 19649152 Already included in 12.1.0.2.5 PSU
20204035 12.1.0.2.0 version. No conflicts
13542050 A mutex related hang with holder around 65534 (0xfffe)
19855835 Upgrade slow when reorganizing large stats history tables
20879889 Open cursor leak from DML on table with a materialized view log
20807398 ORA-600 [kgl-hash-collision] with fix to bug 20465582 installed
21091518 Extend fix of bug 18304693 to Partition Views
2. Patch 21845960 (CPU Oct2015 Fusion Middleware 10.1.3.5)
$ cd $INST_TOP/ora/10.1.3
$ . xxxx.env <== source the env file there
$ echo $ORACLE_HOME <== make sure it points to 10.1.3 IAS_ORACLE_HOME
/path/to/apps/tech_st/10.1.3
$ unzip p21845960_101350_LINUX.zip
$ export OPATCH_PLATFORM_ID=46 <== need it on 64-bit server
$ cd 21845960
$ opatch napply <== take about 15 minutes and roll back previously installed CPU patches
Notes:
(1) After 21845960 was applied, "opatch lsinventory | grep 21845960" will show nothing because it is a CPU patch. But, below line shall show 21845942 (one of the included patches) was installed
$ opatch lsinventory | grep 21845942
(2) Without "export OPATCH_PLATFORM_ID=46" on 64-bit server, opatch exited with error message "NApply failed to apply the following patches "21815758". After that, I tried "opatch napply -skip_duplicate" (I used this option when applied database SPU patch), it says "The input directory could not processed". Then, I went to sub-folder 21815758 and used "opatch apply" to apply it successfully.
3. Patch 21103001
NOTES: This patch replaces CPU Oct2014 Fusion Middleware 10.1.2.3 patch 19434967. See Doc ID 437878.1 CHANGE LOG of Oct 16, 2015.
Start a new OS session or $ . .profile <== run the main env file
$ echo $ORACLE_HOME <== make sure it is the 10.1.2 ORACLE HOME
/path/to/apps/tech_st/10.1.2
(1) p14825718_10123_LINUX.zip (asked by 21103001 README)
Verify conditions are met to apply Patch 14825718 (Step 3.2 notes of Document 437878.1):
- 12881480, 13808590, 14041415, 14262118, 14614795, 14577216, 9593176 were NOT applied.
- 6995251, 7121788 were APPLIED.
$ cd 14825718
$ opatch apply
Note1: README of Bundle Patch is not applicable to E-Business Suite Customers.
Note2: It may rollback some patches. If you get below message, answer "N" to continue:
If you continue, all patches will be rolled back and the new patch (14825718) will be installed.
Do you want to STOP?
Please respond Y|N > N
(2) p6640838_10106_Linux-x86-64.zip (asked by 21103001 README)
This patch 6640838 only added missing components to OUI 10.1.0.6.0, not install a new OUI
Steps with screenshots: Doc ID 1301320.1 How to patch OUI for installing overlay patches on top of Forms Bundle Patch - 9593176
$ mv $ORACLE_HOME/OPatch $ORACLE_HOME/OPatch.pre_6640838
to reserve the higher version of opatch.
$ cd cd/Disk1/install
$ ./runInstaller -ignoreSysPrereqs
$ cd $ORACLE_HOME
$ mv OPatch OPatch_delete
$ mv OPatch.pre_6640838 OPatch
(3) p8551790_10123_LINUX.zip (asked by 21103001 README)
$ cd 8551790
$ opatch apply
(4) p21103001_101232_LINUX.zip
$ cd 21103001
$ opatch apply <== very quick
Note1: Post steps are not needed for EBS. See "Post Steps For Patch 19434967 Files Do Not Exist (Doc ID 1945012.1)"
Note2: If opatch version is older than 1.0.0.0.63, it may give a misleading error message:
The patch directory area must be a number.
ERROR: OPatch failed because of problems in patch area.
4. Run the new adgrants.sql as SYSDBA ( get it from 21507207/admin/adgrants.sql after unzip p21507207_12.1.0_R12_LINUX.zip). Without it, ADFIXUSER.sql will fail.
SQL> @adgrants.sql apps ( <- lower case)
... ... ...
Start of Creating AD_JAR context
PL/SQL procedure successfully completed.
End of Creating AD_JAR context
Creating PL/SQL Package AD_DBMS_METADATA.
Package created.
Package body created.
End of Creating PL/SQL Package AD_DBMS_METADATA.
Creating PL/SQL Package AD_ZD_SYS
Package created.
Package body created.
End of Creating PL/SQL Package AD_ZD_SYS.
PL/SQL procedure successfully completed.
Commit complete.
5. Check pre-requisites on EBS apps
select * from ad_bugs where bug_number in (
'9239089', -- R12.AD.B.Delta.3
'19273341', -- R12.BNE.B.delta.4
'8919491', -- R12.ATG_PF.B.Delta.3
'18936791', -- R12.FWK.B.delta.4 see Don ID 1931412.1
'19393542' -- 12.1.0.2 database patch. It will not show up here :)
);
6. Apply R12.1 Oct2015 CPU 21507207
In my system, I merged 19273341, 18936791, 21507207 together and applied them in one run by adpatch. It may take more than 1 hour on first node (around 18 minutes silence on afprgshrlog.sql !).
Tuesday, December 22, 2015
Saturday, December 19, 2015
Upgrade R12.1.1 to R12.1.3
I followed Doc ID 1080973.1 in below steps to apply patch 9239090 for upgrading R12.1.1 to R12.1.3 on an Oracle 11.2.0.3.0 database.
Current system status:
SQL> select * from ad_bugs where bug_number in ('9239089', '9239090', '9239095', '9817770', '9966055');
only 9239089 was applied.
$ $IAS_ORACLE_HOME/Apache/Apache/bin/httpd -v
Server version: Oracle-Application-Server-10g/10.1.3.5.0 Oracle-HTTP-Server
Server built: Jul 21 2009 11:12:22
$ $ORACLE_HOME/bin/frmcmp_batch|grep Forms| grep Version
Forms 10.1 (Form Compiler) Version 10.1.2.3.0 (Production)
$ cat $FORMS_WEB_CONFIG_FILE|grep sun_plugin_version| cut -c 1-35
sun_plugin_version=1.6.0_06
Pre steps in database (to avoid headaches during the upgrading):
1. Setup two database parameters:
_disable_fast_validate=TRUE <-- this parameter shall be removed after upgrade completed.
pga_aggregate_target=2G
2. Follow Doc ID 1112223.1 to grant XDBADMIN the right role to avoid ORA-31000 error with csrrsreg.sql.
SQL> show user
apps
SQL> select * from USER_ROLE_PRIVS;
USERNAME GRANTED_ROLE ADM DEF OS_
----------------------------------- ------------------------------------------------- ------- ------ ---
APPS AQ_ADMINISTRATOR_ROLE YES YES NO
APPS CTXAPP NO YES NO
APPS JAVASYSPRIV NO YES NO
APPS RESOURCE NO YES NO
APPS XDBADMIN NO YES NO
3. Follow Doc ID 1357760.1 & 1081227.1 to avoid autoconfig error from jtfictx.sh
DECLARE
*
ERROR at line 1:
ORA-29874: warning in the execution of ODCIINDEXALTER routine
ORA-29960: line 1,
DRG-10595: ALTER INDEX JTF_AMV_ITEMS_URL_CTX failed
DRG-10758: index owner does not have the privilege to use file or URL datastore
ORA-06512: at "SYS.DBMS_SQL", line 1199
ORA-06512: at line 96
Fixes:
SQL> create role apps_ctx_role;
SQL> grant apps_ctx_role to apps;
SQL> exec ctxsys.ctx_adm.set_parameter('file_access_role', 'apps_ctx_role');
SQL> grant apps_ctx_role to JTF; -- Grant it to the index owners
SQL> grant apps_ctx_role to AMV;
SQL> grant apps_ctx_role to APPLSYS;
SQL> grant apps_ctx_role to HR;
SQL> grant apps_ctx_role to IBC;
SQL> grant apps_ctx_role to OKC;
4. Follow Doc ID 1551854.1 & 847687.1 to avoid error from EGOSILDU.sql
DECLARE
*
ERROR at line 1:
ORA-02149: Specified partition does not exist
ORA-06512: at line 408
SQL> SELECT ATTR_GROUP_ID
FROM EGO_FND_DSC_FLX_CTX_EXT WHERE APPLICATION_ID = 431
AND DESCRIPTIVE_FLEXFIELD_NAME = 'EGO_ITEMMGMT_GROUP'
AND DESCRIPTIVE_FLEX_CONTEXT_CODE = 'ItemDetailDesc';
ATTR_GROUP_ID
-------------------------
2
SQL> SELECT 1 FROM EGO_MTL_SY_ITEMS_EXT_TL WHERE ATTR_GROUP_ID = 2
AND ROWNUM = 1;
no rows selected
SQL> SELECT 1 FROM EGO_MTL_SY_ITEMS_EXT_TL WHERE ATTR_GROUP_ID = 2
AND ROWNUM = 1;
no rows selected
Fixes: Open a SR with Oracle Support to get a datafix script from them. And run it as sysdba:
SQL> @Partition_Fix_Script_Ver2.sql
Notes: After I used $AD_TOP/sql/adutconf.sql to confirm module EGO is inactive, I let the failed worker to quit and then skipped EGOSILDU.sql in my Dev instance. Seems ignoring the error worked fine.
5. Make sure account OLAPSYS is not locked.
SQL> select username, account_status from dba_users like '%OLAP%';
If locked, it will hit errors in Doc ID 1054417.1:
ERROR at line 1:
ORA-33292: Insufficient permissions to access analytic workspace APPS.ODPCODE
using the specified access mode.
ORA-06512: at "SYS.DBMS_AW", line 67
ORA-06512: at "SYS.DBMS_AW", line 131
ORA-06512: at "SYS.DBMS_AW", line 984
ORA-06512: at "SYS.DBMS_AW", line 937
ORA-33262: Analytic workspace ODPCODE does not exist.
ORA-06512: at "SYS.DBMS_AW", line 908
ORA-06512: at "APPS.MSD_AW_LOADER", line 10
Pre steps on Apps side:
1. If any schema owner password (such as HR) was changed by Sql*plus abnormally (not by FNDCPASS), the upgrade will fail because adpatch would not have the right password for that module. I had to tell all workers to quit and then "kill" adpatch process on OS level because adpatch kept remembering the bad password. After I used FNDCPASS to change the wrong password, I re-ran adpatch from the beginning and it worked.
2. Apply patch 10257394 (Doc ID 1286779.1) to avoid error with IEX_SCORE_COMP_DET_U1
(or, copy iexscore.lct from patch .zip file and manually replace the old one)
Apply the patch:
$ unzip 6 .zip files (for patch 9239090) to same folder /path/to/R1213CD
($ unzip xxxX.zip -d /path/to/R1213CD)
$ adadmin to enable maintenance mode
$ cd /path/to/R1213CD/9239090 <== make folder 9239090 10GB free and rw permissions
$ adpatch <== it takes 3 or more hours
Then, optional step: start apps to do a quick check out.
Post steps:
$ cd 9239095
$ adpatch
$ cd 9817770
$ adpatch
$ cd 9966055
$ adpatch
$ cd $AD_TOP/bin
$ perl admkappsutil.pl ==> then copy appsutil.zip file to DB server.
$ adconfig.sh on DB tier
$ adautocfg.sh on each apps tiers
disable maintenance mode
start apps services
remove _disable_fast_validate=TRUE
NOTES: In a R12.1.1 instance, a lot of database sessions kept in INACTIVE for a long time (15+ days) and never got closed. So, the number of db sessions increased daily and exceeded easily database parameter PROCESSES (3000). We had to recycle Apps services each two weeks. Modules AR and GL are used most in this instance and most idle sessions are from "e::bes:oracle.apps.icx.security.session.created" by JDBC Thin Client. We tried and did not find a fix to this problem. But, after the instance was upgraded to R12.1.3 and its database was upgraded to 12c, the idle session problem went away surprisingly.
Current system status:
SQL> select * from ad_bugs where bug_number in ('9239089', '9239090', '9239095', '9817770', '9966055');
only 9239089 was applied.
$ $IAS_ORACLE_HOME/Apache/Apache/bin/httpd -v
Server version: Oracle-Application-Server-10g/10.1.3.5.0 Oracle-HTTP-Server
Server built: Jul 21 2009 11:12:22
$ $ORACLE_HOME/bin/frmcmp_batch|grep Forms| grep Version
Forms 10.1 (Form Compiler) Version 10.1.2.3.0 (Production)
$ cat $FORMS_WEB_CONFIG_FILE|grep sun_plugin_version| cut -c 1-35
sun_plugin_version=1.6.0_06
Pre steps in database (to avoid headaches during the upgrading):
1. Setup two database parameters:
_disable_fast_validate=TRUE <-- this parameter shall be removed after upgrade completed.
pga_aggregate_target=2G
2. Follow Doc ID 1112223.1 to grant XDBADMIN the right role to avoid ORA-31000 error with csrrsreg.sql.
SQL> show user
apps
SQL> select * from USER_ROLE_PRIVS;
USERNAME GRANTED_ROLE ADM DEF OS_
----------------------------------- ------------------------------------------------- ------- ------ ---
APPS AQ_ADMINISTRATOR_ROLE YES YES NO
APPS CTXAPP NO YES NO
APPS JAVASYSPRIV NO YES NO
APPS RESOURCE NO YES NO
APPS XDBADMIN NO YES NO
3. Follow Doc ID 1357760.1 & 1081227.1 to avoid autoconfig error from jtfictx.sh
DECLARE
*
ERROR at line 1:
ORA-29874: warning in the execution of ODCIINDEXALTER routine
ORA-29960: line 1,
DRG-10595: ALTER INDEX JTF_AMV_ITEMS_URL_CTX failed
DRG-10758: index owner does not have the privilege to use file or URL datastore
ORA-06512: at "SYS.DBMS_SQL", line 1199
ORA-06512: at line 96
Fixes:
SQL> create role apps_ctx_role;
SQL> grant apps_ctx_role to apps;
SQL> exec ctxsys.ctx_adm.set_parameter('file_access_role', 'apps_ctx_role');
SQL> grant apps_ctx_role to JTF; -- Grant it to the index owners
SQL> grant apps_ctx_role to AMV;
SQL> grant apps_ctx_role to APPLSYS;
SQL> grant apps_ctx_role to HR;
SQL> grant apps_ctx_role to IBC;
SQL> grant apps_ctx_role to OKC;
4. Follow Doc ID 1551854.1 & 847687.1 to avoid error from EGOSILDU.sql
DECLARE
*
ERROR at line 1:
ORA-02149: Specified partition does not exist
ORA-06512: at line 408
SQL> SELECT ATTR_GROUP_ID
FROM EGO_FND_DSC_FLX_CTX_EXT WHERE APPLICATION_ID = 431
AND DESCRIPTIVE_FLEXFIELD_NAME = 'EGO_ITEMMGMT_GROUP'
AND DESCRIPTIVE_FLEX_CONTEXT_CODE = 'ItemDetailDesc';
ATTR_GROUP_ID
-------------------------
2
SQL> SELECT 1 FROM EGO_MTL_SY_ITEMS_EXT_TL WHERE ATTR_GROUP_ID = 2
AND ROWNUM = 1;
no rows selected
SQL> SELECT 1 FROM EGO_MTL_SY_ITEMS_EXT_TL WHERE ATTR_GROUP_ID = 2
AND ROWNUM = 1;
no rows selected
Fixes: Open a SR with Oracle Support to get a datafix script from them. And run it as sysdba:
SQL> @Partition_Fix_Script_Ver2.sql
Notes: After I used $AD_TOP/sql/adutconf.sql to confirm module EGO is inactive, I let the failed worker to quit and then skipped EGOSILDU.sql in my Dev instance. Seems ignoring the error worked fine.
5. Make sure account OLAPSYS is not locked.
SQL> select username, account_status from dba_users like '%OLAP%';
If locked, it will hit errors in Doc ID 1054417.1:
ERROR at line 1:
ORA-33292: Insufficient permissions to access analytic workspace APPS.ODPCODE
using the specified access mode.
ORA-06512: at "SYS.DBMS_AW", line 67
ORA-06512: at "SYS.DBMS_AW", line 131
ORA-06512: at "SYS.DBMS_AW", line 984
ORA-06512: at "SYS.DBMS_AW", line 937
ORA-33262: Analytic workspace ODPCODE does not exist.
ORA-06512: at "SYS.DBMS_AW", line 908
ORA-06512: at "APPS.MSD_AW_LOADER", line 10
Pre steps on Apps side:
1. If any schema owner password (such as HR) was changed by Sql*plus abnormally (not by FNDCPASS), the upgrade will fail because adpatch would not have the right password for that module. I had to tell all workers to quit and then "kill" adpatch process on OS level because adpatch kept remembering the bad password. After I used FNDCPASS to change the wrong password, I re-ran adpatch from the beginning and it worked.
2. Apply patch 10257394 (Doc ID 1286779.1) to avoid error with IEX_SCORE_COMP_DET_U1
(or, copy iexscore.lct from patch .zip file and manually replace the old one)
Apply the patch:
$ unzip 6 .zip files (for patch 9239090) to same folder /path/to/R1213CD
($ unzip xxxX.zip -d /path/to/R1213CD)
$ adadmin to enable maintenance mode
$ cd /path/to/R1213CD/9239090 <== make folder 9239090 10GB free and rw permissions
$ adpatch <== it takes 3 or more hours
Then, optional step: start apps to do a quick check out.
Post steps:
$ cd 9239095
$ adpatch
$ cd 9817770
$ adpatch
$ cd 9966055
$ adpatch
$ cd $AD_TOP/bin
$ perl admkappsutil.pl ==> then copy appsutil.zip file to DB server.
$ adconfig.sh on DB tier
$ adautocfg.sh on each apps tiers
disable maintenance mode
start apps services
remove _disable_fast_validate=TRUE
NOTES: In a R12.1.1 instance, a lot of database sessions kept in INACTIVE for a long time (15+ days) and never got closed. So, the number of db sessions increased daily and exceeded easily database parameter PROCESSES (3000). We had to recycle Apps services each two weeks. Modules AR and GL are used most in this instance and most idle sessions are from "e::bes:oracle.apps.icx.security.session.created" by JDBC Thin Client. We tried and did not find a fix to this problem. But, after the instance was upgraded to R12.1.3 and its database was upgraded to 12c, the idle session problem went away surprisingly.
Sunday, December 13, 2015
Configure printer to work with EBS
This summary is not available. Please
click here to view the post.
Saturday, December 12, 2015
EBS Refresh
Refresh from EBSPROD (under /u06/app/EBSPROD) to EBSQA (under /u01/app/EBSQA) on a different host in Linux OS.
1. On source nodes, run PERL script.
$ cd $ADMIN_SCRIPTS_HOME
$ perl adpreclone.pl appsTier <-- No password is needed
NOTE: Pre-clone log file located at $INST_TOP/admin/log/StageAppsTier_MMDDHHMM.log
$ cd $COMMON_TOP/clone <-- to verify new folders are created
$ ls -altr $COMMON_TOP/clone
total 40
drwxr-xr-x 2 applmgr appsuser 4096 Jan 22 2008 html
drwxr-xr-x 3 applmgr appsuser 4096 Jan 22 2008 context
drwxr-xr-x 2 applmgr appsuser 4096 Jan 22 2008 bin
drwxr-xr-x 9 applmgr appsuser 4096 Sep 15 2013 ..
drwxr-xr-x 5 applmgr appsuser 4096 Dec 1 10:06 appsts
drwx------ 7 applmgr appsuser 4096 Dec 1 10:07 jre
drwxr-xr-x 3 applmgr appsuser 4096 Dec 1 10:07 oui
drwxr-xr-x 4 applmgr appsuser 4096 Dec 1 10:07 jlib
drwxr-xr-x 5 applmgr appsuser 4096 Dec 1 10:07 appl
drwxr-xr-x 10 applmgr appsuser 4096 Dec 1 10:07 .
Do the same on other source hosts.
2. On target nodes, backup filess
Create a folder, such as /u01/app/admin_scripts/backups, to hold backed-up files. Then run the script backup_4refresh.sh to back up <SID_HOST> specific files.
$ cd admin_scripts/backups
$ ./backup_4refresh.sh
Confirm necessary files are just backed up:
$ ls -al /u01/app/admin_scripts/backups
$ ls -al /u01/app/admin_scripts/backups/admin
$ ls -al /u01/app/admin_scripts/backups/Apache
Optional: Backup the entire folder to aother partition
$ cd /u01/app
$ tar -zhcvf /u02/app/$CONTEXT_NAME_backup_MMDDYY.tar.gz EBSQA >> backup_MMDD.log
Do the same on other hosts.
3. Make sure all apps services were stopped and database refresh was completed.
Now, remove directories
$ cd /u01/app
$ rm -rf EBSQA
$ rm -rf oraEBSQAinventory; mkdir oraEBSQAinventory
4. Copy directories from source node.
NOTE: copy CM host files to CM host, web/forms host to web/forms host.
$ cd /u01/app
$ rsync --progress -avze ssh applmgr@source_host:/u06/app/EBSPROD /u01/app
enter password
sent 12619428 bytes received 9649138852 bytes 1312649.72 bytes/sec
total size is 33018009660 speedup is 3.42
It may give warning message:
rsync warning: some files vanished before they could be transferred (code 24) at main.c(1532) [generator=3.0.6]
5. Rename the folder and verify the size
$ cd /u01/app
$ mv EBSPROD EBSQA
$ ls -al EBSQAtotal 16
drwxr-xr-x 4 applmgr appsuser 4096 May 7 2013 .
drwxr-xr-x 16 applmgr appsuser 4096 Apr 15 11:27 ..
drwxr-xr-x 4 applmgr appsuser 4096 May 2 2013 apps
drwxr-xr-x 3 applmgr appsuser 4096 May 2 2013 inst
$cd apps
$ du -ks
31640672 .
$ cd ../inst
$ du -ks
2502420 .
6. Edit /etc/oraInst.loc file to make sure below line is used:
inventory_loc=/u01/app/oraEBSQAInventory
7. Start a new OS session (without running the .env file)
If you do not modify the .profile, re-login will get message:
$APPL_TOP/${CONTEXT_NAME}.env: cannot open [No such file or directory]
Note: by this time, the .env file was deleted when old folder EBSQA was removed in Step 5.
So env vairables do not exist
8. Go to /u01/app/EBSQA/apps/apps_st/comn/clone/bin ($COMMON_TOP/clone/bin)
option: delete old files in /u01/app/EBSQA/apps/apps_st/comn/clone
$ cd /u01/app/EBS/apps/apps_st/comn/clone
$ rm -rf jre_BAK_*
Now, get below answers ready -
Database Server Node => dbhost1q.domain.com
database SID => EBSQA
apps => appsPWD
Base Directory => /u01/app/EBSQA
PORT => 1542
port pool: 21 (1542 -1521)
9. Run clone perl script and answer questions
$ perl adcfgclone.pl appsTier <<-- run this in CM node first!!
++++++++++++++++ answers on CM node ++++++++++++++++
Target System Root Service [disabled] : <-- for CM node. WEB/Form node will be opposite
Target System Web Entry Point Services [disabled] :
Target System Web Application Services [disabled] :
Target System Batch Processing Services [enabled] :
Target System Other Services [enabled] : <-- for CM node
Do you want to preserve the Display [y] (y/n) : n
Target System Display [host1q:0.0] : host1q.domain.com:21.0 <-- OAM will use it for display
Do you want the the target system to have the same port values as the source system (y/n) [y] ? : n
Target System Port Pool [0-99] : 21
++++++++++++++++ answers on Form/Web node ++++++++++++++++
Target System Root Service [enabled] : <-- for Form node
Target System Web Entry Point Services [enabled] :
Target System Web Application Services [enabled] :
Target System Batch Processing Services [enabled] : disabled <-- for Form node
Target System Other Services [enabled] : disabled
Do you want to preserve the Display [xifsapm3q:0.0] (y/n) : n
Target System Display [host2q:0.0] : host2q.domain.com:21.0 <-- OAM will use it for display
Do you want the the target system to have the same port values as the source system (y/n) [y] ? : n
Target System Port Pool [0-99] : 21
The script may remain silent for a few minutes (or even 40 minutes) on executing $INST_TOP/admin/install/txkWfClone.sql !
(only statement SELECT * from v$session where upper(machine) like 'HOSTNAME%'; can tell if the sql script is running).
Answer "no" - Dot not start apps services after autoconfig completed.
Check the log file. Any more lines than below need your attention or fix:
$ egrep -i 'fail|error|ora-' $INST_TOP/admin/log/ApplyAppsTier_04151536.log
Error while running adlnktools.sh.
ERRORCODE = 0 ERRORCODE_END
ERRORCODE = 0 ERRORCODE_END
AC-50480: Internal error occurred: java.lang.Exception: Error while generating listener.ora.
Error generating tnsnames.ora from the database, temporary tnsnames.ora will be generated using templates
ERRORCODE = 2 ERRORCODE_END
Result : FAILED
ERRORCODE = 0 ERRORCODE_END
Unique constraint error (00001) is OK if key already exists
ERRORCODE = 0 ERRORCODE_END
ERRORCODE = 0 ERRORCODE_END
. . . . . . . . .
10. Start a new OS session. If necessary, modify the .profile to run .env
$ . ./.profile
11. Remove obsolete files
$ cd $APPLPTMP <-- make sure $APPLPTMP points to the right folder!
$ rm -f *.tmp
Note: seems the clone script cleans the folder $APPLTMP automatically.
12. Generate autoconfig file for DBA (only do this on ONE node)
$ cd $AD_TOP/bin
$ ls -al ad*mk*
$ perl admkappsutil.pl
Starting the generation of appsutil.zip
Log file located at $INST_TOP/admin/log/MakeAppsUtil_04151607.log
output located at $INST_TOP/admin/out/appsutil.zip
MakeAppsUtil completed successfully.
$ cp -p $INST_TOP/admin/out/appsutil.zip $APPLPTMP
13. Verify apps password before email DBA with the password. And clean CM tables
$ sqlplus apps/passwd
SQL> @cmclean.sql
SQL> EXEC FND_CONC_CLONE.SETUP_CLEAN;
SQL> select * from fnd_nodes;
(This SELECT shell get 0 row returned. After DBA runs adconfig.sh below, it shall have one row.)
Notes: cmclean.sql is an old file but it still works for me in my R12.1.3 instances
REM $Id: cmclean.sql,v 1.4 2001/04/07 15:55:07 pferguso Exp $
14. Ask DBA to run ADCONFIG
$ adconfig.sh contextfile=$ORACLE_HOME/db_ebsqa/appsutil/$CONTEXT_NAME.xml
NOTE: if .xml does not exist on a new instance, generate it first. here is how to generate the .xml file after unzip appsutil.zip on database server :
$ export PERL5LIB=$ORACLE_HOME/appsutil/perl
$ export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$OARCLE_HOME/ctx/lib
$ perl $ORACLE_HOME/appsutil/bin/adbldxml.pl
<enter apps password when prompted>
<enter hostname when prompted>
<enter listener port when prompted>
<enter SID name when prompted>
Output should indicate:
The context file has been created at: $ORACLE_HOME/db_ebsdev/appsutil/EBSDEV_ebsdb1d.xml
(or $ perl adbldxml.pl tier=db appsuser=<APPSuser> )
15. Run script to restore files from backup
$ cd admin_scripts/backups
$ ./resotore_refresh.sh
15. Verify files and make sure two files for site custom
$FND_TOP/fndenv.env
$INST_TOP/ora/10.1.2/forms/server/default.env
Note: two files shall have a section in below format (Otherwise it will be overwritten by autoconfig). Or, modify them after adautocfg.
#Begin Customizations
custom tops
#End Customizations
$ ls -al $INST_TOP/certs/Apache
cwallet.sso ewallet.p12
17. Update $CONTEXT_FILE
- APPLCSF & APPLPTMP on ALL nodes !
- Below ones are ONLY on Form/Web nodes:
webport
activewebport
login
external_url
webentryhost
webentryurlprotocol
Notes: For real port (not as 443 in F5) when SSL is enabled, make sure the port (such as 4472) for s_active_webport match the port in file ssl.conf and s_webssl_port in $CONTEXT_FILE .
18. Run AD autocfg : always make CM node work first (after DBA is done with adconfig!)
$ cd $ADMIN_SCRIPTS_HOME
$ ./adautocfg.sh
Optional: change APPS password using FNDCPASS here, and then run adautocfg.sh again on all nodes.
19. Source .profile to take new env variables after autocfg (or re-login)
$ . ./.profile
20. Start services
21. Login as XML Publisher Admin --> Administration --> General --> replace the path with $APPLPTMP.
22. Update profile "Site%name%"
Check profile option: BNE Debug Log Directory, BNE Upload Staging Directory (then, bounce services)
RRA: Service Prefix (this should be BLANK. Not hard coded !!)
RRA: Enabled (someones say this must be set to YES. Seems not necessary)
Viewer: Text (this should be blank or browser)
23. Modify files for customization. Maybe only on the web node. Such as:
- TEMPLATE FILE : $FND_TOP/admin/template/httpd_conf_1013.tmp
add a section to it
- TEMPLATE FILE : $FND_TOP/admin/template/oracle_apache_conf_1013.tmp
#include "%s_ora_config_home%/10.1.3/Apache/modplsql/conf/plsql.conf"
include "%s_weboh_oh%/Apache/modplsql/conf/plsql.conf" <<-- add this line
- Add APPS password to $IAS_ORACLE_HOME/Apache/modplsql/conf/dads.conf
24. Re-create database links, check DBA_DIRECTORIES (DBA task)
25. Re-link some folders for $CUSTOM_TOP:
$ cd $CUSTOM_TOP/ftp
$ rm in
$ rm out
$ ln -s /path/to/ftp/in in
$ ln -s /path/to/ftp/out out
$ chmod 777 in
$ chmod 777 out
26. Optional: disable production users
-- Doc. ID: 1064798.1. Disable EBS users
DECLARE
-- get user list which should be disabled
cc number :=0;
cursor c1 is
select user_name
from fnd_user
where (end_date is null or end_date > sysdate)
-- access all users that has no end-date or end-date is future
and (user_name like 'B%' or user_name like 'M%')
and user_name not in ('SYSADMIN', 'USERVIP1', 'USERVIP2', 'A999999'); -- excluding VIPs
BEGIN
for c in c1 loop
-- disable user
fnd_user_pkg.disableuser(c.user_name);
cc := cc+1;
If ( MOD(cc, 50) = 0) then
commit;
end if;
end loop;
dbms_output.put_line('Totla disabled ' || cc);
commit; --commit changes
END;
/
++++++++ Troubleshooting +++++++
1. After CM node is up in the first time, run a CM job got error when opening the log
An error occurred while attempting to establish an Applications File Server connection with the
node FNDFS_CMHOST. There may be a network configuration problem, or the TNS listener
on node FNDFS_CMHOST may not be running. Please contact your system administrator.
you can confirm the problem by running below command on Web Host:
$ tnsping FNDFS_CMHOST
TNS-12547: TNS:lost contact
after host names were deleted by "EXEC FND_CONC_CLONE.SETUP_CLEAN" from database tables and when autoconfig was executed first on Concurrent Host CMhost, web host names are not saved to database tables yet and so tnsnames.ora file on CMhost does not have entries for web hosts.
Fix: two ways to fix it
1). on CM host
(a). Replace tnsnames.ora and sqlnet.ora in $TNS_ADMIN with files from the backup before refresh.
(b). adalnctl.sh stop
(c). adalnctl.sh start
Now, on the Web Host server, "tnsping FNDFS_CMHOST" returns "OK"
2). Run autoconfig AGAIN on all hosts, starting with CM host first.
2. When stopping CM services (to run autoconfig again for the new APPS password), 5 processes on
OS level did not stop. See all five of them in the Concurrent Manager:
CONCURRENT MANAGER Actual Target
Output Post Processor 2 0
Workflow Agent Listener Service 1 0
Workflow Mailer Service 1 0
Workflow Document Web Services Service 1 0
I clicked "Terminate" on them, but that did not kill the OS processes. I had to kill them by OS command:
$ kill -12 15524 15529 15544 15548 15555
After I ran autoconfig and started CM services, they showed "Terminted" in GUI Concurrent Manager.
I had to clicked each of them to "Activate" them. They did get started after the clicks !!
Should not try to terminate them in the GUI !!
3. When I ran the autoconfig last on CM node, the login page did not work!!
So, run autoconfig on web/forms node last.
4. After clone script adcfgclone.pl worked, Autoconfig ended with below error in adconfig.log & ApplyAppsTier_11011132.log:
[CVM Error Report]
The following report lists errors encountered during CVM Phase
<filename> <return code where appropriate>
/u03/app/EBSUAT/apps/apps_st/appl/ad/12.0.0/bin/adgentns.pl 2
Updating s_tnsmode to 'generateTNS'
UpdateContext exited with status: 0
AC-50480: Internal error occurred: java.lang.Exception: Error while generating listener.ora.
Error generating tnsnames.ora from the database, temporary tnsnames.ora will be generated using templates
Instantiating Tools tnsnames.ora
Tools tnsnames.ora instantiated
Web tnsnames.ora instantiated
adgentns.pl exiting with status 2
Fix: run below line AGAIN
SQL> EXEC FND_CONC_CLONE.SETUP_CLEAN;
Then, run adconfig.sh on DB host. After that, make sure the DB host name shows up in below query before run adautocfg.sh on Apps tiers:
SQL> select * from fnd_nodes;
~~~~~~~~~~~~~~~~~~ script: backup_4refresh.sh ~~~~~~~~~~~~~~~~~~
#!/bin/ksh
# backup files before the refresh Dec. 2015
cd $HOME/admin_scripts/backups
DT=`date +"%h_%d_%y"` # backup date
echo It will delete old backup files. Continue ? yes or no
read answer
case $answer in
[yY][eE][sS]|[yY])
if [ -n "$CONTEXT_FILE" ]; then
echo 'running'
else
echo 'environment variable $CONTEXT_FILE does not exit. Exit ...'
exit 1;
fi;
if [ -f $CONTEXT_FILE ]; then
echo 'starting copy ...'
else
echo 'CONTEXT file $CONTEXT_FILE does not exist. Exit ...'
exit 1;
fi
file1=`basename $CONTEXT_FILE`
if [ -f $file1 ]; then
mv $file1 $file1'_'$DT # backup date
fi
file2=custom`basename $CONTEXT_FILE .xml`'.env'
if [ -f $APPL_TOP/$file2 ]; then
cp -p $APPL_TOP/$file2 .
fi
rm -rf Apache admin
cp -p $CONTEXT_FILE .
cp -rp $INST_TOP/certs/Apache .
cp -rp $TNS_ADMIN .
cp -p $FND_TOP/fndenv.env .
cp -p $INST_TOP/ora/*2/forms/server/default.env .
cp -p $APPL_TOP/admin/adkeystore.dat .
cp -p $APPL_TOP/admin/adsign.txt .
## more optional files. Need to manually copy them back if needed
cp -p $FND_TOP/admin/template/custom/rwbuilder_conf_1012.tmp .
cp -p $FND_TOP/admin/template/ssl_conf_1013.tmp .
cp -p $FND_TOP/admin/template/httpd_conf_1013.tmp .
cp -p $FND_TOP/admin/template/oracle_apache_conf_1013.tmp .
cp -p $IAS_ORACLE_HOME/Apache/modplsql/conf/dads.conf . echo 'Done with backing up files'
ls -altr $HOME/admin_scripts/backups
exit 0
;;
[nN][oO]|[nN])
echo "No"
;;
*)
echo "Invalid input..."
exit 1
;;
esac
~~~~~~~~~~~~~~~~~~ scripts: restore_refresh.sh ~~~~~~~~~~~~~~~~~~~
#!/bin/ksh
# restore files AFTER clone script during the refresh Dec. 2015
cd $HOME/admin_scripts/backups
DT=`date +"%h_%d_%y"`
echo It will overwrite some files. Continue? yes or no
read answer
case $answer in
[yY][eE][sS]|[yY])
if [ -n "$CONTEXT_FILE" ]; then
echo 'running'
else
echo 'environment variable $CONTEXT_FILE does not exit. Exit ...'
exit 1;
fi;
file2=custom`basename $CONTEXT_FILE .xml`'.env'
if [ -f $file2 ]; then
cp -p $file2 $APPL_TOP/.
fi
mv $FND_TOP/fndenv.env $FND_TOP/fndenv.env_clone
cp -p fndenv.env $FND_TOP/.
mv $INST_TOP/ora/10.1.2/forms/server/default.env $INST_TOP/ora/10.1.2/forms/server/default.env_clone
cp -p default.env $INST_TOP/ora/10.1.2/forms/server/.
mv $INST_TOP/certs/Apache $INST_TOP/certs/Apache'_'$DT
cp -pr Apache $INST_TOP/certs/.
## files for JRE 1.8. They will be used if Java signing is re-ran.
mv $APPL_TOP/admin/adkeystore.dat $APPL_TOP/admin/adkeystore.dat_clone
cp -p adkeystore.dat $APPL_TOP/admin/.
mv $APPL_TOP/admin/adsign.txt $APPL_TOP/admin/adsign.txt_clone
cp -p adsign.txt $APPL_TOP/admin/.
echo 'Done with restoring files.'
;;
[nN][oO]|[nN])
echo "No"
;;
*)
echo "Invalid input..."
exit 1
;;
esac
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The first task of EBS refresh is to refresh the database. Here are the steps:
0. Stop EBS apps
1. In the source database EBSPROD (in 10G and 11G)
1.1 Identify the SCN nmuber for the RMAN script. Check source database backup log to find the sequence.
1.2 SQL> select first_change#, next_change#, first_time from v$log_history
where sequence# = 11047;
2. In the target database EBSQA
2.1 Shutdown the target database
SQL> alter database backup controlfile to trace;
SQL> shutdown immediate;
2.2 Drop the databae
SQL> startup mount restrict;
SQL> drop database;
2.3 Get ready for the RMAN restore
SQL> startup nomount;
2.4 Make sure host has enough disk space
2.5 Assume the source database was backed up by RMAN. Run the script
$ cat db_refresh.sh
ORACLE_SID=targetDB_ID
export ORACLE_SID
export TNS_ADMIN=$ORACLE_HOME/network/admin
ORACLE_USER=oracle
NLS_LANG=AMERICAN_AMERICA.AL32UTF8
NLS_DATE_FORMAT='YYYY-MM-DD hh24:mi:ss'
export NLS_LANG
export NLS_DATE_FORMAT
NB_ORA_CLASS=sourceDBserver
export NB_ORA_CLASS
DATE=`date +"%h%d%y_%H%M"`
rman catalog rman/rmanPWD@rmanDB_ID target sys/PSWD@sourceDB_ID auxiliary / msglog db_refresh.${DATE}.log << EOF
run {
ALLOCATE AUXILIARY CHANNEL ch00 TYPE 'SBT_TAPE';
ALLOCATE AUXILIARY CHANNEL ch01 TYPE 'SBT_TAPE';
ALLOCATE AUXILIARY CHANNEL ch02 TYPE 'SBT_TAPE';
SEND 'NB_ORA_CLIENT=sourceDBserver';
SET NEWNAME FOR TEMPFILE 1 to '/path/to/temp01.dbf';
SET NEWNAME FOR TEMPFILE 2 to '/path/to/temp02.dbf';
duplicate target database to targetDB_ID
UNTIL scn = 5968220384735;
}
EOF
$ ./db_refresh.sh &
3. Bring the database up.
Do NOT change any password (except SYSTEM).
1. On source nodes, run PERL script.
$ cd $ADMIN_SCRIPTS_HOME
$ perl adpreclone.pl appsTier <-- No password is needed
NOTE: Pre-clone log file located at $INST_TOP/admin/log/StageAppsTier_MMDDHHMM.log
$ cd $COMMON_TOP/clone <-- to verify new folders are created
$ ls -altr $COMMON_TOP/clone
total 40
drwxr-xr-x 2 applmgr appsuser 4096 Jan 22 2008 html
drwxr-xr-x 3 applmgr appsuser 4096 Jan 22 2008 context
drwxr-xr-x 2 applmgr appsuser 4096 Jan 22 2008 bin
drwxr-xr-x 9 applmgr appsuser 4096 Sep 15 2013 ..
drwxr-xr-x 5 applmgr appsuser 4096 Dec 1 10:06 appsts
drwx------ 7 applmgr appsuser 4096 Dec 1 10:07 jre
drwxr-xr-x 3 applmgr appsuser 4096 Dec 1 10:07 oui
drwxr-xr-x 4 applmgr appsuser 4096 Dec 1 10:07 jlib
drwxr-xr-x 5 applmgr appsuser 4096 Dec 1 10:07 appl
drwxr-xr-x 10 applmgr appsuser 4096 Dec 1 10:07 .
Do the same on other source hosts.
2. On target nodes, backup filess
Create a folder, such as /u01/app/admin_scripts/backups, to hold backed-up files. Then run the script backup_4refresh.sh to back up <SID_HOST> specific files.
$ cd admin_scripts/backups
$ ./backup_4refresh.sh
Confirm necessary files are just backed up:
$ ls -al /u01/app/admin_scripts/backups
$ ls -al /u01/app/admin_scripts/backups/admin
$ ls -al /u01/app/admin_scripts/backups/Apache
Optional: Backup the entire folder to aother partition
$ cd /u01/app
$ tar -zhcvf /u02/app/$CONTEXT_NAME_backup_MMDDYY.tar.gz EBSQA >> backup_MMDD.log
Do the same on other hosts.
3. Make sure all apps services were stopped and database refresh was completed.
Now, remove directories
$ cd /u01/app
$ rm -rf EBSQA
$ rm -rf oraEBSQAinventory; mkdir oraEBSQAinventory
4. Copy directories from source node.
NOTE: copy CM host files to CM host, web/forms host to web/forms host.
$ cd /u01/app
$ rsync --progress -avze ssh applmgr@source_host:/u06/app/EBSPROD /u01/app
enter password
sent 12619428 bytes received 9649138852 bytes 1312649.72 bytes/sec
total size is 33018009660 speedup is 3.42
It may give warning message:
rsync warning: some files vanished before they could be transferred (code 24) at main.c(1532) [generator=3.0.6]
5. Rename the folder and verify the size
$ cd /u01/app
$ mv EBSPROD EBSQA
$ ls -al EBSQAtotal 16
drwxr-xr-x 4 applmgr appsuser 4096 May 7 2013 .
drwxr-xr-x 16 applmgr appsuser 4096 Apr 15 11:27 ..
drwxr-xr-x 4 applmgr appsuser 4096 May 2 2013 apps
drwxr-xr-x 3 applmgr appsuser 4096 May 2 2013 inst
$cd apps
$ du -ks
31640672 .
$ cd ../inst
$ du -ks
2502420 .
6. Edit /etc/oraInst.loc file to make sure below line is used:
inventory_loc=/u01/app/oraEBSQAInventory
7. Start a new OS session (without running the .env file)
If you do not modify the .profile, re-login will get message:
$APPL_TOP/${CONTEXT_NAME}.env: cannot open [No such file or directory]
Note: by this time, the .env file was deleted when old folder EBSQA was removed in Step 5.
So env vairables do not exist
8. Go to /u01/app/EBSQA/apps/apps_st/comn/clone/bin ($COMMON_TOP/clone/bin)
option: delete old files in /u01/app/EBSQA/apps/apps_st/comn/clone
$ cd /u01/app/EBS/apps/apps_st/comn/clone
$ rm -rf jre_BAK_*
Now, get below answers ready -
Database Server Node => dbhost1q.domain.com
database SID => EBSQA
apps => appsPWD
Base Directory => /u01/app/EBSQA
PORT => 1542
port pool: 21 (1542 -1521)
9. Run clone perl script and answer questions
$ perl adcfgclone.pl appsTier <<-- run this in CM node first!!
++++++++++++++++ answers on CM node ++++++++++++++++
Target System Root Service [disabled] : <-- for CM node. WEB/Form node will be opposite
Target System Web Entry Point Services [disabled] :
Target System Web Application Services [disabled] :
Target System Batch Processing Services [enabled] :
Target System Other Services [enabled] : <-- for CM node
Do you want to preserve the Display [y] (y/n) : n
Target System Display [host1q:0.0] : host1q.domain.com:21.0 <-- OAM will use it for display
Do you want the the target system to have the same port values as the source system (y/n) [y] ? : n
Target System Port Pool [0-99] : 21
++++++++++++++++ answers on Form/Web node ++++++++++++++++
Target System Root Service [enabled] : <-- for Form node
Target System Web Entry Point Services [enabled] :
Target System Web Application Services [enabled] :
Target System Batch Processing Services [enabled] : disabled <-- for Form node
Target System Other Services [enabled] : disabled
Do you want to preserve the Display [xifsapm3q:0.0] (y/n) : n
Target System Display [host2q:0.0] : host2q.domain.com:21.0 <-- OAM will use it for display
Do you want the the target system to have the same port values as the source system (y/n) [y] ? : n
Target System Port Pool [0-99] : 21
The script may remain silent for a few minutes (or even 40 minutes) on executing $INST_TOP/admin/install/txkWfClone.sql !
(only statement SELECT * from v$session where upper(machine) like 'HOSTNAME%'; can tell if the sql script is running).
Answer "no" - Dot not start apps services after autoconfig completed.
Check the log file. Any more lines than below need your attention or fix:
$ egrep -i 'fail|error|ora-' $INST_TOP/admin/log/ApplyAppsTier_04151536.log
Error while running adlnktools.sh.
ERRORCODE = 0 ERRORCODE_END
ERRORCODE = 0 ERRORCODE_END
AC-50480: Internal error occurred: java.lang.Exception: Error while generating listener.ora.
Error generating tnsnames.ora from the database, temporary tnsnames.ora will be generated using templates
ERRORCODE = 2 ERRORCODE_END
Result : FAILED
ERRORCODE = 0 ERRORCODE_END
Unique constraint error (00001) is OK if key already exists
ERRORCODE = 0 ERRORCODE_END
ERRORCODE = 0 ERRORCODE_END
. . . . . . . . .
10. Start a new OS session. If necessary, modify the .profile to run .env
$ . ./.profile
11. Remove obsolete files
$ cd $APPLPTMP <-- make sure $APPLPTMP points to the right folder!
$ rm -f *.tmp
Note: seems the clone script cleans the folder $APPLTMP automatically.
12. Generate autoconfig file for DBA (only do this on ONE node)
$ cd $AD_TOP/bin
$ ls -al ad*mk*
$ perl admkappsutil.pl
Starting the generation of appsutil.zip
Log file located at $INST_TOP/admin/log/MakeAppsUtil_04151607.log
output located at $INST_TOP/admin/out/appsutil.zip
MakeAppsUtil completed successfully.
$ cp -p $INST_TOP/admin/out/appsutil.zip $APPLPTMP
13. Verify apps password before email DBA with the password. And clean CM tables
$ sqlplus apps/passwd
SQL> @cmclean.sql
SQL> EXEC FND_CONC_CLONE.SETUP_CLEAN;
SQL> select * from fnd_nodes;
(This SELECT shell get 0 row returned. After DBA runs adconfig.sh below, it shall have one row.)
Notes: cmclean.sql is an old file but it still works for me in my R12.1.3 instances
REM $Id: cmclean.sql,v 1.4 2001/04/07 15:55:07 pferguso Exp $
14. Ask DBA to run ADCONFIG
$ adconfig.sh contextfile=$ORACLE_HOME/db_ebsqa/appsutil/$CONTEXT_NAME.xml
NOTE: if .xml does not exist on a new instance, generate it first. here is how to generate the .xml file after unzip appsutil.zip on database server :
$ export PERL5LIB=$ORACLE_HOME/appsutil/perl
$ export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$OARCLE_HOME/ctx/lib
$ perl $ORACLE_HOME/appsutil/bin/adbldxml.pl
<enter apps password when prompted>
<enter hostname when prompted>
<enter listener port when prompted>
<enter SID name when prompted>
Output should indicate:
The context file has been created at: $ORACLE_HOME/db_ebsdev/appsutil/EBSDEV_ebsdb1d.xml
(or $ perl adbldxml.pl tier=db appsuser=<APPSuser> )
15. Run script to restore files from backup
$ cd admin_scripts/backups
$ ./resotore_refresh.sh
15. Verify files and make sure two files for site custom
$FND_TOP/fndenv.env
$INST_TOP/ora/10.1.2/forms/server/default.env
Note: two files shall have a section in below format (Otherwise it will be overwritten by autoconfig). Or, modify them after adautocfg.
#Begin Customizations
custom tops
#End Customizations
$ ls -al $INST_TOP/certs/Apache
cwallet.sso ewallet.p12
17. Update $CONTEXT_FILE
- APPLCSF & APPLPTMP on ALL nodes !
- Below ones are ONLY on Form/Web nodes:
webport
activewebport
login
external_url
webentryhost
webentryurlprotocol
Notes: For real port (not as 443 in F5) when SSL is enabled, make sure the port (such as 4472) for s_active_webport match the port in file ssl.conf and s_webssl_port in $CONTEXT_FILE .
18. Run AD autocfg : always make CM node work first (after DBA is done with adconfig!)
$ cd $ADMIN_SCRIPTS_HOME
$ ./adautocfg.sh
Optional: change APPS password using FNDCPASS here, and then run adautocfg.sh again on all nodes.
19. Source .profile to take new env variables after autocfg (or re-login)
$ . ./.profile
20. Start services
21. Login as XML Publisher Admin --> Administration --> General --> replace the path with $APPLPTMP.
22. Update profile "Site%name%"
Check profile option: BNE Debug Log Directory, BNE Upload Staging Directory (then, bounce services)
RRA: Service Prefix (this should be BLANK. Not hard coded !!)
RRA: Enabled (someones say this must be set to YES. Seems not necessary)
Viewer: Text (this should be blank or browser)
23. Modify files for customization. Maybe only on the web node. Such as:
- TEMPLATE FILE : $FND_TOP/admin/template/httpd_conf_1013.tmp
add a section to it
- TEMPLATE FILE : $FND_TOP/admin/template/oracle_apache_conf_1013.tmp
#include "%s_ora_config_home%/10.1.3/Apache/modplsql/conf/plsql.conf"
include "%s_weboh_oh%/Apache/modplsql/conf/plsql.conf" <<-- add this line
- Add APPS password to $IAS_ORACLE_HOME/Apache/modplsql/conf/dads.conf
24. Re-create database links, check DBA_DIRECTORIES (DBA task)
25. Re-link some folders for $CUSTOM_TOP:
$ cd $CUSTOM_TOP/ftp
$ rm in
$ rm out
$ ln -s /path/to/ftp/in in
$ ln -s /path/to/ftp/out out
$ chmod 777 in
$ chmod 777 out
26. Optional: disable production users
-- Doc. ID: 1064798.1. Disable EBS users
DECLARE
-- get user list which should be disabled
cc number :=0;
cursor c1 is
select user_name
from fnd_user
where (end_date is null or end_date > sysdate)
-- access all users that has no end-date or end-date is future
and (user_name like 'B%' or user_name like 'M%')
and user_name not in ('SYSADMIN', 'USERVIP1', 'USERVIP2', 'A999999'); -- excluding VIPs
BEGIN
for c in c1 loop
-- disable user
fnd_user_pkg.disableuser(c.user_name);
cc := cc+1;
If ( MOD(cc, 50) = 0) then
commit;
end if;
end loop;
dbms_output.put_line('Totla disabled ' || cc);
commit; --commit changes
END;
/
++++++++ Troubleshooting +++++++
1. After CM node is up in the first time, run a CM job got error when opening the log
An error occurred while attempting to establish an Applications File Server connection with the
node FNDFS_CMHOST. There may be a network configuration problem, or the TNS listener
on node FNDFS_CMHOST may not be running. Please contact your system administrator.
you can confirm the problem by running below command on Web Host:
$ tnsping FNDFS_CMHOST
TNS-12547: TNS:lost contact
after host names were deleted by "EXEC FND_CONC_CLONE.SETUP_CLEAN" from database tables and when autoconfig was executed first on Concurrent Host CMhost, web host names are not saved to database tables yet and so tnsnames.ora file on CMhost does not have entries for web hosts.
Fix: two ways to fix it
1). on CM host
(a). Replace tnsnames.ora and sqlnet.ora in $TNS_ADMIN with files from the backup before refresh.
(b). adalnctl.sh stop
(c). adalnctl.sh start
Now, on the Web Host server, "tnsping FNDFS_CMHOST" returns "OK"
2). Run autoconfig AGAIN on all hosts, starting with CM host first.
2. When stopping CM services (to run autoconfig again for the new APPS password), 5 processes on
OS level did not stop. See all five of them in the Concurrent Manager:
CONCURRENT MANAGER Actual Target
Output Post Processor 2 0
Workflow Agent Listener Service 1 0
Workflow Mailer Service 1 0
Workflow Document Web Services Service 1 0
I clicked "Terminate" on them, but that did not kill the OS processes. I had to kill them by OS command:
$ kill -12 15524 15529 15544 15548 15555
After I ran autoconfig and started CM services, they showed "Terminted" in GUI Concurrent Manager.
I had to clicked each of them to "Activate" them. They did get started after the clicks !!
Should not try to terminate them in the GUI !!
3. When I ran the autoconfig last on CM node, the login page did not work!!
So, run autoconfig on web/forms node last.
4. After clone script adcfgclone.pl worked, Autoconfig ended with below error in adconfig.log & ApplyAppsTier_11011132.log:
[CVM Error Report]
The following report lists errors encountered during CVM Phase
<filename> <return code where appropriate>
/u03/app/EBSUAT/apps/apps_st/appl/ad/12.0.0/bin/adgentns.pl 2
Updating s_tnsmode to 'generateTNS'
UpdateContext exited with status: 0
AC-50480: Internal error occurred: java.lang.Exception: Error while generating listener.ora.
Error generating tnsnames.ora from the database, temporary tnsnames.ora will be generated using templates
Instantiating Tools tnsnames.ora
Tools tnsnames.ora instantiated
Web tnsnames.ora instantiated
adgentns.pl exiting with status 2
Fix: run below line AGAIN
SQL> EXEC FND_CONC_CLONE.SETUP_CLEAN;
Then, run adconfig.sh on DB host. After that, make sure the DB host name shows up in below query before run adautocfg.sh on Apps tiers:
SQL> select * from fnd_nodes;
~~~~~~~~~~~~~~~~~~ script: backup_4refresh.sh ~~~~~~~~~~~~~~~~~~
#!/bin/ksh
# backup files before the refresh Dec. 2015
cd $HOME/admin_scripts/backups
DT=`date +"%h_%d_%y"` # backup date
echo It will delete old backup files. Continue ? yes or no
read answer
case $answer in
[yY][eE][sS]|[yY])
if [ -n "$CONTEXT_FILE" ]; then
echo 'running'
else
echo 'environment variable $CONTEXT_FILE does not exit. Exit ...'
exit 1;
fi;
if [ -f $CONTEXT_FILE ]; then
echo 'starting copy ...'
else
echo 'CONTEXT file $CONTEXT_FILE does not exist. Exit ...'
exit 1;
fi
file1=`basename $CONTEXT_FILE`
if [ -f $file1 ]; then
mv $file1 $file1'_'$DT # backup date
fi
file2=custom`basename $CONTEXT_FILE .xml`'.env'
if [ -f $APPL_TOP/$file2 ]; then
cp -p $APPL_TOP/$file2 .
fi
rm -rf Apache admin
cp -p $CONTEXT_FILE .
cp -rp $INST_TOP/certs/Apache .
cp -rp $TNS_ADMIN .
cp -p $FND_TOP/fndenv.env .
cp -p $INST_TOP/ora/*2/forms/server/default.env .
cp -p $APPL_TOP/admin/adkeystore.dat .
cp -p $APPL_TOP/admin/adsign.txt .
## more optional files. Need to manually copy them back if needed
cp -p $FND_TOP/admin/template/custom/rwbuilder_conf_1012.tmp .
cp -p $FND_TOP/admin/template/ssl_conf_1013.tmp .
cp -p $FND_TOP/admin/template/httpd_conf_1013.tmp .
cp -p $FND_TOP/admin/template/oracle_apache_conf_1013.tmp .
cp -p $IAS_ORACLE_HOME/Apache/modplsql/conf/dads.conf . echo 'Done with backing up files'
ls -altr $HOME/admin_scripts/backups
exit 0
;;
[nN][oO]|[nN])
echo "No"
;;
*)
echo "Invalid input..."
exit 1
;;
esac
~~~~~~~~~~~~~~~~~~ scripts: restore_refresh.sh ~~~~~~~~~~~~~~~~~~~
#!/bin/ksh
# restore files AFTER clone script during the refresh Dec. 2015
cd $HOME/admin_scripts/backups
DT=`date +"%h_%d_%y"`
echo It will overwrite some files. Continue? yes or no
read answer
case $answer in
[yY][eE][sS]|[yY])
if [ -n "$CONTEXT_FILE" ]; then
echo 'running'
else
echo 'environment variable $CONTEXT_FILE does not exit. Exit ...'
exit 1;
fi;
file2=custom`basename $CONTEXT_FILE .xml`'.env'
if [ -f $file2 ]; then
cp -p $file2 $APPL_TOP/.
fi
mv $FND_TOP/fndenv.env $FND_TOP/fndenv.env_clone
cp -p fndenv.env $FND_TOP/.
mv $INST_TOP/ora/10.1.2/forms/server/default.env $INST_TOP/ora/10.1.2/forms/server/default.env_clone
cp -p default.env $INST_TOP/ora/10.1.2/forms/server/.
mv $INST_TOP/certs/Apache $INST_TOP/certs/Apache'_'$DT
cp -pr Apache $INST_TOP/certs/.
## files for JRE 1.8. They will be used if Java signing is re-ran.
mv $APPL_TOP/admin/adkeystore.dat $APPL_TOP/admin/adkeystore.dat_clone
cp -p adkeystore.dat $APPL_TOP/admin/.
mv $APPL_TOP/admin/adsign.txt $APPL_TOP/admin/adsign.txt_clone
cp -p adsign.txt $APPL_TOP/admin/.
echo 'Done with restoring files.'
;;
[nN][oO]|[nN])
echo "No"
;;
*)
echo "Invalid input..."
exit 1
;;
esac
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The first task of EBS refresh is to refresh the database. Here are the steps:
0. Stop EBS apps
1. In the source database EBSPROD (in 10G and 11G)
1.1 Identify the SCN nmuber for the RMAN script. Check source database backup log to find the sequence.
1.2 SQL> select first_change#, next_change#, first_time from v$log_history
where sequence# = 11047;
2. In the target database EBSQA
2.1 Shutdown the target database
SQL> alter database backup controlfile to trace;
SQL> shutdown immediate;
2.2 Drop the databae
SQL> startup mount restrict;
SQL> drop database;
2.3 Get ready for the RMAN restore
SQL> startup nomount;
2.4 Make sure host has enough disk space
2.5 Assume the source database was backed up by RMAN. Run the script
$ cat db_refresh.sh
ORACLE_SID=targetDB_ID
export ORACLE_SID
export TNS_ADMIN=$ORACLE_HOME/network/admin
ORACLE_USER=oracle
NLS_LANG=AMERICAN_AMERICA.AL32UTF8
NLS_DATE_FORMAT='YYYY-MM-DD hh24:mi:ss'
export NLS_LANG
export NLS_DATE_FORMAT
NB_ORA_CLASS=sourceDBserver
export NB_ORA_CLASS
DATE=`date +"%h%d%y_%H%M"`
rman catalog rman/rmanPWD@rmanDB_ID target sys/PSWD@sourceDB_ID auxiliary / msglog db_refresh.${DATE}.log << EOF
run {
ALLOCATE AUXILIARY CHANNEL ch00 TYPE 'SBT_TAPE';
ALLOCATE AUXILIARY CHANNEL ch01 TYPE 'SBT_TAPE';
ALLOCATE AUXILIARY CHANNEL ch02 TYPE 'SBT_TAPE';
SEND 'NB_ORA_CLIENT=sourceDBserver';
SET NEWNAME FOR TEMPFILE 1 to '/path/to/temp01.dbf';
SET NEWNAME FOR TEMPFILE 2 to '/path/to/temp02.dbf';
duplicate target database to targetDB_ID
UNTIL scn = 5968220384735;
}
EOF
$ ./db_refresh.sh &
3. Bring the database up.
Do NOT change any password (except SYSTEM).
EBS apps RDA
Two types of RDA on Oracle EBS apps.
1. Data collection test
- Run it for R12.1 from browser (Doc ID 732091.1)
$ cd $IZU_TOP/bin
$ ./rda.sh -vdCRP -e APPL _SHORT='AR' ACT
If getting error "RDA-00001: Cannot change to the output directory", rename the setup.cfg file in the directory and try it again.
2. Download file p20603454_8715310_Linux-x86-64.zip from Oracle support site and install it from unzip. Then
$ cd rda
$ ./rda.sh -T hcve
1. Data collection test
- Run it for R12.1 from browser (Doc ID 732091.1)
- Log onto Oracle E-Business Suite
- Click on responsibility Application Diagnostics
- Select the Diagnose menu option
- Click button Select Application and select an appropriate Application, such as payables (SQLAP), receivables (AR)
- Scroll down to group "System Snapshot"
- Select test name "RDA"
- Input Parameters (* required)
- Responsibility Id (LOV) * (seems System Administrator works)
- Application Shortname (LOV) * (such as AR, XLA)
- Mask sensitive data (LOV) * (seems optional)
- APPS Schema Username
- APPS Schema Password
$ cd $IZU_TOP/bin
$ ./rda.sh -vdCRP -e APPL _SHORT='AR' ACT
If getting error "RDA-00001: Cannot change to the output directory", rename the setup.cfg file in the directory and try it again.
2. Download file p20603454_8715310_Linux-x86-64.zip from Oracle support site and install it from unzip. Then
$ cd rda
$ ./rda.sh -T hcve
Subscribe to:
Posts (Atom)