Friday, February 11, 2022

Renew Java Signing

When the current Java Code Signing file will expire in days, a big question is: if we do not renew it, will the EBS Forms still work as usual?  From my experience, even the Java Signing cert expired, EBS Forms will continue functioning. 

But, if a expired adkeystore.dat file is used to re-sign JAR files by adadmin (as a patching step, etc),  the newly signed forms may be blocked by pop-up message in a "Java Application Blocked" box: Your security settings have blocked an application signed with an expired or not-yet-valid certificate from running.

In this case, you can go to Java console on local machine, Java >> Security >> Exception Site List >> add the EBS site URL. Then, you can open EBS GUI forms as a workaround.

When a wrong adkeystore.dat is used by adadmin to generate JAR files, EBS Forms will not open up by popup message:
JAR resources in JNLP file are not signed by same certificate

Also see Oracle Doc ID 2605947.1 (Forms Jar File Certificate Expiration and Timestamp Functionality). 

Warning on JAR file when it closes to expiration:
$ jarsigner -verify -verbose -certs $AD_TOP/java/jar/adxlib.jar
... ...
Warning:
This jar contains entries whose signer certificate will expire within six months.
This jar contains signatures that does not include a timestamp. Without a timestamp, users may not be able to validate this jar after the signer certificate's expiration date (2022-01-12) or after any future revocation date.

Warning after expired:
$ jarsigner -verify -verbose -certs $AD_TOP/java/jar/adxlib.jar
... ...
Warning:
This jar contains entries whose signer certificate has expired.

Steps to renew Java Signing cert, if you have to:

1. I emailed the old adkeystore.dat file to a colleague in Security team. He emailed back the renewed file. He said he downloaded it from Venafi (Venafi Trust Protection Platform) site as JKS (.jks) file with the same old password, and then he renamed it to adkeystore.dat. 

There is no need to create separate cert for each server. Code signing is the same for all servers but has a different alias. Key command to change the alias for another instance (e.g. from EBSDEV instance to EBSQA):

$ keytool -changealias -alias ebsdev_devNode1d -destalias EBSQA_qaNode1q -keystore adkeystore.dat

SQL> set serveroutput on    -- pull out passwords
SQL> declare
spass varchar2(30);
kpass varchar2(30);
begin
ad_jar.get_jripasswords(spass, kpass);
dbms_output.put_line(spass);
dbms_output.put_line(kpass);
end;  
/
puneet
myxuan

2. Copy the new cert file to $NE_BASE/EBSapps/appl/ad/admin (in R12.2)

$ cd $NE_BASE/EBSapps/appl/ad/admin

$ mv adkeystore.dat adkeystore.dat_OLD
$ keytool -list -keystore adkeystore.dat_OLD
Enter keystore password:

Keystore type: JKS
Keystore provider: SUN

Your keystore contains 1 entry

ebsdev_devNode1d, Apr 5, 2019, PrivateKeyEntry,      <== that is the file creation/modification date
Certificate fingerprint (SHA1): A7:01:6E:76:D7:2D:F5:69:8B:93:2B:C0:CA:E5:41:7C:DB:51:45:AA

$ keytool -list -keystore adkeystore.dat    <== new file from renewal
Enter keystore password:

Keystore type: JKS
Keystore provider: SUN

Your keystore contains 3 entries      <== new file could still say "1 entry". Not sure why

cn=digicert assured id root ca, ou=www.digicert.com, o=digicert inc, c=us, Jan 3, 2022, trustedCertEntry,
Certificate fingerprint (SHA1): 05:63:B8:63:0D:62:D7:5A:BB:C8:AB:1E:4B:DF:B5:A8:99:B2:4D:43
cn=digicert sha2 assured id code signing ca, ou=www.digicert.com, o=digicert inc, c=us, Jan 3, 2022, trustedCertEntry,
Certificate fingerprint (SHA1): 92:C1:58:8E:85:AF:22:01:CE:79:15:E8:53:8B:49:2F:60:5B:80:C6
ebsdev_devNode1d, Jan 3, 2022, PrivateKeyEntry,
Certificate fingerprint (SHA1): AF:78:49:69:75:3D:D8:6E:70:F9:74:69:42:BB:A9:19:6B:03:9D:E

Use "keytool -list -v -keystore adkeystore.dat" to see the keystore's validation period and other details.

3. Run command to change the alias for QA server:

$ keytool -changealias -alias ebsdev_devNode1d -destalias EBSQA_qaNode1q -keystore adkeystore.dat
Enter keystore password:
Enter key password for <ebsdev_devNode1d>

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore adkeystore.dat  -destkeystore adkeystore.dat -deststoretype pkcs12".

$ keytool -list -keystore adkeystore.dat
Enter keystore password:

Keystore type: JKS
Keystore provider: SUN

Your keystore contains 3 entries

cn=digicert assured id root ca, ou=www.digicert.com, o=digicert inc, c=us, Jan 3, 2022, trustedCertEntry,
Certificate fingerprint (SHA1): 05:63:B8:63:0D:62:D7:5A:BB:C8:AB:1E:4B:DF:B5:A8:99:B2:4D:43
cn=digicert sha2 assured id code signing ca, ou=www.digicert.com, o=digicert inc, c=us, Jan 3, 2022, trustedCertEntry,
Certificate fingerprint (SHA1): 92:C1:58:8E:85:AF:22:01:CE:79:15:E8:53:8B:49:2F:60:5B:80:C6
ebsqa_qaNode1q, Jan 3, 2022, PrivateKeyEntry,
Certificate fingerprint (SHA1): AF:78:49:69:75:3D:D8:6E:70:F9:74:69:42:BB:A9:19:6B:03:9D:E

Extra steps (that may not be necessary):
$ cp -p adkeystore.dat adkeystore.dat_BK

$ keytool -importkeystore -srckeystore adkeystore.dat -destkeystore adkeystore.dat -deststoretype pkcs12
Enter source keystore password:
Entry for alias cn=digicert assured id root ca, ou=www.digicert.com, o=digicert inc, c=us successfully imported.
Enter key password for <ebsqa_qaNode1q>
Entry for alias ebsqa_qaNode1q successfully imported.
Entry for alias cn=digicert sha2 assured id code signing ca, ou=www.digicert.com, o=digicert inc, c=us successfully imported.
Import command completed:  3 entries successfully imported, 0 entries failed or cancelled

Warning:
<cn=digicert assured id root ca, ou=www.digicert.com, o=digicert inc, c=us> uses the SHA1withRSA signature algorithm which is considered a security risk. This algorithm will be disabled in a future update.
Migrated "adkeystore.dat" to Non JKS/JCEKS. The JKS keystore is backed up as "adkeystore.dat.old".

$ ls -altr adkeystore.dat*
-rw-r----- 1 user users 4992 Apr  2  2019 adkeystore.dat_OLD
-rw-r--r-- 1 user users 7498 Mar  1 11:39 adkeystore.dat_BK
-rw-r--r-- 1 user users 7498 Mar  1 11:41 adkeystore.dat.old
-rw-r--r-- 1 user users 8721 Mar  1 11:41 adkeystore.dat

4. Make sure file adsign.txt is correct
$ more adsign.txt
EBSQA_qanode1q 1 CUST

5. Run adadmin to generate JAR files
$ adamin
 ==> 1 ==> 4
Do you wish to force regeneration of all jar files? [No] ? Yes

Forcing generation of all product jar files.
Creating and signing every jar file can take about thirty minutes depending on the hardware being used.
... ...
Generating customall.jar ...
Not creating customall.jar as no custom java directories found under JAVA_TOP.
customall.jar generated successfully.
   ** Updating appsborg.zip ...
    Reading adjborg.txt...
adearea();
No files listed in $RUN_BASE/EBSapps/appl/admin/adjborg.txt
Ignoring ...
adjborg.txt file seems to be empty.
    Done reading adjborg.txt.
As adjborg.txt file is empty, skipping the  generation of appsborg.zip
   ** Updating appsborg2.zip ...
    Reading adjborg2.txt...
adearea();
No files listed in $RUN_BASE/EBSapps/appl/admin/adjborg2.txt
Ignoring ...
adjborg.txt file seems to be empty.
    Done reading adjborg2.txt.
As adjborg2.txt file is empty, skipping the  generation of appsborg2.zip

6. Confirm the new expiration date
jarsigner -verify -verbose -certs $AD_TOP/java/jar/adxlib.jar
... ...
- Signed by "CN=Company_name, O=Company_name, L=City, ST=State_, C=US"
    Digest algorithm: SHA-256
    Signature algorithm: SHA256withRSA, 4096-bit key

jar verified.

Warning:
This jar contains signatures that do not include a timestamp. Without a timestamp, users may not be able to validate this jar after any of the signer certificates expire (as early as 2024-04-05).

The signer certificate will expire on 2024-04-05.

7. Test the result by opening a few GUI forms. 
It is not necessary to test the whole site.

Tuesday, February 1, 2022

Re-new R12.2 ssl certificate

Certificate has to be renewed before it expires, because after it expires, "adapcctl.sh start" will fail to start Apache service in R12.2. Steps to renew it in R12.2.10:

1. Renew the cert file.
This the key step and may be outside of your control. The process of creating a certificate is always:
- Create public\private key pair.
- Create CSR.
- CA issues certificate based on CSR.

In most cases, the first two steps are done in a cert tool, such as Venafi. Then Venafi talks to DigiCert to get the signed certificate which can be download as .pfx file. After it is renamed to ewallet.p12, OWM (Oracle Wallet Manager) 10.1.0.5.0 in 10.1.3 ORACLE_HOME of R12.1 can read it and turn "Auto Login" on.

But OWM 11.1.1.9.0 at $FMW_HOME/webtier/bin/ in R12.2.10 can not read .pfx file correctly, simply showing "Certificate[Empty]". We have to use orapki under $FMW_HOME/oracle_common/bin to turn "Auto Login" on as a workaround.

2. Copy the new cert file to $NE_BASE/inst/$CONTEXT_NAME/certs/Apache as ewallet.p12
This Apache folder can serve as a staging location. Depending on instance setup, it may not be used in R12.2 at all.

3. Verify the cert can be open and its password is good (optional):
$ alias orapki=$FMW_HOME/oracle_common/bin/orapki
$ which orapki                     <= do not use the one in 10.1.2 ORACLE_HOME

$ orapki wallet display -wallet $NE_BASE/inst/$CONTEXT_NAME/certs/Apache/ewallet.p12
Enter wallet password:   xxxxx
Requested Certificates:
User Certificates:
Subject:        CN=site_name.domian.com,O=company_name,L=city,ST=AZ,C=US
Trusted Certificates:
Subject:        CN=DigiCert High Assurance EV Root CA,OU=www.digicert.com,O=DigiCert Inc,C=US
Subject:        CN=company_name Secure CA2,O=company_name,C=US

Notes:  Without specifying the path to file ewallet.p12, orapki in $FMW_HOME (vs. in 10.1.2 ORACLE_HOME) will read a default cert file in somewhere and may give incorrect password error.

4. Run below line to enable Auto Login

$ orapki wallet create -wallet $NE_BASE/inst/$CONTEXT_NAME/certs/Apache/ewallet.p12 -pwd 'sslPWD' -auto_login
Oracle PKI Tool : Version 11.1.1.9.0
Copyright (c) 2004, 2015, Oracle and/or its affiliates. All rights reserved.

"orapki wallet create ..." will not destroy or modify the original wallet! It creates cwallet.sso in the current directory. You can now view its contents without a password:
$ orapki wallet display -wallet $NE_BASE/inst/$CONTEXT_NAME/certs/Apache/cwallet.sso
Oracle PKI Tool : Version 11.1.1.9.0
Copyright (c) 2004, 2015, Oracle and/or its affiliates. All rights reserved.

Requested Certificates:
User Certificates:
Subject:        CN= ... ...
... ...

5. Stop apps services, and copy cwallet.sso to other locations:

$ adstpall.sh apps/appsPWD

$ iName=$(tr < $CONTEXT_FILE '<>' '  ' | awk '/"s_ohs_instance"/ {print $(NF-1)}' )
$ echo $iName
EBS_web_EBSDEV_OHS1

$ cd $FMW_HOME/webtier/instances/$iName

$ find . -name cwallet.sso | fgrep -v /webgate/
./config/OPMN/opmn/wallet_ORIG/cwallet.sso
./config/OPMN/opmn/wallet/cwallet.sso
./config/OHS/EBS_web_EBSDEV/keystores/default/cwallet.sso
./config/OHS/EBS_web_EBSDEV/proxy-wallet/cwallet.sso

Use one line to see their contents:
$ find . -name cwallet.sso | fgrep -v /webgate/ | while read w ; do echo -e "\n$w"; orapki wallet display -nologo -complete -wallet $w ; done

$ ls -al config/OPMN/opmn/wallet/cwallet.sso
-rw------- 1 user group 6369 Feb 13  2020 config/OPMN/opmn/wallet/cwallet.sso
$ ls -al config/OHS/EBS_web_EBSDEV/keystores/default/cwallet.sso
-rw------- 1 user group 6369 Feb 13  2020 config/OHS/EBS_web_EBSDEV/keystores/default/cwallet.sso
$ ls -al config/OHS/EBS_web_EBSDEV/proxy-wallet/cwallet.sso
-rw------- 1 user group 4373 Jun 29  2021 config/OHS/EBS_web_EBSDEV/proxy-wallet/cwallet.sso
                         <== a default file from upgrade installation? Seems it is not used.

Now, back up old .sso files and use new file $NE_BASE/inst/$CONTEXT_NAME/certs/Apache/cwallet.sso to replace them.

6. Copy ewallet.p12 and cwallet.sso to all other nodes.
Note: New cwallet.sso has to be copied to all folders on concurrent node as well, only IF ssl was enabled on that node. Otherwise, "adop phase=fs_clone" may fail with message "Unable to initialize SSL environment, nzos call nzosSetCredential returned 28791" and strange error on the node:
LOG FILE: $INST_TOP/logs/appl/rgf/TXK/txkSetAppsConf_10141635.log ***
ERROR: The value <$FMW_HOME/webtier/instances/EBS_web_OHS3> for s_ohs_instance_loc in $CONTEXT_FILE isn't a directory.

7. Start/stop Apache as a quick test
$ adapcctl.sh start             <== Do NOT use "adopmnctl.sh startall"
$ adopmnctl.sh status
$ adopmnctl.sh stopall

8. Run autoconfig 
9. Star apps services

UPDATE in 2024: I put all above steps to a shell script for renewing cert.

- TROUBLESHOOTING
$ SUBiName=${iName%?????}
$ echo $SUBiName

Its OHS logs from adapcctl.sh are at $FMW_HOME/webtier/instances/$iName/diagnostics/logs/OHS/$SUBiName
And opmn.log is at $FMW_HOME/webtier/instances/$iName/diagnostics/logs/OPMN/opmn

a cwallet.sso file also exists in two folders under $EBS_DOMAIN_HOME/opmn/EBS*/wallet. I am not sure what it is for. But you can use orapki to view them, such as 
$ $ $FMW_HOME/oracle_common/bin/orapki wallet display -wallet $EBS_DOMAIN_HOME/opmn/EBS_web_OHS1/wallet/cwallet.sso

$ $FMW_HOME/oracle_common/bin/orapki wallet display -wallet $EBS_DOMAIN_HOME/opmn/EBS_web_OHS2/wallet/cwallet.sso

- A way to get server.cer file, read the contents of cert file, and find the expiration date of a cert file (Doc ID 2674842.1):
a) Verify the password works
$ $FMW_HOME/oracle_common/bin/orapki wallet display -wallet /path/to/ewallet.p12
Oracle PKI Tool : Version 11.1.1.9.0
Copyright (c) 2004, 2015, Oracle and/or its affiliates. All rights reserved.

Enter wallet password:
Requested Certificates:
User Certificates:
Subject:        CN=site_name.domian.com,O=company_name,L=city,ST=AZ,C=US
Trusted Certificates:
Subject:        CN=DigiCert High Assurance EV Root CA,OU=www.digicert.com,O=DigiCert Inc,C=US
Subject:        CN=company_name. Secure CA2,O=company_name,C=US

b) If troubleshooting is needed, export the certificate present in the wallet to a file
Copy the "Subject" information under the "User Certificates" section above and run command to generate server.cer file:

$ orapki wallet export -wallet /path/to/ewallet.p12 -dn 'CN=site_name.domian.com,O=company_name,L=city,ST=AZ,C=US' -cert server.crt
                                  <== it generates .crt file or .cer file, or both are the same>
$ more server.crt

c) Use the keytool to check the signature algorithm and expiry details
$ which keytool
$COMMON_TOP/util/jdk32/jre/bin/keytool      (It is a Java tool)
$ keytool -printcert -file server.crt
Owner: CN=site_name.domian.com,O=company_name,L=city,ST=AZ,C=US
Issuer: CN=company_name Secure CA2,O=company_name,C=US
Serial number: bbef1120a01b29a5c3c213623xxxxxxxx
Valid from: Thu Jan 12 20:00:00 EDT 2021 until: Tue Jan 16 19:59:59 EDT 2022
... ...
d) To read the full contents of server.cer, run openssl:
$ which openssl
/usr/bin/openssl
$ openssl version
OpenSSL 1.0.2k-fips  26 Jan 2017
$ openssl x509 -in server.crt -noout -text >> server.txt
$ more server.txt
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            05:36:63:2d:8f:8d:4b:1f:64:45:84:5a:ce:49:1a:8e
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: ... ...
... ...
- Workarounds
Since OWM 11.1.1.9 in R12.2 can not be used to read cert file, one option is to create a key pair and CSR in OWM using old cert file. After the CSR is used within Venafi to enroll for the certificate, manually install the certificate back in OWM. So, we have to create the request within OWM in multiple steps. In this case, the password for old certificate must be good. 

The another option is to use OWM in 10.1.3 ORACLE_HOME on R12.1 file system to open and save the new cert file (if old R12.1 file system is still available).

References (for creating/renewing cert file. I did not use all of them):

- Oracle Wallet Manager Shows "Certificate:Empty" When Generating Wallets Using An Existing Certificate & Key (Doc ID 1582186.1)
- How to Check Validity of Server Certificate from ORAPKI Wallet in Command Line (Doc ID 2674842.1)
- How to Replace an Expired or Expiring Certificate in FMW 11g Wallets? (Doc ID 1371209.1)
- 12.2 E-Business Suite Technology Stack Upgrade Steps To Reuse/ Migrate 11i Certificates To 12.2 Wallet Format To Replace Cwallet.sso (Doc ID 2315134.1)

Sunday, January 30, 2022

SQL to check if a R12.2 patch is applied

In EBS R12.2,  ad_bugs does not tell if an EBS patch is applied or not due to the implications of online patching and its "abort" feature. 

1. The simple way to check if a parch (33207251, for example) was applied or not in R12.2 (if node info is not important in a multi-node instance):

SQL> SELECT ad_patch.is_patch_applied('R12',-1,33207251) FROM dual;

AD_PATCH.IS_PATCH_APPLIED('R12',-1,33207251)
----------------
EXPLICIT

2. More accurate way to check patching status:

Query (modified from a query in Doc ID 443761.1) for finding out more details on patching (if it was applied by ADOP). In a multi-nodes environment, it does not tell which node, but the timestamp gives you some clue.

SQL> SELECT DISTINCT a.bug_number, e.patch_name, to_char(c.end_date, 'DD-MON-RRRR HH24:MI:SS') End_date, e.patch_type, d.patch_abstract
FROM
ad_bugs a, ad_patch_run_bugs b, ad_patch_runs c, ad_patch_drivers d , ad_applied_patches e
WHERE
a.bug_id = b.bug_id AND
b.patch_run_id = c.patch_run_id AND
c.patch_driver_id = d.patch_driver_id AND
d.applied_patch_id = e.applied_patch_id AND
a.bug_number = '33207251'
order by 1,3 ASC;

BUG_NUMBER  PATCH_NAME  END_DATE   PATCH_TYPE    PATCH_ABSTRACT
---------------  ------------  ----------------------------  ---------------------------------------
33207251  33487428  28-JAN-2022 09:14:10  ONE-OFF  ORACLE APPLICATIONS RELEASE 
                                                                                             12.2: CPU PATCH FOR JAN 2022
33207251  33487428  29-JAN-2022 08:20:21  ONE-OFF  ORACLE APPLICATIONS RELEASE
                                                                                             12.2: CPU PATCH FOR JAN 2022

NOTES: If a patch is part of another big patch, such a CPU patch, it might not show up in table ad_applied_patches or ad_adop_session_patches. For instance, patch 33207251 was applied successfully as part of Jan 2022 CPU patch 33487428, but it does not show up in queries:
SQL> select * from ad_applied_patches where patch_name in ('33207251', '33487428');
SQL> select * from ad_adop_session_patches where bug_number in ('33207251', '33487428');

3. To get the node info of applied patches, first of all, run query to check if it returns duplicate Server name or not:
SQL> select distinct node_name, appltop_id, edition_name 
            from AD_ADOP_SESSIONS
          where node_name in (select node_name from ADOP_VALID_NODES)
               and edition_name in (select edition_name from all_editions where usable='YES');
     
If it does NOT return duplicate NODE_NAME, below query could be a good one to tell if a patch was applied successfully or not on node level. But, for a cloned instance, it lists only patches that were applied after clone.

SQL> SELECT adb.bug_number, aas.name appl_top_name, adb.creation_date, adb.language,
decode(ad_patch.is_patch_applied('R12',aas.appl_top_id,adb.bug_number,adb.language),'EXPLICIT','APPLIED','NOT_APPLIED','NOT APPLIED') status
FROM ad_bugs adb,
(select distinct appltop_id appl_top_id, node_name name from ad_adop_sessions 
           where node_name in (select node_name from ADOP_VALID_NODES) ) aas
where adb.bug_number in (
'33168664',
'33207251', 
'33487428',
'26521736'
) order by creation_date desc, adb.bug_number,aas.name,adb.language;

EXPLANATION

Oracle  Doc ID 1963046.1 give a query to find if a patch was applied or not:
SQL> SELECT adb.bug_number,ad_patch.is_patch_applied('R12', \'$appl_top_id\', adb.bug_number)
FROM ad_bugs adb
WHERE adb.bug_number in ('\$patch_no\');

expected results:
       EXPLICIT = applied
       NOT APPLIED = not applied / aborted

The problem is table APPLSYS.AD_APPL_TOPS has too many obsolete data (Doc ID 2075234.1). Especially for cloned instance, it may be useless in finding APPL_TOP_ID. I find table AD_ADOP_SESSIONS has more accurate APPLTOP_ID info.  Below is an example to confirm patching 33168664 failed on 2nd node.

SQL> select distinct appltop_id, node_name, node_type from ad_adop_sessions 
           where node_name in (select node_name from ADOP_VALID_NODES);

APPLTOP_ID NODE_NAME  NODE_TYPE
----------------- -------------------- ----------
               2088  appNode1d      slave
               1088  appNode2d      master

SQL> SELECT adb.bug_number,ad_patch.is_patch_applied('R12', 1088, adb.bug_number) status
           FROM ad_bugs adb WHERE adb.bug_number in ('33168664');

BUG_NUMBER  STATUS
--------------------- -----------
33168664     EXPLICIT

SQL> SELECT adb.bug_number,ad_patch.is_patch_applied('R12', 2088, adb.bug_number) status
           FROM ad_bugs adb WHERE adb.bug_number in ('33168664');

BUG_NUMBER  STATUS
--------------------- -----------
33168664     NOT_APPLIED

Additional NOTES:

A). Doc ID 443761.1 (How To Check If a Certain Patch Was Applied to Oracle E-Business Suite Instance) also gives a way to check which patches were applied in each ADOP_SESSION_ID (patching cycle).  But, I found ad_adop_session_patches is NOT very reliable.

set pagesize 200;
set linesize 160;
column adop_session_id format 999999999999;
column bug_number format a15;
column status format a15;
column applied_file_system_base format a23;
column patch_file_system_base format a23;
column adpatch_options format a15;
column node_name format a15;
column end_date format a15;
column exec_time format a15;
column clone_status format a15;
SQL> select ADOP_SESSION_ID, BUG_NUMBER, APPLIED_FILE_SYSTEM_BASE, PATCH_FILE_SYSTEM_BASE, ADPATCH_OPTIONS, NODE_NAME, 
END_DATE,  ROUND((end_date - start_date) * 24*60,2) exec_time, CLONE_STATUS,
       DECODE(status,'N','Applied on other nodes',
                     'R','Running',
                     'H','Failed (Hard)',
                     'F','Failed (Jobs Skipped)',
                     'S','Success (worked after skipping the failed)',
                     'Y','Success',
                     'C','Clone Complete') status
from ad_adop_session_patches
order by end_date desc;

B). Snapshot update: If you are sure a patch is applied but it is not showing as applied via the above steps, then update the snapshot manually with the steps below:
1) Start adadmin after source the RUN FS env.
2) Select "2. Maintain Applications Files menu" in "AD Administration Main Menu".
3) In "Maintain Applications Files", select "4. Maintain snapshot information".
4) Select "2. Update current view snapshot" in the "Maintain Snapshot Information".
5) Select "1. Update Complete APPL_TOP" in the "Maintain Current View Snapshot Information".

C). To determine the RUN files system:
SQL> SELECT EXTRACTVALUE(XMLType(text),'//oa_context_file_loc')
 appl_top, status
  FROM fnd_oam_context_files
 WHERE name NOT IN ('TEMPLATE','METADATA','config.txt')
  AND CTX_TYPE='A'
 AND (status IS NULL OR UPPER(status) IN ('S','F'))
  AND EXTRACTVALUE(XMLType(text),'//file_edition_type') = 'run';

To determine the PATCH files system:
SQL> SELECT EXTRACTVALUE(XMLType(text),'//oa_context_file_loc')
 appl_top, status
  FROM fnd_oam_context_files
 WHERE name NOT IN ('TEMPLATE','METADATA','config.txt')
  AND CTX_TYPE='A'
 AND (status IS NULL OR UPPER(status) IN ('S','F'))
  AND EXTRACTVALUE(XMLType(text),'//file_edition_type') = 'patch';

Friday, January 21, 2022

Change Apps password in R12.2

Two steps to change APPS password in R12.2: FNDCPASS and WLS Admin Console.

1) Stop Apps services
2) $ FNDCPASS apps/old_appsPWS 0 Y system/'systemPWD' SYSTEM APPLSYS new_AppsPWD
3) AutoConfig
4) start WLS Admin services
./adadminsrvctl.sh start

5) Manually update the password in WLS Admin Console for starting oacore, etc.
See Doc ID 1674462.1 (R12.2: Steps to Change the APPS, APPLSYS, and APPS_NE Password Using FNDCPASS or AFPASSWD)
a. Log in to WLS Administration Console.
    http://[node_name.domain.com]:s_wls_adminport/console
b. Click Lock & Edit in Change Center.
c. In the Domain Structure tree, expand Services, then select Data Sources.
d. On the "Summary of JDBC Data Sources" page, select EBSDataSource.
e. On the "Settings for EBSDataSource" page, select the Connection Pool tab.
f. Enter the new password in the "Password" field.
g. Enter the new password in the "Confirm Password" field.
h. Click Save.
i. Click Activate Changes in Change Center.
6) Start all Apps services

Notes: Doc ID 2360475.1 states APPS password is not saved in any files but stored in database.

Tuesday, January 18, 2022

Change Weblogic password in R12.2

Many posts on changing Weblogic password. Below steps worked for me.

1. Stop all EBS services
2. Start Admin Server on Primary node
$ $ADMIN_SCRIPTS_HOME/adadminsrvctl.sh start
3. Run below line to change Weblogic password.
WARNING: It does not prompt to confirm the new password. The safest way is to enter it by copy/paste. If you type it wrong, it will be a disaster because Oracle does not supply a way to decrypt passwords.
 
$ perl $FND_TOP/patch/115/bin/txkUpdateEBSDomain.pl -action=updateAdminPassword

Program: txkUpdateEBSDomain.pl started at Tue ... ...

AdminServer will be re started after changing WebLogic Admin Password
All Mid Tier services should be SHUTDOWN before changing WebLogic Admin Password
Confirm if all Mid Tier services are in SHUTDOWN state. Enter "Yes" to proceed or anything else to exit: Yes

Enter the full path of Applications Context File [DEFAULT - $CONTEXT_FILE]:
Enter the WLS Admin Password:
Enter the new WLS Admin Password:
Enter the APPS user password:
... ...
*************** IMPORTANT ****************
WebLogic Admin Password is changed.
Restart all application tier services using control scripts.
********************************************
----------------------------------------
Inside generateMimeMappingsPropFile()...
----------------------------------------
$FMW_HOME/user_projects/domains/EBS_domain/config/mimemappings.properties already exists, updating it.
--------------------------------------
Inside updateMimeMappingsPropFile()...
--------------------------------------
-------------------------------------
Inside resetExistingMimeMappings()...
-------------------------------------
Overwriting the value for the parameter: png
Overwriting the value for the parameter: xml
Overwriting the value for the parameter: js
Overwriting the value for the parameter: svg
Overwriting the value for the parameter: swf
Reset of mime mappings completed.
---------------------------
Inside addMimeMappings()...
---------------------------
Adding of mime mappings completed.
Taking backup of existing mimemappings.properties.
Copying the file
----------------
SOURCE : $FMW_HOME/user_projects/domains/EBS_domain/config/mimemappings.properties
TARGET : $FMW_HOME/user_projects/domains/EBS_domain/config/mimemappings.properties_bkp
Copying temporary file as mimemappings.properties.
Copying the file
----------------
SOURCE : $FMW_HOME/user_projects/domains/EBS_domain/config/mimemappings.properties_temp
TARGET : $FMW_HOME/user_projects/domains/EBS_domain/config/mimemappings.properties
$FMW_HOME/user_projects/domains/EBS_domain/config/mimemappings.properties updated successfully.
Program: txkUpdateEBSDomain.pl completed at Tue ... ...

4. Log onto Weblogic Console and EM as weblogic using the new password.
5. Start all EBS services

Different Weblogic version may have different way to change the password. Version info (for details, see Doc ID 1051959.1 How To Find the Full WebLogic Server Version and Full Patch Level):

$ cd $FMW_HOME/user_projects/domains/EBS_domain_${TWO_TASK}/servers/AdminServer/logs
$ grep WebLogic AdminServer.log
... ...
<WebLogic Server "AdminServer" version:
WebLogic Server 10.3.6.0.210119 PSU Patch for BUG32052267 Mon Nov 23 07:28:31 UTC 2020
WebLogic Server Temporary Patch for BUG13964737 Fri Dec 20 11:32:08 IST 2013
WebLogic Server Temporary Patch for BUG20474010 Sun Mar 01 17:22:18 IST 2015
WebLogic Server Temporary Patch for ${CRS} Mon Jul 30 16:45:20 EDT 2012
WebLogic Server Temporary Patch for ${CRS} Mon Jul 30 16:45:20 EDT 2012
WebLogic Server 10.3.6.0  Tue Nov 15 08:52:36 PST 2011 1441050  Copyright (c) 1995, 2011, Oracle and/or its affiliates. All rights reserved.>   ... ...

REFERENCES:

How to Change /Reset /Retrieve the WebLogic Server Administrator Password - All Versions (Doc ID 1082299.1)
How to Decrypt WLS Passwords using WLST? ( Doc ID 2732961.1 )