After EBS R12 was copied/moved to a new server due to Linux OS upgrade, autoconfig worked and all EBS services started without any issue. But after successful login, EBS R12 forms failed to open during startup. The error is
FRM-92101 : There was a failure in the Forms Server during startup.
This could happen due to invalid configuration. Please look into the web-server log file for details.
Click on Detail to see
Java Exception:
oracle.forms.net.ConnectionException: Forms session <2> failed during startup: no response from runtime process
at oracle.forms.net.ConnectionException.createConnectionException(Unknown Source)
at oracle.forms.net.HTTPNStream.getResponse(Unknown Source)
at oracle.forms.net.HTTPNStream.doFlush(Unknown Source)
at oracle.forms.net.HTTPNStream.flush(Unknown Source)
at java.io.DataOutputStream.flush(Unknown Source)
at oracle.forms.net.HTTPConnection.connect(Unknown Source)
at oracle.forms.engine.FormsDispatcher.initConnection(Unknown Source)
at oracle.forms.engine.FormsDispatcher.init(Unknown Source)
at oracle.forms.engine.Runform.initConnection(Unknown Source)
at oracle.forms.engine.Runform.startRunform(Unknown Source)
at oracle.forms.engine.Main.createRunform(Unknown Source)
at oracle.forms.engine.Main.start(Unknown Source)
… ...
Look into log file application.log in folder $LOG_HOME/ora/10.1.3/j2ee/forms/forms_default_group_1, see luckily a message:
runtime process failed during startup with errors $ORACLE_HOME/bin/frmweb: error while loading shared libraries: libXm.so.2: cannot open shared object file: No such file or directory
The problem was caused by missing Linux package openmotif21 when the new server was configured. EBS forms function works after install the right openmotif rpm and bounce EBS services.
$ rpm -qa | grep openmotif
openmotif21-2.1.30-11.EL6.i686
Saturday, May 16, 2020
Wednesday, April 22, 2020
Move EBS from Linux 6 to Linux 7
1. Build new server with a temporary name and install RHEL7 on it (by Linux Admin)
It is very important to rename it later to the same server name for EBS apps to work properly.
Linux Admin has to follow Oracle Doc ID 761566.1 (Oracle E-Business Suite Installation and Upgrade Notes Release 12 (12.1.1) for Linux x86-64) to install all required RHEL7 (Red Hat Enterprise Linux 7) components.
$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.7 (Maipo)
$ cd /usr/bin
$ ls -altr java*
-rwxr-xr-x 1 root root 5530 Jul 22 2019 javaws.itweb
lrwxrwxrwx 1 root root 24 Feb 13 09:24 javaws -> /etc/alternatives/javaws
lrwxrwxrwx 1 root root 27 Feb 18 19:16 javac -> /usr/java/default/bin/javac
lrwxrwxrwx 1 root root 29 Feb 18 19:16 javadoc -> /usr/java/default/bin/javadoc
lrwxrwxrwx 1 root root 22 Feb 18 19:45 java -> /etc/alternatives/java
$ java -version
openjdk version "1.8.0_232"
OpenJDK Runtime Environment (build 1.8.0_232-b09)
OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode)
$ javac -version
javac 1.7.0_131
3. Verify the new RHEL7 server
$ ls -l ebs*
-rwxr-xr-x 1 root root 545 Jun 21 2016 ebs1
-rwxr-xr-x 1 root root 549 Jun 21 2016 ebs2
4) A soft link for libdb-4.3.so under /usr/lib on RHEL7 server is necessary. Otherwise, Apache service will not start on RHEL7 sever by error
$ $ADJVAPRG -version
java version "1.7.0_231"
Java(TM) SE Runtime Environment (build 1.7.0_231-b08)
Java HotSpot(TM) Client VM (build 24.231-b08, mixed mode)
$ echo $IAS_ORACLE_HOME
/path/to/apps/tech_st/10.1.3
$ $IAS_ORACLE_HOME/appsutil/jdk/jre/bin/java -fullversion
java full version "1.7.0_231-b08"
$ echo $ORACLE_HOME
/path/to/apps/tech_st/10.1.2
$ $ORACLE_HOME/jdk/bin/java -fullversion
java full version "1.7.0_231-b08
5. Make final switch
1) Stop EBS apps services on RHEL6 servers
2) Copy all file partitions from RHEL6 servers to RHEL7 servers (using rsync by Linux Admin)
3) Rename new RHEL7 server back to the old server name
4) Change network DNS with the new IP address for RHEL7 server name
5) Modify F5 configuration with the new IP addresses of new EBS hosts
6) Run autoconfig on all EBS nodes on RHEL7 servers
7) Start EBS apps services on RHEL7 servers
8) Make sure file system backup is activated on new servers
It is very important to rename it later to the same server name for EBS apps to work properly.
Linux Admin has to follow Oracle Doc ID 761566.1 (Oracle E-Business Suite Installation and Upgrade Notes Release 12 (12.1.1) for Linux x86-64) to install all required RHEL7 (Red Hat Enterprise Linux 7) components.
$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.7 (Maipo)
$ cd /usr/bin
$ ls -altr java*
-rwxr-xr-x 1 root root 5530 Jul 22 2019 javaws.itweb
lrwxrwxrwx 1 root root 24 Feb 13 09:24 javaws -> /etc/alternatives/javaws
lrwxrwxrwx 1 root root 27 Feb 18 19:16 javac -> /usr/java/default/bin/javac
lrwxrwxrwx 1 root root 29 Feb 18 19:16 javadoc -> /usr/java/default/bin/javadoc
lrwxrwxrwx 1 root root 22 Feb 18 19:45 java -> /etc/alternatives/java
$ java -version
openjdk version "1.8.0_232"
OpenJDK Runtime Environment (build 1.8.0_232-b09)
OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode)
$ javac -version
javac 1.7.0_131
3. Verify the new RHEL7 server
1) Make sure /etc/oraInst.loc is copied over and has right permission.
2) Check all files under /etc/init.d are copied over for auto stop/start EBS services by server's reboot. For example, below two shell scripts stop/start EBS services of two instances.
$ cd /etc/init.d$ ls -l ebs*
-rwxr-xr-x 1 root root 545 Jun 21 2016 ebs1
-rwxr-xr-x 1 root root 549 Jun 21 2016 ebs2
3) It is good to have /usr/bin/xclock, ftp, sftp, gzip installed.
httpd: error while loading shared libraries: libdb-4.3.so: cannot open shared object file: No such file or directory
$ cd /usr/lib
$ ls -al libdb*.*
lrwxrwxrwx 1 root root 12 Feb 4 11:30 libdb-4.3.so -> libdb-5.3.so
-rwxr-xr-x 1 root root 1868348 Jan 30 2019 libdb-5.3.so
lrwxrwxrwx 1 root root 12 Dec 10 15:21 libdb-5.so -> libdb-5.3.so
-rw-r--r-- 1 root root 5825 Dec 11 10:52 libdb.so.2
lrwxrwxrwx 1 root root 12 Feb 4 11:30 libdb-4.3.so -> libdb-5.3.so
-rwxr-xr-x 1 root root 1868348 Jan 30 2019 libdb-5.3.so
lrwxrwxrwx 1 root root 12 Dec 10 15:21 libdb-5.so -> libdb-5.3.so
-rw-r--r-- 1 root root 5825 Dec 11 10:52 libdb.so.2
Note: This is different from RHEL6 server, and Apache worked fine.
$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 6.10 (Santiago)
$ ls -al /usr/lib/libdb*.*
-rw-r--r-- 1 root root 5825 Nov 11 2016 /usr/lib/libdb.so.2
4. Upgrade EBS JDK from JDK 6 to JDK 7 on RHEL6 host. This step is an important pre-step for RHEL7 upgrade, and is not necessary if JDK 7 was used within EBS already. See blow on how to upgrade JDK:
https://erpondb.blogspot.com/2019/12/upgrade-jdk-to-jdk-7-in-ebs-r121.html$ $ADJVAPRG -version
java version "1.7.0_231"
Java(TM) SE Runtime Environment (build 1.7.0_231-b08)
Java HotSpot(TM) Client VM (build 24.231-b08, mixed mode)
$ echo $IAS_ORACLE_HOME
/path/to/apps/tech_st/10.1.3
$ $IAS_ORACLE_HOME/appsutil/jdk/jre/bin/java -fullversion
java full version "1.7.0_231-b08"
$ echo $ORACLE_HOME
/path/to/apps/tech_st/10.1.2
$ $ORACLE_HOME/jdk/bin/java -fullversion
java full version "1.7.0_231-b08
5. Make final switch
1) Stop EBS apps services on RHEL6 servers
2) Copy all file partitions from RHEL6 servers to RHEL7 servers (using rsync by Linux Admin)
3) Rename new RHEL7 server back to the old server name
4) Change network DNS with the new IP address for RHEL7 server name
5) Modify F5 configuration with the new IP addresses of new EBS hosts
6) Run autoconfig on all EBS nodes on RHEL7 servers
7) Start EBS apps services on RHEL7 servers
8) Make sure file system backup is activated on new servers
Friday, March 27, 2020
Re-new Java Signing Key
The server certificate for JAR files expires after some years, for example, in each two years. That is the reason we have to renew it periodically. After run renewal, I got better understanding on the process than in my post Keystore & Java Signing in 2016.
1. Find the current keystore password and key password embedded in EBS R12.1 instance
SQL> set serveroutput on
SQL> declare
spass varchar2(30);
kpass varchar2(30);
begin
ad_jar.get_jripasswords(spass, kpass);
dbms_output.put_line(spass);
dbms_output.put_line(kpass);
end;
/
puneet
myxuan
If you want to them, use "$ adjkey -storepasswd" and "$ adjkey -keypasswd" to do that.
$ echo $TWO_TASK
EBSDEV
$ hostname
devserver1d
$ keytool -list -keystore adkeystore.dat
Enter keystore password:
Keystore type: JKS
Keystore provider: SUN
Your keystore contains 1 entry
ebsdev_devserver1d, Apr 15, 2019, PrivateKeyEntry,
Certificate fingerprint (SHA1): A7:01:6E:76:D7:2D:F5: … ...
$ keytool -list -v -keystore adkeystore.dat <= View details
Enter keystore password:
Keystore type: JKS
Keystore provider: SUN
Your keystore contains 1 entry
Alias name: ebsdev_devserver1d
Creation date: Apr 15, 2019
Entry type: PrivateKeyEntry
Certificate chain length: 3
Certificate[1]:
Owner: CN=Company name, O=Company name, L=City, ST=State, C=US
Issuer: CN=DigiCert SHA2 Assured ID Code Signing CA, OU=www.digicert.com, O=DigiCert Inc, C=US
Serial number: f7d45adf32f16e ….
Valid from: Thu Jan 03 19:00:00 EST 2019 until: Wed Jan 12 07:00:00 EST 2022
Certificate fingerprints:
MD5: 17:68:2F:67:CC:F0: … …
SHA1: A7:01:6E:76:D7:2D: … …
Signature algorithm name: SHA256withRSA
Version: 3
Certificate[2]:
Owner: CN=DigiCert SHA2 Assured ID Code Signing CA, OU=www.digicert.com, O=DigiCert Inc, C=US
Issuer: CN=DigiCert Assured ID Root CA, OU=www.digicert.com, O=DigiCert Inc, C=US
Serial number: 409181b5fd5bb667 ......
Valid from: Tue Oct 22 08:00:00 EDT 2013 until: Sun Oct 22 08:00:00 EDT 2028
Certificate fingerprints:
MD5: B6:56:37:6C:3D:2A: … …
SHA1: 92:C1:58:8E:85:AF: … …
Signature algorithm name: SHA256withRSA
Version: 3
Extensions:
#1:
… … …
Certificate[3]:
Owner: CN=DigiCert Assured ID Root CA, OU=www.digicert.com, O=DigiCert Inc, C=US
Issuer: CN=DigiCert Assured ID Root CA, OU=www.digicert.com, O=DigiCert Inc, C=US
Serial number: ce7e0e517d8 … …
Valid from: Thu Nov 09 19:00:00 EST 2006 until: Sun Nov 09 19:00:00 EST 2031
Certificate fingerprints:
MD5: 87:CE:0B:7B:2A: … …
SHA1: 05:63:B8:63:0D: … …
Signature algorithm name: SHA1withRSA
Version: 3
Extensions:
#1: ObjectId: 2.5.29.15 Criticality=true
KeyUsage [
DigitalSignature
Key_CertSign
Crl_Sign
]
#2: ObjectId: 2.5.29.19 Criticality=true
BasicConstraints:[
CA:true
PathLen:2147483647
]
#3: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 45 EB 9C A7 A7 21 9D E.......1-Q...!.
0010: F3 6D C8 0F .m..
]
]
#4: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: 45 EB 8B A7 A7 21 9D E.......1-Q...!.
0010: F3 6D C8 0F .m..
]
]
Enter keystore password: <= puneet
Enter key password for <ebsdev_devserver1d> <= keyPWD
New key password for <ebsdev_devserver1d>: <= myxuan
Re-enter new key password for <ebsdev_devserver1d>: <= myxuan
4. Make adkeystore.dat to work in other EBS instances.
The original adkeystore.dat is only for instance EBSDEV. If you have multiple instances, below command worked for me to make it work for QA instance EBSQA (on different servers):
$ cp adkeystore.dat $APPL_TOP/admin
$ cd $APPL_TOP/admin
$ keytool -changealias -alias ebsdev_devserver1d -destalias EBSQA_qaserver1q -keystore adkeystore.dat
Enter keystore password:
Enter key password for <ebsdev_devserver1d>
$ keytool -list -keystore adkeystore.dat
Enter keystore password:
Keystore type: JKS
Keystore provider: SUN
Your keystore contains 1 entry
ebsqa_qaserver1q, Apr 20, 2019, PrivateKeyEntry,
Certificate fingerprint (SHA1): A7:01:6E:76:D7:2D:F5: … ...
NOTES: After JDK was upgraded to JDK 7, adadmin gives warning on more files when generating JAR files. Depending on what EBS components are used, usually it's safe to ignore them.
adearea();
Warning=> unable to locate $FND_TOP/java/3rdparty/stdalone/odi-sdk-invocation10g.zip file.
Warning=> unable to locate $FND_TOP/java/3rdparty/stdalone/ucmscript.zip file.
Warning=> unable to locate $FND_TOP/java/3rdparty/stdalone/oracle-ucm-ridc-12_1_2_0_0.zip file.
Warning=> unable to locate $FND_TOP/java/3rdparty/stdalone/gcm-server.zip file.
Warning=> unable to locate $FND_TOP/java/3rdparty/stdalone/json_simple-1-1.zip file.
6. Verify file's new timestamp and new expiration date
Warning:
1. Find the current keystore password and key password embedded in EBS R12.1 instance
SQL> set serveroutput on
SQL> declare
spass varchar2(30);
kpass varchar2(30);
begin
ad_jar.get_jripasswords(spass, kpass);
dbms_output.put_line(spass);
dbms_output.put_line(kpass);
end;
/
puneet
myxuan
If you want to them, use "$ adjkey -storepasswd" and "$ adjkey -keypasswd" to do that.
2. $APPL_TOP/admin/adkeystore.dat is a binary file, which includes, in my file, one key entry and 3 certificates. Use keytool to view its contents:
$ echo $TWO_TASK
EBSDEV
$ hostname
devserver1d
$ keytool -list -keystore adkeystore.dat
Enter keystore password:
Keystore type: JKS
Keystore provider: SUN
Your keystore contains 1 entry
ebsdev_devserver1d, Apr 15, 2019, PrivateKeyEntry,
Certificate fingerprint (SHA1): A7:01:6E:76:D7:2D:F5: … ...
$ keytool -list -v -keystore adkeystore.dat <= View details
Enter keystore password:
Keystore type: JKS
Keystore provider: SUN
Your keystore contains 1 entry
Alias name: ebsdev_devserver1d
Creation date: Apr 15, 2019
Entry type: PrivateKeyEntry
Certificate chain length: 3
Certificate[1]:
Owner: CN=Company name, O=Company name, L=City, ST=State, C=US
Issuer: CN=DigiCert SHA2 Assured ID Code Signing CA, OU=www.digicert.com, O=DigiCert Inc, C=US
Serial number: f7d45adf32f16e ….
Valid from: Thu Jan 03 19:00:00 EST 2019 until: Wed Jan 12 07:00:00 EST 2022
Certificate fingerprints:
MD5: 17:68:2F:67:CC:F0: … …
SHA1: A7:01:6E:76:D7:2D: … …
Signature algorithm name: SHA256withRSA
Version: 3
Extensions:
#1:
… … …Certificate[2]:
Owner: CN=DigiCert SHA2 Assured ID Code Signing CA, OU=www.digicert.com, O=DigiCert Inc, C=US
Issuer: CN=DigiCert Assured ID Root CA, OU=www.digicert.com, O=DigiCert Inc, C=US
Serial number: 409181b5fd5bb667 ......
Valid from: Tue Oct 22 08:00:00 EDT 2013 until: Sun Oct 22 08:00:00 EDT 2028
Certificate fingerprints:
MD5: B6:56:37:6C:3D:2A: … …
SHA1: 92:C1:58:8E:85:AF: … …
Signature algorithm name: SHA256withRSA
Version: 3
Extensions:
#1:
… … …
Certificate[3]:
Owner: CN=DigiCert Assured ID Root CA, OU=www.digicert.com, O=DigiCert Inc, C=US
Issuer: CN=DigiCert Assured ID Root CA, OU=www.digicert.com, O=DigiCert Inc, C=US
Serial number: ce7e0e517d8 … …
Valid from: Thu Nov 09 19:00:00 EST 2006 until: Sun Nov 09 19:00:00 EST 2031
Certificate fingerprints:
MD5: 87:CE:0B:7B:2A: … …
SHA1: 05:63:B8:63:0D: … …
Signature algorithm name: SHA1withRSA
Version: 3
Extensions:
#1: ObjectId: 2.5.29.15 Criticality=true
KeyUsage [
DigitalSignature
Key_CertSign
Crl_Sign
]
#2: ObjectId: 2.5.29.19 Criticality=true
BasicConstraints:[
CA:true
PathLen:2147483647
]
#3: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 45 EB 9C A7 A7 21 9D E.......1-Q...!.
0010: F3 6D C8 0F .m..
]
]
#4: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: 45 EB 8B A7 A7 21 9D E.......1-Q...!.
0010: F3 6D C8 0F .m..
]
]
3. Before cert's expiration date (01/12/2022 in above keystore), you have to renew it. First, you make request to Security team for a new keystore file that will work for one EBS instance. Then, run adadmin to re-generate JAR files to that instance (as in Step 5).
I do not know what tool Security team uses to create file adkeystore.dat. If the password in new adkeystore.dat do not match EBS instance's password in Step 1, adadmin in Step 5 will fail with error - jarsigner: unable to recover key from keystore. Commands to change keystore password and key password:
$ keytool -storepasswd -keystore /path/to/temp/adkeystore.dat
Enter keystore password: <= storePwd
New keystore password: <= puneet
Re-enter new keystore password: <= puneet
I do not know what tool Security team uses to create file adkeystore.dat. If the password in new adkeystore.dat do not match EBS instance's password in Step 1, adadmin in Step 5 will fail with error - jarsigner: unable to recover key from keystore. Commands to change keystore password and key password:
$ keytool -storepasswd -keystore /path/to/temp/adkeystore.dat
Enter keystore password: <= storePwd
New keystore password: <= puneet
Re-enter new keystore password: <= puneet
Enter keystore password: <= puneet
Enter key password for <ebsdev_devserver1d> <= keyPWD
New key password for <ebsdev_devserver1d>: <= myxuan
Re-enter new key password for <ebsdev_devserver1d>: <= myxuan
4. Make adkeystore.dat to work in other EBS instances.
The original adkeystore.dat is only for instance EBSDEV. If you have multiple instances, below command worked for me to make it work for QA instance EBSQA (on different servers):
$ cp adkeystore.dat $APPL_TOP/admin
$ cd $APPL_TOP/admin
$ keytool -changealias -alias ebsdev_devserver1d -destalias EBSQA_qaserver1q -keystore adkeystore.dat
Enter keystore password:
Enter key password for <ebsdev_devserver1d>
$ keytool -list -keystore adkeystore.dat
Enter keystore password:
Keystore type: JKS
Keystore provider: SUN
Your keystore contains 1 entry
ebsqa_qaserver1q, Apr 20, 2019, PrivateKeyEntry,
Certificate fingerprint (SHA1): A7:01:6E:76:D7:2D:F5: … ...
5. Generate JAR using adadmin utility which will take the new certificates in adkeystore.dat
$ echo $TWO_TASK
EBSQA
$ adstpall.sh
$ adadmin
==> 1 Select Generate Applications Files
==> 4 Generate Product JAR Files Yes. (Do force the regeneration of all JAR files.)
$ adstpall.sh
$ adadmin
==> 1 Select Generate Applications Files
==> 4 Generate Product JAR Files Yes. (Do force the regeneration of all JAR files.)
(adadmin can be used to re-load JAR files, which is not required, by
==> 3 Compile/Reload Applications Database Entities menu
==> 3 Reload JAR Files to Database. Seems it uploads some .jar files, but not all .jar files)
==> 3 Compile/Reload Applications Database Entities menu
==> 3 Reload JAR Files to Database. Seems it uploads some .jar files, but not all .jar files)
NOTES: After JDK was upgraded to JDK 7, adadmin gives warning on more files when generating JAR files. Depending on what EBS components are used, usually it's safe to ignore them.
adearea();
Warning=> unable to locate $FND_TOP/java/3rdparty/stdalone/odi-sdk-invocation10g.zip file.
Warning=> unable to locate $FND_TOP/java/3rdparty/stdalone/ucmscript.zip file.
Warning=> unable to locate $FND_TOP/java/3rdparty/stdalone/oracle-ucm-ridc-12_1_2_0_0.zip file.
Warning=> unable to locate $FND_TOP/java/3rdparty/stdalone/gcm-server.zip file.
Warning=> unable to locate $FND_TOP/java/3rdparty/stdalone/json_simple-1-1.zip file.
6. Verify file's new timestamp and new expiration date
$ ls -al $AD_TOP/java/jar/adxlib.jar <= pick up a JAR file
$ jarsigner -verify -verbose -certs $AD_TOP/java/jar/adxlib.jar
... ... ...
sm 7732 Wed Jan 02 05:02:02 EST 2002 oracle/apps/ad/util/zip/ZipOutputStream.class
$ jarsigner -verify -verbose -certs $AD_TOP/java/jar/adxlib.jar
... ... ...
sm 7732 Wed Jan 02 05:02:02 EST 2002 oracle/apps/ad/util/zip/ZipOutputStream.class
X.509, CN=Company name, O=Company name, L=City, ST=State, C=US
[certificate is valid from 1/3/19 7:00 PM to 1/12/22 7:00 AM]
X.509, CN=DigiCert SHA2 Assured ID Code Signing CA, OU=www.digicert.com, O=DigiCert Inc, C=US
[certificate is valid from 10/22/13 8:00 AM to 10/22/28 8:00 AM]
X.509, CN=DigiCert Assured ID Root CA, OU=www.digicert.com, O=DigiCert Inc, C=US
[certificate is valid from 11/9/06 7:00 PM to 11/9/31 7:00 PM]
... ... ...
- Signed by "CN=Company name, O=Company name, L=City, ST=State, C=US"
Digest algorithm: SHA1
Signature algorithm: SHA1withRSA, 2048-bit key
[certificate is valid from 1/3/19 7:00 PM to 1/12/22 7:00 AM]
X.509, CN=DigiCert SHA2 Assured ID Code Signing CA, OU=www.digicert.com, O=DigiCert Inc, C=US
[certificate is valid from 10/22/13 8:00 AM to 10/22/28 8:00 AM]
X.509, CN=DigiCert Assured ID Root CA, OU=www.digicert.com, O=DigiCert Inc, C=US
[certificate is valid from 11/9/06 7:00 PM to 11/9/31 7:00 PM]
... ... ...
- Signed by "CN=Company name, O=Company name, L=City, ST=State, C=US"
Digest algorithm: SHA1
Signature algorithm: SHA1withRSA, 2048-bit key
jar verified.
Warning:
This jar contains signatures that does not include a timestamp. Without a timestamp, users may not
be able to validate this jar after the signer certificate's expiration date (2019-06-05) or after
any future revocation date.
be able to validate this jar after the signer certificate's expiration date (2019-06-05) or after
any future revocation date.
(Not sure why getting this warning)
NOTE: keytool is part of JDK
NOTE: keytool is part of JDK
$ which keytool
$IAS_ORACLE_HOME/appsutil/jdk/jre/bin/keytool
Tuesday, March 24, 2020
Enable TLS1.2 in EBS R12.1
I followed Doc ID 376700.1 (Enabling TLS in Oracle E-Business Suite Release 12.1) to encrypt all connections for Oracle E-Business Suite Release 12.1.3 in RHEL7 using Transport Layer Security (TLS) protocol 1.2.
To check which TLS 1.X is used, on Chrome Ctrl+Shift+I to bring up a DevTool. Click on Security tab to see security info.
======================
- PRE REQUISITES
======================
1) HTTP Server (OHS) 10.1.3.5.0
If needed, use Doc ID 454811.1 (Upgrading to the Latest OracleAS 10g 10.1.3.x Patch Set in Oracle E-Business Suite Release 12) to upgrade it.
$ cd $INST_TOP/ora/10.1.3
$ . XXXX_xxxx.env
2) JDK 7 is required
I followed Doc ID 1467892.1 to upgrade JDK 6 to JDK 7 before working on this. Verify:
$ echo $IAS_ORACLE_HOME
/path/to/apps/tech_st/10.1.3
$ $IAS_ORACLE_HOME/appsutil/jdk/jre/bin/java -fullversion
java full version "1.7.0_231-b08"
$ echo $ORACLE_HOME
/path/to/apps/tech_st/10.1.2
$ $ORACLE_HOME/jdk/bin/java -fullversion
java full version "1.7.0_231-b08"
3) EBS site uses HTTPS
In my EBS sites, digital certificate uses PKCS#12 (Public Key Cryptography Standard #12) encryption. We download and re-new it from a website managed by company Security team. When we download it, the website ask to enter a password, say 'sslPWD01'.
$ cd $INST_TOP/certs/Apache
$ ls -al
-rw-r----- 1 user ogroup 6413 Jan 15 16:05 ewallet.p12
-rw------- 1 user ogroup 6441 Jan 15 16:05 cwallet.sso
======================
- Apply patches for enabling TLS1.2
======================
1) 3 patches to 10.1.3.5 HOME
$ cd $ADMIN_SCRIPTS_HOME
$ adstpall.sh
$ cd $INST_TOP/ora/10.1.3
$ . XXXXX_xxxxxx.env
$ echo $ORACLE_HOME
$ vi /etc/oraInst.loc
$ opatch lsinventory | grep 21845942 <== yes, 2015 CPU was applied before
$ cd /ebsu01/app/patchTLS12 (where all files are saved in)
-- 29292327 (Product Patched: Oracle HTTP Server. It includes newer "openssl")
-- unzip p29292327_101350_LINUX.zip
$ cd 29292327
$ opatch apply
-- 27208670 (Product Patched: OPMN. It include newer "opmn")
Note it is necessary to install a newer SDK to 10.1.3.5 OH to avoid error (from applying patch 27208670, Doc ID 2555323.1):
OPATCH_JAVA_ERROR=CheckConflict: OPatch cannot process overlay patches because of no OUI support. Please take latest OUI 10.1 patchset from "My Oracle Support" and try again.
$ cd $ORACLE_HOME
$ mv OPatch OPatch.pre_6640838
$ cd /ebsu01/app/patchForms (where p6640838_10106_Linux-x86-64.zip is unzipped)
$ cd cd/Disk1/install
$ export DISPLAY=66.666.666.666:0.0
$ ./runInstaller -ignoreSysPrereqs &
Steps for install (very slowly):
a) Choose next on the welcome page.
b) Choose the install type as Custom
c) Enter the 10.1.3.5 ORACLE_HOME where OUI components have to be installed.
d) On the Available product components page, select the checkbox to show all components.
e) Then, click on Expend All
f) You can de-select
Oracle Installation Libraries 10.1.0.6.0
Ensure below are selected.
x Oracle Universal Installer
x Installer SDK Component 10.1.0.6.0
x Oracle One-Off Patch Installer
g) Then, click on Next to Install.
$ cd $ORACLE_HOME
$ mv OPatch OPatch_delete
$ mv OPatch.pre_6640838 OPatch
-- Now, apply patch 27208670
$ cd /ebsu01/app/patchTLS12
$ unzip p27208670_101350_LINUX.zip
$ cd 27208670
$ opatch apply <== "27208670" overlays " 21845942 "
$ cd ..
$ rm -rf 27208670
-- 22322938 Product Patched: OC4J (update Java Mail API to 1.5.4)
-- unzip p22322938_101350_Generic.zip
$ cd 22322938
$ opatch apply => Answer N to continue (Rolling back patch 8999551)
2) 4 patches by adpatch
SQL> select * from ad_bugs where bug_number in (
'23645824', -- 23645824:R12.TXK.B (R12.TXK.B.delta.3. for FORMSAPP.EAR DEPLOYMENT FAILS)
'22974534', -- 22974534:R12.OWF.B (for OAF EMAIL)
'27881758', -- 27881758:R12.OWF.B (connect to OUTLOOK.OFFICE365.COM. superseded by 28779647)
'24677849' -- 24677849:R12.TXK.B (fix MAILER FAILS)
);
Merge those 4 patches to tls12_merge.zip
NOTE: Start a new OS session to take 10.1.2 ORACLE_HOME to avoid adpatch error:
adogjf() Unable to copy Registry.Dat.
$ adadmin => to enable maintenance mode
$ cd /ebsu01/app/patchTLS12
-- unzip tls12_merge.zip
$ cd tls12_merge
$ echo $ORACLE_HOME => 10.1.2
$ ls
$ adpatch
3) Using EBS openssl (that comes from patch 29292327)
$ openssl version
$ ls -al $IAS_ORACLE_HOME/Apache/open_ssl/bin
-rw-r--r-- 1 user ogroup 616303 Mar 23 12:48 openssl
-rw-r--r-- 1 user ogroup 10835 Mar 23 12:48 openssl.cnf
$ cd $IAS_ORACLE_HOME/Apache/open_ssl/bin
$ chmod 755 openssl
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$IAS_ORACLE_HOME/lib
$ export OPENSSL_CONF=$IAS_ORACLE_HOME/Apache/open_ssl/bin/openssl.cnf
NOTES: without them, openssl may give error/warning
openssl: error while loading shared libraries: libssl.so.1.0.0: cannot open shared object file: No such file or directory
WARNING: can't open config file: /home/nse/workspace/openssl-linux_x86_OL5/install/openssl.cnf
$ export PATH=$IAS_ORACLE_HOME/Apache/open_ssl/bin:$PATH
$ which openssl
~/$TWO_TASK/apps/tech_st/10.1.3/Apache/open_ssl/bin/openssl
$ openssl version
OpenSSL 1.0.2q 20 Nov 2018
1) Get server key and certificates
$ cd $INST_TOP/certs/Apache
$ openssl pkcs12 -in ewallet.p12 -out allcerts.crt -nokeys -password pass:sslPWD01
MAC verified OK
Warning unsupported bag type: secretBag
NOTES: It creates allcerts.crt file in which has 3 certs. Need to manually extract certs from this file:
server.crt (keyword: server name) -
subject=/C=US/ST=state/L=city/O=company name/CN=servername.domain.com
intermediate.crt (with company name) -
subject=/C=US/O=company name/CN=company name. Secure CA2
root cert (ca.crt) -
subject=/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance EV Root CA
But, intermediate.crt and root cert (ca.crt) are universal within the company network. They can be copied from other instances. So, only server.crt is truly new.
$ cat server.crt intermediate.crt ca.crt > opmn.crt
By now, $INST_TOP/certs/Apache has below files:
-rw-r----- 1 user ogroup 6349 Jan 15 16:05 ewallet.p12
-rw------- 1 user ogroup 6377 Jan 15 16:05 cwallet.sso
-rw-r--r-- 1 user ogroup 1367 Mar 17 17:16 ca.crt
-rw-r--r-- 1 user ogroup 1684 Mar 17 17:20 intermediate.crt
-rw-r--r-- 1 user ogroup 1970 Mar 23 13:26 jyy.key
-rw-r--r-- 1 user ogroup 1675 Mar 23 13:26 server.key
-rw-r--r-- 1 user ogroup 2342 Mar 23 13:28 server.crt
-rw-r--r-- 1 user ogroup 5393 Mar 23 15:46 opmn.crt
Additional commands
Do not know what below will do:
openssl pkcs12 -in ewallet.p12 -out A_certs.crt -nokeys -nodes -password pass:sslPWD01
openssl pkcs12 -in ewallet.p12 -out CA_cert.crt -nokeys -cacerts -chain -password pass:sslPWD01
2) Modify or create template files in $FND_TOP/admin/template/custom. They can be shared out to other EBS instances. Below are for enabling TLS 1.2 only (and disabling others):
$ cd $FND_TOP/admin/template/custom
$ mv opmn_xml_1013.tmp opmn_xml_1013.tmp_BK (<= in case there was one)
$ cp -p ../opmn_xml_1013.tmp .
$ vi opmn_xml_1013.tmp
Replace this line in the template:
<ssl enabled="true" wallet-file="%s_web_ssl_directory%/opmn"/>
With the following:
<ssl enabled="true" openssl-certfile="%s_web_ssl_directory%/Apache/opmn.crt" openssl-keyfile="%s_web_ssl_directory%/Apache/server.key" openssl-password="change1t" openssl-lib="%s_weboh_oh%/lib" ssl-versions="TLSv1.2" ssl-ciphers="ECDHE-RSA-AES128-GCM-SHA256,ECDHE-RSA-AES256-GCM-SHA384"/>
$ diff opmn_xml_1013.tmp opmn_xml_1013.tmp_BK
14c14
< <ssl enabled="true" openssl-certfile="%s_web_ssl_directory%/Apache/opmn.crt" openssl-keyfile="%s_web_ssl_directory%/Apache/server.key" openssl-password="change1t" openssl-lib="%s_weboh_oh%/lib" ssl-versions="TLSv1.2" ssl-ciphers="ECDHE-RSA-AES128-GCM-SHA256,ECDHE-RSA-AES256-GCM-SHA384"/>
---
> <ssl enabled="true" wallet-file="%s_web_ssl_directory%/opmn"/>
$ cp -p ../ssl_conf_1013.tmp .
Perform the following
Step 1 - Comment out the following line in the template:
#SSLWallet file:%s_web_ssl_directory%/Apache
Step 2 - Add the following 3 lines into the template:
SSLCertificateFile %s_web_ssl_directory%/Apache/server.crt
SSLCertificateKeyFile %s_web_ssl_directory%/Apache/server.key
SSLCertificateChainFile %s_web_ssl_directory%/Apache/intermediate.crt
Step 3 - Comment out two lines
# SSLProtocol -all +TLSv1 +SSLv3
# SSLCipherSuite HIGH:MEDIUM:!aNULL:+SHA1:+MD5:+HIGH:+MEDIUM
Step 4 - Add two lines to it:
SSLProtocol TLSv1.2
SSLCipherSuite ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384
$ diff oc4j_properties_1013.tmp ../oc4j_properties_1013.tmp
72,73d71
< https.protocols=TLSv1.2
<
$ diff oafm_oc4j_properties_1013.tmp ../oafm_oc4j_properties_1013.tmp
68,69d67
< https.protocols=TLSv1.2
<
$ diff forms_oc4j_properties_1013.tmp ../forms_oc4j_properties_1013.tmp
72,73d71
< https.protocols=TLSv1.2
<
3) Run autoconfig and start services
$ adadmin => disable maintenance mode
$ cd $ADMIN_SCRIPTS_HOME
$ adautocfg.sh
$ adstrtal.sh
$ openssl s_client -connect site_name.domian.com:port -tls1_2
TROUBLESHOOTING:
1. After all above steps completed, adstrtal.sh failed to start oacore service in the first time by Timed-out error (code 204). It is fixed by cleaning session lock files in persistence directories .
2. If the SSL root certificate (ca.crt) is incorrect, EBS GUI will give warning in popup: Java warning: The certificate is not valid and cannot be used to verify the identity of this website . You have to find the problem in ca.crt or manually import it into the Java 'Secure Site CA' certificate store (which I did not try):
Java Control Panel -> Security (tab) -> Manage Certificates (button) -> Certificate Type: Secure Site CA -> Import (button)
After first instance worked with enabling TLS1.2, setup steps can be done by a shell script:
#!/bin/bash
# This script get crts and keys from ssl certificate ewallet.p12 in $INST_TOP/certs/Apache. And copy
6 template files from current location to $FND_TOP/admin/template/custom. This will reduce manual steps and human error.
#
p12PWD='N0Pe'
echo -n "p12 cert password > "
read p12PWD
if [ $p12PWD != 'N0Pe' ]; then
echo "$p12PWD"
CURRPWD=$PWD
echo $CURRPWD
echo "working on template files ..."
echo "$FND_TOP/admin/template/custom"
cp -p *.tmp $FND_TOP/admin/template/custom/.
ls -altr $FND_TOP/admin/template/custom
echo "working on cert files ..."
# Assume below two certs are universal within the company. Just copy over to use them.
cp -p ca.crt $INST_TOP/certs/Apache
cp -p intermediate.crt $INST_TOP/certs/Apache
# certs files
export PATH=$IAS_ORACLE_HOME/Apache/open_ssl/bin:$PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$IAS_ORACLE_HOME/lib
export OPENSSL_CONF=$IAS_ORACLE_HOME/Apache/open_ssl/bin/openssl.cnf
cd $IAS_ORACLE_HOME/Apache/open_ssl/bin
chmod 755 openssl
which openssl
cd $INST_TOP/certs/Apache
openssl pkcs12 -in ewallet.p12 -out jyy.key -nocerts -passout pass:'change1t' -password pass:$p12PWD
openssl rsa -in jyy.key -out server.key -passin pass:'change1t'
# below line assumes the SERVER cert is the 1st position in ewallet.p12. If it is not, need to manual
ly copy it from file allcerts.crt generated by next command line.
openssl pkcs12 -in ewallet.p12 -clcerts -nokeys -password pass:$p12PWD | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p;/-END CERTIFICATE-/q' > server.crt
#
# Use this to get more details if needed
## openssl pkcs12 -in ewallet.p12 -out allcerts.crt -nokeys -password pass:$p12PWD
#
cat server.crt intermediate.crt ca.crt > opmn.crt
#
# Not sure if the order in below output file is always correct
openssl pkcs12 -in ewallet.p12 -clcerts -nokeys -password pass:$p12PWD | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > opmn.crt_2nd
# opmn.crt and opmn.crt_2nd shall be identical except the key order in the files
# echo "cehck difference. should have nothing"
# diff opmn.crt opmn.cer_2nd
ls -altr
echo $INST_TOP/certs/Apache
cd $CURRPWD
else
echo "password seems wrong."
exit 1
fi
To check which TLS 1.X is used, on Chrome Ctrl+Shift+I to bring up a DevTool. Click on Security tab to see security info.
======================
- PRE REQUISITES
======================
1) HTTP Server (OHS) 10.1.3.5.0
If needed, use Doc ID 454811.1 (Upgrading to the Latest OracleAS 10g 10.1.3.x Patch Set in Oracle E-Business Suite Release 12) to upgrade it.
$ cd $INST_TOP/ora/10.1.3
$ . XXXX_xxxx.env
2) JDK 7 is required
I followed Doc ID 1467892.1 to upgrade JDK 6 to JDK 7 before working on this. Verify:
$ echo $IAS_ORACLE_HOME
/path/to/apps/tech_st/10.1.3
$ $IAS_ORACLE_HOME/appsutil/jdk/jre/bin/java -fullversion
java full version "1.7.0_231-b08"
$ echo $ORACLE_HOME
/path/to/apps/tech_st/10.1.2
$ $ORACLE_HOME/jdk/bin/java -fullversion
java full version "1.7.0_231-b08"
3) EBS site uses HTTPS
In my EBS sites, digital certificate uses PKCS#12 (Public Key Cryptography Standard #12) encryption. We download and re-new it from a website managed by company Security team. When we download it, the website ask to enter a password, say 'sslPWD01'.
$ cd $INST_TOP/certs/Apache
$ ls -al
-rw-r----- 1 user ogroup 6413 Jan 15 16:05 ewallet.p12
-rw------- 1 user ogroup 6441 Jan 15 16:05 cwallet.sso
======================
- Apply patches for enabling TLS1.2
======================
1) 3 patches to 10.1.3.5 HOME
$ cd $ADMIN_SCRIPTS_HOME
$ adstpall.sh
$ cd $INST_TOP/ora/10.1.3
$ . XXXXX_xxxxxx.env
$ echo $ORACLE_HOME
$ vi /etc/oraInst.loc
$ opatch lsinventory | grep 21845942 <== yes, 2015 CPU was applied before
$ cd /ebsu01/app/patchTLS12 (where all files are saved in)
-- 29292327 (Product Patched: Oracle HTTP Server. It includes newer "openssl")
-- unzip p29292327_101350_LINUX.zip
$ cd 29292327
$ opatch apply
-- 27208670 (Product Patched: OPMN. It include newer "opmn")
Note it is necessary to install a newer SDK to 10.1.3.5 OH to avoid error (from applying patch 27208670, Doc ID 2555323.1):
OPATCH_JAVA_ERROR=CheckConflict: OPatch cannot process overlay patches because of no OUI support. Please take latest OUI 10.1 patchset from "My Oracle Support" and try again.
$ cd $ORACLE_HOME
$ mv OPatch OPatch.pre_6640838
$ cd /ebsu01/app/patchForms (where p6640838_10106_Linux-x86-64.zip is unzipped)
$ cd cd/Disk1/install
$ export DISPLAY=66.666.666.666:0.0
$ ./runInstaller -ignoreSysPrereqs &
Steps for install (very slowly):
a) Choose next on the welcome page.
b) Choose the install type as Custom
c) Enter the 10.1.3.5 ORACLE_HOME where OUI components have to be installed.
d) On the Available product components page, select the checkbox to show all components.
e) Then, click on Expend All
f) You can de-select
Oracle Installation Libraries 10.1.0.6.0
Ensure below are selected.
x Oracle Universal Installer
x Installer SDK Component 10.1.0.6.0
x Oracle One-Off Patch Installer
g) Then, click on Next to Install.
$ cd $ORACLE_HOME
$ mv OPatch OPatch_delete
$ mv OPatch.pre_6640838 OPatch
-- Now, apply patch 27208670
$ cd /ebsu01/app/patchTLS12
$ unzip p27208670_101350_LINUX.zip
$ cd 27208670
$ opatch apply <== "27208670" overlays " 21845942 "
$ cd ..
$ rm -rf 27208670
-- 22322938 Product Patched: OC4J (update Java Mail API to 1.5.4)
-- unzip p22322938_101350_Generic.zip
$ cd 22322938
$ opatch apply => Answer N to continue (Rolling back patch 8999551)
2) 4 patches by adpatch
SQL> select * from ad_bugs where bug_number in (
'23645824', -- 23645824:R12.TXK.B (R12.TXK.B.delta.3. for FORMSAPP.EAR DEPLOYMENT FAILS)
'22974534', -- 22974534:R12.OWF.B (for OAF EMAIL)
'27881758', -- 27881758:R12.OWF.B (connect to OUTLOOK.OFFICE365.COM. superseded by 28779647)
'24677849' -- 24677849:R12.TXK.B (fix MAILER FAILS)
);
Merge those 4 patches to tls12_merge.zip
NOTE: Start a new OS session to take 10.1.2 ORACLE_HOME to avoid adpatch error:
adogjf() Unable to copy Registry.Dat.
$ adadmin => to enable maintenance mode
$ cd /ebsu01/app/patchTLS12
-- unzip tls12_merge.zip
$ cd tls12_merge
$ echo $ORACLE_HOME => 10.1.2
$ ls
$ adpatch
3) Using EBS openssl (that comes from patch 29292327)
$ openssl version
OpenSSL 1.0.2k-fips 26 Jan 2017
$ which openssl
/usr/bin/openssl <== from OS
/usr/bin/openssl <== from OS
$ ls -al $IAS_ORACLE_HOME/Apache/open_ssl/bin
-rw-r--r-- 1 user ogroup 616303 Mar 23 12:48 openssl
-rw-r--r-- 1 user ogroup 10835 Mar 23 12:48 openssl.cnf
$ cd $IAS_ORACLE_HOME/Apache/open_ssl/bin
$ chmod 755 openssl
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$IAS_ORACLE_HOME/lib
$ export OPENSSL_CONF=$IAS_ORACLE_HOME/Apache/open_ssl/bin/openssl.cnf
NOTES: without them, openssl may give error/warning
openssl: error while loading shared libraries: libssl.so.1.0.0: cannot open shared object file: No such file or directory
WARNING: can't open config file: /home/nse/workspace/openssl-linux_x86_OL5/install/openssl.cnf
$ export PATH=$IAS_ORACLE_HOME/Apache/open_ssl/bin:$PATH
$ which openssl
~/$TWO_TASK/apps/tech_st/10.1.3/Apache/open_ssl/bin/openssl
$ openssl version
OpenSSL 1.0.2q 20 Nov 2018
=========================
- Configure files to enable TLS1.2
=========================
1) Get server key and certificates
$ cd $INST_TOP/certs/Apache
$ openssl pkcs12 -in ewallet.p12 -out jyy.key -nocerts -passout pass:'EBS1t' -password pass:'sslPWD01'
MAC verified OK
Warning unsupported bag type: secretBag
$ openssl rsa -in jyy.key -out server.key -passin pass:'EBS1t' <= Use same PASS PHRASE
writing RSA key
$ openssl pkcs12 -in ewallet.p12 -out allcerts.crt -nokeys -password pass:sslPWD01
MAC verified OK
Warning unsupported bag type: secretBag
NOTES: It creates allcerts.crt file in which has 3 certs. Need to manually extract certs from this file:
server.crt (keyword: server name) -
subject=/C=US/ST=state/L=city/O=company name/CN=servername.domain.com
intermediate.crt (with company name) -
subject=/C=US/O=company name/CN=company name. Secure CA2
root cert (ca.crt) -
subject=/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance EV Root CA
But, intermediate.crt and root cert (ca.crt) are universal within the company network. They can be copied from other instances. So, only server.crt is truly new.
$ cat server.crt intermediate.crt ca.crt > opmn.crt
By now, $INST_TOP/certs/Apache has below files:
-rw-r----- 1 user ogroup 6349 Jan 15 16:05 ewallet.p12
-rw------- 1 user ogroup 6377 Jan 15 16:05 cwallet.sso
-rw-r--r-- 1 user ogroup 1367 Mar 17 17:16 ca.crt
-rw-r--r-- 1 user ogroup 1684 Mar 17 17:20 intermediate.crt
-rw-r--r-- 1 user ogroup 1970 Mar 23 13:26 jyy.key
-rw-r--r-- 1 user ogroup 1675 Mar 23 13:26 server.key
-rw-r--r-- 1 user ogroup 2342 Mar 23 13:28 server.crt
-rw-r--r-- 1 user ogroup 5393 Mar 23 15:46 opmn.crt
Additional commands
$ openssl pkcs12 -in ewallet.p12 -info
will list 3 certs and the key
Do not know what below will do:
openssl pkcs12 -in ewallet.p12 -out A_certs.crt -nokeys -nodes -password pass:sslPWD01
openssl pkcs12 -in ewallet.p12 -out CA_cert.crt -nokeys -cacerts -chain -password pass:sslPWD01
2) Modify or create template files in $FND_TOP/admin/template/custom. They can be shared out to other EBS instances. Below are for enabling TLS 1.2 only (and disabling others):
$ cd $FND_TOP/admin/template/custom
$ mv opmn_xml_1013.tmp opmn_xml_1013.tmp_BK (<= in case there was one)
$ cp -p ../opmn_xml_1013.tmp .
$ vi opmn_xml_1013.tmp
Replace this line in the template:
<ssl enabled="true" wallet-file="%s_web_ssl_directory%/opmn"/>
With the following:
<ssl enabled="true" openssl-certfile="%s_web_ssl_directory%/Apache/opmn.crt" openssl-keyfile="%s_web_ssl_directory%/Apache/server.key" openssl-password="change1t" openssl-lib="%s_weboh_oh%/lib" ssl-versions="TLSv1.2" ssl-ciphers="ECDHE-RSA-AES128-GCM-SHA256,ECDHE-RSA-AES256-GCM-SHA384"/>
$ diff opmn_xml_1013.tmp opmn_xml_1013.tmp_BK
14c14
< <ssl enabled="true" openssl-certfile="%s_web_ssl_directory%/Apache/opmn.crt" openssl-keyfile="%s_web_ssl_directory%/Apache/server.key" openssl-password="change1t" openssl-lib="%s_weboh_oh%/lib" ssl-versions="TLSv1.2" ssl-ciphers="ECDHE-RSA-AES128-GCM-SHA256,ECDHE-RSA-AES256-GCM-SHA384"/>
---
> <ssl enabled="true" wallet-file="%s_web_ssl_directory%/opmn"/>
$ cp -p ../ssl_conf_1013.tmp .
Perform the following
Step 1 - Comment out the following line in the template:
#SSLWallet file:%s_web_ssl_directory%/Apache
Step 2 - Add the following 3 lines into the template:
SSLCertificateFile %s_web_ssl_directory%/Apache/server.crt
SSLCertificateKeyFile %s_web_ssl_directory%/Apache/server.key
SSLCertificateChainFile %s_web_ssl_directory%/Apache/intermediate.crt
Step 3 - Comment out two lines
# SSLProtocol -all +TLSv1 +SSLv3
# SSLCipherSuite HIGH:MEDIUM:!aNULL:+SHA1:+MD5:+HIGH:+MEDIUM
Step 4 - Add two lines to it:
SSLProtocol TLSv1.2
SSLCipherSuite ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384
Follow Oracle document to make similar changes in other 4 template files:
$ diff httpd_conf_1013.tmp ../httpd_conf_1013.tmp
261,262c261
< # LoadModule ossl_module libexec/mod_ossl.so
< LoadModule ssl_module libexec/mod_ssl.so
---
> LoadModule ossl_module libexec/mod_ossl.so
261,262c261
< # LoadModule ossl_module libexec/mod_ossl.so
< LoadModule ssl_module libexec/mod_ssl.so
---
> LoadModule ossl_module libexec/mod_ossl.so
$ diff oc4j_properties_1013.tmp ../oc4j_properties_1013.tmp
72,73d71
< https.protocols=TLSv1.2
<
$ diff oafm_oc4j_properties_1013.tmp ../oafm_oc4j_properties_1013.tmp
68,69d67
< https.protocols=TLSv1.2
<
$ diff forms_oc4j_properties_1013.tmp ../forms_oc4j_properties_1013.tmp
72,73d71
< https.protocols=TLSv1.2
<
3) Run autoconfig and start services
$ adadmin => disable maintenance mode
$ cd $ADMIN_SCRIPTS_HOME
$ adautocfg.sh
$ adstrtal.sh
$ openssl s_client -connect site_name.domian.com:port -tls1_2
TROUBLESHOOTING:
1. After all above steps completed, adstrtal.sh failed to start oacore service in the first time by Timed-out error (code 204). It is fixed by cleaning session lock files in persistence directories .
2. If the SSL root certificate (ca.crt) is incorrect, EBS GUI will give warning in popup: Java warning: The certificate is not valid and cannot be used to verify the identity of this website . You have to find the problem in ca.crt or manually import it into the Java 'Secure Site CA' certificate store (which I did not try):
Java Control Panel -> Security (tab) -> Manage Certificates (button) -> Certificate Type: Secure Site CA -> Import (button)
After first instance worked with enabling TLS1.2, setup steps can be done by a shell script:
#!/bin/bash
# This script get crts and keys from ssl certificate ewallet.p12 in $INST_TOP/certs/Apache. And copy
6 template files from current location to $FND_TOP/admin/template/custom. This will reduce manual steps and human error.
#
p12PWD='N0Pe'
echo -n "p12 cert password > "
read p12PWD
if [ $p12PWD != 'N0Pe' ]; then
echo "$p12PWD"
CURRPWD=$PWD
echo $CURRPWD
echo "working on template files ..."
echo "$FND_TOP/admin/template/custom"
cp -p *.tmp $FND_TOP/admin/template/custom/.
ls -altr $FND_TOP/admin/template/custom
echo "working on cert files ..."
# Assume below two certs are universal within the company. Just copy over to use them.
cp -p ca.crt $INST_TOP/certs/Apache
cp -p intermediate.crt $INST_TOP/certs/Apache
# certs files
export PATH=$IAS_ORACLE_HOME/Apache/open_ssl/bin:$PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$IAS_ORACLE_HOME/lib
export OPENSSL_CONF=$IAS_ORACLE_HOME/Apache/open_ssl/bin/openssl.cnf
cd $IAS_ORACLE_HOME/Apache/open_ssl/bin
chmod 755 openssl
which openssl
cd $INST_TOP/certs/Apache
openssl pkcs12 -in ewallet.p12 -out jyy.key -nocerts -passout pass:'change1t' -password pass:$p12PWD
openssl rsa -in jyy.key -out server.key -passin pass:'change1t'
# below line assumes the SERVER cert is the 1st position in ewallet.p12. If it is not, need to manual
ly copy it from file allcerts.crt generated by next command line.
openssl pkcs12 -in ewallet.p12 -clcerts -nokeys -password pass:$p12PWD | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p;/-END CERTIFICATE-/q' > server.crt
#
# Use this to get more details if needed
## openssl pkcs12 -in ewallet.p12 -out allcerts.crt -nokeys -password pass:$p12PWD
#
cat server.crt intermediate.crt ca.crt > opmn.crt
#
# Not sure if the order in below output file is always correct
openssl pkcs12 -in ewallet.p12 -clcerts -nokeys -password pass:$p12PWD | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > opmn.crt_2nd
# opmn.crt and opmn.crt_2nd shall be identical except the key order in the files
# echo "cehck difference. should have nothing"
# diff opmn.crt opmn.cer_2nd
ls -altr
echo $INST_TOP/certs/Apache
cd $CURRPWD
else
echo "password seems wrong."
exit 1
fi
Thursday, December 12, 2019
Upgrade JDK to JDK 7 in EBS R12.1
There are three parts in upgrading JDK (Java Development Kit) on server for EBS R12.1.3. The document I followed on this upgrade is Doc ID 1467892.1 (Using JDK 7.0 Latest Update with Oracle E-Business Suite Release 12.0 and 12.1).
First, check current JDK/Java version is old (Doc ID 468311.1) :
$ sh -c "`awk -F= '$1 ~ /^JSERVJAVA.*$/ {print $2}' $ADMIN_SCRIPTS_HOME/java.sh` -version;"
java version "1.6.0_17"
Java(TM) SE Runtime Environment (build 1.6.0_17-b04)
Java HotSpot(TM) Server VM (build 14.3-b01, mixed mode)
Note client JRE version is a different thing:
$ cat $FORMS_WEB_CONFIG_FILE|grep sun_plugin_version| cut -c 1-35
sun_plugin_version=1.8.0_152
1) Upgrading to JDK 7.0 on Application Tier 10.1.3 Oracle Home
Upgrading to JDK 7.0 requires Oracle Application Server 10.1.3.5 or higher for the web tier.
By default when R12.1.1 was installed, Web Tier got 10.1.3.4. To upgrade it to 10.1.3.5 follow Doc ID 454811.1 (Upgrading to the Latest OracleAS 10g 10.1.3.x Patch Set in Oracle E-Business Suite Release 12).
1. Download JDK 7.0
I downloaded JDK 1.7.0_231 from a link in Doc ID 1439822.1 (Java SE Downloads on My Oracle Support Knowledge).
Oracle JDK 7 Update 231 - Patch 29657331 (32 bit): p29657331_170231_LINUX.zip => jdk-7u231-linux-i586.tar.gz
2. Before upgrading Application Tier nodes to JDK 7.0, apply two pre patches.
Patch 16545472 ('9239089' -- R12.AD.B.delta.3, '8919491' -- R12.ATG_PF.B.delta.3)
Patch 17309237 (prerequisite: '17932167' -- R12.TXK.B.delta.3)
Below query returns 3 rows for me, indicating their prerequisites were installed
SQL> select * from ad_bugs where bug_number in (
'16545472',
'17309237',
'9239089', -- R12.AD.B.delta.3,
'8919491', -- R12.ATG_PF.B.delta.3
'17932167', -- R12.TXK.B.delta.3
'19671435', -- RHEL7 needs it. Not for JDK upgrade.
'19863797' -- RHEL7 needs it. Not for JDK upgrade.
);
-- stop apps services
$ cd $ADMIN_SCRIPTS_HOME
$ ./adstpall.sh apps/apps
$ ./adadmin
$ cd /aebsu01/app/patchJDK7 <= where all .zip files located
$ -- unzip p16545472_R12.OAM.B_R12_GENERIC.zip
$ cd 16545472
$ ls
$ adpatch
$ cd ..
$ -- unzip p17309237_R12.TXK.B_R12_GENERIC.zip
$ cd 17309237
$ ls
$ adpatch
3. Replace JDK in EBS 10.1.3 HOME
$ echo $IAS_ORACLE_HOME
point to /path/apps/tech_st/10.1.3
$ cd ..
$ -- unzip p29657331_170231_LINUX.zip
$ tar vzxfp jdk-7u231-linux-i586.tar.gz
-- replace old JDK 1.6 Home used by Oracle E-Business Suite R12.1
$ cd $IAS_ORACLE_HOME/appsutil
$ mv jdk jdk_old_for_7
$ mv /aebsu01/app/patchJDK7/jdk1.7.0_231 jdk
$ ls -ald jdk* <= if jdk64 exists, rename it.
$ cd $IAS_ORACLE_HOME/jre
$ mv 1.4.2 1.4.2_old_for_7
$ ls -al $IAS_ORACLE_HOME/jre
-- copy 5 EBS font files to the new JDK
$ ls -al $FND_TOP/resource/ALB*.ttf
$ ls $IAS_ORACLE_HOME/appsutil/jdk/jre/lib/fonts
$ cp -p $FND_TOP/resource/ALB*.ttf $IAS_ORACLE_HOME/appsutil/jdk/jre/lib/fonts/.
$ ls $IAS_ORACLE_HOME/appsutil/jdk/jre/lib/fonts
2) Upgrade to JDK 7.0 in OracleAS 10.1.2 Oracle_Home
1. apply pre patch
$ echo $ORACLE_HOME
point to /path/apps/tech_st/10.1.2
== patch 12848228
$ cd /aebsu01/app/patchJDK7
$ opatch lsinventory
$ opatch lsinventory | grep 5659594 -- show 5659594 was installed before
$ -- unzip p12848228_10123_GENERIC.zip
$ cd 12848228
$ opatch apply
2. Replace old JDK 1.4.2 Home
$ cd ..
$ tar vzxfp jdk-7u231-linux-i586.tar.gz
$ cd $ORACLE_HOME
$ mv jdk jdk_old_for_7
$ mv /aebsu01/app/patchJDK7/jdk1.7.0_231 jdk
$ mv $ORACLE_HOME/jre/1.4.2 $ORACLE_HOME/jre/1.4.2_old_for_7
$ ls -ald $ORACLE_HOME/jdk* <= if jdk64 exists, rename it
$ ls -al $ORACLE_HOME/jre
Note: upto this step, "opatch lsinventory" (opatch version 1.0.0.0.63) gets warning on liboraInstaller.so
3. Apply 5 patches to 10.1.2 Home in below sequence:
== patch 16271876
$ cd /aebsu01/app/patchJDK7
$ -- unzip p16271876_10123_LINUX.zip
$ cd 16271876
$ opatch apply
$ cd $ORACLE_HOME/lib/stubs
$ ln -s libjvm-1.7-stub.so libjvm.so
$ ls -al libjvm.so
== patch 17907988
$ cd /aebsu01/app/patchJDK7
$ -- unzip p17907988_10123_LINUX.zip
$ cd 17907988
$ opatch apply <-- it rolls back subset patch 7121788
$ chmod +x $ORACLE_HOME/bin/genshlib
$ ls -al $ORACLE_HOME/bin/genshlib
== patch 17653437
$ cd ..
$ -- unzip p17653437_10123_LINUX.zip
$ cd 17653437
$ ls
$ opatch apply
<-- it rolls back conflicting patch 6995251 (answer: N (not STOP). Doc ID 1921974.1 )
$ cd $ORACLE_HOME/forms/lib
$ make -f ins_forms.mk sharedlib install <= Rebuild Forms executables
== patch 17645157
$ cd /aebsu01/app/patchJDK7
$ -- unzip p17645157_10123_LINUX.zip
$ cd 17645157
$ opatch apply
$ cd $ORACLE_HOME/reports/lib
$ make -f ins_reports.mk install <= Rebuild Reports executables
== patch 16241466
$ cd /aebsu01/app/patchJDK7
$ -- unzip p16241466_10123_LINUX.zip
$ cd 16241466
$ opatch apply <-- It rolls back subset patch 8551790
$ opatch lsinventory -- confirm 6 new patches installed
4. Re-generate Oracle E-Business Suite Forms and Reports
$ adadmin --> 1 --> 2 & 3
AND disable maintenance mode
The following Oracle Forms objects did not generate successfully:
igi forms/US IGIRRMSC.fmx
I ignored it. (Doc ID 2206725.1: Ignore the issue if you do not use Public Sector Financials International. It will not affect your system.)
5. Verify
$ $ADJVAPRG -version
java version "1.7.0_231"
Java(TM) SE Runtime Environment (build 1.7.0_231-b08)
Java HotSpot(TM) Server VM (build 24.231-b08, mixed mode)
$ $AFJVAPRG -version
java version "1.7.0_231"
Java(TM) SE Runtime Environment (build 1.7.0_231-b08)
Java HotSpot(TM) Server VM (build 24.231-b08, mixed mode)
$ cd $ADMIN_SCRIPT_HOME
- start EBS services
3) Upgrade to JRE 7.0 on Database Tier Node
This step is needed if Oracle database version is below 11gR2. This upgrade is independent to the JDK upgrade on the Oracle E-Business Suite application tier.
Download Latest JRE 7.0 Update. Note: Download the 32-bit JRE only, not the Java SE Development Kit (JDK)
To replace existing JRE:
$ cd $ORACLE_HOME/appsutil
$ mv jre jre_old
$ mv jre1.7.0_231 jre
TROUBLESHOOTING:
1. After all above steps, script adstpall.sh got Timed-out error on three processes OAFM, FORMS, and OACORE
$ ./adstpall.sh apps/appPWD
You are running adstpall.sh version 120.10.12010000.4
The logfile for this session is located at $LOG_HOME/appl/admin/log/adstpall.log
Executing service control script: $ADMIN_SCRIPTS_HOME/adoafmctl.sh stop
script returned:
****************************************************
ERROR : Timed out( 100000 ): Interrupted Exception
You are running adoafmctl.sh version 120.8
Stopping OPMN managed OAFM OC4J instance ...
****************************************************
The error in $LOG_HOME/ora/10.1.3/opmn/default_group~oafm~default_group~1.log
--------
20/01/29 16:22:07 Stop process
--------
Error: Could not connect to the remote server. Please check if the server is down or the client is using invalid host, ORMI port or password to connect: <no message>oracle.oc4j.security.ExchangingEncryptor.getEncryptedValue(ExchangingEncryptor.java:161)
com.evermind.server.rmi.RMIProtocol$SecureCredentials.send(RMIProtocol.java:278)
com.evermind.server.rmi.RMIProtocol.sendCredentials(RMIProtocol.java:95)
oracle.oc4j.rmi.ClientRmiTransport.connectToServer(ClientRmiTransport.java:92)
oracle.oc4j.rmi.ClientSocketRmiTransport.connectToServer(ClientSocketRmiTransport.java:69)
com.evermind.server.rmi.RMIClientConnection.connect(RMIClientConnection.java:765)
com.evermind.server.rmi.RMIClientConnection.sendLookupRequest(RMIClientConnection.java:247)
com.evermind.server.rmi.RMIClientConnection.lookup(RMIClientConnection.java:231)
com.evermind.server.rmi.RMIClient.lookup(RMIClient.java:302)
com.evermind.server.rmi.RMIClientContext.lookup(RMIClientContext.java:59)
com.evermind.client.orion.Oc4jAdminConsole.executeCommand(Oc4jAdminConsole.java:138)
com.evermind.client.orion.Oc4jAdminConsole.main(Oc4jAdminConsole.java:31)
caused by: oracle.oc4j.security.KeyExchange.getSecretKey(KeyExchange.java:136)
oracle.oc4j.security.ExchangingEncryptor.getEncryptedValue(ExchangingEncryptor.java:152)
com.evermind.server.rmi.RMIProtocol$SecureCredentials.send(RMIProtocol.java:278)
com.evermind.server.rmi.RMIProtocol.sendCredentials(RMIProtocol.java:95)
oracle.oc4j.rmi.ClientRmiTransport.connectToServer(ClientRmiTransport.java:92)
oracle.oc4j.rmi.ClientSocketRmiTransport.connectToServer(ClientSocketRmiTransport.java:69)
com.evermind.server.rmi.RMIClientConnection.connect(RMIClientConnection.java:765)
com.evermind.server.rmi.RMIClientConnection.sendLookupRequest(RMIClientConnection.java:247)
com.evermind.server.rmi.RMIClientConnection.lookup(RMIClientConnection.java:231)
com.evermind.server.rmi.RMIClient.lookup(RMIClient.java:302)
com.evermind.server.rmi.RMIClientContext.lookup(RMIClientContext.java:59)
com.evermind.client.orion.Oc4jAdminConsole.executeCommand(Oc4jAdminConsole.java:138)
com.evermind.client.orion.Oc4jAdminConsole.main(Oc4jAdminConsole.java:31)
caused by: Unsupported secret key algorithm: DES
The fix is to follow Doc ID 2353710.1:
1) Copy opmn_xml_1013.tmp from $FND_TOP/admin/template to $FND_TOP/admin/template/custom
2) Modify 6 lines in $FND_TOP/admin/template/custom/opmn_xml_1013.tmp by attaching "-Djdk.crypto.KeyAgreement.legacyKDF=true" to the end of each line for oafm_jvm, forms_jvm, oacore_jvm.
3) Run autoconfig.
After that, confirm file opmn/xml is updated with new entries:
$ grep legacyKDF $ORA_CONFIG_HOME/10.1.3/opmn/conf/opmn.xml
2. After above, I still have Timed-out problem with stopping FORMS. I looked into further by running below line to deploy EAR files and it failed with exact error as in Doc ID 1399491.1 (but no similar error in my formsstd.err file). The fix is to apply patch 12965674 to 10.1.3 ORACLE_HOME.
$ $FND_TOP/bin/txkrun.pl -script=CfgOC4JApp -applicationname=forms -oc4jpass=welcome123 -runautoconfig=No
Command error: <rc> = 52224, <command> = $IAS_ORACLE_HOME/opmn/bin/opmnctl stopproc instancename=forms
STACK TRACE
TXK::Error::abort('TXK::Error','HASH(0x8bf2334)').......
3. If EBS installation is a 32bit build while JDK is 64bit, opatch may give error:
Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library
/PATH/apps/tech_st/10.1.2/oui/lib/linux/liboraInstaller.so which might have disabled stack guard.
The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'. java.lang.UnsatisfiedLinkError: /PATH/apps/tech_st/10.1.2/oui/lib/linux/liboraInstaller.so:
/PATH/apps/tech_st/10.1.2/oui/lib/linux/liboraInstaller.so: wrong ELF class: ELFCLASS32
Another way to download Java files from Oracle Support site: click on Patches &Updates tab. Download the right version JDK. For example, JDK 1.7 Update 231 on Linux 64bit.
It may returns multiple rows.
. JDK 7 update 231 is the one that you would need for JDK
. Server JRE 7 is the JRE used on server side
. JRE 7 update 231 is the regular JRE version of Java used for applications on client side.
References:
1. Using JDK 7.0 Latest Update with Oracle E-Business Suite Release 12.0 and 12.1 (Doc ID 1467892.1)
2. Using the Latest JDK 7.0 Update with Oracle E-Business Suite Release 12.2 (Doc ID 1530033.1)
It today says EBS R12.2.2 or higher is certified with "JDK 7.0 Update 9 or higher" (1.7.0_09).
3. All Java SE Downloads on MOS (Doc ID 1439822.1).
4. E-WL: How to Upgrade the Java JDK Version for WebLogic 12c (Doc ID 2168514.1)
First, check current JDK/Java version is old (Doc ID 468311.1) :
$ sh -c "`awk -F= '$1 ~ /^JSERVJAVA.*$/ {print $2}' $ADMIN_SCRIPTS_HOME/java.sh` -version;"
java version "1.6.0_17"
Java(TM) SE Runtime Environment (build 1.6.0_17-b04)
Java HotSpot(TM) Server VM (build 14.3-b01, mixed mode)
Note client JRE version is a different thing:
$ cat $FORMS_WEB_CONFIG_FILE|grep sun_plugin_version| cut -c 1-35
sun_plugin_version=1.8.0_152
1) Upgrading to JDK 7.0 on Application Tier 10.1.3 Oracle Home
Upgrading to JDK 7.0 requires Oracle Application Server 10.1.3.5 or higher for the web tier.
By default when R12.1.1 was installed, Web Tier got 10.1.3.4. To upgrade it to 10.1.3.5 follow Doc ID 454811.1 (Upgrading to the Latest OracleAS 10g 10.1.3.x Patch Set in Oracle E-Business Suite Release 12).
1. Download JDK 7.0
I downloaded JDK 1.7.0_231 from a link in Doc ID 1439822.1 (Java SE Downloads on My Oracle Support Knowledge).
Oracle JDK 7 Update 231 - Patch 29657331 (32 bit): p29657331_170231_LINUX.zip => jdk-7u231-linux-i586.tar.gz
2. Before upgrading Application Tier nodes to JDK 7.0, apply two pre patches.
Patch 16545472 ('9239089' -- R12.AD.B.delta.3, '8919491' -- R12.ATG_PF.B.delta.3)
Patch 17309237 (prerequisite: '17932167' -- R12.TXK.B.delta.3)
Below query returns 3 rows for me, indicating their prerequisites were installed
SQL> select * from ad_bugs where bug_number in (
'16545472',
'17309237',
'9239089', -- R12.AD.B.delta.3,
'8919491', -- R12.ATG_PF.B.delta.3
'17932167', -- R12.TXK.B.delta.3
'19671435', -- RHEL7 needs it. Not for JDK upgrade.
'19863797' -- RHEL7 needs it. Not for JDK upgrade.
);
-- stop apps services
$ cd $ADMIN_SCRIPTS_HOME
$ ./adstpall.sh apps/apps
$ ./adadmin
$ cd /aebsu01/app/patchJDK7 <= where all .zip files located
$ -- unzip p16545472_R12.OAM.B_R12_GENERIC.zip
$ cd 16545472
$ ls
$ adpatch
$ cd ..
$ -- unzip p17309237_R12.TXK.B_R12_GENERIC.zip
$ cd 17309237
$ ls
$ adpatch
3. Replace JDK in EBS 10.1.3 HOME
$ echo $IAS_ORACLE_HOME
point to /path/apps/tech_st/10.1.3
$ cd ..
$ -- unzip p29657331_170231_LINUX.zip
$ tar vzxfp jdk-7u231-linux-i586.tar.gz
-- replace old JDK 1.6 Home used by Oracle E-Business Suite R12.1
$ cd $IAS_ORACLE_HOME/appsutil
$ mv jdk jdk_old_for_7
$ mv /aebsu01/app/patchJDK7/jdk1.7.0_231 jdk
$ ls -ald jdk* <= if jdk64 exists, rename it.
$ cd $IAS_ORACLE_HOME/jre
$ mv 1.4.2 1.4.2_old_for_7
$ ls -al $IAS_ORACLE_HOME/jre
-- copy 5 EBS font files to the new JDK
$ ls -al $FND_TOP/resource/ALB*.ttf
$ ls $IAS_ORACLE_HOME/appsutil/jdk/jre/lib/fonts
$ cp -p $FND_TOP/resource/ALB*.ttf $IAS_ORACLE_HOME/appsutil/jdk/jre/lib/fonts/.
$ ls $IAS_ORACLE_HOME/appsutil/jdk/jre/lib/fonts
2) Upgrade to JDK 7.0 in OracleAS 10.1.2 Oracle_Home
1. apply pre patch
$ echo $ORACLE_HOME
point to /path/apps/tech_st/10.1.2
== patch 12848228
$ cd /aebsu01/app/patchJDK7
$ opatch lsinventory
$ opatch lsinventory | grep 5659594 -- show 5659594 was installed before
$ -- unzip p12848228_10123_GENERIC.zip
$ cd 12848228
$ opatch apply
2. Replace old JDK 1.4.2 Home
$ cd ..
$ tar vzxfp jdk-7u231-linux-i586.tar.gz
$ cd $ORACLE_HOME
$ mv jdk jdk_old_for_7
$ mv /aebsu01/app/patchJDK7/jdk1.7.0_231 jdk
$ mv $ORACLE_HOME/jre/1.4.2 $ORACLE_HOME/jre/1.4.2_old_for_7
$ ls -ald $ORACLE_HOME/jdk* <= if jdk64 exists, rename it
$ ls -al $ORACLE_HOME/jre
Note: upto this step, "opatch lsinventory" (opatch version 1.0.0.0.63) gets warning on liboraInstaller.so
3. Apply 5 patches to 10.1.2 Home in below sequence:
== patch 16271876
$ cd /aebsu01/app/patchJDK7
$ -- unzip p16271876_10123_LINUX.zip
$ cd 16271876
$ opatch apply
$ cd $ORACLE_HOME/lib/stubs
$ ln -s libjvm-1.7-stub.so libjvm.so
$ ls -al libjvm.so
== patch 17907988
$ cd /aebsu01/app/patchJDK7
$ -- unzip p17907988_10123_LINUX.zip
$ cd 17907988
$ opatch apply <-- it rolls back subset patch 7121788
$ chmod +x $ORACLE_HOME/bin/genshlib
$ ls -al $ORACLE_HOME/bin/genshlib
== patch 17653437
$ cd ..
$ -- unzip p17653437_10123_LINUX.zip
$ cd 17653437
$ ls
$ opatch apply
<-- it rolls back conflicting patch 6995251 (answer: N (not STOP). Doc ID 1921974.1 )
$ cd $ORACLE_HOME/forms/lib
$ make -f ins_forms.mk sharedlib install <= Rebuild Forms executables
== patch 17645157
$ cd /aebsu01/app/patchJDK7
$ -- unzip p17645157_10123_LINUX.zip
$ cd 17645157
$ opatch apply
$ cd $ORACLE_HOME/reports/lib
$ make -f ins_reports.mk install <= Rebuild Reports executables
== patch 16241466
$ cd /aebsu01/app/patchJDK7
$ -- unzip p16241466_10123_LINUX.zip
$ cd 16241466
$ opatch apply <-- It rolls back subset patch 8551790
$ opatch lsinventory -- confirm 6 new patches installed
4. Re-generate Oracle E-Business Suite Forms and Reports
$ adadmin --> 1 --> 2 & 3
AND disable maintenance mode
The following Oracle Forms objects did not generate successfully:
igi forms/US IGIRRMSC.fmx
I ignored it. (Doc ID 2206725.1: Ignore the issue if you do not use Public Sector Financials International. It will not affect your system.)
5. Verify
$ $ADJVAPRG -version
java version "1.7.0_231"
Java(TM) SE Runtime Environment (build 1.7.0_231-b08)
Java HotSpot(TM) Server VM (build 24.231-b08, mixed mode)
$ $AFJVAPRG -version
java version "1.7.0_231"
Java(TM) SE Runtime Environment (build 1.7.0_231-b08)
Java HotSpot(TM) Server VM (build 24.231-b08, mixed mode)
$ cd $ADMIN_SCRIPT_HOME
- start EBS services
3) Upgrade to JRE 7.0 on Database Tier Node
This step is needed if Oracle database version is below 11gR2. This upgrade is independent to the JDK upgrade on the Oracle E-Business Suite application tier.
Download Latest JRE 7.0 Update. Note: Download the 32-bit JRE only, not the Java SE Development Kit (JDK)
To replace existing JRE:
$ cd $ORACLE_HOME/appsutil
$ mv jre jre_old
$ mv jre1.7.0_231 jre
TROUBLESHOOTING:
1. After all above steps, script adstpall.sh got Timed-out error on three processes OAFM, FORMS, and OACORE
$ ./adstpall.sh apps/appPWD
You are running adstpall.sh version 120.10.12010000.4
The logfile for this session is located at $LOG_HOME/appl/admin/log/adstpall.log
Executing service control script: $ADMIN_SCRIPTS_HOME/adoafmctl.sh stop
script returned:
****************************************************
ERROR : Timed out( 100000 ): Interrupted Exception
You are running adoafmctl.sh version 120.8
Stopping OPMN managed OAFM OC4J instance ...
****************************************************
The error in $LOG_HOME/ora/10.1.3/opmn/default_group~oafm~default_group~1.log
--------
20/01/29 16:22:07 Stop process
--------
Error: Could not connect to the remote server. Please check if the server is down or the client is using invalid host, ORMI port or password to connect: <no message>oracle.oc4j.security.ExchangingEncryptor.getEncryptedValue(ExchangingEncryptor.java:161)
com.evermind.server.rmi.RMIProtocol$SecureCredentials.send(RMIProtocol.java:278)
com.evermind.server.rmi.RMIProtocol.sendCredentials(RMIProtocol.java:95)
oracle.oc4j.rmi.ClientRmiTransport.connectToServer(ClientRmiTransport.java:92)
oracle.oc4j.rmi.ClientSocketRmiTransport.connectToServer(ClientSocketRmiTransport.java:69)
com.evermind.server.rmi.RMIClientConnection.connect(RMIClientConnection.java:765)
com.evermind.server.rmi.RMIClientConnection.sendLookupRequest(RMIClientConnection.java:247)
com.evermind.server.rmi.RMIClientConnection.lookup(RMIClientConnection.java:231)
com.evermind.server.rmi.RMIClient.lookup(RMIClient.java:302)
com.evermind.server.rmi.RMIClientContext.lookup(RMIClientContext.java:59)
com.evermind.client.orion.Oc4jAdminConsole.executeCommand(Oc4jAdminConsole.java:138)
com.evermind.client.orion.Oc4jAdminConsole.main(Oc4jAdminConsole.java:31)
caused by: oracle.oc4j.security.KeyExchange.getSecretKey(KeyExchange.java:136)
oracle.oc4j.security.ExchangingEncryptor.getEncryptedValue(ExchangingEncryptor.java:152)
com.evermind.server.rmi.RMIProtocol$SecureCredentials.send(RMIProtocol.java:278)
com.evermind.server.rmi.RMIProtocol.sendCredentials(RMIProtocol.java:95)
oracle.oc4j.rmi.ClientRmiTransport.connectToServer(ClientRmiTransport.java:92)
oracle.oc4j.rmi.ClientSocketRmiTransport.connectToServer(ClientSocketRmiTransport.java:69)
com.evermind.server.rmi.RMIClientConnection.connect(RMIClientConnection.java:765)
com.evermind.server.rmi.RMIClientConnection.sendLookupRequest(RMIClientConnection.java:247)
com.evermind.server.rmi.RMIClientConnection.lookup(RMIClientConnection.java:231)
com.evermind.server.rmi.RMIClient.lookup(RMIClient.java:302)
com.evermind.server.rmi.RMIClientContext.lookup(RMIClientContext.java:59)
com.evermind.client.orion.Oc4jAdminConsole.executeCommand(Oc4jAdminConsole.java:138)
com.evermind.client.orion.Oc4jAdminConsole.main(Oc4jAdminConsole.java:31)
caused by: Unsupported secret key algorithm: DES
The fix is to follow Doc ID 2353710.1:
1) Copy opmn_xml_1013.tmp from $FND_TOP/admin/template to $FND_TOP/admin/template/custom
2) Modify 6 lines in $FND_TOP/admin/template/custom/opmn_xml_1013.tmp by attaching "-Djdk.crypto.KeyAgreement.legacyKDF=true" to the end of each line for oafm_jvm, forms_jvm, oacore_jvm.
3) Run autoconfig.
After that, confirm file opmn/xml is updated with new entries:
$ grep legacyKDF $ORA_CONFIG_HOME/10.1.3/opmn/conf/opmn.xml
2. After above, I still have Timed-out problem with stopping FORMS. I looked into further by running below line to deploy EAR files and it failed with exact error as in Doc ID 1399491.1 (but no similar error in my formsstd.err file). The fix is to apply patch 12965674 to 10.1.3 ORACLE_HOME.
$ $FND_TOP/bin/txkrun.pl -script=CfgOC4JApp -applicationname=forms -oc4jpass=welcome123 -runautoconfig=No
Command error: <rc> = 52224, <command> = $IAS_ORACLE_HOME/opmn/bin/opmnctl stopproc instancename=forms
STACK TRACE
TXK::Error::abort('TXK::Error','HASH(0x8bf2334)').......
3. If EBS installation is a 32bit build while JDK is 64bit, opatch may give error:
Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library
/PATH/apps/tech_st/10.1.2/oui/lib/linux/liboraInstaller.so which might have disabled stack guard.
The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'. java.lang.UnsatisfiedLinkError: /PATH/apps/tech_st/10.1.2/oui/lib/linux/liboraInstaller.so:
/PATH/apps/tech_st/10.1.2/oui/lib/linux/liboraInstaller.so: wrong ELF class: ELFCLASS32
Another way to download Java files from Oracle Support site: click on Patches &Updates tab. Download the right version JDK. For example, JDK 1.7 Update 231 on Linux 64bit.
. JDK 7 update 231 is the one that you would need for JDK
. Server JRE 7 is the JRE used on server side
. JRE 7 update 231 is the regular JRE version of Java used for applications on client side.
References:
1. Using JDK 7.0 Latest Update with Oracle E-Business Suite Release 12.0 and 12.1 (Doc ID 1467892.1)
2. Using the Latest JDK 7.0 Update with Oracle E-Business Suite Release 12.2 (Doc ID 1530033.1)
It today says EBS R12.2.2 or higher is certified with "JDK 7.0 Update 9 or higher" (1.7.0_09).
3. All Java SE Downloads on MOS (Doc ID 1439822.1).
4. E-WL: How to Upgrade the Java JDK Version for WebLogic 12c (Doc ID 2168514.1)
Subscribe to:
Posts (Atom)