Sunday, September 13, 2015

Re-new SSL certificate file & How to enable SSL

In my environment, two files ewallet.p12 and cwallet.sso are saved in $INST_TOP/certs/Apache. The path is defined in ssl.conf file. Autoconfig will add cwallet.sso as the keystore variable to $CONTEXT_FILE.
 
Seems to me that certificate file works for the host name or site name. If one server hosts multiple EBS instances, same certificate file can be used for all Apache instances on the host. Certificate expires after a period of time.

Steps to renew certificate files (from .p12 file):

1. Generate the new cert file (renew the file) from certificate tool
Depending on the tool provided by CA (certificate authority) of the company for generating the certificate file, select to Renew cert for the site or host with key size of 2048 kb.
2. Download the .p12 file in my company's tool
Check Yes to include Root Chain and Private key (in PKCS12 format, depending on the tool). Also enter a password to protect the file.
3. Copy the new cert file to $INST_TOP/certs/Apache and name it ewallet.p12 (after backing up the old one).
4. Go to $INST_TOP/ora/10.1.3 directory, and source the .env file
Make sure $ORACLE_HOME point to 10.1.3 HOME ( for all infrastructure changes).
5. $ owm (run Oracle Wallet Manager after starting Exceed for XWindow).
NOTES: if run owm from 10.1.2 Oracle Home, it may give an error, such as
owm: line 155: /d2/R12/ab/apps/R1211XB9/apps/tech_st/10.1.2/jdk/jre//bin/java: No such file or directory
6. In OWM GUI, Wallet => Open,  navigate to folder $INST_TOP/certs/Apache, click on the path => Open
7. Enter the password (the one entered when re-new and download certificate file) and then click Wallet => check "Auto Login" to enable it, then => Save.

Note in February 2016:  If new certificate standard SHA2 (DigiCert) replaces SHA1 (GeoTrust) on generating new .p12 file, OWM may not open file ewallet.p12 with invalid password error. The fix is to apply patch 21845960 (October 2015 CPU Patch) to 10.1.3 ORACLE HOME (for Apache).

8. verify file cwallet.sso gets created in the same directory $INST_TOP/certs/Apache.
9. Copy the two files to other nodes of same SITE (if site uses multiple nodes for network load balance).
10. Open a new session to take regular .env file and then run autoconfig in web nodes.
11. Start apps services.  Click on the lock icon in the URL, you shall see the new date in certificate file.
12. Copy the two files to other instance(s) on the same node (if applicable).

How to enable SSL?

1. Create new certificate file for site ebssitename.domain.com by the certificate tool and get the two files ready in $INST_TOP/certs/Apache (similar to Steps 1 to 9 above).
2. Update 5 entries in Context file on web tiers: active web port, login, external url, webentryhost, webentryurlprotocol
  - active web port (s_active_webport) : 80 --> 443
    Usually F5 listens to 443, the HTTPS default, and redirects connections to real ssl port, such as 4472. For real port (without F5), makes sure the port (such as 4472) for s_active_webport and s_webssl_port in $CONTEXT_FILE are the same and matches the port in $INST_TOP/ora/10.1.3/Apache/Apache/conf/ssl.conf.
  - login (s_login_page): https://ebssitename.domain.com:s_active_webport/OA_HTML/AppsLogin
  - external url (s_external_url): https://ebssitename.domain.com:s_active_webport
  - webentryhost (s_webentryhost): ebssitename (no change)
  - webentryurlprotocol (s_webentryurlprotocol): http --> https
3. It may be necessary to modify ssl.conf in $INST_TOP/ora/10.1.3/Apache/Apache/conf to enable TLS1.x by editing file $FND_TOP/admin/template/ssl_conf_1013.tmp
4. Run auotconfig on all web nodes (this step will update Port number in httpd.conf, etc).

How to disable SSL?

1. Change above 5 variables in $CONTEXT_FILE. or
2. After shurdown apps tier, run
$ txkrun.pl -script=SetAdvCfg \-appsuser=apps -appspass=<Apps Password> -disable=SSL \-s_webport=<Web port>(Replace your Web Port Number)
3. "Running Configuration Wizards from the Command Line in Oracle E-Business Suite Release 12 [ID 1364368.1]", under "Disabling SSL" section.

Reference:
Doc ID 376700.1 - Enabling SSL or TLS in Oracle E-Business Suite Release 12

UPDATES in 2020:  After TLS1.2 is enabled (see https://erpondb.blogspot.com/2020/03/enable-tls12-in-ebs-r121.html), more steps are needed in re-newing ssl certificate because more configuration files need and use new keys from .p12 file.  I write a script to extract keys.

#!/bin/bash
# To get new files server.key and opmn.crt, from new cert file ewallet.p12, used by template files
# in $FND_TOP/admin/template/custom
# Assumpation: TLS1.2 was enabled. And, intermediate.crt & ca.crt are company-wide
#                     and are unchanged from year to year.
# Steps: 1. go to $INST_TOP/ora/10.1.3 to make 10.1.3 the ORACLE_HOME, and then modify the .p12 file by owm
#        2. place this file in $INST_TOP/certs/Apache, and then run it.
#
p12PWD='N0Pe'
echo -n "p12 cert password > "
read p12PWD
if [ $p12PWD != 'N0Pe' ]; then
 echo "$p12PWD"
 CURRPWD=$PWD
 echo $CURRPWD
echo "working on cert files ..."
# Assume two certs ca.crt and intermediate.crt are universal within the company.
# certs files
export PATH=$IAS_ORACLE_HOME/Apache/open_ssl/bin:$PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$IAS_ORACLE_HOME/lib
export OPENSSL_CONF=$IAS_ORACLE_HOME/Apache/open_ssl/bin/openssl.cnf
cd $IAS_ORACLE_HOME/Apache/open_ssl/bin
chmod 755 openssl
which openssl
cd $INST_TOP/certs/Apache
if [ -f 'jyy.key' ]; then
   mv jyy.key jyy.key_OLD
fi
if [ -f 'server.key' ]; then
   mv server.key server.key_OLD
fi
if [ -f 'server.crt' ]; then
   mv server.crt server.crt_OLD
fi
if [ -f 'opmn.crt' ]; then
   mv opmn.crt opmn.crt_OLD
fi
if [ -f 'allcerts.jyy' ]; then
   mv allcerts.jyy allcerts.jyy_OLD
fi

openssl pkcs12 -in ewallet.p12 -out jyy.key -nocerts -passout pass:'change1t' -password pass:$p12PWD
openssl rsa -in jyy.key -out server.key -passin pass:'change1t'
# below line assumes the SERVER cert is the 1st position in ewallet.p12.
# If it is not, need to manually copy it from allcerts.jyy by next command line.
openssl pkcs12 -in ewallet.p12 -clcerts -nokeys -password pass:$p12PWD | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p;/-END CERTIFICATE-/q' > server.crt
# Use this to get details of 3 certs. Note allcerts.jyy is just for
# troubleshooting/comparison purpose when needed.
openssl pkcs12 -in ewallet.p12 -out allcerts.jyy -nokeys -password pass:$p12PWD
#
cat server.crt intermediate.crt ca.crt > opmn.crt
#
# Not sure if the order in below output file is always correct
openssl pkcs12 -in ewallet.p12 -clcerts -nokeys -password pass:$p12PWD | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > opmn.crt_2nd
# opmn.crt and opmn.crt_2nd shall be identical except the key order in the files
# echo "Use below line to check difference. should have nothing"
# diff opmn.crt opmn.cer_2nd
ls -altr
echo $INST_TOP/certs/Apache
cd $CURRPWD
else
  echo "password seems wrong."
  exit 1
fi

UPDATES:
I tried to use orapki to get server.crt file from ewallet.p12 (in R12.1.3). It was very close to make it work.

$ cd $INST_TOP/ora/10.1.3
$ ls
$ . $CONTEXT_NAME.env
$ which orapki             <= 10.1.3
$ cd /path/to/Apache
$ mv server.crt server.crt_BK_good
$ orapki wallet display -wallet /path/to/Apache/ewallet.p12
Enter wallet password:   xxxxxx
Requested Certificates:
Subject:        CN=site_name.domian.com,O=company_name,L=city,ST=AZ,C=US
User Certificates:
Trusted Certificates:
Subject:        CN=site_name.domian.com,O=company_name,L=city,ST=AZ,C=US
Subject:        CN=company_name Secure CA2,O=company_name,C=US
Subject:        CN=DigiCert High Assurance EV Root CA,OU=www.digicert.com,O=DigiCert Inc,C=US
$ orapki wallet export -wallet /path/to/Apache/ewallet.p12 -dn 'CN=site_name.domian.com,O=company_name,L=city,ST=AZ,C=US' -cert server.crt
$ chmod +r server.crt
$ more server.crt

$ diff server.crt server.crt_BK_good
... ...                                                  <== the length of lines is different
\ No newline at end of file                <== also this message

I used openssl to view their contents:

$ export PATH=$IAS_ORACLE_HOME/Apache/open_ssl/bin:$PATH
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$IAS_ORACLE_HOME/lib
$ export OPENSSL_CONF=$IAS_ORACLE_HOME/Apache/open_ssl/bin/openssl.cnf
$ which openssl
~/$TWO_TASK/tech_st/10.1.3/Apache/open_ssl/bin/openssl
$ openssl version
OpenSSL 1.0.2q  20 Nov 2018

$ openssl x509 -in server.crt -noout -text  > server.txt
$ openssl x509 -in server.crt_BK_good -noout -text > server.txt_good
$ diff  server.txt server.txt_good    <== two text files are identical!

$ openssl x509 -noout -modulus -in server.crt | openssl md5
(stdin)= 637615c9fcc4c8945817b4941bbff708
$ openssl rsa -noout -modulus -in server.key | openssl md5
(stdin)= 637615c9fcc4c8945817b4941bbff708 

But when using this server.crt to start Apache, it gets error (which is misleading). :

$ ./adapcctl.sh start
You are running adapcctl.sh version 120.7.12010000.2

Starting OPMN managed Oracle HTTP Server (OHS) instance ...
sslSecureInit: SSL_CTX_use_certificate_chain_file($INST_TOP/certs/Apache/server.key): failed
4148582144:error:0906D066:PEM routines:PEM_read_bio:bad end line:pem_lib.c:805:
4148582144:error:140DC009:SSL routines:SSL_CTX_use_certificate_chain_file:PEM lib:ssl_rsa.c:708:
opmnctl: opmn start failed.
opmnctl: opmn is not running.

adapcctl.sh: exiting with status 0

The errors match Doc ID 2721500.1 (OPMN Start Fails with Error : SslSecureInit: SSL_CTX_use_certificate_chain_file Failed). But the solutions do not work. It may have a format or "new line" issue. Or, orapki only works well after version 11.1.1.7.0 (in R12.2)

No comments: