Tuesday, December 2, 2025

October 2025 CPU patchset and CVE-2025-61882

Oracle emailed Security Alert CVE-2025-61882 out on Oct 5, 2025 and published Doc ID 3106344.1 (Security Alert CVE-2025-61882 Patch Availability Document for Oracle E-Business Suite) for it. The document first strongly recommends a set of patches 38501230, 38501349, 38501757 as hotfixes. Then, Oracle modified the document and recommended October 2025 CPU patch set or 6 hotfix patches 38507994, 38518258, 38523311, 38523302, and 38510732, 38501757.

I applied October 2025 CPU patch and found it includes all individual patches mentioned above. So, it is a good and clean way to fix CVE-2025-61882 vulnerabilities. Below are EBS patches I applied to my instances

38298685 12.2.0  Oct 2025 CPU
38261405 R12.FWK.C  Oracle Applications Framework
37450688 R12.OWF.C  Oracle Workflow
38180394 R12.FND.C  (prerequisite: OCT 2020 CPU: 31643029:12.2.0)
38510732 R12.XDO.C  BI Publisher (formerly XML Publisher)

Note ECPUC.sql only lists patches that are required or recommended by a CPU patch release but does NOT list prerequisites by each patch. For example, October 2025 CPU patch 38298685 requires R12.AD.C.DELTA.15 & R12.TXK.C.DELTA.15 (or higher) and so I had first to apply below two patches in instances that had AD and TXK Delta 14:

36119925 (R12.AD.C.DELTA.16) Built: JUL-16-2024
36117775 (R12.TXK.C.DELTA.16) Built: JUL-16-2024

Run new adgrants.sql as SYSDBA using 
SQL> @/path/to/adgrants.sql <APPS schema name>
$ adop phase=apply apply_mode=downtime patches=36119925,36303698,36989014,37988551 merge=yes patchtop=/path/to/Oct2025_CPU/AD workers=16 wait_on_failed_job=yes
$ adop phase=apply apply_mode=downtime patches=36117775,36641685,37500697 merge=yes patchtop=/path/to/Oct2025_CPU/TXK workers=16 wait_on_failed_job=yes

Detailed steps in applying patches in October 2025 CPU release are almost the same as in applying January 2025 patches.

To get the list of CVEs addressed by each CPU patchset, go to https://www.oracle.com/security-alerts and then click on the individual CPU release.




Tuesday, November 25, 2025

ADOP and adstrtal.sh may skip reading $HOME/.profile

When user applMgr on a primary node of RHEL8 runs ADOP to apply patches to slave/remote nodes or runs adstrtal.sh in allnodes mode to start EBS services on remote nodes, it does not read applMgr's .profile on remote nodes but get some OS settings from /etc/login.defs of the server instead. That makes some custom or important behaviors, such permission on file $CONTEXT_FILE or on files in $APPLCSF/out, inconsistent in a multi-node instance.

Two ways to avoid the problem:

1. Add line(s) to custom env file
$INST_TOP/appl/admin/custom$CONTEXT_NAME.env on remote nodes, e.g.
umask 0022
echo "this is for debug"

2. Comment out line(s), such as "# UMASK   077", in /etc/login.defs to force/make ADOP and adstrtal.sh get OS env settings from applMgr's .profile on remote nodes.

Similarly software tools, such as ftp & scp, also read OS env variables from file /etc/login.defs of server level. 

Sunday, October 19, 2025

How to retrieve WebLogic password in R12.2

WebLogic is used as part of Oracle EBS R12.2. "weblogic" account is used to start EBS services and is used to log onto WebLogic Admin console (at hostname.domain.com:s_wls_adminport). Its password is encrypted and saved on the file system. Here are two steps I ran in my R12.2 environments to decrypt it.

1. Identify WebLogic Domain path and boot.properties file

$ echo $EBS_DOMAIN_HOME
$RUN_BASE/FMW_Home/user_projects/domains/EBS_domain
$ cd $EBS_DOMAIN_HOME/servers/AdminServer/security
$ ls 
boot.properties
$ more boot.properties
username={AES}mE0mATL4+Lv/gLcIuuuuuuuuu=
password={AES}2kGMi4fcZ7FwYXWIxxxxxxxxx=

2. Run wlst.sh to decrypt the password

$ cd $FMW_HOME/wlserver_10.3/common/bin
$ ls
commEnv.sh       config.sh  security    startDerby.sh   stopDerby.sh  upgrade.sh     wlsifconfig.sh
config_builder.sh  pack.sh    setPatchEnv.sh  startManagedWebLogic.sh  unpack.sh     wlscontrol.sh  wlst.sh
$ sh wlst.sh
CLASSPATH=/u01/app/... ......
Initializing WebLogic Scripting Tool (WLST) ...
Welcome to WebLogic Server Administration Scripting Shell
Type help() for help on available commands
wls:/offline> domain = '$EBS_DOMAIN_HOME'   # <= replace the env variable with the real path
wls:/offline> service = weblogic.security.internal.SerializedSystemIni.getEncryptionService(domain)
wls:/offline> encryption = weblogic.security.internal.encryption.ClearOrEncryptedService(service)
wls:/offline> print "Weblogic server password is: %s" %encryption.decrypt("{AES}2kGMi4fcZ7FwYXWIxxxxxxxxx=")
Weblogic server password is: webLogicPWD
wls:/offline> ^C

If you get Java error, most likely you entered a wrong path or a wrong encrypted password/string.

My WebLogic version is 10.3.6.0.210119. It can be found by two lines:
$ . $FMW_HOME/wlserver_10.3/server/bin/setWLSEnv.sh
... ...
Your environment has been set.
$  java weblogic.version
... ...
WebLogic Server 10.3.6.0.210119 PSU Patch for ...
... ...
If you want to change weblogic password, please read How to change weblogic password

Wednesday, October 15, 2025

adcgnjar gets Java memory error

After custom Java code files are copied to EBS file system, adcgnjar gets Java error.

$ which adcgnjar
$AD_TOP//bin/adcgnjar

$ adcgnjar
Copyright (c) 2002, 2012 Oracle Corporation
Redwood Shores, California, USA
AD Custom Jar Generation
Version 12.2.0
... ...
About to Generate customall.jar : Fri Sep 26 2025 11:59:42

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at oracle.apps.ad.jri.fwk.ZipFileResourceUnitArea.getResourceUnit(ZipFileResourceUnitArea.java:914)
at oracle.apps.ad.jri.fwk.JRIArchiveOutputUtils.scanAllResourceUnits(JRIArchiveOutputUtils.java:659)
at oracle.apps.ad.jri.fwk.JRIArchiveOutputUtils.updateJRIArchiveZip(JRIArchiveOutputUtils.java:285)
at oracle.apps.ad.jri.fwk.JRIArchiveOutputUtils.writeJRIArchive(JRIArchiveOutputUtils.java:218)
at oracle.apps.ad.jri.adjmx.mergeAndExtract(adjmx.java:1395)
at oracle.apps.ad.jri.adjmx.main(adjmx.java:592)

error:
Failed to generate customall.jar
Restoring customall.bak to customall.jar

Add memory to server may not fix the problem. Fix:
$ export JAVA_TOOL_OPTIONS="-verbose -Xmx2500M -Xms2500M"
Then, run adcgnjar again.
 

Saturday, September 6, 2025

RHEL7 to RHEL8 upgrade on EBS server

Oracle EBS did not support in-place OS upgrade until April 30, 2024 when they updated Oracle document 1330701.1 (Oracle E-Business Suite Installation and Upgrade Notes Release 12 (12.2) for Linux x86-64):

An in-place OS upgrade from Oracle Linux 7 to 8 using the Leapp utility is certified for Oracle E-Business Suite Release 12.2. While you were previously required to install Oracle Linux 8 on a separate target environment before cloning or re-mounting file systems to it, you can now choose to preserve your current Oracle Linux 7 environment and perform an in-place upgrade to Oracle Linux 8 using Leapp. You should review the Operating System upgrades using Leapp section below.

If server team allows the server name unchanged as we did in RHEL 5, 6, and 7 upgrades, it is an easy task for EBS Apps Admin to bring EBS up on the new RHEL 8 server. But, if server team has to change the server name (e.g. migrate the host to a cloud), we have to clone EBS apps to the new server and to make other areas (such as in network, F5, security/firewall, FTP/email, custom codes and PPM) also work with cloned EBS instances.

RHEL 8 in-place upgrade steps on our R12.2.10 servers:

1. Check current RHEL version
$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.9 (Maipo)
2. Apply two patches to EBS: 
- WebLogic patch 32931976. Download the .zip file to get patch file p31090393_1036_Linux-x86-64.zip (which has no relation with patch 31090393). Without this patch, users get a blank page without error message when viewing output or log of concurrent requests. Unzip the file to cache_dir folder and then use bsu.sh to apply EG54. 

$ cd $FMW_HOME/utils/bsu/
$ ./bsu.sh -prod_dir=$FMW_HOME/wlserver_10.3 -status=applied -verbose -view | egrep -i 'EG54'

- Patch 33471111. Follow special steps in Doc ID 13307011.1 to apply it by using folder prereq from p33471111_111190_Generic.zip to replace both $FMW_HOME/webtier/inventory/Scripts/prereq and $PATCH_BASE/FMW_Home/webtier/inventory/Scripts/prereq

3. Verify apps (login page & forms) works and back up all file systems.
4. Shutdown all EBS services.
5. Linux Admin upgrades RHEL to RHEL 8 by using Leapp.
6. Verify it is RHEL8 now
$ cat /etc/redhat-release
Red Hat Enterprise Linux release 8.10 (Ootpa)

- Kernel Settings: /etc/sysctl.conf
Restore sysctl.conf from the backup that was taken prior to the upgrade.
- Make sure to include the options parameter in the resolve.conf.
$ cat /etc/resolv.conf | grep -i options
options timeout:1 attempts:5 rotate
- If the /etc/sysconfig/networking/profiles/default/network file exists, remove it.
- Values in /etc/security/limits.conf
Restore /etc/security/limits.conf from the backup
- Make sure /etc/oraInst.loc has proper permission.

7. Verify all rpm packages required by Oracle doc 1330701.1 exist on the server. Use YUM to install them, such as
$ yum install libXp-devel-1.0.3-3.el8.i686

Some important packages installed on my servers:
$ rpm -qa | grep libXp
libXp-devel-1.0.3-3.el8.i686
libXp-1.0.3-3.el8.x86_64
libXp-1.0.3-3.el8.i686
libXp-devel-1.0.3-3.el8.x86_64
libXpm-3.5.12-11.el8.x86_64

$ rpm -qa | grep motif
motif-devel-2.3.4-20.el8.x86_64
motif-2.3.4-20.el8.x86_64
motif-devel-2.3.4-20.el8.i686
motif-2.3.4-20.el8.i686
Notes: opemmotif21 is not installed

$ rpm -qa | grep redhat-lsb
redhat-lsb-desktop-4.1-47.el8.x86_64
redhat-lsb-printing-4.1-47.el8.x86_64
redhat-lsb-submod-security-4.1-47.el8.x86_64
redhat-lsb-cxx-4.1-47.el8.x86_64
redhat-lsb-submod-multimedia-4.1-47.el8.x86_64
redhat-lsb-languages-4.1-47.el8.x86_64
redhat-lsb-core-4.1-47.el8.x86_64
redhat-lsb-4.1-47.el8.x86_64

$ rpm -qa | grep xorg-x11     (Optional?)
xorg-x11-fonts-misc-7.5-19.el8.noarch
xorg-x11-xbitmaps-1.1.1-13.el8.noarch
xorg-x11-server-utils-7.7-27.el8.x86_64
xorg-x11-apps-7.7-21.el8.x86_64            <= xclock
xorg-x11-utils-7.5-28.el8.x86_64
xorg-x11-proto-devel-2020.1-3.el8.noarch
xorg-x11-xauth-1.0.9-12.el8.x86_64
xorg-x11-font-utils-7.5-41.el8.x86_64
xorg-x11-xinit-1.3.4-18.el8.x86_64

$ rpm -qa | grep libXrender
libXrender-devel-0.9.10-7.el8.x86_64
libXrender-0.9.10-7.el8.x86_64
libXrender-0.9.10-7.el8.i686

8. Run AutoConfig 
9. Start all EBS  services
10. After RHEL8 upgrade, some ports may not be opened on the server. If Apache services start without error, but EBS login webpage does not load up on browsers, the port number may become not open for TCP or may be blocked by a firewall after OS upgrade. In a case, "wget https://sitename.domain.com:4470" returned "connected" on the Linux server itself but got "failed: No route to host." from any remote server. After I asked Linux Admin to fix the issue on port 4470, the login page https://sitename.domain.com:4470 works on browsers.
Please check my post on port or https://runcloud.io/blog/check-tcp-port-linux,

11. Listener issue in opening concurrent job log. 
It was also due to port problem. When trying to open concurrent request log, get message an error occurred while attempting to establish an Applications File Server connection with the node FNDFS_nodeName. There may be a network configuration problem, or the TNS listener on node FNDFS_nodeName may not be running. Please contact your system administrator.

"tnsping FNDFS_nodeName" should work and tell the listener port on the server where FNDFS_nodeName is an entry of tnsnames.ora file. Most likely, the listener port is blocked on the server.

12. SELinux may be enabled in RHEL8. Check file /etc/selinux/config. That may make ssh passwordless login fails. "-Z" option in "ls -alZ" command is available only when selinux is enabled.

13. Make sure applMgr account has the right umask value in $HOME/.profile. This is very important if other OS users need to view output file (in $APPLCSF) of concurrent jobs or to submit concurrent jobs from a shell script.
$ umask 0022

14. Kill switches
Minimum Security Baseline (MSB), also known as a Minimum Baseline Security Standard (MBSS), is a set of foundational security guidelines and requirements that organizations must meet to protect their information systems and data, ensuring a basic level of security. 
MSB says $HOME directories should be 700 permissions (drwx------.). If an EBS is installed under /u01/app ($HOME) directory, the proper permission on app is 750 (drwxr-x---.) and its owner is applMgr. In that case, Server Admin has to add a kill switch so that Chef will not remove those permissions. A Kill Switch is one line (such as cis_rhel8_01_02_03=false) added to a company custom file (usually, it is /etc/xxxxx.info) to prevent something is uninstalled or is removed by Chef.

15. X11 is removed on RHEL8 unless an Exception (such as Archer exception. See https://www.archerirm.com) is filed, and a kill switch is added.
In RHEL8, folder lib under /usr/X11R6 may still exist, but /usr/X11R6/lib does not have any .so files or is empty. openmofit21 is replaced by another package in RHEL8. Without it, EBS Forms still works. If Forms Server failed to start with error FRM-92101, see Doc ID 3052209.1. Also check logs in $EBS_DOMAIN_HOME/servers/forms_server1/log/*.out

16. X-Windows is disabled in RHEL8 by default per Security Requirements as it does not meet the MSB as set in the CIS standards. An exception is needed to make X-Window (such xclock) work on laptop/PC.

17. Server tools in our RHEL8 servers are installed under /opt.
$ ls -d */
BESClient
Tivoli
tivoli
Symantec
appdynamics
chef
Tanium
splunkforwarder
CrowdStrike

Each of them is a folder. Use "ps -ef | grep chef" (e.g.) to check if the tool is running on the server.