umask and default file permission in Linux

You can find the umask value in your Linux account by typing "umask" on the command line:

$ umask

0077

The value on umask in a Linux server level is defined in file /etc/login.defs (or maybe in /etc/profile, /etc/bashrc or /etc/cshrc). But you can set up your own umask in .profile for your account. 

$ more /etc/login.defs

MAIL_DIR                /var/spool/mail

PASS_MAX_DAYS   90

PASS_MIN_DAYS    7 

PASS_WARN_AGE  7

PASS_MIN_LEN     8

UID_MIN               1000

UID_MAX              60000

GID_MIN              1000

GID_MAX             60000

CREATE_HOME     yes

UMASK                   077

USERGROUPS_ENAB yes

ENCRYPT_METHOD   sha512

Under 0077, any file you created will have "-rw-------" permission, which means only yourself can read and write it.

$ touch test1.del

$ ls -al test1.del

-rw-------. 1 userID Group 0 Feb 02 13:25 test1.del

Sometimes, it is necessary for other users to read or modify a file created by a service account or you want others to read your files. To change the default from server level, you can add one line to the account's .profile:

$ vi $HOME/.profile

umask u=rwx,g=rwx,o=rx

or

umask  0002

After re-login, umask will change to 0002 in the account. Then, any file created by that account will get "-rw-rw-r--" permission. 

$ umask

0002

$ touch test2.del

$ ls -al test2.del

-rw-rw-r--. 1 userID Group 0 Feb 02 13:55 test2.del

Now, other users can read it.

If you put "umask u=rwx,g=rwx,o=rwx" or "umask  0000" in .profile, any new file will get "-rw-rw-rw-" permission (666). 

$ vi $HOME/.profile

umask u=rwx,g=rwx,o=rwx

$ umask

0000

$ touch test3.del

$ ls -al test3.del

-rw-rw-rw-. 1 userID Group 0 Feb 02 15:55 test3.del

If you have "umask  0022" in the profile, new file will get permission "-rw-r--r--".

Note that "x" in the .profile only applies to new folder creation. Linux allows only manually to grant executable to a file.

Apply R12.2 October 2024 CPU patches

The document for October 2024 CPU patches is Doc ID 3037725.1 (Oracle E-Business Suite Release 12.2 Critical Patch Update Availability Document - October 2024). We have to follow it to apply this CPU patches.

1. First of all, specify a location for holding all patch files. All zip file shall be downloaded/saved in sub-folders of this location

$ patch_folder=/u02/apps/CPU_Patches/Oct2024

2. Run EJCPUC script on database server to check the Java versions. A new utility, Oracle E-Business Suite Java Critical Patch Update Checker (EJCPUC, Patch 37171025), was released with the Oct 2024 CPU. EJCPUC Output:

$ bash ejcpuc.sh

#############################################################

## Checking DB tier Java for CPU 2024.10 on Platform IBM_AIX

#############################################################

## Check Database Version

#############################################################

Your database version is 19.25.0.0.0

         ORACLE_HOME     $ORACLE_HOME

         ORACLE_SID          EBSDEV

         ORACLE_UNQNAME

## Check Java Version of OJVM, Database JDK and EBS's appsutil JRE

#############################################################

 Latest Version  action  Your Version  bitness Java Location

 -------------- -------- ------------  ------- ---------------

 1.8.0_431       o)    _.__.101034000   64-bit   OJVM In database

 1.8.0_411               1.8.0_421             64-bit   $ORACLE_HOME/jdk/bin/java

 1.8.0_411        u)   1.8.0_271             64-bit   $ORACLE_HOME/appsutil/jre/bin/java

o) Apply the Database Release Update (DBRU) recommended by ETCC which will update the DB OJVM version to the latest

u) When the DB JDK version is updated to the latest - then follow section 3 of 1530033.1 to update this JRE

3. Download ETCC (patch 17537119). Make sure both database and apps use the same release of ETCC

$ cd $patch_folder/ETCC

$ grep 120 *.sh

checkDBpatch.sh:# $Header: checkDBpatch.sh 120.127 2024/11/12 15:02:31 chrhill noship $

checkMTpatch.sh:# $Header: checkMTpatch.sh 120.0.12020000.68 2024/04/12 16:35:27 chrhill noship 

3. DBAs apply database patches (after EBS services are stopped) to the database and make sure all requirements are met. ETCC Output :

$ ./checkDBpatch.sh

 +==========================================+

 |    Copyright (c) 2005, 2024 Oracle and/or its affiliates.        |

 |                     All rights reserved.                                             |

 |             Oracle E-Business Suite Release 12.2                       |

 |          Database EBS Technology Codelevel Checker            |

 +==========================================+

Database environment not set, going to check for GridHome.

Oracle Grid Infrastructure not identified.

Database environment not set and no Grid home found, so context file must be specified.

Enter full path to database context file: $ORACLE_HOME/appsutil/<CONTEXT_NAME>.xml

Validating context file: $ORACLE_HOME/appsutil/<CONTEXT_NAME>.xml

Using context file from user input:

$ORACLE_HOME/appsutil/<CONTEXT_NAME>.xml

Starting Database EBS Technology Codelevel Checker, Version 120.127

Mon Nov 25 16:41:09 EST 2024

Log file for this session: $ORACLE_HOME/appsutil/etcc/log/checkDBpatch_45744600.log

Identifying database release.

Database release set to 19.25.0.0.

Multitenant identified.

 - Container database (CDB) identified via s_cdb_name is CEBSDEV

 - Pluggable database (PDB) identified via s_pdb_name is EBSDEV

Connecting to database.

Database connection successful.

Database EBSDEV is in READ WRITE mode.

Identifying APPS and APPLSYS schema names.

 - APPS schema:   APPS

 - APPLSYS schema: APPLSYS

Checking for existence DB-ETCC results table.

Table to store DB-ETCC results already exists in the database.

Bugfix file ./db/onprem/txk_R1220_DB_base_bugs.xml: 120.0.12020000.85

This file will be used for identifying missing bugfixes.

Mapping file ./db/onprem/txk_R1220_DB_mappings.xml: 120.0.12020000.61

This file will be used for mapping bugfixes to patches.

+---------------------------------------------------------------------------------------+

  Always use the latest version of ETCC available in patch 17537119,

  as new bugfixes will not be checked by older versions of the utility.

  You should apply the latest recommended RU, BP, or PSU as appropriate.

+---------------------------------------------------------------------------------------+

Identified database DST version: 32

Checking Bugfix XML file for tag 19.25.0.0_RU.

Obtained list of bugfixes to be applied and list to be rolled back.

Validating OPatch version:

The OPatch utility is version 12.2.0.1.44.

DB-ETCC is compatible with this OPatch version.

Checking for applied patch history:

Found patch history in the inventory.

Checking mapping XML file for tag 19.25.0.0.241015DBRU.

All the required one-off bugfixes are present in database ORACLE_HOME.

5. Proceed with Apps patches

- Back up Apps file systems

- Start Apps services to make sure EBS works after database patching. 

- Stop Apps services 

6. ECPUC script (from patch p35583866) to identify what EBS CPU patches and security fixes are needed.

$ cd $patch_folder/ECPUC

$ sqlplus apps/appsPWD

SQL> @ECPUC.sql

... ...

E-Business Suite Critical Patch Update Checker (ECPUC)

ECPUC.sql may be run on any EBS 12.2 environment to identify missing patches that are in the latest EBS CPU.

You can download the latest version of ECPUC via Patch 35583866.

Refer to the README.txt in Patch 35583866 for instructions for running ECPUC and information regarding the generated ECPUC.lst report.

The checker generates the report ECPUC_YYYY-MM-DD_HH24-MI.lst that lists recommended EBS CPU patches and security fixes for your environment per Table 1 'CPU Patches for Oracle E-Business Suite' and Table 2 'Additional Patches Required' documented in the quarterly EBS CPU MOS Note.

Each quarterly EBS CPU MOS Note ID is unique. Refer to My Oracle Support (MOS) Knowledge Document 2484000.1, 'Identifying the Latest Critical Patch Update for Oracle E-Business Suite Release 12' which includes a link to the current EBS CPU MOS document.

******************************************************************

=================================================

SECTION-1 ECPUC Version

=================================================

EBS CPU Checker Version

-----------------------

                 2024.1

=================================================

SECTION-2 Oracle E-Business Suite (EBS): Instance Information

=================================================

******************************************************************

Instance Summary

******************************************************************

EBS Release EBS CPU Level

----------- -------------

12.2.10     2023.10

Instance Name    Database         Database Version

---------------- ---------------- ------------------

CEBSDEV           EBSDEV            19.25.0.0.0

ATG Product  Product Name                             Code Level

------------ ----------------------------------------------- ----------

ad           Applications DBA                                    C.14

txk         Oracle Applications Technology Stack     C.14

atg_pf    Oracle Applications Technology Family  C.9

fwk        Oracle Applications Framework               C.9

=================================================

SECTION-3 Required EBS CPU and Security Fixes

=================================================

******************************************************************

The following output is a list of required patches and security fixes that are missing in your environment.

It is strongly recommended that you apply all of the listed patches as soon as possible.

If no patches (no rows) are listed then no additional action is required at this time as your environment includes all patches for this EBS CPU.

******************************************************************

The following patches are required for this EBS CPU

--------------------------------------------------------------

36944346:12.2.0

37078855:R12.ATG_PF.C

37078915:R12.SCM_PF.C

37120482:R12.CC_PF.C

35362524:R12.IGI.C

34979060:R12.MSC.C

33457157:R12.HXT.C

30448458:R12.HXT.C

8 rows selected.

7. Apply two EBS patches. After consulting with Developers and Business users, we will apply only first two patches because we do not use the EBS modules on the list.

36944346:12.2.0

37078855:R12.ATG_PF.C

Make sure all Apps services are down

$ ps -ef | grep $LOGNAME

$ vi /etc/oraInst.loc

$ adop -status

If FS_CLONE has not been run, run it. This will help to screen out errors in applying next patches.

$ adop phase=fs_clone 

-- Apply patch to multiple nodes

$ adop phase=apply apply_mode=downtime patches=36944346 patchtop=$patch_folder/Oct24_CPU

$ adop phase=apply apply_mode=downtime patches=37078855 patchtop=$patch_folder/Post

Run query to confirm Oct 2024 CPU in the instance. See Doc ID 2484000.1 (Identifying the Latest Critical Patch Update for Oracle E-Business Suite Release 12.2)

SQL> col CPU format a9

SQL> select max(CODELEVEL) "CPU" from ad_trackable_entities where abbreviation in ('ebscpu');

CPU

---------

2024.10

8. Run autoconfig on database server

$ perl $AD_TOP/bin/admkappsutil.pl

Starting the generation of appsutil.zip

Log file located at $INST_TOP/admin/log/MakeAppsUtil_11260913.log

output located at $INST_TOP/admin/out/appsutil.zip

MakeAppsUtil completed successfully.

$ cp -p $INST_TOP/admin/out/appsutil.zip $APPLPTMP

$ chmod 777 $APPLPTMP/appsutil.zip

DBA run the below steps on EBSDEV database.

1. cp /u04/shared/utl_dir/appsutil.zip $ORACLE_HOME/

2. cd $ORACLE_HOME; unzip -o appsutil.zip

3. Set env and then run autoconfig on the database.

9. Run ETCC

$ cd $patch_folder/ETCC

$ ./checkMTpatch.sh

... ...

Starting Application Tier EBS Technology Codelevel Checker, Version 120.0.12020000.68.

... ...

===========================================================

PATCH RECOMMENDATION SUMMARY

===========================================================

One or more products have bugfixes missing.

The default patch recommendations to install these missing bugfixes are:

-------------------------------------------------------------------------------

Oracle Fusion Middleware (FMW) - Oracle Common 11.1.1.9.0

-------------------------------------------------------------------------------

  Patch 34714760

    - Filename: p34714760_111190_Generic.zip

-------------------------------------------------------------------------------

WLS 10.3.6.0.231017

-------------------------------------------------------------------------------

  Patch 35476084 [SU Patch [KMHV]]

    - Filename: p35476084_1036_Linux-x86-64.zip

Apply the required patches and rerun this script.

+-----------------------------------------------------------------------------+

A consolidated zip file with the required application tier patches is

available on My Oracle Support via:

  Patch 36616672

10.  Apply WLS patches: KMHV, WY44

$ echo $APPL_TOP 

$ echo $ORACLE_HOME

$RUN_BASE/EBSapps/10.1.2

$ echo $FMW_HOME

$RUN_BASE/FMW_Home

$ cd $FMW_HOME/utils/bsu

-- Check if patch p13845626 was applied:

$ ./bsu.sh -prod_dir=$FMW_HOME/wlserver_10.3 -status=applied -verbose -view | egrep -i 'WY44'

Patch ID:            WY44

PatchContainer:    WY44.jar

$ cd $FMW_HOME/utils/bsu/cache_dir

$ cp -p $patch_folder/WLS/35476084/p35476084_1036_Linux-x86-64.zip .   # (KMHV)

$ cp -p $patch_folder/WLS/13845626/p13845626_10360231017_Generic.zip . 

# (WY44. Asked by Doc ID 3037725.1)

$ unzip p35476084_1036_Linux-x86-64.zip

Archive:  p35476084_1036_Linux-x86-64.zip

replace README.txt? [y]es, [n]o, [A]ll, [N]one, [r]ename: A

  inflating: README.txt

  inflating: patch-catalog_27986.xml

  inflating: KMHV.jar

$ cd ..

$ ./bsu.sh -install -patch_download_dir=$FMW_HOME/utils/bsu/cache_dir -patchlist=KMHV -prod_dir=$FMW_HOME/wlserver_10.3

Checking for conflicts..

Conflict(s) detected - resolve conflict condition and execute patch installation again

Conflict condition details follow:

Patch KMHV is mutually exclusive and cannot coexist with patch(es): CW7X

$ ./bsu.sh -remove -patchlist=CW7X -prod_dir=$FMW_HOME/wlserver_10.3

Checking for conflicts..

No conflict(s) detected

Removing Patch ID: CW7X..

Result: Success

$ ./bsu.sh -install -patch_download_dir=$FMW_HOME/utils/bsu/cache_dir -patchlist=KMHV -prod_dir=$FMW_HOME/wlserver_10.3

Checking for conflicts..

No conflict(s) detected

Installing Patch ID: KMHV..

Result: Success

$ ./bsu.sh -prod_dir=$FMW_HOME/wlserver_10.3 -status=applied -verbose -view | egrep -i 'KMHV'

Patch ID:          KMHV

PatchContainer:    KMHV.jar

-- if patch p13845626 was NOT applied:

$ ./bsu.sh -install -patch_download_dir=$FMW_HOME/utils/bsu/cache_dir -patchlist=WY44 -prod_dir=$FMW_HOME/wlserver_10.3

11. Apply patch 34714760 to Oracle Fusion Middleware (FMW) - Common

$ export ORACLE_HOME=$FMW_HOME/oracle_common

$ export PATH=$ORACLE_HOME/OPatch:$PATH

$ which opatch

$ cd /tmpshrs/ebsarupgradeTEMP/CPU_Patches/Oct2024/FMW_OHS_Orcl_Comn

$ ls -al

$ mv 34714760 34714760_Feb19

$ unzip p34714760_111190_Generic.zip

$ cd 34714760

$ opatch apply

... ...

Patching component oracle.jrf.opss, 11.1.1.9.0...

... ...

-- Check if 4 patches were applied

$ opatch lsinventory | egrep -i '34330735|33974106|33960746|34714760'

-- if patch 33960746 was not applied

$ cd /tmpshrs/ebsarupgradeTEMP/CPU_Patches/Oct2024/FMW_OHS_Orcl_Comn

$ unzip p33960746_111190_Generic.zip    (required by Doc ID 3037725.1)

$ cd 33960746

$ opatch apply

... ...

Patching component oracle.sysman.common, 10.2.0.5.6...

Patching component oracle.sysman.oms.core, 11.1.1.9.0...

Patching component oracle.sysman.plugin.ai.main.oms, 11.1.1.9.0...

... ...

Note: if a patch was applied previously, "opatch apply" again will give below message: 

$ opatch apply

Applying interim patch '33960746' to OH '$FMW_HOME/oracle_common'

Verifying environment and performing prerequisite checks...

The following patch(es) are duplicate patches with patches installed in the Oracle Home.

 [ 33960746]

You have already installed same patch(es) with same UPI(s) or same version(s).

These patch(es) will be skipped.

... ...

OPatch stopped on request.

 

$ opatch lsinventory | egrep -i '34330735|33974106|33960746|34714760'

Patch  33960746     : applied on Thu Dec 05 12:03:32 EST 2024

     33960746

Patch  34714760     : applied on Thu Dec 05 11:48:27 EST 2024

     34714760

Patch  33974106     : applied on Fri Nov 24 13:31:49 EST 2023

Patch  34330735     : applied on Wed Aug 17 21:59:30 EDT 2022

12. Run ETCC again

Start a new OS session to get the correct env, and make sure no more patches are needed.

$ cd $patch_folder/ETCC

$ ./checkMTpatch.sh

==========================================

All required one-offs are confirmed as present.

Finished checking prerequisite patches for file edition: run.

13. Upgrade JDK

$ cd $patch_folder/EJCPUC

$ ls

ejcpuc.cmd  ejcpuc.sh  p37171025_R12_GENERIC.zip  Readme.txt

$ sh ejcpuc.sh

######################################################################

## Checking Apptier Java 7 for CPU 2024.10 on Platform Linux_x64 - need 1.7.0_441

## 

######################################################################

 2024.10        action  Your Version    bitness Java Location

 ------------   ------  ------------    ------- ---------------

 1.7.0_441      UPDATE   1.7.0_391    32-bit  $ORACLE_HOME/jdk/bin/java

 1.7.0_441      UPDATE   1.7.0_391    32-bit  $COMMON_TOP/util/jdk32/bin/java

 1.7.0_441      UPDATE   1.7.0_391    64-bit  $COMMON_TOP/util/jdk64/bin/java

 1.7.0_441      UPDATE   1.7.0_391    64-bit  $FMW_HOME/webtier/jdk/bin/java

Follow 1530033.1 to update the JDK(s). Your application tier JDK 7 is lower than the 1.7.0_441 update released in CPU 2024.10.

$ cd $patch_folder/JDK_1_7_441/JY

Assume 2 JDK files and my shell script for JDK upgrade exist in this folder:

$ ls

jdk-7u441-linux-i586.tar.gz

jdk-7u441-linux-x64.tar.gz

JDK_upgrade441.sh

$ ./JDK_upgrade441.sh

Verify:

$ cd $COMMON_TOP/util/

$ ls -al

$ cd $patch_folder/EJCPUC

$ sh ejcpuc.sh

##################################################################################

## Checking Apptier Java 7 for CPU 2024.10 on Platform Linux_x64 - need 1.7.0_441

## 

##################################################################################

 2024.10        action  Your Version    bitness Java Location

 ------------   ------  ------------    ------- ---------------

 1.7.0_441      OK      1.7.0_441       32-bit   $ORACLE_HOME/jdk/bin/java

 1.7.0_441      OK      1.7.0_441       32-bit   $COMMON_TOP/util/jdk32/bin/java

 1.7.0_441      OK      1.7.0_441       64-bit   $COMMON_TOP/util/jdk64/bin/java

 1.7.0_441      OK      1.7.0_441       64-bit   $FMW_HOME/webtier/jdk/bin/java

14. Upgrade JRE

$ cd $COMMON_TOP/webapps/oacore/util/javaplugin

# Assume below file is downloaded and unzipped in folder

# $patch_folder/JRE_8_431

# p37063177_180431_WINNT.zip

$ cp $patch_folder/JRE_8_431/jre-8u431-windows-i586.exe j2se18431.exe

$ ls -altr

$ $FND_TOP/bin/txkSetPlugin.sh 18431

$ grep sun_plugin_ver $CONTEXT_FILE

         <sun_plugin_ver oa_var="s_sun_plugin_ver">1.8.0_431</sun_plugin_ver>

$ grep s_forms_launch_method $CONTEXT_FILE

         <config_option type="techstack" oa_var="s_forms_launch_method">jws</config_option>

15. Finalize 

optional: it takes time: "Creating and signing every jar file can take about thirty minutes ..."

$ adadmin

option 1 => 4 => yes

$ autoconfig on all Apps nodes

Start all services

$ adop phase=fs_clone

=== script JDK_upgrade441.sh for upgrading JDK =====

$ more JDK_upgrade441.sh

# JDK upgrade  JY 11/19/2024

# Oracle JDK 7 Update 441 Patch Patch 37063192

DT=`date +"%h_%Y"`

curr=`pwd`

echo $curr

JDKfolder=/tmpshrs/ebsarupgradeTEMP/CPU_Patches/Oct2024/JDK_1_7_441/JY

# Assume two JDK files (from patch 37063192) must be saved in the folder:

# jdk-7u441-linux-i586.tar.gz

# jdk-7u441-linux-x64.tar.gz

ls -al $JDKfolder

echo "Current JDK version:"

$ADJVAPRG -version

$AFJVAPRG -version

# --

echo "$COMMON_TOP/util"

cd $COMMON_TOP/util

tar -czf jdk64_BK_$DT.tar.gz jdk64  # do not use -v (to turn off output)

tar -czf jdk32_BK_$DT.tar.gz jdk32

rm -fr jdk64

rm -fr jdk32

cp -p $JDKfolder/*.tar.gz .

tar -xzf jdk-7u441-linux-i586.tar.gz

mv jdk1.7.0_441 jdk32

tar -xzf jdk-7u441-linux-x64.tar.gz

mv jdk1.7.0_441 jdk64

ls -al jdk*

pwd

sleep 5

# --

echo "$FMW_HOME/webtier"

cd $FMW_HOME/webtier

tar -czf jdk64_BK_$DT.tar.gz jdk

rm -fr jdk

cp -p $JDKfolder/jdk-7u441-linux-x64.tar.gz .

tar -xzf jdk-7u441-linux-x64.tar.gz

mv jdk1.7.0_441 jdk

ls -al jdk*

pwd

sleep 5

# --

echo "$ORACLE_HOME"

cd $ORACLE_HOME

tar -czf jdk32_BK_$DT.tar.gz jdk

rm -fr jdk

cp -p $JDKfolder/jdk-7u441-linux-i586.tar.gz .

tar -xzf jdk-7u441-linux-i586.tar.gz

mv jdk1.7.0_441 jdk

ls -al jdk*

pwd

sleep 5

echo "New JDK version:"

$ADJVAPRG -version

$AFJVAPRG -version

echo "Compiling:"

cd $ORACLE_HOME/forms/lib

make -f ins_forms.mk sharedlib install

cd $ORACLE_HOME/reports/lib

make -f ins_reports.mk install

cd $curr

echo "Done"

fnd_web_sec.change_password in R12.2

fnd_web_sec.change_password still works in R12.2.10. It is recommended to use it only in some special/urgent needs because it ignores the restrictions by EBS Profile options 'Signon%'.

Below statement by APPS will change EBS_userID password:

SQL> select fnd_web_sec.change_password('EBS_userID','newPwd4U') from dual;

FND_WEB_SEC.CHANGE_PASSWORD('EBS_USERID','NEWPWD4U')

-------------------------------------------------------------------------------------

Y

You can use below line to confirm a password:

SQL> select fnd_web_sec.validate_login('EBS_userID', 'newPwd4U') from dual;

FND_WEB_SEC.VALIDATE_LOGIN('EBS_USERID','NEWPWD4U')

--------------------------------------------------------------------------------

Y

SQL> select fnd_web_sec.validate_login('EBS_userID', 'myPWD_01') from dual;

FND_WEB_SEC.VALIDATE_LOGIN('EBS_USERID','MYPWD_01')

--------------------------------------------------------------------------------

N

SQL> select fnd_message.get from dual;

GET

--------------------------------------------------------------------------------

PASSWORD_INVALID

One day, EBS users cannot log onto EBS site. The login webpage shows up but does not allow any users in. Since there is no error on EBS apps side, we do not know it is a security/password problem or other problems.  I used below queries to show it is a database problem 

SQL> show user

USER is "APPS"

SQL> select HOST_NAME, INSTANCE_NAME from v$instance;

HOST_NAME   INSTANCE_NAME

------------------  -------------------------

ebsdb1q             CEBSQA

SQL> select fnd_web_sec.validate_login('EBS_userID', 'XXXXxxx') from dual;   

ERROR at line 1:

ORA-03113: end-of-file on communication channel

SQL> select sysdate from dual;

ERROR:

ORA-03114: not connected to ORACLE

NOTE: FND_WEB_SEC.validate_password( ) is aonther function.

SQL> select fnd_web_sec.validate_password('EBS_userID', 'newPwd4U') from dual;

FND_WEB_SEC.VALIDATE_PASSWORD('EBS_USERID','NEWPWD4U')

--------------------------------------------------------------------------------

N

SQL> select fnd_message.get from dual;

GET

--------------------------------------------------------------------------------

Must not reuse a recently used password. Please supply a different password.

NOTE: fnd_message.get can be used sometimes to get useful information. For example, after a Java load errored out in Sql*Plus, below line gives some details:

SQL> select fnd_message.get from dual;

GET

--------------------------------------------------------------------------------

Unable to load Java class oracle.apps.xxfnd.custom.security.PasswordValidation specified in profile option SIGNON_PASSWORD_CUSTOM.  Please verify that the class exists and that it implements the Java interface oracle.apps.fnd.security.PasswordValidation.

EBS forms failed by CrowdStrike

EBS Forms in our financial applications suddenly does not work. The message on the webpage is

Failure of Web Server bridge:

No backend server available for connection: timed out after 10 seconds or idempotent set to OFF or method not idempotent. 

 

The error message does not tell the true problem. When checking into services on OS level, I saw Oracle EBS Forms service was not running and also saw errors from startup script $ADMIN_SCRIPTS_HOME/adstrtal.sh:

Forms service failed to start. 

The Node Manager is already up.

ERROR: Unable to start up the managed server forms_server1

Server specific logs are located at $EBS_DOMAIN_HOME/servers/forms_server1/logs

05/13/24-20:56:26 :: admanagedsrvctl.sh: exiting with status 1

Java error exists in Forms log file $EBS_DOMAIN_HOME/servers/forms_server1/logs/forms_server1.out

<May 13, 2024 8:56:25 PM EDT> <Emergency> <Store> <BEA-280060> <The persistent store "_WLS_forms_server1" encountered a fatal error, and it must be shut down: weblogic.store.PersistentStoreFatalException: [Store:280020]There was an error while reading from the log file

weblogic.store.PersistentStoreFatalException: [Store:280020]There was an error while reading from the log file

        at weblogic.store.io.file.FileStoreIO.open(FileStoreIO.java:128)

        at weblogic.store.internal.PersistentStoreImpl.recoverStoreConnections(PersistentStoreImpl.java:435)

        at weblogic.store.internal.PersistentStoreImpl.open(PersistentStoreImpl.java:423)

        at weblogic.store.admin.AdminHandler.activate(AdminHandler.java:126)

        at weblogic.store.admin.FileAdminHandler.activate(FileAdminHandler.java:207)

        Truncated.

Caused By: java.io.EOFException: premature EOF: expected=512, actual=126

        at weblogic.store.io.file.StoreFile.readBulk(StoreFile.java:316)

        at weblogic.store.io.file.Heap.readStoreFile(Heap.java:1142)

        at weblogic.store.io.file.Heap.getNextRecoveryFile(Heap.java:1226)

        at weblogic.store.io.file.Heap.open(Heap.java:373)

        at weblogic.store.io.file.FileStoreIO.open(FileStoreIO.java:117)

        Truncated.

Seems WebLogic failed to open a file, but the log did not say which file. I knew that Linux Admins just did server maintenance and rebooted server after they applied monthly patches and Security updates on OS level. That was the only change in the application environment recently.

After searching around, I found the Java errors match the description in Oracle Doc ID 3017110.1 ( Managed Forms Server Fails To Start - Displaying Message: FAILED_NOT_RESTARTABLE - ERROR: <BEA-280061> The persistent store "_WLS_forms_server1" could not be deployed: weblogic.store.PersistentStoreFatalException [Store:280020] ). 

The document points out the problem is caused by CrowdStrike, which locks a Forms file in $EBS_DOMAIN_HOME/servers/forms_server#/data/store/default.

CrowdStrike is installed in /opt/CrowdStrike. It is owned by root, and it is running constantly on the Linux server.

$ ps -ef | grep falcon-sensor

root      1081  1079  0 May13 ?        00:22:23 falcon-sensor

The problem can be fixed temporarily by a workaround:

1. Delete/rename below .DAT file (I guess CrowdStrike does not like the file name and so locks it)

$ cd $EBS_DOMAIN_HOME/servers/forms_server1/data/store/default

$ ls -altr

total 1028

drwxr-xr-x 4 user group      40 Sep 13  2023 ..

-rw-r--r-- 1   user group  1049088 May 13 20:51 _WLS_FORMS_SERVER1000000.DAT

drwxr-xr-x 2 user group     42 May 13 20:56 .

$ rm _WLS_FORMS_SERVER1000000.DAT

2. Re-start services cleanly by

$ADMIN_SCRIPTS_HOME/adstrtal.sh

Or

$ADMIN_SCRIPTS_HOME/admanagedsrvctl.sh start forms_server1

The permanent fix is that the Secure team rolls back the CrowdStrike change (and applies it again until CrowdStrike fixes the problem), because its new update touches an Oracle Forms data file wrongly during its scan. 

Shell script for renewing ssl certificate

My post Re-new R12.2 ssl certificate has details on how to renew a certificate. A shell script helps a lot when there are many EBS instances waiting for renewal. I wrote below script which takes only one minute to renew the cert on each node after the certificate is renewed on Venafi website and downloaded/copied to Linux server. 

As of today, we still have difficulties using .yaml file to extract certificate from Venafi server to Linux server automatically. We tried to set up a "push" way on Venafi website to do the automation. But if the password is changed on the Linux account, the push will fail. 

$ vi renew_cert.sh

# Script for renewing ssl certificate after new cert file is saved to Linux server

walletpwd='putPWDhere'

# walletpwd='tttest'

walletloc=$HOME/temp/Certs_Renew   # path where the Venafi cert file is saved

walletname='ewallet.p12'              # Must rename the file from Venafi to this name

certname='cwallet.sso'

echo "cert at: $walletloc"

echo "cert name: $walletname"

echo $walletpwd

cd $walletloc

errorC=`env| grep RUN_BASE | wc -l`

if [ $errorC -lt 1 ]; then

  echo "No R12.2 environment"

  exit 1

  # . $HOME/AEBSQA2/EBSapps.env RUN

fi

alias orapki=$FMW_HOME/oracle_common/bin/orapki

orapki wallet display -wallet $walletloc/$walletname -pwd $walletpwd > viewCert.log

errorC=`egrep -i 'PKI-' viewCert.log | wc -l`

echo "Error: $errorC"

if [ $errorC -gt 0 ]; then

   echo "The password is incorrect or the Venafi cert file is incorrect."

   exit 2

fi

DT=`date +"%h_%d_%y_%H%M"`

if [ -f $certname ]; then

   mv $certname ${certname}_${DT}

fi

orapki wallet create -wallet $walletloc/$walletname -pwd $walletpwd -auto_login

if [ ! -f $certname ]; then

   echo "Failure in getting new cert file. Exiting."

   exit 3

fi

echo " "

echo "Copy cert file to directories ..."

cd $NE_BASE/inst/$CONTEXT_NAME/certs    # save a copy in this folder

if [ -d Apache ]; then

mv Apache Apache_${DT}

fi

mkdir Apache

cd Apache

pwd

cp -p $walletloc/$walletname ${walletname}

cp -p $walletloc/$certname ${certname}

iName=$(tr < $CONTEXT_FILE '<>' '  ' | awk '/"s_ohs_instance"/ {print $(NF-1)}' )

SUBiName=${iName%?????}

cd $FMW_HOME/webtier/instances/$iName/config/OPMN/opmn/wallet

pwd

if [ -f $certname ]; then

   mv $certname ${certname}_${DT}

fi

cp -p $walletloc/$certname ${certname}

cd $FMW_HOME/webtier/instances/$iName/config/OHS/$SUBiName/keystores/default

pwd

if [ -f $certname ]; then

   mv $certname ${certname}_${DT}

fi

cp -p $walletloc/$certname ${certname}

cd $FMW_HOME/webtier/instances/$iName/config/OHS/$SUBiName/proxy-wallet

pwd

if [ -f $certname ]; then

   mv $certname ${certname}_${DT}

fi

cp -p $walletloc/$certname ${certname}

echo " "

echo "Recycle Apache service..."

cd $ADMIN_SCRIPTS_HOME

./adopmnctl.sh stop

sleep 10

./adopmnctl.sh status

./adapcctl.sh start

./adopmnctl.sh status

echo "Paths for log files:"

echo $FMW_HOME/webtier/instances/$iName/diagnostics/logs/OHS/$SUBiName

echo $FMW_HOME/webtier/instances/$iName/diagnostics/logs/OPMN/opmn

cd

==== Run the script ====

$ ./renew_cert.sh

cert at: $HOME/temp/Certs_Renew

cert name: ewallet.p12

putPWDhere

Error: 0

Oracle PKI Tool : Version 11.1.1.9.0

Copyright (c) 2004, 2015, Oracle and/or its affiliates. All rights reserved.

Copy cert file to directories ...

/u04/app/EBSQA/fs_ne/inst/EBSQA_nodeName/certs/Apache

$FMW_HOME/webtier/instances/EBS_web_EBSQA_OHS1/config/OPMN/opmn/wallet

$FMW_HOME/webtier/instances/EBS_web_EBSQA_OHS1/config/OHS/EBS_web_EBSQA/keystores/default

$FMW_HOME/webtier/instances/EBS_web_EBSQA_OHS1/config/OHS/EBS_web_EBSQA/proxy-wallet

Recycle Apache service ...

You are running adopmnctl.sh version 120.0.12020000.2

Stopping Oracle Process Manager (OPMN)  and the managed processes ...

opmnctl stopall: stopping opmn and all managed processes...

adopmnctl.sh: exiting with status 0

adopmnctl.sh: check the logfile $LOG_HOME/appl/admin/log/adopmnctl.txt for more information ...  

You are running adopmnctl.sh version 120.0.12020000.2

Checking status of OPMN managed processes...

opmnctl status: opmn is not running.

adopmnctl.sh: exiting with status 0

adopmnctl.sh: check the logfile $LOG_HOME/appl/admin/log/adopmnctl.txt for more information ...  

You are running adapcctl.sh version 120.0.12020000.6

Starting OPMN managed Oracle HTTP Server (OHS) instance ...

adapcctl.sh: exiting with status 0

adapcctl.sh: check the logfile $LOG_HOME/appl/admin/log/adapcctl.txt for more information ...  

You are running adopmnctl.sh version 120.0.12020000.2

Checking status of OPMN managed processes...

Processes in Instance: EBS_web_ARQA_OHS1

--------------------------------+--------------------+---------+---------

ias-component                    | process-type       |     pid    | status  

--------------------------------+--------------------+---------+---------

EBS_web_EBSQA             | OHS                   |   14542 | Alive   

adopmnctl.sh: exiting with status 0

adopmnctl.sh: check the logfile $LOG_HOME/appl/admin/log/adopmnctl.txt for more information ...  

Paths for log files:

$FMW_HOME/webtier/instances/EBS_web_EBSQA_OHS1/diagnostics/logs/OHS/EBS_web_EBSQA

$FMW_HOME/webtier/instances/EBS_web_EBSQA_OHS1/diagnostics/logs/OPMN/opmn

==== Files in the folder ====

$ ls 

renew_cert.sh

ewallet.p12

ewallet.p12.lck

cwallet.sso.lck

viewCert.log

cwallet.sso