Wednesday, October 10, 2018

adoacorectl.sh exiting with status 204

After server rebooted unexpectedly, adoacorectl.sh failed to start OACORE OC4J due to opmn issue and made R12.1.3 initial page unable to re-direct to login page. The error is

adoacorectl.sh: exiting with status 204

File $INST_TOP/logs/appl/admin/log/adoacorectl.txt shows more detail:
10/09/18-11:37:25 :: adoacorectl.sh version 120.13
10/09/18-11:37:25 :: adoacorectl.sh: starting OPMN if it is not running
opmnctl: opmn is already running.
10/09/18-11:37:25 :: adoacorectl.sh: Starting OPMN managed OACORE OC4J instance
opmnctl: starting opmn managed processes...
============================================
opmn id=
hostname.domian.com:6240
    0 of 1 processes started.

ias-instance id=EBSDEV_hostname.hostname.domian.com
++++++++++++++++++++++++++++++++++++++++++++
ias-component/process-type/process-set:
    default_group/oacore/default_group/

Error
--> Process (index=1,uid=1230729787,pid=17205)
    failed to start a managed process after the maximum retry limit
    Log:
   $INST_TOP/logs/ora/10.1.3/opmn/default_group~oacore~default_group~1.log

10/09/18-11:37:37 :: adoacorectl.sh: exiting with status 204

File $INST_TOP/logs/ora/10.1.3/opmn/default_group~oacore~default_group~1.log has below warning which was not new and existed before:
--------
18/10/09 11:37:25 Start process
--------
18/10/09 11:37:26 WARNING: ApplicationLogManager is not installed, may result in loader leaks. Set -Djava.util.logging.manager=oracle.classloader.util.ApplicationLogManager


I did not make any changes in EBS in past months. Seems the problem was with opmn, but I do not know what it is. So I just ran autocofig. After that, "adoacorectl.sh start" worked surprisingly and login page showed up. The warning message with ApplicationLogManager still stays in the opmn log file.

The problem could be some session lock files got inconsistent by the server crash.

BTW, to turn trace (increase logging) on OC4J oacore by editing two files:

Edit j2ee-logging.xml adjust the following in file:
$ORA_CONFIG_HOME/10.1.3/j2ee/oacore/config/j2ee-logging.xml
<logger name=’oracle’ level='TRACE:32′ useParentHandlers=’false’>
Edit orion-web.xml adjust the following in file:
$ORA_CONFIG_HOME/10.1.3/j2ee/oacore/application-deployments/oacore/html/orion-web.xml
<param-name>debug_mode</param-name>
<param-value>true</param-value>
But it is difficult to read and understand the log log.xml under $LOG_HOME/ora/10.1.3/j2ee/oacore/oacore_default_group_1