Today was terrible day in office I had to face lots of issue for critical instances. All of sudden it was noticed that one of R12 instance was irresponsive. I was asked to jump to issue. I was asked not to restart concurrent manager. I was only suppose to bound applications (web server). In order to resolve issue I performed following.
Stopped Web services in following order
Sourced application environment
$ADMIN_SCRIPTS_HOME/adformsrvctl.sh stop
$ADMIN_SCRIPTS_HOME /jtffmctl.sh stop
$ADMIN_SCRIPTS_HOME /adoafmctl.sh stop
$ADMIN_SCRIPTS_HOME /adoacorectl.sh stop
$ADMIN_SCRIPTS_HOME /adapcctl.sh stop
$ADMIN_SCRIPTS_HOME/adopmnctl.sh stop
$ADMIN_SCRIPTS_HOME/mwactl.sh -login <sysadmin password> stop_force <MWAPort>
Started Services
Sourced application enviroment
$ADMIN_SCRIPTS_HOME/adopmnctl.sh start
$ADMIN_SCRIPTS_HOME/adapcctl.sh start
$ADMIN_SCRIPTS_HOME/adoacorectl.sh start
$ADMIN_SCRIPTS_HOME/adoafmctl.sh start
$ADMIN_SCRIPTS_HOME/jtffmctl.sh start
${SCRIPT_DIR}/mwactl.sh start
$ADMIN_SCRIPTS_HOME/adopmnctl.sh status
$ADMIN_SCRIPTS_HOME/adformsrvctl.sh start
Even after restarting services did not came up. I noticed given below errors for "s_java_object_cache_port" context variaable (from $CONTEXT_FILE) port.
$LOG_HOME/logs/ora/10.1.3/opmn/oafm_default_group_1/oafmstd.err
10/02/05 11:47:38 Error: <connector name="OracleASjms" path="OracleASjms.rar" /> will not be bootstrapped since corresponding module declaration was not found in application.xml.
10/02/05 11:47:38 Error: <connector name="OracleASjms" path="OracleASjms.rar" /> will not be bootstrapped since corresponding module declaration was not found
in application.xml.
10/02/05 11:48:06 Exception in static block of jtf.cache.CacheManager. Stack trace is: oracle.apps.jtf.base.resources.FrameworkException: IAS Cache initializa
tion failed. The Distributed Caching System failed to initialize on port: s_java_object_cache_port. The list of hosts in the distributed caching system is: <ipaddress>. The port <port> should be free on each host running the JVMs.
at oracle.apps.jtf.cache.IASCacheProvider.init(IASCacheProvider.java:335)
at oracle.apps.jtf.cache.CacheManager.activateCache(CacheManager.java:1451)
at oracle.apps.jtf.cache.CacheManager.initCache(CacheManager.java:768)
$LOG_HOME /logs/ora/10.1.3/opmn/oacore_default_group_2/oacorestd.err
Error: <connector name="OracleASjms" path="OracleASjms.rar" /> will not be bootstrapped since corresponding module declaration was not found
in application.xml.
10/02/05 11:47:32 Exception in static block of jtf.cache.CacheManager. Stack trace is: oracle.apps.jtf.base.resources.FrameworkException: IAS Cache initializa
tion failed. The Distributed Caching System failed to initialize on port: s_java_object_cache_port. The list of hosts in the distributed caching system is: <IPADDRESS>. The po
rt s_java_object_cache_port should be free on each host running the JVMs.
at oracle.apps.jtf.cache.IASCacheProvider.init(IASCacheProvider.java:335)
at oracle.apps.jtf.cache.CacheManager.activateCache(CacheManager.java:1451)
$LOG_HOME /logs/ora/10.1.3/opmn/oacore_default_group_3/oacorestd.err
Error: <connector name="OracleASjms" path="OracleASjms.rar" /> will not be bootstrapped since corresponding module declaration was not found
in application.xml.
10/02/05 11:46:49 Exception in static block of jtf.cache.CacheManager. Stack trace is: oracle.apps.jtf.base.resources.FrameworkException: IAS Cache initializa
tion failed. The Distributed Caching System failed to initialize on port: s_java_object_cache_port. The list of hosts in the distributed caching system is: <IPAddress> The po
rt <port> should be free on each host running the JVMs.
at oracle.apps.jtf.cache.IASCacheProvider.init(IASCacheProvider.java:335)
at oracle.apps.jtf.cache.CacheManager.activateCache(CacheManager.java:1451)
at oracle.apps.jtf.cache.CacheManager.initCache(CacheManager.java:768)
at oracle.apps.jtf.cache.CacheManager.<clinit>(CacheManager.java:390)
Solution
I had to restart oracle applications Rel 12 web services along with concurrent manager.
I performed following.
- Source environmennt
- Stoped all web services running on middle tier
- Stopped concurrent manager
- Started web services along with concurrent manager
- I also checked $LOG_HOME /logs/ora/10.1.3/opmn/*-default_group_?/oa*std.err file and did not noticed above errors