The Distributed Caching System failed to initialize on port

Today was terrible day in office I had to face lots of issue for critical instances. All of sudden it was noticed that one of R12 instance was irresponsive. I was asked to jump to issue. I was asked not to restart concurrent manager. I was only suppose to bound applications (web server). In order to resolve issue I performed following.

1. Stopped Web services in following order

   Sourced application environment

   $ADMIN_SCRIPTS_HOME/adformsrvctl.sh stop

   $ADMIN_SCRIPTS_HOME /jtffmctl.sh stop

   $ADMIN_SCRIPTS_HOME /adoafmctl.sh stop

   $ADMIN_SCRIPTS_HOME /adoacorectl.sh stop

   $ADMIN_SCRIPTS_HOME /adapcctl.sh stop

   $ADMIN_SCRIPTS_HOME/adopmnctl.sh stop

   $ADMIN_SCRIPTS_HOME/mwactl.sh -login <sysadmin password> stop_force <MWAPort>

    

2. Started Services

   Sourced application enviroment

   $ADMIN_SCRIPTS_HOME/adopmnctl.sh start

   $ADMIN_SCRIPTS_HOME/adapcctl.sh start

   $ADMIN_SCRIPTS_HOME/adoacorectl.sh start

   $ADMIN_SCRIPTS_HOME/adoafmctl.sh start

   $ADMIN_SCRIPTS_HOME/jtffmctl.sh start

   ${SCRIPT_DIR}/mwactl.sh start

   $ADMIN_SCRIPTS_HOME/adopmnctl.sh status

   $ADMIN_SCRIPTS_HOME/adformsrvctl.sh start

Even after restarting services did not came up. I noticed given below errors for "s_java_object_cache_port" context variaable (from $CONTEXT_FILE) port.

$LOG_HOME/logs/ora/10.1.3/opmn/oafm_default_group_1/oafmstd.err

10/02/05 11:47:38 Error: <connector name="OracleASjms" path="OracleASjms.rar" /> will not be bootstrapped since corresponding module declaration was not found in application.xml.

10/02/05 11:47:38 Error: <connector name="OracleASjms" path="OracleASjms.rar" /> will not be bootstrapped since corresponding module declaration was not found

in application.xml.

10/02/05 11:48:06 Exception in static block of jtf.cache.CacheManager. Stack trace is: oracle.apps.jtf.base.resources.FrameworkException: IAS Cache initializa

tion failed. The Distributed Caching System failed to initialize on port: s_java_object_cache_port. The list of hosts in the distributed caching system is: <ipaddress>. The port <port> should be free on each host running the JVMs.

at oracle.apps.jtf.cache.IASCacheProvider.init(IASCacheProvider.java:335)

at oracle.apps.jtf.cache.CacheManager.activateCache(CacheManager.java:1451)

at oracle.apps.jtf.cache.CacheManager.initCache(CacheManager.java:768)

$LOG_HOME /logs/ora/10.1.3/opmn/oacore_default_group_2/oacorestd.err

Error: <connector name="OracleASjms" path="OracleASjms.rar" /> will not be bootstrapped since corresponding module declaration was not found

in application.xml.

10/02/05 11:47:32 Exception in static block of jtf.cache.CacheManager. Stack trace is: oracle.apps.jtf.base.resources.FrameworkException: IAS Cache initializa

tion failed. The Distributed Caching System failed to initialize on port: s_java_object_cache_port. The list of hosts in the distributed caching system is: <IPADDRESS>. The po

rt s_java_object_cache_port should be free on each host running the JVMs.

at oracle.apps.jtf.cache.IASCacheProvider.init(IASCacheProvider.java:335)

at oracle.apps.jtf.cache.CacheManager.activateCache(CacheManager.java:1451)

$LOG_HOME /logs/ora/10.1.3/opmn/oacore_default_group_3/oacorestd.err

Error: <connector name="OracleASjms" path="OracleASjms.rar" /> will not be bootstrapped since corresponding module declaration was not found

in application.xml.

10/02/05 11:46:49 Exception in static block of jtf.cache.CacheManager. Stack trace is: oracle.apps.jtf.base.resources.FrameworkException: IAS Cache initializa

tion failed. The Distributed Caching System failed to initialize on port: s_java_object_cache_port. The list of hosts in the distributed caching system is: <IPAddress> The po

rt <port> should be free on each host running the JVMs.

at oracle.apps.jtf.cache.IASCacheProvider.init(IASCacheProvider.java:335)

at oracle.apps.jtf.cache.CacheManager.activateCache(CacheManager.java:1451)

at oracle.apps.jtf.cache.CacheManager.initCache(CacheManager.java:768)

at oracle.apps.jtf.cache.CacheManager.<clinit>(CacheManager.java:390)

 

Solution

I had to restart oracle applications Rel 12 web services along with concurrent manager.

I performed following.

1. Source environmennt
2. Stoped all web services running on middle tier
3. Stopped concurrent manager
4. Started web services along with concurrent manager
5. I also checked $LOG_HOME /logs/ora/10.1.3/opmn/*-default_group_?/oa*std.err file and did not noticed above errors