Apache Stratos has a significant amount of read/write locks in the codebase, to synchronize data structure updates made by multiple threads.  However, if these read/write locks are not properly implemented to handle the following scenarios, it can lead to several problems:
  • A thread trying to acquire a write lock while having a read lock.
  • Locks acquired but not released.
  • Trying to release a lock that has not been acquired by the same thread.

The ReadWriteLock class, which is available in the Stratos common module, can be used to detect any of the latter mentioned issues. This class makes use of the ReadWriteLockMonitor.java and ReadWriteLock.java Java modules, which are in the <STRATOS_SOURCE_HOME>/components/org.apache.stratos.common/src/main/java/org/apache/stratos/common/concurrent/locks/ directory,  and implements additional logic to detect the above mentioned issues. By default, the problem detection logic for read/write locks is disabled.

Enabling the detection logic for read/write locks

Follow the instructions below to enable the detection logic for read/write locks:

  1. Navigate to the stratos.sh file, which is in the <STRATOS_HOME>/bin/ directory.

  2. Enable the detection logic for read/write locks as follows:

    -Dread.write.lock.monitor.enabled=true

Example scenario

The following is an example scenario where an unreleased lock in the REST API was detected:

org.apache.stratos.common.exception.LockNotReleasedException
	at org.apache.stratos.common.concurrent.locks.ReadWriteLockMonitor.checkTimeout(ReadWriteLockMonitor.java:71)
	at org.apache.stratos.common.concurrent.locks.ReadWriteLockMonitor.run(ReadWriteLockMonitor.java:50)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
[2015-01-20 18:30:12,952] ERROR {org.apache.stratos.common.concurrent.locks.ReadWriteLockMonitor} -  System error, lock has not released for 30 seconds: [lock-name] topology [lock-type] Read [thread-id] 405 [thread-name] http-nio-9443-exec-21 [stack-trace]
java.lang.Thread.getStackTrace(Thread.java:1589)
org.apache.stratos.common.concurrent.locks.ReadWriteLock.acquireReadLock(ReadWriteLock.java:160)
org.apache.stratos.messaging.message.receiver.topology.TopologyManager.acquireReadLockForCluster(TopologyManager.java:173)
org.apache.stratos.rest.endpoint.api.StratosApiV41Utils.addClustersInstancesToGroupInstanceBean(StratosApiV41Utils.java:1015)
org.apache.stratos.rest.endpoint.api.StratosApiV41Utils.setSubGroupInstances(StratosApiV41Utils.java:1027)
org.apache.stratos.rest.endpoint.api.StratosApiV41Utils.addGroupsInstancesToApplicationInstanceBean(StratosApiV41Utils.java:979)
org.apache.stratos.rest.endpoint.api.StratosApiV41Utils.getApplicationRuntime(StratosApiV41Utils.java:963)
org.apache.stratos.rest.endpoint.api.StratosApiV41.getApplicationRuntime(StratosApiV41.java:579)
  • No labels