Package org.eclipse.persistence.internal.helper

Class ConcurrencyUtil

  • public class ConcurrencyUtil
extends Object

    • Field Detail

      • DEFAULT_USE_SEMAPHORE_TO_SLOW_DOWN_OBJECT_BUILDING_CONCURRENCY

        public static final boolean DEFAULT_USE_SEMAPHORE_TO_SLOW_DOWN_OBJECT_BUILDING_CONCURRENCY
        See Also:
        Constant Field Values

      • DEFAULT_USE_SEMAPHORE_TO_SLOW_DOWN_WRITE_LOCK_MANAGER_ACQUIRE_REQUIRED_LOCKS

        public static final boolean DEFAULT_USE_SEMAPHORE_TO_SLOW_DOWN_WRITE_LOCK_MANAGER_ACQUIRE_REQUIRED_LOCKS
        See Also:
        Constant Field Values

      • DEFAULT_CONCURRENCY_MANAGER_OBJECT_BUILDING_NO_THREADS

        public static final int DEFAULT_CONCURRENCY_MANAGER_OBJECT_BUILDING_NO_THREADS
        See Also:
        Constant Field Values

      • DEFAULT_CONCURRENCY_MANAGER_WRITE_LOCK_MANAGER_ACQUIRE_REQUIRED_LOCKS_NO_THREADS

        public static final int DEFAULT_CONCURRENCY_MANAGER_WRITE_LOCK_MANAGER_ACQUIRE_REQUIRED_LOCKS_NO_THREADS
        See Also:
        Constant Field Values

      • DEFAULT_CONCURRENCY_SEMAPHORE_MAX_TIME_PERMIT

        public static final long DEFAULT_CONCURRENCY_SEMAPHORE_MAX_TIME_PERMIT
        See Also:
        Constant Field Values

      • DEFAULT_CONCURRENCY_SEMAPHORE_LOG_TIMEOUT

        public static final long DEFAULT_CONCURRENCY_SEMAPHORE_LOG_TIMEOUT
        See Also:
        Constant Field Values

    • Method Detail

      • determineIfReleaseDeferredLockAppearsToBeDeadLocked

        public void determineIfReleaseDeferredLockAppearsToBeDeadLocked​(ConcurrencyManager concurrencyManager,
                                                                long whileStartTimeMillis,
                                                                DeferredLockManager lockManager,
                                                                ReadLockManager readLockManager,
                                                                boolean callerIsWillingToAllowInterruptedExceptionToBeFiredUpIfNecessary)
                                                         throws InterruptedException
        Throw an interrupted exception if appears that eclipse link code is taking too long to release a deferred lock.
        Parameters:
        whileStartTimeMillis - the start date of the while tru loop for releasing a deferred lock
        callerIsWillingToAllowInterruptedExceptionToBeFiredUpIfNecessary - this flag is to allow the write lock manager to say that it is afraid of a concurrency exception being fire up because the thread in a dead lock might be trying to do a commit and blowing these threads up is most likely too dangerous and possibly the eclipselink code is not robust enough to code with such scenarios We do not care so much about blowing up exception during object building but during committing of transactions we are very afraid
        Throws:
        InterruptedException - we fire an interrupted exception to ensure that the code blows up and releases all of the locks it had.

      • getAcquireWaitTime

        public long getAcquireWaitTime()
        Returns:
        "eclipselink.concurrency.manager.waittime" persistence property value.

      • setAcquireWaitTime

        public void setAcquireWaitTime​(long acquireWaitTime)

      • getBuildObjectCompleteWaitTime

        public long getBuildObjectCompleteWaitTime()
        Returns:
        "eclipselink.concurrency.manager.build.object.complete.waittime" persistence property value.

      • setBuildObjectCompleteWaitTime

        public void setBuildObjectCompleteWaitTime​(long buildObjectCompleteWaitTime)

      • getMaxAllowedSleepTime

        public long getMaxAllowedSleepTime()
        Returns:
        property to control how long we are willing to wait before firing up an exception

      • setMaxAllowedSleepTime

        public void setMaxAllowedSleepTime​(long maxAllowedSleepTime)

      • getMaxAllowedFrequencyToProduceTinyDumpLogMessage

        public long getMaxAllowedFrequencyToProduceTinyDumpLogMessage()
        Just like we have a massive dump log message see getMaxAllowedFrequencyToProduceMassiveDumpLogMessage() we also want threads to produce "tiny" dump about the fact that they rae stuck. We want to avoid these threads spaming too much the server log ... once the log message is out there not much point in continuously pumping the same log message out over and over again. Controlling how frequently the tiny dump is important especially when the user configures the hacked eclipselink to not fire up a blow up exception and instead to allow eclipselink to remain frozen forever.
        Returns:
        the frequency with which we are allowed to create a tiny dump log message

      • setMaxAllowedFrequencyToProduceTinyDumpLogMessage

        public void setMaxAllowedFrequencyToProduceTinyDumpLogMessage​(long maxAllowedFrequencyToProduceTinyDumpLogMessage)

      • getMaxAllowedFrequencyToProduceMassiveDumpLogMessage

        public long getMaxAllowedFrequencyToProduceMassiveDumpLogMessage()
        If the system is perceived to be frozen and not evolving any longer, we will allow that every so often (e.g. once a minute) the logic complaining that the thread is stuck and going nowhere logs a very big dump message where the FULL concurrency manager state is explained. So that we can (manually) try to understand the dead lock based on the dumped information See also dateWhenLastConcurrencyManagerStateFullDumpWasPerformed.

      • setMaxAllowedFrequencyToProduceMassiveDumpLogMessage

        public void setMaxAllowedFrequencyToProduceMassiveDumpLogMessage​(long maxAllowedFrequencyToProduceMassiveDumpLogMessage)

      • isAllowInterruptedExceptionFired

        public boolean isAllowInterruptedExceptionFired()

      • setAllowInterruptedExceptionFired

        public void setAllowInterruptedExceptionFired​(boolean allowInterruptedExceptionFired)

      • isAllowConcurrencyExceptionToBeFiredUp

        public boolean isAllowConcurrencyExceptionToBeFiredUp()
        Returns:
        true if we are supposed to be firing up exception to abort the thread in a dead lock, false we are afraid of trying to abort the transaction and not managing to resolve the dead lock and prefer to system frozen and be forced into restarting it.

      • setAllowConcurrencyExceptionToBeFiredUp

        public void setAllowConcurrencyExceptionToBeFiredUp​(boolean allowConcurrencyExceptionToBeFiredUp)

      • isAllowTakingStackTraceDuringReadLockAcquisition

        public boolean isAllowTakingStackTraceDuringReadLockAcquisition()

      • setAllowTakingStackTraceDuringReadLockAcquisition

        public void setAllowTakingStackTraceDuringReadLockAcquisition​(boolean allowTakingStackTraceDuringReadLockAcquisition)

      • isUseSemaphoreInObjectBuilder

        public boolean isUseSemaphoreInObjectBuilder()

      • setUseSemaphoreInObjectBuilder

        public void setUseSemaphoreInObjectBuilder​(boolean useSemaphoreInObjectBuilder)

      • isUseSemaphoreToLimitConcurrencyOnWriteLockManagerAcquireRequiredLocks

        public boolean isUseSemaphoreToLimitConcurrencyOnWriteLockManagerAcquireRequiredLocks()

      • setUseSemaphoreToLimitConcurrencyOnWriteLockManagerAcquireRequiredLocks

        public void setUseSemaphoreToLimitConcurrencyOnWriteLockManagerAcquireRequiredLocks​(boolean useSemaphoreToLimitConcurrencyOnWriteLockManagerAcquireRequiredLocks)

      • getNoOfThreadsAllowedToObjectBuildInParallel

        public int getNoOfThreadsAllowedToObjectBuildInParallel()

      • setNoOfThreadsAllowedToObjectBuildInParallel

        public void setNoOfThreadsAllowedToObjectBuildInParallel​(int noOfThreadsAllowedToObjectBuildInParallel)

      • getNoOfThreadsAllowedToDoWriteLockManagerAcquireRequiredLocksInParallel

        public int getNoOfThreadsAllowedToDoWriteLockManagerAcquireRequiredLocksInParallel()

      • setNoOfThreadsAllowedToDoWriteLockManagerAcquireRequiredLocksInParallel

        public void setNoOfThreadsAllowedToDoWriteLockManagerAcquireRequiredLocksInParallel​(int noOfThreadsAllowedToDoWriteLockManagerAcquireRequiredLocksInParallel)

      • getConcurrencySemaphoreMaxTimePermit

        public long getConcurrencySemaphoreMaxTimePermit()

      • setConcurrencySemaphoreMaxTimePermit

        public void setConcurrencySemaphoreMaxTimePermit​(long concurrencySemaphoreMaxTimePermit)

      • getConcurrencySemaphoreLogTimeout

        public long getConcurrencySemaphoreLogTimeout()

      • setConcurrencySemaphoreLogTimeout

        public void setConcurrencySemaphoreLogTimeout​(long concurrencySemaphoreLogTimeout)

      • createToStringExplainingOwnedCacheKey

        public String createToStringExplainingOwnedCacheKey​(ConcurrencyManager concurrencyManager)
        Returns:
        A to string of the cache key (e.g. that we are trying to lock

      • currentThreadIsStuckForSomeTimeProduceTinyLogMessage

        protected String currentThreadIsStuckForSomeTimeProduceTinyLogMessage​(long elapsedTime,
                                                                      ConcurrencyManager concurrencyManager,
                                                                      DeferredLockManager lockManager,
                                                                      ReadLockManager readLockManager)
        We have a thread that is not evolving for quite some while. This is a fairy good indication of eclipselink being stuck in a dead lock. So we log some information about the thread that is stuck.
        Parameters:
        elapsedTime - how many ms have passed since the thread stopped moving
        concurrencyManager - the current cache key that the thread is trying to acquire or the object where the thread is waiting for the release deferred locks .
        lockManager - the lock manager
        readLockManager - the read lock manager
        Returns:
        Return the string with the tiny message we logged on the server log. This message can be interesting if we decide to fire up an interrupted exception

      • tooMuchTimeHasElapsed

        public boolean tooMuchTimeHasElapsed​(long whileStartTimeMillis,
                                     long maxAllowedSleepTimeMs)

      • dumpConcurrencyManagerInformationIfAppropriate

        public void dumpConcurrencyManagerInformationIfAppropriate()
        Invoke the dumpConcurrencyManagerInformationStep01(Map, Map, Map, Map, Map, Map, Map, Set, Map, Map) if sufficient time has passed. This log message will potentially create a massive dump in the server log file. So we need to check when was the last time that the masive dump was produced and decide if we can log again the state of the concurrency manager. The access to dateWhenLastConcurrencyManagerStateFullDumpWasPerformedLock is synchronized, because we do not want two threads in parallel to star deciding to dump the complete state of the concurrency manager at the same time. Only one thread should succeed in producing the massive dump in a given time window.

      • dumpConcurrencyManagerInformationStep01

        protected void dumpConcurrencyManagerInformationStep01​(Map<Thread,​DeferredLockManager> deferredLockManagers,
                                                       Map<Thread,​ReadLockManager> readLockManagersOriginal,
                                                       Map<Thread,​ConcurrencyManager> mapThreadToWaitOnAcquireOriginal,
                                                       Map<Thread,​String> mapThreadToWaitOnAcquireMethodNameOriginal,
                                                       Map<Thread,​Set<ConcurrencyManager>> mapThreadToWaitOnAcquireInsideWriteLockManagerOriginal,
                                                       Map<Thread,​ConcurrencyManager> mapThreadToWaitOnAcquireReadLockOriginal,
                                                       Map<Thread,​String> mapThreadToWaitOnAcquireReadLockMethodNameOriginal,
                                                       Set<Thread> setThreadWaitingToReleaseDeferredLocksOriginal,
                                                       Map<Thread,​String> mapThreadsThatAreCurrentlyWaitingToReleaseDeferredLocksJustificationClone,
                                                       Map<Thread,​Set<Object>> mapThreadToObjectIdWithWriteLockManagerChangesOriginal)
        The current working thread is having problems. It seems to not go forward being stuck either trying to acquire a cache key for writing, as a deferred cache key or it is at the end of the process and it is waiting for some other thread to finish building some objects it needed to defer. Now that the system is frozen we want to start spamming into the server log file the state of the concurrency manager since this might help us understand the situation of the system.
        Parameters:
        deferredLockManagers - static map coming from the concurrency manager telling us all the threds and their defferred locks and active locks
        readLockManagersOriginal - static map coming from the concurrency manager telling us all the threads and their read locks
        mapThreadToWaitOnAcquireOriginal - static map of threads that have registered themselves as waiting for some cache key
        mapThreadToWaitOnAcquireInsideWriteLockManagerOriginal - this map relates to the fact that the write lock manager during transaction commits is very illusive. The write lock manger is not allowing itself to get stuck on acquiring any cache key. It uses waits with timings and therefore the locks needed to write and that cannot be obtained are not appearing inside our tracebility maps of the concurrency manager. We needed add the WriteLockManager.THREAD_TO_FAIL_TO_ACQUIRE_CACHE_KEYS but semantically this map is 100 percent the same thing as the mapThreadToWaitOnAcquireOriginal. It still represents a thread wanting to grab a write lock and not managing to get it. Being stuck in that step. Wo we will want to fuse together the (mapThreadToWaitOnAcquireOriginal and the mapThreadToWaitOnAcquireInsideWriteLockManagerOriginal) to make our lives much easier.
        setThreadWaitingToReleaseDeferredLocksOriginal - static map of threads that have stopped going deeped in the recursion of object building and are waiting for the confirmation that some of the objects they needed to build are finished building.
        mapThreadToObjectIdWithWriteLockManagerChangesOriginal - The write lock manager has been tweaked to store information about objects ids that the current thread has in its hands and that will required for write locks to be acquired by a committing thread. This information is especially interesting if any thread participating in a dead lock is getting stuck in the acquisition of write locks as part of the commit process. This information might end up revealing a thread that has done too many changes and is creating a bigger risk fo dead lock. The more resources an individual thread tries to grab the worse it is for the concurrency layer. The size of the change set can be interesting.

      • dumpConcurrencyManagerInformationStep02

        protected void dumpConcurrencyManagerInformationStep02​(ConcurrencyManagerState concurrencyManagerState)
        Dump the server log all of the information that we managed to aggregate about the current state of the concurrency manager.
        Parameters:
        concurrencyManagerState - a snapshot of the current state of the concurrency manager and threads accessing locks.

      • enrichGenerateThreadDumpForCurrentThread

        public String enrichGenerateThreadDumpForCurrentThread()
        This helper API is created due to the problem of the corruption of the eclipselink cache. The idea is to have a tool that allows us to know specifically where the current thread was located when it acquired a READ LOCK.

        Cache corruption problem:
        namely the fact that when dead locks are seen to be taking place some of the threads that seem to be primary culprits of the dead lock are actually idle doing nothing but they have have left the number of readers of the cache corrupted (e.g. typically forever incremnted).

        Returns:
        get the stack trace of the current thread.

      • createInformationAboutAllThreadsWaitingToAcquireReadCacheKeys

        protected String createInformationAboutAllThreadsWaitingToAcquireReadCacheKeys​(Map<Thread,​ConcurrencyManager> mapThreadToWaitOnAcquireReadLockClone,
                                                                               Map<Thread,​String> mapThreadToWaitOnAcquireReadLockMethodNameClone)
        In this page of log dumping information we want to give a summary to the user of threads that appear to be stuck doing an acquire of the cache key.
        Parameters:
        mapThreadToWaitOnAcquireReadLockClone - this a cloned map that has an association between thread and cache keys the thread would like to acquire for READING but cannot because there is some active thread (other than themselves) holding the cache key (e.g. for writing)

      • createInformationAboutAllThreadsWaitingToReleaseDeferredLocks

        protected String createInformationAboutAllThreadsWaitingToReleaseDeferredLocks​(Set<Thread> setThreadWaitingToReleaseDeferredLocksClone)
        Log information about threads not moving forward because they are waiting for the isBuildObjectOnThreadComplete to return true.
        Parameters:
        setThreadWaitingToReleaseDeferredLocksClone - threads waiting for the release deferred lock process to complete.

      • createInformationAboutAllResourcesAcquiredAndDeferredByAllThreads

        protected String createInformationAboutAllResourcesAcquiredAndDeferredByAllThreads​(ConcurrencyManagerState concurrencyManagerState)
        Log information about all threads tracked in the concurrency manager.
        Parameters:
        concurrencyManagerState - and object that represents a snapshot of the current state of the concurrency manager.

      • createInformationAboutAllResourcesAcquiredAndDeferredByThread

        protected String createInformationAboutAllResourcesAcquiredAndDeferredByThread​(ReadLockManager readLockManager,
                                                                               DeferredLockManager lockManager,
                                                                               Set<ConcurrencyManager> waitingOnAcquireCacheKeys,
                                                                               ConcurrencyManager waitingOnAcquireReadCacheKey,
                                                                               boolean threadWaitingToReleaseDeferredLocks,
                                                                               Thread thread,
                                                                               int currentThreadNumber,
                                                                               int totalNumberOfThreads,
                                                                               Set<Object> writeManagerThreadPrimaryKeysWithChangesToBeMerged,
                                                                               String waitingToReleaseDeferredLocksJustification)
        Build a string that tries to describe in as much detail as possible the resources associated to the current thread.
        Parameters:
        readLockManager - the read lock manager for the current thread
        lockManager - the lock manager for the current thread
        waitingOnAcquireCacheKeys - null if the current thread is not waiting to acquire a cache key otherwise the cachekey that the current thread wants to acquire and that is making it block. This field evolved to be a set and not just one cache key because when we needed to tweak the write lock manager code to report about why the write lock manager is stuck we need it to create the map WriteLockManager.THREAD_TO_FAIL_TO_ACQUIRE_CACHE_KEYS whereby during a commit where entiteis are merged into the shared cache a thread might be trying to grab several write locks. so here we have a mix between the concurrency manager cache key a thread currently wants together with cache keys the write lock managed is not managing to grab.
        waitingOnAcquireReadCacheKey - cache key the thread is failing to acquire in the ConcurrencyManager.acquireReadLock()
        threadWaitingToReleaseDeferredLocks - true if the curren thread is now blocked waiting to confirm the locks it deferred have finished building the corresponding objects.
        thread - the thread eing described
        currentThreadNumber - just loop incremented index to help the dump log messages give the feeling of the current thread being described and how many more threads are still to be described
        totalNumberOfThreads - the total number of threads being described in a for loop
        writeManagerThreadPrimaryKeysWithChangesToBeMerged - Null for all threads excep those that are currently about to commit and merge changes to the shared cache. In this case it holds the primary keys of the objects that were changed by the transaction. The write lock manager has been tweaked to store information about objects ids that the current thread has in its hands and that will required for write locks to be acquired by a committing thread. This information is especially interesting if any thread participating in a dead lock is getting stuck in the acquisition of write locks as part of the commit process. This information might end up revealing a thread that has done too many changes and is creating a bigger risk fo dead lock. The more resources an individual thread tries to grab the worse it is for the concurrency layer. The size of the change set can be interesting.
        waitingToReleaseDeferredLocksJustification - when a thread is stuck for more than 500 ms in the release defferred locks algorithm, the concurrency manager starts try to justify why the method isBuildObjectComplete keeps returning false. This information is important whenever the param thread waiting to release deferred locks is true
        Returns:
        a string describing the thread provided. We can see the active locks, deferred locks, read locks etc... as well sa if the thread is waiting to acquire a specific cache key or waiting for build object to complete.

      • cloneMapThreadToWaitOnAcquire

        public static Map<Thread,​ConcurrencyManager> cloneMapThreadToWaitOnAcquire​(Map<Thread,​ConcurrencyManager> mapThreadToWaitOnAcquireOriginal)
        Clone the static map of the concurrency manager that tells us about threads waiting to acquire locks.
        Parameters:
        mapThreadToWaitOnAcquireOriginal - the original map we want to clone
        Returns:
        a cloned map

      • cloneMapThreadToMethodName

        public static Map<Thread,​String> cloneMapThreadToMethodName​(Map<Thread,​String> mapThreadToWaitOnAcquireOriginal)
        Clone the map of the method names that tells us justification where threads acquire locks.
        Parameters:
        mapThreadToWaitOnAcquireOriginal - the original map we want to clone
        Returns:
        a cloned map

      • cloneMapThreadToWaitOnAcquireInsideWriteLockManagerOriginal

        public static Map<Thread,​Set<ConcurrencyManager>> cloneMapThreadToWaitOnAcquireInsideWriteLockManagerOriginal​(Map<Thread,​Set<ConcurrencyManager>> mapThreadToWaitOnAcquireInsideWriteLockManagerOriginal)
        Clone the static map of the concurrency manager that tells us about threads waiting to acquire locks.
        Parameters:
        mapThreadToWaitOnAcquireInsideWriteLockManagerOriginal - the original map we want to clone
        Returns:
        a cloned map

      • enrichMapThreadToWaitOnAcquireInsideWriteLockManagerClone

        public static void enrichMapThreadToWaitOnAcquireInsideWriteLockManagerClone​(Map<Thread,​Set<ConcurrencyManager>> mapThreadToWaitOnAcquireInsideWriteLockManagerClone,
                                                                             Map<Thread,​ConcurrencyManager> mapThreadToWaitOnAcquireClone)
        We have two maps we are using to trace threads that are stuck acquiring locks. One map is found in the concurrency manager the other in the write lock manager. When we start dumping information we only care about working with one and only one map. Therefore we merge the two maps together since semantically they both mean the exact same thing: a thread A wants a cachekey B for writing and is not getting it.
        Parameters:
        mapThreadToWaitOnAcquireInsideWriteLockManagerClone - this is the map we want o enrich
        mapThreadToWaitOnAcquireClone - this is the map whose entries we want to copy into the map to enrich

      • cloneSetThreadsThatAreCurrentlyWaitingToReleaseDeferredLocks

        public static Set<Thread> cloneSetThreadsThatAreCurrentlyWaitingToReleaseDeferredLocks​(Set<Thread> setThreadWaitingToReleaseDeferredLocksOriginal)
        A set of threads that are at the end of object building and are waiting for the deferred locks to be resolved.
        Parameters:
        setThreadWaitingToReleaseDeferredLocksOriginal - the original set of threads that are waiting for deferred locks to be resolved.
        Returns:
        A cloned has set of threads waiting for their deferred locks to be resolved.

      • cloneMapThreadToObjectIdWithWriteLockManagerChanges

        public static Map<Thread,​Set<Object>> cloneMapThreadToObjectIdWithWriteLockManagerChanges​(Map<Thread,​Set<Object>> mapThreadToObjectIdWithWriteLockManagerChangesOriginal)
        Clone the information about threads that are in the write lock manager trying to commit and the object ids they are holding with some arbitrary changes.
        Parameters:
        mapThreadToObjectIdWithWriteLockManagerChangesOriginal - map of thread to the primary keys of of objects changed by a transaction in the commit phase. This is the original map grabbed from the WriteLockManager.
        Returns:
        a cloned map of thread to object id primary keys that a thread committing might have changed.

      • cloneReadLockManagerMap

        public Map<Thread,​ReadLockManager> cloneReadLockManagerMap​(Map<Thread,​ReadLockManager> readLockManagersOriginal)
        To facilitate algorithms that want to dump a snapshot of the current state of the concurrency manager or to start a hunt for dead locks this api faciliates the boostraping logic of such algorithms by giving the algorithm a stable clone of the map of read locks that we know will not change throughout the time the algorithm is running.
        Parameters:
        readLockManagersOriginal - This the original map of read locks referred by the concurrency manager. This is a very bad platform to work with because if for whatever reason not all threads are frozen and some are actualy managing to complete their transactions the contents of this map are systematically changing with threds being added in and removed.
        Returns:
        A clone of the readLockManagersOriginal. Essentially the map instance returned is new and independent and the values ReadLockManager are also clones and independent. The only thing that is not cloned here - whose state could be changing - are the cache key themselves. The cache keys pointed by the vector ReadLockManager.getReadLocks() are the original values. So our clone from the read lock manager is not a perfectly stable clone. It will not be blowing up telling us concurrent access modification when we loop through the vector. But no one can guarnate the number of readers on the cache key stays the same nor that the active thread on a cache key stays the same... Those values can definitely be fluctuating (not ideal ... but it would be quite hard to get better than this).

      • cloneDeferredLockManagerMap

        public Map<Thread,​DeferredLockManager> cloneDeferredLockManagerMap​(Map<Thread,​DeferredLockManager> deferredLockManagersOriginal)
        The exact same thing as the cloneReadLockManagerMap(Map) but the map we are cloning here is the one of threads to deferred locks
        Parameters:
        deferredLockManagersOriginal - the original map taken from the conrruency manager itself
        Returns:
        A clone of that map that is a relatively stable data structure to work with since no new threads will register in or out in the map nor will the DeferredLockManager values be changing. As for the read lock manager we have no assurance as to what is happening with the cache keys themselves refered by the DeferredLockManager values, the cache keys are always changing their metadata as new threads come in to do work or finish doing work. So it is not a perfect snapshot of the state of the system, but it is as close as we can get.

      • cloneDeferredLockManager

        public DeferredLockManager cloneDeferredLockManager​(DeferredLockManager deferredLockManagerOriginal)
        Clone an original DeferredLockManager so that our algorithms of state dump or dead lock search can safely work ina stable model state that is not constantly changing.
        Parameters:
        deferredLockManagerOriginal - an object that is originating from the map of thread to deferred locks from the concurrency manager class. We do not want to be carrying around the original object while try to make a dump/snapshot of the current state of the concurrency manager since these objects are always mutating. Locks are being acquired and released etc... All the tiem. The only objest thta will be stable are those of threads involved ina dead lock. And those are the threads that matter the most to us anyway.
        Returns:
        a cloned deferred lock manager. The newly created deferred lock manager will have its vectors of cache keys holding references the same cache keys as the original object. The cache keys themselves are not cloned. That measn that the DeferredLockManager will be immuatable in terms of its vectors and held references. But the objects it refers to (e.g. cache keys) can be mutating all the time if new readers or active threads arrive.

      • enrichMapOfCacheKeyToDtosExplainingThreadExpectationsOnCacheKeyInfoThreadsStuckOnAcquire

        public void enrichMapOfCacheKeyToDtosExplainingThreadExpectationsOnCacheKeyInfoThreadsStuckOnAcquire​(Map<ConcurrencyManager,​CacheKeyToThreadRelationships> mapOfCacheKeyToDtosExplainingThreadExpectationsOnCacheKey,
                                                                                                     Map<Thread,​Set<ConcurrencyManager>> unifiedMapOfThreadsStuckTryingToAcquireWriteLock)
        Enrich the mapOfCacheKeyToDtosExplainingThreadExpectationsOnCacheKey by setting on the cache keys the threads that are stuck trying to acquire the cache key.
        Parameters:
        mapOfCacheKeyToDtosExplainingThreadExpectationsOnCacheKey - the map we are trying to enirhc with additional information
        unifiedMapOfThreadsStuckTryingToAcquireWriteLock - a map telling us about threads that at a certain point in time were not progressing anywhere because they were waiting to acquire a write lock. These are threads either stuck on the concurrency manager or in the write lock manager during a transaction commmit

      • enrichMapOfCacheKeyToDtosExplainingThreadExpectationsOnCacheKeyInfoThreadsStuckOnAcquireLockForReading

        public void enrichMapOfCacheKeyToDtosExplainingThreadExpectationsOnCacheKeyInfoThreadsStuckOnAcquireLockForReading​(Map<ConcurrencyManager,​CacheKeyToThreadRelationships> mapOfCacheKeyToDtosExplainingThreadExpectationsOnCacheKey,
                                                                                                                   Map<Thread,​ConcurrencyManager> mapThreadToWaitOnAcquireReadLockClone)
        Enrich the mapOfCacheKeyToDtosExplainingThreadExpectationsOnCacheKey by setting on the cache keys the threads that are stuck trying to acquire the cache key with a read lock. These are threads stuck on the ConcurrencyManager.acquireReadLock()
        Parameters:
        mapOfCacheKeyToDtosExplainingThreadExpectationsOnCacheKey - the map we are trying to enirhc with additional information
        mapThreadToWaitOnAcquireReadLockClone - a map telling us about threads that at a certain point in time were not progressing anywhere because they were waiting to acquire a lock.

      • enrichMapOfCacheKeyToDtosExplainingThreadExpectationsOnCacheKeyInfoAboutReadLocks

        public void enrichMapOfCacheKeyToDtosExplainingThreadExpectationsOnCacheKeyInfoAboutReadLocks​(Map<ConcurrencyManager,​CacheKeyToThreadRelationships> mapOfCacheKeyToDtosExplainingThreadExpectationsOnCacheKey,
                                                                                              Map<Thread,​ReadLockManager> readLockManagerMapClone)
        Enrich the mapOfCacheKeyToDtosExplainingThreadExpectationsOnCacheKey based on the read locks
        Parameters:
        mapOfCacheKeyToDtosExplainingThreadExpectationsOnCacheKey - a map whose metadata we need to enrich
        readLockManagerMapClone - map cloned from the original map and that gives us a snapshot of threads that acquired read locks

      • enrichMapOfCacheKeyToDtosExplainingThreadExpectationsOnCacheKeyInfoAboutActiveAndDeferredLocks

        public void enrichMapOfCacheKeyToDtosExplainingThreadExpectationsOnCacheKeyInfoAboutActiveAndDeferredLocks​(Map<ConcurrencyManager,​CacheKeyToThreadRelationships> mapOfCacheKeyToDtosExplainingThreadExpectationsOnCacheKey,
                                                                                                           Map<Thread,​DeferredLockManager> deferredLockManagerMapClone)
        Enrich our map map of cache key to threads having a relationship with that object in regards to active locks on the cache key and deferred locks on the cache key
        Parameters:
        mapOfCacheKeyToDtosExplainingThreadExpectationsOnCacheKey - the map we want to enrich with more information
        deferredLockManagerMapClone - the cloned map with information about threads and their deferred locks.

      • get

        protected CacheKeyToThreadRelationships get​(ConcurrencyManager cacheKey,
                                            Map<ConcurrencyManager,​CacheKeyToThreadRelationships> mapOfCacheKeyToDtosExplainingThreadExpectationsOnCacheKey)
        Helper method to make sure we never get null dto from the mapOfCacheKeyToDtosExplainingThreadExpectationsOnCacheKey.
        Parameters:
        cacheKey - the cache key we are search for
        mapOfCacheKeyToDtosExplainingThreadExpectationsOnCacheKey - the map of cache key to concurrency manager locking metadata
        Returns:
        never returls null. If the cache key is not yet in the map a ney entry is returned.

      • readLockManagerProblem01CreateLogErrorMessageToIndicateThatCurrentThreadHasNullReadLockManagerWhileDecrementingNumberOfReaders

        public String readLockManagerProblem01CreateLogErrorMessageToIndicateThatCurrentThreadHasNullReadLockManagerWhileDecrementingNumberOfReaders​(int currentNumberOfReaders,
                                                                                                                                             int decrementedNumberOfReaders,
                                                                                                                                             ConcurrencyManager cacheKey)
        If when we are decrement the counter of number of readers on a cache key we find ourselves lacking the read lock manager at the time of the decrement we want to log a big fat error on the server log protecting that the current thread is misbehaving.
        Parameters:
        currentNumberOfReaders - the current count of readers on the cache key about to be decremented
        decrementedNumberOfReaders - the number of readers of the cache key if we subtract one reader
        cacheKey - the cache key that is about to suffer a decrement on the number of readers

      • readLockManagerProblem02ReadLockManageHasNoEntriesForThread

        public String readLockManagerProblem02ReadLockManageHasNoEntriesForThread​(ConcurrencyManager cacheKey,
                                                                          long threadId)

      • readLockManagerProblem03ReadLockManageHasNoEntriesForThread

        public String readLockManagerProblem03ReadLockManageHasNoEntriesForThread​(ConcurrencyManager cacheKey,
                                                                          long threadId)

      • createReadLockAcquisitionMetadata

        public ReadLockAcquisitionMetadata createReadLockAcquisitionMetadata​(ConcurrencyManager concurrencyManager)
        The concurrency managers about to acquire a cache key. And since we have been suffering from cache corruption on the acquire read locks we need to collect a lot more information about the time of acquisition of a read lock.
        Parameters:
        concurrencyManager - the cache key we are about to increment and acquire for reading
        Returns:
        object that have all the context information to allow us to know when and where exactly this key acquisition took place.