Package com.github.ddth.queue.impl.universal

Class BaseLessLockingUniversalJdbcQueue<T extends BaseUniversalQueueMessage<ID>,ID>

java.lang.Object

com.github.ddth.queue.impl.AbstractQueue<ID,DATA>

com.github.ddth.queue.impl.AbstractEphemeralSupportQueue<ID,DATA>

com.github.ddth.queue.impl.JdbcQueue<ID,byte[]>

com.github.ddth.queue.impl.universal.BaseUniversalJdbcQueue<T,ID>

com.github.ddth.queue.impl.universal.BaseLessLockingUniversalJdbcQueue<T,ID>

Type Parameters:

T -

All Implemented Interfaces:

IQueue<ID,byte[]>, AutoCloseable

Direct Known Subclasses:

AbstractLessLockingUniversalJdbcQueue, AbstractLessLockingUniversalJdbcQueue

public abstract class BaseLessLockingUniversalJdbcQueue<T extends BaseUniversalQueueMessage<ID>,ID>
extends BaseUniversalJdbcQueue<T,ID>

Same as BaseUniversalJdbcQueue, but using a "less-locking" algorithm, requires only one single db table for both queue and ephemeral storage.

Queue db table schema:

• queue_id: <ID>, see IMessage.getId(), COL_QUEUE_ID
• ephemeral_id: bigint, not null, default value = 0, see COL_EPHEMERAL_ID
• msg_org_timestamp: datetime, see IMessage.getTimestamp(), COL_ORG_TIMESTAMP
• msg_timestamp: datetime, see IQueueMessage.getQueueTimestamp(), COL_TIMESTAMP
• msg_num_requeues: int, see IQueueMessage.getNumRequeues(), COL_NUM_REQUEUES
• msg_content: blob, message's content, see COL_CONTENT

Since:

1.0.0

Author:

Thanh Nguyen

Nested Class Summary

Nested classes/interfaces inherited from class com.github.ddth.queue.impl.AbstractQueue

AbstractQueue.PutToQueueCase

Field Summary

Fields

Modifier and Type	Field	Description
static String	COL_CONTENT	Table's column name to store message's content
static String	COL_EPHEMERAL_ID	Table's column name to store ephemeral id
static String	COL_NUM_REQUEUES	Table's column name to store message's number of requeues
static String	COL_ORG_TIMESTAMP	Table's column name to store message's original timestamp
static String	COL_QUEUE_ID	Table's column name to store queue-id
static String	COL_TIMESTAMP	Table's column name to store message's timestamp

Fields inherited from class com.github.ddth.queue.impl.JdbcQueue

DEFAULT_MAX_RETRIES, DEFAULT_TRANX_ISOLATION_LEVEL

Fields inherited from interface com.github.ddth.queue.IQueue

SIZE_NOT_SUPPORTED

Constructor Summary

Constructors

Constructor	Description
BaseLessLockingUniversalJdbcQueue()

Method Summary

Modifier and Type	Method	Description
protected void	_finishWithRetries(Connection conn, IQueueMessage<ID,byte[]> msg, int numRetries, int maxRetries)	Perform "finish" action, retry if deadlock.
protected boolean	_requeueSilentWithRetries(Connection conn, IQueueMessage<ID,byte[]> msg, int numRetries, int maxRetries)	Re-queue (silentlt) a message, retry if deadlock.
protected boolean	_requeueWithRetries(Connection conn, IQueueMessage<ID,byte[]> msg, int numRetries, int maxRetries)	Re-queue a message, retry if deadlock.
protected T	_takeWithRetries(Connection conn, int numRetries, int maxRetries)	Take a message from queue, retry if deadlock.
protected int	ephemeralSize(Connection conn)	Get number of items currently in ephemeral storage.
protected Collection<T>	getOrphanMessagesFromEphemeralStorage(Connection conn, long thresholdTimestampMs)	Primitive operation: Get all orphan messages (messages that were left in ephemeral storage for a long time).
String	getTableNameEphemeral()	Name of database table to store ephemeral messages.
BaseLessLockingUniversalJdbcQueue<T,ID>	init()	Initializing method.
protected T	peekFromQueueStorage(Connection conn)	Primitive operation: Read (but not removed) message from head of queue storage.
protected boolean	putToEphemeralStorage(Connection conn, IQueueMessage<ID,byte[]> _msg)	Primitive operation: Put a message to the ephemeral storage.
protected boolean	putToQueueStorage(Connection conn, IQueueMessage<ID,byte[]> msg)	Primitive operation: Put a message to tail of the queue storage.
protected int	queueSize(Connection conn)	Get number of items currently in queue storage.
protected T	readFromEphemeralStorage(Connection conn, ID id)	Primitive operation: Read (but not removed) a message from the ephemeral storage.
protected boolean	removeFromEphemeralStorage(Connection conn, IQueueMessage<ID,byte[]> msg)	Primitive operation: Remove a message from the queue storage.
protected boolean	removeFromQueueStorage(Connection conn, IQueueMessage<ID,byte[]> msg)	Primitive operation: Remove a message from the queue storage.

Methods inherited from class com.github.ddth.queue.impl.universal.BaseUniversalJdbcQueue

createMessge, ensureMessageType, getFifo, isFifo, markFifo, selectMessages, setFifo, take

Methods inherited from class com.github.ddth.queue.impl.JdbcQueue

_getOrphanMessagesWithRetries, _queueWithRetries, buildJdbcHelper, destroy, doPutToQueue, doSize, ephemeralSize, executeWithRetries, executeWithRetries, executeWithRetries, finish, getDataSource, getJdbcHelper, getMaxRetries, getOrphanMessages, getTableName, getTransactionIsolationLevel, queueSize, readFromEphemeralStorage, setDataSource, setJdbcHelper, setJdbcHelper, setMaxRetries, setTableName, setTableNameEphemeral, setTransactionIsolationLevel

Methods inherited from class com.github.ddth.queue.impl.AbstractEphemeralSupportQueue

getEphemeralDisabled, getEphemeralMaxSize, isEphemeralDisabled, setEphemeralDisabled, setEphemeralMaxSize

Methods inherited from class com.github.ddth.queue.impl.AbstractQueue

close, createMessage, createMessage, createMessage, deserialize, deserialize, getMessageFactory, getQueueName, getSerDeser, queue, requeue, requeueSilent, serialize, setMessageFactory, setQueueName, setSerDeser

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

COL_QUEUE_ID

public static final String COL_QUEUE_ID

Table's column name to store queue-id

See Also:

Constant Field Values

COL_EPHEMERAL_ID

public static final String COL_EPHEMERAL_ID

Table's column name to store ephemeral id

See Also:

Constant Field Values

COL_ORG_TIMESTAMP

public static final String COL_ORG_TIMESTAMP

Table's column name to store message's original timestamp

See Also:

Constant Field Values

COL_TIMESTAMP

public static final String COL_TIMESTAMP

Table's column name to store message's timestamp

See Also:

Constant Field Values

COL_NUM_REQUEUES

public static final String COL_NUM_REQUEUES

Table's column name to store message's number of requeues

See Also:

Constant Field Values

COL_CONTENT

public static final String COL_CONTENT

Table's column name to store message's content

See Also:

Constant Field Values

Constructor Detail

BaseLessLockingUniversalJdbcQueue

public BaseLessLockingUniversalJdbcQueue()

Method Detail

getTableNameEphemeral

public String getTableNameEphemeral()

Name of database table to store ephemeral messages.

Overrides:

getTableNameEphemeral in class JdbcQueue<ID,byte[]>

Returns:

init

public BaseLessLockingUniversalJdbcQueue<T,ID> init()
                                                   throws Exception

Initializing method.

Overrides:

init in class JdbcQueue<ID,byte[]>

Returns:

Throws:

Exception

queueSize

protected int queueSize(Connection conn)

Get number of items currently in queue storage.

Overrides:

queueSize in class JdbcQueue<ID,byte[]>

Returns:

ephemeralSize

protected int ephemeralSize(Connection conn)

Get number of items currently in ephemeral storage.

Overrides:

ephemeralSize in class JdbcQueue<ID,byte[]>

Returns:

peekFromQueueStorage

protected T peekFromQueueStorage(Connection conn)

Primitive operation: Read (but not removed) message from head of queue storage.

Specified by:

peekFromQueueStorage in class JdbcQueue<ID,byte[]>

Returns:

readFromEphemeralStorage

protected T readFromEphemeralStorage(Connection conn,
                                     ID id)

Primitive operation: Read (but not removed) a message from the ephemeral storage.

Specified by:

readFromEphemeralStorage in class JdbcQueue<ID,byte[]>

Returns:

putToEphemeralStorage

protected boolean putToEphemeralStorage(Connection conn,
                                        IQueueMessage<ID,byte[]> _msg)

Primitive operation: Put a message to the ephemeral storage.

Specified by:

putToEphemeralStorage in class JdbcQueue<ID,byte[]>

removeFromQueueStorage

protected boolean removeFromQueueStorage(Connection conn,
                                         IQueueMessage<ID,byte[]> msg)

Primitive operation: Remove a message from the queue storage.

Specified by:

removeFromQueueStorage in class JdbcQueue<ID,byte[]>

Returns:

removeFromEphemeralStorage

protected boolean removeFromEphemeralStorage(Connection conn,
                                             IQueueMessage<ID,byte[]> msg)

Primitive operation: Remove a message from the queue storage.

Specified by:

removeFromEphemeralStorage in class JdbcQueue<ID,byte[]>

Returns:

getOrphanMessagesFromEphemeralStorage

protected Collection<T> getOrphanMessagesFromEphemeralStorage(Connection conn,
                                                              long thresholdTimestampMs)

Primitive operation: Get all orphan messages (messages that were left in ephemeral storage for a long time).

Specified by:

getOrphanMessagesFromEphemeralStorage in class JdbcQueue<ID,byte[]>

thresholdTimestampMs - get all orphan messages that were queued before this timestamp

Returns:

putToQueueStorage

protected boolean putToQueueStorage(Connection conn,
                                    IQueueMessage<ID,byte[]> msg)

Primitive operation: Put a message to tail of the queue storage.

Specified by:

putToQueueStorage in class JdbcQueue<ID,byte[]>

Returns:

_finishWithRetries

protected void _finishWithRetries(Connection conn,
                                  IQueueMessage<ID,byte[]> msg,
                                  int numRetries,
                                  int maxRetries)

Perform "finish" action, retry if deadlock.

Note: http://dev.mysql.com/doc/refman/5.0/en/innodb-deadlocks.html

InnoDB uses automatic row-level locking. You can get deadlocks even in the case of transactions that just insert or delete a single row. That is because these operations are not really "atomic"; they automatically set locks on the (possibly several) index records of the row inserted or deleted.

Note: the supplied queue message is mutable.

Implementation: remove message completely from storage.

Overrides:

_finishWithRetries in class JdbcQueue<ID,byte[]>

_requeueWithRetries

protected boolean _requeueWithRetries(Connection conn,
                                      IQueueMessage<ID,byte[]> msg,
                                      int numRetries,
                                      int maxRetries)

Re-queue a message, retry if deadlock.

Note: http://dev.mysql.com/doc/refman/5.0/en/innodb-deadlocks.html

InnoDB uses automatic row-level locking. You can get deadlocks even in the case of transactions that just insert or delete a single row. That is because these operations are not really "atomic"; they automatically set locks on the (possibly several) index records of the row inserted or deleted.

Implementation:

• Reset value of COL_EPHEMERAL_ID to 0.
• Update value of COL_NUM_REQUEUES and COL_TIMESTAMP.

Overrides:

_requeueWithRetries in class JdbcQueue<ID,byte[]>

Returns:

_requeueSilentWithRetries

protected boolean _requeueSilentWithRetries(Connection conn,
                                            IQueueMessage<ID,byte[]> msg,
                                            int numRetries,
                                            int maxRetries)

Re-queue (silentlt) a message, retry if deadlock.

Note: http://dev.mysql.com/doc/refman/5.0/en/innodb-deadlocks.html

InnoDB uses automatic row-level locking. You can get deadlocks even in the case of transactions that just insert or delete a single row. That is because these operations are not really "atomic"; they automatically set locks on the (possibly several) index records of the row inserted or deleted.

Implementation:

• Reset value of COL_EPHEMERAL_ID to 0.
• Update value of COL_NUM_REQUEUES and COL_TIMESTAMP.

Overrides:

_requeueSilentWithRetries in class JdbcQueue<ID,byte[]>

Returns:

_takeWithRetries

protected T _takeWithRetries(Connection conn,
                             int numRetries,
                             int maxRetries)

Take a message from queue, retry if deadlock.

Note: http://dev.mysql.com/doc/refman/5.0/en/innodb-deadlocks.html

InnoDB uses automatic row-level locking. You can get deadlocks even in the case of transactions that just insert or delete a single row. That is because these operations are not really "atomic"; they automatically set locks on the (possibly several) index records of the row inserted or deleted.

Implementation:

• Get the first available queue message.
• Generate a unique-id and assign to the message's COL_EPHEMERAL_ID.
• Return the message object.

Overrides:

_takeWithRetries in class JdbcQueue<ID,byte[]>

Returns: