If you can't read please download the document
Upload
philip-norton
View
10.263
Download
1
Embed Size (px)
Citation preview
Drupal 7Queues
Drupal Queues
New to Drupal 7
First in first out data structure
Used internally by Drupal
Uses OO principles
Fully customisable
Why Use Queues?
Batch processing of large amounts of data/items
Delay processing of complex calculations so system load is more stable
Sequential processing of items
Preventing API service black listing
Drupal 7 Implementations
Batch API
Cron
Aggregator Module
Update Module
Drupal Queues
Found in the file:
/modules/system/system.queue.inc
Classes are used to wrap queue functionality
Reliable vs Non-reliable
Reliable
Generally kept in database table
Every item will be executed at least once
Items exist over several requests
If the request fails then queue remains intact
Non-reliable
Generally kept in memory only
All items might exist in a single request
No guarantee that all queue items will be executed
No guarantee that all items will be executed in order
If request fails then queue might be lost
Drupal Queue Classes
DrupalQueue
Single static method get()
DrupalQueue::get('my_queue');
Returns a queue object of a given name and a given type (reliable or non-reliable)
Force reliable queue by passing TRUE as second parameter
Default returned class is SystemQueue
Essentially a queue object factory
SystemQueue
Example of a reliable queue class
Implements DrupalReliableQueueInterface
Uses the database table queue to store and retrieve the queue
Items are 'leased' to ensure no two processes get the same queue item
Default class for new queues
queue Table
Created at Drupal install
MemoryQueue
Example of an non-reliable queue class
All queue items are stored in memory
$queue parameter in class contains the queue
Also implements item leasing
System Variables
Three system variables are used by DrupalQueue to decide what sort of object to return
'queue_class_' . $nameDefault is NULL
queue_default_classDefault is SystemQueue
queue_default_reliable_classDefault is SystemQueue
Using Drupal Queues
Create A Queue
$queue = DrupalQueue::get('my_queue', TRUE);
$item = array('dataitem1' => 'something','int' => 123);
$queue->createItem($item);
echo $queue->numberOfItems(); // 1
Create A MemoryQueue
variable_set('queue_default_class', 'MemoryQueue');
$queue = DrupalQueue::get('my_queue');
$item = array('dataitem1' => 'something','int' => 123);
$queue->createItem($item);
echo $queue->numberOfItems(); // 1
Get Item From Queue
$queue = DrupalQueue::get('my_queue');
$got_item = $queue->claimItem();
echo $got_item->data['dataitem1'];
Retrieved Item Structure
stdClass Object( [data] => Array ( [dataitem1] => something [qwe] => 123 )
[item_id] => 89)
Change Lease Length
Can pass the lease time (in seconds) to claimItem()
$got_item = $queue->claimItem(100);
Release Or Delete
releaseItem() resets the lease time
$queue->releaseItem($got_item);
deleteItem() removes item from queue
$queue->deleteItem($got_item);
Customizing
Create custom queue class
Create variable called queue_class_ . $name
Value is the class name of your queue class
Call DrupalQueue::get() with the variable $name as a string
Customizing
Reliable class MUST implement DrupalReliableQueueInterface
Can also extend SystemQueue
All Queue classes should at least implement DrupalQueueInterface
Using Custom Classes
variable_set('queue_class_mycustom', 'MyCustomQueueClass');
$queue = DrupalQueue::get('mycustom');
Stack Class
Last in first out implementation
class Stack extends SystemQueue {
public function claimItem($lease_time = 30) { while (TRUE) { $item = db_query_range('SELECT data, item_id FROM {queue} q WHERE expire = 0 AND name = :name ORDER BY created DESC', 0, 1, array(':name' => $this->name))->fetchObject();// ...
EventQueue Class
Required as part of the NWDUG website
Any items added would not be available for 45 minutes to give a period of grace after creating an event node
class EventQueue extends SystemQueue {
public function claimItem($lease_time = 30) { while (TRUE) { $item = db_query_range('SELECT data, item_id FROM {queue} q WHERE expire = 0 AND name = :name AND created >= UNIX_TIMESTAMP(DATE_ADD(NOW(), INTERVAL 2700 SECOND)) ORDER BY created ASC', 0, 1, array(':name' => $this->name))->fetchObject();
WatchdogSystemQueue
Extends SystemQueue and creates a log every time anything is done
Maintains SystemQueue functionality
WatchdogSystemQueue
class WatchdogSystemQueue extends SystemQueue {
public function __construct($name) { watchdog('queue', '%name queue created', array('%name' => $name)); parent::__construct($name); }
public function createItem($data) { watchdog('queue', 'Item created : %data', array('%data' => print_r($data, TRUE))); parent::createItem($data); }
public function claimItem($lease_time = 3600) { $return_value = parent::claimItem($lease_time); watchdog('queue', 'Item claimed %item (lease time = %lease)', array('%item' => print_r($return_value, TRUE),'%lease' => $lease_time)); return $return_value; }//....
RandomMemoryQueue
Extends the MemoryQueue class
Items are added in order but are retrieved in random order
RandomMemoryQueue
class RandomMemoryQueue extends MemoryQueue {
public function claimItem($lease_time = 30) { $available_items = array(); // Extract the remining available items foreach ($this->queue as $key => $item) { if ($item->expire == 0) { $available_items[] = $item; } } // Randomly select one (if available) if (count($available_items) > 0) { $queue_length = count($this->queue); $rand_item = rand(0, $queue_length - 1); $item = $available_items[$rand_item]; $item->expire = time() + $lease_time; return $item; }
return FALSE; }}
Creating And Destroying
DrupalQueueInterface has two methods available for creating and destroying the queue
createQueue()Should be called within an install hook
deleteQueue()Should be called within an uninstall hook
Tips
Unless you really need to rewrite the entire class it is best to extend SystemQueue or MemoryQueue
For open source projects try to keep the same retrieved item structure as the system queues
No checks for unique items in default system queues
Resources
Drupal Queues APIhttp://api.drupal.org/api/drupal/modules--system--system.queue.inc/group/queue/7
http://bit.ly/kuphnc
Queue UI Modulehttp://drupal.org/project/queue_ui
Source code is well documented/modules/system/system.queue.inc
Full write up of this talk on #! codehttp://www.hashbangcode.com/
Questions?
$queue = DrupalQueue::get('questions');
$question = $queue->claimItem();
echo $question->data['question'];
Bloghttp:///www.norton42.org.uk/
Twitter@philipnorton42
#! codehttp://www.hashbangcode.com/
#! code on Twitter@hashbangcode
Philip Norton
Philip Nortonwww.hashbangcode.com
#!