API Documentation

Redis Collections are composed of only several classes. All items listed below are exposed as public API, so you can (and you should) import them directly from redis_collections package.


base

class redis_collections.base.RedisCollection(redis=None, key=None)[source]

Abstract class providing backend functionality for all the other Redis collections.

__init__(redis=None, key=None)[source]
Parameters:
  • data – Initial data.
  • redis (redis.StrictRedis) – Redis client instance. If not provided, a new Redis connection is created.
  • key (str) – The key at which the collection will be stored in Redis. Collections with the same key point to the same data. If not provided a random key is generated.
redis = None

Redis client instance. StrictRedis object with default connection settings is used if not set by __init__().

key = None

Redis key of the collection.


dicts

The dicts module contains standard collections based on Python dictionaries. Included collections are Dict, Counter, and DefaultDict. Each collection stores its items in a Redis hash structure.

Note

If you need to store mutable values like lists or sets in a collection, be sure to enable writeback. See Synchronization for more information.

When storing numeric types (e.g. float) as keys, be aware that these collections behave slightly differently from standard Python dictionaries. See Hashing dictionary keys and set elements for more information.

class redis_collections.dicts.Dict(*args, **kwargs)[source]

Collection based on the built-in Python dict type. Items are stored in a Redis hash structure. See Python’s dict documentation for usage notes.

The viewitems(), viewkeys(), and viewvalues() methods from Python 2.7’s dictionary type are not implemented.

__init__(*args, **kwargs)[source]

Create a new Dict object.

If the first argument (data) is another mapping type, create the new Dict with its items as the initial data. Or, If the first argument is an iterable of (key, value) pairs, create the new Dict with those items as the initial data.

Unlike Python’s built-in dict type, initial items cannot be set using keyword arguments. Keyword arguments are passed to the RedisCollection constructor.

Parameters:
  • data (iterable or mapping) – Initial data.
  • redis (redis.StrictRedis) – Redis client instance. If not provided, default Redis connection is used.
  • key (str) – Redis key for the collection. Collections with the same key point to the same data. If not provided, a random string is generated.
  • writeback (bool) – If True, keep a local cache of changes for storing modifications to mutable values. Changes will be written to Redis after calling the sync method.
__len__(pipe=None)[source]

Return the number of items in the dictionary.

__iter__(pipe=None)[source]

Return an iterator over the keys of the dictionary.

__contains__(key)[source]

Return True if key is present, else False.

getmany(*keys)[source]

Return a list of values corresponding to the keys in the iterable of keys. If a key is not present in the collection, its corresponding value will be None.

Note

This method is not implemented by standard Python dictionary classes.

__getitem__(key)[source]

Return the item of dictionary with key key. Raises a KeyError if key is not in the map.

If a subclass of Dict defines a method __missing__(), and key is not present, the d[key] operation calls that method with the key key as argument. The d[key] operation then returns or raises whatever is returned or raised by the __missing__(key) call.

__setitem__(key, value)[source]

Set d[key] to value.

__delitem__(key)[source]

Remove d[key] from dictionary. Raises a KeyError() if key is not in the map.

items(pipe=None)[source]

Return a copy of the dictionary’s list of (key, value) pairs.

iteritems(pipe=None)[source]

Return an iterator over the dictionary’s (key, value) pairs.

keys()[source]

Return a copy of the dictionary’s list of keys.

iter()[source]

Return an iterator over the keys of the dictionary. This is a shortcut for iterkeys().

iterkeys()[source]

Return an iterator over the dictionary’s keys.

values()[source]

Return a copy of the dictionary’s list of values.

itervalues()[source]

Return an iterator over the dictionary’s values.

pop(key, default=<missing value>)[source]

If key is in the dictionary, remove it and return its value, else return default. If default is not given and key is not in the dictionary, a KeyError is raised.

popitem()[source]

Remove and return an arbitrary (key, value) pair from the dictionary.

popitem() is useful to destructively iterate over a dictionary, as often used in set algorithms. If the dictionary is empty, calling popitem() raises a KeyError.

setdefault(key, default=None)[source]

If key is in the dictionary, return its value. If not, insert key with a value of default and return default. default defaults to None.

update(other=None, **kwargs)[source]

Update the dictionary with the key/value pairs from other, overwriting existing keys. Return None.

update() accepts either another dictionary object or an iterable of key/value pairs (as tuples or other iterables of length two). If keyword arguments are specified, the dictionary is then updated with those key/value pairs: d.update(red=1, blue=2).

copy(key=None)[source]

Return a new collection with the same items as this one. If key is specified, create the new collection with the given Redis key.

classmethod fromkeys(seq, value=None, **kwargs)[source]

Create a new dictionary with keys from seq and values set to value.

Note

fromkeys() is a class method that returns a new dictionary. It is possible to specify additional keyword arguments to be passed to __init__() of the new object.

scan_items()[source]

Yield each of the (key, value) pairs from the collection, without pulling them all into memory.

Warning

This method is not available on the dictionary collections provided by Python.

This method may return the same (key, value) pair multiple times. See the Redis SCAN documentation for details.

class redis_collections.dicts.Counter(*args, **kwargs)[source]

Collection based on the Python standard library’s collections.Counter type. Items are stored in a Redis hash structure. See Python’s Counter documentation for usage notes.

Counter inherits from Dict, so see its API documentation for information on other methods.

The viewitems(), viewkeys(), and viewvalues() methods from Python 2.7’s Counter type are not implemented.

__init__(*args, **kwargs)[source]

Create a new Counter object.

If the first argument (data) is another mapping type, create the new Counter with the counts of the input items as the initial data. Or, If the first argument is an iterable of (key, value) pairs, create the new Counter with those items as the initial data.

Unlike Python’s standard collections.Counter type, initial items cannot be set using keyword arguments. Keyword arguments are passed to the RedisCollection constructor.

Parameters:
  • data (iterable or mapping) – Initial data.
  • redis (redis.StrictRedis) – Redis client instance. If not provided, default Redis connection is used.
  • key (str) – Redis key for the collection. Collections with the same key point to the same data. If not provided, a random string is generated.
  • writeback (bool) – If True, keep a local cache of changes for storing modifications to mutable values. Changes will be written to Redis after calling the sync method.
most_common(n=None)[source]

Return a list of the n most common elements and their counts from the most common to the least. If n is not specified, most_common() returns all elements in the counter. Elements with equal counts are ordered arbitrarily.

elements(n=None)[source]

Return an iterator over elements repeating each as many times as its count. Elements are returned in arbitrary order. If an element’s count is less than one, elements() will ignore it.

update(other=None, **kwargs)[source]

Elements are counted from an iterable or added-in from another mapping (or counter). Like dict.update() but adds counts instead of replacing them. Also, the iterable is expected to be a sequence of elements, not a sequence of (key, value) pairs.

subtract(other=None, **kwargs)[source]

Elements are subtracted from an iterable or from another mapping (or counter). Like dict.update() but subtracts counts instead of replacing them.

__delitem__(key)[source]

Like dict.__delitem__(), but does not raise KeyError for missing values.

class redis_collections.dicts.DefaultDict(*args, **kwargs)[source]

Collection based on the Python standard library’s collections.defaultdict type. Items are stored in a Redis hash structure. See Python’s defaultdict documentation for usage notes.

DefaultDict inherits from Dict, so see its API documentation for information on other methods.

The viewitems(), viewkeys(), and viewvalues() methods from Python 2.7’s Counter type are not implemented.

__init__(*args, **kwargs)[source]

Create a new DefaultDict object.

The first argument provides the initial value for the default_factory attribute; it defaults to None. All other arguments are passed to the Dict constructor.

Unlike Python’s standard collections.defaultdict type, initial items cannot be set using keyword arguments. Keyword arguments are passed to the RedisCollection constructor via the Dict constructor.

Parameters:
  • default_factory (callable or None) – Used to provide default values for missing keys.
  • data (iterable or mapping) – Initial data.
  • redis (redis.StrictRedis) – Redis client instance. If not provided, default Redis connection is used.
  • key (str) – Redis key for the collection. Collections with the same key point to the same data. If not provided, a random string is generated.
copy(key=None)[source]

Return a new collection with the same items as this one. If key is specified, create the new collection with the given Redis key.


lists

The lists module contains standard collections based on Python lists. Included collections are List and Deque. Each collection stores its values in a Redis list structure.

Note

If you need to store mutable values like lists or sets in a collection, be sure to enable writeback. See Synchronization for more information.

class redis_collections.lists.List(*args, **kwargs)[source]

Collection based on the built-in Python list type. Items are stored in a Redis list structure. See Python’s list documentation for usage notes.

__init__(*args, **kwargs)[source]

Create a new List object.

If the first argument (data) is an iterable object, create the new List with its values as the initial data.

Parameters:
  • data (iterable) – Initial data.
  • redis (redis.StrictRedis) – Redis client instance. If not provided, default Redis connection is used.
  • key (str) – Redis key for the collection. Collections with the same key point to the same data. If not provided, a random string is generated.
  • writeback (bool) – If True keep a local cache of changes for storing modifications to mutable values. Changes will be written to Redis after calling the sync method.
__delitem__(index)[source]

Delete the item at index.

__getitem__(index)[source]

If index is an int, return the value at that index. If index is a slice, return the values from that slice as a list.

__iter__(pipe=None)[source]

Return a list of all values from Redis (overriding those with values from the local cache)

__len__(pipe=None)[source]

Return the length of this collection.

__reversed__()[source]

Return an iterator over this collection’s items in reverse order.

__setitem__(index, value)[source]

If index is an int, set the value for that index to value. If index is a slice, set the values for the indexes associated with that slice to the contents of the iterable value.

append(value)[source]

Insert value at the end of this collection.

clear(pipe=None)[source]

Delete all values from this collection.

copy(key=None)[source]

Return a new collection with the same items as this one. If key is specified, create the new collection with the given Redis key.

count(value)[source]

Return the number of occurences of value.

Note

Counting is implemented in Python.

extend(other)[source]

Adds the values from the iterable other to the end of this collection.

index(value, start=None, stop=None)[source]

Return the index of the first occurence of value. If start or stop are provided, return the smallest index such that s[index] == value and start <= index < stop.

insert(index, value)[source]

Insert value into the collection at index.

pop(index=-1)[source]

Retrieve the value at index, remove it from the collection, and return it.

remove(value)[source]

Remove the first occurence of value.

reverse()[source]

Reverses the items of this collection “in place” (only two values are retrieved from Redis at a time).

sort(key=None, reverse=False)[source]

Sort the items of this collection according to the optional callable key. If reverse is set then the sort order is reversed.

Note

This sort requires all items to be retrieved from Redis and stored in memory.

class redis_collections.lists.Deque(iterable=None, maxlen=None, **kwargs)[source]

Collection based on the Python standard library’s collections.deque type. Items are stored in a Redis hash structure. See Python’s deque documentation for usage notes.

Dequq inherits from List, so see its API documentation for information on other methods.

__init__(iterable=None, maxlen=None, **kwargs)[source]

Create a new Deque object.

If the first argument (iterable) is an iterable object, create the new Deque with its values as the initial data.

If the second argument (maxlen) is an integer, create the Deque with the given maximum length. If the second argument is not given or is None, create the Deque without a maximum length.

If the Deque is full (the number of values stored is equal to the maximum length), adding new items to one side will cause a corresponding number of items to be removed from the other side.

Parameters:
  • iterable – Initial data.
  • maxlen (int) – Maximum size.
  • redis (redis.StrictRedis) – Redis client instance. If not provided, default Redis connection is used.
  • key (bool) – Redis key for the collection. Collections with the same key point to the same data. If not provided, a random string is generated.
  • writeback – If True keep a local cache of changes for storing modifications to mutable values. Changes will be written to Redis after calling the sync method.

Warning

The maxlen of the collection can’t be enforced when multiple processes are accessing its Redis collection.

append(value)[source]

Add value to the right side of the collection.

appendleft(value)[source]

Add value to the left side of the collection.

copy(key=None)[source]

Return a new collection with the same items as this one. If key is specified, create the new collection with the given Redis key.

extend(other)[source]

Extend the right side of the the collection by appending values from the iterable other.

extendleft(other)[source]

Extend the left side of the the collection by appending values from the iterable other. Note that the appends will reverse the order of the given values.

insert(index, value)[source]

Insert value into the collection at index. If the insertion would the collection to grow beyond maxlen, raise IndexError.

pop()[source]

Remove and return an value from the right side of the collection.

popleft()[source]

Remove and return an value from the right side of the collection.

rotate(n=1)[source]

Rotate the deque n steps to the right. If n is negative, rotate to the left.


sets

The sets module contains a standard collection, Set, which is based on Python’s built-in set type. Its elements are stored in a Redis set structure.

class redis_collections.sets.Set(*args, **kwargs)[source]

Collection based on the built-in Python set type. Items are stored in a Redis hash structure. See Python’s set documentation for usage notes.

__init__(*args, **kwargs)[source]

Create a new Set object.

If the first argument (data) is an iterable object, create the new Set with its elements as the initial data.

Parameters:
  • data (iterable) – Initial data.
  • redis (redis.StrictRedis) – Redis client instance. If not provided, default Redis connection is used.
  • key (str) – Redis key for the collection. Collections with the same key point to the same data. If not provided, a random string is generated.
__contains__(value, pipe=None)[source]

Test for membership of value in the set.

__iter__(pipe=None)[source]

Return an iterator over elements of the set.

__len__(pipe=None)[source]

Return cardinality of the set.

add(value)[source]

Add element value to the set.

clear(pipe=None)[source]

Remove all elements from the set.

discard(value)[source]

Remove element value from the set if it is present.

isdisjoint(other)[source]

Return True if the set has no elements in common with other. Sets are disjoint if and only if their intersection is the empty set.

Parameters:other – Any kind of iterable.
Return type:boolean
pop()[source]

Remove and return an arbitrary element from the set. Raises KeyError if the set is empty.

random_sample(k=1)[source]

Return a k length list of unique elements chosen from the Set. Elements are not removed. Similar to random.sample() function from standard library.

Parameters:k – Size of the sample, defaults to 1.
Return type:list

Note

This method is not available on the Python set. When Redis version < 2.6 is being used the whole set is stored in memory and the sample is computed in Python.

remove(value)[source]

Remove element value from the set. Raises KeyError if it is not contained in the set.

scan_elements()[source]

Yield each of the elements from the collection, without pulling them all into memory.

Warning

This method is not available on the set collections provided by Python.

This method may return the element multiple times. See the Redis SCAN documentation for details.

intersection(*others)[source]

Return a new set with elements common to the set and all others.

Parameters:others – Iterables, each one as a single positional argument.
Return type:set

Note

The same behavior as at union() applies.

intersection_update(*others)[source]

Update the set, keeping only elements found in it and all others.

Parameters:others – Iterables, each one as a single positional argument.
Return type:None

Note

The same behavior as at difference_update() applies.

issuperset(other)[source]

Test whether every element in other is in the set.

Parameters:other – Any kind of iterable.
Return type:boolean
issubset(other)[source]

Test whether every element in the set is in other.

Parameters:other – Any kind of iterable.
Return type:boolean
union(*others)[source]

Return a new set with elements from the set and all others.

Parameters:others – Iterables, each one as a single positional argument.
Return type:set

Note

If all others are Set instances, the operation is performed completely in Redis. Otherwise, values are retrieved from Redis and the operation is performed in Python.

update(*others)[source]

Update the set, adding elements from all others.

Parameters:others – Iterables, each one as a single positional argument.
Return type:None

Note

If all others are Set instances, the operation is performed completely in Redis. Otherwise, values are retrieved from Redis and the operation is performed in Python.

difference(*others)[source]

Return a new set with elements in the set that are not in the others.

Parameters:others – Iterables, each one as a single positional argument.
Return type:set

Note

The same behavior as at union() applies.

difference_update(*others)[source]

Update the set, removing elements found in others.

Parameters:others – Iterables, each one as a single positional argument.
Return type:None

Note

The same behavior as at update() applies.

symmetric_difference(other)[source]

Return a new set with elements in either the set or other but not both.

Parameters:other – Any kind of iterable.
Return type:set

Note

The same behavior as at union() applies.

symmetric_difference_update(other)[source]

Update the set, keeping only elements found in either set, but not in both.

Parameters:other – Any kind of iterable.
Return type:None

Note

The same behavior as at update() applies.


sortedsets

The sortedsets module contains a collection, SortedSetCounter, which provides an interface to Redis’s Sorted Set type.

class redis_collections.sortedsets.SortedSetCounter(*args, **kwargs)[source]

SortedSetCounter is a collection based on the Redis Sorted Set type. Instances map a unique set of member objects to floating point score values.

>>> ssc = SortedSetCounter([('earth', 300), ('mercury', 100)])
>>> ssc.set_score('venus', 200)
>>> ssc.get_score('venus')
200.0

When retrieving members they are returned in order by score:

>>> ssc.items()
[('mercury', 100.0), ('venus', 200.0), ('earth', 300.0)]

Ranges of items by rank can be computed and returned efficiently, as can ranges by score:

>>> ssc.items(min_rank=1)  # 'mercury' has rank 0
[('venus', 200.0), ('earth', 300.0)]
>>> ssc.items(min_score=99, max_score=299)
[('mercury', 100.0), ('venus', 200.0)]

Warning

The API for SortedSetCounter does not attempt to match an existing Python collection’s.

  • Unlike Dict or Set objects, equal numeric types are considered distinct when used as members. For example, a collection can contain both 1 and 1.0.
  • Unlike Counter or collections.Counter objects, only float scores can be stored.
__init__(*args, **kwargs)[source]

Create a new SortedSetCounter object.

If the first argument (data) is an iterable object, create the new SortedSetCounter with its elements as the initial data.

Parameters:
  • data (iterable or mapping) – Initial data.
  • redis (redis.StrictRedis) – Redis client instance. If not provided, default Redis connection is used.
  • key (str) – Redis key for the collection. Collections with the same key point to the same data. If not provided, a random string is generated.
__contains__(member, pipe=None)[source]

Return True if member is present, else False.

__iter__(pipe=None)[source]

Return an iterator of (member, score) tuples from the collection.

__len__(pipe=None)[source]

Return the number of members in the collection.

count_between(min_score=None, max_score=None)[source]

Returns the number of members whose score is between min_score and max_score (inclusive).

discard_between(min_rank=None, max_rank=None, min_score=None, max_score=None)[source]

Remove members whose ranking is between min_rank and max_rank OR whose score is between min_score and max_score (both ranges inclusive). If no bounds are specified, no members will be removed.

discard_member(member, pipe=None)[source]

Remove member from the collection, unconditionally.

get_score(member, default=None, pipe=None)[source]

Return the score of member, or default if it is not in the collection.

get_or_set_score(member, default=0)[source]

If member is in the collection, return its value. If not, store it with a score of default and return default. default defaults to 0.

get_rank(member, reverse=False, pipe=None)[source]

Return the rank of member in the collection. By default, the member with the lowest score has rank 0. If reverse is True, the member with the highest score has rank 0.

increment_score(member, amount=1)[source]

Adjust the score of member by amount. If member is not in the collection it will be stored with a score of amount.

items(min_rank=None, max_rank=None, min_score=None, max_score=None, reverse=False, pipe=None)[source]

Return a list of (member, score) tuples whose ranking is between min_rank and max_rank AND whose score is between min_score and max_score (both ranges inclusive). If no bounds are specified, all items will be returned.

scan_items()[source]

Yield each of the (member, score) tuples from the collection, without pulling them all into memory.

Warning

This method may return the same (member, score) tuple multiple times. See the Redis SCAN documentation for details.

set_score(member, score, pipe=None)[source]

Set the score of member to score.

update(other)[source]

Update the collection with items from other. Accepts other SortedSetCounter instances, dictionaries mapping members to numeric scores, or sequences of (member, score) tuples.


syncable

Persistent Python collections that can be written to and read from Redis. The collections are kept in memory, so their operations run as fast as their standard counterparts’.

Use in a with block to automatically sync to Redis after the block executes, or call the sync() method explicitly.

>>> with SyncableDict() as D:
...     D['one'] = 1
...
>>> D  # Contents are available locally and are stored in Redis
{'one': 1}
>>> D['two'] = 2  # Changes are available locally, but not in Redis...
>>> D.sync()  # ...until sync is called.

If you specify a key pointing to an existing collection, its contents will be loaded.

>>> D.key
'f4a78a6faacb4d8e97829f9888ac6740'
>>> E = SyncableDict(key='f4a78a6faacb4d8e97829f9888ac6740')
>>> E
{'one': 1, 'two': 2}
class redis_collections.syncable.SyncableDict(**kwargs)[source]

dict subclass whose contents can be synced to Redis.

See Python’s dict documentation for details.

class redis_collections.syncable.SyncableCounter(**kwargs)[source]

collections.Counter subclass whose contents can be synced to Redis.

See Python’s Counter documentation for details.

class redis_collections.syncable.SyncableDefaultDict(*args, **kwargs)[source]

collections.defaultdict subclass whose contents can be synced to Redis.

See Python’s defaultdict documentation for details.

class redis_collections.syncable.SyncableList(**kwargs)[source]

list subclass whose contents can be synced to Redis.

See Python’s list documentation for details.

class redis_collections.syncable.SyncableDeque(iterable=None, maxlen=None, **kwargs)[source]

deque subclass whose contents can be synced to Redis.

See Python’s deque documentation for details.

class redis_collections.syncable.SyncableSet(**kwargs)[source]

set subclass whose contents can be synced to Redis.

See Python’s set documentation for details.

class redis_collections.syncable.LRUDict(maxsize=None, **kwargs)[source]

dict-like class that stores recently-used items in memory and automatically pushes older items to Redis. Useful for constraining a Python process’s memory footprint when a Redis server with lots of memory is available.

>>> D = LRUDict(maxsize=2)
>>> D['a'] = 1  # 'a' is stored locally
>>> D['b'] = 2  # 'b' is stored locally
>>> D['c'] = 2  # 'a' is pushed to Redis and 'c' is stored locally
>>> assert D['a'] == 1  # 'a' is retrieved from Redis and replaces 'b'
>>> assert D['b'] == 2  # 'b' is retrieved from Redis and replaces 'c'
>>> D['b'] = -2
>>> D.sync()  # All items are copied to Redis
clear()[source]

Removes all items from both the local cache and the collection’s Redis key.

copy(key=None)[source]

Creates another collection with the same items and maxsize with the given key.

classmethod fromkeys(seq, value=None, **kwargs)[source]

Create a new collection with keys from seq and values set to value. The keyword arguments are passed to the persistent Dict.

sync(clear_cache=False)[source]

Copy items from the local cache to the persistent Dict. If clear_cache is True, clear out the local cache after pushing its items to Redis.