dictutils - Mapping types (OMD)

Python has a very powerful mapping type at its core: the dict type. While versatile and featureful, the dict prioritizes simplicity and performance. As a result, it does not retain the order of item insertion [1], nor does it store multiple values per key. It is a fast, unordered 1:1 mapping.

The OrderedMultiDict contrasts to the built-in dict, as a relatively maximalist, ordered 1:n subtype of dict. Virtually every feature of dict has been retooled to be intuitive in the face of this added complexity. Additional methods have been added, such as collections.Counter-like functionality.

A prime advantage of the OrderedMultiDict (OMD) is its non-destructive nature. Data can be added to an OMD without being rearranged or overwritten. The property can allow the developer to work more freely with the data, as well as make more assumptions about where input data will end up in the output, all without any extra work.

One great example of this is the OMD.inverted() method, which returns a new OMD with the values as keys and the keys as values. All the data and the respective order is still represented in the inverted form, all from an operation which would be outright wrong and reckless with a built-in dict or collections.OrderedDict.

The OMD has been performance tuned to be suitable for a wide range of usages, including as a basic unordered MultiDict. Special thanks to Mark Williams for all his help.

[1]

As of 2015, basic dicts on PyPy are ordered, and as of December 2017, basic dicts in CPython 3 are now ordered, as well.

class boltons.dictutils.FrozenDict

An immutable dict subtype that is hashable and can itself be used as a dict key or set entry. What frozenset is to set, FrozenDict is to dict.

There was once an attempt to introduce such a type to the standard library, but it was rejected: PEP 416.

Because FrozenDict is a dict subtype, it automatically works everywhere a dict would, including JSON serialization.

clear(*a, **kw)

raises a TypeError, because FrozenDicts are immutable

classmethod fromkeys(keys, value=None)

Create a new dictionary with keys from iterable and values set to value.

pop(*a, **kw)

raises a TypeError, because FrozenDicts are immutable

popitem(*a, **kw)

raises a TypeError, because FrozenDicts are immutable

setdefault(*a, **kw)

raises a TypeError, because FrozenDicts are immutable

update(*a, **kw)

raises a TypeError, because FrozenDicts are immutable

updated(*a, **kw)

Make a copy and add items from a dictionary or iterable (and/or keyword arguments), overwriting values under an existing key. See dict.update() for more details.

class boltons.dictutils.ManyToMany(items=None)

a dict-like entity that represents a many-to-many relationship between two groups of objects

behaves like a dict-of-tuples; also has .inv which is kept up to date which is a dict-of-tuples in the other direction

also, can be used as a directed graph among hashable python objects

add(key, val)

get(key, default=frozenset({}))

iteritems()

keys()

remove(key, val)

replace(key, newkey)

replace instances of key by newkey

update(iterable)

given an iterable of (key, val), add them all

boltons.dictutils.MultiDict

alias of OrderedMultiDict

boltons.dictutils.OMD

alias of OrderedMultiDict

class boltons.dictutils.OneToOne(*a, **kw)

Implements a one-to-one mapping dictionary. In addition to inheriting from and behaving exactly like the builtin dict, all values are automatically added as keys on a reverse mapping, available as the inv attribute. This arrangement keeps key and value namespaces distinct.

Basic operations are intuitive:

>>> oto = OneToOne({'a': 1, 'b': 2})
>>> print(oto['a'])
1
>>> print(oto.inv[1])
a
>>> len(oto)
2

Overwrites happens in both directions:

>>> oto.inv[1] = 'c'
>>> print(oto.get('a'))
None
>>> len(oto)
2

For a very similar project, with even more one-to-one functionality, check out bidict.

clear() → None.  Remove all items from D.

copy() → a shallow copy of D

inv

pop(k[, d]) → v, remove specified key and return the corresponding value.

If the key is not found, return the default if given; otherwise, raise a KeyError.

popitem()

Remove and return a (key, value) pair as a 2-tuple.

Pairs are returned in LIFO (last-in, first-out) order. Raises KeyError if the dict is empty.

setdefault(key, default=None)

Insert key with a value of default if key is not in the dictionary.

Return the value for key if key is in the dictionary, else default.

classmethod unique(*a, **kw)

This alternate constructor for OneToOne will raise an exception when input values overlap. For instance:

>>> OneToOne.unique({'a': 1, 'b': 1})
Traceback (most recent call last):
...
ValueError: expected unique values, got multiple keys for the following values: ...

This even works across inputs:

>>> a_dict = {'a': 2}
>>> OneToOne.unique(a_dict, b=2)
Traceback (most recent call last):
...
ValueError: expected unique values, got multiple keys for the following values: ...

update([E, ]**F) → None.  Update D from dict/iterable E and F.

If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

class boltons.dictutils.OrderedMultiDict(*a, **kw)

A MultiDict is a dictionary that can have multiple values per key and the OrderedMultiDict (OMD) is a MultiDict that retains original insertion order. Common use cases include:

• handling query strings parsed from URLs

• inverting a dictionary to create a reverse index (values to keys)

• stacking data from multiple dictionaries in a non-destructive way

The OrderedMultiDict constructor is identical to the built-in dict, and overall the API constitutes an intuitive superset of the built-in type:

>>> omd = OrderedMultiDict()
>>> omd['a'] = 1
>>> omd['b'] = 2
>>> omd.add('a', 3)
>>> omd.get('a')
3
>>> omd.getlist('a')
[1, 3]

Some non-dict-like behaviors also make an appearance, such as support for reversed():

>>> list(reversed(omd))
['b', 'a']

Note that unlike some other MultiDicts, this OMD gives precedence to the most recent value added. omd['a'] refers to 3, not 1.

>>> omd
OrderedMultiDict([('a', 1), ('b', 2), ('a', 3)])
>>> omd.poplast('a')
3
>>> omd
OrderedMultiDict([('a', 1), ('b', 2)])
>>> omd.pop('a')
1
>>> omd
OrderedMultiDict([('b', 2)])

If you want a safe-to-modify or flat dictionary, use OrderedMultiDict.todict().

>>> from pprint import pprint as pp  # preserve printed ordering
>>> omd = OrderedMultiDict([('a', 1), ('b', 2), ('a', 3)])
>>> pp(omd.todict())
{'a': 3, 'b': 2}
>>> pp(omd.todict(multi=True))
{'a': [1, 3], 'b': [2]}

With multi=False, items appear with the keys in to original insertion order, alongside the most-recently inserted value for that key.

>>> OrderedMultiDict([('a', 1), ('b', 2), ('a', 3)]).items(multi=False)
[('a', 3), ('b', 2)]

Warning

dict(omd) changed behavior in Python 3.7 due to changes made to support the transition from collections.OrderedDict to the built-in dictionary being ordered. Before 3.7, the result would be a new dictionary, with values that were lists, similar to omd.todict(multi=True) (but only shallow-copy; the lists were direct references to OMD internal structures). From 3.7 onward, the values became singular, like omd.todict(multi=False). For reliable cross-version behavior, just use todict().

add(k, v)

Add a single value v under a key k. Existing values under k are preserved.

addlist(k, v)

Add an iterable of values underneath a specific key, preserving any values already under that key.

>>> omd = OrderedMultiDict([('a', -1)])
>>> omd.addlist('a', range(3))
>>> omd
OrderedMultiDict([('a', -1), ('a', 0), ('a', 1), ('a', 2)])

Called addlist for consistency with getlist(), but tuples and other sequences and iterables work.

clear()

Empty the dictionary.

copy()

Return a shallow copy of the dictionary.

counts()

Returns a mapping from key to number of values inserted under that key. Like collections.Counter, but returns a new OrderedMultiDict.

classmethod fromkeys(keys, default=None)

Create a dictionary from a list of keys, with all the values set to default, or None if default is not set.

get(k, default=None)

Return the value for key k if present in the dictionary, else default. If default is not given, None is returned. This method never raises a KeyError.

To get all values under a key, use OrderedMultiDict.getlist().

getlist(k, default=_MISSING)

Get all values for key k as a list, if k is in the dictionary, else default. The list returned is a copy and can be safely mutated. If default is not given, an empty list is returned.

inverted()

Returns a new OrderedMultiDict with values and keys swapped, like creating dictionary transposition or reverse index. Insertion order is retained and all keys and values are represented in the output.

>>> omd = OMD([(0, 2), (1, 2)])
>>> omd.inverted().getlist(2)
[0, 1]

Inverting twice yields a copy of the original:

>>> omd.inverted().inverted()
OrderedMultiDict([(0, 2), (1, 2)])

items(multi=False)

Returns a list containing the output of iteritems(). See that method’s docs for more details.

iteritems(multi=False)

Iterate over the OMD’s items in insertion order. By default, yields only the most-recently inserted value for each key. Set multi to True to get all inserted items.

iterkeys(multi=False)

Iterate over the OMD’s keys in insertion order. By default, yields each key once, according to the most recent insertion. Set multi to True to get all keys, including duplicates, in insertion order.

itervalues(multi=False)

Iterate over the OMD’s values in insertion order. By default, yields the most-recently inserted value per unique key. Set multi to True to get all values according to insertion order.

keys(multi=False)

Returns a list containing the output of iterkeys(). See that method’s docs for more details.

pop(k, default=_MISSING)

Remove all values under key k, returning the most-recently inserted value. Raises KeyError if the key is not present and no default is provided.

popall(k, default=_MISSING)

Remove all values under key k, returning them in the form of a list. Raises KeyError if the key is not present and no default is provided.

poplast(k=_MISSING, default=_MISSING)

Remove and return the most-recently inserted value under the key k, or the most-recently inserted key if k is not provided. If no values remain under k, it will be removed from the OMD. Raises KeyError if k is not present in the dictionary, or the dictionary is empty.

setdefault(k, default=_MISSING)

If key k is in the dictionary, return its value. If not, insert k with a value of default and return default. default defaults to None. See dict.setdefault() for more information.

sorted(key=None, reverse=False)

Similar to the built-in sorted(), except this method returns a new OrderedMultiDict sorted by the provided key function, optionally reversed.

Parameters:

• key (callable) – A callable to determine the sort key of each element. The callable should expect an item (key-value pair tuple).

• reverse (bool) – Set to True to reverse the ordering.

>>> omd = OrderedMultiDict(zip(range(3), range(3)))
>>> omd.sorted(reverse=True)
OrderedMultiDict([(2, 2), (1, 1), (0, 0)])

Note that the key function receives an item (key-value tuple), so the recommended signature looks like:

>>> omd = OrderedMultiDict(zip('hello', 'world'))
>>> omd.sorted(key=lambda i: i[1])  # i[0] is the key, i[1] is the val
OrderedMultiDict([('o', 'd'), ('l', 'l'), ('e', 'o'), ('l', 'r'), ('h', 'w')])

sortedvalues(key=None, reverse=False)

Returns a copy of the OrderedMultiDict with the same keys in the same order as the original OMD, but the values within each keyspace have been sorted according to key and reverse.

Parameters:

• key (callable) – A single-argument callable to determine the sort key of each element. The callable should expect an item (key-value pair tuple).

• reverse (bool) – Set to True to reverse the ordering.

>>> omd = OrderedMultiDict()
>>> omd.addlist('even', [6, 2])
>>> omd.addlist('odd', [1, 5])
>>> omd.add('even', 4)
>>> omd.add('odd', 3)
>>> somd = omd.sortedvalues()
>>> somd.getlist('even')
[2, 4, 6]
>>> somd.keys(multi=True) == omd.keys(multi=True)
True
>>> omd == somd
False
>>> somd
OrderedMultiDict([('even', 2), ('even', 4), ('odd', 1), ('odd', 3), ('even', 6), ('odd', 5)])

As demonstrated above, contents and key order are retained. Only value order changes.

todict(multi=False)

Gets a basic dict of the items in this dictionary. Keys are the same as the OMD, values are the most recently inserted values for each key.

Setting the multi arg to True is yields the same result as calling dict on the OMD, except that all the value lists are copies that can be safely mutated.

update(E, **F)

Add items from a dictionary or iterable (and/or keyword arguments), overwriting values under an existing key. See dict.update() for more details.

update_extend(E, **F)

Add items from a dictionary, iterable, and/or keyword arguments without overwriting existing items present in the dictionary. Like update(), but adds to existing keys instead of overwriting them.

values(multi=False)

Returns a list containing the output of itervalues(). See that method’s docs for more details.

viewitems() → a set-like object providing a view on OMD's items

viewkeys() → a set-like object providing a view on OMD's keys

viewvalues() → an object providing a view on OMD's values

boltons.dictutils.subdict(d, keep=None, drop=None)

Compute the “subdictionary” of a dict, d.

A subdict is to a dict what a subset is a to set. If A is a subdict of B, that means that all keys of A are present in B.

Returns a new dict with any keys in drop removed, and any keys in keep still present, provided they were in the original dict. keep defaults to all keys, drop defaults to empty, so without one of these arguments, calling this function is equivalent to calling dict().

>>> from pprint import pprint as pp
>>> pp(subdict({'a': 1, 'b': 2}))
{'a': 1, 'b': 2}
>>> subdict({'a': 1, 'b': 2, 'c': 3}, drop=['b', 'c'])
{'a': 1}
>>> pp(subdict({'a': 1, 'b': 2, 'c': 3}, keep=['a', 'c']))
{'a': 1, 'c': 3}