iterutils - itertools improvements

itertools is full of great examples of Python generator usage. However, there are still some critical gaps. iterutils fills many of those gaps with featureful, tested, and Pythonic solutions.

Many of the functions below have two versions, one which returns an iterator (denoted by the *_iter naming pattern), and a shorter-named convenience form that returns a list. Some of the following are based on examples in itertools docs.

Iteration

These are generators and convenient list-producing counterparts comprising several common patterns of iteration not present in the standard library.

boltons.iterutils.chunked(src, size, count=None, **kw)

Returns a list of count chunks, each with size elements, generated from iterable src. If src is not evenly divisible by size, the final chunk will have fewer than size elements. Provide the fill keyword argument to provide a pad value and enable padding, otherwise no padding will take place.

>>> chunked(range(10), 3)
[[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]
>>> chunked(range(10), 3, fill=None)
[[0, 1, 2], [3, 4, 5], [6, 7, 8], [9, None, None]]
>>> chunked(range(10), 3, count=2)
[[0, 1, 2], [3, 4, 5]]

See chunked_iter() for more info.

boltons.iterutils.chunked_iter(src, size, **kw)

Generates size-sized chunks from src iterable. Unless the optional fill keyword argument is provided, iterables not evenly divisible by size will have a final chunk that is smaller than size.

>>> list(chunked_iter(range(10), 3))
[[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]
>>> list(chunked_iter(range(10), 3, fill=None))
[[0, 1, 2], [3, 4, 5], [6, 7, 8], [9, None, None]]

Note that fill=None in fact uses None as the fill value.

boltons.iterutils.chunk_ranges(input_size, chunk_size, input_offset=0, overlap_size=0, align=False)

Generates chunk_size-sized chunk ranges for an input with length input_size. Optionally, a start of the input can be set via input_offset, and and overlap between the chunks may be specified via overlap_size. Also, if align is set to True, any items with i % (chunk_size-overlap_size) == 0 are always at the beginning of the chunk.

Returns an iterator of (start, end) tuples, one tuple per chunk.

>>> list(chunk_ranges(input_offset=10, input_size=10, chunk_size=5))
[(10, 15), (15, 20)]
>>> list(chunk_ranges(input_offset=10, input_size=10, chunk_size=5, overlap_size=1))
[(10, 15), (14, 19), (18, 20)]
>>> list(chunk_ranges(input_offset=10, input_size=10, chunk_size=5, overlap_size=2))
[(10, 15), (13, 18), (16, 20)]

>>> list(chunk_ranges(input_offset=4, input_size=15, chunk_size=5, align=False))
[(4, 9), (9, 14), (14, 19)]
>>> list(chunk_ranges(input_offset=4, input_size=15, chunk_size=5, align=True))
[(4, 5), (5, 10), (10, 15), (15, 19)]

>>> list(chunk_ranges(input_offset=2, input_size=15, chunk_size=5, overlap_size=1, align=False))
[(2, 7), (6, 11), (10, 15), (14, 17)]
>>> list(chunk_ranges(input_offset=2, input_size=15, chunk_size=5, overlap_size=1, align=True))
[(2, 5), (4, 9), (8, 13), (12, 17)]
>>> list(chunk_ranges(input_offset=3, input_size=15, chunk_size=5, overlap_size=1, align=True))
[(3, 5), (4, 9), (8, 13), (12, 17), (16, 18)]

boltons.iterutils.pairwise(src, end=Sentinel('_UNSET'))

Convenience function for calling windowed() on src, with size set to 2.

>>> pairwise(range(5))
[(0, 1), (1, 2), (2, 3), (3, 4)]
>>> pairwise([])
[]

Unless end is set, the number of pairs is always one less than the number of elements in the iterable passed in, except on an empty input, which will return an empty list.

With end set, a number of pairs equal to the length of src is returned, with the last item of the last pair being equal to end.

>>> list(pairwise(range(3), end=None))
[(0, 1), (1, 2), (2, None)]

This way, end values can be useful as sentinels to signal the end of the iterable.

boltons.iterutils.pairwise_iter(src, end=Sentinel('_UNSET'))

Convenience function for calling windowed_iter() on src, with size set to 2.

>>> list(pairwise_iter(range(5)))
[(0, 1), (1, 2), (2, 3), (3, 4)]
>>> list(pairwise_iter([]))
[]

Unless end is set, the number of pairs is always one less than the number of elements in the iterable passed in, or zero, when src is empty.

With end set, a number of pairs equal to the length of src is returned, with the last item of the last pair being equal to end.

>>> list(pairwise_iter(range(3), end=None))
[(0, 1), (1, 2), (2, None)]

This way, end values can be useful as sentinels to signal the end of the iterable. For infinite iterators, setting end has no effect.

boltons.iterutils.windowed(src, size, fill=Sentinel('_UNSET'))

Returns tuples with exactly length size. If fill is unset and the iterable is too short to make a window of length size, no tuples are returned. See windowed_iter() for more.

boltons.iterutils.windowed_iter(src, size, fill=Sentinel('_UNSET'))

Returns tuples with length size which represent a sliding window over iterable src.

>>> list(windowed_iter(range(7), 3))
[(0, 1, 2), (1, 2, 3), (2, 3, 4), (3, 4, 5), (4, 5, 6)]

If fill is unset, and the iterable is too short to make a window of length size, then no window tuples are returned.

>>> list(windowed_iter(range(3), 5))
[]

With fill set, the iterator always yields a number of windows equal to the length of the src iterable.

>>> windowed(range(4), 3, fill=None)
[(0, 1, 2), (1, 2, 3), (2, 3, None), (3, None, None)]

This way, fill values can be useful to signal the end of the iterable. For infinite iterators, setting fill has no effect.

boltons.iterutils.unique(src, key=None)

unique() returns a list of unique values, as determined by key, in the order they first appeared in the input iterable, src.

>>> ones_n_zeros = '11010110001010010101010'
>>> ''.join(unique(ones_n_zeros))
'10'

See unique_iter() docs for more details.

boltons.iterutils.unique_iter(src, key=None)

Yield unique elements from the iterable, src, based on key, in the order in which they first appeared in src.

>>> repetitious = [1, 2, 3] * 10
>>> list(unique_iter(repetitious))
[1, 2, 3]

By default, key is the object itself, but key can either be a callable or, for convenience, a string name of the attribute on which to uniqueify objects, falling back on identity when the attribute is not present.

>>> pleasantries = ['hi', 'hello', 'ok', 'bye', 'yes']
>>> list(unique_iter(pleasantries, key=lambda x: len(x)))
['hi', 'hello', 'bye']

boltons.iterutils.redundant(src, key=None, groups=False)

The complement of unique().

By default returns non-unique/duplicate values as a list of the first redundant value in src. Pass groups=True to get groups of all values with redundancies, ordered by position of the first redundant value. This is useful in conjunction with some normalizing key function.

>>> redundant([1, 2, 3, 4])
[]
>>> redundant([1, 2, 3, 2, 3, 3, 4])
[2, 3]
>>> redundant([1, 2, 3, 2, 3, 3, 4], groups=True)
[[2, 2], [3, 3, 3]]

An example using a key function to do case-insensitive redundancy detection.

>>> redundant(['hi', 'Hi', 'HI', 'hello'], key=str.lower)
['Hi']
>>> redundant(['hi', 'Hi', 'HI', 'hello'], groups=True, key=str.lower)
[['hi', 'Hi', 'HI']]

key should also be used when the values in src are not hashable.

Note

This output of this function is designed for reporting duplicates in contexts when a unique input is desired. Due to the grouped return type, there is no streaming equivalent of this function for the time being.

Stripping and splitting

A couple of str-inspired mechanics that have come in handy on iterables, too:

boltons.iterutils.split(src, sep=None, maxsplit=None)

Splits an iterable based on a separator. Like str.split(), but for all iterables. Returns a list of lists.

>>> split(['hi', 'hello', None, None, 'sup', None, 'soap', None])
[['hi', 'hello'], ['sup'], ['soap']]

See split_iter() docs for more info.

boltons.iterutils.split_iter(src, sep=None, maxsplit=None)

Splits an iterable based on a separator, sep, a max of maxsplit times (no max by default). sep can be:

• a single value

• an iterable of separators

• a single-argument callable that returns True when a separator is encountered

split_iter() yields lists of non-separator values. A separator will never appear in the output.

>>> list(split_iter(['hi', 'hello', None, None, 'sup', None, 'soap', None]))
[['hi', 'hello'], ['sup'], ['soap']]

Note that split_iter is based on str.split(), so if sep is None, split() groups separators. If empty lists are desired between two contiguous None values, simply use sep=[None]:

>>> list(split_iter(['hi', 'hello', None, None, 'sup', None]))
[['hi', 'hello'], ['sup']]
>>> list(split_iter(['hi', 'hello', None, None, 'sup', None], sep=[None]))
[['hi', 'hello'], [], ['sup'], []]

Using a callable separator:

>>> falsy_sep = lambda x: not x
>>> list(split_iter(['hi', 'hello', None, '', 'sup', False], falsy_sep))
[['hi', 'hello'], [], ['sup'], []]

See split() for a list-returning version.

boltons.iterutils.strip(iterable, strip_value=None)

Strips values from the beginning and end of an iterable. Stripped items will match the value of the argument strip_value. Functionality is analogous to that of the method str.strip. Returns a list.

>>> strip(['Fu', 'Foo', 'Bar', 'Bam', 'Fu'], 'Fu')
['Foo', 'Bar', 'Bam']

boltons.iterutils.strip_iter(iterable, strip_value=None)

Strips values from the beginning and end of an iterable. Stripped items will match the value of the argument strip_value. Functionality is analogous to that of the method str.strip. Returns a generator.

>>> list(strip_iter(['Fu', 'Foo', 'Bar', 'Bam', 'Fu'], 'Fu'))
['Foo', 'Bar', 'Bam']

boltons.iterutils.lstrip(iterable, strip_value=None)

Strips values from the beginning of an iterable. Stripped items will match the value of the argument strip_value. Functionality is analogous to that of the method str.lstrip. Returns a list.

>>> lstrip(['Foo', 'Bar', 'Bam'], 'Foo')
['Bar', 'Bam']

boltons.iterutils.lstrip_iter(iterable, strip_value=None)

Strips values from the beginning of an iterable. Stripped items will match the value of the argument strip_value. Functionality is analogous to that of the method str.lstrip. Returns a generator.

>>> list(lstrip_iter(['Foo', 'Bar', 'Bam'], 'Foo'))
['Bar', 'Bam']

boltons.iterutils.rstrip(iterable, strip_value=None)

Strips values from the end of an iterable. Stripped items will match the value of the argument strip_value. Functionality is analogous to that of the method str.rstrip. Returns a list.

>>> rstrip(['Foo', 'Bar', 'Bam'], 'Bam')
['Foo', 'Bar']

boltons.iterutils.rstrip_iter(iterable, strip_value=None)

Strips values from the end of an iterable. Stripped items will match the value of the argument strip_value. Functionality is analogous to that of the method str.rstrip. Returns a generator.

>>> list(rstrip_iter(['Foo', 'Bar', 'Bam'], 'Bam'))
['Foo', 'Bar']

Nested

Nested data structures are common. Yet virtually all of Python’s compact iteration tools work with flat data: list comprehensions, map/filter, generator expressions, itertools, even other iterutils.

The functions below make working with nested iterables and other containers as succinct and powerful as Python itself.

boltons.iterutils.remap(root, visit=<function default_visit>, enter=<function default_enter>, exit=<function default_exit>, **kwargs)

The remap (“recursive map”) function is used to traverse and transform nested structures. Lists, tuples, sets, and dictionaries are just a few of the data structures nested into heterogeneous tree-like structures that are so common in programming. Unfortunately, Python’s built-in ways to manipulate collections are almost all flat. List comprehensions may be fast and succinct, but they do not recurse, making it tedious to apply quick changes or complex transforms to real-world data.

remap goes where list comprehensions cannot.

Here’s an example of removing all Nones from some data:

>>> from pprint import pprint
>>> reviews = {'Star Trek': {'TNG': 10, 'DS9': 8.5, 'ENT': None},
...            'Babylon 5': 6, 'Dr. Who': None}
>>> pprint(remap(reviews, lambda p, k, v: v is not None))
{'Babylon 5': 6, 'Star Trek': {'DS9': 8.5, 'TNG': 10}}

Notice how both Nones have been removed despite the nesting in the dictionary. Not bad for a one-liner, and that’s just the beginning. See this remap cookbook for more delicious recipes.

remap takes four main arguments: the object to traverse and three optional callables which determine how the remapped object will be created.

Parameters:

• root – The target object to traverse. By default, remap supports iterables like list, tuple, dict, and set, but any object traversable by enter will work.

• visit (callable) –

  This function is called on every item in root. It must accept three positional arguments, path, key, and value. path is simply a tuple of parents’ keys. visit should return the new key-value pair. It may also return True as shorthand to keep the old item unmodified, or False to drop the item from the new structure. visit is called after enter, on the new parent.

  The visit function is called for every item in root, including duplicate items. For traversable values, it is called on the new parent object, after all its children have been visited. The default visit behavior simply returns the key-value pair unmodified.

• enter (callable) –

  This function controls which items in root are traversed. It accepts the same arguments as visit: the path, the key, and the value of the current item. It returns a pair of the blank new parent, and an iterator over the items which should be visited. If False is returned instead of an iterator, the value will not be traversed.

  The enter function is only called once per unique value. The default enter behavior support mappings, sequences, and sets. Strings and all other iterables will not be traversed.

• exit (callable) –

  This function determines how to handle items once they have been visited. It gets the same three arguments as the other functions – path, key, value – plus two more: the blank new parent object returned from enter, and a list of the new items, as remapped by visit.

  Like enter, the exit function is only called once per unique value. The default exit behavior is to simply add all new items to the new parent, e.g., using list.extend() and dict.update() to add to the new parent. Immutable objects, such as a tuple or namedtuple, must be recreated from scratch, but use the same type as the new parent passed back from the enter function.

• reraise_visit (bool) – A pragmatic convenience for the visit callable. When set to False, remap ignores any errors raised by the visit callback. Items causing exceptions are kept. See examples for more details.

• trace (bool) – Pass trace=True to print out the entire traversal. Or pass a tuple of 'visit', 'enter', or 'exit' to print only the selected events.

remap is designed to cover the majority of cases with just the visit callable. While passing in multiple callables is very empowering, remap is designed so very few cases should require passing more than one function.

When passing enter and exit, it’s common and easiest to build on the default behavior. Simply add from boltons.iterutils import default_enter (or default_exit), and have your enter/exit function call the default behavior before or after your custom logic. See this example.

Duplicate and self-referential objects (aka reference loops) are automatically handled internally, as shown here.

boltons.iterutils.get_path(root, path, default=Sentinel('_UNSET'))

Retrieve a value from a nested object via a tuple representing the lookup path.

>>> root = {'a': {'b': {'c': [[1], [2], [3]]}}}
>>> get_path(root, ('a', 'b', 'c', 2, 0))
3

The path tuple format is intentionally consistent with that of remap(), but a single dotted string can also be passed.

One of get_path’s chief aims is improved error messaging. EAFP is great, but the error messages are not.

For instance, root['a']['b']['c'][2][1] gives back IndexError: list index out of range

What went out of range where? get_path currently raises PathAccessError: could not access 2 from path ('a', 'b', 'c', 2, 1), got error: IndexError('list index out of range',), a subclass of IndexError and KeyError.

You can also pass a default that covers the entire operation, should the lookup fail at any level.

Parameters:

• root – The target nesting of dictionaries, lists, or other objects supporting __getitem__.

• path (tuple) – A sequence of strings and integers to be successively looked up within root. A dot-separated (a.b) string may also be passed.

• default – The value to be returned should any PathAccessError exceptions be raised.

boltons.iterutils.research(root, query=<function <lambda>>, reraise=False, enter=<function default_enter>)

The research() function uses remap() to recurse over any data nested in root, and find values which match a given criterion, specified by the query callable.

Results are returned as a list of (path, value) pairs. The paths are tuples in the same format accepted by get_path(). This can be useful for comparing values nested in two or more different structures.

Here’s a simple example that finds all integers:

>>> root = {'a': {'b': 1, 'c': (2, 'd', 3)}, 'e': None}
>>> res = research(root, query=lambda p, k, v: isinstance(v, int))
>>> print(sorted(res))
[(('a', 'b'), 1), (('a', 'c', 0), 2), (('a', 'c', 2), 3)]

Note how query follows the same, familiar path, key, value signature as the visit and enter functions on remap(), and returns a bool.

Parameters:

• root – The target object to search. Supports the same types of objects as remap(), including list, tuple, dict, and set.

• query (callable) – The function called on every object to determine whether to include it in the search results. The callable must accept three arguments, path, key, and value, commonly abbreviated p, k, and v, same as enter and visit from remap().

• reraise (bool) – Whether to reraise exceptions raised by query or to simply drop the result that caused the error.

With research() it’s easy to inspect the details of a data structure, like finding values that are at a certain depth (using len(p)) and much more. If more advanced functionality is needed, check out the code and make your own remap() wrapper, and consider submitting a patch!

boltons.iterutils.flatten(iterable)

flatten() returns a collapsed list of all the elements from iterable while collapsing any nested iterables.

>>> nested = [[1, 2], [[3], [4, 5]]]
>>> flatten(nested)
[1, 2, 3, 4, 5]

boltons.iterutils.flatten_iter(iterable)

flatten_iter() yields all the elements from iterable while collapsing any nested iterables.

>>> nested = [[1, 2], [[3], [4, 5]]]
>>> list(flatten_iter(nested))
[1, 2, 3, 4, 5]

Numeric

Number sequences are an obvious target of Python iteration, such as the built-in range(), and itertools.count(). Like the Iteration members above, these return iterators and lists, but take numeric inputs instead of iterables.

boltons.iterutils.backoff(start, stop, count=None, factor=2.0, jitter=False)

Returns a list of geometrically-increasing floating-point numbers, suitable for usage with exponential backoff. Exactly like backoff_iter(), but without the 'repeat' option for count. See backoff_iter() for more details.

>>> backoff(1, 10)
[1.0, 2.0, 4.0, 8.0, 10.0]

boltons.iterutils.backoff_iter(start, stop, count=None, factor=2.0, jitter=False)

Generates a sequence of geometrically-increasing floats, suitable for usage with exponential backoff. Starts with start, increasing by factor until stop is reached, optionally stopping iteration once count numbers are yielded. factor defaults to 2. In general retrying with properly-configured backoff creates a better-behaved component for a larger service ecosystem.

>>> list(backoff_iter(1.0, 10.0, count=5))
[1.0, 2.0, 4.0, 8.0, 10.0]
>>> list(backoff_iter(1.0, 10.0, count=8))
[1.0, 2.0, 4.0, 8.0, 10.0, 10.0, 10.0, 10.0]
>>> list(backoff_iter(0.25, 100.0, factor=10))
[0.25, 2.5, 25.0, 100.0]

A simplified usage example:

for timeout in backoff_iter(0.25, 5.0):
    try:
        res = network_call()
        break
    except Exception as e:
        log(e)
        time.sleep(timeout)

An enhancement for large-scale systems would be to add variation, or jitter, to timeout values. This is done to avoid a thundering herd on the receiving end of the network call.

Finally, for count, the special value 'repeat' can be passed to continue yielding indefinitely.

Parameters:

• start (float) – Positive number for baseline.

• stop (float) – Positive number for maximum.

• count (int) – Number of steps before stopping iteration. Defaults to the number of steps between start and stop. Pass the string, ‘repeat’, to continue iteration indefinitely.

• factor (float) – Rate of exponential increase. Defaults to 2.0, e.g., [1, 2, 4, 8, 16].

• jitter (float) – A factor between -1.0 and 1.0, used to uniformly randomize and thus spread out timeouts in a distributed system, avoiding rhythm effects. Positive values use the base backoff curve as a maximum, negative values use the curve as a minimum. Set to 1.0 or True for a jitter approximating Ethernet’s time-tested backoff solution. Defaults to False.

boltons.iterutils.frange(stop, start=None, step=1.0)

A range() clone for float-based ranges.

>>> frange(5)
[0.0, 1.0, 2.0, 3.0, 4.0]
>>> frange(6, step=1.25)
[0.0, 1.25, 2.5, 3.75, 5.0]
>>> frange(100.5, 101.5, 0.25)
[100.5, 100.75, 101.0, 101.25]
>>> frange(5, 0)
[]
>>> frange(5, 0, step=-1.25)
[5.0, 3.75, 2.5, 1.25]

boltons.iterutils.xfrange(stop, start=None, step=1.0)

Same as frange(), but generator-based instead of returning a list.

>>> tuple(xfrange(1, 3, step=0.75))
(1.0, 1.75, 2.5)

See frange() for more details.

Categorization

These functions operate on iterables, dividing into groups based on a given condition.

boltons.iterutils.bucketize(src, key=<class 'bool'>, value_transform=None, key_filter=None)

Group values in the src iterable by the value returned by key.

>>> bucketize(range(5))
{False: [0], True: [1, 2, 3, 4]}
>>> is_odd = lambda x: x % 2 == 1
>>> bucketize(range(5), is_odd)
{False: [0, 2, 4], True: [1, 3]}

key is bool by default, but can either be a callable or a string or a list if it is a string, it is the name of the attribute on which to bucketize objects.

>>> bucketize([1+1j, 2+2j, 1, 2], key='real')
{1.0: [(1+1j), 1], 2.0: [(2+2j), 2]}

if key is a list, it contains the buckets where to put each object

>>> bucketize([1,2,365,4,98],key=[0,1,2,0,2])
{0: [1, 4], 1: [2], 2: [365, 98]}

Value lists are not deduplicated:

>>> bucketize([None, None, None, 'hello'])
{False: [None, None, None], True: ['hello']}

Bucketize into more than 3 groups

>>> bucketize(range(10), lambda x: x % 3)
{0: [0, 3, 6, 9], 1: [1, 4, 7], 2: [2, 5, 8]}

bucketize has a couple of advanced options useful in certain cases. value_transform can be used to modify values as they are added to buckets, and key_filter will allow excluding certain buckets from being collected.

>>> bucketize(range(5), value_transform=lambda x: x*x)
{False: [0], True: [1, 4, 9, 16]}

>>> bucketize(range(10), key=lambda x: x % 3, key_filter=lambda k: k % 3 != 1)
{0: [0, 3, 6, 9], 2: [2, 5, 8]}

Note in some of these examples there were at most two keys, True and False, and each key present has a list with at least one item. See partition() for a version specialized for binary use cases.

boltons.iterutils.partition(src, key=<class 'bool'>)

No relation to str.partition(), partition is like bucketize(), but for added convenience returns a tuple of (truthy_values, falsy_values).

>>> nonempty, empty = partition(['', '', 'hi', '', 'bye'])
>>> nonempty
['hi', 'bye']

key defaults to bool, but can be carefully overridden to use either a function that returns either True or False or a string name of the attribute on which to partition objects.

>>> import string
>>> is_digit = lambda x: x in string.digits
>>> decimal_digits, hexletters = partition(string.hexdigits, is_digit)
>>> ''.join(decimal_digits), ''.join(hexletters)
('0123456789', 'abcdefABCDEF')

Sorting

The built-in sorted() is great, but what do you do when you want to partially override the sort order?

boltons.iterutils.soft_sorted(iterable, first=None, last=None, key=None, reverse=False)

For when you care about the order of some elements, but not about others.

Use this to float to the top and/or sink to the bottom a specific ordering, while sorting the rest of the elements according to normal sorted() rules.

>>> soft_sorted(['two', 'b', 'one', 'a'], first=['one', 'two'])
['one', 'two', 'a', 'b']
>>> soft_sorted(range(7), first=[6, 15], last=[2, 4], reverse=True)
[6, 5, 3, 1, 0, 2, 4]
>>> import string
>>> ''.join(soft_sorted(string.hexdigits, first='za1', last='b', key=str.lower))
'aA1023456789cCdDeEfFbB'

Parameters:

• iterable (list) – A list or other iterable to sort.

• first (list) – A sequence to enforce for elements which should appear at the beginning of the returned list.

• last (list) – A sequence to enforce for elements which should appear at the end of the returned list.

• key (callable) – Callable used to generate a comparable key for each item to be sorted, same as the key in sorted(). Note that entries in first and last should be the keys for the items. Defaults to passthrough/the identity function.

• reverse (bool) – Whether or not elements not explicitly ordered by first and last should be in reverse order or not.

Returns a new list in sorted order.

boltons.iterutils.untyped_sorted(iterable, key=None, reverse=False)

A version of sorted() which will happily sort an iterable of heterogeneous types and return a new list, similar to legacy Python’s behavior.

>>> untyped_sorted(['abc', 2.0, 1, 2, 'def'])
[1, 2.0, 2, 'abc', 'def']

Note how mutually orderable types are sorted as expected, as in the case of the integers and floats above.

Note

Results may vary across Python versions and builds, but the function will produce a sorted list, except in the case of explicitly unorderable objects.

Reduction

reduce() is a powerful function, but it is also very open-ended and not always the most readable. The standard library recognized this with the addition of sum(), all(), and any(). All these functions take a basic operator (+, and, and or) and use the operator to turn an iterable into a single value.

Functions in this category follow that same spirit, turning iterables like lists into single values:

boltons.iterutils.one(src, default=None, key=None)

Along the same lines as builtins, all() and any(), and similar to first(), one() returns the single object in the given iterable src that evaluates to True, as determined by callable key. If unset, key defaults to bool. If no such objects are found, default is returned. If default is not passed, None is returned.

If src has more than one object that evaluates to True, or if there is no object that fulfills such condition, return default. It’s like an XOR over an iterable.

>>> one((True, False, False))
True
>>> one((True, False, True))
>>> one((0, 0, 'a'))
'a'
>>> one((0, False, None))
>>> one((True, True), default=False)
False
>>> bool(one(('', 1)))
True
>>> one((10, 20, 30, 42), key=lambda i: i > 40)
42

See Martín Gaitán’s original repo for further use cases.

boltons.iterutils.first(iterable, default=None, key=None)

Return first element of iterable that evaluates to True, else return None or optional default. Similar to one().

>>> first([0, False, None, [], (), 42])
42
>>> first([0, False, None, [], ()]) is None
True
>>> first([0, False, None, [], ()], default='ohai')
'ohai'
>>> import re
>>> m = first(re.match(regex, 'abc') for regex in ['b.*', 'a(.*)'])
>>> m.group(1)
'bc'

The optional key argument specifies a one-argument predicate function like that used for filter(). The key argument, if supplied, should be in keyword form. For example, finding the first even number in an iterable:

>>> first([1, 1, 3, 4, 5], key=lambda x: x % 2 == 0)
4

Contributed by Hynek Schlawack, author of the original standalone module.

boltons.iterutils.same(iterable, ref=Sentinel('_UNSET'))

same() returns True when all values in iterable are equal to one another, or optionally a reference value, ref. Similar to all() and any() in that it evaluates an iterable and returns a bool. same() returns True for empty iterables.

>>> same([])
True
>>> same([1])
True
>>> same(['a', 'a', 'a'])
True
>>> same(range(20))
False
>>> same([[], []])
True
>>> same([[], []], ref='test')
False

Type Checks

In the same vein as the feature-checking builtin, callable().

boltons.iterutils.is_iterable(obj)

Similar in nature to callable(), is_iterable returns True if an object is iterable, False if not.

>>> is_iterable([])
True
>>> is_iterable(object())
False

boltons.iterutils.is_scalar(obj)

A near-mirror of is_iterable(). Returns False if an object is an iterable container type. Strings are considered scalar as well, because strings are more often treated as whole values as opposed to iterables of 1-character substrings.

>>> is_scalar(object())
True
>>> is_scalar(range(10))
False
>>> is_scalar('hello')
True

boltons.iterutils.is_collection(obj)

The opposite of is_scalar(). Returns True if an object is an iterable other than a string.

>>> is_collection(object())
False
>>> is_collection(range(10))
True
>>> is_collection('hello')
False