jsonutils - JSON interactions

jsonutils aims to provide various helpers for working with JSON. Currently it focuses on providing a reliable and intuitive means of working with JSON Lines-formatted files.

class boltons.jsonutils.JSONLIterator(file_obj, ignore_errors=False, reverse=False, rel_seek=None)[source]

The JSONLIterator is used to iterate over JSON-encoded objects stored in the JSON Lines format (one object per line).

Most notably it has the ability to efficiently read from the bottom of files, making it very effective for reading in simple append-only JSONL use cases. It also has the ability to start from anywhere in the file and ignore corrupted lines.

Parameters:
  • file_obj (file) – An open file object.
  • ignore_errors (bool) – Whether to skip over lines that raise an error on deserialization (json.loads()).
  • reverse (bool) – Controls the direction of the iteration. Defaults to False. If set to True and rel_seek is unset, seeks to the end of the file before iteration begins.
  • rel_seek (float) – Used to preseek the start position of iteration. Set to 0.0 for the start of the file, 1.0 for the end, and anything in between.
cur_byte_pos

A property representing where in the file the iterator is reading.

next()[source]

Yields one dict loaded with json.loads(), advancing the file object by one line. Raises StopIteration upon reaching the end of the file (or beginning, if reverse was set to True.

boltons.jsonutils.reverse_iter_lines(file_obj, blocksize=4096, preseek=True, encoding=None)[source]

Returns an iterator over the lines from a file object, in reverse order, i.e., last line first, first line last. Uses the file.seek() method of file objects, and is tested compatible with file objects, as well as StringIO.StringIO.

Parameters:
  • file_obj (file) – An open file object. Note that reverse_iter_lines mutably reads from the file and other functions should not mutably interact with the file object after being passed. Files can be opened in bytes or text mode.
  • blocksize (int) – The block size to pass to file.read(). Warning: keep this a fairly large multiple of 2, defaults to 4096.
  • preseek (bool) – Tells the function whether or not to automatically seek to the end of the file. Defaults to True. preseek=False is useful in cases when the file cursor is already in position, either at the end of the file or in the middle for relative reverse line generation.