b.i.GraphIndex(object) : class documentation

An index for data with embedded graphs.

The index maps keys to a list of key reference lists, and a value. Each node has the same number of key reference lists. Each key reference list can be empty or an arbitrary length. The value is an opaque NULL terminated string without any newlines. The storage of the index is hidden in the interface: keys and key references are always tuples of bytestrings, never the internal representation (e.g. dictionary offsets).

It is presumed that the index will not be mutated - it is static data.

Successive iter_all_entries calls will read the entire index each time. Additionally, iter_entries calls will read the index linearly until the desired keys are found. XXX: This must be fixed before the index is suitable for production use. :XXX

Method	__init__	Open an index called name on transport.
Method	__eq__	Equal when self and other were created with the same parameters.
Method	__ne__	Undocumented
Method	__repr__	Undocumented
Method	clear_cache	Clear out any cached/memoized values.
Method	external_references	Return references that are not present in this index.
Method	iter_all_entries	Iterate over all keys within the index.
Method	iter_entries	Iterate over keys within the index.
Method	iter_entries_prefix	Iterate over keys within the index using prefix matching.
Method	key_count	Return an estimate of the number of keys in this index.
Method	validate	Validate that everything in the index can be accessed.
Method	_buffer_all	Buffer all the index data.
Method	_get_nodes_by_key	Undocumented
Method	_read_prefix	Undocumented
Method	_resolve_references	Return the resolved key references for references.
Method	_find_index	Helper for the _parsed_*_index calls.
Method	_parsed_byte_index	Return the index of the entry immediately before offset.
Method	_parsed_key_index	Return the index of the entry immediately before key.
Method	_is_parsed	Returns True if offset has been parsed.
Method	_iter_entries_from_total_buffer	Iterate over keys when the entire index is parsed.
Method	_find_ancestors	See BTreeIndex._find_ancestors.
Method	_lookup_keys_via_location	Public interface for implementing bisection.
Method	_parse_header_from_bytes	Parse the header from a region of bytes.
Method	_parse_region	Parse node data returned from a readv operation.
Method	_parse_segment	Parse one segment of data.
Method	_parse_lines	Undocumented
Method	_parsed_bytes	Mark the bytes from start to end as parsed.
Method	_read_and_parse	Read the ranges and parse the resulting data.
Method	_signature	The file signature for this index type.

def __init__(self, transport, name, size, unlimited_cache=False, offset=0):

Open an index called name on transport.

Parameters	transport	A bzrlib.transport.Transport.
	name	A path to provide to transport API calls.
	size	The size of the index in bytes. This is used for bisection logic to perform partial index reads. While the size could be obtained by statting the file this introduced an additional round trip as well as requiring stat'able transports, both of which are avoided by having it supplied. If size is None, then bisection support will be disabled and accessing the index will just stream all the data.
	offset	Instead of starting the index data at offset 0, start it at an arbitrary offset.

def __eq__(self, other):

Equal when self and other were created with the same parameters.

def __ne__(self, other):

Undocumented

def __repr__(self):

Undocumented

def _buffer_all(self, stream=None):

Buffer all the index data.

Mutates self._nodes and self.keys_by_offset.

def clear_cache(self):

Clear out any cached/memoized values.

This can be called at any time, but generally it is used when we have extracted some information, but don't expect to be requesting any more from this index.

def external_references(self, ref_list_num):

Return references that are not present in this index.

def _get_nodes_by_key(self):

Undocumented

def iter_all_entries(self):

Iterate over all keys within the index.

Returns An iterable of (index, key, value) or (index, key, value, reference_lists). The former tuple is used when there are no reference lists in the index, making the API compatible with simple key:value index types. There is no defined order for the result iteration - it will be in the most efficient order for the index.

def _read_prefix(self, stream):

Undocumented

def _resolve_references(self, references):

Return the resolved key references for references.

References are resolved by looking up the location of the key in the _keys_by_offset map and substituting the key name, preserving ordering.

Parameters	references	An iterable of iterables of key locations. e.g. [[123, 456], [123]]
Returns	A tuple of tuples of keys.

def _find_index(self, range_map, key):

Helper for the _parsed_*_index calls.

Given a range map - [(start, end), ...], finds the index of the range in the map for key if it is in the map, and if it is not there, the immediately preceeding range in the map.

def _parsed_byte_index(self, offset):

Return the index of the entry immediately before offset.

e.g. if the parsed map has regions 0,10 and 11,12 parsed, meaning that there is one unparsed byte (the 11th, addressed as[10]). then: asking for 0 will return 0 asking for 10 will return 0 asking for 11 will return 1 asking for 12 will return 1

def _parsed_key_index(self, key):

Return the index of the entry immediately before key.

e.g. if the parsed map has regions (None, 'a') and ('b','c') parsed, meaning that keys from None to 'a' inclusive, and 'b' to 'c' inclusive have been parsed, then: asking for '' will return 0 asking for 'a' will return 0 asking for 'b' will return 1 asking for 'e' will return 1

def _is_parsed(self, offset):

Returns True if offset has been parsed.

def _iter_entries_from_total_buffer(self, keys):

Iterate over keys when the entire index is parsed.

def iter_entries(self, keys):

Iterate over keys within the index.

Parameters	keys	An iterable providing the keys to be retrieved.
Returns	An iterable as per iter_all_entries, but restricted to the keys supplied. No additional keys will be returned, and every key supplied that is in the index will be returned.

def iter_entries_prefix(self, keys):

Iterate over keys within the index using prefix matching.

Prefix matching is applied within the tuple of a key, not to within the bytestring of each key element. e.g. if you have the keys ('foo', 'bar'), ('foobar', 'gam') and do a prefix search for ('foo', None) then only the former key is returned.

WARNING: Note that this method currently causes a full index parse unconditionally (which is reasonably appropriate as it is a means for thunking many small indices into one larger one and still supplies iter_all_entries at the thunk layer).

Parameters	keys	An iterable providing the key prefixes to be retrieved. Each key prefix takes the form of a tuple the length of a key, but with the last N elements 'None' rather than a regular bytestring. The first element cannot be 'None'.
Returns	An iterable as per iter_all_entries, but restricted to the keys with a matching prefix to those supplied. No additional keys will be returned, and every match that is in the index will be returned.

def _find_ancestors(self, keys, ref_list_num, parent_map, missing_keys):

See BTreeIndex._find_ancestors.

def key_count(self):

Return an estimate of the number of keys in this index.

For GraphIndex the estimate is exact.

def _lookup_keys_via_location(self, location_keys):

Public interface for implementing bisection.

If _buffer_all has been called, then all the data for the index is in memory, and this method should not be called, as it uses a separate cache because it cannot pre-resolve all indices, which buffer_all does for performance.

Parameters	location_keys	A list of location(byte offset), key tuples.
Returns	A list of (location_key, result) tuples as expected by bzrlib.bisect_multi.bisect_multi_bytes.

def _parse_header_from_bytes(self, bytes):

Parse the header from a region of bytes.

Parameters	bytes	The data to parse.
Returns	An offset, data tuple such as readv yields, for the unparsed data. (which may length 0).

def _parse_region(self, offset, data):

Parse node data returned from a readv operation.

Parameters	offset	The byte offset the data starts at.
	data	The data to parse.

def _parse_segment(self, offset, data, end, index):

Parse one segment of data.

Parameters	offset	Where 'data' begins in the file.
	data	Some data to parse a segment of.
	end	Where data ends
	index	The current index into the parsed bytes map.
Returns	high_parsed_byte, last_segment. high_parsed_byte is the location of the highest parsed byte in this segment, last_segment is True if the parsed segment is the last possible one in the data block.

def _parse_lines(self, lines, pos):

Undocumented

def _parsed_bytes(self, start, start_key, end, end_key):

Mark the bytes from start to end as parsed.

Calling self._parsed_bytes(1,2) will mark one byte (the one at offset 1) as parsed.

Parameters	start	The start of the parsed region.
	end	The end of the parsed region.

def _read_and_parse(self, readv_ranges):

Read the ranges and parse the resulting data.

Parameters readv_ranges A prepared readv range list.

def _signature(self):

The file signature for this index type.

def validate(self):

Validate that everything in the index can be accessed.

API Documentation for Bazaar, generated by pydoctor at 2022-06-16 00:25:16.