b.k._KndxIndex(object) : class documentation

Part of bzrlib.knit View In Hierarchy

Manages knit index files

    The index is kept in memory and read on startup, to enable
    fast lookups of revision information.  The cursor of the index
    file is always pointing to the end, making it easy to append
    entries.

    _cache is a cache for fast mapping from version id to a Index
    object.

    _history is a cache for fast mapping from indexes to version ids.

    The index data format is dictionary compressed when it comes to
    parent references; a index entry may only have parents that with a
    lover index number.  As a result, the index is topological sorted.

    Duplicate entries may be written to the index for a single version id
    if this is done then the latter one completely replaces the former:
    this allows updates to correct version and parent information.
    Note that the two entries may share the delta, and that successive
    annotations and references MUST point to the first entry.

    The index file on disc contains a header, followed by one line per knit
    record. The same revision can be present in an index file more than once.
    The first occurrence gets assigned a sequence number starting from 0.

    The format of a single line is
    REVISION_ID FLAGS BYTE_OFFSET LENGTH( PARENT_ID|PARENT_SEQUENCE_ID)* :

    REVISION_ID is a utf8-encoded revision id
    FLAGS is a comma separated list of flags about the record. Values include
        no-eol, line-delta, fulltext.
    BYTE_OFFSET is the ascii representation of the byte offset in the data file
        that the compressed data starts at.
    LENGTH is the ascii representation of the length of the data file.
    PARENT_ID a utf-8 revision id prefixed by a '.' that is a parent of
        REVISION_ID.
    PARENT_SEQUENCE_ID the ascii representation of the sequence number of a
        revision id already in the knit that is a parent of REVISION_ID.
    The ' :' marker is the end of record marker.

    partial writes:
    when a write is interrupted to the index file, it will result in a line
    that does not end in ' :'. If the ' :' is not present at the end of a line,
    or at the end of the file, then the record that is missing it will be
    ignored by the parser.

    When writing new records to the index file, the data is preceded by '
'
    to ensure that records always start on new lines even if the last write was
    interrupted. As a result its normal for the last line in the index to be
    missing a trailing newline. One can be added with no harmful effects.

    :ivar _kndx_cache: dict from prefix to the old state of KnitIndex objects,
        where prefix is e.g. the (fileid,) for .texts instances or () for
        constant-mapped things like .revisions, and the old state is
        tuple(history_vector, cache_dict).  This is used to prevent having an
        ABI change with the C extension that reads .kndx files.
    
Method __init__ Create a _KndxIndex on transport using mapper.
Method add_records Add multiple records to the index.
Method scan_unvalidated_index See _KnitGraphIndex.scan_unvalidated_index.
Method get_missing_compression_parents See _KnitGraphIndex.get_missing_compression_parents.
Method check_header Undocumented
Method get_build_details Get the method, index_memo and compression parent for keys.
Method get_method Return compression method of specified key.
Method get_options Return a list representing options.
Method find_ancestry See CombinedGraphIndex.find_ancestry()
Method get_parent_map Get a map of the parents of keys.
Method get_position Return details needed to access the version.
Method keys Get all the keys in the collection.
Method _cache_key Cache a version record in the history array and index cache.
Method _check_read Undocumented
Method _check_write_ok Assert if not writes are permitted.
Method _init_index Initialize an index.
Method _load_prefixes Load the indices for prefixes.
Method _partition_keys Turn keys into a dict of prefix:suffix_list.
Method _dictionary_compress Dictionary compress keys.
Method _reset_cache Undocumented
Method _sort_keys_by_io Figure out an optimal order to read the records for the given keys.
Method _split_key Split key into a prefix and suffix.
def __init__(self, transport, mapper, get_scope, allow_writes, is_locked):
Create a _KndxIndex on transport using mapper.
def add_records(self, records, random_id=False, missing_compression_parents=False):
Add multiple records to the index.
Parametersrecordsa list of tuples: (key, options, access_memo, parents).
random_idIf True the ids being added were randomly generated and no check for existence will be performed.
missing_compression_parentsIf True the records being added are only compressed against texts already in the index (or inside records). If False the records all refer to unavailable texts (or texts inside records) as compression parents.
def scan_unvalidated_index(self, graph_index):
See _KnitGraphIndex.scan_unvalidated_index.
def get_missing_compression_parents(self):
See _KnitGraphIndex.get_missing_compression_parents.
def _cache_key(self, key, options, pos, size, parent_keys):
Cache a version record in the history array and index cache.

This is inlined into _load_data for performance. KEEP IN SYNC.
(It saves 60ms, 25% of the __init__ overhead on local 4000 record
 indexes).
def check_header(self, fp):
Undocumented
def _check_read(self):
Undocumented
def _check_write_ok(self):
Assert if not writes are permitted.
def get_build_details(self, keys):
Get the method, index_memo and compression parent for keys.

Ghosts are omitted from the result.

:param keys: An iterable of keys.
:return: A dict of key:(index_memo, compression_parent, parents,
    record_details).
    index_memo
        opaque structure to pass to read_records to extract the raw
        data
    compression_parent
        Content that this record is built upon, may be None
    parents
        Logical parents of this node
    record_details
        extra information about the content which needs to be passed to
        Factory.parse_record
def get_method(self, key):
Return compression method of specified key.
def get_options(self, key):
Return a list representing options.

e.g. ['foo', 'bar']

def find_ancestry(self, keys):
See CombinedGraphIndex.find_ancestry()
def get_parent_map(self, keys):
Get a map of the parents of keys.
ParameterskeysThe keys to look up parents for.
ReturnsA mapping from keys to parents. Absent keys are absent from the mapping.
def get_position(self, key):
Return details needed to access the version.
Returnsa tuple (key, data position, size) to hand to the access logic to get the record.
def _init_index(self, path, extra_lines=):
Initialize an index.
def keys(self):
Get all the keys in the collection.

The keys are not ordered.

def _load_prefixes(self, prefixes):
Load the indices for prefixes.
def _partition_keys(self, keys):
Turn keys into a dict of prefix:suffix_list.
def _dictionary_compress(self, keys):
Dictionary compress keys.
ParameterskeysThe keys to generate references to.
ReturnsA string representation of keys. keys which are present are dictionary compressed, and others are emitted as fulltext with a '.' prefix.
def _reset_cache(self):
Undocumented
def _sort_keys_by_io(self, keys, positions):
Figure out an optimal order to read the records for the given keys.

Sort keys, grouped by index and sorted by position.

ParameterskeysA list of keys whose records we want to read. This will be sorted 'in-place'.
positionsA dict, such as the one returned by _get_components_positions()
ReturnsNone
def _split_key(self, key):
Split key into a prefix and suffix.
API Documentation for Bazaar, generated by pydoctor at 2019-10-20 00:34:54.