b.t.Transport(object) : class documentation

Part of bzrlib.transport View In Hierarchy

Known subclasses: bzrlib.tests.test_store.MockTransport, bzrlib.tests.test_transport.BackupTransportHandler, bzrlib.tests.test_transport.BadTransportHandler, bzrlib.transport.ConnectedTransport, bzrlib.transport.decorator.TransportDecorator, bzrlib.transport.local.LocalTransport, bzrlib.transport.memory.MemoryTransport, bzrlib.transport.pathfilter.PathFilteringTransport, bzrlib.transport.remote.HintingSSHTransport

This class encapsulates methods for retrieving or putting a file from/to a storage location.

Most functions have a _multi variant, which allows you to queue up multiple requests. They generally have a dumb base implementation which just iterates over the arguments, but smart Transport implementations can do pipelining. In general implementations should support having a generator or a list as an argument (ie always iterate, never index)

Instance VariablesbaseBase URL for the transport; should always end in a slash.
Method __init__ Undocumented
Method clone Return a new Transport object, cloned from the current location,
Method create_prefix Create all the directories leading down to self.base.
Method ensure_base Ensure that the directory this transport references exists.
Method external_url Return a URL for self that can be given to an external process.
Method get_segment_parameters Return the segment parameters for the top segment of the URL.
Method abspath Return the full url to the given relative path.
Method recommended_page_size Return the recommended page size for this transport.
Method relpath Return the local path portion from a given absolute path.
Method local_abspath Return the absolute path on the local filesystem.
Method has Does the file relpath exist?
Method has_multi Return True/False for each entry in relpaths
Method has_any Return True if any of the paths exist.
Method iter_files_recursive Iter the relative paths of files in the transports sub-tree.
Method get Get the file at the given relative path.
Method get_bytes Get a raw string of the bytes for a file at the given location.
Method get_smart_medium Return a smart client medium for this transport if possible.
Method readv Get parts of the file at the given relative path.
Method get_multi Get a list of file-like objects, one for each entry in relpaths.
Method put_bytes Atomically put the supplied bytes into the given location.
Method put_bytes_non_atomic Copy the string into the target location.
Method put_file Copy the file-like object into the location.
Method put_file_non_atomic Copy the file-like object into the target location.
Method mkdir Create a directory at the given path.
Method mkdir_multi Create a group of directories
Method open_write_stream Open a writable file stream at relpath.
Method append_file Append bytes from a file-like object to a file at relpath.
Method append_bytes Append bytes to a file at relpath.
Method append_multi Append the text in each file-like or string object to
Method copy Copy the item at rel_from to the location at rel_to.
Method copy_multi Copy a bunch of entries.
Method copy_to Copy a set of entries from self into another Transport.
Method copy_tree Copy a subtree from one relpath to another.
Method copy_tree_to_transport Copy a subtree from one transport to another.
Method rename Rename a file or directory.
Method move Move the item at rel_from to the location at rel_to.
Method move_multi Move a bunch of entries.
Method move_multi_to Move a bunch of entries to a single location.
Method delete Delete the item at relpath
Method delete_multi Queue up a bunch of deletes to be done.
Method delete_tree Delete an entire tree. This may require a listable transport.
Method __repr__ Undocumented
Method stat Return the stat information for a file.
Method rmdir Remove a directory at the given path.
Method stat_multi Stat multiple files and return the information.
Method readlink Return a string representing the path to which the symbolic link points.
Method hardlink Create a hardlink pointing to source named link_name.
Method symlink Create a symlink pointing to source named link_name.
Method listable Return True if this store supports listing.
Method list_dir Return a list of all files at the given location.
Method lock_read Lock the given file for shared (read) access.
Method lock_write Lock the given file for exclusive (write) access.
Method is_readonly Return true if this connection cannot be written to.
Method disconnect Undocumented
Method _translate_error Translate an IOError or OSError into an appropriate bzr error.
Method _pump Most children will need to copy from one file-like
Method _get_total Try to figure out how many entries are in multi,
Method _report_activity Notify that this transport has activity.
Method _update_pb Update the progress bar based on the current count
Method _iterate_over Iterate over all entries in multi, passing them to func,
Method _readv Get parts of the file at the given relative path.
Method _seek_and_read An implementation of readv that uses fp.seek and fp.read.
Method _sort_expand_and_combine Helper for readv.
Static Method _coalesce_offsets Yield coalesced offsets.
Method _can_roundtrip_unix_modebits Return true if this transport can store and retrieve unix modebits.
Method _reuse_for Undocumented
Method _redirected_to Returns a transport suitable to re-issue a redirected request.
def __init__(self, base):
Undocumented
def _translate_error(self, e, path, raise_generic=True):
Translate an IOError or OSError into an appropriate bzr error.

This handles things like ENOENT, ENOTDIR, EEXIST, and EACCESS

def clone(self, offset=None):
Return a new Transport object, cloned from the current location, using a subdirectory or parent directory. This allows connections to be pooled, rather than a new one needed for each subdir.
def create_prefix(self):
Create all the directories leading down to self.base.
def ensure_base(self):
Ensure that the directory this transport references exists.

This will create a directory if it doesn't exist. :return: True if the directory was created, False otherwise.

def external_url(self):
Return a URL for self that can be given to an external process.

There is no guarantee that the URL can be accessed from a different machine - e.g. file:/// urls are only usable on the local machine, sftp:/// urls when the server is only bound to localhost are only usable from localhost etc.

NOTE: This method may remove security wrappers (e.g. on chroot transports) and thus should only be used when the result will not be used to obtain a new transport within bzrlib. Ideally chroot transports would know enough to cause the external url to be the exact one used that caused the chrooting in the first place, but that is not currently the case.

ReturnsA URL that can be given to another process.
RaisesInProcessTransportIf the transport is one that cannot be accessed out of the current process (e.g. a MemoryTransport) then InProcessTransport is raised.
def get_segment_parameters(self):
Return the segment parameters for the top segment of the URL.
def _pump(self, from_file, to_file):
Most children will need to copy from one file-like object or string to another one. This just gives them something easy to call.
def _get_total(self, multi):
Try to figure out how many entries are in multi, but if not possible, return None.
def _report_activity(self, bytes, direction):
Notify that this transport has activity.

Implementations should call this from all methods that actually do IO. Be careful that it's not called twice, if one method is implemented on top of another.

ParametersbytesNumber of bytes read or written.
direction'read' or 'write' or None.
def _update_pb(self, pb, msg, count, total):
Update the progress bar based on the current count and total available, total may be None if it was not possible to determine.
def _iterate_over(self, multi, func, pb, msg, expand=True):
Iterate over all entries in multi, passing them to func, and update the progress bar as you go along.
ParametersexpandIf True, the entries will be passed to the function by expanding the tuple. If False, it will be passed as a single parameter.
def abspath(self, relpath):
Return the full url to the given relative path.
Parametersrelpatha string of a relative path
def recommended_page_size(self):
Return the recommended page size for this transport.

This is potentially different for every path in a given namespace. For example, local transports might use an operating system call to get the block size for a given path, which can vary due to mount points.

ReturnsThe page size in bytes.
def relpath(self, abspath):
Return the local path portion from a given absolute path.

This default implementation is not suitable for filesystems with aliasing, such as that given by symlinks, where a path may not start with our base, but still be a relpath once aliasing is resolved.

def local_abspath(self, relpath):
Return the absolute path on the local filesystem.

This function will only be defined for Transports which have a physical local filesystem representation.

Raiseserrors.NotLocalUrlWhen no local path representation is available.
def has(self, relpath):
Does the file relpath exist?

Note that some transports MAY allow querying on directories, but this is not part of the protocol. In other words, the results of t.has("a_directory_name") are undefined.

Returns (type: bool)
def has_multi(self, relpaths, pb=None):
Return True/False for each entry in relpaths
def has_any(self, relpaths):
Return True if any of the paths exist.
def iter_files_recursive(self):
Iter the relative paths of files in the transports sub-tree.

NOTE: This only lists files, not subdirectories!

As with other listing functions, only some transports implement this,. you may check via listable() to determine if it will.

def get(self, relpath):
Get the file at the given relative path.

This may fail in a number of ways:
  • HTTP servers may return content for a directory. (unexpected content failure)
  • FTP servers may indicate NoSuchFile for a directory.
  • SFTP servers may give a file handle for a directory that will fail on read().

For correct use of the interface, be sure to catch errors.PathError when calling it and catch errors.ReadError when reading from the returned object.

ParametersrelpathThe relative path to the file
Returns (type: File-like object.)
def get_bytes(self, relpath):
Get a raw string of the bytes for a file at the given location.
ParametersrelpathThe relative path to the file
def get_smart_medium(self):
Return a smart client medium for this transport if possible.

A smart medium doesn't imply the presence of a smart server: it implies that the smart protocol can be tunnelled via this transport.

RaisesNoSmartMediumif no smart server medium is available.
def readv(self, relpath, offsets, adjust_for_latency=False, upper_limit=None):
Get parts of the file at the given relative path.
ParametersrelpathThe path to read data from.
offsetsA list of (offset, size) tuples.
adjust_for_latencyAdjust the requested offsets to accomodate transport latency. This may re-order the offsets, expand them to grab adjacent data when there is likely a high cost to requesting data relative to delivering it.
upper_limitWhen adjust_for_latency is True setting upper_limit allows the caller to tell the transport about the length of the file, so that requests are not issued for ranges beyond the end of the file. This matters because some servers and/or transports error in such a case rather than just satisfying the available ranges. upper_limit should always be provided when adjust_for_latency is True, and should be the size of the file in bytes.
ReturnsA list or generator of (offset, data) tuples
def _readv(self, relpath, offsets):
Get parts of the file at the given relative path.
ParametersrelpathThe path to read.
offsetsA list of (offset, size) tuples.
ReturnsA list or generator of (offset, data) tuples
def _seek_and_read(self, fp, offsets, relpath='<unknown>'):
An implementation of readv that uses fp.seek and fp.read.

This uses _coalesce_offsets to issue larger reads and fewer seeks.

ParametersfpA file-like object that supports seek() and read(size). Note that implementations are allowed to call .close() on this file handle, so don't trust that you can use it for other work.
offsetsA list of offsets to be read from the given file.
Returnsyield (pos, data) tuples for each request
def _sort_expand_and_combine(self, offsets, upper_limit):
Helper for readv.
ParametersoffsetsA readv vector - (offset, length) tuples.
upper_limitThe highest byte offset that may be requested.
ReturnsA readv vector that will read all the regions requested by offsets, in start-to-end order, with no duplicated regions, expanded by the transports recommended page size.
@staticmethod
def _coalesce_offsets(offsets, limit=0, fudge_factor=0, max_size=0):
Yield coalesced offsets.

With a long list of neighboring requests, combine them into a single large request, while retaining the original offsets. Turns [(15, 10), (25, 10)] => [(15, 20, [(0, 10), (10, 10)])] Note that overlapping requests are not permitted. (So [(15, 10), (20, 10)] will raise a ValueError.) This is because the data we access never overlaps, and it allows callers to trust that we only need any byte of data for 1 request (so nothing needs to be buffered to fulfill a second request.)

ParametersoffsetsA list of (start, length) pairs
limitOnly combine a maximum of this many pairs Some transports penalize multiple reads more than others, and sometimes it is better to return early. 0 means no limit
fudge_factorAll transports have some level of 'it is better to read some more data and throw it away rather than seek', so collapse if we are 'close enough'
max_sizeCreate coalesced offsets no bigger than this size. When a single offset is bigger than 'max_size', it will keep its size and be alone in the coalesced offset. 0 means no maximum size.
Returnsreturn a list of _CoalescedOffset objects, which have members for where to start, how much to read, and how to split those chunks back up
def get_multi(self, relpaths, pb=None):
Get a list of file-like objects, one for each entry in relpaths.
ParametersrelpathsA list of relative paths.
pbAn optional ProgressTask for indicating percent done.
ReturnsA list or generator of file-like objects
def put_bytes(self, relpath, bytes, mode=None):
Atomically put the supplied bytes into the given location.
ParametersrelpathThe location to put the contents, relative to the transport base.
bytesA bytestring of data.
modeCreate the file with the given mode.
ReturnsNone
def put_bytes_non_atomic(self, relpath, bytes, mode=None, create_parent_dir=False, dir_mode=None):
Copy the string into the target location.

This function is not strictly safe to use. See Transport.put_bytes_non_atomic for more information.

ParametersrelpathThe remote location to put the contents.
bytesA string object containing the raw bytes to write into the target file.
modePossible access permissions for new file. None means do not set remote permissions.
create_parent_dirIf we cannot create the target file because the parent directory does not exist, go ahead and create it, and then try again.
dir_modePossible access permissions for new directories.
def put_file(self, relpath, f, mode=None):
Copy the file-like object into the location.
ParametersrelpathLocation to put the contents, relative to base.
fFile-like object.
modeThe mode for the newly created file, None means just use the default.
ReturnsThe length of the file that was written.
def put_file_non_atomic(self, relpath, f, mode=None, create_parent_dir=False, dir_mode=None):
Copy the file-like object into the target location.

This function is not strictly safe to use. It is only meant to be used when you already know that the target does not exist. It is not safe, because it will open and truncate the remote file. So there may be a time when the file has invalid contents.

ParametersrelpathThe remote location to put the contents.
fFile-like object.
modePossible access permissions for new file. None means do not set remote permissions.
create_parent_dirIf we cannot create the target file because the parent directory does not exist, go ahead and create it, and then try again.
dir_modePossible access permissions for new directories.
def mkdir(self, relpath, mode=None):
Create a directory at the given path.
def mkdir_multi(self, relpaths, mode=None, pb=None):
Create a group of directories
def open_write_stream(self, relpath, mode=None):
Open a writable file stream at relpath.

A file stream is a file like object with a write() method that accepts bytes to write.. Buffering may occur internally until the stream is closed with stream.close(). Calls to readv or the get_* methods will be synchronised with any internal buffering that may be present.

ParametersrelpathThe relative path to the file.
modeThe mode for the newly created file, None means just use the default
ReturnsA FileStream. FileStream objects have two methods, write() and close(). There is no guarantee that data is committed to the file if close() has not been called (even if get() is called on the same path).
def append_file(self, relpath, f, mode=None):
Append bytes from a file-like object to a file at relpath.

The file is created if it does not already exist.

Parametersfa file-like object of the bytes to append.
modeUnix mode for newly created files. This is not used for existing files.
Returnsthe length of relpath before the content was written to it.
def append_bytes(self, relpath, bytes, mode=None):
Append bytes to a file at relpath.

The file is created if it does not already exist.

Parametersfa string of the bytes to append. (type: str)
modeUnix mode for newly created files. This is not used for existing files.
Returnsthe length of relpath before the content was written to it.
def append_multi(self, files, pb=None):
Append the text in each file-like or string object to the supplied location.
ParametersfilesA set of (path, f) entries
pbAn optional ProgressTask for indicating percent done.
def copy(self, rel_from, rel_to):
Copy the item at rel_from to the location at rel_to.

Override this for efficiency if a specific transport can do it faster than this default implementation.

def copy_multi(self, relpaths, pb=None):
Copy a bunch of entries.
ParametersrelpathsA list of tuples of the form [(from, to), (from, to),...]
def copy_to(self, relpaths, other, mode=None, pb=None):
Copy a set of entries from self into another Transport.

:param relpaths: A list/generator of entries to be copied.
:param mode: This is the target mode for the newly created files
TODO: This interface needs to be updated so that the target location
      can be different from the source location.
def copy_tree(self, from_relpath, to_relpath):
Copy a subtree from one relpath to another.

If a faster implementation is available, specific transports should implement it.

def copy_tree_to_transport(self, to_transport):
Copy a subtree from one transport to another.

self.base is used as the source tree root, and to_transport.base is used as the target. to_transport.base must exist (and be a directory).

def rename(self, rel_from, rel_to):
Rename a file or directory.

This must fail if the destination is a nonempty directory - it must not automatically remove it. It should raise DirectoryNotEmpty, or some other PathError if the case can't be specifically detected.

If the destination is an empty directory or a file this function may either fail or succeed, depending on the underlying transport. It should not attempt to remove the destination if overwriting is not the native transport behaviour. If at all possible the transport should ensure that the rename either completes or not, without leaving the destination deleted and the new file not moved in place.

This is intended mainly for use in implementing LockDir.

def move(self, rel_from, rel_to):
Move the item at rel_from to the location at rel_to.

The destination is deleted if possible, even if it's a non-empty directory tree.

If a transport can directly implement this it is suggested that it do so for efficiency.

def move_multi(self, relpaths, pb=None):
Move a bunch of entries.
ParametersrelpathsA list of tuples of the form [(from1, to1), (from2, to2),...]
def move_multi_to(self, relpaths, rel_to):
Move a bunch of entries to a single location. This differs from move_multi in that you give a list of from, and a single destination, rather than multiple destinations.
ParametersrelpathsA list of relative paths [from1, from2, from3, ...]
rel_toA directory where each entry should be placed.
def delete(self, relpath):
Delete the item at relpath
def delete_multi(self, relpaths, pb=None):
Queue up a bunch of deletes to be done.
def delete_tree(self, relpath):
Delete an entire tree. This may require a listable transport.
def __repr__(self):
Undocumented
def stat(self, relpath):
Return the stat information for a file. WARNING: This may not be implementable for all protocols, so use sparingly. NOTE: This returns an object with fields such as 'st_size'. It MAY or MAY NOT return the literal result of an os.stat() call, so all access should be via named fields. ALSO NOTE: Stats of directories may not be supported on some transports.
def rmdir(self, relpath):
Remove a directory at the given path.
def stat_multi(self, relpaths, pb=None):
Stat multiple files and return the information.
def readlink(self, relpath):
Return a string representing the path to which the symbolic link points.
def hardlink(self, source, link_name):
Create a hardlink pointing to source named link_name.
def symlink(self, source, link_name):
Create a symlink pointing to source named link_name.
def listable(self):
Return True if this store supports listing.
def list_dir(self, relpath):
Return a list of all files at the given location. WARNING: many transports do not support this, so trying avoid using it if at all possible.
def lock_read(self, relpath):
Lock the given file for shared (read) access.

WARNING: many transports do not support this, so trying avoid using it. These methods may be removed in the future.

Transports may raise TransportNotPossible if OS-level locks cannot be taken over this transport.

ReturnsA lock object, which should contain an unlock() function.
def lock_write(self, relpath):
Lock the given file for exclusive (write) access.

WARNING: many transports do not support this, so trying avoid using it. These methods may be removed in the future.

Transports may raise TransportNotPossible if OS-level locks cannot be taken over this transport.

ReturnsA lock object, which should contain an unlock() function.
def is_readonly(self):
Return true if this connection cannot be written to.
def _can_roundtrip_unix_modebits(self):
Return true if this transport can store and retrieve unix modebits.

(For example, 0700 to make a directory owner-private.)

Note: most callers will not want to switch on this, but should rather just try and set permissions and let them be either stored or not. This is intended mainly for the use of the test suite.

Warning: this is not guaranteed to be accurate as sometimes we can't be sure: for example with vfat mounted on unix, or a windows sftp server.

def _reuse_for(self, other_base):
Undocumented
def disconnect(self):
Undocumented
def _redirected_to(self, source, target):
Returns a transport suitable to re-issue a redirected request.

The redirection can be handled only if the relpath involved is not renamed by the redirection.

ParameterssourceThe source url as returned by the server.
targetThe target url as returned by the server.
ReturnsA transport or None.
API Documentation for Bazaar, generated by pydoctor at 2019-09-16 00:30:59.