l.a.uploadprocessor : module documentation

Part of lp.archiveuploader

Code for 'processing' 'uploads'. Also see nascentupload.py.

Uploads are directories in the 'incoming' queue directory. They may have arrived manually from a distribution contributor, via a poppy upload, or they may have come from a build.

Within an upload, we may find no changes file, one, or several. One is the usual number. To process the upload, we process each changes file in turn. These changes files may be within a structure of sub-directories, in which case we extract information from the names of these, to calculate which distribution and which PPA are being uploaded to.

To process a changes file, we make checks such as that the other files referenced by it are present, formatting is valid, signatures are correct, checksums match, and that the .changes file represents an upload which makes sense, eg. it is not a binary for which we have no source, or an older version than already exists in the same target distroseries pocket.

Depending on the outcome of these checks, the changes file will either be accepted (and the information from it, and the referenced files, imported into the database) or it won't (and the database will be unchanged). If not accepted, a changes file might be 'failed' or 'rejected', where failed changes files are dropped silently, but rejected ones generate a rejection email back to the uploader.

There are several valid reasons to fail (the changes file is so mangled that we can't read who we should send a rejection to, or it's not correctly signed, so we can't be sure a rejection wouldn't be spam (it may not have been uploaded by who it says it was uploaded by). In practice, in the code as it stands, we also consider the processing of a changes file to have failed if it generates an unexpected exception, and there are some known cases where it does this and a rejection would have been more useful (see bug 35965).

Each upload directory is saved after processing, in case it is needed for debugging purposes. This is done by moving it to a directory inside the queue directory, beside incoming, named after the result - 'failed', 'rejected' or 'accepted'. Where there are no changes files, the upload is considered failed, and where there is more than one changes file, the upload is assigned the worst of the results from the various changes files found (in the order above, failed being worst).

Function parse_build_upload_leaf_name Parse the leaf directory name of a build upload.
Class UploadStatusEnum Possible results from processing an upload.
Class UploadPathError This exception happened when parsing the upload path.
Class PPAUploadPathError Exception when parsing a PPA upload path.
Class UploadProcessor Responsible for processing uploads. See module docstring.
Class UploadHandler Handler for processing a single upload.
Class UserUploadHandler No class docstring; 1/4 methods documented
Class CannotGetBuild Attempting to retrieve the build for this upload failed.
Class BuildUploadHandler No class docstring; 4/8 methods documented
Function parse_upload_path Locate the distribution and archive for the upload.
Function _getDistributionAndSuite Return an IDistribution and a valid suite name for the given path.
def parse_build_upload_leaf_name(name):
Parse the leaf directory name of a build upload.
ParametersnameDirectory name.
ReturnsTuple with build farm job id.
def _getDistributionAndSuite(parts, exc_type):
Return an IDistribution and a valid suite name for the given path.

Helper function used within parse_upload_path for extracting and verifying the part of the upload path targeting a existing distribution and optionally one of its suite.

It will fail with AssertionError if the given parts is not a list with one or two elements.

Parameterspartsa list of path parts to be processed.
exc_typea specific Exception type that should be raised on errors.
Returnsa tuple containing a IDistribution and a suite name if it's appropriate. The suite name will be None if it wasn't present in the given path parts.
Raisesthe given exc_type if the corresponding distribution or suite could not be found.
def parse_upload_path(relative_path):
Locate the distribution and archive for the upload.

We do this by analysing the path to which the user has uploaded, ie. the relative path within the upload folder to the changes file.

Current paths are:
/<distro>[/suite] - any primary archive /~<person>/<distro>/<ppa>[/suite] - any PPA
One deprecated form is still supported for the Ubuntu primary archive:
/ - Ubuntu primary archive
Three deprecated forms are still supported for Ubuntu PPAs:
/~<person>/<ppa>/ubuntu[/suite] - any Ubuntu PPA /~<person>/<ppa> - any Ubuntu PPA /~<person>/ubuntu - Ubuntu PPA named "ppa"

The original /~<person>/ubuntu/<suite> form is no longer supported as it clashes with /~<person>/<distro>/<ppa>

suite is an optional distroseries with an optional pocket suffix.

I raises UploadPathError if something was wrong when parsing it.

On success it returns a tuple of IDistribution, suite-name, IArchive for the given path, where the second field can be None.

API Documentation for Launchpad, generated by pydoctor at 2019-12-09 00:00:08.