l.a.s.p.PublishFTPMaster(LaunchpadCronScript) : class documentation

Part of lp.archivepublisher.scripts.publish_ftpmaster View In Hierarchy

Publish a distro (update).

The publishable files are kept in the filesystem. Most of the work is done in a working "dists" directory, which then replaces the current "dists" in the archive root.

For performance reasons, the old "dists" is not discarded. It is kept as the dists-copy version for the next run. Its contents don't matter in detail; an rsync updates it based on the currently published dists directory before we start working with it.

At the end of one pass of the script, the "dists" directory in the archive root and its backup copy in the dists-copy root will have traded places.

However the script normally does 2 passes: once just for security updates, to expedite publication of critical fixes, and once for the whole distribution. At the end of this, the directories will be back in their original places (though with updated contents).

Method add_my_options See LaunchpadScript.
Method processOptions Handle command-line options.
Method getConfigs Set up configuration objects for archives to be published.
Method locateIndexesMarker Give path for marker file whose presence marks index creation.
Method listSuitesNeedingIndexes Find suites in distroseries that need indexes created.
Method markIndexCreationComplete Note that archive indexes for suite have been created.
Method createIndexes Create archive indexes for suites of distroseries.
Method processAccepted Run the process-accepted script.
Method getDirtySuites Return set of suites that have packages pending publication.
Method getDirtySecuritySuites List security suites with pending publications.
Method rsyncBackupDists Populate the backup dists with a copy of distsroot.
Method recoverArchiveWorkingDir Recover working dists dir for archive_config.
Method recoverWorkingDists Look for and recover any dists left in transient working state.
Method setUpDirs Create archive roots and such if they did not yet exist.
Method runPublishDistro Execute publish-distro.
Method publishDistroArchive Publish the results for an archive.
Method runPublishDistroParts Execute the publish-distro hooks.
Method installDists Put the new dists into place, as near-atomically as possible.
Method clearEmptyDirs Clear out any redundant empty directories.
Method runFinalizeParts Run the finalize.d parts to finalize publication.
Method publishSecurityUploads Quickly process just the pending security uploads.
Method publishDistroUploads Publish the distro's complete uploads.
Method updateStagedFilesForSuite Install all staged files for a single archive and suite.
Method updateStagedFiles Install all staged files for a distribution's archives.
Method publish Do the main publishing work.
Method prepareFreshSeries If there are any new distroseries, prepare them for publishing.
Method setUp Process options, and set up internal state.
Method processDistro Process distribution.
Method main See LaunchpadScript.

Inherited from LaunchpadCronScript:

Method __init__ Construct new LaunchpadScript.
Method get_last_activity Return the last activity, if any.
Method record_activity Record the successful completion of the script.

Inherited from LaunchpadScript (via LaunchpadCronScript):

Method handle_options Undocumented
Method name Enable subclasses to override with command-line arguments.
Method dbuser Enable subclasses to override with command-line arguments.
Method login Super-convenience method that avoids the import.
Method lockfilename Return lockfilename.
Method lockfilepath Undocumented
Method setup_lock Create lockfile.
Method lock_or_die Attempt to lock, and sys.exit(1) if the lock's already taken.
Method unlock Release the lock. Do this before going home.
Method run Actually run the script, executing zcml and initZopeless.
Method lock_and_run Call lock_or_die(), and then run() the script.
Method _init_zca Initialize the ZCA, this can be overridden for testing purposes.
Method _init_db Initialize the database transaction.
def add_my_options(self):
See LaunchpadScript.
def processOptions(self):
Handle command-line options.

Sets `self.distributions` to the `Distribution`s to publish.
def getConfigs(self):
Set up configuration objects for archives to be published.

The configs dict maps each distribution to another dict that maps the archive purposes that are relevant for publishing to the respective archives' configurations.

So: getConfigs[distro][purpose] gives you a config.

def locateIndexesMarker(self, distribution, suite):
Give path for marker file whose presence marks index creation.

The file will be created once the archive indexes for suite have been created. This is how future runs will know that this work is done.

def listSuitesNeedingIndexes(self, distroseries):
Find suites in distroseries that need indexes created.

Checks for the marker left by markIndexCreationComplete.

def markIndexCreationComplete(self, distribution, suite):
Note that archive indexes for suite have been created.

This tells listSuitesNeedingIndexes that no, this suite no longer needs archive indexes to be set up.

def createIndexes(self, distribution, suites):
Create archive indexes for suites of distroseries.
def processAccepted(self, distribution):
Run the process-accepted script.
def getDirtySuites(self, distribution):
Return set of suites that have packages pending publication.
def getDirtySecuritySuites(self, distribution):
List security suites with pending publications.
def rsyncBackupDists(self, distribution):
Populate the backup dists with a copy of distsroot.

Uses "rsync -aH --delete" so that any obsolete files that may still be in the backup dists are cleaned out (bug 58835).

Parametersarchive_purposeThe (purpose of the) archive to copy.
def recoverArchiveWorkingDir(self, archive_config):
Recover working dists dir for archive_config.

If there is a dists directory for archive_config in the working location, kick it back to the backup location.

def recoverWorkingDists(self):
Look for and recover any dists left in transient working state.

An archive's dists directory is temporarily moved into the archive root for running publish-distro. If a previous script run died while in this state, restore the directory to its permanent location.

def setUpDirs(self):
Create archive roots and such if they did not yet exist.
def runPublishDistro(self, distribution, args=, suites=None):
Execute publish-distro.
def publishDistroArchive(self, distribution, archive, security_suites=None):
Publish the results for an archive.
ParametersarchiveArchive to publish.
security_suitesAn optional list of suites to restrict the publishing to.
def runPublishDistroParts(self, distribution, archive):
Execute the publish-distro hooks.
def installDists(self, distribution):
Put the new dists into place, as near-atomically as possible.

For each archive, this switches the dists directory and the backup dists directory around.

def clearEmptyDirs(self, distribution):
Clear out any redundant empty directories.
def runFinalizeParts(self, distribution, security_only=False):
Run the finalize.d parts to finalize publication.
def publishSecurityUploads(self, distribution):
Quickly process just the pending security uploads.

Returns True if publications were made, False otherwise.

def publishDistroUploads(self, distribution):
Publish the distro's complete uploads.
def updateStagedFilesForSuite(self, archive_config, suite):
Install all staged files for a single archive and suite.
ReturnsTrue if any files were installed, otherwise False.
def updateStagedFiles(self, distribution):
Install all staged files for a distribution's archives.
def publish(self, distribution, security_only=False):
Do the main publishing work.
Parameterssecurity_onlyIf True, limit publication to security updates on the main archive. This is much faster, so it makes sense to do a security-only run before the main event to expedite critical fixes.
ReturnsTrue if any publication was made to the archive.
def prepareFreshSeries(self, distribution):
If there are any new distroseries, prepare them for publishing.
ReturnsTrue if a series did indeed still need some preparation, of False for the normal case.
def setUp(self):
Process options, and set up internal state.
def processDistro(self, distribution):
Process distribution.
def main(self):
See LaunchpadScript.
API Documentation for Launchpad, generated by pydoctor at 2020-01-25 00:00:13.