lp.services.webapp.batching.StormRangeFactory : API documentation

A range factory for Storm result sets.

It creates the endpoint memo values from the expressions used in the ORDER BY clause.

Limitations:

The order_by expressions must be Storm PropertyColumn instances, e.g. Bug.title. Simple strings (e.g., resultset.order_by('Bug.id') or general Storm SQL expression are not supported.
The objects representing rows of the PropertyColumn's table must be contained in the result. I.e.,

store.find(Bug.id, Bug.id < 10)

does not work, while

store.find(Bug, Bug.id < 10)

works.

Note: This factory assumes that the result set is fully sorted, i.e. that the set of the column values used for sorting is distinct for each result row.

Method	__init__	Create a new StormRangeFactory instance.
Method	empty_resultset	Undocumented
Method	getOrderBy	Return the order_by expressions of the result set.
Method	getOrderValuesFor	Return the values of the order_by expressions for the given row.
Method	getEndpointMemos	See `IRangeFactory`.
Method	reportError	Undocumented
Method	parseMemo	Convert the given memo string into a sequence of Python objects.
Method	reverseSortOrder	Return a list of reversed sort expressions.
Method	limitsGroupedByOrderDirection	Group sort expressions and memo values by order direction.
Method	lessThanOrGreaterThanExpression	Return an SQL expression "(expressions) OP (memos)".
Method	equalsExpressionsFromLimits	Return a list [expression == memo, ...] for the given limits.
Method	whereExpressionsFromGroupedLimits	Build a sequence of WHERE expressions from the given limits.
Method	whereExpressions	WHERE expressions for the given sort columns and memos values.
Method	getSliceFromMemo	Return a result set for the given memo values.
Method	getSlice	See `IRangeFactory`.
Method	getSliceByIndex	See `IRangeFactory.
Method	rough_length	See `IRangeFactory.
Method	_get_shadowed_list	Undocumented

def __init__(self, resultset, error_cb=None):

Create a new StormRangeFactory instance.

Parameters	resultset	A Storm ResultSet instance or a DecoratedResultSet instance.
	error_cb	A function which takes one string as a parameter. It is called when the parameter endpoint_memo of getSlice() does not match the order settings of a resultset.

def parseMemo(self, memo):

Convert the given memo string into a sequence of Python objects.

memo should be a JSON string as returned by getEndpointMemos().

Note that memo originates from a URL query parameter and can thus not be trusted to always contain formally valid and consistent data.

Parsing errors or data not matching the sort parameters of the result set are simply ignored.

def lessThanOrGreaterThanExpression(self, expressions, memos):

Return an SQL expression "(expressions) OP (memos)".

OP is >, if the elements of expressions are PropertyColumns; else the elements of expressions are instances of Desc(PropertyColumn) and OP is <.

def whereExpressionsFromGroupedLimits(self, limits):

Build a sequence of WHERE expressions from the given limits.

limits is a list of tuples (expressions, memos), where
expressions is a list of PropertyColumn instances or of
instances of Desc(PropertyColumn). Desc(PropertyColumn)
and PropertyColumn instances must not appear in the same
expressions list.

memos are the memo values asociated with the columns in
expressions.

Given a limits value of
    [([c11, c12 ...], [m11, m12 ...]),
     ([c21, c22 ...], [m21, m22 ...]),
     ...
     ([cN1, cN2 ...], [mN1, mN2 ...])]

this method returns a sequence of these Storm/SQL expressions:

    * (c11, c12 ...) = (m11, m12 ...) AND
      (c21, c22 ...) = (m21, m22 ...) AND
      ...
      (cN1, cN2 ...) < (mN1, mN2 ...)
    * (c11, c12 ...) = (m11, m12 ...) AND
      (c21, c22 ...) = (m21, m22 ...) AND
      ...
      (cM1, cM2 ...) < (mM1, mM2 ...)

      (where M = N - 1)
    ...
    * (c11, c12 ...) < (m11, m12 ...)

The getSlice() should return rows matching any of these
expressions. Note that the result sets returned by each
expression are disjuct, hence they can be simply ORed,
as well as used in a UNION ALL query.

def getSliceFromMemo(self, size, memo):

Return a result set for the given memo values.

Note that at least two other implementations are possible: Instead of OR-combining the expressions returned by whereExpressions(), these expressions could be used for separate SELECTs which are then merged with UNION ALL.

We could also issue separate Storm queries for each expression and combine the results here.

Which variant is more efficient is yet unknown; it may differ between different queries.

l.s.w.b.StormRangeFactory : class documentation