Part of lp.services.webapp.batching View In Hierarchy

A range factory for Storm result sets.

It creates the endpoint memo values from the expressions used in the ORDER BY clause.

Limitations:

The order_by expressions must be Storm PropertyColumn instances, e.g. Bug.title. Simple strings (e.g., resultset.order_by('Bug.id') or general Storm SQL expression are not supported.

The objects representing rows of the PropertyColumn's table must be contained in the result. I.e.,

store.find(Bug.id, Bug.id < 10)

does not work, while

store.find(Bug, Bug.id < 10)

works.

Note: This factory assumes that the result set is fully sorted, i.e. that the set of the column values used for sorting is distinct for each result row.

Method | __init__ | Create a new StormRangeFactory instance. |

Method | empty_resultset | Undocumented |

Method | getOrderBy | Return the order_by expressions of the result set. |

Method | getOrderValuesFor | Return the values of the order_by expressions for the given row. |

Method | getEndpointMemos | See `IRangeFactory` . |

Method | reportError | Undocumented |

Method | parseMemo | Convert the given memo string into a sequence of Python objects. |

Method | reverseSortOrder | Return a list of reversed sort expressions. |

Method | limitsGroupedByOrderDirection | Group sort expressions and memo values by order direction. |

Method | lessThanOrGreaterThanExpression | Return an SQL expression "(expressions) OP (memos)". |

Method | equalsExpressionsFromLimits | Return a list [expression == memo, ...] for the given limits. |

Method | whereExpressionsFromGroupedLimits | Build a sequence of WHERE expressions from the given limits. |

Method | whereExpressions | WHERE expressions for the given sort columns and memos values. |

Method | getSliceFromMemo | Return a result set for the given memo values. |

Method | getSlice | See `IRangeFactory` . |

Method | getSliceByIndex | See `IRangeFactory. |

Method | rough_length | See `IRangeFactory. |

Method | _get_shadowed_list | Undocumented |

def
__init__(self, resultset, error_cb=None):

Create a new StormRangeFactory instance.

Parameters | resultset | A Storm ResultSet instance or a DecoratedResultSet instance. |

error_cb | A function which takes one string as a parameter. It is called when the parameter endpoint_memo of getSlice() does not match the order settings of a resultset. |

def
parseMemo(self, memo):

Convert the given memo string into a sequence of Python objects.

memo should be a JSON string as returned by getEndpointMemos().

Note that memo originates from a URL query parameter and can thus not be trusted to always contain formally valid and consistent data.

Parsing errors or data not matching the sort parameters of the result set are simply ignored.

def
limitsGroupedByOrderDirection(self, sort_expressions, memos):

Group sort expressions and memo values by order direction.

def
lessThanOrGreaterThanExpression(self, expressions, memos):

Return an SQL expression "(expressions) OP (memos)".

OP is >, if the elements of expressions are PropertyColumns; else the elements of expressions are instances of Desc(PropertyColumn) and OP is <.

def
equalsExpressionsFromLimits(self, limits):

Return a list [expression == memo, ...] for the given limits.

def
whereExpressionsFromGroupedLimits(self, limits):

Build a sequence of WHERE expressions from the given limits. limits is a list of tuples (expressions, memos), where expressions is a list of PropertyColumn instances or of instances of Desc(PropertyColumn). Desc(PropertyColumn) and PropertyColumn instances must not appear in the same expressions list. memos are the memo values asociated with the columns in expressions. Given a limits value of [([c11, c12 ...], [m11, m12 ...]), ([c21, c22 ...], [m21, m22 ...]), ... ([cN1, cN2 ...], [mN1, mN2 ...])] this method returns a sequence of these Storm/SQL expressions: * (c11, c12 ...) = (m11, m12 ...) AND (c21, c22 ...) = (m21, m22 ...) AND ... (cN1, cN2 ...) < (mN1, mN2 ...) * (c11, c12 ...) = (m11, m12 ...) AND (c21, c22 ...) = (m21, m22 ...) AND ... (cM1, cM2 ...) < (mM1, mM2 ...) (where M = N - 1) ... * (c11, c12 ...) < (m11, m12 ...) The getSlice() should return rows matching any of these expressions. Note that the result sets returned by each expression are disjuct, hence they can be simply ORed, as well as used in a UNION ALL query.

def
whereExpressions(self, sort_expressions, memos):

WHERE expressions for the given sort columns and memos values.

def
getSliceFromMemo(self, size, memo):

Return a result set for the given memo values.

Note that at least two other implementations are possible: Instead of OR-combining the expressions returned by whereExpressions(), these expressions could be used for separate SELECTs which are then merged with UNION ALL.

We could also issue separate Storm queries for each expression and combine the results here.

Which variant is more efficient is yet unknown; it may differ between different queries.