Abstract
We present DBToaster, a novel query compilation framework for
producing high performance compiled query executors that
incrementally and continuously answer standing aggregate queries
using in-memory views. DBToaster targets applications that
require efficient main-memory processing of standing queries
(views) fed by high-volume data streams, recursively compiling
view maintenance (VM) queries into simple C++ functions for
evaluating database updates (deltas). While today's VM
algorithms consider the impact of single deltas on view queries
to produce maintenance queries, we recursively consider deltas
of maintenance queries and compile to thoroughly transform
queries into code. Recursive compilation successively elides
certain scans and joins, and eliminates significant query plan
interpreter overheads. In this demonstration, we walk through
our compilation algorithm, and show the significant performance
advantages of our compiled executors over other query
processors. We are able to demonstrate 1--3 orders of magnitude
improvements in processing times for a financial application and
a data warehouse loading application, both implemented across a
wide range of database systems, including PostgreSQL, HSQLDB, a
commercial DBMS 'A', the Stanford STREAM engine, and a
commercial stream processor 'B'.
Users
Please
log in to take part in the discussion (add own reviews or comments).