Rethinking the application-database interface

Abstract: From social networking websites to bank transactions, we interact with data-intensive applications every day. Such applications are typically hosted on an application server that interacts with a database server to manipulate persistent data. To make such applications efficient, developers face the daunting task of mastering the intricacies of both the programming system and the database management system. For instance, while many application features can be implemented in either the application or pushed into the database, it is difficult for a programmer to decide where to place a given computation as the decision is typically workload-driven. Unfortunately, making the wrong choice often results in drastic performance hit.

In this talk, I will show how examining both the programming system and the database management system at the same time allows us to significantly improve the performance of data-intensive applications. To illustrate such cross-system optimization opportunities, I have designed, built, and evaluated three systems: Query By Synthesis, a tool that converts functionality written as imperative code into relational queries; Sloth, a system that combines queries embedded in applications into batches; and Pyxis, a system that seamlessly moves computation between application and database servers. Using real-world examples, I will show that these systems allow orders of magnitude performance improvement and graceful adaptation to changing server environments while preserving the high-level programming interface to the developer. My work has been featured in ACM Tech News and included in graduate seminar courses. We have also received requests from more than 50 individuals and institutions who are interested in trying out our tools.

Slides