Performance Management in Distributed Systems:
Workload Transformation and System Insulation (1997-present)

Web proxies are intermediaries between Web clients and servers in which requests from Web clients are forwarded to Web servers and replies are not only returned to the client but also cached at the intermediary in the hope that future requests can be satisfied from the intermediary without having to contact Web servers. The two main uses of Web proxies are (1) to reduce bandwidth and latency by reducing the number of requests over expensive or slow links, and (2) to reduce the load on Web servers. Caching Web proxies are surprisingly hard to get right: the main challenges are the interaction of the caching subsystem with the underlying storage, the management of a large number of open network connections, and isolation from a variety of unreliable network services. Recently, Web proxies have emerged as a successful way to separate the logic of a Web service that is often much more reliably implemented as an application with low or no parallelism from the need of providing this service in a highly parallel architecture to meet high levels of demand. Web proxies have therefore become focal points of performance management in distributed systems of Web services.

A common approach is to evaluate Web proxy architectures in terms of hit ratio, average throughput, and average latency using traces over relatively short periods of time. I took a different approach by instrumenting Web proxies to reveal the latencies of all network- and system services on which a Web caching service depended [10] and by collecting long-term traces of real world workloads that captured many transient but frequently occurring conditions in the networked environment of Web proxies. We found that under real world conditions a commonly used simple Web caching architecture frequently outperformed a popular and sophisticated architecture [13]. The main reason was that the simple architecture more successfully utilized underlying operating system services while the the more sophisticated architecture duplicated some of those services and did not preserve inherent workload locality in its utilization of storage services. One of the key insight of this work was the importance of isolating Web proxies (and network services in general) from latencies or disruptions of other network services such as the domain name service. In a follow-up study we showed how to further improve interaction of Web proxies and storage systems by an order of magnitude [14]. This work ultimately lead to a patent with Network Appliance [16]. The Sigmetrics paper [13] has been cited in 95 publications, including 3 books (according to Google Scholar).

One common approach to system insulation from latencies is caching. One way to increase the hit ratio of caching is prefetching. Most prefetch approaches are based on past behavior in terms of seconds or minutes. I took a different approach that took advantage of the diurnal pattern of Web cache bandwidth utilization and the fact that bandwidth is purchased according to expected peak bandwidth capacity during a month. The approach is to only prefetch during diurnal low utilization periods (late night until early morning) based on patterns observed during the previous high utilization period [15]. Access patterns are evaluated using off-line machine learning techniques that run at the beginning of each diurnal low utilization period and generate a decision tree for the following prefetch period. This makes this approach robust against changes in Internet traffic over time, traffic idiosyncrasies at a particular location, and the diversity of Internet traffic that lead to complex prefetching rules. The evaluation based on Web traces and scans of 320 popular servers shows that nightly prefetching was able to reduce peak bandwidth utilization by up to 15%.

Many applications have I/O performance requirements but most large, distributed storage systems do not provide performance guarantees nor any kind of performance isolation. Introducing reliable and predictable performance isolation in large, shared storage systems is challenging be- cause of the statefulness of disks (i.e. seek and rotation distance to a target location depends on the current location), the connection management in a large scale storage network with 100,000s of clients and 10,000s of servers, and the interaction of client and server cache pressures with network congestion avoidance algorithms. To address these issues I’m involved in a project that looks at end-to-end performance management in scalable, distributed storage systems [3].

REFERENCES