Y.C. Tay
NU Singapore/ Singapore
Title:
Dynamic equation-based memory allocation for stream processing engines
Abstract:
Applications that continuously process an input data stream are increasingly common. Such streaming applications are often latency-sensitive (e.g. surgery, autonomous driving, high-frequency trading). The input-to-output latency depends on the operators that process the data, and the operator latency in turn depends on the available resources. For an operator that is stateful (e.g. joins) or accesses stored data (e.g. key-value pairs), one crucial resource is memory. This talk presents the use of a Cache Miss Equation to dynamically size the RocksDB cache for the key-value store in Apache Flink. The equation works for different read/write patterns and replacement policies. (Work done in collaboration with Rengan Dou and Richard T.B. Ma.)
Bio:
Y.C. Tay received the B.Sc. degree from the University of Singapore, and the Ph.D. degree from Harvard University. He is a Professor with the Department of Computer Science at the National University of Singapore. He has spent sabbaticals at Princeton, MIT, Cambridge, UCLA, National Taiwan University, Microsoft, Intel, and VMware. He is author of Analytical Performance Modeling for Computer Systems (Springer). His main research interest is performance modeling (database transactions, wireless protocols, Internet traffic, and cache misses). Other interests include database systems (social networks and synthetic generation of data) and the use of local time in distributed computing. He has served on program committees for ACM SIGMETRICS, ACM SIGMOD, VLDB, ICDE, IFIP PERFORMANCE, IFIP NETWORKING, MASCOTS, WWW, ICDCS and ICS. He is also a Senior Associate Editor of the ACM Transactions on Modeling and Performance Evaluation of Computer Systems. He has won several teaching awards.
