Scientific benchmarking of parallel computing systems: twelve ways to tell the masses when reporting performance results

T Hoefler, R Belli - Proceedings of the international conference for high …, 2015 - dl.acm.org
Measuring and reporting performance of parallel computers constitutes the basis for
scientific advancement of high-performance computing (HPC). Most scientific reports show …

Performance overhead comparison between hypervisor and container based virtualization

Z Li, M Kihl, Q Lu, JA Andersson - 2017 IEEE 31st International …, 2017 - ieeexplore.ieee.org
The current virtualization solution in the Cloud widely relies on hypervisor-based
technologies. Along with the recent popularity of Docker, the container-based virtualization …

uFLIP: Understanding flash IO patterns

L Bouganim, B Jónsson, P Bonnet - arXiv preprint arXiv:0909.1780, 2009 - arxiv.org
Does the advent of flash devices constitute a radical change for secondary storage? How
should database systems adapt to this new form of secondary storage? Before we can …

Dynamic software updates: a VM-centric approach

S Subramanian, M Hicks, KS McKinley - Proceedings of the 30th ACM …, 2009 - dl.acm.org
Software evolves to fix bugs and add features. Stopping and restarting programs to apply
changes is inconvenient and often costly. Dynamic software updating (DSU) addresses this …

What's wrong with my benchmark results? Studying bad practices in JMH benchmarks

D Costa, CP Bezemer, P Leitner… - IEEE Transactions on …, 2019 - ieeexplore.ieee.org
Microbenchmarking frameworks, such as Java's Microbenchmark Harness (JMH), allow
developers to write fine-grained performance test suites at the method or statement level …

Write-rationing garbage collection for hybrid memories

S Akram, JB Sartor, KS McKinley, L Eeckhout - ACM SIGPLAN Notices, 2018 - dl.acm.org
Emerging Non-Volatile Memory (NVM) technologies offer high capacity and energy
efficiency compared to DRAM, but suffer from limited write endurance and longer latencies …

A data generator for cloud-scale benchmarking

T Rabl, M Frank, HM Sergieh, H Kosch - … Evaluation, Measurement and …, 2011 - Springer
In many fields of research and business data sizes are breaking the petabyte barrier. This
imposes new problems and research possibilities for the database community. Usually, data …

CEEM: A practical methodology for cloud services evaluation

Z Li, L OBrien, H Zhang - 2013 IEEE Ninth World Congress on …, 2013 - ieeexplore.ieee.org
Given an increasing number of Cloud services available in the market, evaluating candidate
Cloud services is crucial and beneficial for both service customers (eg cost benefit analysis) …

Looking back on the language and hardware revolutions: measured power, performance, and scaling

H Esmaeilzadeh, T Cao, Y Xi, SM Blackburn… - ACM SIGARCH …, 2011 - dl.acm.org
This paper reports and analyzes measured chip power and performance on five process
technology generations executing 61 diverse benchmarks with a rigorous methodology. We …

Repeatability, reproducibility, and rigor in systems research

J Vitek, T Kalibera - Proceedings of the ninth ACM international …, 2011 - dl.acm.org
Computer systems research spans sub-disciplines that include embedded and real-time
systems, compilers, networking, and operating systems. Our contention is that a number of …