I am trying to see the performance improvement of HTTP/2 compared to HTTP/1.1 from header compression. I have simulated a network of 300ms latency and 50 KB/s bandwidth (I tried few other combinations as well).
I did multiple attempts of tests which sends different number of requests to sever varying from 1 to 100.
In each test, I load my page multiple times and measure time between request sent and first byte of response received (I use Navigation Timing API for this). There is a reduction of this time between the very first request and consequent requests. However similar reduction is seen with HTTP/1.1 as well.
So there is no visible gain compared to HTTP/1.1. There are considerable improvements to total page load time, but I have no way to say that it is from header compression or multiplexing. So I think measuring time between request sent and first byte of response received should give an accurate measure. But I am unable to experience that.
Find my sample test results
What should be the correct way to measure performance improvement from header compression ?
If you want to know the performance improvements from header compression alone, you need two scenarios whose only difference is header compression. Since HTTP/1.1 doesn't support header compression, you need to make the experiment using HTTP/2. In one, you ask your HPACK encoder to not use any sort of compression (no Huffman encoding, no use of tables at all).
The way I see it, you need quite elaborate server support to measure this over a full-blown website. And you probably want a full-blown website so that your headers are interesting enough. My best suggestion therefore is that you check out NginX source code and edit it's code to basically have HPACK slack as much as possible.
Failing such a comprehensive solution, you can simply measure how many bytes are used in HTTP/2 frames dedicated to headers versus how much bytes are used for headers in HTTP/1.1 fetches of your site. Right now Wireshark and some parsing can help you there. We are planning to add such counters to ShimmerCat, but unfortunately is not there yet (there are quick command line switches to test different protocols though....).
After increasing the header size ~2 KB and emulating low bandwidth and high latency conditions, I was able to see TTFB reduction between first and second requests.