| This directory contains code uses to measure the performance of the Veyron IPC |
| stack. |
| |
| ================================================================================ |
| |
| The ipc_test.go file uses GO's testing package to run benchmarks. Each |
| benchmark involves one server and one client. The server has two very simple |
| methods that echo the data received from the client back to the client. |
| |
| client ---- Echo(payload) ----> server |
| client <--- return payload ---- server |
| |
| There are two versions of the Echo method: |
| - Echo(payload []byte) ([]byte], error) |
| - EchoStream() <[]byte,[]byte> error |
| |
| The first benchmarks use the non-streaming version of Echo with a varying |
| payload size. The second benchmarks use the streaming version with varying |
| number of chunks and payload sizes. The third one is for measuring the |
| performance with multiple clients hosted in the same process. |
| |
| All these benchmarks create a VC before measurements begin. So, the VC creation |
| overhead is excluded. |
| |
| $ veyron go test -test.bench=. -timeout=1h -test.cpu=1 -test.benchtime=5s \ |
| veyron/runtimes/google/ipc/benchmarks |
| |
| Benchmark____1B 2000 3895391 ns/op 0.00 MB/s |
| Benchmark___10B 2000 3982372 ns/op 0.01 MB/s |
| Benchmark___1KB 5000 3251297 ns/op 0.62 MB/s |
| Benchmark_100KB 2000 6244664 ns/op 32.03 MB/s |
| Benchmark____1_chunk_____1B 5000 4070866 ns/op 0.00 MB/s |
| Benchmark____1_chunk____10B 2000 4242328 ns/op 0.00 MB/s |
| Benchmark____1_chunk____1KB 2000 3679679 ns/op 0.54 MB/s |
| Benchmark____1_chunk___10KB 2000 4070936 ns/op 4.91 MB/s |
| Benchmark___10_chunks____1B 2000 3828552 ns/op 0.01 MB/s |
| Benchmark___10_chunks___10B 5000 3685269 ns/op 0.05 MB/s |
| Benchmark___10_chunks___1KB 2000 6831116 ns/op 2.93 MB/s |
| Benchmark___10_chunks__10KB 1000 9662880 ns/op 20.70 MB/s |
| Benchmark__100_chunks____1B 2000 8938980 ns/op 0.02 MB/s |
| Benchmark__100_chunks___10B 2000 5924969 ns/op 0.34 MB/s |
| Benchmark__100_chunks___1KB 500 37264103 ns/op 5.37 MB/s |
| Benchmark__100_chunks__10KB 100 64999728 ns/op 30.77 MB/s |
| Benchmark__per_chunk____1B 500000 1535312 ns/op 0.00 MB/s |
| Benchmark__per_chunk___10B 2000 9416017 ns/op 0.00 MB/s |
| Benchmark__per_chunk___1KB 1000 7803789 ns/op 0.26 MB/s |
| Benchmark__per_chunk__10KB 1000 7828585 ns/op 2.55 MB/s |
| Benchmark____1B_mux___10_chunks___10B 1000 9233379 ns/op 0.00 MB/s |
| Benchmark____1B_mux___10_chunks___1KB 1000 8639613 ns/op 0.00 MB/s |
| Benchmark____1B_mux__100_chunks___10B 500 30530925 ns/op 0.00 MB/s |
| Benchmark____1B_mux__100_chunks___1KB 200 40886630 ns/op 0.00 MB/s |
| |
| 'Benchmark___1KB' shows that it takes an average of 3.251 ms to |
| execute a simple Echo RPC with a 1 KB payload. |
| |
| 'Benchmark___10_chunks___1KB' shows that a streaming RPC with the |
| same payload (i.e. 10 chunks of 1 KB) takes about 6.831 ms on average. |
| |
| 'Benchmark__per_chunk___1KB' shows that sending a stream of 1 KB chunks |
| takes an average of 7.804 ms per chunk. |
| |
| 'Benchmark____1B_mux___10_chunks___1KB' shows that it takes an average |
| of 9.233 ms to execute a simple Echo RPC with a 1 B payload while streaming |
| 10 chunks of 1 KB payloads continuously in the same process. |
| |
| bm/main.go does the same benchmarks as ipc_test.go but with more varying |
| configurations and optional histogram outputs. |
| |
| $ veyron go run veyron/runtimes/google/ipc/benchmarks/bm/main.go \ |
| -test.cpu=1,2 -test.benchtime=5s -histogram |
| |
| RESULTS.txt has the latest benchmark results with main.go |
| |
| ================================================================================ |
| |
| Running the client and server as separate processes. |
| |
| In this case, we can see the cost of name resolution, creating the VC, etc. in |
| the first RPC. |
| |
| $ $VEYRON_ROOT/veyron/go/bin/bmserver --address=localhost:8888 --acl='{"...":"A"}' |
| |
| (In a different shell) |
| $ $VEYRON_ROOT/veyron/go/bin/bmclient --server=/localhost:8888 --count=10 \ |
| --payload_size=1000 |
| CallEcho 0 64133467 |
| CallEcho 1 766223 |
| CallEcho 2 703860 |
| CallEcho 3 697590 |
| CallEcho 4 601134 |
| CallEcho 5 601142 |
| CallEcho 6 624079 |
| CallEcho 7 644664 |
| CallEcho 8 605195 |
| CallEcho 9 637037 |
| |
| It took about 64 ms to execute the first RPC, and then 0.60-0.70 ms to execute |
| the next ones. |
| |
| |
| On a Raspberry Pi, everything is much slower. The same tests show the following |
| results: |
| |
| $ ./benchmarks.test -test.bench=. -test.cpu=1 -test.benchtime=5s 2>/dev/null |
| PASS |
| Benchmark____1B 500 21316148 ns/op |
| Benchmark___10B 500 23304638 ns/op |
| Benchmark__100B 500 21860446 ns/op |
| Benchmark___1KB 500 24000346 ns/op |
| Benchmark__10KB 200 37530575 ns/op |
| Benchmark_100KB 100 136243310 ns/op |
| Benchmark_N_RPCs____1_chunk_____1B 500 19957506 ns/op |
| Benchmark_N_RPCs____1_chunk____10B 500 22868392 ns/op |
| Benchmark_N_RPCs____1_chunk___100B 500 19635412 ns/op |
| Benchmark_N_RPCs____1_chunk____1KB 500 22572190 ns/op |
| Benchmark_N_RPCs____1_chunk___10KB 500 37570948 ns/op |
| Benchmark_N_RPCs___10_chunks___1KB 100 51670740 ns/op |
| Benchmark_N_RPCs__100_chunks___1KB 50 364938740 ns/op |
| Benchmark_N_RPCs_1000_chunks___1KB 2 3586374500 ns/op |
| Benchmark_1_RPC_N_chunks_____1B 10000 1034042 ns/op |
| Benchmark_1_RPC_N_chunks____10B 5000 1894875 ns/op |
| Benchmark_1_RPC_N_chunks___100B 5000 2857289 ns/op |
| Benchmark_1_RPC_N_chunks____1KB 5000 6465839 ns/op |
| Benchmark_1_RPC_N_chunks___10KB 100 80019430 ns/op |
| Benchmark_1_RPC_N_chunks__100KB Killed |
| |
| The simple 1 KB RPCs take an average of 24 ms. The streaming equivalent takes |
| about 22 ms, and streaming many 1 KB chunks takes about 6.5 ms per chunk. |
| |
| |
| $ ./bmserver --address=localhost:8888 --acl='{"...":"A"}' |
| |
| $ ./bmclient --server=/localhost:8888 --count=10 --payload_size=1000 |
| CallEcho 0 2573406000 |
| CallEcho 1 44669000 |
| CallEcho 2 54442000 |
| CallEcho 3 33934000 |
| CallEcho 4 47985000 |
| CallEcho 5 61324000 |
| CallEcho 6 51654000 |
| CallEcho 7 47043000 |
| CallEcho 8 44995000 |
| CallEcho 9 53166000 |
| |
| On the pi, the first RPC takes ~2.5 sec to execute. |
