Want to subscribe to topics you're interested in?
Become a Member

Google's perfkit for testing cloud infrastructure.

Discussion in 'System Administration' started by s.molinari, Feb 15, 2015.

  1. s.molinari

    s.molinari New Member

    10
    4
    3
    Aug 31, 2014
    Ratings:
    +4
    Local Time:
    11:49 AM
    • Like Like x 1
  2. eva2000

    eva2000 Administrator Staff Member

    28,995
    6,579
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,769
    Local Time:
    7:49 PM
    Nginx 1.13.x
    MariaDB 5.5
    Nice thanks for heads up Scott..

    PerfKit Benchmarker sounds a bit like my bench.centminmod.com benchmark script (used to evaluate VPS/Server performance and connectivity) but on steroids :)

    edit: more media coverage
    seems right now PerfKit Benchmarker only supports Amazon AWS, Microsoft Azure and Google's GCE platforms though

    nice set of benchmarks though - might borrow some for integration into bench.centminmod.com (already have a few like fio, ping, unixbench and also have basic support for sysbench) :D

     
    Last edited: Feb 15, 2015
    • Like Like x 1
  3. eva2000

    eva2000 Administrator Staff Member

    28,995
    6,579
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,769
    Local Time:
    7:49 PM
    Nginx 1.13.x
    MariaDB 5.5
    Going to borrow memtier_benchmark tool can benchmark both redis and memcached servers. I use to use Twemperf from twitter for memcached server benchmarks.

    memcached server benchmarked

    Code:
    memtier_benchmark -P memcache_text -s 127.0.0.1 -p 11211 --random-data --data-size-range=4-204 --data-size-pattern=S --key-minimum=200 --key-maximum=400 --key-pattern=G:G --key-stddev=10 --key-median=300 2>&1 > memtier_benchmark.log
    
    [RUN #1] Preparing benchmark client...
    [RUN #1] Launching threads now...
    [RUN #1 100%,  82 secs]  0 threads:     2000000 ops,   24720 (avg:   24334) ops/sec, 3.50MB/sec (avg: 3.47MB/sec),  8.08 (avg:  8.16) msec latency
    
    Code:
    cat memtier_benchmark.log
    4         Threads
    50        Connections per thread
    10000     Requests per thread
    Type        Ops/sec     Hits/sec   Misses/sec      Latency       KB/sec
    ------------------------------------------------------------------------
    Sets        2214.42          ---          ---      8.17400       299.06
    Gets       22119.88     21900.87       219.01      8.15900      3257.78
    Totals     24334.30     21900.87       219.01      8.16000      3556.83
    
    
    Request Latency Distribution
    Type        <= msec      Percent
    ------------------------------------------------------------------------
    SET               0         0.11
    SET               1         1.52
    SET               2         2.40
    SET               3         3.15
    SET               4         4.74
    SET               5        10.27
    SET               6        25.38
    SET               7        50.51
    SET               8        73.71
    SET               9        86.63
    SET              10        93.06
    SET              11        96.13
    SET              12        97.47
    SET              13        98.09
    SET              14        98.49
    SET              15        98.71
    SET              16        98.92
    SET              17        99.10
    SET              18        99.25
    SET              19        99.39
    SET              20        99.47
    SET              21        99.56
    SET              22        99.64
    SET              23        99.68
    SET              24        99.71
    SET              25        99.74
    SET              26        99.78
    SET              27        99.80
    SET              28        99.81
    SET              29        99.84
    SET              30        99.85
    SET              31        99.87
    SET              32        99.89
    SET              33        99.90
    SET              34        99.91
    SET              35        99.92
    SET              36        99.93
    SET              37        99.94
    SET              38        99.94
    SET              39        99.95
    SET              40        99.95
    SET              41        99.96
    SET              42        99.96
    SET              43        99.97
    SET              44        99.97
    SET              45        99.97
    SET              46        99.97
    SET              47        99.98
    SET              48        99.99
    SET              49        99.99
    SET              53        99.99
    SET              54        99.99
    SET              59       100.00
    SET              60       100.00
    ---
    GET               0         0.10
    GET               1         1.38
    GET               2         2.31
    GET               3         3.09
    GET               4         4.73
    GET               5        10.04
    GET               6        25.10
    GET               7        50.69
    GET               8        73.53
    GET               9        86.73
    GET              10        93.17
    GET              11        96.26
    GET              12        97.66
    GET              13        98.32
    GET              14        98.69
    GET              15        98.93
    GET              16        99.10
    GET              17        99.25
    GET              18        99.37
    GET              19        99.47
    GET              20        99.55
    GET              21        99.63
    GET              22        99.68
    GET              23        99.72
    GET              24        99.75
    GET              25        99.78
    GET              26        99.80
    GET              27        99.82
    GET              28        99.83
    GET              29        99.85
    GET              30        99.87
    GET              31        99.88
    GET              32        99.89
    GET              33        99.90
    GET              34        99.91
    GET              35        99.91
    GET              36        99.92
    GET              37        99.93
    GET              38        99.94
    GET              39        99.94
    GET              40        99.95
    GET              41        99.95
    GET              42        99.95
    GET              43        99.96
    GET              44        99.96
    GET              45        99.96
    GET              46        99.97
    GET              47        99.98
    GET              48        99.98
    GET              49        99.99
    GET              50        99.99
    GET              51        99.99
    GET              52        99.99
    GET              53        99.99
    GET              54        99.99
    GET              56       100.00
    GET              57       100.00
    GET              58       100.00
    GET              59       100.00
    GET              60       100.00
    GET              61       100.00
    GET              62       100.00
    GET              63       100.00
    
    redis 2.8.19 benchmarked

    Code:
    memtier_benchmark -P redis -s 127.0.0.1 -p 6379 --random-data --data-size-range=4-204 --data-size-pattern=S --key-minimum=200 --key-maximum=400 --key-pattern=G:G --key-stddev=10 --key-median=300 2>&1 > memtier_benchmark.redis.log      
    [RUN #1] Preparing benchmark client...
    [RUN #1] Launching threads now...
    [RUN #1 100%,  60 secs]  0 threads:     2000000 ops,   32549 (avg:   33281) ops/sec, 4.32MB/sec (avg: 4.30MB/sec),  6.13 (avg:  5.99) msec latency
    
    Code:
    cat memtier_benchmark.redis.log
    4         Threads
    50        Connections per thread
    10000     Requests per thread
    Type        Ops/sec     Hits/sec   Misses/sec      Latency       KB/sec
    ------------------------------------------------------------------------
    Sets        3032.24          ---          ---      5.98800       436.15
    Gets       30289.11     28122.66      2166.45      5.99200      3967.32
    Totals     33321.35     28122.66      2166.45      5.99200      4403.47
    
    
    Request Latency Distribution
    Type        <= msec      Percent
    ------------------------------------------------------------------------
    SET               0         0.07
    SET               1         1.11
    SET               2         5.00
    SET               3        15.42
    SET               4        34.90
    SET               5        57.72
    SET               6        74.66
    SET               7        84.81
    SET               8        90.85
    SET               9        94.73
    SET              10        96.87
    SET              11        98.07
    SET              12        98.84
    SET              13        99.32
    SET              14        99.62
    SET              15        99.79
    SET              16        99.85
    SET              17        99.89
    SET              18        99.91
    SET              19        99.93
    SET              20        99.96
    SET              21        99.97
    SET              22        99.99
    SET              23        99.99
    SET              24       100.00
    SET              25       100.00
    SET              28       100.00
    ---
    GET               0         0.05
    GET               1         1.03
    GET               2         4.86
    GET               3        15.35
    GET               4        34.74
    GET               5        57.71
    GET               6        74.72
    GET               7        84.78
    GET               8        90.81
    GET               9        94.71
    GET              10        96.85
    GET              11        98.09
    GET              12        98.86
    GET              13        99.33
    GET              14        99.60
    GET              15        99.78
    GET              16        99.84
    GET              17        99.87
    GET              18        99.91
    GET              19        99.93
    GET              20        99.96
    GET              21        99.97
    GET              22        99.99
    GET              23        99.99
    GET              24       100.00
    GET              25       100.00
    GET              26       100.00
    GET              28       100.00
    
    redis AOF enabled with appendfsync everysec

    Code:
    memtier_benchmark -P redis -s 127.0.0.1 -p 6379 --random-data --data-size-range=4-204 --data-size-pattern=S --key-minimum=200 --key-maximum=400 --key-pattern=G:G --key-stddev=10 --key-median=300 2>&1 > memtier_benchmark.redis.aofenabled.log
    [RUN #1] Preparing benchmark client...
    [RUN #1] Launching threads now...
    [RUN #1 100%,  60 secs]  0 threads:     2000000 ops,   34892 (avg:   32788) ops/sec, 4.64MB/sec (avg: 4.23MB/sec),  5.72 (avg:  6.08) msec latency
    
    Code:
    cat memtier_benchmark.redis.aofenabled.log
    4         Threads
    50        Connections per thread
    10000     Requests per thread
    Type        Ops/sec     Hits/sec   Misses/sec      Latency       KB/sec
    ------------------------------------------------------------------------
    Sets        2983.83          ---          ---      6.08000       429.19
    Gets       29805.53     27679.11      2126.42      6.08100      3904.52
    Totals     32789.37     27679.11      2126.42      6.08100      4333.70
    
    
    Request Latency Distribution
    Type        <= msec      Percent
    ------------------------------------------------------------------------
    SET               0         0.07
    SET               1         0.75
    SET               2         3.92
    SET               3        13.54
    SET               4        32.11
    SET               5        55.51
    SET               6        73.98
    SET               7        84.69
    SET               8        90.85
    SET               9        94.78
    SET              10        96.98
    SET              11        98.10
    SET              12        98.80
    SET              13        99.25
    SET              14        99.56
    SET              15        99.74
    SET              16        99.81
    SET              17        99.86
    SET              18        99.91
    SET              19        99.93
    SET              20        99.94
    SET              21        99.96
    SET              22        99.97
    SET              23        99.97
    SET              24        99.98
    SET              25        99.98
    SET              26        99.98
    SET              27        99.99
    SET              28        99.99
    SET              29        99.99
    SET              59       100.00
    SET              60       100.00
    SET              61       100.00
    SET              62       100.00
    ---
    GET               0         0.05
    GET               1         0.75
    GET               2         3.89
    GET               3        13.46
    GET               4        32.12
    GET               5        55.56
    GET               6        74.01
    GET               7        84.60
    GET               8        90.84
    GET               9        94.74
    GET              10        96.98
    GET              11        98.12
    GET              12        98.83
    GET              13        99.27
    GET              14        99.57
    GET              15        99.73
    GET              16        99.80
    GET              17        99.85
    GET              18        99.90
    GET              19        99.92
    GET              20        99.94
    GET              21        99.97
    GET              22        99.98
    GET              23        99.98
    GET              24        99.98
    GET              25        99.98
    GET              26        99.99
    GET              27        99.99
    GET              28        99.99
    GET              29        99.99
    GET              30       100.00
    GET              56       100.00
    GET              57       100.00
    GET              59       100.00
    GET              60       100.00
    GET              61       100.00
    GET              62       100.00
    
    my custom redisinfo.sh script
    Code:
    ./redisinfo.sh
    --------------------------------------------------------------------------
    redis-cli 2.8.19
    --------------------------------------------------------------------------
    redis-cli info
    # Server
    redis_version:2.8.19
    redis_git_sha1:00000000
    redis_git_dirty:0
    redis_build_id:8012391549b7b328
    redis_mode:standalone
    os:Linux 2.6.32-504.3.3.el6.x86_64 x86_64
    arch_bits:64
    multiplexing_api:epoll
    gcc_version:4.4.7
    process_id:26296
    run_id:d370932bd82e7aeab4c750b2b2803131cc1470ab
    tcp_port:6379
    uptime_in_seconds:224
    uptime_in_days:0
    hz:10
    lru_clock:14698116
    config_file:/etc/redis.conf
    
    # Clients
    connected_clients:1
    client_longest_output_list:0
    client_biggest_input_buf:0
    blocked_clients:0
    
    # Memory
    used_memory:810848
    used_memory_human:791.84K
    used_memory_rss:5468160
    used_memory_peak:810848
    used_memory_peak_human:791.84K
    used_memory_lua:35840
    mem_fragmentation_ratio:6.74
    mem_allocator:jemalloc-3.6.0
    
    # Persistence
    loading:0
    rdb_changes_since_last_save:0
    rdb_bgsave_in_progress:0
    rdb_last_save_time:1423984036
    rdb_last_bgsave_status:ok
    rdb_last_bgsave_time_sec:-1
    rdb_current_bgsave_time_sec:-1
    aof_enabled:0
    aof_rewrite_in_progress:0
    aof_rewrite_scheduled:0
    aof_last_rewrite_time_sec:-1
    aof_current_rewrite_time_sec:-1
    aof_last_bgrewrite_status:ok
    aof_last_write_status:ok
    
    # Stats
    total_connections_received:3
    total_commands_processed:2
    instantaneous_ops_per_sec:0
    total_net_input_bytes:42
    total_net_output_bytes:3720
    instantaneous_input_kbps:0.01
    instantaneous_output_kbps:1.12
    rejected_connections:0
    sync_full:0
    sync_partial_ok:0
    sync_partial_err:0
    expired_keys:0
    evicted_keys:0
    keyspace_hits:0
    keyspace_misses:0
    pubsub_channels:0
    pubsub_patterns:0
    latest_fork_usec:0
    
    # Replication
    role:master
    connected_slaves:0
    master_repl_offset:0
    repl_backlog_active:0
    repl_backlog_size:1048576
    repl_backlog_first_byte_offset:0
    repl_backlog_histlen:0
    
    # CPU
    used_cpu_sys:0.05
    used_cpu_user:0.39
    used_cpu_sys_children:0.00
    used_cpu_user_children:0.00
    
    # Keyspace
    
    --------------------------------------------------------------------------
    redis-cli --intrinsic-latency 10
    Max latency so far: 8 microseconds.
    Max latency so far: 22 microseconds.
    Max latency so far: 63 microseconds.
    Max latency so far: 74 microseconds.
    Max latency so far: 743 microseconds.
    Max latency so far: 1176 microseconds.
    Max latency so far: 2541 microseconds.
    Max latency so far: 7983 microseconds.
    Max latency so far: 8382 microseconds.
    
    1497243 total runs (avg latency: 6.6789 microseconds / 66789.43 nanoseconds per run).
    Worst run took 1255x longer than the average latency.
    
    --------------------------------------------------------------------------
    vmstat 1 10
    procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu-----
    r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
    0  0      0 233104 118092 481232    0    0    67    48  147  582  3  3 95  0  0
    0  0      0 233096 118092 481252    0    0     0    68   66   60  0  1 99  0  0
    0  0      0 233104 118100 481236    0    0     0    28   66   81  0  0 100  0  0
    0  0      0 233104 118100 481240    0    0     0     0   54   57  0  0 100  0  0
    1  0      0 233104 118100 481240    0    0     0     0   61   64  0  1 99  0  0
    0  0      0 233104 118100 481240    0    0     0     0   70   77  0  0 100  0  0
    0  0      0 233104 118100 481240    0    0     0     0   56   67  0  0 100  0  0
    2  0      0 233104 118100 481240    0    0     0     0   47   54  0  0 100  0  0
    0  0      0 233104 118100 481240    0    0     0     0   46   55  1  0 99  0  0
    1  0      0 233104 118100 481240    0    0     0     0   54   54  0  0 100  0  0
    --------------------------------------------------------------------------
    
    smem -kU redis
      PID User     Command                         Swap      USS      PSS      RSS
    26296 redis    /usr/bin/redis-server 127.0        0     4.7M     4.7M     5.2M
    --------------------------------------------------------------------------
    
    --------------------------------------------------------------------------
    smem -kmU redis
    Map                                       PIDs   AVGPSS      PSS
    /usr/lib/locale/locale-archive               1        0        0
    [vdso]                                       1        0        0
    [vsyscall]                                   1        0        0
    /lib64/libdl-2.12.so                         1     8.0K     8.0K
    /lib64/libm-2.12.so                          1     8.0K     8.0K
    /lib64/ld-2.12.so                            1     9.0K     9.0K
    /lib64/libpthread-2.12.so                    1    12.0K    12.0K
    [stack]                                      1    20.0K    20.0K
    /lib64/libc-2.12.so                          1    47.0K    47.0K
    /usr/lib64/libjemalloc.so.1                  1    96.0K    96.0K
    /usr/bin/redis-server                        1   360.0K   360.0K
    <anonymous>                                  1     4.1M     4.1M
    --------------------------------------------------------------------------
    
    --------------------------------------------------------------------------
    smem -tpk
      PID User     Command                         Swap      USS      PSS      RSS
    2403 root     /sbin/mingetty /dev/tty1           0    80.0K    99.0K   584.0K
    2407 root     /sbin/mingetty /dev/tty3           0    80.0K    99.0K   584.0K
    2409 root     /sbin/mingetty /dev/tty4           0    80.0K    99.0K   584.0K
    2411 root     /sbin/mingetty /dev/tty5           0    80.0K    99.0K   584.0K
    2418 root     /sbin/mingetty /dev/tty6           0    80.0K    99.0K   584.0K
    2405 root     /sbin/mingetty /dev/tty2           0    84.0K   103.0K   588.0K
      974 nsd      /usr/local/sbin/nsd -c /etc        0    84.0K   224.0K   908.0K
      977 nsd      /usr/local/sbin/nsd -c /etc        0   152.0K   263.0K   604.0K
    2415 root     /sbin/udevd -d                     0    80.0K   303.0K   944.0K
    2416 root     /sbin/udevd -d                     0    80.0K   303.0K   944.0K
      357 root     /sbin/udevd -d                     0   108.0K   312.0K   976.0K
      914 root     auditd                             0   308.0K   327.0K   776.0K
    26321 root     /bin/bash ./redisinfo.sh           0   216.0K   401.0K     1.4M
    1232 root     /bin/sh /usr/bin/mysqld_saf        0   316.0K   504.0K     1.6M
    1026 dbus     dbus-daemon --system               0   528.0K   546.0K   908.0K
        1 root     /sbin/init                         0   440.0K   551.0K     1.4M
    2390 root     crond                              0   644.0K   676.0K     1.2M
    1209 root     /usr/sbin/sshd                     0   708.0K   794.0K     1.2M
    1113 root     /usr/sbin/VBoxService              0   700.0K   798.0K     1.5M
      934 root     /sbin/rsyslogd -i /var/run/        0   876.0K   908.0K     1.5M
    1218 ntp      ntpd -u ntp:ntp -p /var/run        0   956.0K     1.1M     2.0M
    2362 root     /usr/libexec/postfix/master        0  1008.0K     1.4M     3.3M
    2374 postfix  pickup -l -t fifo -u               0     1.0M     1.4M     3.3M
    2375 postfix  qmgr -l -t fifo -u                 0     1.1M     1.5M     3.4M
    2378 root     nginx: master process /usr/        0   788.0K     1.7M     2.8M
    1124 root     /usr/sbin/console-kit-daemo        0     1.7M     1.8M     2.6M
    2425 root     sshd: root@pts/0,pts/              0     2.2M     2.6M     4.3M
    2427 root     -bash                              0     2.4M     2.7M     4.0M
    11654 root     -bash                              0     2.4M     2.7M     4.0M
    2268 root     /usr/sbin/haveged -w 3072 -        0     3.1M     3.1M     3.5M
      976 nsd      /usr/local/sbin/nsd -c /etc        0     4.2M     4.3M     4.7M
    26296 redis    /usr/bin/redis-server 127.0        0     4.7M     4.7M     5.2M
    2380 nginx    nginx: worker process              0     4.0M     4.9M     6.3M
    26354 root     python /usr/bin/smem -tpk          0     5.8M     5.9M     6.6M
    6215 nobody   /usr/local/bin/memcached -d        0     6.1M     6.1M     6.5M
    6230 root     php-fpm: master process (/u        0     6.2M     6.3M     6.8M
    1016 root     lfd - sleepin                      0    17.0M    17.1M    17.7M
    2255 mysql    /usr/sbin/mysqld --basedir=        0    38.6M    38.8M    40.2M
    -------------------------------------------------------------------------------
       38 9                                           0   108.8M   115.4M   146.2M
    --------------------------------------------------------------------------
    
    --------------------------------------------------------------------------
    free -mlt
                 total       used       free     shared    buffers     cached
    Mem:           996        769        227         21        115        470
    Low:           996        769        227
    High:            0          0          0
    -/+ buffers/cache:        183        812
    Swap:         1839          0       1839
    Total:        2836        769       2067
    --------------------------------------------------------------------------
    I'm liking memtier_benchmark

    with test paramters
    Code:
     memtier_benchmark -P memcache_text -s 127.0.0.1 -p 11211 --random-data --data-size-range=4-204 --data-size-pattern=S --key-minimum=200 --key-maximum=400 --key-pattern=G:G --key-stddev=10 --key-median=300 2>&1 > memtier_benchmark.log
    default of box memcached server benchmark for Centmin Mod LEMP stack
    Code:
    cat memtier_benchmark.log
    4         Threads
    50        Connections per thread
    10000     Requests per thread
    Type        Ops/sec     Hits/sec   Misses/sec      Latency       KB/sec
    ------------------------------------------------------------------------
    Sets        2214.42          ---          ---      8.17400       299.06
    Gets       22119.88     21900.87       219.01      8.15900      3257.78
    Totals     24334.30     21900.87       219.01      8.16000      3556.83
    do some tuning and tweaking of memcached server on Centmin Mod LEMP stack
    Code:
    cat memtier_benchmark.log
    4         Threads
    50        Connections per thread
    10000     Requests per thread
    Type        Ops/sec     Hits/sec   Misses/sec      Latency       KB/sec
    ------------------------------------------------------------------------
    Sets        3455.32          ---          ---      5.22500       466.64
    Gets       34515.24     32113.53      2401.71      5.22100      4820.50
    Totals     37970.56     32113.53      2401.71      5.22200      5287.13
    more tweaking for memcached server
    Code:
    cat memtier_benchmark.log
    4         Threads
    50        Connections per thread
    10000     Requests per thread
    Type        Ops/sec     Hits/sec   Misses/sec      Latency       KB/sec
    ------------------------------------------------------------------------
    Sets        3545.16          ---          ---      5.21700       478.77
    Gets       35412.66     33082.59      2330.07      5.21300      4962.82
    Totals     38957.82     33082.59      2330.07      5.21400      5441.59
     
    Last edited: Feb 15, 2015