Memcached Memcached Server 1.4.29 Released

eva2000 · Jul 15, 2016

Memcached Server 1.4.29 Updated

Updated both Centmin Mod 123.08stable and 123.09beta01 builds to default to Memcached server 1.4.29 release. Once you update your local server Centmin Mod branch code via centmin.sh menu option 23 outlined below. You will need to exit centmin.sh and re-run centmin.sh again and run centmin.sh menu option 10 to update Memcached server to 1.4.29.

To update your Centmin Mod builds follow instructions at centminmod.com/upgrade.html and respective version threads below:

Centmin Mod 1.2.3-eva2000.08 stable install & update thread

Centmin Mod 123.09beta thread

Example of centmin.sh menu option 23 git updating code.

ReleaseNotes1429 · memcached/memcached Wiki · GitHub

Overview
Improved support for large items over the 1 megabyte default. Also includes a handful of features.

Large items now have much better memory efficiency, and can get very large (potentially hundreds of megabytes) safely.

Many thanks to Premium Bare Metal Servers and Container Hosting - Packet for providing high speed baremetal hardware which was used for performance and burn-in testing of this new feature.

Fixes

fix items with a "0" hash result from failing to evict sometimes.

allow cachedump'ing of slab 63.

fix flakiness of slabs-reassign2.t test.

Fix ancient binprot bug with sets resulting in OOM errors desycning the protocol.

New Features
After over a decade of having an item size target of a single megabyte, items can now safely raise above that. Up to hundreds of megabytes, potentially.

This feature automatically enables if you raise the item size limit above 1mb. You can also enable it manually with a limit of 1mb or less by setting -o slab_chunk_max=16384 (16384 is the recommended default).

Full details in the pull request: Chunked (large) item support by dormando · Pull Request #181 · memcached/memcached · GitHub

In summary:

Memcached uses a slab allocator, which splits up memory into chunks of specific sizes.

The "item size limit" has always been the largest possible slab chunk.

Increasing the size limit thus spreads out available slab classes, reducing memory efficiency.

Now:

The maximum slab chunk size is no longer tied to the maximum item size.

Items larger than the slab chunk size now comprise of multiple chunks of the largest size.

Benefits:

With a smaller maximum chunk size, the slab classes for small items are now more fine grained raising memory efficiency.

Items that approach 1mb can waste more memory as the classes are far apart. Chaining items together can help a lot here.

Performance should be good, as small items are still 1:1 and large items soak more bandwidth, allowing extra room for a very small amount of CPU overhead.

Current caveats:

The system has always been able to store an item after evicting exactly one item. This is no longer the case for large items. If you mix very large items (say 5 megabyte), with many small-ish items (say 100k), with a slab chunk max of 16k, memcached may return more OOM errors than normal while attempting to free enough space to store a new 5 megabyte item.

With a slab chunk max of 16k, an item that requires 16k + 1 byte of memory will use 32k of memory. The larger the item the less effective overhead this has, but at the smaller levels it can lead to problems. Simply leave settings at their defaults or raise the slab_chunk_max if you run into this.

There be dragons if you attempt to store multiple gigabyte items. The item size attributes are 32bit.

These issues should improve in future releases after some refactoring. IE: It should be possible to comprise a chunked item of more than one chunk size, alleviating the last-chunk inefficiency.

To get the most out of this feature, we again recommend you start memcached with -o modern, or many of its new defaults. This allows it to reassign memory as necessary and improves the LRU algorithm significantly.
Click to expand...

Memcached 1.4.29
Code (Text):
echo stats | nc 127.0.0.1 11211
STAT pid 27623
STAT uptime 63
STAT time 1466919513
STAT version 1.4.29
STAT libevent 2.0.22-stable
STAT pointer_size 64
STAT rusage_user 0.167974
STAT rusage_system 0.050992
STAT curr_connections 6
STAT total_connections 7
STAT rejected_connections 0
STAT connection_structures 7
STAT reserved_fds 20
STAT cmd_get 4
STAT cmd_set 3
STAT cmd_flush 0
STAT cmd_touch 0
STAT get_hits 1
STAT get_misses 3
STAT get_expired 0
STAT delete_misses 0
STAT delete_hits 0
STAT incr_misses 0
STAT incr_hits 0
STAT decr_misses 0
STAT decr_hits 0
STAT cas_misses 0
STAT cas_hits 0
STAT cas_badval 0
STAT touch_hits 0
STAT touch_misses 0
STAT auth_cmds 0
STAT auth_errors 0
STAT bytes_read 50226
STAT bytes_written 25014
STAT limit_maxbytes 25165824
STAT accepting_conns 1
STAT listen_disabled_num 0
STAT time_in_listen_disabled_us 0
STAT threads 4
STAT conn_yields 0
STAT hash_power_level 16
STAT hash_bytes 524288
STAT hash_is_expanding 0
STAT slab_reassign_rescues 0
STAT slab_reassign_evictions_nomem 0
STAT slab_reassign_inline_reclaim 0
STAT slab_reassign_busy_items 0
STAT slab_reassign_running 0
STAT slabs_moved 0
STAT lru_crawler_running 0
STAT lru_crawler_starts 2
STAT lru_maintainer_juggles 350
STAT malloc_fails 0
STAT log_worker_dropped 0
STAT log_worker_written 0
STAT log_watcher_skipped 0
STAT log_watcher_sent 0
STAT bytes 25034
STAT curr_items 1
STAT total_items 3
STAT slab_global_page_pool 0
STAT expired_unfetched 0
STAT evicted_unfetched 0
STAT evictions 0
STAT reclaimed 0
STAT crawler_reclaimed 0
STAT crawler_items_checked 2
STAT lrutail_reflocked 0
STAT moves_to_cold 0
STAT moves_to_warm 0
STAT moves_within_lru 0
STAT direct_reclaims 0
END

eva2000 · Jul 15, 2016

huge change for memcached being able to cache items over 1MB !

And AFAIK, with Centmin Mod now using the -o modern flag, the new changes should be utilised out of the box ?

but seems according to Chunked (large) item support by dormando · Pull Request #181 · memcached/memcached · GitHub the default is 1MB item max size unless you pass the -o slab_chunk_max parameter.
Code (Text):
      settings.item_size_max = 1024 * 1024; /* The famous 1MB upper limit. */
+    settings.slab_chunk_size_max = settings.item_size_max;
+    settings.slab_page_size = 1024 * 1024; /* chunks are split from 1MB pages. */
so for 2MB max size
Code (Text):
-o slab_chunk_max=2048
asking on their mailing list for confirmation Google Groups

edit: okay the unit is kilobytes so 2MB = 2048 KB

eva2000 · Jul 15, 2016

sample stats with Centmin Mod already installed memcached-tool

Code (Text):

memcached-tool 127.0.0.1:11211 stats
#127.0.0.1:11211   Field       Value
         accepting_conns           1
               auth_cmds           0
             auth_errors           0
                   bytes      174368
              bytes_read      233371
           bytes_written      135920
              cas_badval           0
                cas_hits           0
              cas_misses           0
               cmd_flush           0
                 cmd_get         132
                 cmd_set         128
               cmd_touch           0
             conn_yields           0
   connection_structures          11
   crawler_items_checked          94
       crawler_reclaimed           0
        curr_connections          10
              curr_items          87
               decr_hits           0
             decr_misses           0
             delete_hits           0
           delete_misses           0
         direct_reclaims           0
       evicted_unfetched           0
               evictions           0
       expired_unfetched           0
             get_expired           0
             get_flushed           0
                get_hits          55
              get_misses          77
              hash_bytes      524288
       hash_is_expanding           0
        hash_power_level          16
               incr_hits           0
             incr_misses           0
                libevent 2.0.22-stable
          limit_maxbytes   268435456
     listen_disabled_num           0
        log_watcher_sent           0
     log_watcher_skipped           0
      log_worker_dropped           0
      log_worker_written           0
     lru_crawler_running           0
      lru_crawler_starts          69
  lru_maintainer_juggles        1032
       lrutail_reflocked           0
            malloc_fails           0
           moves_to_cold         104
           moves_to_warm           9
        moves_within_lru           3
                     pid       21995
            pointer_size          64
               reclaimed           0
    rejected_connections           0
            reserved_fds          20
           rusage_system    0.076666
             rusage_user    0.083333
   slab_global_page_pool           0
slab_reassign_busy_items           0
slab_reassign_chunk_rescues           0
slab_reassign_evictions_nomem           0
slab_reassign_inline_reclaim           0
   slab_reassign_rescues           0
   slab_reassign_running           0
             slabs_moved           0
                 threads           4
                    time  1468589866
time_in_listen_disabled_us           0
       total_connections          15
             total_items         128
              touch_hits           0
            touch_misses           0
                  uptime          67
                 version      1.4.29

my settings, i have 256MB memory allocated to this instance

Code (Text):

memcached-tool 127.0.0.1:11211 settings
#127.0.0.1:11211   Field       Value
       auth_enabled_sasl          no
        binding_protocol auto-negotiate
             cas_enabled         yes
              chunk_size          72
          detail_enabled          no
           domain_socket        NULL
               evictions          on
expirezero_does_not_evict          no
           flush_enabled         yes
           growth_factor        1.08
          hash_algorithm     murmur3
          hashpower_init           0
             hot_lru_pct          32
            idle_timeout           0
                   inter   127.0.0.1
           item_size_max     1048576
             lru_crawler         yes
       lru_crawler_sleep         100
     lru_crawler_tocrawl           0
   lru_maintainer_thread         yes
                maxbytes   268435456
                maxconns        2048
           maxconns_fast         yes
             num_threads           4
     num_threads_per_udp           4
                  oldest           0
          reqs_per_event         200
           slab_automove           1
           slab_reassign         yes
         stat_key_prefix           :
        tail_repair_time           0
             tcp_backlog        2048
                 tcpport       11211
             track_sizes          no
                 udpport       11211
                   umask         700
               verbosity           0
            warm_lru_pct          32
     watcher_logbuf_size      262144
      worker_logbuf_size       65536

or using netcat to get settings

Code (Text):

echo stats settings | nc 127.0.0.1 11211
STAT maxbytes 268435456
STAT maxconns 2048
STAT tcpport 11211
STAT udpport 11211
STAT inter 127.0.0.1
STAT verbosity 0
STAT oldest 0
STAT evictions on
STAT domain_socket NULL
STAT umask 700
STAT growth_factor 1.08
STAT chunk_size 72
STAT num_threads 4
STAT num_threads_per_udp 4
STAT stat_key_prefix :
STAT detail_enabled no
STAT reqs_per_event 200
STAT cas_enabled yes
STAT tcp_backlog 2048
STAT binding_protocol auto-negotiate
STAT auth_enabled_sasl no
STAT item_size_max 1048576
STAT maxconns_fast yes
STAT hashpower_init 0
STAT slab_reassign yes
STAT slab_automove 1
STAT lru_crawler yes
STAT lru_crawler_sleep 100
STAT lru_crawler_tocrawl 0
STAT tail_repair_time 0
STAT flush_enabled yes
STAT hash_algorithm murmur3
STAT lru_maintainer_thread yes
STAT hot_lru_pct 32
STAT warm_lru_pct 32
STAT expirezero_does_not_evict no
STAT idle_timeout 0
STAT watcher_logbuf_size 262144
STAT worker_logbuf_size 65536
STAT track_sizes no
END

pamamolf · Jul 16, 2016

And AFAIK, with Centmin Mod now using the -o modern flag, the new changes should be utilised out of the box ?
Click to expand...

Yes

eva2000 · Jul 16, 2016

pamamolf said: ↑

Yes
Click to expand...

was actually referring to -o slab_chunk_max new feature not the other new features heh

Log in / Register

Forums

Join the community today

Memcached Memcached Server 1.4.29 Released

eva2000 Administrator Staff Member

Memcached Server 1.4.29 Updated

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

pamamolf Premium Member Premium Member

eva2000 Administrator Staff Member

.

Log in / Register

Useful Searches

Join the community today

Memcached Memcached Server 1.4.29 Released

eva2000 Administrator Staff Member

Memcached Server 1.4.29 Updated

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

pamamolf Premium Member Premium Member

eva2000 Administrator Staff Member

.