Beta Branch addons/haproxy.sh preview

eva2000 · Aug 27, 2017

Haproxy load balancer is another 'to do' list item I am planning for future Centmin Mod versions. Most likely will be added as an addon at addons/haproxy.sh when eventually ready.

This is just a preview thread

Haproxy 1.7.9 with OpenSSL 1.1.0f + Lua + zlib + Pcre/Pcre Jit + Transparent Proxy

Code (Text):

haproxy -vv
HA-Proxy version 1.7.9 2017/08/18
Copyright 2000-2017 Willy Tarreau <willy@haproxy.org>

Build options :
  TARGET  = linux2628
  CPU     = native
  CC      = gcc
  CFLAGS  = -march=native -m64 -march=x86-64 -O2 -g
  OPTIONS = USE_LINUX_SPLICE=1 USE_LINUX_TPROXY=1 USE_ZLIB=1 USE_REGPARM=1 USE_OPENSSL=1 USE_LUA=1 USE_PCRE=1 USE_PCRE_JIT=1

Default settings :
  maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Encrypted password support via crypt(3): yes
Built with zlib version : 1.2.11
Running on zlib version : 1.2.11
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with OpenSSL version : OpenSSL 1.1.0f  25 May 2017
Running on OpenSSL version : OpenSSL 1.1.0f  25 May 2017
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports prefer-server-ciphers : yes
Built with PCRE version : 8.41 2017-07-05
Running on PCRE version : 8.41 2017-07-05
PCRE library supports JIT : yes
Built with Lua version : Lua 5.3.4
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available filters :
        [COMP] compression
        [TRACE] trace
        [SPOE] spoe

Haproxy statistics

eva2000 · Aug 27, 2017

haproxytool

haproxytool command line support

Code (Text):

       
haproxytool frontend -c --file /var/run/haproxy/haproxy.sock
www 20000
haproxy-stats 20000

Code (Text):

haproxytool frontend -s --file /var/run/haproxy/haproxy.sock
www OPEN
haproxy-stats OPEN

Code (Text):

haproxytool frontend -r --file /var/run/haproxy/haproxy.sock
www 3730
haproxy-stats 27

Code (Text):

haproxytool backend -s --file /var/run/haproxy/haproxy.sock
haproxy-stats UP
spider_backend UP
backend_tarpit UP
www_backend UP

Code (Text):

haproxytool backend -r --file /var/run/haproxy/haproxy.sock
haproxy-stats 16
spider_backend 220
backend_tarpit 0
www_backend 3510

Code (Text):

haproxytool backend -S --file /var/run/haproxy/haproxy.sock
haproxy-stats
spider_backend
    server1
    server2
    server3
backend_tarpit
www_backend
    server1
    server2
    server3

Code (Text):

haproxytool server -W --file /var/run/haproxy/haproxy.sock
# backendname servername
spider_backend                 server1                                    2
spider_backend                 server2                                    2
spider_backend                 server3                                    2
www_backend                    server1                                    2
www_backend                    server2                                    2
www_backend                    server3                                    2

Code (Text):

haproxytool server -s --file /var/run/haproxy/haproxy.sock
# backendname servername
spider_backend                 server1                                    UP
spider_backend                 server2                                    UP
spider_backend                 server3                                    UP
www_backend                    server1                                    UP
www_backend                    server2                                    UP
www_backend                    server3                                    UP

Code (Text):

haproxytool server -r --file /var/run/haproxy/haproxy.sock
# backendname servername
spider_backend                 server1                                    110
spider_backend                 server2                                    110
spider_backend                 server3                                    0
www_backend                    server1                                    1710
www_backend                    server2                                    1800
www_backend                    server3                                    0

Code (Text):

haproxytool dump --file /var/run/haproxy/haproxy.sock
# frontend name, status, requests, process_nb
www,OPEN,3730,[1]
haproxy-stats,OPEN,27,[1]
# backend name, status, requests, servers
haproxy-stats,UP,16,
spider_backend,UP,220,server1,server2,server3
backend_tarpit,UP,0,
www_backend,UP,3510,server1,server2,server3
# server name, status, requests, backend
server1,UP,110,spider_backend
server2,UP,110,spider_backend
server3,UP,0,spider_backend
server1,UP,1710,www_backend
server2,UP,1800,www_backend
server3,UP,0,www_backend

Code (Text):

haproxytool acl -l --file /var/run/haproxy/haproxy.sock
# id (file) description
0 () acl 'always_true' file '/etc/haproxy/haproxy.cfg' line 54
1 () acl 'src_get_gpc0' file '/etc/haproxy/haproxy.cfg' line 86
2 () acl 'hdr_cnt' file '/etc/haproxy/haproxy.cfg' line 92
3 () acl 'hdr_cnt' file '/etc/haproxy/haproxy.cfg' line 93
4 () acl 'hdr_val' file '/etc/haproxy/haproxy.cfg' line 94
5 () acl 'hdr_cnt' file '/etc/haproxy/haproxy.cfg' line 95
6 () acl 'url_sub' file '/etc/haproxy/haproxy.cfg' line 100
7 () acl 'path_end' file '/etc/haproxy/haproxy.cfg' line 101
8 () acl 'path_dir' file '/etc/haproxy/haproxy.cfg' line 104
9 (/etc/haproxy/spiderbotlist.lst) pattern loaded from file '/etc/haproxy/spiderbotlist.lst' used by acl at file '/etc/haproxy/haproxy.cfg' line 109
10 () acl 'hdr_sub' file '/etc/haproxy/haproxy.cfg' line 109
11 () acl 'sc2_conn_rate' file '/etc/haproxy/haproxy.cfg' line 137
12 () acl 'sc2_bytes_out_rate' file '/etc/haproxy/haproxy.cfg' line 138
13 () acl 'be_sess_rate' file '/etc/haproxy/haproxy.cfg' line 159
14 () acl 'be_conn' file '/etc/haproxy/haproxy.cfg' line 160
15 () acl 'wait_end' file '/etc/haproxy/haproxy.cfg' line 163
-1 () acl 'http_auth' file 'internal-stats-auth-compat' line 0

eva2000 · Feb 7, 2018

Updated Haproxy routine with updated Haproxy 1.8.3 version with OpenSSL 1.1.0g. Added optional support to replace custom official zlib 1.2.11 with Cloudflare zlib performance fork for better HTTP compressed request performance [benchmarks]. Inspired to do some more testing thanks to Vultr Baremetal Nginx Proxy Loadbalancer testing.

Code (Text):

haproxy -vv
HA-Proxy version 1.8.3-205f675 2017/12/30
Copyright 2000-2017 Willy Tarreau <willy@haproxy.org>

Build options :
  TARGET  = linux2628
  CPU     = native
  CC      = gcc
  CFLAGS  = -march=native -m64 -march=x86-64 -O2 -g
  OPTIONS = USE_LINUX_SPLICE=1 USE_LINUX_TPROXY=1 USE_ZLIB=1 USE_REGPARM=1 USE_THREAD=1 USE_OPENSSL=1 USE_LUA=1 USE_PCRE=1 USE_PCRE_JIT=1

Default settings :
  maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with OpenSSL version : OpenSSL 1.1.0g  2 Nov 2017
Running on OpenSSL version : OpenSSL 1.1.0g  2 Nov 2017
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2
Built with Lua version : Lua 5.3.4
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Encrypted password support via crypt(3): yes
Built with multi-threading support.
Built with PCRE version : 8.41 2017-07-05
Running on PCRE version : 8.41 2017-07-05
PCRE library supports JIT : yes
Built with zlib version : 1.2.8
Running on zlib version : 1.2.8
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with network namespace support.

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available filters :
        [SPOE] spoe
        [COMP] compression
        [TRACE] trace

Cloudflare zlib performance fork = 1.2.8

Code (Text):

Built with zlib version : 1.2.8
Running on zlib version : 1.2.8

Built with new Haproxy multithreading support

Code (Text):

Built with multi-threading support.

eva2000 · Feb 8, 2018

Haproxy 1.8.3 with HTTP/2 and HTTP/1.1 HTTPS

eva2000 · Feb 10, 2018

Looks like Haproxy 1.8.4 out with lots of fixes. Last day with Vultr Bare Metal promo so last update for now [ANNOUNCE] haproxy-1.8.4

Optimised compilation of Haproxy with GCC 7.2.1 and LTO + Gold linker built Haproxy binary
Code (Text):
haproxy -vv                            
HA-Proxy version 1.8.4-1deb90d 2018/02/08
Copyright 2000-2018 Willy Tarreau <willy@haproxy.org>

Build options :
  TARGET  = linux2628
  CPU     = native
  CC      = gcc
  CFLAGS  = -march=native -m64 -O3 -g -flto -fuse-ld=gold -Wimplicit-fallthrough=0
  OPTIONS = USE_LINUX_SPLICE=1 USE_LINUX_TPROXY=1 USE_ZLIB=1 USE_REGPARM=1 USE_THREAD=1 USE_OPENSSL=1 USE_LUA=1 USE_PCRE=1 USE_PCRE_JIT=1

Default settings :
  maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with network namespace support.
Built with zlib version : 1.2.8
Running on zlib version : 1.2.8
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with PCRE version : 8.41 2017-07-05
Running on PCRE version : 8.41 2017-07-05
PCRE library supports JIT : yes
Built with multi-threading support.
Encrypted password support via crypt(3): yes
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built with Lua version : Lua 5.3.4
Built with OpenSSL version : OpenSSL 1.1.0g  2 Nov 2017
Running on OpenSSL version : OpenSSL 1.1.0g  2 Nov 2017
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available filters :
        [TRACE] trace
        [COMP] compression
        [SPOE] spoe
http://www.haproxy.org/download/1.8/src/CHANGELOG

ChangeLog :
===========

2018/02/08 : 1.8.4
- BUG/MEDIUM: h2: properly handle the END_STREAM flag on empty DATA frames
- BUILD: ssl: silence a warning when building without NPN nor ALPN support
- BUG/MEDIUM: ssl: cache doesn't release shctx blocks
- BUG/MINOR: lua: Fix default value for pattern in Socket.receive
- DOC: lua: Fix typos in comments of hlua_socket_receive
- BUG/MEDIUM: lua: Fix IPv6 with separate port support for Socket.connect
- BUG/MINOR: lua: Fix return value of Socket.settimeout
- MINOR: dns: Handle SRV record weight correctly.
- BUG/MEDIUM: mworker: execvp failure depending on argv[0]
- MINOR: hathreads: add support for gcc < 4.7
- BUILD/MINOR: ancient gcc versions atomic fix
- BUG/MEDIUM: stream: properly handle client aborts during redispatch
- DOC: clarify the scope of ssl_fc_is_resumed
- CONTRIB: debug: fix a few flags definitions
- BUG/MINOR: poll: too large size allocation for FD events
- BUG/MEDIUM: peers: fix expire date wasn't updated if entry is modified remotely.
- MINOR: servers: Don't report duplicate dyncookies for disabled servers.
- MINOR: global/threads: move cpu_map at the end of the global struct
- MINOR: threads: add a MAX_THREADS define instead of LONGBITS
- MINOR: global: add some global activity counters to help debugging
- MINOR: threads/fd: Use a bitfield to know if there are FDs for a thread in the FD cache
- BUG/MEDIUM: threads/polling: Use fd_cache_mask instead of fd_cache_num
- BUG/MEDIUM: fd: maintain a per-thread update mask
- MINOR: fd: add a bitmask to indicate that an FD is known by the poller
- BUG/MEDIUM: epoll/threads: use one epoll_fd per thread
- BUG/MEDIUM: kqueue/threads: use one kqueue_fd per thread
- BUG/MEDIUM: threads/mworker: fix a race on startup
- BUG/MINOR: mworker: only write to pidfile if it exists
- MINOR: threads: Fix build when we're not compiling with threads.
- BUG/MINOR: threads: always set an owner to the thread_sync pipe
- BUG/MEDIUM: threads/server: Fix deadlock in srv_set_stopping/srv_set_admin_flag
- BUG/MEDIUM: checks: Don't try to release undefined conn_stream when a check is freed
- BUG/MINOR: kqueue/threads: Don't forget to close kqueue_fd[tid] on each thread
- MINOR: threads: Use __decl_hathreads instead of #ifdef/#endif
- BUILD: epoll/threads: Add test on MAX_THREADS to avoid warnings when complied without threads
- BUILD: kqueue/threads: Add test on MAX_THREADS to avoid warnings when complied without threads
- CLEANUP: sample: Fix comment encoding of sample.c
- CLEANUP: sample: Fix outdated comment about sample casts functions
- BUG/MINOR: sample: Fix output type of c_ipv62ip
- CLEANUP: Fix typo in ARGT_MSK6 comment
- BUG/MINOR: cli: use global.maxsock and not maxfd to list all FDs
- BUG/MINOR: threads: Update labels array because of changes in lock_label enum
- BUG/MINOR: epoll/threads: only call epoll_ctl(DEL) on polled FDs
- BUG/MEDIUM: spoe: Always try to receive or send the frame to detect shutdowns
- BUG/MEDIUM: spoe: Allow producer to read and to forward shutdown on request side
- BUG/MINOR: time/threads: ensure the adjusted time is always correct
- BUG/MEDIUM: standard: Fix memory leak in str2ip2()
- MINOR: init: emit warning when -sf/-sd cannot parse argument
- DOC: Describe routing impact of using interface keyword on bind lines
- DOC: Mention -Ws in the list of available options
- BUG/MINOR: config: don't emit a warning when global stats is incompletely configured
Click to expand...

eva2000 · Dec 22, 2018

Nice Haproxy 1.9.0 has been released with support for HTTP/2 HTTPS communication between Haproxy load balancer and origin backends that are HTTP/2 HTTPS based ! Also threaded mode has up to 60% faster performance too and improved caching performance.

From haproxy.org

One year after 1.8.0 comes 1.9.0. This one's primary target is a significant technical improvement over 1.8, bringing much higher multi-threaded performance, improvements on connection management, process management, caching, H2... Please read the announcement here and the detailed blog article for more information. An important point to note, this technical release is not suitable for inclusion in distros, as it will only be maintained for approximately one year (till 2.1 is out). Version 2.0 coming in May will be long-lived and more suitable for distros.
Click to expand...

Change log https://www.haproxy.org/download/1.9/src/CHANGELOG

2018/12/19 : 1.9.0
- BUG/MEDIUM: compression: Use the right buffer pointers to compress input data
- BUG/MINOR: mux_pt: Set CS_FL_WANT_ROOM when count is zero in rcv_buf() callback
- BUG/MEDIUM: connection: Add a new CS_FL_ERR_PENDING flag to conn_streams.
- CONTRIB: debug: teach the "flags" utility about new conn_stream flags
- BUG/MEDIUM: stream-int: always clear CS_FL_WANT_ROOM before receiving
- BUG/MEDIUM: mux-h2: also restart demuxing when data are pending in demux
- BUG/MEDIUM: mux-h2: restart demuxing as soon as demux data are available
- BUG/MEDIUM: h2: fix aggregated cookie length computation in HTX mode
- MINOR: mux-h2: report more h2c, last h2s and cs information on "show fd"
- CONTRIB: debug: report stream-int's flag SI_FL_CLEAN_ABRT
- MINOR: cli/stream: add the conn_stream in "show sess" output
- BUG/MINOR: mux-h2: don't report a fantom h2s in "show fd"
- BUG/MINOR: cli/fd: don't isolate the thread for each individual fd
- MINOR: objtype: report a few missing types in names and base pointers
- BUG/MEDIUM: mux-h2: make sure to report synchronous errors after EOS
- BUG/MEDIUM: mux-h2: report asynchronous errors in h2_wake_some_streams()
- BUG/MEDIUM: mux-h2: make sure the demux also wakes streams up on errors
- BUG/MINOR: mux-h1: report the correct frontend in error captures
- BUG/MEDIUM: stream-int: also wake the stream up on end of transfer
- MEDIUM: h2: properly check and deduplicate the content-length header in HTX
- BUG/MEDIUM: stream: Forward the right amount of data before infinite forwarding
- BUG/MINOR: proto_htx: Call the HTX version of the function managing client cookies
- BUG/MEDIUM: lua/htx: Handle EOM in receive/get_line calls in HTTP applets
- BUG/MINOR: lua: Return an error if a legacy HTTP applet doesn't send anything
- MINOR: compression: Remove the thread_local variable buf_output
- CLEANUP: connection: rename subscription events values and event field
- CLEANUP: connection: rename conn->mux_ctx to conn->ctx
- MINOR: connection: remove an unwelcome dependency on struct stream
- CLEANUP: stream-int: consistently call the si/stream_int functions
- BUG/MEDIUM: h1: Don't shutw/shutr the connection if we have keepalive.
- BUG/MEDIUM: H2: Make sure htx is set even on empty frames.
- BUG/MEDIUM: mux-h2: pass CS_FL_ERR_PENDING to h2_wake_some_streams()
- MEDIUM: stream-int: always consider all CS errors on the send side
- BUG/MEDIUM: h2: Make sure we don't set CS_FL_ERROR if there's still data.
- CLEANUP: mux-h2: implement h2s_notify_{send,recv} to report events to subscribers
- MINOR: mux-h2: add a new function h2s_alert() to call the data layer
- BUG/MEDIUM: mux-h2: make use of h2s_alert() to report aborts
- MINOR: connection: add cs_set_error() to set the error bits
- CLEANUP: mux-h2: make use of cs_set_error()
- BUG/MINOR: mux-h2: make sure we check the conn_stream in early data
- BUG/MEDIUM: h2: Don't wait for flow control if the connection had a shutr.
- MINOR: cli/show_fd: report that a connection is back or not
- SCRIPTS: add the slack channel URL to the announce script
- CLEANUP: remove my name and address from the copyright banner
- DOC: mention in the readme that 1.9 is a stable version now
Click to expand...

Optimised compilation of Haproxy 1.9.0 with OpenSSL 1.1.1a for TLS 1.3 support + Cloudflare zlib performance library compiled with GCC 7.3.1 and LTO + Gold linker built binary
Code (Text):
haproxy -vv
HA-Proxy version 1.9.0 2018/12/19 - https://haproxy.org/
Build options :
  TARGET  = linux2628
  CPU     = native
  CC      = gcc
  CFLAGS  = -march=native -m64 -O3 -g -flto -fuse-ld=gold -Wimplicit-fallthrough=0
  OPTIONS = USE_LINUX_SPLICE=1 USE_LINUX_TPROXY=1 USE_ZLIB=1 USE_REGPARM=1 USE_THREAD=1 USE_OPENSSL=1 USE_LUA=1 USE_PCRE=1 USE_PCRE_JIT=1

Default settings :
  maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with OpenSSL version : OpenSSL 1.1.1a  20 Nov 2018
Running on OpenSSL version : OpenSSL 1.1.1a  20 Nov 2018
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
Built with Lua version : Lua 5.3.4
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built with zlib version : 1.2.8
Running on zlib version : 1.2.8
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with PCRE version : 8.42 2018-03-20
Running on PCRE version : 8.42 2018-03-20
PCRE library supports JIT : yes
Encrypted password support via crypt(3): yes
Built with multi-threading support.

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
              h2 : mode=HTTP       side=FE
              h2 : mode=HTX        side=FE|BE
       <default> : mode=HTX        side=FE|BE
       <default> : mode=TCP|HTTP   side=FE|BE

Available filters :
        [SPOE] spoe
        [COMP] compression
        [CACHE] cache
        [TRACE] trace
New haproxy 1.9.0 master socket for haproxy API querying
Code (Text):
socat /var/run/haproxy/haproxy-master.sock readline
command seems to hang until I hit enter and get
Code (Text):
socat /var/run/haproxy/haproxy-master.sock readline
Unknown command. Please enter one of the following commands only :
  help           : this message
  prompt         : toggle interactive mode with prompt
  quit           : disconnect
  @<relative pid> : send a command to the <relative pid> process
  @!<pid>        : send a command to the <pid> process
  @master        : send a command to the master process
  show cli sockets : dump list of cli sockets
  show cli level   : display the level of the current CLI session
  show proc      : show processes status
  operator       : lower the level of the current CLI session to operator
  user           : lower the level of the current CLI session to user
  reload         : reload haproxy
Query haproxy API show haproxy process status
Code (Text):
show proc
#<PID>          <type>          <relative PID>  <reloads>       <uptime>
9127            master          0               0               0d 00h02m26s
# workers
9128            worker          1               0               0d 00h02m25s
Show haproxy worker 1 info on 2 cpu thread server
Code (Text):
@1 show info
Name: HAProxy
Version: 1.9.0
Release_date: 2018/12/19
Nbthread: 1
Nbproc: 1
Process_num: 1
Pid: 9128
Uptime: 0d 0h02m50s
Uptime_sec: 170
Memmax_MB: 0
PoolAlloc_MB: 0
PoolUsed_MB: 0
PoolFailed: 0
Ulimit-n: 1040064
Maxsock: 1040064
Maxconn: 500000
Hard_maxconn: 500000
CurrConns: 0
CumConns: 1
CumReq: 1
MaxSslConns: 0
CurrSslConns: 0
CumSslConns: 0
Maxpipes: 20000
PipesUsed: 0
PipesFree: 0
ConnRate: 0
ConnRateLimit: 4000
MaxConnRate: 0
SessRate: 0
SessRateLimit: 0
MaxSessRate: 0
SslRate: 0
SslRateLimit: 0
MaxSslRate: 0
SslFrontendKeyRate: 0
SslFrontendMaxKeyRate: 0
SslFrontendSessionReuse_pct: 0
SslBackendKeyRate: 0
SslBackendMaxKeyRate: 0
SslCacheLookups: 0
SslCacheMisses: 0
CompressBpsIn: 0
CompressBpsOut: 0
CompressBpsRateLim: 0
ZlibMemUsage: 0
MaxZlibMemUsage: 0
Tasks: 36
Run_queue: 1
Idle_pct: 100
node: test.com
Stopping: 0
Jobs: 5
Unstoppable Jobs: 0
Listeners: 4
ActivePeers: 0
ConnectedPeers: 0
DroppedLogs: 0
BusyPolling: 0
haproxy runtime API query for show activity

A new show activity command has also been added to the Runtime API. It shows for each thread the total CPU time that was detected as stolen by the system, possibly in other processes running on the same processor, or by another VM shared by the same hypervisor. It also indicates the average processing latency experienced by all tasks, which may indicate that some heavy operations are in progress, such as very high usage of asymmetric cryptography, or extremely large ACLs involving thousands of regular expressions.
Click to expand...
Code (Text):
echo "show activity" |socat /var/run/haproxy/haproxy.sock stdio
thread_id: 0
date_now: 1545454623.205764
loops: 9496
wake_cache: 3168
wake_tasks: 3167
wake_signal: 0
poll_exp: 6335
poll_drop: 0
poll_dead: 0
poll_skip: 0
fd_skip: 0
fd_lock: 0
fd_del: 0
conn_dead: 0
stream: 3
empty_rq: 3160
long_rq: 0
cpust_ms_tot: 0
cpust_ms_1s: 0
cpust_ms_15s: 0
avg_loop_us: 82
Similarly, CPU time and latency values can be reported in logs when profiling is enabled in the global section or enabled using the Runtime API. This helps indicate which TCP/HTTP requests cost a lot to process and which ones suffer from the other ones.
Click to expand...
Optionally, to set it using the Runtime API:
Code (Text):
echo "set profiling tasks on" |socat /var/run/haproxy/haproxy.sock -
To verify that it’s been enabled:
Code (Text):
echo "show profiling" |socat /var/run/haproxy/haproxy.sock -
Per-task CPU profiling              : on      # set profiling tasks {on|off}
Profiling exposes the following fetches which can be captured within the HAProxy log:

Fetch method Description
date_us The microseconds part of the date.
cpu_calls The number of calls to the task processing the stream or current request since it was allocated. It is reset for each new request on the same connection.
cpu_ns_avg The average number of nanoseconds spent in each call to the task processing the stream or current request.
cpu_ns_tot The total number of nanoseconds spent in each call to the task processing the stream or current request.
lat_ns_avg The average number of nanoseconds spent between the moment the task handling the stream is woken up and the moment it is effectively called.
lat_ns_tot The total number of nanoseconds between the moment the task handling the stream is woken up and the moment it is effectively called.
Click to expand...
Server Queue Priority Control feature !
HAProxy 1.9 allows you to prioritize some queued connections over others. This can be helpful to, for example, deliver JavaScript or CSS files before images. Or, you might use it to improve loading times for premium-level customers. Another way to use it is to give a lower priority to bots.

Set a higher server queue priority for JS or CSS files over images by adding an http-request set-priority-class directive that specifies the level of importance to assign to a request. In order to avoid starvation caused by a contiguous stream of high-priority requests, there is also the set-priority-offset directive which sets an upper bound to the extra wait time that certain requests should experience compared to others. When you combine this with ACL rules, you gain the flexibility to decide when and how to prioritize connections.
Code (Text):
acl is_jscss path_end .js .css
acl is_image path_end .png .jpg .jpeg
http-request set-priority-class int(1) if is_jscss
http-request set-priority-class int(10) if is_image
http-request set-priority-class int(100) if !is_jscss !is_image
Lower numbers are given a higher priority. So, in this case, JavaScript and CSS files are given the utmost priority, followed by images, and then by everything else.
Click to expand...
Noticed one thing though haproxy refuses to start up if you enable haproxy 1.9 new Native HTTP Representation (HTX) feature and also use haproxy caching ?
Code (Text):
Dec 22 04:44:34 test.com haproxy[8884]: [ALERT] 355/044434 (8887) : Cache 'cache1': cannot be used by HTX and legacy HTTP proxies in same time.
Dec 22 04:44:34 test.com haproxy[8884]: [ALERT] 355/044434 (8887) : Fatal errors found in configuration.
But I didn't set a HTTP/2 HTTPS backend yet via alpn h2 so not sure if that is the reason ? Lots to read and learn for Haproxy 1.9

Haproxy 2.0 preview will have FastCGI support too !

HAProxy 2.0 Preview
HAProxy 1.9 will allow us to support the latest protocols and features that are becoming a necessity in the rapidly evolving technology landscape. You can expect to see the following features in HAProxy 2.0, which is scheduled to be released in May 2019:

HAProxy Data Plane API

gRPC

Layer 7 Retries

FastCGI integration

Circuit Breaking

Separation of TLS certificates from private keys

Ability to update TLS certificates and private keys using the Runtime API

Stay tuned, as we will continue to provide updates the closer we get to our next release!
Click to expand...

eva2000 · Dec 17, 2019

Test build latest Haproxy 2.1.1 OpenSSL 1.1.1d for TLS 1.3 support + Cloudflare zlib performance library compiled with GCC 8.3.1 and LTO + Gold linker built binary. And switch from pcre 8.4 branch to pcre2 10.3 version and switch from system glibc to jemalloc memory allocator.

Code (Text):

haproxy -vv
HA-Proxy version 2.1.1 2019/12/11 - https://haproxy.org/
Status: stable branch - will stop receiving fixes around Q1 2021.
Known bugs: http://www.haproxy.org/bugs/bugs-2.1.1.html
Build options :
  TARGET  = linux-glibc
  CPU     = native
  CC      = gcc
  CFLAGS  = -march=native -m64 -O3 -g -flto -fuse-ld=gold -Wimplicit-fallthrough=0 -fcode-hoisting -falign-functions=32
  OPTIONS = USE_PCRE2_JIT=1 USE_THREAD=1 USE_REGPARM=1 USE_STATIC_PCRE2=1 USE_LINUX_TPROXY=1 USE_LINUX_SPLICE=1 USE_OPENSSL=1 USE_LUA=1 USE_ZLIB=1 USE_SYSTEMD=1

Feature list : +EPOLL -KQUEUE -MY_EPOLL -MY_SPLICE +NETFILTER -PCRE -PCRE_JIT -PCRE2 +PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD -PTHREAD_PSHARED +REGPARM -STATIC_PCRE +STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT +CRYPT_H -VSYSCALL +GETADDRINFO +OPENSSL +LUA +FUTEX +ACCEPT4 -MY_ACCEPT4 +ZLIB -SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL +SYSTEMD -OBSOLETE_LINKER +PRCTL +THREAD_DUMP -EVPORTS

Default settings :
  bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_THREADS=64, default=2).
Built with OpenSSL version : OpenSSL 1.1.1d  10 Sep 2019
Running on OpenSSL version : OpenSSL 1.1.1d  10 Sep 2019
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
Built with Lua version : Lua 5.3.5
Built with network namespace support.
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built with PCRE2 version : 10.34 2019-11-21
PCRE2 library supports JIT : yes
Encrypted password support via crypt(3): yes
Built with zlib version : 1.2.8
Running on zlib version : 1.2.8
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
              h2 : mode=HTTP       side=FE|BE     mux=H2
            fcgi : mode=HTTP       side=BE        mux=FCGI
       <default> : mode=HTTP       side=FE|BE     mux=H1
       <default> : mode=TCP        side=FE|BE     mux=PASS

Available services : none

Available filters :
        [SPOE] spoe
        [CACHE] cache
        [FCGI] fcgi-app
        [TRACE] trace
        [COMP] compression

Also more performant haproxy binary switch Cloudflare zlib compression library to haproxy's optional libslz stateless zlib library. Libslz compression library is up to 3x times faster than zlib library and uses up to 90% less memory and between 2.5-3.5x times less cpu resources for processing HTTP compressed requests :cool:

Code (Text):

haproxy -vv
HA-Proxy version 2.1.1 2019/12/11 - https://haproxy.org/
Status: stable branch - will stop receiving fixes around Q1 2021.
Known bugs: http://www.haproxy.org/bugs/bugs-2.1.1.html
Build options :
  TARGET  = linux-glibc
  CPU     = native
  CC      = gcc
  CFLAGS  = -march=native -m64 -O3 -g -flto -fuse-ld=gold -Wimplicit-fallthrough=0 -fcode-hoisting -falign-functions=32
  OPTIONS = USE_PCRE2_JIT=1 USE_THREAD=1 USE_REGPARM=1 USE_STATIC_PCRE2=1 USE_LINUX_TPROXY=1 USE_LINUX_SPLICE=1 USE_OPENSSL=1 USE_LUA=1 USE_SLZ=1 USE_SYSTEMD=1

Feature list : +EPOLL -KQUEUE -MY_EPOLL -MY_SPLICE +NETFILTER -PCRE -PCRE_JIT -PCRE2 +PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD -PTHREAD_PSHARED +REGPARM -STATIC_PCRE +STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT +CRYPT_H -VSYSCALL +GETADDRINFO +OPENSSL +LUA +FUTEX +ACCEPT4 -MY_ACCEPT4 -ZLIB +SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL +SYSTEMD -OBSOLETE_LINKER +PRCTL +THREAD_DUMP -EVPORTS

Default settings :
  bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_THREADS=64, default=2).
Built with OpenSSL version : OpenSSL 1.1.1d  10 Sep 2019
Running on OpenSSL version : OpenSSL 1.1.1d  10 Sep 2019
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
Built with Lua version : Lua 5.3.5
Built with network namespace support.
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built with PCRE2 version : 10.34 2019-11-21
PCRE2 library supports JIT : yes
Encrypted password support via crypt(3): yes
Built with libslz for stateless compression.
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
              h2 : mode=HTTP       side=FE|BE     mux=H2
            fcgi : mode=HTTP       side=BE        mux=FCGI
       <default> : mode=HTTP       side=FE|BE     mux=H1
       <default> : mode=TCP        side=FE|BE     mux=PASS

Available services : none

Available filters :
        [SPOE] spoe
        [CACHE] cache
        [FCGI] fcgi-app
        [TRACE] trace
        [COMP] compression

eva2000 · Mar 20, 2022

Revisited latest Haproxy 2.5.5 for testing some compatibility issues with Cloudflare's Full Strict SSL mode

GCC 10.2 compiled Haproxy 2.5.5 built using OpenSSL 1.1.1n with TLSv1.3 support + jemalloc + Lua 5.4 + PCRE2 10.39 + Haproxy's optional libslz stateless zlib library. Libslz compression library is up to 3x times faster than zlib library and uses up to 90% less memory and between 2.5-3.5x times less cpu resources for processing HTTP compressed requests.

Code (Text):

haproxy -vv
HAProxy version 2.5.5-384c5c5 2022/03/14 - https://haproxy.org/
Status: stable branch - will stop receiving fixes around Q1 2023.
Known bugs: http://www.haproxy.org/bugs/bugs-2.5.5.html
Running on: Linux 3.10.0-1160.41.1.el7.x86_64 #1 SMP Tue Aug 31 14:52:47 UTC 2021 x86_64
Build options :
  TARGET  = linux-glibc
  CPU     = native
  CC      = cc
  CFLAGS  = -O2 -march=native -g -Wall -Wextra -Wundef -Wdeclaration-after-statement -fwrapv -Wno-address-of-packed-member -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-clobbered -Wno-missing-field-initializers -Wno-cast-function-type -Wtype-limits -Wshift-negative-value -Wshift-overflow=2 -Wduplicated-cond -Wnull-dereference
  OPTIONS = USE_PCRE2_JIT=1 USE_THREAD=1 USE_STATIC_PCRE2=1 USE_LINUX_TPROXY=1 USE_LINUX_SPLICE=1 USE_OPENSSL=1 USE_LUA=1 USE_SLZ=1 USE_SYSTEMD=1 USE_PROMEX=1
  DEBUG   =

Feature list : +EPOLL -KQUEUE +NETFILTER -PCRE -PCRE_JIT -PCRE2 +PCRE2_JIT +POLL +THREAD +BACKTRACE -STATIC_PCRE +STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT +CRYPT_H +GETADDRINFO +OPENSSL +LUA +ACCEPT4 -CLOSEFROM -ZLIB +SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL +SYSTEMD -OBSOLETE_LINKER +PRCTL -PROCCTL +THREAD_DUMP -EVPORTS -OT -QUIC +PROMEX -MEMORY_PROFILING

Default settings :
  bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_THREADS=64, default=4).
Built with OpenSSL version : OpenSSL 1.1.1n  15 Mar 2022
Running on OpenSSL version : OpenSSL 1.1.1n  15 Mar 2022
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
Built with Lua version : Lua 5.4.4
Built with the Prometheus exporter as a service
Built with network namespace support.
Built with libslz for stateless compression.
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built with PCRE2 version : 10.39 2021-10-29
PCRE2 library supports JIT : yes
Encrypted password support via crypt(3): yes
Built with gcc compiler version 10.2.1 20210130 (Red Hat 10.2.1-11)

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
              h2 : mode=HTTP       side=FE|BE     mux=H2       flags=HTX|CLEAN_ABRT|HOL_RISK|NO_UPG
            fcgi : mode=HTTP       side=BE        mux=FCGI     flags=HTX|HOL_RISK|NO_UPG
       <default> : mode=HTTP       side=FE|BE     mux=H1       flags=HTX
              h1 : mode=HTTP       side=FE|BE     mux=H1       flags=HTX|NO_UPG
       <default> : mode=TCP        side=FE|BE     mux=PASS     flags=
            none : mode=TCP        side=FE|BE     mux=PASS     flags=NO_UPG

Available services : prometheus-exporter
Available filters :
        [SPOE] spoe
        [CACHE] cache
        [FCGI] fcgi-app
        [COMP] compression
        [TRACE] trace

Log in / Register

Forums

Welcome to Centmin Mod Community

Beta Branch addons/haproxy.sh preview

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

haproxytool

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

.

Log in / Register

Useful Searches

Welcome to Centmin Mod Community

Beta Branch addons/haproxy.sh preview

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

haproxytool

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

eva2000 Administrator Staff Member

.