Learn about Centmin Mod LEMP Stack today
Become a Member

Beta Branch addons/haproxy.sh preview

Discussion in 'Beta release code' started by eva2000, Aug 27, 2017.

Tags:
Thread Status:
Not open for further replies.
  1. eva2000

    eva2000 Administrator Staff Member

    41,346
    9,279
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +14,236
    Local Time:
    2:27 PM
    Nginx 1.17.x
    MariaDB 5.5/10.x
    Haproxy load balancer is another 'to do' list item I am planning for future Centmin Mod versions. Most likely will be added as an addon at addons/haproxy.sh when eventually ready.

    This is just a preview thread :)

    Haproxy 1.7.9 with OpenSSL 1.1.0f + Lua + zlib + Pcre/Pcre Jit + Transparent Proxy
    Code (Text):
    haproxy -vv
    HA-Proxy version 1.7.9 2017/08/18
    Copyright 2000-2017 Willy Tarreau <[email protected]>
    
    Build options :
      TARGET  = linux2628
      CPU     = native
      CC      = gcc
      CFLAGS  = -march=native -m64 -march=x86-64 -O2 -g
      OPTIONS = USE_LINUX_SPLICE=1 USE_LINUX_TPROXY=1 USE_ZLIB=1 USE_REGPARM=1 USE_OPENSSL=1 USE_LUA=1 USE_PCRE=1 USE_PCRE_JIT=1
    
    Default settings :
      maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
    
    Encrypted password support via crypt(3): yes
    Built with zlib version : 1.2.11
    Running on zlib version : 1.2.11
    Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
    Built with OpenSSL version : OpenSSL 1.1.0f  25 May 2017
    Running on OpenSSL version : OpenSSL 1.1.0f  25 May 2017
    OpenSSL library supports TLS extensions : yes
    OpenSSL library supports SNI : yes
    OpenSSL library supports prefer-server-ciphers : yes
    Built with PCRE version : 8.41 2017-07-05
    Running on PCRE version : 8.41 2017-07-05
    PCRE library supports JIT : yes
    Built with Lua version : Lua 5.3.4
    Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
    
    Available polling systems :
          epoll : pref=300,  test result OK
           poll : pref=200,  test result OK
         select : pref=150,  test result OK
    Total: 3 (3 usable), will use epoll.
    
    Available filters :
            [COMP] compression
            [TRACE] trace
            [SPOE] spoe


    Haproxy statistics

    haproxy-1.7.9-stats-01.png haproxy-1.7.9-stats-02.png haproxy-1.7.9-stats-03.png
     
    • Like Like x 2
    • Winner Winner x 1
  2. eva2000

    eva2000 Administrator Staff Member

    41,346
    9,279
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +14,236
    Local Time:
    2:27 PM
    Nginx 1.17.x
    MariaDB 5.5/10.x

    haproxytool



    haproxytool command line support
    Code (Text):
           
    haproxytool frontend -c --file /var/run/haproxy/haproxy.sock
    www 20000
    haproxy-stats 20000
    
    

    Code (Text):
    haproxytool frontend -s --file /var/run/haproxy/haproxy.sock
    www OPEN
    haproxy-stats OPEN
    
    

    Code (Text):
    haproxytool frontend -r --file /var/run/haproxy/haproxy.sock
    www 3730
    haproxy-stats 27
    
    

    Code (Text):
    haproxytool backend -s --file /var/run/haproxy/haproxy.sock
    haproxy-stats UP
    spider_backend UP
    backend_tarpit UP
    www_backend UP
    

    Code (Text):
    haproxytool backend -r --file /var/run/haproxy/haproxy.sock
    haproxy-stats 16
    spider_backend 220
    backend_tarpit 0
    www_backend 3510
    

    Code (Text):
    haproxytool backend -S --file /var/run/haproxy/haproxy.sock
    haproxy-stats
    spider_backend
        server1
        server2
        server3
    backend_tarpit
    www_backend
        server1
        server2
        server3
    

    Code (Text):
    haproxytool server -W --file /var/run/haproxy/haproxy.sock
    # backendname servername
    spider_backend                 server1                                    2
    spider_backend                 server2                                    2
    spider_backend                 server3                                    2
    www_backend                    server1                                    2
    www_backend                    server2                                    2
    www_backend                    server3                                    2
    

    Code (Text):
    haproxytool server -s --file /var/run/haproxy/haproxy.sock
    # backendname servername
    spider_backend                 server1                                    UP
    spider_backend                 server2                                    UP
    spider_backend                 server3                                    UP
    www_backend                    server1                                    UP
    www_backend                    server2                                    UP
    www_backend                    server3                                    UP
    

    Code (Text):
    haproxytool server -r --file /var/run/haproxy/haproxy.sock
    # backendname servername
    spider_backend                 server1                                    110
    spider_backend                 server2                                    110
    spider_backend                 server3                                    0
    www_backend                    server1                                    1710
    www_backend                    server2                                    1800
    www_backend                    server3                                    0
    

    Code (Text):
    haproxytool dump --file /var/run/haproxy/haproxy.sock
    # frontend name, status, requests, process_nb
    www,OPEN,3730,[1]
    haproxy-stats,OPEN,27,[1]
    # backend name, status, requests, servers
    haproxy-stats,UP,16,
    spider_backend,UP,220,server1,server2,server3
    backend_tarpit,UP,0,
    www_backend,UP,3510,server1,server2,server3
    # server name, status, requests, backend
    server1,UP,110,spider_backend
    server2,UP,110,spider_backend
    server3,UP,0,spider_backend
    server1,UP,1710,www_backend
    server2,UP,1800,www_backend
    server3,UP,0,www_backend
    

    Code (Text):
    haproxytool acl -l --file /var/run/haproxy/haproxy.sock
    # id (file) description
    0 () acl 'always_true' file '/etc/haproxy/haproxy.cfg' line 54
    1 () acl 'src_get_gpc0' file '/etc/haproxy/haproxy.cfg' line 86
    2 () acl 'hdr_cnt' file '/etc/haproxy/haproxy.cfg' line 92
    3 () acl 'hdr_cnt' file '/etc/haproxy/haproxy.cfg' line 93
    4 () acl 'hdr_val' file '/etc/haproxy/haproxy.cfg' line 94
    5 () acl 'hdr_cnt' file '/etc/haproxy/haproxy.cfg' line 95
    6 () acl 'url_sub' file '/etc/haproxy/haproxy.cfg' line 100
    7 () acl 'path_end' file '/etc/haproxy/haproxy.cfg' line 101
    8 () acl 'path_dir' file '/etc/haproxy/haproxy.cfg' line 104
    9 (/etc/haproxy/spiderbotlist.lst) pattern loaded from file '/etc/haproxy/spiderbotlist.lst' used by acl at file '/etc/haproxy/haproxy.cfg' line 109
    10 () acl 'hdr_sub' file '/etc/haproxy/haproxy.cfg' line 109
    11 () acl 'sc2_conn_rate' file '/etc/haproxy/haproxy.cfg' line 137
    12 () acl 'sc2_bytes_out_rate' file '/etc/haproxy/haproxy.cfg' line 138
    13 () acl 'be_sess_rate' file '/etc/haproxy/haproxy.cfg' line 159
    14 () acl 'be_conn' file '/etc/haproxy/haproxy.cfg' line 160
    15 () acl 'wait_end' file '/etc/haproxy/haproxy.cfg' line 163
    -1 () acl 'http_auth' file 'internal-stats-auth-compat' line 0
    
     
    • Like Like x 3
  3. eva2000

    eva2000 Administrator Staff Member

    41,346
    9,279
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +14,236
    Local Time:
    2:27 PM
    Nginx 1.17.x
    MariaDB 5.5/10.x
    Updated Haproxy routine with updated Haproxy 1.8.3 version with OpenSSL 1.1.0g. Added optional support to replace custom official zlib 1.2.11 with Cloudflare zlib performance fork for better HTTP compressed request performance [benchmarks]. Inspired to do some more testing thanks to Vultr Baremetal Nginx Proxy Loadbalancer testing.

    Code (Text):
    haproxy -vv
    HA-Proxy version 1.8.3-205f675 2017/12/30
    Copyright 2000-2017 Willy Tarreau <[email protected]>
    
    Build options :
      TARGET  = linux2628
      CPU     = native
      CC      = gcc
      CFLAGS  = -march=native -m64 -march=x86-64 -O2 -g
      OPTIONS = USE_LINUX_SPLICE=1 USE_LINUX_TPROXY=1 USE_ZLIB=1 USE_REGPARM=1 USE_THREAD=1 USE_OPENSSL=1 USE_LUA=1 USE_PCRE=1 USE_PCRE_JIT=1
    
    Default settings :
      maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
    
    Built with OpenSSL version : OpenSSL 1.1.0g  2 Nov 2017
    Running on OpenSSL version : OpenSSL 1.1.0g  2 Nov 2017
    OpenSSL library supports TLS extensions : yes
    OpenSSL library supports SNI : yes
    OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2
    Built with Lua version : Lua 5.3.4
    Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
    Encrypted password support via crypt(3): yes
    Built with multi-threading support.
    Built with PCRE version : 8.41 2017-07-05
    Running on PCRE version : 8.41 2017-07-05
    PCRE library supports JIT : yes
    Built with zlib version : 1.2.8
    Running on zlib version : 1.2.8
    Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
    Built with network namespace support.
    
    Available polling systems :
          epoll : pref=300,  test result OK
           poll : pref=200,  test result OK
         select : pref=150,  test result OK
    Total: 3 (3 usable), will use epoll.
    
    Available filters :
            [SPOE] spoe
            [COMP] compression
            [TRACE] trace
    

    Cloudflare zlib performance fork = 1.2.8
    Code (Text):
    Built with zlib version : 1.2.8
    Running on zlib version : 1.2.8
    

    Built with new Haproxy multithreading support
    Code (Text):
    Built with multi-threading support.
    


    haproxy-183-admin-stats-01.png
     
    • Winner Winner x 1
  4. eva2000

    eva2000 Administrator Staff Member

    41,346
    9,279
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +14,236
    Local Time:
    2:27 PM
    Nginx 1.17.x
    MariaDB 5.5/10.x
    Haproxy 1.8.3 with HTTP/2 and HTTP/1.1 HTTPS

    https-haproxy-183-admin-stats-01.png https-haproxy-183-admin-stats-02.png
     
  5. eva2000

    eva2000 Administrator Staff Member

    41,346
    9,279
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +14,236
    Local Time:
    2:27 PM
    Nginx 1.17.x
    MariaDB 5.5/10.x
    Looks like Haproxy 1.8.4 out with lots of fixes. Last day with Vultr Bare Metal promo so last update for now [ANNOUNCE] haproxy-1.8.4

    Optimised compilation of Haproxy with GCC 7.2.1 and LTO + Gold linker built Haproxy binary
    Code (Text):
    haproxy -vv                            
    HA-Proxy version 1.8.4-1deb90d 2018/02/08
    Copyright 2000-2018 Willy Tarreau <[email protected]>
    
    Build options :
      TARGET  = linux2628
      CPU     = native
      CC      = gcc
      CFLAGS  = -march=native -m64 -O3 -g -flto -fuse-ld=gold -Wimplicit-fallthrough=0
      OPTIONS = USE_LINUX_SPLICE=1 USE_LINUX_TPROXY=1 USE_ZLIB=1 USE_REGPARM=1 USE_THREAD=1 USE_OPENSSL=1 USE_LUA=1 USE_PCRE=1 USE_PCRE_JIT=1
    
    Default settings :
      maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
    
    Built with network namespace support.
    Built with zlib version : 1.2.8
    Running on zlib version : 1.2.8
    Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
    Built with PCRE version : 8.41 2017-07-05
    Running on PCRE version : 8.41 2017-07-05
    PCRE library supports JIT : yes
    Built with multi-threading support.
    Encrypted password support via crypt(3): yes
    Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
    Built with Lua version : Lua 5.3.4
    Built with OpenSSL version : OpenSSL 1.1.0g  2 Nov 2017
    Running on OpenSSL version : OpenSSL 1.1.0g  2 Nov 2017
    OpenSSL library supports TLS extensions : yes
    OpenSSL library supports SNI : yes
    OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2
    
    Available polling systems :
          epoll : pref=300,  test result OK
           poll : pref=200,  test result OK
         select : pref=150,  test result OK
    Total: 3 (3 usable), will use epoll.
    
    Available filters :
            [TRACE] trace
            [COMP] compression
            [SPOE] spoe
    


    http://www.haproxy.org/download/1.8/src/CHANGELOG

     
    • Like Like x 3
    • Informative Informative x 1
  6. eva2000

    eva2000 Administrator Staff Member

    41,346
    9,279
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +14,236
    Local Time:
    2:27 PM
    Nginx 1.17.x
    MariaDB 5.5/10.x
    Nice Haproxy 1.9.0 has been released with support for HTTP/2 HTTPS communication between Haproxy load balancer and origin backends that are HTTP/2 HTTPS based ! Also threaded mode has up to 60% faster performance too and improved caching performance.

    From haproxy.org
    Change log https://www.haproxy.org/download/1.9/src/CHANGELOG
    Optimised compilation of Haproxy 1.9.0 with OpenSSL 1.1.1a for TLS 1.3 support + Cloudflare zlib performance library compiled with GCC 7.3.1 and LTO + Gold linker built binary
    Code (Text):
    haproxy -vv
    HA-Proxy version 1.9.0 2018/12/19 - https://haproxy.org/
    Build options :
      TARGET  = linux2628
      CPU     = native
      CC      = gcc
      CFLAGS  = -march=native -m64 -O3 -g -flto -fuse-ld=gold -Wimplicit-fallthrough=0
      OPTIONS = USE_LINUX_SPLICE=1 USE_LINUX_TPROXY=1 USE_ZLIB=1 USE_REGPARM=1 USE_THREAD=1 USE_OPENSSL=1 USE_LUA=1 USE_PCRE=1 USE_PCRE_JIT=1
    
    Default settings :
      maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
    
    Built with OpenSSL version : OpenSSL 1.1.1a  20 Nov 2018
    Running on OpenSSL version : OpenSSL 1.1.1a  20 Nov 2018
    OpenSSL library supports TLS extensions : yes
    OpenSSL library supports SNI : yes
    OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
    Built with Lua version : Lua 5.3.4
    Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
    Built with zlib version : 1.2.8
    Running on zlib version : 1.2.8
    Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
    Built with PCRE version : 8.42 2018-03-20
    Running on PCRE version : 8.42 2018-03-20
    PCRE library supports JIT : yes
    Encrypted password support via crypt(3): yes
    Built with multi-threading support.
    
    Available polling systems :
          epoll : pref=300,  test result OK
           poll : pref=200,  test result OK
         select : pref=150,  test result OK
    Total: 3 (3 usable), will use epoll.
    
    Available multiplexer protocols :
    (protocols marked as <default> cannot be specified using 'proto' keyword)
                  h2 : mode=HTTP       side=FE
                  h2 : mode=HTX        side=FE|BE
           <default> : mode=HTX        side=FE|BE
           <default> : mode=TCP|HTTP   side=FE|BE
    
    Available filters :
            [SPOE] spoe
            [COMP] compression
            [CACHE] cache
            [TRACE] trace
    

    New haproxy 1.9.0 master socket for haproxy API querying
    Code (Text):
    socat /var/run/haproxy/haproxy-master.sock readline
    

    command seems to hang until I hit enter and get
    Code (Text):
    socat /var/run/haproxy/haproxy-master.sock readline
    Unknown command. Please enter one of the following commands only :
      help           : this message
      prompt         : toggle interactive mode with prompt
      quit           : disconnect
      @<relative pid> : send a command to the <relative pid> process
      @!<pid>        : send a command to the <pid> process
      @master        : send a command to the master process
      show cli sockets : dump list of cli sockets
      show cli level   : display the level of the current CLI session
      show proc      : show processes status
      operator       : lower the level of the current CLI session to operator
      user           : lower the level of the current CLI session to user
      reload         : reload haproxy
    

    Query haproxy API show haproxy process status
    Code (Text):
    show proc
    #<PID>          <type>          <relative PID>  <reloads>       <uptime>
    9127            master          0               0               0d 00h02m26s
    # workers
    9128            worker          1               0               0d 00h02m25s
    

    Show haproxy worker 1 info on 2 cpu thread server
    Code (Text):
    @1 show info
    Name: HAProxy
    Version: 1.9.0
    Release_date: 2018/12/19
    Nbthread: 1
    Nbproc: 1
    Process_num: 1
    Pid: 9128
    Uptime: 0d 0h02m50s
    Uptime_sec: 170
    Memmax_MB: 0
    PoolAlloc_MB: 0
    PoolUsed_MB: 0
    PoolFailed: 0
    Ulimit-n: 1040064
    Maxsock: 1040064
    Maxconn: 500000
    Hard_maxconn: 500000
    CurrConns: 0
    CumConns: 1
    CumReq: 1
    MaxSslConns: 0
    CurrSslConns: 0
    CumSslConns: 0
    Maxpipes: 20000
    PipesUsed: 0
    PipesFree: 0
    ConnRate: 0
    ConnRateLimit: 4000
    MaxConnRate: 0
    SessRate: 0
    SessRateLimit: 0
    MaxSessRate: 0
    SslRate: 0
    SslRateLimit: 0
    MaxSslRate: 0
    SslFrontendKeyRate: 0
    SslFrontendMaxKeyRate: 0
    SslFrontendSessionReuse_pct: 0
    SslBackendKeyRate: 0
    SslBackendMaxKeyRate: 0
    SslCacheLookups: 0
    SslCacheMisses: 0
    CompressBpsIn: 0
    CompressBpsOut: 0
    CompressBpsRateLim: 0
    ZlibMemUsage: 0
    MaxZlibMemUsage: 0
    Tasks: 36
    Run_queue: 1
    Idle_pct: 100
    node: test.com
    Stopping: 0
    Jobs: 5
    Unstoppable Jobs: 0
    Listeners: 4
    ActivePeers: 0
    ConnectedPeers: 0
    DroppedLogs: 0
    BusyPolling: 0
    

    haproxy runtime API query for show activity

    Code (Text):
    echo "show activity" |socat /var/run/haproxy/haproxy.sock stdio
    thread_id: 0
    date_now: 1545454623.205764
    loops: 9496
    wake_cache: 3168
    wake_tasks: 3167
    wake_signal: 0
    poll_exp: 6335
    poll_drop: 0
    poll_dead: 0
    poll_skip: 0
    fd_skip: 0
    fd_lock: 0
    fd_del: 0
    conn_dead: 0
    stream: 3
    empty_rq: 3160
    long_rq: 0
    cpust_ms_tot: 0
    cpust_ms_1s: 0
    cpust_ms_15s: 0
    avg_loop_us: 82
    

    Server Queue Priority Control feature !
    Noticed one thing though haproxy refuses to start up if you enable haproxy 1.9 new Native HTTP Representation (HTX) feature and also use haproxy caching ?
    Code (Text):
    Dec 22 04:44:34 test.com haproxy[8884]: [ALERT] 355/044434 (8887) : Cache 'cache1': cannot be used by HTX and legacy HTTP proxies in same time.
    Dec 22 04:44:34 test.com haproxy[8884]: [ALERT] 355/044434 (8887) : Fatal errors found in configuration.
    

    But I didn't set a HTTP/2 HTTPS backend yet via alpn h2 so not sure if that is the reason ? Lots to read and learn for Haproxy 1.9 :)

    Haproxy 2.0 preview will have FastCGI support too !
     
    • Agree Agree x 1
    • Informative Informative x 1
Thread Status:
Not open for further replies.