Learn about Centmin Mod LEMP Stack today
Become a Member

Nginx [PATCH] Fix error when building a Nginx source RPM with custom Zlib - Nginx 1.13.7

Discussion in 'Nginx and PHP-FPM news & discussions' started by bassie, Dec 12, 2017.

  1. bassie

    bassie Active Member

    869
    203
    43
    Apr 29, 2016
    Ratings:
    +609
    Local Time:
    3:32 PM
    When building a Nginx source RPM with your own custom Zlib.
    The following error could appear. Even if -fPIC is already defined in the .spec:

    Code:
    "Relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
    libz.a: error adding symbols: Bad value collect2: error: ld returned 1 exit status"
    This patch will fix the error:

    Nginx-fix-error-building-srpm-custom-zlib

    Error not directly related to Centminmod.
    But nonetheless annoying.

    That's why I made a patch.
    I am sure that there are others on the Internet with the same problem.
     
    • Informative Informative x 1
  2. eva2000

    eva2000 Administrator Staff Member

    35,076
    7,742
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +11,941
    Local Time:
    11:32 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    interesting you using plain vanilla zlib 1.2.11 or a zlib fork ? Not sure why you'd get that error only on source RPM builds and not source installs ? I know Centmin Mod Ngixn 1.13.7 with zlib 1.2.11 compiles fine to date.
     
  3. bassie

    bassie Active Member

    869
    203
    43
    Apr 29, 2016
    Ratings:
    +609
    Local Time:
    3:32 PM
    Zlib and Zlib forks (Zlib-ng, Cloudflare, Intel), even the Fedora and EL7 Zlib source,
    all custom Zlib's will fail without patch when building source RPM's with the Nginx option: --with-zlib=/.

    As written before the error is not directly related to Centminmod :)
    Manual compiling works fine as long as it is not related to building source RPM's. It could be dynamically building, it could be building in a rpmbuild environment.

    Honestly. I don't know.
    I found no comparable case. Apparently no one seems to compile Zlib into source RPM Nginx.

    So it takes a lot more time to find out (many unanswered questions on the internet about -fPIC), than to write a simple patch.

    Nginx + Cloudflare Zlib works fine now.
    Nice and fast too :)

    Goal achieved.
     
    Last edited: Dec 12, 2017
  4. bassie

    bassie Active Member

    869
    203
    43
    Apr 29, 2016
    Ratings:
    +609
    Local Time:
    3:32 PM
    My reaction is above this. A question came to me. Hence 2 replys.

    @eva2000 Question for you. Why using Zlib-ng?
    Is Zlib-ng stable? (thoroughly tested?) as it is a blend between Cloudflare and Intel's fork.

    But perhaps more important, is it safe?
    I do not see anywhere that they fix CVE's?

    And then. What about speed.
    mtl1979 one of the code maintainer of Zlib-ng writes:

    As far as I can read it, Cloudflare's fork is the best choice.
    Well tested, secure, maintained, fast.

    * I know the research of their own product is not always objective.
     
    Last edited: Dec 12, 2017
  5. eva2000

    eva2000 Administrator Staff Member

    35,076
    7,742
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +11,941
    Local Time:
    11:32 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    I am just using stock zlib 1.2.11. But you're using Cloudflare's zlib right ?
     
  6. bassie

    bassie Active Member

    869
    203
    43
    Apr 29, 2016
    Ratings:
    +609
    Local Time:
    3:32 PM
    Yes using Cloudflare's Zlib fork here.
    Aha so Zlib-ng is your fail-back then?
     
    • Informative Informative x 1
  7. eva2000

    eva2000 Administrator Staff Member

    35,076
    7,742
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +11,941
    Local Time:
    11:32 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    it's a work in progress development - just fails to properly compile right now so not used heh
     
  8. bassie

    bassie Active Member

    869
    203
    43
    Apr 29, 2016
    Ratings:
    +609
    Local Time:
    3:32 PM
    It is advisable to switch to Cloudflare's Zlib.
    If you can't compile it properly.

    Zlib-ng seems more of a project to support more CPU's and to do cleanups in the code, rather than speed.

    Centminmod seems to find speed important :)
     
  9. eva2000

    eva2000 Administrator Staff Member

    35,076
    7,742
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +11,941
    Local Time:
    11:32 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
  10. bassie

    bassie Active Member

    869
    203
    43
    Apr 29, 2016
    Ratings:
    +609
    Local Time:
    3:32 PM
    Working fine here. Try again, I'm sure you can :)

    Although I did not read the thread you quote.
    Based on CentOS 7.4 minimal, without Kdump.

    For everyone who wants to know.
    I have executed the following to compile Nginx 1.13.7 with Cloudflare Zlib.
    Please note that a service is not available.
    The idea was purely to test whether it works with Cloudflare Zlib.
    Before I continued building a source RPM.
     
    Last edited: Dec 13, 2017
    • Informative Informative x 1
  11. eva2000

    eva2000 Administrator Staff Member

    35,076
    7,742
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +11,941
    Local Time:
    11:32 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    if you wrap links in [PLAIN][/PLAIN] tags they don't get parsed :)

    thanks, will re-try and see if it works :)
     
  12. eva2000

    eva2000 Administrator Staff Member

    35,076
    7,742
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +11,941
    Local Time:
    11:32 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
  13. bassie

    bassie Active Member

    869
    203
    43
    Apr 29, 2016
    Ratings:
    +609
    Local Time:
    3:32 PM
    Try:
    cd /svr-setup/zlib-cloudflare
    make -f Makefile.in distclean
    cd /svr-setup/nginx-1.13.7
    etc etc.
     
    • Like Like x 1
  14. eva2000

    eva2000 Administrator Staff Member

    35,076
    7,742
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +11,941
    Local Time:
    11:32 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    that worked in allowing cloudflare zlib compile with nginx centmin mod 123.09beta01 with cloudflare zlib fork manual compile · GitHub

    though lsof still shows zlib 1.2.11
    Code (Text):
    lsof | grep nginx | grep -v php-fpm | grep libz
    nginx      1720            root  mem       REG                9,2     117592   12089342 /usr/local/lib/libz.so.1.2.11
    nginx      1721           nginx  mem       REG                9,2     117592   12089342 /usr/local/lib/libz.so.1.2.11
    nginx      1722           nginx  mem       REG                9,2     117592   12089342 /usr/local/lib/libz.so.1.2.11
    nginx      1724           nginx  mem       REG                9,2     117592   12089342 /usr/local/lib/libz.so.1.2.11
    nginx      1725           nginx  mem       REG                9,2     117592   12089342 /usr/local/lib/libz.so.1.2.11
     
  15. eva2000

    eva2000 Administrator Staff Member

    35,076
    7,742
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +11,941
    Local Time:
    11:32 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    k fixed it now Cloudflare zlib is v1.2.8 vs standard zlib v1.2.11
    Code (Text):
    lsof | grep nginx | grep -v php-fpm | grep libz
    nginx     12345            root  mem       REG                9,2     116840   12089344 /usr/local/lib/libz.so.1.2.8
    nginx     12346           nginx  mem       REG                9,2     116840   12089344 /usr/local/lib/libz.so.1.2.8
    nginx     12347           nginx  mem       REG                9,2     116840   12089344 /usr/local/lib/libz.so.1.2.8
    nginx     12349           nginx  mem       REG                9,2     116840   12089344 /usr/local/lib/libz.so.1.2.8
    nginx     12350           nginx  mem       REG                9,2     116840   12089344 /usr/local/lib/libz.so.1.2.8
     
  16. bassie

    bassie Active Member

    869
    203
    43
    Apr 29, 2016
    Ratings:
    +609
    Local Time:
    3:32 PM
    Nice.
     
    Last edited: Dec 13, 2017
    • Like Like x 1
  17. eva2000

    eva2000 Administrator Staff Member

    35,076
    7,742
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +11,941
    Local Time:
    11:32 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    cheers mate for the pointers :)

    long term testing to see how this fairs first :D
     
  18. eva2000

    eva2000 Administrator Staff Member

    35,076
    7,742
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +11,941
    Local Time:
    11:32 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    wonder if cloudflare zlib requires SSE4.2 to work and what happens on cpus without SSE4.2

    on OVH MC-64-OC i7 7700K server i see during nginx compile
    Code (Text):
    make[2]: Entering directory `/svr-setup/zlib-cloudflare'
    ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o adler32.o adler32.c
    ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o crc32.o crc32.c
    ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o deflate.o deflate.c
    ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o infback.o infback.c
    ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o inffast.o inffast.c
    ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o inflate.o inflate.c
    ok
    ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o inftrees.o inftrees.c
    ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o trees.o trees.c
    ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o zutil.o zutil.c
    ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o compress.o compress.c
    checking for sysroot... ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o uncompr.o uncompr.c
    no
    checking for a working dd... ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o gzclose.o gzclose.c
    ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o gzlib.o gzlib.c
    ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o gzread.o gzread.c
    ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o gzwrite.o gzwrite.c
    ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN -c contrib/amd64/crc32-pclmul_asm.S -o crc32-pclmul_asm.o
    /usr/bin/dd
    checking how to truncate binary pipes... ar rc libz.a adler32.o crc32.o deflate.o infback.o inffast.o inflate.o inftrees.o trees.o zutil.o compress.o uncompr.o gzclose.o gzlib.o gzread.o gzwrite.o  crc32-pclmul_asm.o
    /usr/bin/dd bs=4096 count=1
    make[2]: Leaving directory `/svr-setup/zlib-cloudflare'
    
     
  19. bassie

    bassie Active Member

    869
    203
    43
    Apr 29, 2016
    Ratings:
    +609
    Local Time:
    3:32 PM
    Something else first. When building Cloudflare Zlib.
    Make sure that you are cloning branch gcc.amd64, as the Cloudflare Zlib master branch is the original Zlib 1.2.8 without Cloudflare's touch.

    About CPU's without SSE4.2.
    Don't know as Intel introduced it almost 10 years ago.
    Don't have those CPU's anymore to test.

    The Cloudflare Zlib compile script will check for compatibility.
    So no problem. There aren't that much server CPU's up and running from pre 2008 anymore. And if so, you can easily create a fail-back.

    It's all about optimization these days. And every percent counts.
    That's one of the reasons why I started with Cloudflare Zlib.

     
    • Informative Informative x 1
  20. eva2000

    eva2000 Administrator Staff Member

    35,076
    7,742
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +11,941
    Local Time:
    11:32 PM
    Nginx 1.15.x
    MariaDB 5.5/10.x
    cheers the github repo has gcc.amd64 as the default so git clone goes there by default
    Code (Text):
     git branch
    * gcc.amd64
    
     
..