/ Register

Join the community today
Become a Member

Nginx [PATCH] Fix error when building a Nginx source RPM with custom Zlib - Nginx 1.13.7

Discussion in 'Nginx and PHP-FPM news & discussions' started by buik, Dec 12, 2017.

Tags:
Previous Thread Next Thread
Loading...
Page 1 of 3
  1. Dec 12, 2017 #1
    buik

    buik “The best traveler is one without a camera.”

    2,026
    524
    113
    Apr 29, 2016
    Flanders
    Ratings:
    +1,674
    Local Time:
    10:43 PM
    When building a Nginx source RPM with your own custom Zlib.
    The following error could appear. Even if -fPIC is already defined in the .spec:

    Code:
    "Relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
libz.a: error adding symbols: Bad value collect2: error: ld returned 1 exit status"
    This patch will fix the error:

    Nginx-fix-error-building-srpm-custom-zlib

    Error not directly related to Centminmod.
    But nonetheless annoying.

    That's why I made a patch.
    I am sure that there are others on the Internet with the same problem.
     
  2. Dec 12, 2017 #2
    eva2000

    eva2000 Administrator Staff Member

    54,107
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    7:43 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    interesting you using plain vanilla zlib 1.2.11 or a zlib fork ? Not sure why you'd get that error only on source RPM builds and not source installs ? I know Centmin Mod Ngixn 1.13.7 with zlib 1.2.11 compiles fine to date.
     
  3. Dec 12, 2017 #3
    buik

    buik “The best traveler is one without a camera.”

    2,026
    524
    113
    Apr 29, 2016
    Flanders
    Ratings:
    +1,674
    Local Time:
    10:43 PM
    Zlib and Zlib forks (Zlib-ng, Cloudflare, Intel), even the Fedora and EL7 Zlib source,
    all custom Zlib's will fail without patch when building source RPM's with the Nginx option: --with-zlib=/.

    As written before the error is not directly related to Centminmod :)
    Manual compiling works fine as long as it is not related to building source RPM's. It could be dynamically building, it could be building in a rpmbuild environment.

    Honestly. I don't know.
    I found no comparable case. Apparently no one seems to compile Zlib into source RPM Nginx.

    So it takes a lot more time to find out (many unanswered questions on the internet about -fPIC), than to write a simple patch.

    Nginx + Cloudflare Zlib works fine now.
    Nice and fast too :)

    Goal achieved.
     
    Last edited: Dec 12, 2017
  4. Dec 12, 2017 #4
    buik

    buik “The best traveler is one without a camera.”

    2,026
    524
    113
    Apr 29, 2016
    Flanders
    Ratings:
    +1,674
    Local Time:
    10:43 PM
    My reaction is above this. A question came to me. Hence 2 replys.

    @eva2000 Question for you. Why using Zlib-ng?
    Is Zlib-ng stable? (thoroughly tested?) as it is a blend between Cloudflare and Intel's fork.

    But perhaps more important, is it safe?
    I do not see anywhere that they fix CVE's?

    And then. What about speed.
    mtl1979 one of the code maintainer of Zlib-ng writes:

    As far as I can read it, Cloudflare's fork is the best choice.
    Well tested, secure, maintained, fast.

    * I know the research of their own product is not always objective.
     
    Last edited: Dec 12, 2017
  5. Dec 12, 2017 #5
    eva2000

    eva2000 Administrator Staff Member

    54,107
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    7:43 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    I am just using stock zlib 1.2.11. But you're using Cloudflare's zlib right ?
     
  6. Dec 13, 2017 #6
    buik

    buik “The best traveler is one without a camera.”

    2,026
    524
    113
    Apr 29, 2016
    Flanders
    Ratings:
    +1,674
    Local Time:
    10:43 PM
    Yes using Cloudflare's Zlib fork here.
    Aha so Zlib-ng is your fail-back then?
     
  7. Dec 13, 2017 #7
    eva2000

    eva2000 Administrator Staff Member

    54,107
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    7:43 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    it's a work in progress development - just fails to properly compile right now so not used heh
     
  8. Dec 13, 2017 #8
    buik

    buik “The best traveler is one without a camera.”

    2,026
    524
    113
    Apr 29, 2016
    Flanders
    Ratings:
    +1,674
    Local Time:
    10:43 PM
    It is advisable to switch to Cloudflare's Zlib.
    If you can't compile it properly.

    Zlib-ng seems more of a project to support more CPU's and to do cleanups in the code, rather than speed.

    Centminmod seems to find speed important :)
     
  9. Dec 13, 2017 #9
    eva2000

    eva2000 Administrator Staff Member

    54,107
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    7:43 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
  10. Dec 13, 2017 #10
    buik

    buik “The best traveler is one without a camera.”

    2,026
    524
    113
    Apr 29, 2016
    Flanders
    Ratings:
    +1,674
    Local Time:
    10:43 PM
    Working fine here. Try again, I'm sure you can :)

    Although I did not read the thread you quote.
    Based on CentOS 7.4 minimal, without Kdump.

    For everyone who wants to know.
    I have executed the following to compile Nginx 1.13.7 with Cloudflare Zlib.
    Please note that a service is not available.
    The idea was purely to test whether it works with Cloudflare Zlib.
    Before I continued building a source RPM.
     
    Last edited: Dec 13, 2017
  11. Dec 13, 2017 #11
    eva2000

    eva2000 Administrator Staff Member

    54,107
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    7:43 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    if you wrap links in [PLAIN][/PLAIN] tags they don't get parsed :)

    thanks, will re-try and see if it works :)
     
  12. Dec 13, 2017 #12
    eva2000

    eva2000 Administrator Staff Member

    54,107
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    7:43 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
  13. Dec 13, 2017 #13
    buik

    buik “The best traveler is one without a camera.”

    2,026
    524
    113
    Apr 29, 2016
    Flanders
    Ratings:
    +1,674
    Local Time:
    10:43 PM
    Try:
    cd /svr-setup/zlib-cloudflare
    make -f Makefile.in distclean
    cd /svr-setup/nginx-1.13.7
    etc etc.
     
  14. Dec 13, 2017 #14
    eva2000

    eva2000 Administrator Staff Member

    54,107
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    7:43 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    that worked in allowing cloudflare zlib compile with nginx centmin mod 123.09beta01 with cloudflare zlib fork manual compile · GitHub

    though lsof still shows zlib 1.2.11
    Code (Text):
    lsof | grep nginx | grep -v php-fpm | grep libz
nginx      1720            root  mem       REG                9,2     117592   12089342 /usr/local/lib/libz.so.1.2.11
nginx      1721           nginx  mem       REG                9,2     117592   12089342 /usr/local/lib/libz.so.1.2.11
nginx      1722           nginx  mem       REG                9,2     117592   12089342 /usr/local/lib/libz.so.1.2.11
nginx      1724           nginx  mem       REG                9,2     117592   12089342 /usr/local/lib/libz.so.1.2.11
nginx      1725           nginx  mem       REG                9,2     117592   12089342 /usr/local/lib/libz.so.1.2.11
     
  15. Dec 13, 2017 #15
    eva2000

    eva2000 Administrator Staff Member

    54,107
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    7:43 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    k fixed it now Cloudflare zlib is v1.2.8 vs standard zlib v1.2.11
    Code (Text):
    lsof | grep nginx | grep -v php-fpm | grep libz
nginx     12345            root  mem       REG                9,2     116840   12089344 /usr/local/lib/libz.so.1.2.8
nginx     12346           nginx  mem       REG                9,2     116840   12089344 /usr/local/lib/libz.so.1.2.8
nginx     12347           nginx  mem       REG                9,2     116840   12089344 /usr/local/lib/libz.so.1.2.8
nginx     12349           nginx  mem       REG                9,2     116840   12089344 /usr/local/lib/libz.so.1.2.8
nginx     12350           nginx  mem       REG                9,2     116840   12089344 /usr/local/lib/libz.so.1.2.8
     
  16. Dec 13, 2017 #16
    buik

    buik “The best traveler is one without a camera.”

    2,026
    524
    113
    Apr 29, 2016
    Flanders
    Ratings:
    +1,674
    Local Time:
    10:43 PM
    Nice.
     
    Last edited: Dec 13, 2017
  17. Dec 13, 2017 #17
    eva2000

    eva2000 Administrator Staff Member

    54,107
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    7:43 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    cheers mate for the pointers :)

    long term testing to see how this fairs first :D
     
  18. Dec 13, 2017 #18
    eva2000

    eva2000 Administrator Staff Member

    54,107
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    7:43 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    wonder if cloudflare zlib requires SSE4.2 to work and what happens on cpus without SSE4.2

    on OVH MC-64-OC i7 7700K server i see during nginx compile
    Code (Text):
    make[2]: Entering directory `/svr-setup/zlib-cloudflare'
ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o adler32.o adler32.c
ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o crc32.o crc32.c
ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o deflate.o deflate.c
ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o infback.o infback.c
ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o inffast.o inffast.c
ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o inflate.o inflate.c
ok
ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o inftrees.o inftrees.c
ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o trees.o trees.c
ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o zutil.o zutil.c
ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o compress.o compress.c
checking for sysroot... ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o uncompr.o uncompr.c
no
checking for a working dd... ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o gzclose.o gzclose.c
ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o gzlib.o gzlib.c
ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o gzread.o gzread.c
ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN   -c -o gzwrite.o gzwrite.c
ccache gcc -DHAS_PCLMUL -mpclmul -DHAS_SSE42 -msse4.2 -O2 -fomit-frame-pointer -pipe   -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN -c contrib/amd64/crc32-pclmul_asm.S -o crc32-pclmul_asm.o
/usr/bin/dd
checking how to truncate binary pipes... ar rc libz.a adler32.o crc32.o deflate.o infback.o inffast.o inflate.o inftrees.o trees.o zutil.o compress.o uncompr.o gzclose.o gzlib.o gzread.o gzwrite.o  crc32-pclmul_asm.o
/usr/bin/dd bs=4096 count=1
make[2]: Leaving directory `/svr-setup/zlib-cloudflare'
     
  19. Dec 13, 2017 #19
    buik

    buik “The best traveler is one without a camera.”

    2,026
    524
    113
    Apr 29, 2016
    Flanders
    Ratings:
    +1,674
    Local Time:
    10:43 PM
    Something else first. When building Cloudflare Zlib.
    Make sure that you are cloning branch gcc.amd64, as the Cloudflare Zlib master branch is the original Zlib 1.2.8 without Cloudflare's touch.

    About CPU's without SSE4.2.
    Don't know as Intel introduced it almost 10 years ago.
    Don't have those CPU's anymore to test.

    The Cloudflare Zlib compile script will check for compatibility.
    So no problem. There aren't that much server CPU's up and running from pre 2008 anymore. And if so, you can easily create a fail-back.

    It's all about optimization these days. And every percent counts.
    That's one of the reasons why I started with Cloudflare Zlib.

     
  20. Dec 13, 2017 #20
    eva2000

    eva2000 Administrator Staff Member

    54,107
    12,179
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,738
    Local Time:
    7:43 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    cheers the github repo has gcc.amd64 as the default so git clone goes there by default
    Code (Text):
     git branch
* gcc.amd64
     
Previous Thread Next Thread
Loading...
Page 1 of 3

.