Nginx segfaults

Hello
I know this is not related to centmin and centmin installer script but just hope somebody may have same issue.
I'm getting segfaults on nginx :

Code:

kernel: nginx[2007]: segfault at 1b8 ip 0000000000527c62 sp 00007fff3187c190 error 4 in nginx[400000+c19000]
kernel: nginx[2012]: segfault at 1b8 ip 0000000000527c62 sp 00007fff3187c190 error 4 in nginx[400000+c19000]
kernel: nginx[2002]: segfault at 1b8 ip 0000000000527c62 sp 00007fff3187c190 error 4 in nginx[400000+c19000]
kernel: nginx[2025]: segfault at 1b8 ip 0000000000527c62 sp 00007fff3187c190 error 4 in nginx[400000+c19000]
kernel: nginx[2001]: segfault at 1b8 ip 0000000000527c62 sp 00007fff3187c190 error 4 in nginx[400000+c19000]
kernel: nginx[1998]: segfault at 1b8 ip 0000000000527c62 sp 00007fff3187c190 error 4 in nginx[400000+c19000]
kernel: nginx[2022]: segfault at 1b8 ip 0000000000527c62 sp 00007fff3187c190 error 4 in nginx[400000+c19000]
kernel: nginx[2016]: segfault at 1b8 ip 0000000000527c62 sp 00007fff3187c1f0 error 4 in nginx[400000+c19000]

I'm on 123.08stable and nginx 1.9.4 , already tried both openssl and libressl , different nginx versions like 1.8.0 or 1.7.7 and no luck, still getting those errors from kernel.

---

 

Sorry for not posting details.
CentOS release 6.7 which is a vmware esxi 6 vm
uname output:

Code:

Linux xxx.xxx.xxx 2.6.32-573.3.1.el6.x86_64 #1 SMP Thu Aug 13 22:55:16 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

nginx details:

Code:

nginx version: nginx/1.9.4
built by clang 3.4.2 (tags/RELEASE_34/dot2-final)
built with OpenSSL 1.0.2d 9 Jul 2015
TLS SNI support enabled
configure arguments: --with-ld-opt='-ljemalloc -Wl,-z,relro -Wl,-rpath,/usr/local/lib' --with-cc-opt='-m64 -mtune=native -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wno-deprecated-declarations -Wno-unused-parameter -Wno-unused-const-variable -Wno-conditional-uninitialized -Wno-mismatched-tags -Wno-c++11-extensions -Wno-sometimes-uninitialized -Wno-parentheses-equality -Wno-tautological-compare -Wno-self-assign -Wno-deprecated-register -Wno-deprecated -Wno-invalid-source-encoding -Wno-pointer-sign -Wno-parentheses -Wno-enum-conversion' --sbin-path=/usr/local/sbin/nginx --conf-path=/usr/local/nginx/conf/nginx.conf --with-http_ssl_module --with-http_gzip_static_module --with-http_stub_status_module --with-http_sub_module --with-http_addition_module --with-http_image_filter_module --with-http_secure_link_module --with-http_realip_module --with-http_geoip_module --with-openssl-opt=enable-tlsext --add-module=../ngx-fancyindex-ngx-fancyindex --add-module=../ngx_cache_purge-2.3 --add-module=../nginx-accesskey-2.0.3 --add-module=../nginx-http-concat-master --add-module=../openresty-memc-nginx-module-4f6f78f --add-module=../openresty-srcache-nginx-module-ffa9ab7 --add-module=../ngx_devel_kit-0.2.19 --add-module=../set-misc-nginx-module-0.29 --add-module=../echo-nginx-module-0.58 --add-module=../redis2-nginx-module-0.12 --add-module=../ngx_http_redis-0.3.7 --add-module=../lua-nginx-module-0.9.16 --add-module=../lua-upstream-nginx-module-0.03 --add-module=../lua-upstream-cache-nginx-module-0.1.1 --add-module=../nginx_upstream_check_module-0.3.0 --add-module=../nginx-module-vts --add-module=../headers-more-nginx-module-0.261 --with-openssl=../openssl-1.0.2d --with-libatomic --with-threads --with-stream --with-stream_ssl_module --with-pcre=../pcre-8.37 --with-pcre-jit --with-http_spdy_module --add-module=../ngx_pagespeed-release-1.9.32.6-beta

 

Thanks for the debug tip. I will give it a try.
I'm not using SSL on this server. I searched through some old bugs in nginx website and there were some issues with ssl options. Do you recommend recompiling nginx without ssl ? (and how?)

 

Maybe I'm lucky but I updated the centminmod source regarding the debugmod switch and recompiled the nginx, so far so good! I will post if error occurs again.
Thanks for support anyway.

 

This debug-mode generate huge amount of data for a very busy server with a complicated nginx config. +300mb in 5 mins.
So could not extract useful data when error happens!
maybe this:

Code:

2015/09/14 17:13:05 [debug] 27365#27365: *24691 close cached open file: /home/proxy_cache/b/0f/4ad84e6cf941ea873aef5c0288ef60fb, fd:210, c:0, u:1, 0
2015/09/14 17:13:05 [debug] 27365#27365: *24691 run cleanup: 00007FC3CE630228
2015/09/14 17:13:05 [debug] 27365#27365: *24691 http file cache cleanup
2015/09/14 17:13:05 [debug] 27365#27365: *24691 http file cache free, fd: 210
2015/09/14 17:13:05 [debug] 27365#27365: *24691 free: 00007FC3CE62E000, unused: 21328
2015/09/14 17:13:05 [debug] 27365#27365: *24691 free: 00007FC3CE2ED000
2015/09/14 17:13:05 [debug] 27365#27365: *24691 hc free: 0000000000000000 0
2015/09/14 17:13:05 [debug] 27365#27365: *24691 hc busy: 0000000000000000 0
2015/09/14 17:13:05 [debug] 27365#27365: *24691 reusable connection: 1
2015/09/14 17:13:05 [debug] 27365#27365: *24691 event timer add: 177: 10000:1442234595317
2015/09/14 17:13:05 [debug] 27365#27365: *24691 post event 00007FC4D31C4180
2015/09/14 17:13:05 [debug] 27365#27365: *24691 delete posted event 00007FC4D31C4180
2015/09/14 17:13:05 [debug] 27365#27365: *24691 http keepalive handler
2015/09/14 17:13:05 [debug] 27365#27365: *24691 malloc: 00007FC3CE2ED000:65536
2015/09/14 17:13:05 [debug] 27365#27365: *24691 recv: fd:177 -1 of 65536
2015/09/14 17:13:05 [debug] 27365#27365: *24691 recv() not ready (11: Resource temporarily unavailable)
2015/09/14 17:13:05 [debug] 27365#27365: *24691 free: 00007FC3CE2ED000
2015/09/14 17:13:05 [debug] 27365#27365: *24691 http empty handler
2015/09/14 17:13:15 [debug] 27365#27365: *24691 event timer del: 177: 1442234595317
2015/09/14 17:13:15 [debug] 27365#27365: *24691 http keepalive handler
2015/09/14 17:13:15 [debug] 27365#27365: *24691 close http connection: 177
2015/09/14 17:13:15 [debug] 27365#27365: *24691 reusable connection: 0
2015/09/14 17:13:15 [debug] 27365#27365: *24691 free: 0000000000000000
2015/09/14 17:13:15 [debug] 27365#27365: *24691 free: 00007FC3CE866200, unused: 128

I am trying to simplify my conf and see what happens.