Learn about Centmin Mod LEMP Stack today
Register Now

Sysadmin Broke something; feel stupid

Discussion in 'System Administration' started by Derek, Sep 29, 2016.

  1. Derek

    Derek Member

    47
    9
    8
    Aug 5, 2016
    Ratings:
    +22
    Local Time:
    10:04 AM
    So I've been using centminmod as a base for Xenforo as I work to migrate my vBulletin forum. So far so good, everything running as planned, until I installed Elasticsearch and tried to get searches to work.

    I couldn't figure out the problem, but I noticed the server was hitting swap, so I powered down the VM, changed the memory from 32G to 64G (yeah, I've got 23G as innodb cache), booted back up, and now I can't reach the web server. I can access it from the command line using Lynx, but remote servers both on the local subnet and out on the Internet are just getting timeouts.


    • Networking seems to be working - Yum installations work, for instance, and I can ssh in without problem.
    • CSF firewall being turned off didn't make a difference.
    • Downgrading the memory back to 32G didn't make a difference.
    • Connections to the domain I host and the IP address of the centminmod machine itself both work via lynx, but not from remote servers.
    I'm not sure what to do at this point. I'm certain it's something simple, but my brain isn't working well enough, apparently.

    Thoughts?
     
    Last edited: Sep 29, 2016
  2. eva2000

    eva2000 Administrator Staff Member

    54,907
    12,240
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,811
    Local Time:
    12:04 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    what do you get for these commands output - post output in CODE bbcode tags
    Code (Text):
    ping -c4 google.com
    curl -Isv https://www.google.com
    curl -Isv http://yourdomain.com
    csf -g YOUR_ISP_IPADDRESS
    csf -g YOUR_REMOTESERVER_IPADDRESS
    

    who's your web host ? server plan you are on ?

    if you're connecting from remote servers to Centmin Mod server, you need to whitelist the remote server's IP in CSF firewall

    So to be able to remotely connect to external servers, you need to whitelist and allow the remote IP address see the info below:
     
  3. Derek

    Derek Member

    47
    9
    8
    Aug 5, 2016
    Ratings:
    +22
    Local Time:
    10:04 AM
    It was failing with the without CSF enabled. I reenabled CSF for your request.
    Code (Text):
    deleted.  Problem solved, so why leave diagnostic data up?
    
    




    It's a bunch of servers in a rack. Virtualized, so the plan is whatever I want. Max memory is 128G per server, so there's that.

    Even for known services like ssh?
     
    Last edited: Sep 29, 2016
  4. Derek

    Derek Member

    47
    9
    8
    Aug 5, 2016
    Ratings:
    +22
    Local Time:
    10:04 AM
    Aaaaand now it's working. Nothing's changed other than turning CSF back on.

    That worries me.
     
  5. eva2000

    eva2000 Administrator Staff Member

    54,907
    12,240
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,811
    Local Time:
    12:04 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    yup CSF Firewall secures all ports on server inbound and outbound. If a server remotely wants to connect, you need to whitelist in CSF Firewall the ip of remote server or whitelist the ip and port as outlined at CSF - CSF Firewall info | Centmin Mod Community (fine grain control to allow remote ip to connect to only a specific port).

    Same with outbound on Centmin Mod server, if you want to remotely connect to remote server on non-standard ports. CSF outbound whitelists port 80 and 443 for http and https and 22 for SSHD. But if your remote server's SSHD port is non-standard and not 22, then need to whitelist in CSF Firewall TCP_OUT/TCP6_OUT in /etc/csf/csf.conf the non-standard port. Otherwise you can't connect outside of Centmin Mod server. CSF Firewall secures your server inbound and outbound as outlined at Getting Started Guide step 4 and centminmod.com/csf_firewall.html :)
     
  6. eva2000

    eva2000 Administrator Staff Member

    54,907
    12,240
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,811
    Local Time:
    12:04 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    CSF Firewall whitelists known common ports, if you disable CSF firewall, all ports would be blocked unless you manually set them up directly in iptables.
     
  7. Derek

    Derek Member

    47
    9
    8
    Aug 5, 2016
    Ratings:
    +22
    Local Time:
    10:04 AM
    Well, we figured out exactly where my knowledge was lacking, didn't we? ;)

    Thanks!
     
  8. eva2000

    eva2000 Administrator Staff Member

    54,907
    12,240
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,811
    Local Time:
    12:04 AM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
  9. Derek

    Derek Member

    47
    9
    8
    Aug 5, 2016
    Ratings:
    +22
    Local Time:
    10:04 AM
    Yeah. I should really print it out and stick it in a binder...