Join the community today
Become a Member

Sort urls by connections from access.log file

Discussion in 'System Administration' started by pamamolf, May 5, 2016.

  1. pamamolf

    pamamolf Well-Known Member

    2,521
    231
    63
    May 31, 2014
    Ratings:
    +394
    Local Time:
    8:26 PM
    Nginx-1.13.x
    MariaDB 10.1.x
    Hi

    I am using this command to short ip's with more connections:

    Code:
    netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n
    But using Cloudflare i can see only there ip's and not from users.....

    What command should i use to do the same and get the user's ip's with more connections from hosts access.log file ?

    Thanks
     
  2. eva2000

    eva2000 Administrator Staff Member

    29,015
    6,582
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,772
    Local Time:
    3:26 AM
    Nginx 1.13.x
    MariaDB 5.5
    If you use a reverse proxy in front of Centmin Mod Nginx, you need to setup nginx realip to be passed onto Nginx.

    See Getting Started Guide step 4 and setting correct real ip via nginx module config at Nginx Cloudflare & Incapsula (reverse proxy HttpRealIpModule) - CentminMod.com LEMP Nginx web stack for CentOS.

    If using Centmin Mod 123.09beta01 and newer, there's an added tools/csfcf.sh script to aid in this. Details at:
    You just need to setup a cronjob to run
    Code (Text):
    /usr/local/src/centminmod/tools/csfcf.sh auto

    and ensure your nginx.conf http{} context has the include file /usr/local/nginx/conf/cloudflare.conf and/or your individual nginx vhost's server contexts has the same include file
    Code (Text):
    http {
    map_hash_bucket_size 128;
    map_hash_max_size 2048;
    server_names_hash_bucket_size 128;
    server_names_hash_max_size 2048;
    
    limit_req_zone $binary_remote_addr zone=xwplogin:16m rate=40r/m;
    #limit_conn_zone $binary_remote_addr zone=xwpconlimit:16m;
    
    more_set_headers "Server: nginx centminmod";
    more_set_headers "X-Powered-By: centminmod";
    
    include /usr/local/nginx/conf/cloudflare.conf;
    include /usr/local/nginx/conf/maintenance.conf;
    include /usr/local/nginx/conf/vts_http.conf;
    include /usr/local/nginx/conf/geoip.conf;
    #include /usr/local/nginx/conf/pagespeedadmin.conf;
    include /usr/local/nginx/conf/fastcgi_param_https_map.conf;

    Then restart nginx server via command shortcut
    Code (Text):
    ngxrestart

    or
    Code (Text):
    service nginx restart
     
  3. pamamolf

    pamamolf Well-Known Member

    2,521
    231
    63
    May 31, 2014
    Ratings:
    +394
    Local Time:
    8:26 PM
    Nginx-1.13.x
    MariaDB 10.1.x
    I think i have already this and is working great as i can see on vhost access.log file there are the correct users ip's but i don't know how to short them as i do with:

    Code:
    netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n
     
  4. eva2000

    eva2000 Administrator Staff Member

    29,015
    6,582
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +9,772
    Local Time:
    3:26 AM
    Nginx 1.13.x
    MariaDB 5.5
    • Informative Informative x 1