Join the community today
Register Now

Sort urls by connections from access.log file

Discussion in 'System Administration' started by pamamolf, May 5, 2016.

  1. pamamolf

    pamamolf Premium Member Premium Member

    4,101
    428
    83
    May 31, 2014
    Ratings:
    +837
    Local Time:
    6:12 AM
    Nginx-1.26.x
    MariaDB 10.6.x
    Hi


    I am using this command to short ip's with more connections:

    Code:
    netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n
    But using Cloudflare i can see only there ip's and not from users.....

    What command should i use to do the same and get the user's ip's with more connections from hosts access.log file ?

    Thanks
     
  2. eva2000

    eva2000 Administrator Staff Member

    55,189
    12,251
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,829
    Local Time:
    1:12 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    If you use a reverse proxy in front of Centmin Mod Nginx, you need to setup nginx realip to be passed onto Nginx.

    See Getting Started Guide step 4 and setting correct real ip via nginx module config at Nginx Cloudflare & Incapsula (reverse proxy HttpRealIpModule) - CentminMod.com LEMP Nginx web stack for CentOS.

    If using Centmin Mod 123.09beta01 and newer, there's an added tools/csfcf.sh script to aid in this. Details at:
    You just need to setup a cronjob to run
    Code (Text):
    /usr/local/src/centminmod/tools/csfcf.sh auto

    and ensure your nginx.conf http{} context has the include file /usr/local/nginx/conf/cloudflare.conf and/or your individual nginx vhost's server contexts has the same include file
    Code (Text):
    http {
    map_hash_bucket_size 128;
    map_hash_max_size 2048;
    server_names_hash_bucket_size 128;
    server_names_hash_max_size 2048;
    
    limit_req_zone $binary_remote_addr zone=xwplogin:16m rate=40r/m;
    #limit_conn_zone $binary_remote_addr zone=xwpconlimit:16m;
    
    more_set_headers "Server: nginx centminmod";
    more_set_headers "X-Powered-By: centminmod";
    
    include /usr/local/nginx/conf/cloudflare.conf;
    include /usr/local/nginx/conf/maintenance.conf;
    include /usr/local/nginx/conf/vts_http.conf;
    include /usr/local/nginx/conf/geoip.conf;
    #include /usr/local/nginx/conf/pagespeedadmin.conf;
    include /usr/local/nginx/conf/fastcgi_param_https_map.conf;

    Then restart nginx server via command shortcut
    Code (Text):
    ngxrestart

    or
    Code (Text):
    service nginx restart
     
  3. pamamolf

    pamamolf Premium Member Premium Member

    4,101
    428
    83
    May 31, 2014
    Ratings:
    +837
    Local Time:
    6:12 AM
    Nginx-1.26.x
    MariaDB 10.6.x
    I think i have already this and is working great as i can see on vhost access.log file there are the correct users ip's but i don't know how to short them as i do with:

    Code:
    netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n
     
  4. eva2000

    eva2000 Administrator Staff Member

    55,189
    12,251
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,829
    Local Time:
    1:12 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+