Welcome to Centmin Mod Community
Register Now

Sysadmin Blocking robots.txt

Discussion in 'System Administration' started by inthecloudblog, Aug 12, 2016.

  1. inthecloudblog

    inthecloudblog Active Member

    178
    33
    28
    Jan 26, 2016
    Ratings:
    +74
    Local Time:
    12:56 AM
    1.4.6
    Hi . I'm not sure if this is the proper category.
    I want to block robots.txt until one of the webs is done.. any simple way to achieve such thing?

    EDIT:

    just read

    Blocking Certain Robots | Centmin Mod Community

    still remains unclear to me how to make my web unreacheable to yahoo,google, bing,etc..
     
    Last edited: Aug 12, 2016
  2. eva2000

    eva2000 Administrator Staff Member

    30,152
    6,782
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,133
    Local Time:
    1:56 PM
    Nginx 1.13.x
    MariaDB 5.5
    • Friendly Friendly x 1
  3. inthecloudblog

    inthecloudblog Active Member

    178
    33
    28
    Jan 26, 2016
    Ratings:
    +74
    Local Time:
    12:56 AM
    1.4.6
    I will re read about the robots stuff.. ( I was reading while installing centminmod)..

    I was waiting for centminmod to be installed and after it had been done I entered :
    nano /usr/local/nginx/conf/nginx.conf

    I found ( I use CF), so I've uncommented since my domain is not resolving :

    include /usr/local/nginx/conf/cloudflare.conf;


    and I also restarted nginx...

    still does not resolve.. Should I open a new thread?

    I get a 522 error.. ( using the beta build)


    I'm typing nonsense.. going to bed..


    EDIT: damn ... was entering old ips into cf...need some sleep
     
    Last edited: Aug 12, 2016
    • Funny Funny x 1
  4. eva2000

    eva2000 Administrator Staff Member

    30,152
    6,782
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,133
    Local Time:
    1:56 PM
    Nginx 1.13.x
    MariaDB 5.5
    yeah that would cause 522s
     
  5. inthecloudblog

    inthecloudblog Active Member

    178
    33
    28
    Jan 26, 2016
    Ratings:
    +74
    Local Time:
    12:56 AM
    1.4.6
    I've realized that :
    1- I posted in a non suitable section.
    2- Posted asking something when the setup was in progress.
    3- The original post was derrailed by me just because I was way too drowsy and used the old IP's from a droplet( DO) I had just destroyed and placing those in CF

    I really appreciate George's huge and steady dedication and effort for this project. Also please accept and apology for being so confusing
     
    • Like Like x 1
  6. eva2000

    eva2000 Administrator Staff Member

    30,152
    6,782
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +10,133
    Local Time:
    1:56 PM
    Nginx 1.13.x
    MariaDB 5.5
    • Like Like x 1