Discover Centmin Mod today
Register Now

Sysadmin Blocking robots.txt

Discussion in 'System Administration' started by inthecloudblog, Aug 12, 2016.

  1. inthecloudblog

    inthecloudblog Active Member

    199
    36
    28
    Jan 26, 2016
    Ratings:
    +83
    Local Time:
    8:11 AM
    1.4.6
    Hi . I'm not sure if this is the proper category.
    I want to block robots.txt until one of the webs is done.. any simple way to achieve such thing?

    EDIT:

    just read

    Blocking Certain Robots | Centmin Mod Community


    still remains unclear to me how to make my web unreacheable to yahoo,google, bing,etc..
     
    Last edited: Aug 12, 2016
  2. eva2000

    eva2000 Administrator Staff Member

    53,866
    12,160
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,712
    Local Time:
    9:11 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
  3. inthecloudblog

    inthecloudblog Active Member

    199
    36
    28
    Jan 26, 2016
    Ratings:
    +83
    Local Time:
    8:11 AM
    1.4.6
    I will re read about the robots stuff.. ( I was reading while installing centminmod)..

    I was waiting for centminmod to be installed and after it had been done I entered :
    nano /usr/local/nginx/conf/nginx.conf

    I found ( I use CF), so I've uncommented since my domain is not resolving :

    include /usr/local/nginx/conf/cloudflare.conf;


    and I also restarted nginx...

    still does not resolve.. Should I open a new thread?

    I get a 522 error.. ( using the beta build)


    I'm typing nonsense.. going to bed..


    EDIT: damn ... was entering old ips into cf...need some sleep
     
    Last edited: Aug 12, 2016
  4. eva2000

    eva2000 Administrator Staff Member

    53,866
    12,160
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,712
    Local Time:
    9:11 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    yeah that would cause 522s
     
  5. inthecloudblog

    inthecloudblog Active Member

    199
    36
    28
    Jan 26, 2016
    Ratings:
    +83
    Local Time:
    8:11 AM
    1.4.6
    I've realized that :
    1- I posted in a non suitable section.
    2- Posted asking something when the setup was in progress.
    3- The original post was derrailed by me just because I was way too drowsy and used the old IP's from a droplet( DO) I had just destroyed and placing those in CF

    I really appreciate George's huge and steady dedication and effort for this project. Also please accept and apology for being so confusing
     
  6. eva2000

    eva2000 Administrator Staff Member

    53,866
    12,160
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,712
    Local Time:
    9:11 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+