Want to subscribe to topics you're interested in?
Become a Member

Sending Nginx Upgrade Logs To DropBox Via Rclone

Discussion in 'Centmin Mod Insights' started by eva2000, Jan 1, 2022.

Thread Status:
Not open for further replies.
  1. eva2000

    eva2000 Administrator Staff Member

    49,265
    11,294
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +17,571
    Local Time:
    6:30 PM
    Nginx 1.21.x
    MariaDB 10.x

    Using Rclone + Dropbox To Share Nginx Upgrade Logs



    Starting from January 27, 2022 AEST, Centmin Mod 123.09beta01 support sending Nginx upgrade logs via centmin.sh menu option 4 to a Dropbox designated account via Rclone. Below are the steps to enabled this.


    Centmin Mod 123.09beta01 already has an addons/rclone.sh script to install and configure Rclone allowing you to sync server side files with multiple Cloud storage providers which includes Dropbox. We can use Rclone instead to send Nginx upgrade logs including sanitized pcre2log.log to a Dropbox account - preferably a separate Dropbox account used only for this specific purposes.

    Steps to enable this include:

    Step 1. Setting persistent config file /etc/centminmod/custom_config.inc variables to enable Dropbox auto sending of log files and to enable addons/rclone.sh to install.

    Add to persistent config file /etc/centminmod/custom_config.inc (create it if it doesn't exist)
    Code (Text):
    RCLONE_ENABLE='y'
    DROPBOX_SEND='y'
    


    Step 2. Sign up for a new Dropbox account (my referral link) allows me to add 500MB storage quota for each referral. Then create your own Dropbox App Id at Login - Dropbox as per https://rclone.org/dropbox/#get-your-own-dropbox-app-id instructions but also Generated access token and set no expiration for token.

    dropbox-app-id-create-01.png
    dropbox-app-id-create-02.png


    Step 3. Install Rclone via addons/rclone.sh and configure it. The resulting created Rclone config file will be at /root/.config/rclone/rclone.conf

    Code (Text):
    /usr/local/src/centminmod/addons/rclone.sh install
    

    The addons/rclone.sh install command will install latest rclone binary to /usr/sbin/rclone and then immediately prompt you to configure your first Rclone remote (storage provider i.e. Dropbox). Pay attention to the instructions for configuration below. Centmin Mod will look for the Rclone remote named = dropbox1 so make sure you use that for remote name.
    1. select n to create new remote named = dropbox1
    2. select 11 for storage option = Dropbox
    3. hit enter to leave blank at client_id prompt & client_secret prompt
    4. enter y for advanced config
    5. for OAuth Access Token prompt, enter the generated token from DropBox App Id creation in step 2 above
    6. hit enter to leave bank for auth_url, token_url etc or enter your own configurations until you reach auto config prompt
    7. enter n for auto config
    8. at already have a token refresh prompt, enter n
    9. at prompt if the config is okay, enter y
    10. then q to quit
    Then you should have a working Rclone remote named dropbox1 that allows you to send files on your server to your desired Dropbox account. Whenever you run Nginx upgrades via centmin.sh menu option 4, you will now automatically send the logs to your Dropbox account. The one we are most focused on is the sanitized pcre2log.log compressed *.gz log. They're basically your Nginx upgrade logs but with sensitive info removed.

    Example usage from centmin.sh menu option 4 where RCLONE_ENABLE='y' + DROPBOX_SEND='y' and /usr/sbin/rclone exists and rclone dropbox1 remote exists. At end of Nginx recompiles, there would be this output
    Code (Text):
    dropbox_generate 270122-031534
    
    Send Nginx Upgrade Logs To Dropbox Account
    
    /usr/sbin/rclone copy nginx_pcre_checks_270122-031534.log.gz dropbox1:
    
    /usr/sbin/rclone copy private-cm_123.09beta01.b786_270122-031534_nginx_upgrade.log.gz dropbox1:
    
    /usr/sbin/rclone copy public-nginx-pcre2log-270122-031534.log.gz dropbox1:
    
    /usr/sbin/rclone copy centminmod_123.09beta01.b786_270122-031534_pcre2_install.log.gz dropbox1:
    
    /usr/sbin/rclone copy nginx_autoconf.err.270122-031534.log.gz dropbox1:
    
    /usr/sbin/rclone copy nginx_pcre2_download_270122-031534.log.gz dropbox1:
    

    And you'll see this uploaded to your Dropbox account. You just select the pcre2log with date timestamp and can select the 3 horizontal dots to far right of the desired file to select the share option and create and copy the share link.

    nginx-upgrade-logs-dropbox-rclone-01.png

    cm-dropbox-rclone-share-01.png

    cm-dropbox-rclone-share-02.png

    Note the default limits for max file size are 48-150MB, so more than enough for sending compressed Nginx upgrade logs.

    You can check the remotes using listremotes command
    Code (Text):
    rclone listremotes
    dropbox1:
    

    Listing currently uploaded Dropbox files which will be in a directory called Apps/yourchosen_app_folder_name. For this example the App folder name = cmlogs
    Code (Text):
    rclone ls dropbox1:
         5362 centminmod_123.09beta01.b786_270122-031534_pcre2_install.log.gz
         2916 nginx_autoconf.err.270122-031534.log.gz
          804 nginx_pcre2_download_270122-031534.log.gz
         1639 nginx_pcre_checks_270122-031534.log.gz
        58985 private-cm_123.09beta01.b786_270122-031534_nginx_upgrade.log.gz
        57545 public-nginx-pcre2log-270122-031534.log.gz
    


    Dropbox auto sent logs on centmin.sh menu option 4 nginx upgrade runs with renamed private and public Nginx upgrade logs denoting that the public one is sanitized with removal of identifying Nginx vhost domain name references etc. Thus suited for public sharing.
     
    Last edited: Jan 27, 2022
Thread Status:
Not open for further replies.