Wordpress Moving WordPress media to Amazon S3

Hey guys, it's good to be back here!

Recently, a major dance music portal became a client of mine. They came to me with a infected/hacked WordPress site and a overloaded MediaTemple VPS. I've cleaned the site, moved it to a new CentminMod server and now I'm working on a new cloud infrastructure for them.

I think everybody here would be pleased to know that only by moving the site from a 16GB RAM MediaTemple DV with LAMP to a 4GB DigitalOcean with CentminMod, I was able to reduce their memory and CPU usage from 96% to 16% and the site is now faster than ever according to my client.

Long live @eva2000 !!

---

The site is 3 years old and has a HUGE collection of mp3 files from it's weekly podcast. I'm planning to move all mp3 files to Amazon S3. Without it's mp3 files, the site occupies a total of 2.5GB disk space, however, including the mp3 files, the site is a 50GB monster. I believe that 47.5GB of static files is a waste of disk space on a SSD VPS, that's why I'm planning to move all mp3 files to AWS.

What I'm looking for here, is to share experiences about this kind of migration, specially about moving static (mp3) files from WordPress to S3 without breaking the links. I'm aware of AWS WordPress plugin, but it can only point the URL of newly uploaded files and what I'm looking for is a safe (non-regex sorcery) way to move all previously uploaded mp3 files and redirect old URLs to new S3 address.

Any ideas?

Cheers,
Guigo

 

Nice, thanks @eva2000. I was thinking the same, but was trying to avoid search and replace the database. I've used the InterConnectIt search and replace tool in the past, but more recently, I've became a fan of WP-CLI's search-replace command which also takes care of serialized data.

I'm aware of the above mentioned plugin, however, it says...

I'll try the CDN approach and let you guys know about the results.

Thanks!

 

@eva2000 I want to try a different approach, however, I'm having trouble with the proper rewrite rule. I want to rewrite all requests to .mp3 files...

from: http://www.example.com/wp-content/uploads/2012/08/artist-song.mp3
to: http://mp3.example.com/artist-song.mp3

This is what I have so far, unfortunately, it doesn't seems to work

Code:

location ~* \.mp3$ {
    rewrite ^(.*)\/(.*)\.mp3$ http://mp3.example.com/$2.mp3 last;
}

Below is the nginx vhost file (main nginx conf file is pure CentminMod).

Spoiler

Code:

server { 

    listen 80; listen 443;
    server_name www.example.com;
    return 301 $scheme://example.com$request_uri;
}

server {

    listen 80;
    server_name example.com;
   
    root         /var/www/example.com/public;
    error_log     /var/www/example.com/log/error.log;
    access_log     /var/www/example.com/log/access.log combined buffer=32k;

    add_header X-Frame-Options SAMEORIGIN;
    add_header X-Content-Type-Options nosniff;
    add_header X-XSS-Protection "1; mode=block";
    add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;";

    rewrite ^/wp-(.*)$ /wp/wp-$1 last;
    rewrite ^/sitemap_index\.xml$ /index.php?sitemap=1 last;
    rewrite ^/([^/]+?)-sitemap([0-9]+)?\.xml$ /index.php?sitemap=$1&sitemap_n=$2 last;

#    include /usr/local/nginx/conf/pagespeed.conf;
#    include /usr/local/nginx/conf/pagespeedhandler.conf;
#    include /usr/local/nginx/conf/pagespeedstatslog.conf;

    include /usr/local/nginx/conf/wp_secure.conf;
    include /usr/local/nginx/conf/wp_supercache.conf;
#    include /usr/local/nginx/conf/wp_fastcgicache.conf;

    location ~* \.mp3$ { rewrite ^(.*)\/(.*)\.mp3$ http://mp3.example.com/$2.mp3 last; }
    location ~* /wp-(admin|login|includes) { return 301 https://$server_name$request_uri; }
    location / {

#        auth_basic "Private";
#        auth_basic_user_file /var/www/example.com/conf/nginx/htpasswd;
       
        if (-f /var/www/example.com/conf/maintenance) { return 503; }
#        try_files $uri $uri/ /index.php?$args;
        try_files /wp/wp-content/cache/supercache/$http_host/$cache_uri/index.html $uri $uri/ /index.php?$args;
        autoindex off;
       
        location ~* \.php$ {

            try_files $uri =404;

            fastcgi_split_path_info ^(.+\.php)(/.+)$;
#            fastcgi_pass   127.0.0.1:9000;
            fastcgi_pass   unix:/tmp/example.com.sock;
            fastcgi_index  index.php;
#            fastcgi_param  SCRIPT_FILENAME    $document_root$fastcgi_script_name;
            fastcgi_param  SCRIPT_FILENAME    $request_filename;

            # NinjaFirewall PHPRC
            fastcgi_param PHPRC /var/www/example.com/public/.user.ini;

            include fastcgi_param.conf;
#            include fastcgi_cache.conf;
        }
    }
   
    include /usr/local/nginx/conf/block.conf;
    include /usr/local/nginx/conf/drop.conf;
    include /usr/local/nginx/conf/staticfiles.conf;
    include /usr/local/nginx/conf/errorpage.conf;
}

server {

    listen 443 ssl spdy;
    server_name example.com;

    root         /var/www/example.com/public;
    error_log     /var/www/example.com/log/error.log;
    access_log     /var/www/example.com/log/access.log combined buffer=32k;

    ssl_certificate            /var/www/example.com/ssl/example.com.crt;
    ssl_certificate_key        /var/www/example.com/ssl/example.com.key;
   
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_session_cache shared:SSL:10m;

    ssl_ciphers 'AES128+EECDH:AES128+EDH:!aNULL';
    ssl_prefer_server_ciphers on;
    ssl_session_timeout 10m;
   
    add_header Alternate-Protocol 443:npn-spdy/2;
    add_header X-Frame-Options SAMEORIGIN;
    add_header X-Content-Type-Options nosniff;
    add_header X-XSS-Protection "1; mode=block";
    add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;";
   
    rewrite ^/wp-(.*)$ /wp/wp-$1 last;
    rewrite ^/sitemap_index\.xml$ /index.php?sitemap=1 last;
    rewrite ^/([^/]+?)-sitemap([0-9]+)?\.xml$ /index.php?sitemap=$1&sitemap_n=$2 last;
    rewrite ^/wp/favicon(.*)$ $scheme://$server_name/favicon$1 last;
   
#    include /usr/local/nginx/conf/pagespeed.conf;
#    include /usr/local/nginx/conf/pagespeedhandler.conf;
#    include /usr/local/nginx/conf/pagespeedstatslog.conf;

    include /usr/local/nginx/conf/wp_secure.conf;
#    include /usr/local/nginx/conf/wp_supercache.conf;
#    include /usr/local/nginx/conf/wp_fastcgicache.conf;

    location / { return 301 http://$server_name$request_uri; }   
    location ~* \.mp3$ { rewrite ^(.*)\/(.*)\.mp3$ http://mp3.example.com/$2.mp3 last; }
    location ~* /wp-(admin|login|includes|content) {

#        auth_basic "Private";
#        auth_basic_user_file /var/www/example.com/conf/nginx/htpasswd;

        if (-f /var/www/example.com/conf/maintenance) { return 503; }
        try_files $uri $uri/ /wp/index.php?$args;
#        try_files /wp/wp-content/cache/supercache/$http_host/$cache_uri/index.html $uri $uri/ /index.php?$args;
        autoindex off;

        location ~* \.php$ {

            try_files $uri =404;

            fastcgi_split_path_info ^(.+\.php)(/.+)$;
#            fastcgi_pass   127.0.0.1:9000;
            fastcgi_pass   unix:/tmp/$server_name.sock;
            fastcgi_index  index.php;
#            fastcgi_param  SCRIPT_FILENAME    $document_root$fastcgi_script_name;
            fastcgi_param  SCRIPT_FILENAME    $request_filename;

            # NinjaFirewall PHPRC 
            fastcgi_param PHPRC /var/www/example.com/public/.user.ini;

            include fastcgi_param.conf;
#            include fastcgi_cache.conf;
        }
    }

    include /usr/local/nginx/conf/block.conf;
    include /usr/local/nginx/conf/drop.conf;
    include /usr/local/nginx/conf/staticfiles.conf;
    include /usr/local/nginx/conf/errorpage.conf;
}

Do you have some clue of what's wrong?
Cheers

 

Well, it works beautifully!
I had to move rewrite ^/wp-(.*)$ /wp/wp-$1 last; into the / location though.

I've asked the same question on StackExchange two days ago and besides critics, I got no real answer. This forum, and specially @eva2000, are really awesome! Thanks one more time.

Cheers!