I would not be concerned about files being uploaded via FTP being compromised, but rather be concerned about compromised files being uploaded and executed via PHP upload, etc.