Friday, June 24, 2011

Remote backup of SimpleDB domains

So Amazon Web Services is pretty good but they're still lacking some of the small stuff. I'd like to see automated usage reports, warnings about suspicious traffic surges, and better backup options. I really don't understand why they don't assign a few dudes to build these things out because I know people have been begging for them for years.

Anyway, I coded a really simple Python script that will back up my domains. In no way is this a scalable solution - if you have a couple million items in your database, this is likely not a cost effective option. I've got less than a thousand entries today and I'm adding only a few hundred per week. It's hosted on WebFaction, which is a great hosting company - for only a few bucks per month you get access to your own very capable virtual server (no more mooching off friends or using inferior free services). It also uses boto, which is a very mature Python library for all things AWS. I've added it to my crontab to run a few days per week and it sends me an email when it's done. Of course, all usernames, passwords, email addresses removed. So here it is:

#! /usr/bin/env python2.7

from boto.sdb.connection import SDBConnection
from datetime import date
from smtplib import SMTP
from email.mime.text import MIMEText
import os
import settings

conn = SDBConnection('access_key','secret_key')

# backup account domain
account_filename = 'backup/account-' + + '.xml'
f = open(account_filename, 'w')
domain = conn.get_domain("account")
account_size = os.stat(account_filename)

# backup timelog domain
timelog_filename = 'backup/timelog-' + + '.xml'
f = open(timelog_filename, 'w')
domain = conn.get_domain("timelog")
timelog_size = os.stat(timelog_filename)

msg = MIMEText("wrote " + account_filename + " (" + str(account_size.st_size) + " bytes)\nwrote " + timelog_filename + " (" + str(timelog_size.st_size) + " bytpes)")
msg['Subject'] = 'Payroll Nanny backup report for ' +
msg['From'] = 'my_app_address'
msg['To'] = 'my_email_address'

s = SMTP()
s.login('my_webfaction_username', 'my_webfaction_password')
s.sendmail('my_app_address', 'my_email_address', msg.as_string())

No comments:

Post a Comment