Remote backup of SimpleDB domains

So Amazon Web Services is pretty good but they're still lacking some of the small stuff. I'd like to see automated usage reports, warnings about suspicious traffic surges, and better backup options. I really don't understand why they don't assign a few dudes to build these things out because I know people have been begging for them for years.

Anyway, I coded a really simple Python script that will back up my domains. In no way is this a scalable solution - if you have a couple million items in your database, this is likely not a cost effective option. I've got less than a thousand entries today and I'm adding only a few hundred per week. It's hosted on WebFaction, which is a great hosting company - for only a few bucks per month you get access to your own very capable virtual server (no more mooching off friends or using inferior free services). It also uses boto, which is a very mature Python library for all things AWS. I've added it to my crontab to run a few days per week and it sends me an email when it's done. Of course, all usernames, passwords, email addresses removed. So here it is:

#! /usr/bin/env python2.7

from boto.sdb.connection import SDBConnection
from datetime import date
from smtplib import SMTP
from email.mime.text import MIMEText
import os
import settings

conn = SDBConnection('access_key','secret_key')

# backup account domain
account_filename = 'backup/account-' + date.today().isoformat() + '.xml'
f = open(account_filename, 'w')
domain = conn.get_domain("account")
domain.to_xml(f)
f.close()
account_size = os.stat(account_filename)

# backup timelog domain
timelog_filename = 'backup/timelog-' + date.today().isoformat() + '.xml'
f = open(timelog_filename, 'w')
domain = conn.get_domain("timelog")
domain.to_xml(f)
f.close()
timelog_size = os.stat(timelog_filename)

msg = MIMEText("wrote " + account_filename + " (" + str(account_size.st_size) + " bytes)\nwrote " + timelog_filename + " (" + str(timelog_size.st_size) + " bytpes)")
msg['Subject'] = 'Payroll Nanny backup report for ' + date.today().isoformat()
msg['From'] = 'my_app_address'
msg['To'] = 'my_email_address'

s = SMTP()
s.connect('smtp.webfaction.com')
s.login('my_webfaction_username', 'my_webfaction_password')
s.sendmail('my_app_address', 'my_email_address', msg.as_string())
s.quit()

Android Market observations

It's been a few weeks since my last post - I've been super busy launching my first app on Android Market. The prototype was built a couple months ago and it's mature enough to be released into the wild. Homepage here and Android Market entry here.

Anyway, the work necessary to get my app on Android Market was suprisingly minimal. Here are some general observations:

• Google does a great job tying into Eclipse: You can code with any IDE you want but for my money, I have to use Eclipse because it works so seemlessly with Google stuff. Even signing the app is simple.



• Be very distrustful about network connections: I've been using my app for a couple months and never got presented with a "Force Close" (a crash). However, some of the errors being logged are the result of devices dropping network connection and my code was not catching these conditions sufficiently. I'm pushing an update this week which catches these exceptions, presents an alert dialog that states the network isn't working, and takes the user to a safe activity.



• Be careful about security: There as a good article this week warning users of Amazon Web Services (which I am one) to get better with protecting credentials. I spent a good amount of time developing authentication middleware and don't use the VMs this article focuses on but it scared me enough to take a closer look at how I use IAM. I'm rolling credentials more, beefing up my personal password security, and paying more attention to my server logs.



• Testing is hard: I got caught not checking for some basic weird user behavior, like putting spaces in usernames then putting that string in a GET request. Now that I have a bunch of users, I'm fixing these shortcomings.



• Android Market exhibits some weird behavior for publishers: When trying to push my update, I was presented with "An unexpected error occurred. Please try again later." after clicking the "Upload" button. It seems like this is a catch-all error message that the Market displays whenever there's a problem. Back in April 2011, some servers were down and this was getting displayed all the time. I finally resolved it by logging out of Android Market and back in.