"Multiple dex files" error with Eclipse + ADT

I haven't done much Android development in the last year but this week wanted to update my app a bit.  So I installed Eclipse Juno (4.2), grabbed the latest Android Developer Tools (r21), and the latest AWS SDK for Android (1.4.4) since my app persists data on AWS.

These are all pretty big jumps from what I used to have - Eclipse Indigo, probably a single digit ADT, and AWS SDK 0.2.X IIRC.  I've only been using my System76 laptop for grad school work of late, not "real" coding obviously.  Almost everything worked fine.  Thanks go out to these teams for honoring backward compatibility!  The one problem I had was "Unable to execute dex: Multiple dex files define X" (X being something in the AWS package).

So off to StackOverflow.  I tried all the suggestions:

• Open/close Eclipse a few times, doing project cleans
• Reinstalling ADT
• Deleting my /bin and /gen directories, then cleaning
• Made sure my build path was legit - several people traced it back to having /bin in the build path

The AWS JARs I imported were core, debug, and sdb (my app uses SimpleDB - don't ask).  Since my testing only consisted of moving the .apk to dropbox and making sure it worked on my Galaxy S3, I didn't need the debug JAR.  Once I removed it, everything worked okay.

Kinda perturbed this is still a problem - many SO posts and blogs I've seen are by people who appear to know what they're talking about so I don't think it's always a silly oversight by junior developers.  I'm thinking it's a dependency problem - maybe AWS SDK 1.4.4 wasn't developed with ADT r21?  If that was the case, Eclipse and SDK providers make it really hard to grab older versions like you would with a POM file.  If I needed to debug in this instance, I'd be in real trouble.

Android Market observations

It's been a few weeks since my last post - I've been super busy launching my first app on Android Market. The prototype was built a couple months ago and it's mature enough to be released into the wild. Homepage here and Android Market entry here.

Anyway, the work necessary to get my app on Android Market was suprisingly minimal. Here are some general observations:

• Google does a great job tying into Eclipse: You can code with any IDE you want but for my money, I have to use Eclipse because it works so seemlessly with Google stuff. Even signing the app is simple.



• Be very distrustful about network connections: I've been using my app for a couple months and never got presented with a "Force Close" (a crash). However, some of the errors being logged are the result of devices dropping network connection and my code was not catching these conditions sufficiently. I'm pushing an update this week which catches these exceptions, presents an alert dialog that states the network isn't working, and takes the user to a safe activity.



• Be careful about security: There as a good article this week warning users of Amazon Web Services (which I am one) to get better with protecting credentials. I spent a good amount of time developing authentication middleware and don't use the VMs this article focuses on but it scared me enough to take a closer look at how I use IAM. I'm rolling credentials more, beefing up my personal password security, and paying more attention to my server logs.



• Testing is hard: I got caught not checking for some basic weird user behavior, like putting spaces in usernames then putting that string in a GET request. Now that I have a bunch of users, I'm fixing these shortcomings.



• Android Market exhibits some weird behavior for publishers: When trying to push my update, I was presented with "An unexpected error occurred. Please try again later." after clicking the "Upload" button. It seems like this is a catch-all error message that the Market displays whenever there's a problem. Back in April 2011, some servers were down and this was getting displayed all the time. I finally resolved it by logging out of Android Market and back in.

A solution in search of a problem

I don't believe in algorithmic trading methods though I am aware that most stocks bought and sold every day are done so because a computer decided to pull the trigger.

Be that as it may, the real reason for my foray into this world was to write a program with server-side Python which utilizes a CouchDB storage solution and an Android app which visualizes the data on a canvas. Oh yeah, and lots and lots of JSON.

I don't really have a name for these programs - I call the collection of Python scripts ZapDome. The Android app is called WebFrenzy. I've open sourced the Python stuff but like I told a startup guy this week, please don't judge me by the excellence of this code - I'm a Python noob. I'll post the WebFrenzy code after I clean it up a bit. It's a real mess right now. Perhaps most importantly, it's yielded a pretty clever (in my opinion) lightweight Java CouchDB library for Android devices specifically built for interacting with Cloudant servers called BarcaJolt. I'll blog about that later.

Anyway, my algorithm is to look for significant volume trends throughout the day and compare those intraday prices with the closing price. So I collect data at the top of the hour 10AM - 3PM EST then sweep up the closing price at 5PM EST. If the volume is >= 125% or <= 75% what SHOULD be according to it's moving volume average at that time, I flag it.

For example, if a stock is averaging 1,000,000 shares of volume a day and at noon EST it's already racked up 900,000 shares traded, that obviously meets my definition of significant volume. Let's say at noon EST, this stock's price is $2.00 and at the end of the day, it closes at $3.00. What if this happens often? That is, hours when the volume is "high", it almost always posts a profit according to it's closing price?

Honestly, I don't really care what the algorithm is. I'm never going to use this data - I'm a value investor!. But it's a fun hack and gets me experience in things I want to learn more about (Python, CouchDB, JSON, Android apps).

So here's a couple screen caps of my Android app showing the behavior of 2 stocks I've been tracking over the last week (I've only pulled a week's worth of data). Obviously you can't pull any trends out of such a small data sample. The data is on an X/Y graph (Vollume/Price). So the top right quadrant shows instances where the stock's volume was high and the stock price rose before the close. More to follow.

A CouchDB library for Android

So I really liked this article that was featured on Hacker News a week ago and it got me to thinking that maybe I'm using too many 3rd party libraries to help with areas I don't have much experience in (couchdbkit, for example, is awesome). So instead of using Ektorp to help me with Java/CouchDB/JSON stuff, I think I'm going to write my own library to help me with the Android portion of an algorithmic trading system I'm playing around with.

There already is a well-known CouchDB API build specifically for Android called droidcouch but I think this space can use more than 1 option and I have some ideas.

The most striking change between 2003, when I typed my last professional line of code, and now is the influx of 3rd party libraries. The infrastructure necessary to get a lot of adopters is already built out with github/Google Code/sourceforge.

Ext4 delayed allocation data loss

My Ubuntu (Maverick Meerkat) netbook crashed this week and completely wiped an open .odt document. The behavior was very similiar to what happened here. I'm not a file system guru so I wasn't aware until now of the long standing issue with Ext4 file system delayed allocation. From LinuxInsight:

In Ext4 file system, Delayed Allocation causes some extra risks of the data loss if your system gets crashed before all the data is written to hard drive.

This was also covered on slashdot over 2 years ago.

It appears that the transition to Ext4 requires applications properly use fsync() to prevent data loss. Theodore T'so, who played a major role in developing Ext4, includes a good summary here. In fact, Android devices which run on Ext4 are rolling out and preventing data loss has been a major concern.

I can't imagine the Open Office that ships with Ubuntu 10.10 doesn't adhere to the Ext4 spec and use fsync() properly but I've never had this problem in other apps. Very aggravating and perhaps another reason to use Google Docs or give Libre Office a try!

PayNanny

PayNanny is a household employer payroll app I'm working on for Android devices. So far, the only functionality I've built consists of a way for the user to "log in" (identification only - no authentication yet), create/delete/edit employee names, and link or delink their account with another.

I've open sourced it on github.

NoSQL databases - now with SQL!

So I'm writing an app and wanted to take the opportunity to see what the NoSQL buzz was all about. By no means would I need to scale to the level most programmers expect when they employ NoSQL (like Netflix) but I think I've worked with RDBMS enough (though it's been awhile). Ted Dziuba has a pretty funny critique of the NoSQL craze here.

Amazon Web Services (AWS) SimpleDB was a perfect fit - not only is it free for small fish like me but the new AWS Android SDK includes SimpleDB support. I'm not interested in writing a bunch of HTTP libraries - the AWS Android API does everything for me.

The big knock against NoSQL is its data consistency. You're not always guaranteed to get the data you're expecting. SimpleDB counters with a Consistent Read option which I'm employing. You give up a little speed but for a small app like mine it's a no-brainer. But there are no transactions to track so there's still some risk.

Another surprise was SimpleDB's support of SQl through SelectRequest. You can't do stuff like JOINs (these kinds of "advanced" operations are handled in application code) but it's convenient when you need to pull a targeted set of data.

Special shout out to the folk(s) who made sdbtool, a Firefox plug in that let's you interact with your SimpleDB account.

Here are a couple snippets from my class which handles SimpleDB interaction. Connecting is as easy as:

    BasicAWSCredentials credentials;
    Properties properties = new Properties();  
    try {            
        properties.load(getClass().getResourceAsStream(AWS_PROPERTIES));
                  
        String accessKeyId = properties.getProperty("accessKey");
        String secretKey = properties.getProperty("secretKey");

        // some boring error checking

        credentials = new BasicAWSCredentials( properties.getProperty( "accessKey" ), properties.getProperty( "secretKey" ) );
        // note mDB is an AmazonSimpleDBClient
        mDB = new AmazonSimpleDBClient(credentials); 
    }

Executing queries and putting results in a List is fairly easy:

    SelectRequest selectRequest = new SelectRequest("select * from accounts where m_username = '" + username + "'").withConsistentRead(true);
    SelectResult selectResult = mDB.select(selectRequest);  
    List resultList = selectResult.getItems();

I'm able to really take advantage of the API simplicity when adding stuff to the database. NoSQL architectures seem to be pretty great for these types of actions (as long as the data gets there!).

    List attributes = new ArrayList(1);
    attributes.add(new ReplaceableAttribute().withName("m_username").withValue( username));
    PutAttributesRequest request = new PutAttributesRequest("accounts", username, attributes);
    mDB.putAttributes(request);

You know, if data consistency is critical, you could keep querying the database until your data is positively there...

Super simple identification code

So I'm working on an Android app and I really don't care about authentication (yet) but need a way to identify a user. I can assume the user will have at least 1 google (or gmail) account and I'm also going to leverage off the Google App Engine for many services. I'll probably add in the AuthToken functionality eventually - hopefully the Authentication features of Android and Google App Engine will continue to improve in the meantime.

So here's a class that extends ListActivity:

    Account[] mAccts;
 
    @Override
    public void onCreate(Bundle savedInstanceState) {
     super.onCreate(savedInstanceState);
     
     setContentView(R.layout.main);

     // get google/gmail accounts
     AccountManager acctMan = AccountManager.get(this); 
     mAccts = acctMan.getAccountsByType("com.google");

     // put the account names into a String[]
     String[] acctnames = new String[mAccts.length];
     for (int i=0; i< mAccts.length; i++) {
      acctnames[i] = mAccts[i].name;
     }
          
     this.setListAdapter(new ArrayAdapter(this, R.layout.accts_row, acctnames));
    }

That will show the user a list of their Google accounts. Here's the onListItemClick:

     super.onListItemClick(l, v, position, id);

     // find the account clicked and send the intent on its way
     Account acct = mAccts[position];
     Intent i = new Intent(this, PayMain.class);
     i.putExtra(PayMain.KEY_ACCT, acct);
     startActivity(i);

In your target activity class (PayMain in this case), the account will be bundled into the Intent.

The XML files are basic for ListActivity programs. Make sure you ask for GET_ACCOUNTS permission in AndroidManifest.xml (plus USE_CREDENTIALS and INTERNET if you're going to get an AuthToken).

I ran this on my Droid X with 2 gmail accounts and it works fine. Works with no Google accounts also (says the user needs to get a Google account and try again). Having problems adding an account to the emulator. I believe I'm supposed to do it through Dev Tools but can't seem to get it to work. I'm able to add a gmail account to the email app but it's not added to the system as a real account. Plus I can't locate any good documentation of Dev Tools. Strange...

And finally, Notepad Example #3

The final example changes the manner in which data is passed between the Add/Edit activity and the main activity. Instead of packing them in the extras Bundle, the data is pulled from the database. So, note my first note from the Exercise 2 entry. I failed to consider what would happen when pausing the activity for whatever reason. Here are some random thoughts:

• Not loving the ContextMenu: Doesn't seem to be an intuitive way to access operations but I'll keep an open mind. I would prefer to offer the "Delete Note" option as a button in the NodeEdit activity.
• Moving the SQLiteDatabase object to the NoteEdit activity makes perfect sense. I just wonder if all those reads/writes introduces risk of lag, particularly when a remote/cloud data storage solution is used.

Thoughts about Google's Notepad Example 2

So I'm not going to rewrite the Example 2 app like I did for Example 1 but here are some thoughts after going through the exercise.

• Firing intents via startActivityForResult(): It's a little tedious packing items into the extra Bundle, especially if you have a lot of data but I'm sure it's much faster than hitting the database again.
• Local variables versus accessing the Dalvik VM: It's included as a note in the example text, but it bears repeating, especially for mobile developers: "Accessing a local variable is much more efficient than accessing a field in the Dalvik VM, so by doing this we make only one access to the field, and five accesses to the local variable, making the routine much more efficient."
• XML layouts are hard to debug: While the schema is pretty straightforward, it's not like there's a debugger for your XML files so any dumb mistake can make for very tedious troubleshooting.
• Once you get a Virtual Device going in Eclipse, don't turn it off!: Upon starting up Eclipse, the AVD fires up very quickly when executing a program. But if I shut it down, do some coding or whatnot, and try to run it again, it often sits on the "Waiting for HOME ('android.process.acore') to be launched..." step. Out of habit, I sometimes kill the AVD mistakenly.

Getting into SQLite

I grew bored drawing things on a Canvas so I moved on to the very helpful Notepad example. I like it because it's centered around the SQLite database package. Understand the importance of a good UI but the database interaction is much more my thing.

So I went through Exercise 1 then put it away and tried to code it myself. I'll spare the boring screenshots because my program doest exactly what Example 1 does - add dummy rows to a database and display them.

Couple things:

• Got some compile errors related to Cursor stuff (specifically setListAdapter) in my main class. I was extending Activity not ListActivity. Whoops - there is a difference.
• I like the idea of startManagingCursor and wonder why it's not default. Maybe only noobs like me use it but are people really managing their Cursor's lifecycles so thoroughly?
• When I tried to account for SQLExceptions, Eclipse never recognized its reference even though I imported android.database.SQLException. So I just removed it. I'll come back to that one. I'm sure I'm overlooking something obvious.
• Loving Eclipse's Ctrl-Shift-O shortcut to auto-import.
• Took me awhile to figure out the Context data type so I could access the SQLiteOpenHelper constructor. One difference between my code and Google's example was they kept the Context around as a private variable while I just discarded it. I wonder if keeping Contexts around is useful.

Oh, activities need to be defined within the application?

Continue to play around with the Android SDK. I'm working on a program that makes whatever kind of shape the user requests via a menu. Yes, very advanced which is why I named it Simple App. Anyway, the app was crashing when the shape was selected so I got some hands on experience with the Eclipse debugger.

I was getting ActivityNotFoundException and it was bombing on the startActivity() call. But how can that be? The class was properly included and I declared it in AndroidManifest.xml. Well, for some reason, I added this activity outside the application tag. Rookie mistake but it wasn't self evident in the debugger.

Amazon AWS and Database.com

Much of my application development has been "data structure centric". Future projects might be in the same vein so while tinkering with Android coding, I'm looking into 2 major database products with hooks into Android (among others devices, languages, and platforms, of course).

Amazon's AWS has been a major web services provider for awhile. Their data storage product is Amazon S3 but the interesting part is the AWS SDK for Android which provides S3 storage, database creation and messaging. In fact, Amazon might have made it TOO easy to deploy mobile enterprise ("cloud") apps has this PCWorld article suggests! Once I've secured a moderate amount of Android development knowledge, I'm definitely going to open an AWS account and play around with this API (the AWS Free Usage Tier is free for a year).

Salesforce has moved into the database-as-a-service market with database.com, which as this InfoWorld article states aims to be the "back end for the entire Internet". While other companies are playing catch up, Salesforce appears heavily leveraged into the cloud computing and ASP worlds so they might be well positioned to be a database of choice for mobile developers. Unfortunately, database.com is not really live yet (their FAQ page states the full developer preview is coming out in 2011) so I guess this is the end for now.

CNNMoney - Apple vs. Android: The view from Google

Interesting take from senior Google engineers about what the future holds for Android and iOS. The press is trying to play up an angle that these views aren't exact carbon copies of each other but they seem pretty close to me. I agree with Don Dodge that both Google and Apple can achieve their goals (Apple to continue owning the high-end market and Google to grow their software distribution by being the OS of choice for smartphone hardware makers). Perhaps a better question is "Will a third player be able to make significant inroads to the mobile OS market?". Tim Bray (in probably the best article I've read in several weeks) thinks Windows Phone 7 might make inroads and it's hard to envision Microsoft ceding such a huge market without a fight.

PCWorld - 7 Predictions For Open Source in 2011

Pretty good article from PCWorld about what trends to look for in 2011. Bullish on Android, tablets, and Ubuntu (as I am) but not sure I agree a true Linux distro will every really make it into the mainstream as the author seems to suggest. Casual users won't put up with the anomalies you encounter with Linux (even the slick, user-centric ones). Check out this post from Collin Pruit for more about the need for true Ubuntu stability/interoperability. Maybe the closest Linux will ever come "to the masses" is the Linux kernel which Android uses for its core system services...

Applying Android Themes

Pretty clever to pre-define a bunch of attractive themes so developers don't have to think too hard (or do too much of what they're often not good at - UI design). Check out the R.style page and look at everything that begins with "Theme_". They basically act like CSS. I'm playing around with the Dialog theme for an "About Us" popup. You utilize it while defining the activity in AndroidManifest.xml:

android:theme="@android:style/Theme.Dialog"

My first program in a long time – don’t get excited

Until very recently, I the last line of code I typed was in September 2003 when I traded my programming career for a stint in the US Navy. I was lucky enough to sit down with someone very knowledgeable last month to give me some advice on how I could transition back into the SW world. After pointing out my LAMP Stack experience was pretty much useless in 2010, he advised to me get deep into cloud computing and/or mobile technology.

Application development has always been my preference and I am (was) pretty decent at Java so Android development seems like a good match. After installing the Android SDK on my Ubuntu Maverick Meerkat netbook (not an ideal development platform but isn’t Ubuntu great?!?) and banging out the requisite Hello, World rite of passage, I “developed” my first app in a long time. Here it is in all it’s glory – a button that does nothing. I am truly back:

3 beginner lessons I’ve learned about Android development in Eclipse

My specs:

• OS: Ubuntu Maverick Meerkat 10.10 running in Desktop mode because Remix no worky for me.
• IDE: Eclipse Java EE IDE for Web Developers (Galileo)
• Hardware: Gateway LT31 Netbook w/ AMD Athlon 64
• JDK 1.6.0_22

1. If you’re using low resolution, keep in mind GNOME has a tendency to hide important things (like scrollbars) if it runs out of room. If you’re missing something obvious, increase your resolution. I ran into this when installing the Android plugin. Duh.
2. If Eclipse is throwing lots of timeout errors when you try to run something in the Android emulator, increase your timeout variable under Window -> Preferences -> Android -> DDMS -> ADB Connection Timeout. Mine’s at 50000 ms.
3. For some reason, I get a lot of “Error in an XML file” errors for no apparent reason. Here’s a good explanation. I found cleaning my project (Project -> Clean) seems to work. You have to be really careful about not keeping old XML files in your projects.