vasárnap, november 07, 2010

GMail, Google Apps, Laptop and Android syncing HOWTO

I am a GMail user for a long time. Hence, all of my "mail archives" reside mostly there, in my GMail account. Same for contacts. Also, I was rather "brave" with my GMail address, and was and am using it in "promiscuous mode", giving it away almost to everyone and everything, signing up to mailing lists, creating accounts on some not-so-trusted sites, etc. The GMail spam control did it's job every time and served me very well.

Later, I discovered Google Apps (GA), and started using it extensively. So, I ended up with multiple domains controlled (and having them completely separated), with uses as: "family hub", where all the broader family members have unified "family" addresses and services, other "me only" domains for my OSS stuff etc. Then I founded a micro-company, that also has it's own domain powered by GA. Finally, I do work full time for a company, that also uses GA for it's mail and calendar infrastructure. Naturally, I still use my YouTube account (that requires GMail account, does not work [yet?] with GA ones), and also Picasa for my public photo albums.

As for my "workhorse", I have a Mac, that serves me great. It's integrated applications yada-yada-yada... are just a joy to use. Before Mac, I was a Linux user, and I loved it. Today, I use to say that "Mac (desktop) is a UNIX that Linux (desktop) should be alike (wrt integration)". I was a big KDE fan, but it always tend to "fall apart" after you tweaked it a bit. Also, in a moment you started using alternate apps, you ended up either "disconnected" or syncing same stuff over and over again. We all know, that the "default" KDE apps are great, but many times you find something that makes you switch to another app (they are sometimes just incomplete or somewhat unfinished. Yes, I am an OSS developer too, so I do understand, but Desktop I use everyday should be without any compromises. That's why I did paid few bucks, few Hungarian Forints more precisely, for one Mac that simply makes me more productive ;) ).

And finally, I got recently a superb Android phone, one Samsung Galaxy S (sometimes named as I9000 too). I tried to "focus" all the relevant information sources scattered above into one device. Contacts (private and company ones), mails, videos, etc. And I ended up resetting the phone multiple times (to change the "primary" Google Account being registered on the phone, more about this later). How to do it?

The given

As I said, I had following Google Accounts (GMail and GA accounts, an important difference):

  • t.cservenak@gmail.com (GMail, promiscuous, personal use mainly, but most of the dev-lists are subscribed from here too)
  • tamas@familydomain (GA, this is the "primary" personal address)
  • tamas@ossdomain (DNS MX redirect to .net, personal, will be hosted by GitHub Pages feature)
  • tamas@comdomain (GA, personal)
  • tamas@mycompanydomain (GA, work, mail and calendar is a must)
  • tamas@myemployerdomain (GA, work, mail and calendar is a must)

And finally, I wanted to use my Twitter and Facebook (not sure why this latter, only one use of it is to get alarmed for a friends birthday, but I am not an active FB user). So, a lot of accounts to use in focused manner, but also I wanted to get a grip on other, non-GA apps like Youtube and Picasa for full fun coverage. Those latter services are usable with GMail accounts only.

The goal

The goal was following:

  • Using synced contacts, have them synced and up2date on all devices: my Mac -- my "primary" GA -- my Phone
  • Using mail with my GMail account (and slowly move over to a GA account)
  • Using GTalk with my familydomain account
  • Using all the relevant calendars (there are at least three: familydomain, mycompanydomain and myemployerdomain)
  • Using Picasa with my GMail account
  • Using YouTube with my GMail account

First trial and failure

After unpacking my Android device, I did what most of us would do: started adding Google Accounts to Sync, and naturally started with GMail one. I got at once access to my mails, calendar, most of the contacts, but also to my photos uploaded to Picasa (in Gallery on my phone), and I had my Youtube working just fine, all the playlists and uploads were there and usable in Gallery.app.

Later, I figured my first problem: by browsing back to Sync Accounts, I found severe "anomalies": what kind of information (like contacts, calendar, etc) is synchronized depends on two things:

  • What kind of account it is (GMail or GA)
  • When is account added, as first (primary) or as secondary

This puzzled me initially. I got the "best" results by adding my GMail account first to my new phone, it gave me the most, it synced following services:

  • Contacts
  • Mail
  • Calendar
  • Picasa
  • YouTube app
  • GTalk

With my GA familydomain added as primary account, the items melted down to these:

  • Contacts
  • Mail
  • Calendar
  • (x) No pics from picasa in Gallery.app.
  • (x) YouTube app asked for "normal" GMail account (I did not add it yet)
  • GTalk used familydomain as I wanted.

But, it started to bother me that whenever I added GA accounts of mine as secondary Sync account, all I was getting synced was Contacts and Mail only. No Calendar. And Calendar was very important (as I said, my company uses Calendar and I really like to be able to check all the meetings we will have for following days, except when Toby forgets to add all the attendees ;) ).

If I added one of my GA accounts as primary sync account, I ended up with less synced services, but that's okay since Picasa for example does not work with GA accounts. Luckily, if you add GMail account too, as secondary, YouTube and Gallery (for Picasa image uploads and sync) applications are smart enough to figure out and start using it.

My biggest problem (calendars does have a "workaround", see below) was GTalk: not sure is this due to app I had, or some manufacturer limitation of my phone, or this just how things should work on Android, but I was simply unable to switch/change the user used with GTalk, it was always the primary account only. Same for Latitude, Market and who knows what else. And this did not fit well with me. Simply, this forced me to register tamas@familydomain as primary account. This was actually my goal, since my contacts were about to be pushed there too. I wanted to "phase out" slowly my GMail account anyway (not to close it, just make it "secondary").

The Calendar trick

For calendar access, at start, I set the bar to "ReadOnly" access for other calendars (coming from non-primary accounts). For edits, I would use iCal.app as before the phone, and I was prepared to be happy with read only access from phone.

So, I did the cheapest trick I could do, with the luck that almost all GA domains were controlled by me, and my employer had also the "relaxed" setup: domain admin allowed to share and give control over events to users outside of the domain. Simply, I shared all the calendars I needed on my phone with my primary Sync account (to the familydomain account). Again, this works if your GA domain is set up like this, or, as in my case, you are the domain admin, and you are in position to change this.

And at greatest surprise, this did not just provide read only, but a proper read write access! I can create any kind of event on my phone, and am able to select calendar (familydomain, mycompanydomain, myemployerdomain, or just the phone local) where to save the event. For test, I used my employerdomain. In a moments the event appeared in iCal.app. Then I deleted it using iCal.app, and phone got the update too. It works just great!

The misery with Contacts sync

On Mac, since Leopard OS you get Address book to Google sync that works with both, GMail and GA accounts. But there is a subtle difference between plain Leopard (10.5) and Snow Leopard (10.6) -- and I was using both, and I assume that was the root of some problems --: the latter synchronizes contacts in "My Contacts" group only! More about it here: http://www.google.com/support/contactsync/bin/answer.py?hl=en&answer=92997&ctx=share.

After a few trials, I had not even the faintest idea how it works, but it was doing something. So, I used the GMail Contacts editor to clean-sweep my contacts (my Address Book was syncing to GMail), and then just did an export from GMail, and slurped it into GA familydomain, changed Address Book config to familydomain too.

All in all, it was not smooth, especially if you consider the First name, last name misery. In hungarian, the order is lastName-firstName, but I tend to use UIs in english (never liked translated UI elements, it sounds just funny, or worse, completely misleading in Hungarian, depending in translation quality).

Finally it worked, but I am still puzzled how GMail, GA and Address Book sync works. Not to mention, how the GMail/GA "Contacts" UI is just confusing (and different!). I spent hours banging my head, why "All Contacts" link shows less contacts than "My Contacts", to finally figure out that "All Contacts" is just a badly named group, created by some tool (AddressBook.app maybe?), but on UI nothing says that, no distinction between groups and other "views". There is some black magic (or just some implicit knowledge) that was not so transparent at first.

But finally, I was able to "streamline" my contacts just fine.

Caveats

This setup is okay and works as charm, only if

  • you own (or control) the domains to be pulled to your phone
  • or at least your domain(s) are set up to allow calendar shares outside the domain

Naturally, setting up yourself like this is worth only with your own domains. Setting an employer (or even your company) GA as primary account is not the best option, since you may change employers, shut down the business, but your "familydomain" domain would be eternal. But you can just stick with GMail account as primary one and be fine.