reddragdiva: (geek)
[personal profile] reddragdiva

You get up tomorrow and log into GMail. You can't get in. Your account is locked. Your mail, calendar, documents — all gone. What do you do now?

Remember that Google has no customer service, even for paying customers. If your account is locked for any reason, spurious or not, you're utterly fucked.

I keep a regular backup of my GMail. The way to do this that actually works (unlike Thunderbird, IME) is with OfflineIMAP (cheers to [personal profile] ideological_cuddle for the tipoff). It's command-line and geeky, but by crikey it works.

Using it on Ubuntu or Debian is absurdly simple:

  • sudo apt-get install offlineimap
  • Set up a ~/.offlineimaprc file cut'n'pasted from this one, with your own username and password.
  • offlineimap

This will create a folder with all your mail in it, in maildir format (plain text, one message per file). You will have duplicate messages in different folders. I'm just doing this to get an archive, so zipped the result.

GMail's IMAP interface is subtly broken, to the point where it can crash offlineimap. Just start it running again, repeat as often as necessary. (If you like, get a more current version.)

GMail is still the best email interface I've ever used, and I wish Thunderbird would just get the hint and clone it to the last detail. But this way I also have all my stuff myself, just because I can.

I haven't tried this on a Mac or Windows. Could someone do this and write up instructions?

For other Google services, you can get your data from Google Takeout. While your account's not locked.

(no subject)

Date: 2011-07-24 12:01 am (UTC)
ext_175346: (Default)
From: [identity profile] thefon.net
Just to let you know that I read all your blog stuff. I'll get on and offlineimap now.

I read your earlier article but still didn't quite follow exactly how Thunderbird doesn't cut it?

Why the duplicate folders? Is this because offlineimap doesn't preserve folders/tags?

Might have to catch up with you one day soon, to talk over the technical details of this and a few other linuxy things. :)

(no subject)

Date: 2011-07-24 12:27 am (UTC)
ext_175346: (Default)
From: [identity profile] thefon.net
Running it now. Seems to using maildir format rather than mbox, but still fine.

Would be nice if it didn't have to duplicate messages to preserve tags, maybe by using linking on linux. Downloading and storing several copies of each message seems wasteful.

(no subject)

Date: 2011-07-24 12:38 am (UTC)
ext_175346: (Default)
From: [identity profile] thefon.net
Looking at the docs now, there are some options to speed up downloads, and also to filter which folders get downloaded.

(no subject)

Date: 2011-07-24 09:09 am (UTC)
From: [personal profile] abortrephrase
Yeah, I use the following:


folderfilter = lambda foldername: not re.search('(^\[Google Mail\])', foldername)


to avoid the duplication.

GMail presents labels as IMAP folders, so if you have a message with multiple labels it will appear in multiple folders. But that's probably what you want anyway.

(no subject)

Date: 2011-07-24 09:10 am (UTC)
ideological_cuddle: (Default)
From: [personal profile] ideological_cuddle
Such hilarity. I usually use Safari for the abortrephrase journal, but am using it as my default at the moment...

(no subject)

Date: 2011-07-24 12:47 am (UTC)
greylock: (Default)
From: [personal profile] greylock
I should probably do this too.

I use this on my Windows machine:
http://www.gmail-backup.com/

I don't have that drive plugged in at the moment, but recall it was relatively easy to install and use. Install (or just run the exe) point it where you want to put the backup folder, set the dates and run.

Et viola.

No idea if there's a way of importing the resulting files into any other e-mail program, but the data is saved. I should probably put it back together and run it again.

(no subject)

Date: 2011-07-24 09:12 am (UTC)
ideological_cuddle: (Default)
From: [personal profile] ideological_cuddle
Trying to use offlineimap on OS X I found it was far too prone to crash or eat the CPU, so I stopped doing that.

The latest iteration of the bundled Mail application is remarkably GMail-like in its interface. If ti could get it to play nice with GMail I'd use it instead of the web interface, and at some point once iCloud hits I'll see how their mail service goes -- Apple do at least have a customer service department.

(no subject)

Date: 2011-07-24 10:06 am (UTC)
pndc: (Default)
From: [personal profile] pndc
I can read my GMail account via IMAP just fine on Snow Leopard's Mail.app. What problems are you having?

(no subject)

Date: 2011-07-24 10:25 am (UTC)
ideological_cuddle: (Default)
From: [personal profile] ideological_cuddle
Yeah, IMAP is an afterthought for GMail. If you're okay treating it like POP3 then it's probably adequate, but otherwise it's not so hot.

(no subject)

Date: 2011-07-24 10:25 am (UTC)
pndc: (Default)
From: [personal profile] pndc
I don't use GMail as a mail interface, just as a dropbox for people who complain that my regular email address doesn't work. (Usually it turns out that they're mailing cabal.com or something dumb like that.) Mail is periodically dragged and dropped from the GMail inbox into my real mail store if I want to archive it. And that works fine.

I could probably easily write something that could specifically archive GMail's rather funky mailstore, but it's not an itch I need to scratch.

(no subject)

Date: 2011-07-24 10:37 am (UTC)
pndc: (Default)
From: [personal profile] pndc
Yes, but if your post is any guide, it doesn't actually work reliably against a hostile IMAP server that doesn't want you to archive your mail :)

(no subject)

Date: 2011-07-24 11:11 am (UTC)
pndc: (Default)
From: [personal profile] pndc
I'll acknowledge that, but what I'm seeing is a generic caching IMAP proxy rather than a tool for backing up GMail accounts. The GMail-specific support doesn't really seem to add anything but a couple of workarounds of assumptions about how IMAP servers work made elsewhere in the code.

Were I to write an archiving tool, it'd do just that: a GMail-specific IMAP client that mirrors the remote server into a local Maildir+ with hardlinks for messages with multiple labels. It'd not be threaded, provide an IMAP interface, or two-way sync, because those are just unnecessary complications, which makes it a fairly simple Perl script, really.

(no subject)

Date: 2011-07-25 09:44 pm (UTC)
From: (Anonymous)
Why not patch offlineimap to do hardlinks, rather than reinvent the wheel?

Perhaps the tags could also be inserted as some sort of X-Tags mail tag, does anyone know if there's a standard tag commonly used for this?

(no subject)

Date: 2011-07-24 10:24 am (UTC)
ideological_cuddle: (Default)
From: [personal profile] ideological_cuddle
The default configuration (which uses the "[Google Mail]" prefix) works fine if you don't use labels. If you want to use Mail.app as your primary interface then I imagine it works okay.

If you use GMails filters and labels then you won't see any of the label-to-folder mapping because that all happens in the root prefix.

If you tell Mail.app to not use a prefix, you get everything twice. It also seemed to only pull in a few messages from the inbox -- I got five of the couple thousand -- but that may be because I only gave it half an hour before giving up.

(no subject)

Date: 2011-07-26 06:52 pm (UTC)
mouseworks: A crop of an orchid shot taken with a Nikon 105 macro lens (Default)
From: [personal profile] mouseworks
I've been popping off my gmail for years now. What I want to keep is in mail.app.

(no subject)

Date: 2011-07-27 12:26 am (UTC)
ideological_cuddle: (Default)
From: [personal profile] ideological_cuddle
My difficulty is that I want/need access from both my home machine and at work. At work means the web interface, so whatever I do has to play nice with that model.

(no subject)

Date: 2011-07-24 03:14 pm (UTC)
mstevens: (Default)
From: [personal profile] mstevens
As a paying customer in some contexts, I have to correct you slightly.

For paying customers, you get nearly nonexistent, abysmal customer service.

(no subject)

Date: 2011-07-24 07:06 pm (UTC)
catness: (Default)
From: [personal profile] catness
Hey, thanks for the tip! Saw it on Slashdot, installed the stuff (offlineimap + dovecot mail server to test the archive), works like a charm! To be fair, I'm only locked out of G+, my gmail access was restored right away after the verification procedure via cellphone, but I don't trust Google anymore ;)

(no subject)

Date: 2011-07-24 09:20 pm (UTC)
rbarclay: (adminspotting)
From: [personal profile] rbarclay
GMail is still the best email interface I've ever used, and I wish Thunderbird would just get the hint and clone it

I wish Thunderbird would fucking stop making changes to the UI, and concentrate on not exploding all over the RAM constantly. Why a mere fucking MUA needs its very own GB of RAM just to work disgracefully is beyond me anyway.
(Sadly, my boss insists on TB, because "it works" for him on his Windos. Me, I'd rather pay someone to make better MIME- and IMAP-support for exmh, which is still the best interface for managing 1k+ mails/day in a gazillion folders, and by a long shot.)

(no subject)

Date: 2011-07-25 07:56 pm (UTC)
alexmc: (Default)
From: [personal profile] alexmc
I'm not very happy using a tool which crashes a lot. And this does seem to crash alot.

(I'm not 100% convinced this is Google's fault, but I hear what you are saying).

Anyway, I am using it. We shall see how it goes.

GMail's IMAP implementation

Date: 2011-07-27 05:30 am (UTC)
From: [identity profile] matej.ceplovi.cz
Apparently it is not imaplib's fault (http://weblog.timaltman.com/archive/2008/02/24/gmails-buggy-imap-implementation). This guy is coder (QA for Opera to be exact).

(no subject)

Date: 2011-07-26 07:21 pm (UTC)
damerell: (computers)
From: [personal profile] damerell
Boy, am I glad I never started.

(no subject)

Date: 2011-08-01 11:19 am (UTC)
vayshti: (Default)
From: [personal profile] vayshti
The "utterly fucked" link is now busted - and I'd wanted to link it on to someone else. Any idea of why the entry was deleted?

(no subject)

Date: 2011-08-14 10:11 am (UTC)
dougs: (Default)
From: [personal profile] dougs
> I haven't tried this on a Mac or Windows. Could someone do this and write up instructions?

On Windows, install cygwin from http://www.cygwin.com/ and be sure to include "offlineimap" from the huge list of packages.

(I already had openssh, rsync, vim and others installed, so I just re-ran the installer and selected offlineimap additionally)

Then follow the instructions above.

It's working just fine as I type, and is no doubt subject to all the same caveats about GMail's IMAP implementation as have been expressed already.

The version currently on offer with Cygwin seems to be 6.0.2-1

(no subject)

Date: 2011-08-14 04:00 pm (UTC)
dougs: (Default)
From: [personal profile] dougs
... and it turns out I had to do an "stty sane" after it had finished.

(no subject)

Date: 2011-08-26 10:36 pm (UTC)
oda: monochromatic field of blue-violet (Default)
From: [personal profile] oda
Since I am theoretically only using gmail as a backup to my primary mail, which I read with mutt in a small dark cave somewhere that the mammoths roam free, I have set up forwards back to my primary mail, which is filtering back to gmail, in a merry little forwarding loop controlled by procmail, a tool I am too lazy to learn how to use to full advantage, so I go over to my gmail backing store when I want to filter or categorize. I use gmail to look at attachments if I am going to bother to look at them, and to have web access to my email from places where I don't have shell access as I do not consider a smartphone's terminal emulator to be any form of feasible shell access.

I forward from my home domain on its ancient crusty spam-harvested up the yinyang address (it's the same one that went with my NIC handle, before they deigned to offer to sell us the option to hide it from spammers) spam and all, so I used to giggle as Google's spam filters struggled to catch up with spamprobe in efficacy. They've actually gotten pretty good at catching spam by now, after having been fed oh-so-many reams of spam, but they're still worse about false positives. (I think because many people operate like my husband, and simply mark things as spam if they're too lazy to unsubscribe from a perfectly valid mailing list. I have had words with him over this.)

Hey, worse about false positives. Where Else Does Google Do That, hm? :)

Anyhow, if you have another account somewhere, forwarding is a perfectly feasible way to mirror all new mail, if not the old.

March 2022

S M T W T F S
  12 345
6789101112
13141516171819
20212223242526
2728293031  

Style Credit

Expand Cut Tags

No cut tags