To accomplish all this we use a lot of Google's services, specifically their Business Apps package, and also our own virtual cloud server. Both use heavily redundant systems, back-ups, disk arrays and mirrors to ensure they have 100% (or as close to) uptime. See here for a few of the features we use.
The pitfall of this anywhere computing is that it is hugely reliant on an internet connection. Which means if you're working in the middle of nowhere, or your connection fails, you may as well be looking at a blank screen as without the web there is absolutely nothing to see or do. This gives two choices - keep duplicate local copies so you can keep on working (see Google Gears for their version of this) though the reality is this often causes more problems than it solves - or install redundant connections.
Given the wealth of benefits, we chose the latter and have 2 completely independent connections to our building. It's worth emphasising that they need to be completely independent. For example, two broadband connections on separate phone lines, from separate providers may seem independent, but if the telephone exchange goes down it will take out both lines unless you are lucky enough to be able to get lines from different exchanges.
Much better is to opt for a mix of technologies such as broadband, 3G/HSDPA, Cable, Satellite; giving a higher level of security. Combined with a mutli-WAN router you'll be able to switch seamlessly between several connections at once.
Unfortunately, this doesn't solve the issue of working in the middle of nowhere - unless you are willing to put up with the size, weight and cost of a mobile Satellite dish?