Kris Zyp has created OfflineRest, a new module in Dojo 1.2 that allows for a simple offline pattern to building your application.
Dojo 1.2’s new dojox.rpc.OfflineRest module automates the local storage of data and synchronization by leveraging the Dojo Data and REST abstractions. The OfflineRest module augments the JsonRest service in Dojo such that requests are cached in local storage for offline access, and modification requests (put, post, and delete) modify the cache and are recorded for delivery to the server; immediately if online, otherwise when connectivity is restored. Furthermore, JsonRest is the core engine used by JsonRestStore. Consequently, you can simply use the standard Dojo Data API with the JsonRestStore and effortlessly add offline capability with no modifications to your data interaction code. The underlying Rest service automates the handling of caching, storing data locally, and syncing changes. In addition the new OfflineRest module has no dependency on plugins, but rather progressively utilizes offline features that are available, while still operating properly on legacy browsers without offline support.
He has a demo that involves a CRUD system for hiking trails which interestingly asked me for Gears permission, even though it appears that OfflineRest doesn't abstract on top of both Gears and the emerging storage standard, which is something I would like to see, to extend the reach.
Zoho Writer was quick to release offline support using Gears, which allows you to carry read-only views of your data around when you were offline.
This was just the first step in their offline support, and now they have announced the next step. This new version allows you to edit on the road with a synchronization system that keeps you sane:
Working on the Gears team we also run across applications that we would love to take offline. A lot of these applications aren't Google's so we thought it would be nice to be able to take third party apps offline. This also makes sense since Aaron Boodman (Mr. Greasemonkey) is co-tech lead on Gears itself!
Ben Lisbakken has written up his work taking Gears and Greasemonkey to make this happen. He details the real example of taking Wikipedia offline which has once piece of narly code to do with iframe injection to be able to store data on third party sites (e.g. media site vs. wikipedia main site):
Initialize Gears on page
Check if site has been allowed, if not, trigger allow dialog
Insert iFrame
Initialize Gears on iFrame
Check if site has been allowed, if not, trigger allow dialog
If Gears is initialized on both, insert Cache Page link (unless page is cached)
When user clicks Cache Page:
Capture the HTML and CSS of the Main Page
Store the URLs of all links to HTML, CSS, and media files in the Gears database (so we can remove them from the ResourceStore later, if needed)
Create an iFrame whose src is in the domain of upload.wikimedia.org. Pass all media file URLs to the iFrame in the src URL after the hash, e.g. src="http://upload.wikimedia.org/#thisimgloc.jpg||anotherimgloc.jpg||lastimgloc.jpg"
Initialize Gears in iFrame
Capture all URLs from the iFrame's href hash.
When user clicks [x] to remove an article from cache:
Grab all URLs from the Gears database that correspond to that article
Remove all URLs from the ResourceStore of the Main Page that contain the string "en.wikipedia.org"
Remove all URLs from the Gears database that correspond to that article
Create an iFrame whose src is in the domain of upload.wikimedia.org. Pass all media file URLs to the iFrame in the src URL after the hash, e.g. src="http://upload.wikimedia.org/#thisimgloc.jpg||anotherimgloc.jpg||lastimgloc.jpg||remove||"
Initialize Gears in iFrame
Remove all URLs from the ResourceStore that are listed in the iFrame's href hash.
Here's the script in action as I save pages away. This is just the beginning. Ideally we would have the code automatically save content that you have been too, and do smart spidering to get more on the subject too. We will also work on making GearsMonkey scripts even easier to write.
The tool itself is a nice Ruby on Rails application (includes pink fade effects! yellow is so 2006) that gives you a visual canvas to play with your mind.
Their approach to offline is similar to Google Reader in that the user has to say "hey, take me offline". The interface to that is a nice little slider widget. At the point your maps are sync'd down to the local store.
I would love to see it auto sync, and I noticed a couple of issues when I actually went offline but didn't tell the tool first (would be nice to have the tool grok that) but the mind map tool in general is a nice app to use. It feels like Geni.
One new feature we added to WebRunner is web application styling or theming. You can drop a “webapp.css†file in the bundle and it will be applied to the hosted web application. You can also use platform-specific folders in the bundle to apply specific CSS based on the current platform (Windows, Mac or Linux). We’ll put more details up on the wiki when the feature is released. In the meantime, here is a screenshot of Google Reader hosted in WebRunner, running on Mac, using Jon Hicks’ excellent CSS theme.
Luke Birdeau has remixed Craigslist to produce a desktop-esque Ajax application view on the data that adds features such as being able to save your favorites, add notes to them, and even use the app offline (e.g. take your laptop on the road to go see the stuff for sale of meet that blind date). The app combines aspects of 3 libraries – TIBCO GI 3.5 for the interface, plus Dojo (for offline) and Google Maps.
To get started you first pick a locale, then a category, then do a search. You can also add multiple regions and categories too.
Here is a quick demonstration of the app in action:
eBay's San Dimas project has finally moved to public beta. The application, developed using Adobe's AIR runtime, provides the ability to manage your eBay experience via a well-designed desktop interface:
The product has also graduated to public beta, thanks in no small part to the enthusiasm and support of all of you. Whether or not you installed San Dimas, you can now download eBay Desktop, for free, from http://desktop.ebay.com
eBay Desktop works with eBay US, and in the coming weeks and months we will be extending support to other countries. You can continue to use San Dimas to shop on any eBay site in the meantime. The response to San Dimas worldwide was incredible, and we are committed to bringing this desktop eBay experience to everyone.
The application team made substantial changes for this release to ensure a more personalized feel. Changes included:
Revamping the home screen to dedicate more area to highlighting your eBay activity -- your watch list, bidding list, recent items, recent categories, and favorite searches.
A new browse feature
Favorite search feeds allow you to automatically pull down items that match a favorite search
Ability to run in the background even when the application itself is closed, so that you can get reminders or outbid alerts at any time.
Performance improvements
Numerous bug fixes
eBay Desktop is a free download and all you need to use it is a valid eBay account. You can download eBay Desktop from http://desktop.ebay.com
Brian Dunnington liked what he saw with the Dojo Offline Toolkit, and wanted to abstract it out so you could use functionality with any JavaScript library.
He ended up with a new library called Vortex, a 10k JavaScript file that features:
Ability to automatically detect referenced resources such as images, scripts, and stylesheets (including nested @imported stylesheets)
Automatic detection of network state with corresponding UI/feature changes (also implements .isOnline() and .isOffline() methods)
Auto syncing of events. it essentially records any user-defined actions and automatically plays them back when the network comes back online
A generic storage provider to quickly and easily save javascript objects (JSON-encoded)
Graceful fallback if no offline support is available (google gears not installed or not allowed to run)
Library-independent (does not require dojo, prototype, Ext, YUI, etc) and cross-browser compatible
He has a simple RSS demo that shows the library at work.
I took it for a spin and went online and offline to see how it automatically detects. Also in the video below I take a peek at the code and we see what the definition of "online" means (able to XHR a particular file).
Adobe's AIR runtime gets a lot of press for bringing web apps to the desktop but Mark Finkle's WebRunner is looking to be an alternative force in this space.
WebRunner is a simple XULRunner based browser that hosts web applications without the normal web browser user interface.
WebRunner 0.7 was just released with improved configuration capabilities and file system organization. New webapp bundles have also been added for Facebook and Twitter to compliment the existing profiles for:
He has seen that there is a common case, which is "When you build a true web application, many people do so via the single page model". If you take this assumption, wouldn't it be nice to tag the top page as an application, and have the browser subsystem take care of caching files and using them correctly when offline. His assumption is based on the logic that: "If you have complicated server logic you use a multiple page application, which isn't going to do much in an offline mode anyway, as it has complicated server logic!".
Some people do not subscribe to this view, and think that there is a lot of multiple page applications that can be made to be web enabled, and so his latest thoughts take that into consideration.
The idea is that you can then <html application="manifest-of-urls.txt"> and use a different path. He starts with:
Ok, new proposal:
There's a concept of an application cache. An application cache is a group
of resources, the group being identified by a URI (which typically happens
to resolve to a manifest). Resources in a cache are either top-level or
not; top-level resources are those that are HTML or XML and when parsed
with scripting disabled have with the value of
the attribute pointing to the same URI as identifies the cache.
When you visit a page you first check to see if you have that page in a
cache as a known top-level page.
If you do, skip the next two paragraphs; the 'new cache' flag is set to
false.
.... and the logic keeps going ....
Seeing simpler and simpler offline support is great.
Eric Farrar of Sybase iAnywhere has been developing a prototype that allows direct database-to-database synchronization of a Gears application to Oracle, SQL Server, DB2 and Sybase databases. We asked Eric about his work and he told us:
At Sybase iAnywhere we provide a data sync infrastructure for mobile and remote apps. The Background Sync architecture mentioned on the Gears site is a lot like mobile application architectures we see all the time. As a result, it seems as though the same mobile sync technology can be used to solve the offline sync problem for Google Gears.
This prototype uses a local lightweight database (based on the iAnywhere UltraLite database) with built-in synchronization
capabilities, meaning:
Built in change tracking, so that changes (including deletes and updates) can be sent to the server,
Built in state tracking, so that the local database knows when synchronizations succeed and can take appropriate clean-up operations automatically,
Built in synchronization over TCP/IP or HTTP.
Optional encryption of the data store and of the sync stream
Somewhat stricter data management model than SQLite.
The other piece is the sync server that manages synchronization to Oracle, SQL Server, IBM DB2 and Sybase databases. The sync technology is fully transactional, meaning data does not get partially changed at either end. It also has automatic mechanisms to detect conflicts and hooks that allow users to programmatically handle the conflicts using SQL, .NET, or Java.
As far as scalability, these technologies are currently being used in projects that involve nearly half a million mobile devices.
Eric also recorded some videos show this all in action:
The article moves past an introduction to delve into the design decisions around an offline-capable architecture, and user messaging and presentation of state. We learn why Omar decided to go with the explicit offline mode, and then the five steps to offline conversion:
Ensuring resources are available offline
Decoupling the application from the network
Persisting data on the client
Re-creating application state from persisted data
Developing a synchronization strategy.
There is a lot to learn here.
From their architecture considerations:
RTM was designed as a client-side application from inception. The server-side portion of RTM is mainly used as a "dumb" data store, and the application periodically synchronizes with the server. In this case, using Gears to provide offline access was a natural fit, and was relatively quick to implement as we had some prior experience with data synchronization protocols.
There were some features of the online experience that could not be carried over to the offline mode. One of these was the Google Maps integration, in which users can geolocate their tasks and quickly visualize where their tasks are occurring in the real world. As Google Maps requires access to Google servers to fetch map tiles and data, and such a data set is quite large and thus hard to cache, this functionality is disabled once the user enters offline mode.
The undo feature of RTM is also unavailable in the offline version as this is a complex server-side operation (due to the multi-user nature of RTM and the ability to share tasks and lists). Instead, the user is presented with a dialog box asking for confirmation if they execute a destructive action such as delete. Undo functionality in offline mode is on the RTM roadmap, however.
To the decisions they made on showing the user information on whether they are online or offline:
A fundamental design decision is whether to implement offline support as "modal" or "modeless." Choosing which style to implement will, in most cases, be dictated by the type of data the application works with and how much of that data will be available offline. One style is not necessarily superior to the other, and, for example, it's much easier to implement a modeless style for tasks (in RTM) than it is for feed items (in Google Reader) based on the size of data items and the total data set alone.
Finally, we learn some tips and caveats from the RTM Gears implementation, including dealing with the LocalServer, the different types of offline, defensive coding, debugging, and coding with upgrades in mind.
Omar Kilani wraps it up in his conclusion:
By now, you should be itching to add offline support to your web application (we hope!). If you should take anything away from this article, it's that taking your application offline isn't as hard or complex as it may first seem, and that Gears is a joy to work with (and it'll become even easier and more fun as the project matures and is used by more applications).
As for us at RTM, we couldn't be happier with Gears. The speed at which we were able to provide offline functionality (four days from reading the documentation to a launchable implementation) is a testament to the quality, ease of use, and production-readiness of Gears. Many thanks to the Google Gears engineers for their foresight and for making this an open source project to which members of the Internet community can contribute.
Thanks to the Remember The Milk team for taking their application offline in record speed, and for taking the time to share their experience.
Brad Neuberg and SitePen have released a new beta of Dojo Offline. This release includes a full port to Google Gears, a port from Dojo 0.4 to 0.9, and more.
Features
An offline widget that you can easily embed in your web page with just a few lines of code, automatically providing the user with network feedback, sync messages, offline instructions, and more.
A sync framework to help you store actions done while offline and sync them with a server once back on the network.
A slurp() method that automatically scans the page and figures out all the resources that you need offline, including images, stylesheets, scripts, etc.; this is much easier than having to manually maintain which resources should be available offline, especially during development.
Dojo Storage, an easy to use hashtable abstraction for storing offline data for when you don't need the heaviness of Google Gear's SQL abstraction; under the covers Dojo Storage saves its data into Google Gears.
Dojo SQL, an easy to use SQL layer that executes SQL statements and returns them as ordinary JavaScript objects.
New ENCRYPT() and DECRYPT() SQL keywords that you can mix in when using Dojo SQL, to get transparent cryptography for columns of data. Cryptography is done on a Google Worker Pool thread, so that the browser UI is responsive.
Integration with the rest of Dojo, such as the Dojo Event system.
I personally like Apollo, and it had already built recognition, but it is now called Adobe Air.
What is new in the Air beta?
New features in the Adobe AIR beta include an embedded local database, PDF support, enhanced capabilities for JavaScript developers, and deeper integration with Adobe Flex.
The new beta version of Adobe AIR allows developers to incorporate PDF by leveraging Adobe Reader® 8.1 functionality. Users will be able to view and interact with PDF documents within Adobe AIR applications similarly to how they interact with a PDF in the browser today. New capabilities such as support for transparent HTML windows, drag and drop support, and complete access to Adobe AIR and Flash APIs allow Ajax developers to create truly engaging desktop applications. Developers building Adobe AIR applications can use the Ajax frameworks of their choice, and the latest version of WebKit incorporated into Adobe AIR beta provides more components than were previously available in Apollo alpha. The embedded, cross-platform, open source SQLite local database was one of the most requested features from the Apollo alpha release. It requires no extra setup while providing large data capacity and full text search, enabling Web developers who traditionally rely on a database for storage to easily build desktop applications without changing existing techniques. Additionally, a tool is now available on Adobe Labs for Dreamweaver® CS3 that enables Dreamweaver projects to be delivered as Adobe AIR applications.
Remember The Milk is one of the early Web 2.0 entrants that keeps innovating. I think that they were probably the first popular-non-google web applicationto go offline with Gears.
They decided to take an approach similar to Google Reader's offline model:
What can I do offline?
Just about everything that's possible online with Remember The Milk now works offline too -- not only can you access your lists, but you can add new tasks and notes, edit existing tasks (complete, postpone, prioritise, tag, and change due dates to your heart's content), use your own personal tasks search engine, create new Smart Lists, and more.
We asked Omar Kilani of the team a couple of questions about the offline release / experience:
Why did you take the step of asking the user to go offline (as reader does) versus just taking the todo list data offline in the background?
The app state is always in sync, and everything is updated in the
background (data is stored as in-memory JS objects, and sqlite is used
as a backing store in a write-through fashion). So you can pull the
plug, lose your Internet connection, etc, and the app will notice and
switch you to offline mode.
Right now, we take a sort of hybrid modal/modeless approach --
regardless of whether the app syncs in the background, we also thought
it was important to allow the user to control the mode they're in and
request a "sync".
I think an analogy to this is the 'Get Mail' button in email clients --
most mail clients check for new mail in the background, and if you're
using IMAP, you get notified immediately of new mail, but there's some
security in seeing that button there, and knowing you can press it. :)
I think we need to improve the UI aspects of this, and we're going to
look into doing that very soon. :)
How was the experience using Gears for offline?
Gears is awesome, and I believe it is the future of offline web
applications. :)
I personally feel it gives you all the right building blocks to do
whatever needs to be done to take an app offline.
What would you like to see the framework give you?
Progress indication from LocalServer! (This would be very helpful in
communicating with the user for the situation described above with
initial manifest download
A more stable WorkerPool. (It's currently pretty easy to cause a
browser crash with it.
Off the top of my head, how about bundle support in LocalServer?
Perhaps it can go download a Zip/JAR with all your files in it, and have
the manifest entries point into that. That would be pretty sweet.
