Tuesday, April 3rd, 2007
Category: JSON
, Remoting, Security
Ajax pioneer Brent Ashley has written a Developerworks article about making Ajax mashup secure. It looks at where it’s at today and where it’s all headed.
He begins by surveying current techniques for calling external servers, such as the popular On-Demand Javascript technique. This has well-known security issues.
The scalability benefit of the <script> tag comes at the cost of sidestepping the Same Origin Policy security model, introducing potential attack vulnerabilities:
- Cross-site cookie access becomes possible: Scripts from one site can access cookies from another site.
- There is no opportunity to inspect the retrieved code for safety issues before running it: The code runs immediately upon loading.
One short-term solution is the following IFrame fragment identifier hack.
A more recently developed content-retrieval technique employs communication between a page’s script and a hidden iframe through its src URL’s fragment identifier (the part of the URL that comes after the # sign). Scripts in the parent page and embedded iframe can set each other’s fragment identifiers despite coming from different origins. An agreed-upon communication protocol is maintained between the scripts, driven by JavaScript timers that periodically fire routines to check for changes in the fragment identifier.
We’ll hopefully see more flexible, purpose-built, solutions in the future, and Brent’s article summarizes the proposals under discussion - JSONRequest, <module> tag, content restrictions header, W3C Access Control List (ACL) System, Cross-browser.xml.
With all these facilities potentially in the pipeline, one can only hope there will be a clear winner that works in all major browsers, or at least enough overlap that the Ajax libs can provide a straightforward abstraction!!!
Thursday, March 15th, 2007
Category: JavaScript, Library, Remoting
UED is a tiny library that takes a hash and converts it into a URL. Instead of passing a JSON string, for example, you could just construct a URL containing the transfer object. Update: I should mention that this is more about the format than the library...UED is a proposal for a standard way to encode an object into a URL, in the same way that JSON is more a format than a library.
Since it assumes you are making GET calls, RESTful design would suggest only using UED for non-destructive calls (i.e. calls that don't change server state).
A hash is encoded like this:
JAVASCRIPT:
-
-
//The JS Array format of the example given above
-
var arr = {
-
'name':"Binny",
-
'year':2007,
-
'quote':"Hello, World!",
-
'os':['Windows','Linux','Mac'],
-
'software':{
-
'editor':"vi",
-
'audio':"xmms",
-
'video':"vlc"
-
}
-
}
-
var data = ued_encode(arr);
-
leading to a URL like this:
JAVASCRIPT:
-
-
http://www.example.com/get_data.php?name=Binny&year=2007&
-
quote=Hello%2C+World%21&os[]=Windows&os[]=Linux&os[]=Mac&
-
software[editor]=vi&software[audio]=xmms&software[video]=vlc
-
However, there are no server-side decoders as yet.
Wednesday, January 31st, 2007
Category: JavaScript, Programming, Remoting
Plaxo's Joseph Smarr has been playing with on-demand javascript, i.e. downloading extra JS code after the page has already loaded. When you grab the code via a remote call and eval() it, it doesn't get into global scope. So here's how he dealt with it.
Here’s a simplified version of the situation we faced:
function loadMyFuncModule() {
// imagine this was loaded via XHR/etc
var code = 'function myFunc() { alert("myFunc"); }';
return eval(code); // doesn’t work in FF or IE
}
function runApp() {
loadMyFuncModule(); // load extra code “on demand”
myFunc(); // execute newly loaded code
}
The thing to note above is that just calling eval() doesn’t stick the code in global scope in either browser. Dojo’s loader code solves this in Firefox by creating a dj_global variable that points to the global scope and then calling eval on dj_global if possible:
function loadMyFuncModule() {
// imagine this was loaded via XHR/etc
var code = 'function myFunc() { alert("myFunc"); }';
var dj_global = this; // global scope object
return dj_global.eval ? dj_global.eval(code) : eval(code);
}
This works in Firefox but not in IE (eval is not an object method in IE). So what to do? The answer turns out to be that you can use a proprietary IE method window.execScript to eval code in the global scope (thanks to Ryan “Roger” Moore on our team for figuring this out). The only thing to note about execScript is that it does NOT return any value (unlike eval). However when we’re just loading code on-demand, we aren’t returning anything so this doesn’t matter.
The final working code looks like this:
function loadMyFuncModule() {
var dj_global = this; // global scope reference
if (window.execScript) {
window.execScript(code); // eval in global scope for IE
return null; // execScript doesn’t return anything
}
return dj_global.eval ? dj_global.eval(code) : eval(code);
}
function runApp() {
loadMyFuncModule(); // load extra code “on demand”
myFunc(); // execute newly loaded code
}
And once again all is well in the world.
Tuesday, November 7th, 2006
Category: Books, Editorial, Remoting, Security, Testing, The Ajax Experience
Often when you hear discussions regarding Ajax and security, its said that the issues remain the same as they were ten years ago: don't trust user input, don't expose sensitive data without encryption, code for security from day one, never display system errors messages, etc. While that is all true and good, one thing I heard from the Ajax Experience that stuck with me is that "ajax increases the typical amount of attack vectors". We are hitting the server more often, with different transports, and often talking to remote servers as well for services. This will only become a bigger issue as cross domain ajax becomes more prevalent and libraries and tools make it easier to mash things up without having to know each individual services' API. Do the developers you work with keep up to date on writing secure code? Have you seen your ajax app exploited by cross-site scripting attacks or sql injection, or are do you consider things "safe" because you are only doing intranet work?
With that in mind, Michel Sutton's entry on ten common security mistakes might be a good refresher. His earlier entry on SQL injection is also worth a read, particularly if you are hand-coding sql and aren't using a database library that handles parameterized SQL statements for you (though if thats the case you might have bigger issues...)
Recently I went looking for an authoritative book on web app security for some fun-filled weekend reading, and came up with very few hits. The closest I found was How to Break Web Software
and Hacking Exposed Web Applications, Second Edition
. How to Break Web Software has a bunch of good reviews and looks to be a good high level coverage of many of the common attacks. Hacking Exposed is a bit newer and has less reviews, though the first edition looked to be pretty well received. That title and cover are pretty painful, though. Is there an equivalent to the K&R C Book
for web app security?
There are a ton of books on server security and locking down your OS, but not much that targets web applications specifically. Any other good suggestions? Any web security blogs worth subscribing to?
Friday, August 25th, 2006
Category: Ajax, Books, Google, Remoting
Dave Thomas over at the Pragmatic Programmer let us know about a new Friday (i.e., short book that can be read entirely in a Friday afternoon) that they've published: Google Web Toolkit, by Ed Burnette.
Dave was kind enough to provide us with an excerpt for distribution to our readers: Chapter 5 -- Remote Procedure Calls; seems appropriate for an Ajax readership. If you've been wanting an easy way to learn more about GWT, here's your chance.
Thursday, July 13th, 2006
Category: Comet, Remoting
Comet is mostly considered a server-to-browser thing, but how about a permanent connection in the opposite direction, from browser to server? I've been talking about this on my blog and received some interesting thoughts from Alex Russell.
There are two key issues:
(1) Server needs to start outputting before incoming request is finished. With a specialised server, this problem could be overcome.
(2) (More serious as we can't control the browser) The browser would need to upload data in a continuous stream. You can do it with Flash/Java, but I can't see how to do this with standard JS/HTML. If you use XHR, you're going to call send() and wave goodbye to the entire request...there's no support for sequencing it. Same if you submit a regular form, change IFrame's source etc. Even if you could somehow delay reading of content so it's not immediately uploaded, the browser would probably end up not sending anything at all as it would be waiting to fill up a packet.
Now I've seen various people mention the possibility of HTTP keep-alive, but I've never actually seen any concrete demos or techniques to take advantage of it from a script. So if you know of any ...
Anyway, Alex Russell says it's probably not possible, but we can get around it anyway:
So I've spent some time investigating this (as you might expect), and at
the end of the day there's not much to be done aside from using Flash
and their XMLSocket interface. That's an obvious possibility given the
high-performance Flash communication infrastructure we have in Dojo.
Doing bi-directional HTTP probably won't happen, though, but I don't
think that's cause for despair. In my tests, we can get really good
(relative) performance out of distinct HTTP requests so long as the
content of the request is kept to a minimum and the server can process
the connection fast enough. HTTP keepalive exists at a level somewhat
below what's currently exposed to browsers, so if the client and server
support it, frequent requests through stock XHR objects may verywell be
using it anyway. We'll have to do some significant testing to determine
what conjunctions of servers/clients might do this, however.
As an interesting side note, he also pointed to some work going on at http://cometd.com to build an open Comet protocol.
Friday, July 7th, 2006
Category: Database, Remoting
A new PHP component by Adnan Siddiqi accepts a MySQL result set and converts it into a JSON message. MySQL To JSON:
This class can be used to convert data from MySQL query results into a JavaScript expression in JavaScript Object Notation.
It takes a MySQL query result handle and retrieves the query result column names and the query result data.
The class generates the definition of a JavaScript object in JSON that contains an array of a rows of query result data.
Each array element represents an object with the properties set to the query result column names. The property values are the query results for the respective row and column.
Thursday, June 22nd, 2006
Category: Remoting, Scriptaculous, Toolkit, Yahoo!
Cheng Guangnan reports on a potential issue with the autocompletion/suggestion support offered by both Yahoo UI and Scriptaculous libraries. The problem involves parallel calls - there's the potential for an initial list of suggestions to be displayed after a subequent list. His screencasts show what's going on.
1. “2006†is typed.
2. A request of “2006†sent to the server.
3. User continues typing and now “200607†typed.
4. Another request of “200607†sent to the server.
5. User waiting for feedback.
6. The second request return, it show the popup.
7. The first request return, it show the popup with data returned for “2006â€.
If that's the case (and we haven't verified it!), the problem could be solved by some form of Call Tracking. If the first call comes back after the second, simply discard it.
Tuesday, June 13th, 2006
Category: Remoting, Usability
"A" may stand for Asynchronous, but PPK recently asked his readers if people are really exploiting the asynchronous nature of Ajax. Are there really situations where the user can do something while a request takes place? For instance, GMail makes an asynchronous call to grab some mail data - do you actually play around with other controls while the data downloads? Probably not. So he's wondering...
(W)hether asynchronous communication is all that it's cracked up to be from a practical point of view. If in practice it's not useful to initiate a new request while waiting for the response to a previous request, Ajax's main user interface advantage is kind of nullified.
The comments to that post indicate there are indeed practical uses of asynchrony ... he's subsequently summarized four kinds of asynchronous usage:
- Sending data to the server (AKA fire-and-forget)
- Refreshing data (AKA polling, Periodic Refresh)
- This is the (as yet single) example of Ajax as I always supposed it would work: quietly but incessantly refreshing the page content based on repeated user actions
- New "pages" (AKA Microlink)
Wednesday, May 24th, 2006
Category: Comet, Java, Remoting
More and more, Ajax apps are using various techniques to keep content fresh in the browser. Essentially, we're talking about reversing the usual communication flow- the server notices something's happened, and wants to tell the browser about it. Server "calls" browser, not browser calls server.
The popular Java Ajax framework, DWR, recently released milestone 2 towards a 2.0 release, and introduced the term "Reverse Ajax" in the process, to capture this high-level pattern. This article (via Joe Walker) abstracts the communication details behind a clean API, so you can easily switch between three different implementations of Reverse Ajax:
1. Polling: This is where the browser makes a request of the server at regular and frequent intervals, say every 3 seconds to see if there has been an update to the page. To illustrate imagine a 5 year old (or a donkey) in the back of the car shouting 'are we there yet?' every few seconds and you get the picture.
2. Comet, long lived Http, or the slow load technique: Are all names for the same thing. As already mentioned, the server has to wait for the browser to make contact. But this technique allows the server to start answering the browser's request for information very slowly. Extremely slowly. Actually in the same way I used to answer my French teachers at school, it starts the reply but never actually finishes. This allows the server to keep a communications channel open (unlike me and my French teacher) to pass down additional information when the time comes. The closest we currently get to a server push. See Alex Russell's article for the coining of the phrase and outline of definition of Comet. See Bryce Nesbitt's for a brief description and simple demo of slow-load.
3. PiggyBack Technique: Here the server, having an update to send, waits for the next time the browser asks it a question and... pounce, sends the answer and the update. This technique is well understood in long term relationships and is usually preceded by 'and another thing.' You know, you ask what you thought was a straight forward question and get a lot more than you bargained for.
Monday, April 17th, 2006
Category: Remoting, Showcase
King Ping provides a similar service to sites like Ping-O-Matic, which accept a blog URL and notify sites like Technorati that an update has occurred. Not everyone needs these services anymore as the process is often automated, but for those who do, King Ping gives you a nice Ajax interface for it.
The application uses Multi-Stage Download - an XHR request is uploaded for each engine to be notified. The King Ping service at the other end then acts as a Cross-Domain Proxy, routing the call toward an engine and passing back the result to the browser. In the web page's results area, there's a designated status area for each engine, which gets filled as soon as the browser receives the corresponding response.
Joe Anderson, who "pinged" us about it, says there are still issues with King Ping.
King Ping’s AJAX runs very smoothly, though I don’t like the fonts and colours used throughout the site. For some reason, I find the constant use of grey quite depressing; and I think a nice blue gradient would revamp the site.
King Ping has a few other faults. It claims to be able to ping BlogShares, but in fact it can’t. BlogShares RPC is only available to approved parties (Pingoat and Ping-O-Matic), so BlogShares refuses connections from anyone else. Another fault is it doesn’t provide an RPC. This stops you adding it to your blogging software to automatically ping after each update. Ping-O-Matic and Pingoat, however, do offer RPCs.
Wednesday, April 12th, 2006
Category: Remoting, Security
Some of you will be familiar with TheDailyWTF.com, a website showcasing code in the wild that is, well, less than professional. A recent forum item illustrates the ultimate Ajax antipattern: uploading arbitrary code to be executed on the server.
Gustavo Carvalho discovered what happens when XMLHttpRequest and the Eval() function in PHP are combined. I'll leave it to your immagination as to what the server-side looks like ...
var code =
' $cn = mssql_connect($DB_SERVER, $DB_USERNAME, $DB_PASSWORD) ' +
' or die("ERROR: Cannot Connect to $DB_SERVER"); ' +
' $db = mssql_select_db($DB_NAME, $cn);
...
execPhp(code);
Still, you've got to admit the remote execution is nicely encapsulated in that little execPhp() function - no messing around with XHR here ;-).
(Thanks Matthias.)
Saturday, March 11th, 2006
Category: JavaScript, Remoting
Douglas Crockford, creator of JSON, has proposed that browsers include a new "JSONRequest" service to allow for safe cross-domain calls.
JSONRequest is a service which encodes a JavaScript value as a JSON
text, does an HTTP POST of that text, gets the response, and parses
the response into a JavaScript value. If the parse was successful, it returns
the value to the requesting script. In making the request, no HTTP authentication or cookies are sent.
Any cookies returned by the server are discarded. The JSONRequest service
can only be used to send and receive JSON-encoded values. JSONRequest
cannot be used to retrieve documents or other texts.
JSONRequest is a global function. It takes four parameters:
| parameter |
type |
description |
| url |
string |
The URL to POST to. The URL does not need to be related to the page's URL. |
| send |
object |
The JavaScript object or array to send as the POST data. It will
be serialized as JSON text. Cyclical structures will fail. |
| done |
function (requestNumber, value, exception) |
The function to be called when the request is completed. If the request was successful, the function will receive the request number and the returned value. If it is not successful, it will receive the request number and an exception object. |
| timeout |
number |
The number of milliseconds to wait for the response. This parameter is optional. The default is 10000 (10 seconds). |
It would be nice to have a safe component for cross-browser calls, though maybe an extension to XMLHttpRequest, not tied to a particular format like JSON, is preferable. Nevertheless, the article makes the case for a more constrained approach and lists several reasons why JSONRequest is safe enough for cross-domain requests:
- JSONRequest does not send or receive cookies or passwords in HTTP headers. This avoids false authorization situations. Knowing the name of a site does not grant the ability to use its browser credentials.
- JSONRequest works only with JSON text. The JSONRequest cannot be used to access legacy data or documents or scripts. This avoids attacks on internal websites which assume that access is sufficient authorization. A request will fail if the response is not perfectly UTF-8 encoded. Suboptimal aliases and surrogates will fail. A request will fail if the response is not strictly in JSON format. A request will fail if the server does not respond to POST with a JSON payload.
- JSONRequest reveals very little error information. In some cases, the goal of a miscreant is to access the information that can be obtained from an error message. JSONRequest does not return this information to the requesting script. It may provide the information to the user through a log or other mechanism, but not in a form that the script can ordinarily access.
- JSONRequest accumulates random delays before acting on new requests when previous requests have failed. This is to frustrate timing analysis attacks and denial of service attacks.
See Douglas's article for all the motivation and technical details of the JSONRequest proposal.
Wednesday, March 8th, 2006
Category: Comet, Programming, Remoting
Alex Russell has posted slides for his ETech presentation on Comet. Comet (which we mentioned the other day) is Alex's new term for push-style server-to-browser communication.
The presentation motivates Comet largely in terms of social and multi-user concerns, before moving onto the nuts and bolts.
- Ajax is me driven ... Social apps are also driven by others
- To any user, the server is other users.
- If the web is a conversation ... then stale context kills.
- Today's web servers won't cut it
Check out the slides for Alex's practical advice on implementing Comet. We can only hope the talk will find its way on an ETech podcast feed!
Related:
Saturday, March 4th, 2006
Category: Comet, Dojo, Programming, Remoting
Alex Russell has coined a term for a flavour of Ajax that's been getting more attention of late. Comet describes applications where the server keeps pushing - or streaming - data to the client, instead of having the browser keep polling the server for fresh content. Alex identifies several buzzworthy examples:
Comet uses a modified form of the canonical Ajax architecture:
As is illustrated above, Comet applications can deliver data to the client at any time, not only in response to user input. The data is delivered over a single, previously-opened connection. This approach reduces the latency for data delivery significantly.
The architecture relies on a view of data which is event driven on both sides of the HTTP connection. Engineers familiar with SOA or message oriented middleware will find this diagram to be amazingly familiar. The only substantive change is that the endpoint is the browser.
While Comet is similar to Ajax in that it’s asynchronous, applications that implement the Comet style can communicate state changes with almost negligible latency. This makes it suitable for many types of monitoring and multi-user collaboration applications which would otherwise be difficult or impossible to handle in a browser without plugins.
Like Ajax, Comet's a pattern that's been around for a while, but Alex felt there was no suitable name for it, to help describe the problems and solutions growing around it: "(I)n the spirit of improved communication (and not technology invention), I’m proposing a new name for this stuff"
The article explains why Comet is better for users (responsive) and how it can scale. He also points out that Dojo has supported Comet-style architecture for some time.
As a sidenote, it's interesting that 37Signals' Campfire chat app went with polling as opposed to "Comet" architecture - it polls the server every three seconds for new messages. Since 37Signals presumably has similar constraints to Meebo or JotSpot, why would they prefer polling? More generally, when to use polling and when to use Comet?
(UPDATE: We recently interviewed Jamis Buck of 37signals and addressed the push versus poll question. The answer is less about technology and more about following the agile development process -- more on that when we post the cast in the coming weeks. - Ben)
Remember the Push vision?
(A) new medium is arising, surging across the Web in the preferred, many-to-many way: anything flows from anyone to anyone - from anywhere to anywhere - anytime. In other words, a true network like the telephone system, rather than a radiating system like radio or TV. This new medium doesn't wait for clicks. It doesn't need computers. It means personalized experiences not bound by a page - think of a how-to origami video channel or a 3-D furry-muckers VR space. It means information that cascades, not just through a PC, but across all forms of communication devices - headlines sent to a pager, or a traffic map popping up on a cellular phone. And it means content that will not hesitate to find you - whether you've clicked on something recently or not.
-- Wired, March 1997.
4.1 rating from 105 votes
Wednesday, February 15th, 2006
Category: JavaScript, Remoting
Like it or not, there's plenty of people who want to do cross-domain remoting. The typical technique is via script tags, and since this means the response must be valid Javascript, JSON is usually the message format. But what if you want to grab some XML instead of a JSON-formatted object? Dave Johnson explains how to achieve cross-dom
