Ajaxian

Wednesday, March 18th, 2009

Super fast client side searches - the Flickr way

Category: Examples, JSON, Performance, XmlHttpRequest, Yahoo!

Over at the Flickr development blog, Ross Harmes, one of those lesser sung JavaScript heroes explains in detail how Flickr creates really fast client side searches and one of the implementations of these findings is the newly released find people faster feature:

The main findings of the team were that eval() is not only evil but also very slow whereas dynamic script nodes are fast but insecure. The solution was to do a custom evaluation of string data rather than using JSON:

Having set the performance bar pretty high with the last approach, we dove into custom data formats. The challenge would be to create a format that we could parse ourselves, using JavaScript’s String and RegExp methods, that would also match the speed of JSON executed natively. This would allow us to use Ajax again, but keep the data restricted to our domain.

Since we had already discovered that some methods of string manipulation didn’t perform well on large strings, we restricted ourselves to a method that we knew to be fast: split(). We used control characters to delimit each contact, and a different control character to delimit the fields within each contact. This allowed us to parse the string into contact objects with one split, then loop through that array and split again on each string.

Once this had been speeded up, all they needed to use was the YUI AutoComplete control and voilà - fast client side searches even with massive datasets.

Posted by Chris Heilmann at 2:38 pm
8 Comments

3.6 rating from 16 votes

Friday, August 1st, 2008

Cross domain access now, and support for the future

Category: XmlHttpRequest

Kris Zyp is really leading the charge on various missions such as JSON-* and XHR-*. This time he has a posting on a new cross-site XHR plugin repository that wraps up the myriad of techniques that are both pending in standards (XDomain, XHR++) and work arounds (window.name, magic iframe hackery). It also falls back to the faithful "just put up a proxy" for those poor browsers that can't handle the ninja.

Fantastic work as usual.

Posted by Dion Almaer at 6:45 am
11 Comments

4 rating from 33 votes

Friday, June 27th, 2008

The fight for cross domain XMLHttpRequest

Category: Security, XmlHttpRequest

There is a thread going on secure cross domain requests. Microsoft came out with a paper saying that the W3C standard isn't secure, and pushing the Microsoft XDR spec:

A few proposals and implementations exist like XDomainRequest in IE8, JSONRequest and the W3C’s Web Applications Working Group’s Cross Site XMLHttpRequest (CS-XHR) draft specification, which combines an Access control framework with XMLHttpRequest or other features. While XDomainRequest is focused on enabling anonymous access of third party public data, Cross Site XMLHttpRequest has added functionality and consequently enables a broader set of scenarios that may appeal to the developer who may choose to use cross domain authentication and access control among other features. As can be expected with securing a large cross section of cross domain scenarios, a number of concerns have been identified with the CS-XHR draft by the web development community, the IE team members and members of the Web Apps Working Group. For a list of our recent feedback on security on CS-XHR and our take on important security principles in cross domain, please read our Security Whitepaper on Cross Domain. The paper also covers best practices and guidance for developers who will choose to build on the current draft if it’s supported by a future browser.

The community quickly jumped on this in the comments, and beyond.

Anne van Kesteren said:

After half a year of waiting Microsoft finally posted their feedback on Access Control for Cross-Site Requests and specifically the way XMLHttpRequest Level 2 works with that. Microsoft blogged about this event. I suggest people read this rebuttal from Jonas on the paper Microsoft published. To be clear, while the specifications are not entirely finalized nobody has so far put forward a viable attack scenario that does not already apply when these technologies are not supported by user agents.

(Related: Working group fun and “Concerns” raised about W3C Access Control spec have been little more than FUD.)

As linked from Anne, Jonas posted nice feedback:

I'll start with a mini FAQ to avoid repeating myself below:

Why is the PEP in the client rather than the server?

In order to protect legacy servers some of the enforcement will have to live in the client. We can't expect existing legacy servers to all of a sudden enforce something that they haven't before.

In fact, even XDR using client side PEP. It's the client that looks for the XDomainRequest header and denies the webpage access to the data if the header is not there.

In fact, Access-Control does allow full PEP on the server if it so chooses by providing an "Origin" header.

Is Access-Control designed with "Security by design"

Yes. In many ways. For example Access-Control does not allow any requests to be sent to the server that aren't already possible today, unless the server explicitly asks to receive them.

Additionally Access-Control sets up a safe way to transfer private data. This prevents sites from having to invent their own which risks them inventing something less safe.

Thirdly, Access-Control integrates well with the existing HTTP architecture of the web by supporting REST apis and the Content-Type header. This allows existing security infrastructure to inspect and understand Access-Control requests properly.

What about DNS rebinding attacks.

Even with DNS rebinding attacks Access-Control is designed not to allow any requests which are not possible already in todays web platform as implemented in all major browsers.

Especially the last point is something that seems to have been misunderstood at Microsoft. It is not the case that DNS rebinding attacks affect Access-Control any different than it affects the rest of the web platform.

Posted by Dion Almaer at 9:24 am
19 Comments

4.3 rating from 20 votes

Friday, April 18th, 2008

100 Line Ajax Wrapper

Category: Browsers, IE, XmlHttpRequest

Kris Zyp gives us a glimpse at a potential future with his 100 line Ajax wrapper that tries to do the right thing cross browser for the various cross-domain models:

With IE8’s new XDomainRequest feature, a new API is added for cross-site requests, instead of using the W3C cross-site access proposal. Just for fun, I thought I would provide a little glimpse of what the classic Ajax request wrapper function may look like for the next era of web developers. Just a few simple calls to XMLHttpRequest would be way too easy, so instead we get do this:

I think that says.... please just implement the standard IE team! :)

Posted by Dion Almaer at 6:45 am
2 Comments

3.9 rating from 9 votes

Wednesday, April 16th, 2008

Last call for W3C XMLHttpRequest comments

Category: Standards, XmlHttpRequest

The W3C has issued a last call on the XMLHttpRequest spec:

The Web API Working Group has published the Last Call Working Draft of The XMLHttpRequest Object. The XMLHttpRequest Object specification defines an API that provides scripted client functionality for transferring data between a client and a server. Comments are welcome through 2 June. Learn more about the Rich Web Client Activity.

It is nice to see things all speced out, including the fact that you can now get back real errors (SECURITY_ERR, NETWORK_ERR, ABORT_ERR).

There are future thoughts for the spec.next too:

• load event and onload attribute;
• error event and onerror attribute;
• progress event and onprogress attribute;
• abort event and onabort attribute;
• Timers have been suggested, perhaps an ontimeout attribute;
• Property to disable following redirects;
• responseXML for text/html documents;
• Cross-site XMLHttpRequest;
• responseBody to deal with byte streams;
• overrideMimeType to fix up MIME types;
• getRequestHeader() and
  removeRequestHeader().

If you have some final thoughts, let them know!

Posted by Dion Almaer at 11:01 am
2 Comments

4.5 rating from 17 votes

Tuesday, April 15th, 2008

Dojo XHR Plugins; How do you want your XHR today?

Category: Dojo, XmlHttpRequest

Neil Roberts goes into the XHR Plugins that Dojo uses and how you can extend the system to have your own.

If you look at dojo.xhrGet you will see "Acceptable values are: text (default), json, json-comment-optional, json-comment-filtered, javascript, xml", but:

What you may not know is that the handleAs parameter is merely a way of specifying what plugin to use. Knowing where these plugins are, how they work, and how they can be adapted to suit your project will allow you to make repetitive tasks easy and less error-prone.

He starts by piggybacking on the json style callback, and adds a hook so when you do a query, if there are updated objects out there, they come back in the JSON so you can deal with them without having an extra remote call:

Next, Neil goes on to write a system that auto updates nodes:

which is as easy as:

In conclusion:

Dojo makes it incredibly easy to change the way that your Ajax calls work. You can change the format of JSON your server returns without having to change any of your callbacks, you can change the handleAs type for a single function to change the data given to your callback, you can get rid of callbacks altogether and use the arguments to your xhr call determine what should be done with your results.

Posted by Dion Almaer at 6:59 am
2 Comments

4.8 rating from 17 votes

Thursday, January 10th, 2008

Cross-Site XMLHttpRequest in Firefox 3

Category: Security, XmlHttpRequest

John Resig has written up documentation of Cross-Site XMLHttpRequest that discusses the W3C Access Control working draft which Firefox 3 implements.

He gives us a nice example:

In a nutshell, there are two techniques that you can use to achieve your desired cross-site-request result: Specifying a special Access-Control header for your content or including an access-control processing instruction in your XML.

In HTML:

PLAIN TEXT

PHP:

1.  
2. <?php header('Access-Control: allow <*>'); ?>
3. <b>John Resig</b>
4.  

In XML:

PLAIN TEXT

XML:

1.  
2. <?xml version="1.0" encoding="UTF-8"?>
3. <?access-control allow="*"?>
4. <simple><name>John Resig</name></simple>
5.  

And the XHR code itself isn't different from any other XHR code:

PLAIN TEXT

JAVASCRIPT:

1.  
2. var xhr = new XMLHttpRequest();
3. xhr.open("GET", "http://dev.jquery.com/~john/xdomain/test.php", true);
4. xhr.onreadystatechange = function(){
5.   if ( xhr.readyState == 4 ) {
6.     if ( xhr.status == 200 ) {
7.       document.body.innerHTML = "My Name is: " + xhr.responseText;
8.     } else {
9.       document.body.innerHTML = "ERROR";
10.     }
11.   }
12. };
13. xhr.send(null);
14.  

Some are excited to see the cross domain work, and some are concerned.... e.g.

I agree with Thomas. I never understood the NEED to modify the client security model to allow for this. If this is something the software needs to do, then the developer can implement a proxy on the server side. At least in this way the developer has sole discretion on the connections. Just more to go wrong if you ask me.

-

I'm still under the impression - and correct me if I'm wrong - that all these means are tailored to protect the server and its documents. But I thought the issue was to protect the client!

-

What exactly is the reason we need this? Has anybody here really understood why XMLHttp is currently limited to one host and cannot communicate cross-domain? I really do not understand that. If XMLHttp cannot do this by default, why it is still possible to load scripts and images from other servers? Why can I do exactly the same type of cross-domain communication using Flash, maybe using Silverlight in the future? What is the original reason for this limitation? Is this documented anywhere?

If, as mentioned in the spec, HTTP DELETE is problematic, because it may delete data, why cannot we filter such actions when detecting a cross-domain communication? GET and POST are possible in the same way when submitting simple form. It is even possible to generate these form elements dynamically. And this also works cross-domain. At least these two HTTP methods should be enabled by default to allow cross-domain communication. The open web, as often mentioned by Alex Russell, really needs features comparable with closed source software e.g. Flash or Silverlight.

-

I agree with those saying that this spec is misguided. But bothering users too much is also not good. How are they to know in every case what things mean?

What do you think?

Posted by Dion Almaer at 12:29 pm
12 Comments

4.2 rating from 26 votes

Wednesday, November 21st, 2007

Cross Domain XHR W3C proposal

Category: Security, XmlHttpRequest

The W3C has a new proposal titled Enabling Read Access for Web Resources which defines access control primitives to be used for cross domain XHR.

You can set control via a HTTP header:

or an XML processing instruction:

so no cross_domain.xml.

Kris Zyp has written up his thoughts on the proposal:

A number of things to understand about this proposal:

• It does not create any new vulnerabilities with existing servers. Cross domain XHR will always fail with existing servers until they have specifically added headers to define the access control. In other words it doesn’t add new vulnerabilities to the web, rather it allows those who want to add cross site access the ability to due it in a secure manner without hacks like JSONP or fragment identifier messaging.
• Both GET and POST can currently be executed cross site with scripts tags or form submission, so many threats such as CSRF and DOS already exist, the proposal does not introduce them.
• The proposal states that cookies should be removed from cross site requests. This will reduce the incident of cross site request forgery, and forces developers to use more secure explicit forms of authentication maintanence.
• Developers that allow cross site access still must ensure that they are not providing privileged information to sites that should not be accessing the information. Developers that allow POST and other modifying operations should take similiar precautions.
• This provides a fine-grained access control level. When servers define access control headers that allow cross site access, they can specify which web page domains are allowed to access their resources.

Kris also has a comparison with JSON approaches:

The core XMLHttpRequest object itself has also seen a recent new draft.

Posted by Dion Almaer at 6:56 am
6 Comments

3.4 rating from 45 votes

Friday, September 15th, 2006

Eliminating async Javascript callbacks by preprocessing

Category: Programming, Toolkit, XmlHttpRequest

According to Harry Fuecks in this post on the SitePoint PHP blog, using Ajax should be easier:

The Catch 22 of AJAX is, for the sake of an easy life, most of the time we want to write “synchronous code” but asynchronous is the only way to avoid some rather nasty usability issues. This means rather than being able to write simple code, as we’d like to. We’re required instead to handle this via callbacks, but that’s now introduced a whole load more potential issues.

These issues he mentions include requiring a global XMLHttpRequest object to be available and handling multiple calls to a javascript function (like if the user gets a little too impatient). To help combat these issues, Harry recommends a two projects out there that have the functionality to make life a little bit simpler:

• Narrative Javascript
• jwacs (Javascript With Advanced Continuation Support)

Posted by Chris Cornutt at 8:26 am
10 Comments

3.1 rating from 42 votes

Wednesday, August 9th, 2006

Cross-Domain Ajax Insecurity

Category: Ajax, Security, XmlHttpRequest

Chris Shiflett has posted his look today at cross-domain Ajax requests and some of the security implications that can come with it, especially in a world where more and more developers are beginning to think it's okay.

Since the birth of Ajax (the term, not the technology), there has been an increasing interest in various client-side technologies, especially JavaScript. Those who have forged ahead in an attempt to innovate new ways of applying Ajax have inevitably run into the same-domain security policy of XMLHttpRequest(). As a result, there has been an increasing demand for cross-domain Ajax, and there are several creative techniques in use today to get around the same-domain restriction (none of which I consider cross-domain Ajax).

He talks about other methods that can capture the data in an Ajax request (post scanner), but notes that one of the real dangers is removing a barrier for cross-site request forgeries that most normal sites already have in place.

To illustrate, he mentions an issue Digg had with a "self-digging story" a little while back. He also includes a sort of how-to on the method that they used to accomplish the task - basically a Javascript form submit on each viewing. There are checks in place for it now, but there's still the same kind of issue with cross-domain requests. Sure, you'd have to lure diggers to another page to get the key required for another digg, but since the code runs on the client, digg doesn't have much protection.

It's worth noting that XSS vulnerabilities allow malicious JavaScript to execute within your domain, thereby avoiding the same-domain restrictions. This can have catastrophic consequences. Just ask Myspace.

Posted by Chris Cornutt at 10:03 am
5 Comments

3.8 rating from 53 votes

Friday, July 28th, 2006

IE7 XMLHttpRequest - Native or Not?

Category: IE, XmlHttpRequest

MS announced this week that IE7 will be pushed as a high-priority update, so we can expect it to be popular pretty quickly. Reader Shawn Lauriat brought our attention to the question: How native is IE7's XMLHttpRequest?

The IE team have promoted the new IE7 as including native XMLHttpRequest. This is the case, insofar as you can instantiate an XHR using new XMLHttpRequest(). More importantly than the syntax, XHR will still work when ActiveX has been disabled (unlike IE6 and below).

On the other hand, Shawn notes that some issues exist. Some have pointed out that its more of a native facade than a native Javascript object. Specifically:

• xhr.prototype fails. Indeed, it's reported that any dynamic member creation fails (e.g. xhr.callId = 25; an idiom that can be useful for Call Tracking). If this is still the case, it's not the behavior of a native object and it's not consistent with other browsers.
• It's also worth pointing out that IE has an option to disable native XHR. (Aside: can we switch to positive terminology already - "enable" rather than "disable" ... it's hardly a secret of HCI that options should be stated in the positive :-/). The XHR option is, reasonably enough, motivated by security. Although it sounds like XHR will default to enabled (sorry, "not disabled"), it's still a reality that some users will be continue to be lost if you rely on XHR. Don't throw out that IFrame just yet!

IE7 XHR - Native or Not?

Posted by Michael Mahemoff at 2:42 pm
19 Comments

4.1 rating from 44 votes

Monday, July 24th, 2006

XMLHttpRequest Quirks and PHP

Category: PHP, XmlHttpRequest

On his blog, Jacob Santos has written up two simple "gotchas" that he's come across in his PHP/XMLHttpRequest adventures and wanted to share with other developers forging their way through the same experience.

I didn’t find the AJAX frameworks much use while I was working on my current project. I’m sure they are well thought out and designed, but after going through two or three, I was more lost than when I started. I decided then that I should learn how this whole AJAX thing works from the ground up. Turns out XMLHttpRequest isn’t all that difficult, once you get past a few JavaScript cross browser hiccups.

He talks briefly about his PHP backend and the methods that are available to return data in (XML/HTML/JSON) before talking about the issues he found:

• Don’t Create an Instance of the Same Object For multiple Tasks
• Always Call XMLHttpRequest Object First

For both, he gives a bit of code to explain the issue and to illustrate a workaround method, including the full code at the end of the post.

Posted by Chris Cornutt at 9:34 am
4 Comments

3.7 rating from 30 votes

Thursday, July 20th, 2006

XMLHttpRequest Case-Sensitive in IE7

Category: IE, XmlHttpRequest

Sleepyhead has posted a quick note about the XMLHttpRequest support in Internet Explorer 7, some bad news - it's case-sensitive.

The native XMLHttpRequest object in IE7 is case-sensitive. This is no big surprise - however that was not the case with the XMLHttpRequest active-x object in earlier versions of IE. So if you have old code that used uppercase in method/functions that worked with IE6 you have to change it for it to work in IE7. Took me a while to figure out why an old script of mine wasn't working...

He gives the code example for his fix, noting that "Status" will no longer be the same as "status".

Posted by Chris Cornutt at 6:59 am
8 Comments

3.7 rating from 31 votes

Tuesday, July 11th, 2006

Go forth and API

Category: Programming, XmlHttpRequest

Backend scripts aren't the only way to access the wealth of valuable web services out there. Anyone that's done any playing around with Ajax can see the possibilities of combining the two - accessing the APIs directly from your client-side application. In this brief tutorial from ThinkVitamin.com, they follow this course and give you a simple example to get the ball rolling.

To most, the virtues of Web 2.0 are rather ephemeral; that’s always been one of its main criticisms. However, I like to think that one of the movement’s key aspects is a sense of community, an ability to create sites and applications that bring people together.

Even Web APIs aren’t a new idea. Google’s search API has been available via SOAP since 2002, and there’s definitely older services than that. However, the recent growth in Web API availability has been fuelled by two recent developments. The first, which I’ve already mentioned, was a philosophical change in the way that data is handled. The second was the introduction of AJAX. Again, not a new idea, or even a new technology, but sometimes it’s all about timing.

They mention several of the different APIs out there (including Flickr, Google Maps, and Amazon) as well as some of the mashups that take advantage of the merging of these APIs. To illustrate the point of the simplicity of these interfaces, they include a code example of connecting to the Flickr API to grab photo information. There's also a bit on proxying your XMLHttpRequests and a brief look at using JSON to communitcate with the APIs that support it.

Posted by Chris Cornutt at 8:28 am
12 Comments

3.7 rating from 22 votes

Thursday, June 8th, 2006

The Ajax Transport Method(s)

Category: Ajax, Programming, XmlHttpRequest

The IBM developerWorks site has this new tutorial posted today, a look a the various "Ajax transport methods" - XMLHttpRequest, IFrames, and script tags - what they are, how to use them, and which is best for what situation.

Discover three Ajax data transport mechanisms (XMLHttp, script tags, and frames or iframes) and their relative strengths and weaknesses. This tutorial provides code for both the server side and the client side and explains it in detail to provide the techniques you need to put efficient Ajax controls anywhere you need them.

In addition to understanding how the client requests data from the server, you'll also learn how to transport different data types. In addition to Extensible Markup Language (XML), you can move plain text, Hypertext Markup Language (HTML) pages, or JavaScript code.

You'll either need to register to get to the actual tutorial or you can check out the PDF version of it directly. They start with creating the database foundation for the backend, including the PHP code that you'll need to interface with it (the code is available for direct download as well). They also create the backend interface, various types of services - plain-text, XML, RSS, RDF, and JSON.

From then on, they construct the clients to interface with those backends, first with XMLHttpRequest, then an iframe, finally the Javascript "script" tag. For each, they make a an example call to the backend(s) retrieving information of various books in the created table (author, title, ID number).

Posted by Chris Cornutt at 7:06 am
9 Comments

3.8 rating from 39 votes

Tuesday, June 6th, 2006

IFrame + Script Tags = Portable Comet

Category: Comet, Examples, IE, Office, XmlHttpRequest

In a recent post, I explain the difficulties of Comet (Streaming/Push) in IE.

IE makes it difficult for two reasons: (a) IE's XMLHttpRequest component doesn't tell you anything about the response until the connection has closed - even if you try polling it instead of relying on onReadyStateChange, you'll still get an empty string (Try it); (B) Okay, switch to plan B and inspect IFrame content - we can't rely on onload, which is only called once at the end, so we *must* poll. But no, polling won't help either, as the IFrame content remains empty until (you guessed it) the connection is closed. (Try it).

Fortunately, there *is* a portable solution, demonstrated here.

The portable solution is this: Have the server continuously output script tags that call a known function in the parent frame. When you set the child IFrame's source to point to this service, it will start evaluating the inline scripts as they pop out of the server.

Of course, none of this is really new, at least not to some people. Ajax pioneer Brent Ashley points out:

The ReadyState 3 issue that Michael talks about has been well known (well, apparently not well known) at least since Scott Andrew LePera described the problem in late 2002. It really needs to be fixed.

Hopefully, future versions of IE will expose the ongoing response while the XHR connection is still open. But until then, we at least have a workable technique that works across different browsers.

This area isn't very well-documented; It would be good to hear what others have done to make streaming portable.

Posted by Michael Mahemoff at 4:33 pm
28 Comments

4 rating from 57 votes