Wednesday, February 18th, 2009

Canvas for a Text Editor?

Category: Bespin, Canvas

>The following is a lightly edited repost from an entry on my personal blog.

Culling through the Bespin feedback we received post-launch, it’s clear that a few aspects of the project are controversial. I’d like to address one of them: our choice of canvas as the rendering engine for the editor.

(In addition to feedback, we’ve already seen some interesting community efforts at extending Bespin; check out our blog post on the Labs website for details.)

Bespin and Canvas

Bespin takes advantage of a relatively new feature of browsers to power its text editor: the <canvas> element. Canvas gives developers something the intriguing ability to render arbitrary two-dimensional graphics in a web page; it’s basically an <img> element you can draw into with JavaScript commands. The API looks something like this:

  1. // all drawing performed via the context
  2. var ctx = document.getElementById("canvasId").getContext("2d");
  3.  
  4. ctx.fillStyle = "black";  // accepts same values as CSS "background" property
  5.  
  6. ctx.fillRect(0, 0, 50, 50); // draws a black 50x50 rectange in the canvas

Canvas is available in the currently released versions of Safari, Opera, Firefox, and Chrome; it has been a part of the HTML 5 draft specification for a long time. It is not available in Internet Explorer.

Bespin builds on this API to recreate a text editor from scratch. That’s right; rather than use the native facilities of every modern operating system that give you word processing features for free, Bespin rolls them from scratch, including such basic constructs as a blinking cursor and text selection.

One of the the features we didn’t have to re-implement is text rendering; the HTML 5 specification includes an API to render text on a canvas. Unfortunately, this portion of the canvas API has only been implemented by Firefox, though it’s already in WebKit nightly builds (and thus will very likely be in a future version of Safari) and Chromium nightlies (from which Chrome versions are derived). We expect Opera will soon as well.

But Does It Scale?

We weren’t actually sure this would work when we set out to do it. You wouldn’t think a JavaScript interpreter in a browser could render enough frames on a large, modern 32-bit image display to make the experience anything close to what you get effortlessly in Notepad on Windows XP running on 10-year-old hardware.

To be sure, for some users, the performance does indeed suck–we’re looking into exactly why (it appears that certain video drivers cause performance problems on Linux; Google Chrome’s multi-process architecture appears to slow down performance on that browser). The interesting thing is that for large classes of users, performance is really, really good. There’s a lot to say about topic; we’ll touch on that more in a future blog post.

Why?

The first time we demoed Bespin to someone who understand these implementation details, the reaction was “Wow, you’re crazy!” followed by “That’s really cool.” Pretty much everyone else I’ve spoken with has the same first reaction (the crazy bit) but the follow-up is not always positive (ranging from the aforementioned to “WTF!?”).

Surely there’s a higher-level abstraction we could have built on? Something above pixel painting?

Well, let’s get one thing out of the way first-up: there may very well be a more effective way to do what we’ve done than with canvas. I’m not saying this is the best way to do it; I just want to explain why we did it.

Control, Control, We Must Have Control

One of our goals with Bespin is to be able to (at some point in the future) deliver a fantastically compelling user experience for editing code on the Web. We don’t want Bespin to be the editor you use when you can’t get to your desktop; we want it to be your editor of choice in any context.

It turns out that’s a tall order. Some users like word-wrapping; some don’t. Some users actually like column selection modes while other (more sane) users like line-oriented selection (I kid, I kid). Developers are notoriously finicky with their tools–especially their code editor–and we didn’t want to re-educate the world on how our editor worked; we want to make Bespin adaptable to as wide an audience as we could.

We felt that if we built Bespin on something like <textarea> or the “content editable” feature of HTML, we’d have a built-in set of limitations over which we’d have very little or no control.

The Alternatives

To build on the last paragraph, we considered three different alternatives to canvas:

  1. textarea
  2. contentEditable
  3. “line blocks”

The “control” argument above is one reason we dismissed textarea and contentEditable; further, we’ve seen a large number of other web editor implementations based on these approaches and based on playing with those and by the many horror stories floating around the Web from their authors, we were concerned that we couldn’t overcome performance and compatibility problems, especially when editing large files.

Line Blocks

“Line blocks” are a different story. What if we had a bunch of <div> elements stacked vertically and treated each one as a rendering surface for a line of text in the document being edited? We played with that approach a bit but couldn’t get as much performance out of it as we could with canvas, especially for wide files (>80 columns).

Compatibility

We’ve taken some flak for claiming that Bespin is based on “Open Web technologies” but only supporting Firefox and WebKit nightly. Earlier in this post, I explained the state of canvas support in the browsers and how we’re hopeful that our canvas approach will work in future versions of Chrome, Safari, and Opera.

However, we have a plan in case browsers don’t implement the proper text rendering APIs: bitmap font rendering. By pre-rendering fonts into images, we can then render the images on the canvas. There are a number of proof-of-concept implementations floating around that demonstrate this concept (though unlike some, we wouldn’t render the fonts as vectors–there’s no way it would perform to our needs).

What about Internet Explorer? It’s hard to say what the future of canvas support is for IE; it certainly isn’t coming in IE8. However, there are a number of projects that emulate canvas by mapping it to VML (an abandoned IE proprietary rendering technology), Flash, and Silverlight. So far, none of these bridges offers adequate performance for us to seriously considering supporting it. We’re aware of promising developments in this area and we’ll experiment with IE support as these technologies evolve.

Mobile Support: iPhone, Android, Fennec, and More

We’re also very excited to get Bespin working on mobile devices with fully-featured browsers. The iPhone’s browser–”mobile Safari”–does support canvas, but does not yet have the text rendering APIs. We may implement the aforementioned bitmap font rendering just to get on the iPhone. We are also looking at other mobile platforms, such as Android, Fennec, Palm, and so forth.

How cool would it be to change your code and redeploy it from the beach?

Obviously, it will take some interface changes for Bespin to be usable on a mobile device. We have some ideas here and we look forward to working with the community on developing the concepts further.

What do you think?

What do you think of our use of canvas in Bespin’s editor?

Related Content:

19 Comments »

Comments feed TrackBack URI

Personally I think that HTML5 should include new spec for a tag or suchlike, that way all this “messy” code can be written on the low-level side of things for a *much* better solution.

Having tinkered (a lot) with this sort of thing (codepress) I’d much rather see a non-javascript driven framework. A Java Applet, or even a Flash element will knock spots off in terms of speed…

Decidedly on the fence. :(

Comment by oopstudios — February 18, 2009

I look forward to the way things will be a couple years down the line. Naturally, based on the above, people will develop higher-level libraries on top of canvas so that you don’t have to take care of everything yourself for each new app.

And then the day will come when someone will implement a browser entirely in Javascript, on top of canvas. And it’ll run in your browser too!

I’m not sure which is sickest, that it will actually happen, or that I actually can’t wait for that day :)

Comment by robinberjon — February 18, 2009

The most immediate issue with the use of canvas is the nonsensical right mouse context menu.

Comment by jonathanpeterson — February 18, 2009

I think its a powerful statement to use canvas for this. Not only that, but it looks awesome, so great job.

If browsers can get canvas support and performance up to the point that its a viable option for full screen canvas, we could see a world of change coming. Its not a far cry to building higher levels of abstractions over canvas the same way we do with desktop software over native graphics support. In fact, I guess that’s already happening with Objective-J to a certain degree.

Clearly this is not the kind of thing one would consider part of the “semantic” or “accessible” web, but it could be a shot in the arm for developers of complex web applications thinking they’ll have to move to flash. Here’s hoping it works out.

Comment by genericallyloud — February 18, 2009

Link on “community efforts at extending bespin” should be:

http://labs.mozilla.com/2009/02/thanks-for-the-bespin-contributions-eclipse-xwiki-and-more/

Comment by jeromatron — February 18, 2009

Caveat: I haven’t actually tried Bespin yet.

In principal, I’m fine with a canvas-based text editor if it’s done right, but it has to provide all standard conventions on all platforms (accelerator keys are the biggie in my book). Also it needs to allow copy/paste in from other applications.

Are any of those things a problem in Bespin? Are any of them insurmountable?

Thanks,
Jeff

Comment by codedread — February 18, 2009

I’m definitely in the “WTF?” group. First off, I’m also in the group of Linux users whose performance was horrible, so maybe I’m biased. But I also find it strange that Mozilla Labs of all places is looking for weird performance hacks, rather than trying to make the ‘right way’ perform well.

Presumably the performance gain was mostly from painting only the relevant parts of a file that appear on-screen. Why not do the same thing with your DIV solution (nowrap, overflow:hidden)?

Comment by schuyler1d — February 18, 2009

Good job guys. I think it’s a great idea to take a stand and start pushing canvas for apps like this.

Another point of flexibility is keyboard bindings and vim/emacs emulation. Personally, I can’t justify using an IDE (online or other) for serious development unless it has decent vim emulation. I imagine a canvas solution is the best bet for something like.

Comment by Michael Mahemoff — February 18, 2009

I think you should have considered SVG. Given that entire vector graphics editors have been implemented in SVG, and it supports text better than Canvas ever will, and of course mouse events and javascript, it is a natural fit for your stated goals.

Comment by JonathanLeech — February 18, 2009

I think this is fucking amazing Ben, what you did with bespin is best defined as *magic*. And the fact that you don’t support IE, I consider a *bonus*…!!
.
Go Ben and Dion…!!!!!!! :))))

Comment by ThomasHansen — February 18, 2009

Please don’t even think of wasting your time to create an IE compatible version of Bespin. No sane developer will use IE in any version to do web development. Please invest your time in enhancing Bespin instead of wasting with supporting absolut crap!

Comment by AndiSkater — February 18, 2009

@JonathanLeech I agree not mentioning SVG is an omission. Although SVG is a much older standard, it also has huge support gaps across browsers and is again missing in IE 8. Atleast you can bridge canvas with a JS library. I think the common way to support SVG is with the (moribund) Adobe plugin.

I applaud the choice of canvas. Many great editors have shipped their own UI toolkits. Since Bespin is an experiment, this gives them ultimate latitude in experimentation.

The open web depends on the platform, design, and implementation of the editor. So far so good.

Comment by ozten — February 18, 2009

Congrats Ben and Dion this is really really cool!

I like the idea of rendering complex widgets like a text editor using canvas. However I wonder whether the browser’s key events are sufficient to implement a full editor. I’ve already observed that curly braces do not work in FF3 on my Mac. I’ve written the key event layer for qooxdoo and it was a major pain to get roughly consistent behavior between different browsers on different platforms. I’m really interested in how you solved key handling for Bespin.

Comment by fjakobs — February 18, 2009

Ben and Dion, this is cool as hell! And the best part is this is kind of like Simon Cowell singing!

We kind of already know what Flash and SVG can do (although I am interested in hearing why you eliminated them as possibilities). You’re showing what Canvas can do. And the results are shockingly good.

I agree with not supporting IE. This isn’t an end user app, it’s a dev app. You don’t have to worry about plugin saturation. Likewise, let WebKit play catchup on this one :)

Comment by AnM8tR — February 18, 2009

“We kind of already know what Flash and SVG can do (although I am interested in hearing why you eliminated them as possibilities).”
.
Isn’t that quite obvious…?
I think the answer to that question was in the question… ;)

Comment by ThomasHansen — February 18, 2009

Hey guys, thanks for all the feedback and kind words.

I left out some important bits, specifically thoughts on how accessibility and SVG relate to what we did in canvas. I created a second half to the blog post here:

http://benzilla.galbraiths.org/2009/02/18/bespin-and-canvas-part-2bespin-and-canvas-part-2/

Comment by Ben Galbraith — February 18, 2009

Hey all!

Just quickly coded up some proof of concept Flash stuff to show how one would go about making an editor in Flash:

http://oopstudios.com/stuff/dlite/flash_experiments/?dir

Hopefully this makes it’s way to Dion et all

Comment by oopstudios — February 18, 2009

hi everybody, take a look at: http://www.amyeditor.com/

Comment by steida — February 18, 2009

Amy Editor is smart! Nice

Comment by oopstudios — February 18, 2009

Leave a comment

You must be logged in to post a comment.