GIGO. After watching the video, who thinks this guy can recommend a script? Take it a step further; who thinks his program can recommend a script?

No seriously, the interface is horrible and incredibly unintuitive.

It may help if you view your users as a spectrum: beginner -> advanced or little time commitment -> major time commitment. Right now, it seems as if you’re on the advanced/major time commitment side of the spectrum, which is great if you need or want that, but I suspect you may find more users at the lower end of the spectrum. That’s just a guess, however; you’ll have to get feedback from your users and run tests. It’s important to note, however, that your UI should contain enough options to target your primary user segment and no more, otherwise it will feel too complicated to that user group. Any more advanced features should be placed in something like an advanced options tab- accessible if needed, but not in the main interface so as to be confusing for novice users. Just some food for thought.

You may want to include priority templates that can be applied with a single click. I’d suspect that you could summarize a large majority of project priorities in only a few templates: web app, high/medium/low traffic websites, Adobe Air app, intranet app, etc. Of course, these would only be base templates which you could further customize as needed, but they could save a lot of time.

If you incorporate those two suggestions, not requiring a user to rate the frameworks and being able to choose a priority template, you would be able to minimize the most simple use of your decision center to two steps: choose a ratings input and choose a priority template.

As for the registration, I really like the StackOverflow model. They don’t require you to sign up and they keep track of you with a cookie. If the cookie gets erased, your work on the site no longer is associated with you. However, if you want your account to be permanent, then you sign up. Picnik, the online photo editor does something similar. Personally, I love this type of model, from the perspective of both a user and a website owner. As a user, I’m not required to make a commitment while still being able to use the product. As a website owner, any users that do sign up are higher quality users and more likely to be potential customers for other similar services. I know they like the product and they use it, so they’re pre-qualified. The casual users aren’t ready yet, so my relationship with them is more long term until they take the next step in commitment.

When you separate your users like that, you can focus on trying to get casual users to make more of a commitment by adding additional value if they sign up. Your copy could read something like, “more features are available when you sign up.” You can also focus on making your committed users repeat customers with other offers or services, for which the copy could read, “If you liked ___, you should also try ___. Here’s why…”

I’d highly recommend letting users try out at least a subset of features before requiring sign up, but that’s entirely up to you.

As a final thought, this is a really good idea and I think it could be very useful, especially as you expand beyond just Ajax frameworks. I see a ton of questions about PHP and other types of frameworks, but you could also add database types, programming languages (ROR vs PHP vs .NET, etc.), server software; the list goes on and it isn’t just limited to software/computer related entities. You could do it for electronics, automobiles, music- anything that could be rated. Once you get all the kinks worked out, you could really expand into some interesting markets. Have fun!

Part of the problem is perception, part reality. On the perception side, your priorities do cascade, so you only have to prioritize at the top level.

On the reality side, you say that you have to rate each of the frameworks yourself and that’s a “huge task”. I agree! Here’s what I can do. What if I could enable you to enter your priorities and then combine them with someone’s elses product ratings such that the product score = their product ratings weighted by your priorities? In addition, you get to view their detailed evaluation data in the grid and export it.

You would be able to switch between the following data sets: 1) your ratings and priorities; 2) the vendor self-ratings and self-descriptions; 3) Ratings of expert Ajaxians; 4) a composite of the athenz.com community; 5) a composite of experts.

Number two is in testing now and looks like this. You’ve prodded me to address three, four, and five. What do you think?

Also, the sign up form is a huge deterrent in its current form and may be a little overkill for the perceived benefit. No sign-up form or lazy registration would be best, but even just an OpenID sign-in or a simpler form that just asks for username, email and password would be better.

These are just my thoughts and suggestions, so take them for what they’re worth. Good luck!

Re the 100+ steps, really it’s only 5 steps, but using the tool does require some work and some thinking, but work and thinking that can produce insight. My most recent work has been with very large companies with complex decision processes. I’m trying to find a balance between a tool for people who need to make quick decisions and a tool for big, team decisions. Agility to scale up and down.

If this tool is so generic, why would you limit the available frameworks to a fixed set (and not let users extend it during creating their evaluation). And why would you add Flex and Silverlight to a list of Ajax Frameworks.

Probably because buzzwords get the ball rolling and some news on ajax magazines … even on fridays …

I myself am bewildered by this. This is a generic website to help people make decisions on things?

Seems like this is something that will be incredibly brittle and difficult to maintain. No worse (in terms of timeliness) than a blog on the subject, I suppose.

I really like Jims tool :) I’m a vendor of a js-lib, but didn’t find the possibility to add my lib to the options, where can I do this?