ME Blog

Gesture recognition in GWT and IT Mill Toolkit: SimpleGesture

2009-04-24T09:43:00.013+03:00

SimpleGesture is a pure GWT (and IT Mill Toolkit) implementation of the mouse gesture recognition method described by Didier Brun at bytearray.org (as I understand it). It allows you to register easy to understand (human readable) gestures, and receive events when these occur. Additionally, SimpleGesture can record new gestures, so that you don't have to write them by hand.

Here is a short demo video - I'm using a Wii (as a mouse, no special integration) to control a simple demo application, and you can see me record a new gesture as well:

You can try the demo application live here: http://marc.virtuallypreinstalled.com/SimpleGesture
The demo application is made using IT Mill Toolkit - SimpleGesture is actually a component for IT Mill Toolkit, but it uses GWT on the client side and the actual GWT widget is made to be stand-alone.

SimpleGesture was made using my "10% time" here at IT Mill (yes, we're only half Google, but that's not half bad, if you ask me...;)

Applications
Mouse gestures have their place, and are not always appropriate - for instance, for regular desktop use you might find that keyboard shortcuts are often better. But if the pointing device is your main input device, or you do not have a keyboard, gestures might be the way to go. The video demo, using the Wii, should demonstrate this quite clearly. Also, hitting a small button in the upper left corner with the Wii controller can be quite a challenge, while the gestures work quite well (hello Fitt's Law!)
I'd imagine there are a few uses for touch -based interfaces as well... ;-)

Technical details
I had used the bytearray mouse gesture library for a Flash project before, and I love how simple the method is, and yet it works very well, so I decided to try to implement it in pure GWT.

The method works by assigning a number depending on which direction the mouse moves in (moving right will produce "0", moving down will produce "2", and so on), and comparing the resulting string of numbers to the registered gestures. The gesture with the lowest Levenshtein distance (and below a set threshold), is considered a match.

For instance, the "rotate clockwise" gesture used in the demo looks like this: "7001"
How is that for simplicity?

SimpleGesture uses the Levenshtein Distance method from Jakarta Commons (getLevenshteinDistance() from StringUtils) on the client-side, without modifications, compiled using GWT. I think this shows one of the strengths of GWT - the ability to make use of existing Java code in the browser.

Use it!
Feel free to use and improve SimpleGesture - it uses the Apache License, just as IT Mill Toolkit and Jakarta Commons. Hopefully you'll send me some patches... :-)

If you're using IT Mill Toolkit, you can get the whole demo project from SVN.

To use it in "plain" GWT, (w/o IT Mill Toolkit), just get the GWT stuff from SVN (or just ISimpleGesture.java). ISimpleGesture a GWT Widget, and there are only two IT Mill Toolkit specific things (which you can remove): "implements Paintable", and the "updateFromUIDL()" -method.

Note that SimpleGesture currently uses EventPreview and listens to the whole window, which is not ideal - event preview is somewhat problematic, and it would be nice to be able to use different gestures in different parts of the window. This is one area that should be improved (patches, anyone?), and it could be made more configurable as well.

Feedback
Please feel free to comment here, discuss at the IT Mill Toolkit forums, or email me at marc.englundÄTitmill.com

iPhone Google Reader on Nokia 5800

2009-02-20T20:15:00.005+02:00

I'm used to the iPhone version of Google Reader on my iPod Touch, so I was quite disappointed when I first tried Google Reader on my Nokia 5800: it gave me the basic stripped down webpage with tiny links. Boo.

Then it dawned on me: the 5800 runs webkit, and Google will probably be happy to serve the iPhone Reader... Simple:

http://www.google.com/reader/i/

Works like a charm!

Searching Trac from Firefox

2008-10-28T09:23:00.004+02:00

Not often, but every once in a while one stumbles across something that's so easy to use, and right under your nose, you wonder how you could have missed it (the reason, of course, is that you assume it can't be that simple, because usually it's not). This is one of those occasions. Please don't laugh...

I just realized how easy it is to add Trac search to the Firefox search engine widget. Yes, it's a not even "a small step for man", but it speeds one task up every-so-litte, which is always nice. And one image should be enough to explain:

UI patterns part 1; ComboBox variations

2008-10-10T14:55:00.008+03:00

I've been accumulating some usage and usability patterns for some time (mostly for IT Mill Toolkit, but the ideas are pretty generic, I think), and it's time to start purging the backlog.
One might say that these are "patterns" in two ways: as a way of using components, and in a broader sense as usability/user interface patterns. Hopefully these examples, although simple, will inspire and show some possibilities you might not have thought of.

Some of the components in IT Mill Toolkit are very versatile, and some possibilities might not be immediately obvious. It's possible to create very (re)usable "patterns" just by configuring a component into a certain mode, adding a little CSS, or just using it in a novel way.

The new "FeatureBrowser" to be included in next version of IT Mill Toolkit will also include patterns, and over all be a much better source for cut-and-paste code than the previous version.

Anyway, first up are a few suggest and inplace-editing -examples, made with a ComboBox and some CSS - no client side gwt coding needed. You can try the examples here.

Let's take a look at the details, one example at a time...
(SORRY Blogger screwed up code formatting - I have no idea how to post code here...)

The first one is a ComboBox: immediate, new items allowed, null selection disallowed.

ComboBox cb = new ComboBox();
cb.setImmediate(true);
cb.setNewItemsAllowed(true);
cb.setNullSelectionAllowed(false);
cb.addStyleName("search");
cb.addListener(new ComboBox.ValueChangeListener() {
public void valueChange(ValueChangeEvent event) {
// do search
}
});

The following is the CSS used to hide the regular ComboBox arrow and make it look like a textfield:

.i-filterselect-search .i-filterselect-button {
background-position: right 0;
width: 5px;
}

The second one is basically the above field, but the entered "tag" is added to the layout before the ComboBox, as a Button that removes itself when clicked:

cb = new ComboBox();
cb.setImmediate(true);
cb.setNewItemsAllowed(true);
cb.setNullSelectionAllowed(false);
cb.addStyleName("search");
cb.addListener(new ComboBox.ValueChangeListener() {
public void valueChange(ValueChangeEvent event) {
Object val = event.getProperty().getValue();
if (val != null) {
Button b = new Button(val + " ⊠",
new Button.ClickListener() {
public void buttonClick(ClickEvent event) {
tokenLayout.removeComponent(event.getButton());
tokenCount--;
}
});
b.setStyleName(Button.STYLE_LINK);
tokenLayout.addComponent(b, tokenCount++);
event.getProperty().setValue(null);
}
}
});

The CSS is the same as in the previous example.

The third example is basically the previous one, vertically oriented and using FILTERINGMODE_CONTAINS instead of the default FILTERINGMODE_STARTSWITH to match anywhere (when entering last name or domain).

[...]
cb.setFilteringMode(ComboBox.FILTERINGMODE_CONTAINS);
[...]

The CSS is still the same as above.

The forth one is a little different; it's styled as a text that turns into a suggesting ComboBox when focused; hovering shows an arrow to indicate the text can be changed.

[...]
cb.setImmediate(true);
cb.setNullSelectionAllowed(false);
cb.setNewItemsAllowed(true);
cb.addStyleName("inplace");
[...]

The CSS is the interesting part; IE requires some special attention too (IE6 actually shows the arrow all the time, since it does not support :hover on divs):

.i-filterselect-inplace {
background: transparent none;
}
.i-filterselect-inplace input {
font-size: 18px;
border: 1px solid none;
}
.i-filterselect-inplace:hover input {
font-style: italic;
}
.i-filterselect-inplace input:focus {
background: #fff url(../default/textfield/img/bg.png) repeat-x;
border: 1px solid #b6b6b6;
border-top-color: #9d9d9d;
border-bottom-color: #d6d6d6;
border-right-color: #d6d6d6;
font-style: inherit;
font-size: inherit;
}
.i-filterselect-inplace .i-filterselect-button {
background: transparent none;
}
* html .i-filterselect-inplace .i-filterselect-button {
/* IE6 does not support :hover on div */
background: #f0f0f0 url(../default/select/img/arrow-down.png) no-repeat center 10px;
}
.i-filterselect-inplace:hover .i-filterselect-button,
.i-filterselect-inplace input:focus + .i-filterselect-button {
background: #f0f0f0 url(../default/select/img/arrow-down.png) no-repeat center 10px;
}
.i-filterselect-inplace input:focus + .i-filterselect-button {
background-color: transparent;
}

cb = new ComboBox();
cb.setWidth("70px");
for (int i=0; i

Again, the CSS is the main attraction - sporting the same IE workarounds:

.i-filterselect-icon {
height: auto;
}
.i-filterselect-icon,
.i-filterselect-icon .i-filterselect-button {
background: transparent none;
}
.i-filterselect-icon input {
visibility: hidden;
}
.i-filterselect-icon:hover .i-filterselect-button,
.i-filterselect-icon input:focus + .i-filterselect-button {
background: #ffffff url(../default/select/img/arrow-down.png) no-repeat center 10px;
border: 1px solid #eeeeee;
margin-top: -25px;
}
* html .i-filterselect-icon .i-filterselect-button {
/* IE6 does not support :hover on div */
background: #ffffff url(../default/select/img/arrow-down.png) no-repeat center 10px;
border: 1px solid #eeeeee;
margin-top: -25px;
}

As you probably noticed, the code samples are not complete; the full source can be found here.
The CSS is complete, though, and that might be all you need - there is nothing particularly complicated going on in javaland...

Press Play on Test with IT Mill TestingTools

2008-03-20T07:50:00.001+02:00

Testing ajax and RIA UIs can be a major ~~pita~~ annoyance, and manual testing is often required. IT Mill TestingTools was released yesterday, and though it does not make all Test Engineers obsolete in one swift blow, it hopefully eases UI testing significantly, leading to more testing and better quality. RIA testing is now point-click-save-run, or as the TestingTools slogan puts it: "Press Play on Test".
I've been working on TestingTools (as well as the IT Mill Toolkit), and thought I'd make a quick video, showing the basic functionality that's great for Joe Tester and me:

(~~excuse the bad quality on youtube, I'll see if I can put the full-sized swf somewhere soon...~~)
Update: it wasn't too hard to get the my recording hosted at IT Mill - it's even featured as a quick tour. Watch the full-size version.

No programming skills required, no more waitForCondition scripting - the boss can do it too.
And it's quick enough to be useful for programmers like me; I can record my own tests to make sure my changes to Label.java do not break HelloWorld - so quickly and easily my attention span can actually handle it. (TT replays tests as quickly as possible, as you can see in the video)
But perhaps the biggest potential benefit is for all those projects that do not have the means or will to set up real testing.
A testing tool for non-testers, if you will.

IT Mill TestingTools does not replace all other testing tools - you'll probably still want to use your favorite unit-tests, performance- and profiling tools for your server-side stuff (we do). But I do think it fills a gap in the toolchain, and hopefully the end result will be more testing and better quality.

A final note: Although TestingTools is especially good for testing IT Mill Toolkit applications (it knows about the components, and is able to wait for ajaxy tasks to complete), you can actually test other sites as well. For security reasons, you must first enable testing in the application/site, mainly to get around browser domain restrictions (no, sorry, you can't test google.com with the online demo) .

Lightboxd - A lightbox with direct links

2008-03-19T21:22:00.002+02:00

Quicklink for the impatient: try Lightboxd.

I just love the "lightbox" -pattern for displaying images, and I've tried quite a few implementations. However, there is one feature I've been missing from all of them (admittedly, I should perhaps have rtfm more in some instances), and that feature is direct linking.

For instance, I might want to send a link saying "here, this is the image I was talking about", or the newlyweds I photographed might want to send me a link saying "this is the shot we want printed big".

Being lazy, I tried complaining first, but since that did not seem to work very well, I decided to add this functionality to my favourite lightbox implementation, Lightbox2 by Lokesh Dhakar (I just happen to think it has a very good 'feel'. That, and it was easy to modify... Lightview is looking good too, but it's license does not allow any modifications or derivate work.) However, I'm fully expecting all lightboxes to follow suit and add support for direct linking by the end of next week... ;-)

Lightboxd is a slightly modified Lightbox2, with added direct linking support. It uses standard HTML anchors, so it kind of works even if javascript is disabled (the page jumps to the anchor, but does not open the linked image).
A direct link looks like this (try it!): http://www.subdoc.com/marc/code/lightboxd/#img1

It's by no means perfect - but it's a 1.0.

And for the record: I'm totally hoping this will make it into the original 'distribution' some day - the world does not really need one more lightbox -implementation ;-)

Go get Lightboxd »

Slice animation in plain JavaScript, minus data-uri

2008-03-11T22:04:00.008+02:00

I made a slightly more dynamic version of Joonas Lehtinen's Slice animation in plain JavaScript.
Joonas' example creates a 3d animation by using data-uris to embed 1px wide image slices in html. It's a fun animation example and an interesting use of data-uris, but data-uris are a bit of a pain to encode (though you could of course automate the process server-side). I wanted to see if I could keep the fun part while making the script a little more dynamic by getting rid of the need to encode data-uris.

The resulting example animates any image, in the browser (no server needed), using plain javascript (only the initialization is changed, the animation code is basically unmodified).

Try it »

I have yet to find a use for this, other than checking how quickly your browser (and computer) slows down when you feed it a bigger image - but I find it interesting nonetheless.

ME Blog is online!

2008-03-10T10:09:00.000+02:00

Jup, I finally configured the blog, and here's the first post. I'm actually new to blogging, so I'm sure there will be some changes once I figure out what's possible.

The main idea with this blog is to post about updates to my stuff around the web (like my main site and my flickr photos), and whenever I contribute to other sites, but I'm sure I'll start posting some stuff exclusively on the blog as well, once I get used to it. Actually, I would not be surprised if the point with separate 'main site' is lost completely at some point, and everything is moved here.

Time will tell...