There’s a long time since i’ve blogged about the FlexiTools’2010 workshop and the discussion about a flexible graphic representation made possible in Eclipse graphical editors.

Here we talk about creating an intermediate element which holds a flexible graphic representation, in opposition to it’s canonical one (which in GMF is defined at .gmfgraph model). The users would be able to draw the elements as they wish, and as well be able to create new ones, something like this:

Here is a video explaining how it works, please watch and comment (go to Sketch Forum):

I’ve just submitted a position paper to FlexiTools’2010, in which me and Simone outline our ideas for the Sketch API, which i’m carrying on with Chris and Mariot.
Take a look at the paper:SKETCH: Modeling Using Freehand Drawing in Eclipse Graphical Editors

The main idea is the approach of the modeling task as a twofold process — one of freely sketching models with little interruption from the system; and another more formal, “classic modeling”, recognizing the elements drawn by the user right away, having as output a model in its canonical representation.

By letting users draw the model using their own graphic representation, we are allowing the GEF/GMF editor to be more flexible regarding its visual notation. For instance, assuming a simple model with the elements named flower, sun and cloud – they might be connected and generate an output or might serve as input for other models. A user might draw the flower element, for instance, in infinite ways, having any number of petals, with or without stem, and so on.

This allows the creation of a “graphicless model”, without predefined visual counterparts to the model elements, just elements and relations — the user would choose how elements will look like. That means the .gmfgraph would just hold the canonical representation, but the real one will be the user’s.

 So, on Eclipse GEF/GMF frameworks side, some minor modifications will need to take place:

• The underlying model (Ecore?) will need a generic element to serve as an ‘unrecognized’ element, to be created at the model while the user does not signifies it as anything
• make the editor flexible enough to hold any graphic representation for it’s elements, representing it using an SVG with the sketch. Also, this approach can also be used to make annotations on existing models, since the user might be able to create an “annotation” element.

Those are all feasible, so little adaptation would be needed to plug sketch onto GEF/GMF editors

Don’t you think?

3 weeks ago Sketch API proposal became official. Meanwhile, our job is to foster community around the project.

This project is intended to be applied to any GEF editor, and since that there are so many GEF-based graphical editors out there, our goal now is to try to define a common set of functionalities between them.

We will start to discuss some of the ideas using the Sketch Forum/Newsgroup, please join the discussion and create topics if you are interested on this project or if you know a GEF editor that this project could benefit.

So far, what we have in mind is:

• Gesture (not only shape) recognition – the API would be able to provide a set of configurable gestures, that would translate into commands for moving, resizing, and so on. This feature would massively benefit GEF3D.
• Sketched vs. ‘Beautified’ look – There would be two ways of presenting the diagrams: the Sketched would show the recognized element as the user sketched it (using an SVG Figure created from it); The ‘beautified’ look would be the EditPart’s normal representation.
• Postponed recognition – Maybe to have an Action to trigger the recognition would be better for some domains, instead of always asking the user what he meant to represent like on this demo.

Imagine to have a mockup/wireframe editor built on top of Visual Editor. A prototype builder might be coded using the user’s own drawings. And the prototype might be constructed without a ‘done look’.

Any more ideas?

We are also editing the Sketch proposal’s Wiki so the modifications would be merged onto the official one later. If you are an interested party, please add yourself there.

Pen-based/Tablet technology it’s been around for a long time, but recently this ‘touch-oriented interaction’ has been gaining more attention, with all the buzz around iPad. So if we’re starting to interact to devices with our hands, we are not clicking anymore — perhaps our interfaces should start to be less click-oriented? How can we take advantage of the popularity of those devices? Wouldn’t be nice if we had sketching recognition withing Eclipse?

I think so, I’ve been working on sketch recognition for GEF/GMF editors while creating the MoLIC editor, take a look:

Sketch API (former OmniModeling) on MoLIC Editor from Ugo Sangiorgi on Vimeo.

There are other posts on this blog reporting the evolution until this point — the API is able to recognize shapes the way you draw, learning withing the process if it cant recognize it upfront. It can recognize any shape, really, you will ‘teach’ it how.

Omnimodeling: Multi-Agent Recognition from Ugo Sangiorgi on Vimeo.

So it’s on! I decided to contribute to Eclipse by proposing a Sketch API for GEF/GMF using the former OmniModeling project base, take a look at the proposal we are working on. Mariot Chauvin and Chris Aniszczyk are helping me through, thank you so much guys!

Comments and contributions are welcome

I should acknowledge Simone D.J. Barbosa for the kind support during the OmniModeling project, as well as my colleagues at the Semiotic Engineering Group of PUCRio.