There’s a long time since i’ve blogged about the FlexiTools’2010 workshop and the discussion about a flexible graphic representation made possible in Eclipse graphical editors.

Here we talk about creating an intermediate element which holds a flexible graphic representation, in opposition to it’s canonical one (which in GMF is defined at .gmfgraph model). The users would be able to draw the elements as they wish, and as well be able to create new ones, something like this:

Here is a video explaining how it works, please watch and comment (go to Sketch Forum):

I’ve just submitted a position paper to FlexiTools’2010, in which me and Simone outline our ideas for the Sketch API, which i’m carrying on with Chris and Mariot.
Take a look at the paper:SKETCH: Modeling Using Freehand Drawing in Eclipse Graphical Editors

The main idea is the approach of the modeling task as a twofold process — one of freely sketching models with little interruption from the system; and another more formal, “classic modeling”, recognizing the elements drawn by the user right away, having as output a model in its canonical representation.

By letting users draw the model using their own graphic representation, we are allowing the GEF/GMF editor to be more flexible regarding its visual notation. For instance, assuming a simple model with the elements named flower, sun and cloud – they might be connected and generate an output or might serve as input for other models. A user might draw the flower element, for instance, in infinite ways, having any number of petals, with or without stem, and so on.

This allows the creation of a “graphicless model”, without predefined visual counterparts to the model elements, just elements and relations — the user would choose how elements will look like. That means the .gmfgraph would just hold the canonical representation, but the real one will be the user’s.

 So, on Eclipse GEF/GMF frameworks side, some minor modifications will need to take place:

• The underlying model (Ecore?) will need a generic element to serve as an ‘unrecognized’ element, to be created at the model while the user does not signifies it as anything
• make the editor flexible enough to hold any graphic representation for it’s elements, representing it using an SVG with the sketch. Also, this approach can also be used to make annotations on existing models, since the user might be able to create an “annotation” element.

Those are all feasible, so little adaptation would be needed to plug sketch onto GEF/GMF editors

Don’t you think?

3 weeks ago Sketch API proposal became official. Meanwhile, our job is to foster community around the project.

This project is intended to be applied to any GEF editor, and since that there are so many GEF-based graphical editors out there, our goal now is to try to define a common set of functionalities between them.

We will start to discuss some of the ideas using the Sketch Forum/Newsgroup, please join the discussion and create topics if you are interested on this project or if you know a GEF editor that this project could benefit.

So far, what we have in mind is:

• Gesture (not only shape) recognition – the API would be able to provide a set of configurable gestures, that would translate into commands for moving, resizing, and so on. This feature would massively benefit GEF3D.
• Sketched vs. ‘Beautified’ look – There would be two ways of presenting the diagrams: the Sketched would show the recognized element as the user sketched it (using an SVG Figure created from it); The ‘beautified’ look would be the EditPart’s normal representation.
• Postponed recognition – Maybe to have an Action to trigger the recognition would be better for some domains, instead of always asking the user what he meant to represent like on this demo.

Imagine to have a mockup/wireframe editor built on top of Visual Editor. A prototype builder might be coded using the user’s own drawings. And the prototype might be constructed without a ‘done look’.

Any more ideas?

We are also editing the Sketch proposal’s Wiki so the modifications would be merged onto the official one later. If you are an interested party, please add yourself there.

Pen-based/Tablet technology it’s been around for a long time, but recently this ‘touch-oriented interaction’ has been gaining more attention, with all the buzz around iPad. So if we’re starting to interact to devices with our hands, we are not clicking anymore — perhaps our interfaces should start to be less click-oriented? How can we take advantage of the popularity of those devices? Wouldn’t be nice if we had sketching recognition withing Eclipse?

I think so, I’ve been working on sketch recognition for GEF/GMF editors while creating the MoLIC editor, take a look:

Sketch API (former OmniModeling) on MoLIC Editor from Ugo Sangiorgi on Vimeo.

There are other posts on this blog reporting the evolution until this point — the API is able to recognize shapes the way you draw, learning withing the process if it cant recognize it upfront. It can recognize any shape, really, you will ‘teach’ it how.

Omnimodeling: Multi-Agent Recognition from Ugo Sangiorgi on Vimeo.

So it’s on! I decided to contribute to Eclipse by proposing a Sketch API for GEF/GMF using the former OmniModeling project base, take a look at the proposal we are working on. Mariot Chauvin and Chris Aniszczyk are helping me through, thank you so much guys!

Comments and contributions are welcome

I should acknowledge Simone D.J. Barbosa for the kind support during the OmniModeling project, as well as my colleagues at the Semiotic Engineering Group of PUCRio.

I’ve just bought a 13.3 inch HP dv3, model 1075us with AMD Turion(tm) X2 Dual-Core Mobile RM-72 and after some configuration i’m now able to run Ubuntu 9.04 on it. I decided to write here what i needed to do, in case of anybody facing the same issues i did
First, here is all my hardware, if you have the same, then the solutions might fit for you.
My lspci:

00:00.0 Host bridge: Advanced Micro Devices [AMD] RS780 Host Bridge
00:01.0 PCI bridge: Hewlett-Packard Company Device 9602
00:04.0 PCI bridge: Advanced Micro Devices [AMD] RS780 PCI to PCI bridge (PCIE port 0)
00:06.0 PCI bridge: Advanced Micro Devices [AMD] RS780 PCI to PCI bridge (PCIE port 2)
00:07.0 PCI bridge: Advanced Micro Devices [AMD] RS780 PCI to PCI bridge (PCIE port 3)
00:11.0 SATA controller: ATI Technologies Inc SB700/SB800 SATA Controller [AHCI mode]
00:12.0 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI0 Controller
00:12.1 USB Controller: ATI Technologies Inc SB700 USB OHCI1 Controller
00:12.2 USB Controller: ATI Technologies Inc SB700/SB800 USB EHCI Controller
00:13.0 USB Controller: ATI Technologies Inc SB700/SB800 USB OHCI0 Controller
00:13.1 USB Controller: ATI Technologies Inc SB700 USB OHCI1 Controller
00:13.2 USB Controller: ATI Technologies Inc SB700/SB800 USB EHCI Controller
00:14.0 SMBus: ATI Technologies Inc SBx00 SMBus Controller (rev 3a)
00:14.1 IDE interface: ATI Technologies Inc SB700/SB800 IDE Controller
00:14.2 Audio device: ATI Technologies Inc SBx00 Azalia (Intel HDA)
00:14.3 ISA bridge: ATI Technologies Inc SB700/SB800 LPC host controller
00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge
00:18.0 Host bridge: Advanced Micro Devices [AMD] Family 11h HyperTransport Configuration (rev 40)
00:18.1 Host bridge: Advanced Micro Devices [AMD] Family 11h Address Map
00:18.2 Host bridge: Advanced Micro Devices [AMD] Family 11h DRAM Controller
00:18.3 Host bridge: Advanced Micro Devices [AMD] Family 11h Miscellaneous Control
00:18.4 Host bridge: Advanced Micro Devices [AMD] Family 11h Link Control
01:05.0 VGA compatible controller: ATI Technologies Inc RS780M/RS780MN [Radeon HD 3200 Graphics]
01:05.1 Audio device: ATI Technologies Inc RS780 Azalia controller
08:00.0 Network controller: Broadcom Corporation BCM4322 802.11a/b/g/n Wireless LAN Controller (rev 01)
09:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 02)

Issue #1 No sound (ATI Technologies Inc RS780 Azalia controller)
I got no sound at all, but the solution was easy, just do:

gksudo gedit /etc/modprobe.d/alsa-base.conf

and append this at the end:

options snd slots=snd-hda-intel,snd-hda-intel
options snd-hda-intel model=dell-m4-2
options snd-hda-intel enable_msi=1
alias snd-card-1 snd-hda-intel
alias snd-card-0 snd-hda-intel

save, reset and enjoy.

Issue #2 Broadcomm’s shitty WiFi (Broadcom Corporation BCM4322 802.11a/b/g/n Wireless LAN Controller (rev 01))

It was really not an issue, I’ve saw many people complaining about the broadcomms wifi, but I’ve just enabled the restricted driver and it works fine for me.

Issue #3 ATI acceleration with fglrx
I’m a compiz user, i cant help it! Nevermind the fire effects and all eye candy, what i really find useful is the zooming function, the window drag-and-drop between multiple desktops and the ability to assign a command to any key combination.
For example, I use to hit F12 to turn my screen off, whenever i need to go away from my laptop (to sleep, for example =P) thus i installed the restricted drivers and compizconfig-backend-gconf + compizconfig-settings-manager and assigned F12 to

/usr/bin/xset dpms force off

But the screen didnt completely went off, the backlight was still on. So i installed the latest Catalyst 9.5 driver doing this steps http://wiki.cchtml.com/index.php/Ubuntu_Jaunty_Installation_Guide and now it works.

The fingerprint authenticator(with fprint), the media card reader and the HP webcam works flawlessly.

Thats all for now.
(I will update this post as I find more compatibility issues between HP dv3 and Ubuntu)

When studying some topic I usually take notes to remember things more easily. Sometimes this notes are just drawings, and that was just the case for The Communicability Evaluation Method, or CEM, a method for HCI evaluation based on Semiotic Engineering. It consists on three steps. (1) tagging recorded interaction videos, (2) map the tags onto HCI problems, and (3) where the overall message, which cames through system, from the designer to the user is inspected, this last step is called semiotic profiling.
Well, for the first step there is a table with 13 tags related to what the user would be saying, like “OOPS” when he does something he’s not intented to do, and go to Undo.
I tought it would be funny to draw faces for all the tags and put them into a poster, or a A4 paper so everyone can print and put it on the wall. Enjoy!

Download the vectorial source (SVG) and faces as individual images here

When using any of those, please include the following information somewhere visible:

Communicability Evaluation Method Tags by Ugo Sangiorgi is licensed under a Creative Commons Attribution 3.0 Unported License.
Based on a work at blog.ugosan.com.

First, here is a video showing everything working

TabletPC running Ubuntu Intrepid + Compiz from Ugo Sangiorgi on Vimeo.

Since i’d started with OmniModeling project my advisor borrowed me a TabletPC, it’s a Compaq2710 and it had windows installed by default. I will not explain here the reasons why i dont use windows for a long time now, but I struggled for one week trying to get used to work with it because, well, the pc its not mine, I couldnt mess with the partition table installing Linux. And i didnt knew if X11 supported tablet pcs or not after all.
So i installed Wubi, picked a username and Ubuntu got up and running like a charm, using a big file on windows as hard drive. I didnt even needed a cd image, Wubi downloaded it for me. It’s wonderful. If you want to show Linux to someone who uses windows, do it with Wubi.

But what about the stylus pen? it was not recognized right away. After some search, here is what I did:

1.Install the wacom driver using this script, just execute it

chmod +x install_linuxwacom./install_linuxwacom

2.Then gksudo gedit /etc/X11/xorg.conf and append this at the end of the file:

Section “InputDevice”
Driver “wacom”
Identifier “stylus”
Option “Device” “/dev/ttyS0″ # SERIAL ONLY
Option “Type” “stylus”
Option “ForceDevice” “ISDV4″ # Tablet PC ONLY
EndSection

Section “InputDevice”
Driver “wacom”
Identifier “eraser”
Option “Device” “/dev/ttyS0″ # SERIAL ONLY
Option “Type” “eraser”
Option “ForceDevice” “ISDV4″ # Tablet PC ONLY
EndSection

Section “InputDevice”
Driver “wacom”
Identifier “cursor”
Option “Device” “/dev/ttyS0″ # SERIAL ONLY
Option “Type” “cursor”
Option “ForceDevice” “ISDV4″ # Tablet PC ONLY
EndSection

Section “InputDevice”
Driver “wacom”
Identifier “pad”
Option “Device” “/dev/ttyS0″ # SERIAL ONLY
Option “Type” “pad”
EndSection

# Uncomment the following section if you you have a TabletPC that supports touch
# Section “InputDevice”
# Driver “wacom”
# Identifier “touch”
# Option “Device” “/dev/ttyS0″ # SERIAL ONLY
# Option “Device” “/dev/input/wacom” # USB ONLY
# Option “Type” “touch”
# Option “ForceDevice” “ISDV4″ # Serial Tablet PC ONLY
# Option “USB” “on” # USB ONLY
# EndSection

Section “ServerLayout”
Identifier “Default Layout”
Screen “Default Screen”
InputDevice “stylus” “SendCoreEvents”
InputDevice “eraser” “SendCoreEvents”
InputDevice “pad” # For Intuos3/CintiqV5/Graphire4/Bamboo tablets
EndSection

3. Restart X11 (ctrl+alt+backspace).

Now you can run wacomcpl to calibrate the pen, and fine tune some parameters such as sensivity.
(I like to use the middle button of my pen as the third button of the mouse, you can configure this on wacomcpl, but for some reason the configuration is lost between boots..)

Another application you will probably find useful is CellWriter, install it (sudo apt-get install cellwriter)
And finally, activating the eye-candy effects with Compiz (System – Preferences – Appearance – Visual Effects) you can have on-screen annotation (ctrl+alt to annotate and ctrl+alt+k to clear) and other useful things such as the zoom feature.
Compiz can be configurable installing compizconfig-settings-manager package ( sudo apt-get install compizconfig-settings-manager )

Thats All,
Please comment on the post =)

I’m making some progress implementing a Multi-Agent recognition system for OmniModeling. Right now it’s able to recognize multiple strokes as a gesture, so the user is able to draw dashed connections and composite figures as well:

Omnimodeling: Multi-Agent Recognition from Ugo Sangiorgi on Vimeo.

From here: Eclipse community members are invited to display their Photoshop and Gimp skills by taking a still frame from a movie or TV show, and replacing the MovieOS with a screenshot of an Eclipse IDE running plugins, an RCP or SWT application, or even a mobile OSGi application. The top 5 entries will receive Eclipse wear. Submit your entry by attaching it to bug 252135 by November 17 at 4:00 pm EST.
I’d sent two images so far, from “2001: a Space Odissey” (made with GIMP):

HAL and Dr. David Bowman discussing a GMF model
HAL sees Java Perspective:

Here is a video showing the current development status. For that I used the Levenshtein algorithm partly inspired by the approach proposed on this article [1].

This is a normal GMF generated GEF editor with Square, Triangle and Connection tools.
I’d added another tool called Sketch on the palette, so at this point I’m able to feedback to the user the sketch being performed, and detect what is meant by comparing the sketch with a using the Levenshtein algorithm for string distance. I’m also working on a Multi-agent approach for both recognition and learning of sketches.

Omnimodeling for GMF (in progress…) from Ugo Sangiorgi on Vimeo.

So in a rough way, GMF provides the ability for a user to create/edit/view a XML model containing Squares and Triangles (in this case) but in a graphic way. What OmniModeling tries to accomplish is to add another layer on top of this funcionality and let the user draw the model freely, by hand.

The next step is not only to recognize the shape, but grasp the size of the element being added (by comparing distance between corners maybe) and set it automatically.

.