Onomatopen

I developed a prototype painting system called Onomatopen, which enables the user to switch brushes and apply effects using onomatopoeia. For example, if the user draws a line while saying "Zig-zag Zig-zag...", a jagged line is drawn.

Functions

I have currently implemented 10 features using onomatopoeia.
There are two main uses of onomatopoeia; switching brushes and editing images.

Brush	Edit/Effect
	"Keshi Keshi" Eraser "Choki Choki" Cut "Peta Peta" Paste "Copy Copy" Copy "Biyoon" Deformation

Features

Multimodal Interaction

One of the important features of Onomatopen is the ability to draw lines using the voice. This interaction omits explicit switching operations such as selecting a brush.

Suitable mapping between operation methods and functions

Onomatopoeia is suitable for representing states of objects in a more sensory-based manner. In Onomatopen, users can imagine the texture of a line or executed function more easily than by changing brushes using buttons.

Entertainment

The above features of Onomatopen, such as appealing to sensitivity and visual/auditory feedback, are suited to the entertainment of the user.

Implementation

The voice recognition part uses Julius, which is an open-source voice recognition engine. The Onomatopen system adopts a customized grammer for recognizing repeated onomatopoeia.

Application of Onomatopen

By applying the techniques of the Onomatopen to video game, I developed a game named "Ninja Onomato-kun".

Download

For Windows only
For Windows and Linux
Source code (github.com)