publications

2025

1. Nature

   

   A Generic Non-Invasive Neuromotor Interface for Human-Computer Interaction

   Patrick Kaifosh, Thomas R. Reardon, and CTRL-Labs at Reality Labs (243 contributors including Bruno De Araujo)

   Nature, Jul 2025

   Abs DOI

   Since the advent of computing, humans have sought computer input technologies that are expressive, intuitive and universal. While diverse modalities have been developed, including keyboards, mice and touchscreens, they require interaction with a device that can be limiting, especially in on-the-go scenarios. Gesture-based systems use cameras or inertial sensors to avoid an intermediary device, but tend to perform well only for unobscured movements. By contrast, brain–computer or neuromotor interfaces that directly interface with the body’s electrical signalling have been imagined to solve the interface problem1, but high-bandwidth communication has been demonstrated only using invasive interfaces with bespoke decoders designed for single individuals2,3,4. Here, we describe the development of a generic non-invasive neuromotor interface that enables computer input decoded from surface electromyography (sEMG). We developed a highly sensitive, easily donned sEMG wristband and a scalable infrastructure for collecting training data from thousands of consenting participants. Together, these data enabled us to develop generic sEMG decoding models that generalize across people. Test users demonstrate a closed-loop median performance of gesture decoding of 0.66 target acquisitions per second in a continuous navigation task, 0.88 gesture detections per second in a discrete-gesture task and handwriting at 20.9 words per minute. We demonstrate that the decoding performance of handwriting models can be further improved by 16% by personalizing sEMG decoding models. To our knowledge, this is the first high-bandwidth neuromotor interface with performant out-of-the-box generalization across people.

2022

1. TEI

   

   Sensing Hand Interactions with Everyday Objects by Profiling Wrist Topography

   Julius Cosmo Romeo Rudolph, David Holman, Bruno De Araujo, and 3 more authors

   In Proceedings of the Sixteenth International Conference on Tangible, Embedded, and Embodied Interaction, Daejeon, Republic of Korea, Jul 2022

   Abs DOI Video

   We demonstrate rich inferences about unaugmented everyday objects and hand object interactions by measuring minute skin surface deformations at the wrist using a sensing technique based on capacitance. The wristband prototype infers muscle and tendon tension, pose, and motion, which we then map to force (9 users, 13.66 +/- 9.84 N regression error on classes 0–49.1 N), grasp (9 users, 81 +/- 7 % classification accuracy on 6 grasps), and continuous interaction (10 users, 99 +/- 1 % discrimination accuracy between 6 interactions, 89–97 % accuracy on 3 states within each interaction) using basic machine learning models. We wrapped these sensing capabilities into a proof-of-concept end-to-end system, Ubiquitous Controls, that enables virtual range inputs by sensing continuous interactions with unaugmented objects. Eight users leveraged our system to control UI widgets (like sliders and dials) with object interactions (like “cutting with scissors’’ and “squeezing a ball”). Finally, we discuss the implications and opportunities of using hands as a ubiquitous sensor of our surroundings.

2018

1. CHI

   

   DataInk: Direct and Creative Data-Oriented Drawing

   Haijun Xia, Nathalie Henry Riche, Fanny Chevalier, and 2 more authors

   In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, Montreal QC, Canada, Jul 2018

   Abs DOI Video

   Creating whimsical, personal data visualizations remains a challenge due to a lack of tools that enable for creative visual expression while providing support to bind graphical content to data. Many data analysis and visualization creation tools target the quick generation of visual representations, but lack the functionality necessary for graphics design. Toolkits and charting libraries offer more expressive power, but require expert programming skills to achieve custom designs. In contrast, sketching affords fluid experimentation with visual shapes and layouts in a free-form manner, but requires one to manually draw every single data point. We aim to bridge the gap between these extremes. We propose DataInk, a system supports the creation of expressive data visualizations with rigorous direct manipulation via direct pen and touch input. Leveraging our commonly held skills, coupled with a novel graphical user interface, DataInk enables direct, fluid, and flexible authoring of creative data visualizations.

2017

1. CHI

   Collection Objects: Enabling Fluid Formation and Manipulation of Aggregate Selections

   Haijun Xia, Bruno De Araujo, and Daniel Wigdor

   In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, Colorado, USA, Jul 2017

   Abs DOI

   Despite the long development of Graphical User Interfaces, working with multiple graphical objects remains a challenge, due to the difficulties of forming complex selections, ambiguities of operations, and tediousness of repetitively unselect-reselect or ungroup-regroup objects. Instead of tackling them as individual problems, we attribute it to the lack of system support to the general selection-action cycles. We propose Collection Objects to not only support a single fast selection-action cycle but also allow multiple cycles to be chained together into a fluid workflow. Collection Objects unifies selection, grouping, and manipulation of aggregate selections into a single object, with which selection can be composed with various techniques, modified for later actions, grouped with objects inside still directly accessible, and quasi-moded for less context switching. We implemented Collection Object in the context of a vector drawing application with simultaneous pen and touch input. Results of an expert evaluation show that Collection Objects holds considerable promises for fluid interaction with multiple objects.

2. CHI

   

   GhostID: Enabling Non-Persistent User Differentiation in Frequency-Division Capacitive Multi-Touch Sensors

   Sidharth Sahdev, Clifton Forlines, Ricardo Jota, and 6 more authors

   In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, Colorado, USA, Jul 2017

   Abs DOI Video

   Current touch devices are adept at tracking finger touches, but cannot distinguish if multiple touches are caused by different fingers on a single hand, by fingers from both hands of a single user, or by different users. This limitation significantly reduces the possibilities for interaction techniques in touch interfaces. We present GhostID, a capacitive sensor that can differentiate the origins of multiple simultaneous touches. Our approach analyzes the signal ghosting, already present as an artifact in a frequency-division touch controller, to differentiate touches from the same hand or different hands of a single user (77% reliability at 60 fps) or from two different users (95% reliability at 60 fps). In addition to GhostID, we also develop a framework of user-differentiation capabilities for touch input devices, and illustrate a set of interaction techniques enabled by GhostID.

2016

1. UIST

   

   Next-Point Prediction Metrics for Perceived Spatial Errors

   Mathieu Nancel, Daniel Vogel, Bruno De Araujo, and 2 more authors

   In Proceedings of the 29th Annual Symposium on User Interface Software and Technology, Tokyo, Japan, Jul 2016

   Abs DOI Video

   Touch screens have a delay between user input and corresponding visual interface feedback, called input ’latency’ (or ’lag’). Visual latency is more noticeable during continuous input actions like dragging, so methods to display feedback based on the most likely path for the next few input points have been described in research papers and patents. Designing these ’next-point prediction’ methods is challenging, and there have been no standard metrics to compare different approaches. We introduce metrics to quantify the probability of 7 spatial error ’side-effects’ caused by next-point prediction methods. Types of side-effects are derived using a thematic analysis of comments gathered in a 12 participants study covering drawing, dragging, and panning tasks using 5 state-of-the-art next-point predictors. Using experiment logs of actual and predicted input points, we develop quantitative metrics that correlate positively with the frequency of perceived side-effects. These metrics enable practitioners to compare next-point predictors using only input logs.

2. UIST

   Multi-Device Storyboards for Cinematic Narratives in VR

   Rorik Henrikson, Bruno De Araujo, Fanny Chevalier, and 2 more authors

   In Proceedings of the 29th Annual Symposium on User Interface Software and Technology, Tokyo, Japan, Jul 2016

   Abs DOI

   Virtual Reality (VR) narratives have the unprecedented potential to connect with an audience through presence, placing viewers within the narrative. The onset of consumer VR has resulted in an explosion of interest in immersive storytelling. Planning narratives for VR, however, is a grand challenge due to its unique affordances, its evolving cinematic vocabulary, and most importantly the lack of supporting tools to explore the creative process in VR.In this paper, we distill key considerations with the planning process for VR stories, collected through a formative study conducted with film industry professionals. Based on these insights we propose a workflow, specific to the needs of professionals creating storyboards for VR film, and present a multi-device (tablet and head-mounted display) storyboard tool supporting this workflow. We discuss our design and report on feedback received from interviews following demonstration of our tool to VR film professionals.

3. CHI

   

   Hammer Time! A Low-Cost, High Precision, High Accuracy Tool to Measure the Latency of Touchscreen Devices

   Jonathan Deber, Bruno De Araujo, Ricardo Jota, and 4 more authors

   In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, California, USA, Jul 2016

   Abs DOI Video

   We report on the Latency Hammer, a low-cost yet highaccuracy and high-precision automated tool that measures the interface latency of touchscreen devices. The Hammer directly measures latency by triggering a capacitive touch event on a device using an electrically actuated touch simulator, and a photo sensor to monitor the screen for a visual response. This allows us to measure the full end-toend latency of a touchscreen system exactly as it would be experienced by a user. The Hammer does not require human interaction to perform a measurement, enabling the acquisition of large datasets. We present the operating principles of the Hammer, and discuss its design and construction; full design documents are available online. We also present a series of tools and equipment that were built to assess and validate the performance of the Hammer, and demonstrate that it provides reliable latency measurements.

4. CHI

   

   Storeoboard: Sketching Stereoscopic Storyboards

   Rorik Henrikson, Bruno De Araujo, Fanny Chevalier, and 2 more authors

   In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, California, USA, Jul 2016

   Abs DOI Video

   We present Storeoboard, a system for stereo-cinematic conceptualization, via storyboard sketching directly in stereo. The resurgence of stereoscopic media has motivated filmmakers to evolve a new stereo-cinematic vocabulary, as many principles for stereo 3D film are unique. Concepts like plane separation, parallax position, and depth budgets are missing from early planning due to the 2D nature of existing storyboards. Storeoboard is the first of its kind, allowing filmmakers to explore, experiment and conceptualize ideas in stereo early in the film pipeline, develop new stereo-cinematic constructs and foresee potential difficulties. Storeoboard is the design outcome of interviews and field work with directors, stereographers, and storyboard artists. We present our design guidelines and implementation of a tool combining stereo-sketching, depth manipulations and storyboard features into a coherent and novel workflow. We report on feedback from storyboard artists, industry professionals and the director of a live action, feature film on which Storeoboard was deployed.

5. CHI

   

   Object-Oriented Drawing

   Haijun Xia, Bruno De Araujo, Tovi Grossman, and 1 more author

   In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, California, USA, Jul 2016

   Abs DOI

   We present Object-Oriented Drawing, which replaces most WIMP UI with Attribute Objects. Attribute Objects embody the attributes of digital content as UI objects that can be manipulated through direct touch gestures. In the paper, the fundamental UI concepts are presented, including Attribute Objects, which may be moved, cloned, linked, and freely associated with drawing objects. Other functionalities, such as attribute-level blending and undo, are also demonstrated. We developed a drawing application based on the presented concepts with simultaneous touch and pen input. An expert assessment of our application shows that direct physical manipulation of Attribute Objects enables a user to quickly perform interactions which were previously tedious, or even impossible, with a coherent and consistent interaction experience throughout the entire interface.

6. TEI

   

   Snake Charmer: Physically Enabling Virtual Objects

   Bruno De Araujo, Ricardo Jota, Varun Perumal, and 3 more authors

   In Proceedings of the TEI ’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction, Eindhoven, Netherlands, Jul 2016

   Abs DOI Video

   Augmented and virtual reality have the potential of being indistinguishable from the real world. Holographic displays, including head mounted units, support this vision by creating rich stereoscopic scenes, with objects that appear to float in thin air - often within arm’s reach. However, one has but to reach out and grasp nothing but air to destroy the suspension of disbelief. Snake-charmer is an attempt to provide physical form to virtual objects by revisiting the concept of Robotic Graphics or Encountered-type Haptic interfaces with current commodity hardware. By means of a robotic arm, Snake-charmer brings physicality to a virtual scene and explores what it means to truly interact with an object. We go beyond texture and position simulation and explore what it means to have a physical presence inside a virtual scene. We demonstrate how to render surface characteristics beyond texture and position, including temperature; how to physically move objects; and how objects can physically interact with the user’s hand. We analyze our implementation, present the performance characteristics, and provide guidance for the construction of future physical renderers.

2015

1. ACM Survey

   A Survey on Implicit Surface Polygonization

   B. De Araujo, Daniel S. Lopes, Pauline Jepp, and 2 more authors

   ACM Comput. Surv., May 2015

   Abs DOI

   Implicit surfaces (IS) are commonly used in image creation, modeling environments, modeling objects, and scientific data visualization. In this article, we present a survey of different techniques for fast visualization of IS. The main classes of visualization algorithms are identified along with the advantages of each in the context of the different types of IS commonly used in computer graphics. We focus closely on polygonization methods, as they are the most suited to fast visualization. Classification and comparison of existing approaches are presented using criteria extracted from current research. This enables the identification of the best strategies according to the number of specific requirements, such as speed, accuracy, quality, or stylization.

2014

1. JRTIP

   3D modelling of laser scanned and photogrammetric data for digital documentation: the Mosteiro da Batalha case study

   Adriano Oliveira, João F. Oliveira, João M. Pereira, and 2 more authors

   J. Real-Time Image Process., Dec 2014

   Abs DOI

   Advances in both terrestrial laser scanning hardware and photogrammetric systems combined are creating increasingly precise and rich 3D coloured data. In this article we show how computer graphics and visualization techniques have played an important role in real-time visualization, data management, modelling, and data fusion in an increasing number of applications such as surveying engineering, structure analysis, architecture, archaeology and cultural heritage. Specifically, we describe the typical modelling steps involved in the creation of a range of digital documents provided by the 3D digitization company Artescan to customers. We present how these modelling steps were applied in the context of creating digital documents used in the preservation of Mosteiro da Batalha.

2. SUI

   Exploring tablet surrounding interaction spaces for medical imaging

   Hanaë Rateau, Laurent Grisoni, and Bruno De Araujo

   In Proceedings of the 2nd ACM Symposium on Spatial User Interaction, Honolulu, Hawaii, USA, Dec 2014

   Abs DOI

   Medical imaging is essential to support most diagnosis. It often requires visualizing individual 2D slices from 3D volumetric datasets and switching between both representations. Combining an overview with a detailed view of the data [1] enables to keep the user in context when looking in detail at a slice. Given both their mobility and their adequacy to support direct manipulation, tablets are attractive devices to ease imaging analysis tasks. They have been successfully combined with tabletops [3], allowing new ways to explore volumetric data. However, while touch allows for a more direct manipulation, it suffers from the well-known fat fnger problem which can interfere with the display, making it hard to understand subtle visual changes. To overcome this problem, we propose to explore the space around tablet devices. Such approach has been used for displays [2] to separate several workspaces of the desktop. Here, we use such space to invoke commands that are not required to be performed on the tablet, thus maximizing the visualization space during manipulations.

3. Interaction events in contactless gestural systems: from motion to interaction

   Nicolas Bremard, Laurent Grisoni, and Bruno De Araujo

   In Proceedings of the 2014 International Workshop on Movement and Computing, Paris, France, Dec 2014

   Abs DOI

   Contactless interactive systems appear to have an interesting context of application in interactive art, however practical example often involve low interactivity; we think this is due to the fact that this interaction paradigm is still poorly understood, and tools for interaction design are still missing. We present a simple interaction technique (Point-and-Move) that allows to point-and-click without instrument through distance; through this example, we show that the presented concept and framework has nice and flexible properties. Finally, we illustrate the approach on the two artistic installation we participated to, and also provide uncontrolled user experience feedback on the presented interaction technique, from these installations.

4. IUI

   Mimetic interaction spaces: controlling distant displays in pervasive environments

   Hanae Rateau, Laurent Grisoni, and Bruno De Araujo

   In Proceedings of the 19th International Conference on Intelligent User Interfaces, Haifa, Israel, Dec 2014

   Abs DOI

   Pervasive computing is a vision that has been an inspiring long-term target for many years now. Interaction techniques that allow one user to efficiently control many screens, or that allow several users to collaborate on one distant screen, are still hot topics, and are often considered as two different questions. Standard approaches require a strong coupling between the physical location of input device, and users. We propose to consider these two questions through the same basic concept, that uncouples physical location and user input, using a mid-air approach. We present the concept of mimetic interaction spaces (MIS), a dynamic user-definition of an imaginary input space thanks to an iconic gesture, that can be used to define mid-air interaction techniques. We describe a participative design user-study, that shows this technique has interesting acceptability and elicit some definition and deletion gestures. We finally describe a design space for MIS-based interaction, and show how such concept may be used for multi-screen control, as well as screen sharing in pervasive environments.

2013

1. ITS

   Interactive surfaces for interaction with stereoscopic 3d (ISIS3D): tutorial and workshop at its 2013

   Florian Daiber, Bruno Rodrigues De Araujo, Frank Steinicke, and 1 more author

   In Proceedings of the 2013 ACM International Conference on Interactive Tabletops and Surfaces, St. Andrews, Scotland, United Kingdom, Dec 2013

   Abs DOI

   With the increasing distribution of multi-touch capable devices multi-touch interaction becomes more and more ubiquitous. Multi-touch interaction offers new ways to deal with 3D data allowing a high degree of freedom (DOF) without instrumenting the user. Due to the advances in 3D technologies, designing for 3D interaction is now more relevant than ever. With more powerful engines and high resolution screens also mobile devices can run advanced 3D graphics, 3D UIs are emerging beyond the game industry, and recently, first prototypes as well as commercial systems bringing (auto-) stereoscopic display on touch-sensitive surfaces have been proposed. With the Tutorial and Workshop on “Interactive Surfaces for Interaction with Stereoscopic 3D (ISIS3D)” we aim to provide an interactive forum that focuses on the challenges that appear when the flat digital world of surface computing meets the curved, physical, 3D space we live in.

2. C&G

   Special Section on Touching the 3rd Dimension: Mockup Builder: 3D modeling on and above the surface

   Bruno R. De Araujo, GéRy Casiez, Joaquim A. Jorge, and 1 more author

   Comput. Graph., May 2013

   Abs DOI

   We present Mockup Builder, a semi-immersive environment for conceptual design which allows virtual mockups to be created using gestures. Our goal is to provide familiar ways for people to conceive, create and manipulate three-dimensional shapes. To this end, we developed on-and-above-the-surface interaction techniques based on asymmetric bimanual interaction for creating and editing 3D models in a stereoscopic environment. Our approach combines both hand and finger tracking in the space on and above a multi-touch surface. This combination brings forth an alternative design environment where users can seamlessly switch between interacting on the surface or above it to leverage the benefit of both interaction spaces. A formal user evaluation conducted with experienced users shows very promising avenues for further work towards providing an alternative to current user interfaces for modeling.

3. Building Projection-Based VR Systems

   Joaquim Armando Pires Jorge, Bruno Rodrigues De Araujo, Luciano Pereira Soares, and 3 more authors

   May 2013

   Abs

   Among virtual reality technologies, a multi-projection display is certainly the most attractive and challenging. Engineers and researchersin any number of fieldsare increasingly attracted by immersive multi-projection displays. However, building a multi-projection system is a complex integration task, challenging even VR specialists. This book provides an introduction to the issues to consider when planning the installation of a multi-projection environment for researches and professionals not only in the computer graphics and virtual reality field but also in any other area that wants to use multi-projection displays. The book takes a practical approach, offering guidance in how to create a multi-projection environment. Sections: Evaluation of available display technologies Details of setting up a variety of display infrastructures Options for image generation, from cheapest to most expensive Methods for tracking users in an immersive projection environment Human-computer interaction, with an emphasis on multimodal interaction Audio: how to create a realistic auditory display in terms of sound synthesis and propagation How to utilize virtual reality software for immersive environments Case studies of VR environments the authors have constructed

2012

1. GI

   

   Mockup builder: direct 3D modeling on and above the surface in a continuous interaction space

   Bruno De Araujo, Géry Casiez, and Joaquim A. Jorge

   In Proceedings of Graphics Interface 2012, Toronto, Ontario, Canada, May 2012

   Abs Video

   Our work introduces a semi-immersive environment for conceptual design where virtual mockups are obtained from gestures we aim to get closer to the way people conceive, create and manipulate three-dimensional shapes. We present on-and-above-the-surface interaction techniques following Guiard’s asymmetric bimanual model to take advantage of the continuous interaction space for creating and editing 3D models in a stereoscopic environment. To allow for more expressive interactions, our approach continuously combines hand and finger tracking in the space above the table with multi-touch on its surface. This combination brings forth an alternative design environment where users can seamlessly switch between interacting on the surface or in the space above it depending on the task. Our approach integrates continuous space usage with bimanual interaction to provide an expressive set of 3D modeling operations. Preliminary trials with our experimental setup show this as a very promising avenue for further work.

2. VR

   Title of tutorial: Designing immersive VR systems: From bits to bolts

   Luciano P. Soares, Joaquim A. Jorge, Jose Miguel Salles Dias, and 4 more authors

   In Proceedings of the 2012 IEEE Virtual Reality, May 2012

   Abs DOI

   Immersive 3D Virtual Environments (VE) have become affordable for many research centers. However, a complete solution needs several integration steps to be fully operational. Some of these steps are difficult to accomplish and require an uncommon combination of different skills. This tutorial presents the most recent techniques developed to address this problem, from displays to software tools. The hardware in a typical VR installations combines projectors, screens, speakers, computers, tracking and I/O devices. The tutorial will discuss hardware options, explaining their advantages and disadvantages. We will cover design decisions from basic software and hardware design, through user tracking, multimodal human-computer interfaces and acoustic rendering, to how to administrate the whole solution. Additionally, we will provide an introduction to existing tracking technologies, explaining how the most common devices work, while focusing on infrared optical tracking. Finally, we briefly cover integration software and middleware developed for most VE settings.

2011

1. Combining bimanual manipulation and pen-based input for 3D modelling

   Pedro Lopes, Daniel Mendes, Bruno De Araujo, and 1 more author

   In Proceedings of the Eighth Eurographics Symposium on Sketch-Based Interfaces and Modeling, Vancouver, British Columbia, Canada, May 2011

   Abs DOI

   Multitouch enabled surfaces can bring advantages to modelling scenarios, in particular if bimanual and pen input can be combined. In this work, we assess the suitability of multitouch interfaces to 3D sketching tasks. We developed a multitouch enabled version of ShapeShop, whereby bimanual gestures allow users to explore the canvas through camera operations while using a pen to sketch. This provides a comfortable setting familiar to most users. Our contribution focuses on comparing the combined approach (bimanual and pen) to the pen-only interface for similar tasks. We conducted the evaluation helped by ten sketching experts who exercised both techniques. Results show that our approach both simplifies workflow and lowers task times, when compared to the pen-only interface, which is what most current sketching applications provide.

2010

1. Multi-projector VR Systems

   Luciano P. Soares, Alberto Raposo, Joaquim Jorge, and 3 more authors

   In Proceedings of the 2010 23RD SIBGRAPI - Conference on Graphics, Patterns and Images Tutorials, May 2010

   Abs

   Immersive multi-projection environments are becoming affordable for many research centers, but these solutions needs several integration steps to be fully operational, and some of these steps are difficult and not in a common domain. This paper presents the most recent techniques involved in multi-projection solutions, from projection to computer cluster software. The hardware in these VR (Virtual Reality) installations is a connection of projectors, screen, speaker, computers and tracking devices. This survey paper will introduce hardware options, explaining their advantages and disadvantages. We will cover software design and open source tools available, and how to administrate the whole solution.

2009

1. Style nodes and repolygonization for hierarchical tree-based implicit surface modelling

   Pauline Jepp, Bruno De Araujo, Joaquim Jorge, and 2 more authors

   In Proceedings of the Fifth Eurographics Conference on Computational Aesthetics in Graphics, Visualization and Imaging, Victoria, British Columbia, Canada, May 2009

   Abs

   In this paper we present an extension to a hierarchical tree based implicit surface modelling system that includes interactively controlling style and appearance, and also creating a more accurate curvature based polygonal approximation. Multiple styles can be layered and applied to objects so that they are guided by local geometry although not strictly bound by it. To achieve this a new node, the Style Unary Node, is added to the ShapeShop BlobTree, which creates a style blending region inspired by primitive field blending. As visualization of implicit surfaces in interactive environments is often based on polygonization a more accurate curvature based polygonisation algorithm is also presented.

2007

1. Extensible middleware framework for multimodal interfaces in distributed environments

   Vitor Fernandes, Tiago Guerreiro, Bruno De Araujo, and 2 more authors

   In Proceedings of the 9th International Conference on Multimodal Interfaces, Nagoya, Aichi, Japan, May 2007

   Abs DOI

   We present a framework to manage multimodal applications and interfaces in a reusable and extensible manner. We achieve this by focusing the architecture both on applications’ needs and devices’ capabilities. One particular domain we want to approach is collaborative environments where several modalities and applications make it necessary to provide for an extensible system combining diverse components across heterogeneous platforms on-the-fly. This paper describes the proposed framework and its main contributions in the context of an architectural application scenario. We demonstrate how to connect different non-conventional applications and input modalities around an immersive environment (tiled display wall).

2. IMPROVE: collaborative design review in mobile mixed reality

   In Proceedings of the 2nd International Conference on Virtual Reality, Beijing, China, May 2007

   Abs

   In this paper we introduce an innovative application designed tomake collaborative design review in the architectural and automotive domainmore effective. For this purpose we present a system architecture whichcombines variety of visualization displays such as high resolution multitile displays, TabletPCs and head-mounted displays with innovative 2D and 3DInteraction Paradigms to better support collaborative mobile mixed realitydesign reviews. Our research and development is motivated by two usescenarios: automotive and architectural design review involving real users fromPagePark architects and FIAT Elasis. Our activities are supported by the EUIST project IMPROVE aimed at developing advanced display techniques,fostering activities in the areas of: optical see-through HMD development usingunique OLED technology, marker-less optical tracking, mixed reality rendering,image calibration for large tiled displays, collaborative tablet-based andprojection wall oriented interaction and stereoscopic video streaming for mobileusers. The paper gives an overview of the hardware and software developmentswithin IMPROVE and concludes with results from first user tests.

3. IMPROVE: advanced displays and interaction techniques for collaborative design review

   Pedro Santos, André Stork, Thomas Gierlinger, and 13 more authors

   In Proceedings of the 2nd International Conference on Virtual Reality, Beijing, China, May 2007

   Abs

   In this paper we present evaluation results of an innovative application designed to make collaborative design review in the architectural and automotive domain more effective. Within IMPROVE, a European research project in the area of advanced displays, we are combining high resolution multi-tile displays, TabletPCs and head-mounted displays with innovative 2D and 3D Interaction Paradigms to better support collaborative mobile mixed reality design reviews. Our research and development is motivated by application scenarios in the automotive domain involving FIAT Elasis from Naples, Italy and in the architectural domain involving Page/Park architects from Glasgow, Scotland. User evaluation took place at Glasgow (UK), Naples (ITA) and Darmstadt (GER), where we tested the integrated IMPROVE prototype application. The tests were conducted based on several heuristics such as ergonomics and psychomotorial factors and they were conducted based on guidelines recommended by ISO 9241 to verify whether the developed interfaces were suitable for the applications scenarios. Evaluation results show that there is a strong demand for more interactive design review systems, allowing users greater flexibility and greater choice of input and visualization modalities as well as their combination.

4. IMPROVE: designing effective interaction for virtual and mixed reality environments

   Pedro Santos, André Stork, Thomas Gierlinger, and 13 more authors

   In Proceedings of the 12th International Conference on Human-Computer Interaction: Interaction Platforms and Techniques, Beijing, China, May 2007

   Abs

   In this paper we present evaluation results of an innovative application designed to make collaborative design review in the architectural and automotive domain more effective. Within IMPROVE, a European research project in the area of advanced displays, we are combining high resolution multi-tile displays, TabletPCs and head-mounted displays with innovative 2D and 3D Interaction Paradigms to better support collaborative mobile mixed reality design reviews. Our research and development is motivated by application scenarios in the automotive domain involving FIAT Elasis from Naples, Italy and in the architectural domain involving Page/Park architects from Glasgow, Scotland. User evaluation took place at Glasgow (UK), Naples (ITA) and Darmstadt (GER), where we tested the integrated IMPROVE prototype application. The tests were conducted based on several heuristics such as ergonomics and psychomotorial factors and they were conducted based on guidelines recommended by ISO 9241 to verify whether the developed interfaces were suitable for the applications scenarios. Evaluation results show that there is a strong demand for more interactive design review systems, allowing users greater flexibility and greater choice of input and visualization modalities as well as their combination.

2005

1. A Calligraphic Interface for Interactive Free-Form Modeling with Large Datasets

   Bruno Rodrigues De Araujo and Joaquim Armando Pires Jorge

   In Proceedings of the XVIII Brazilian Symposium on Computer Graphics and Image Processing, May 2005

   Abs DOI

   We present a sketching interface for modeling shapes definedby large sets of points. Our system supports powerful modeling operations that are applied directly to the points defining the surface. These operations are based on sketch input to allow directly creating objects using simple strokes. Objects may be edited by either by boundary over-sketching, or by cutting, relief drawing on surfaces, merging and cloning. By combining these operations we can create complex shapes, including objects with sharp features. Our work uses the Multi-level Partition of Unity Implicits (MPU) technique to convert point clouds into implicit surfaces. Furthermore, we have devised a fast adaptive incremental polygonization algorithm which takes advantage of the MPU structure. This makes local re-polygonization possible and allows real-time modifications to large point sets since it avoids re-calculating the whole polygonal representation from scratch after each modification.

2. Adaptive polygonization of implicit surfaces

   Bruno De Araujo and Joaquim Armando Pires Jorge

   Comput. Graph., Oct 2005

   Abs DOI

   We present an algorithm for polygonizing closed implicit surfaces, which produces meshes adapted to the local curvature of the surface. Our method is similar to, but not based on, Marching Triangles, in that we start from a point on the surface and develop a mesh from that point using a surface-tracking approach. However, our approach works by managing fronts, or sets of points on the border of the current polygonization. Fronts can subdivide to form new fronts or merge if they become adjacent. In a marked departure from previous approaches, our meshes approximate the surface through heuristics relying on curvature. Furthermore, our method works completely on-the-fly, resolving cracks as it proceeds, without the need for any post-remeshing step to correct failures. We have tested the algorithm with three different representations of implicit surfaces, variational, analytical and MPU, using non-trivial data sets, yielding results that illustrate the flexibility and scalability of our technique. Performance comparisons with variants of Marching Cubes show that our approach is capable of good accuracy and meshing quality without sacrificing computing resources.

2004

1. 

   Curvature Dependent Polygonization of Implicit Surfaces

   Bruno Rodrigues De Araujo and Joaquim Armando Pires Jorge

   In Proceedings of the Computer Graphics and Image Processing, XVII Brazilian Symposium, Oct 2004

   Abs

   We present an algorithm for polygonizing closed implicit surfaces, which produces meshes adapted to the local curvature of the surface.Our method is similar to, but NOT based on, Marching Triangles, in that we start from a point on the surface and develop a mesh from that point using a surface-tracking approach.In a marked departure from previous approaches, our meshes approximate the surface through heuristics relying on curvature.Furthermore, our method works completely on-the-fly, resolving cracks as it proceeds, without the need for any post-remeshing step to correct failures.We have tested the algorithm with three different representations of implicit surfaces, Variational, analytical and MPU, using non-trivial data sets, yielding results that illustrate the flexibility and scalability of our technique.Performance comparisons with variants of Marching Cubes show that our approach is capable of good accuracy and meshing quality without sacrificing computing resources.

2. MIBlob: a tool for medical visualization and modelling using sketches

   Bruno De Araujo, Joaquim Jorge, Mario Costa Sousa, and 2 more authors

   In ACM SIGGRAPH 2004 Posters, Los Angeles, California, Oct 2004

   DOI