Here is a small selection of the many projects that I worked on over the years. This selection includes the projects that I feel are most relevant in terms of academic and technical research. For a glimpse at some of the other projects that define my technical expertise please click on the T.B. link on the side.

Kinect in Research

Date: 2011
Collaborators: Petros Faloutsos, Costas Sideris

Microsoft Kinect is an amazing device that has opened up the landscape for many applications and other very interesting avenues of research. I am actively working on a range of research projects to explore ways in which the Kinect can improve my ongoing work on medical research and robust perceptual interfaces. Be sure to check out the open source Python wrapper I developed for OpenNI as part of this project on the software page.

A Robust Framework for Perceptual Interfaces

Date: 2011
Collaborators: Petros Faloutsos

There is substantial interest and demand for perceptual interfaces in the research community and the industry, but key flaws in current approaches prevent these technologies from becoming truly viable and pervasive. I am researching and developing a set of new ideas that are designed to address the current limitations of the field and produce truly usable tools.

Automatic Visual Analysis of Accredited Training Tasks for Laparoscopic Surgery

Date: 2010
Collaborators: Florian Kasper, Brian Allen, Petros Faloutsos, Erik Dutson

Laparoscopic surgery is a delicate medical procedure that requires specialized motor skills that are difficult to learn. I developed with my collaborators an advanced computer vision system that can automatically track and analyze the performance of a surgeon and detect key events of a training task. This technology provides a level of automation that has not been available before in medical training. This project also gave me the opportunity to research and develop new ideas for the robust tracking of multiple targets in a cluttered environment.

Gaming Technologies for Wireless Health

Date: 2009
Collaborators: Majid Sarrafzadeh

I researched and developed a fully playable game that requires the player to perform full body motions to score points. The game is fun, results in substantial calorie consuption and promotes fitness, which are all key goals of the emerging field of wireless health. The game uses an advanced software solution that works with the sensing capabilities of commodity gaming hardware.

Multigrid Methods for High Resolution Physics Simulations

Date: 2009
Collaborators: Rasmus Tamstorf, Andy Selle, Eftychios Sifakis, Joey Teran
, Yongning Zhu, Disney Animation Studios

I researched and developed a multigrid technique for accelerating expensive high resolution simulations for production animation.

Image Based Sculpting

Date: 2009
Collaborators: Petros Faloutsos

I am researching a technique that combines discrete differential geometry, deformable models, variational calculus, and stereo vision for extracting high-quality, high-detail geometry from images.

Unit Testing Graphics Code Effectively in C++

Date: 2009
Collaborators: None

Developing Unit tests to quickly validate code modules is common practice in software engineering. If your module has well-defined input and output specifications, building a unit test is trivial. In addition there are many software libraries that make the task of building unit tests very easy. However, in graphics, most of the time the output of your software is visual and thus there isn't a direct quantitative way to validate its conformance with specifications. The problem is even more substantial if your graphics code implies interactivity. I developed a sophisticated library that makes the task of unit testing graphics code particularly easy. There is a related library in Python called OpenGL Context, but the relevance of this work lies on the fact that devising a suite with the same kind of flexibility in C++ is not trivial at all and yet a large amount of graphics code needs to be written in C++. I am currently using this library extensively to quickly validate my research code. The reward is that I can develop software very rapidly and the incidence of bugs in my code modules is very low.

Effective Software Prototyping for Scientific Research

Date: current
Collaborators: None

The problem of prototyping code effectively has been one of the persistent themes of my research work, since I started the PhD program. Many of the problems that you encounter as a researcher are highly complex and often a mathematical abstraction is simply not good enough to give you sufficient insight. The ability to experiment with ideas and concepts is therefore an extremely valuable asset. Even more so when you are working in a creative field such as computer graphics! Ideally you would like to be able to put together a piece of code very quickly without bells and whistles, and try out an idea. You mustn’t let go of that valuable inspiration, which may turn into your next big break-through! Yet if you think that putting together a bunch of code in half an hour without any regard for software quality gets you there fast, then you are wrong! The time you spend debugging you code, and sifting through apparently incomprehensible bugs, largely offsets the time you gain by writing your code in a very short amount of time. Moreover, the frustration that comes with it, is going to kill your inspiration and enthusiasm. This is not something that you want to underestimate either. Lastly, your PhD advisor or your collaborators are not going to be happy when they have to deal with a piece of code that crashes every minute, even if they know that it is research code.

To tackle this problem I did *not* end up writing a software tool that does everything for me, or devising yet another programming language. This is the common approach and one that has not been particularly successful in my opinion. Instead I came up with an effective software pipeline that leverages a number of existing technologies and the way they communicate with each other. With this I tried to exploit a very beautiful concept in computer science: “levels of abstraction”. A computer scientist knows that you should focus your intellectual magnifying glass on the scope of the problem that you are dealing with at the moment. You don’t want to see any more or any less detail that what is needed. With this in mind, I carefully selected a number of existing technologies, organized them based on their level of abstraction, and performed some upstream work to optimize their intercommunication. Someday I would like to publish my findings.

Flexible Sketch-Recognition Pipeline for Embedded Systems

Date: 2008
Collaborators: Petros Faloutsos

Back to my previous work on sketch-based interfaces, I wanted to exploit the fact that my technique for sketch-recognition was also highly efficient and compact. I wanted also to emphasize that my approach achieves a good degree of generality by analyzing sketches based on their semantics instead of their geometry alone. The reward is that my interface can be easily adapted to work with a variety of other mature animation packages and it is easy to port onto a variety of hardware devices. With this in mind, I turned my original interface into more of generic sketch-recognition pipeline and I implemented a highly efficient version of it that runs in real-time on the Nintendo DS and on the iPhone.

Regularized Non-Linear Geometric Deformations

Date: 2007
Collaborators: Doug Roble, Petros Faloutsos, Digital Domain

Deformable models play an important role both in computer vision and computer graphics. Deformations are described mathematically using a non-linear formulation due to differential geometry, and the resulting model is discretized and often linearized, to meet the computational needs of the task at hand. While linear models are attractive, sometime it is very useful to solve the non-linear problem directly. With this project, I developed a mathematical technique that regularizes the problem and solves the non-linear deformation problem iteratively with good convergence.

Fast Finite Element Method for Skin Deformation

Date: 2007
Collaborators: Sung-Hee Lee, Demetri Terzopoulos

I studied a technique for simulating a thin fatty tissue such a skin in real time. The constraints were to make the simulation physically accurate and fast. Spring-mass systems, though common for these tasks, are not good enough for the level of accuracy that we were seeking. I ended up using an accurate simulation based on the Finite Element Method (FEM). However, FEM is slow and requires fairly small time steps if you want to handle large deformations with the usual linearizations. I augmented regular FEM with a method suggested by Möller called Stiffness Warping to make the simulation robust for large deformations and make it run in real-time.

Robust Stroke Grouping with Support Vector Machines

Date: 2007
Collaborators: Petros Faloutsos

The sketch-recognition technique that I had developed before works well and robustly, if each stroke in the sketch has a well-defined semantic meaning within the drawing. However, this limitation is somewhat restrictive. Most users enjoy drawing sketches freely with many strokes and they may introduce additional strokes for an esthetic purpose that may throw off the recognition engine. I devised a technique that overcomes this limitation successfully. The basic idea is to search for possible ways of grouping strokes, so that each group has a clear semantic purpose in the sketch. The search is based on the semantic information contained in the training set of a statistical tool called Support Vector Machine. While I developed this technique in conjunction with my work on sketch-based interfaces, it does not depend on my particular approach for sketch-recognition.

Neural-Network Model for Grounding Space Relations in Natural Language Processing

Date: 2007
Collaborators: Michael Dyer

This project was a foray for me into a classic problem in artificial intelligence. The goal was to develop a system than can learn to associate English sentences that specify the spacial relationship of two objects with their actual visual arrangement. In other words, I would like to present two shapes to the computer – say a triangle and circle—and teach it that, say, the triangle is on the left of the circle. This problem is known as grounding. Another important research decision was to use connectionist model, that is to use a neural network. One related key problem that I had to research was how to learn the inverse of a function with neural networks. The system that I developed could learn basic relationships within a prescribed visual grid.

Image-Based Modeling with Video Textures

Date: 2006
Collaborators: Petros Faloutsos

Achieving photo-realism with synthetic techniques is one the major goals of computer graphics. Yet deceiving the human eye is certainly not an easy task at all! However, while real-time graphics relies exclusively on synthetic imagery, in visual effects we try to achieve photorealism by combining synthetic elements with live action footage. This project was an attempt to do the latter in a real-time context and was based on the notion that modern graphics hardware is a highly capable at streaming video fast. One key research question was how to augment the video stream with 3D visual cues, so that the texture will not look flat anymore. I studied extensively related problems in image-based rendering and modeling and I developed a library that could stream video directly into texture objects using both the fixed functionality rendering pipeline and shaders.

Robust Sketch-Based Interface for Posing Facial Expressions

Date: 2005
Collaborators: Petros Faloutsos

This was my first project in the domain of sketch-based interfaces and a project that stayed with me throughout my PhD years. A sketch-based interface is an intelligent interface that lets a user control articulated computer tasks with the fluidity offered by drawing strokes on a tablet. This is a challenging task for sketches --unlike traditional computer interfaces-- describe users’ intentions in a rather ambiguous and qualitative form. Thus the computer has to “guess” what the user really wants from ill-defined, noisy, and incomplete data. The research goal of this project was to create a robust, fast, and adaptable sketch-based interface to pose facial expressions for applications in computer graphics and animation. We chose facial expressions, because we needed a focused, well-defined application to start with. After more than a year of research, I devised an approach than analyzes sketches at different layers of abstraction and leverages machine learning with other statistical techniques to deliver an interface that met our original goals successfully. In addition, my algorithm was general enough that could work with other kinds of drawings other than faces.

Music Eye

Date: 2005
Collaborators: Stefano Soatto

This was a fun vision project. The problem is to detect a sequence of notes from the video stream of a performance on a keyboard instrument. After many experiments I found that I can obtain the largest amount of visual cues if I place the camera on the side of the performer giving a perspective view of the keyboard. Detecting the region in the image that corresponds to the keyboard is simple given its distinctive appearance. I compute a homography to rectify the keyboard region and I can exploit the known geometrical structure of the keyboard to divide the rectified image in to regions corresponding roughly to each key. For each segmented region of the keyboard, I use several visual cues to trigger note events.

Automatic Estimation of Leaf Quantity in Trees and Brushes from Images

Date: 2005
Collaborators: William Kaiser, Eric Graham, CENS

This is my second project with the biologist Eric Graham. Another quantity that helps a biologist evaluate the ecology of a natural environment is the number of leaves on trees and brushes. Since an absolute estimate of this quantity is too difficult to obtain, what we are seeking is a relative quantity so that the biologist can compare the ecology of several different regions and at different times of year. In order to tackle this problem I proposed to measure leaf density from an image and I studied a relative metric that can relate this density information to the ratio of the number of leaves.

Automatic Estimation of Leaf Area from Images

Date: 2005
Collaborators: William Kaiser, Eric Graham, CENS

Eric Graham, a biologist at UCLA often needs to study plants to evaluate the ecology of a designated environment. The area of the leaves in that region provides substantial information about the environment for it relates to the amount of surface available for photosynthesis and other important biological mechanisms. However, measuring the area of a sufficient number of leaves by hand with a ruler is a tedious, slow, and inaccurate process. I was presented with the research problem of simplifying this task by devising an effective image-based technique. After several months of research, I developed a technique that uses pixel ratios to provide estimates that are far more accurate than eyeballing the area of a leaf with a ruler. The software that I developed was integrated in Photoshop and emitted its data into Excel worksheets ready for further analysis. An interesting highlight of the project was the technique I devised for calibrating images for real-world measurements. Since I could not afford to do full camera calibration, I only required the biologists to place a couple of regular pennies in the image. Since the geometry and size of a penny is well-known and its appearance makes it easy to segment, I could calibrate images on the fly with no human intervention and the resulting accuracy was more than sufficient for the task.

Image Tracking with the NIMS system

Date: 2004
Collaborators: Chung Hsieh, William Kaiser, CENS

This project was my first collaboration with the Center for Embedded Networked Sensing (CENS) at UCLA. My task was to develop a system for consistently tracking a set of features as a remotely actuated camera is moving. My software was going to be integrated with the NIMS system, which consists of an actuated camera attached to a controllable rig. In NIMS, the camera can be translated within a vertical plane and we have precise control over its pan and tilt. The system that I developed, uses the geometrical information that comes from camera motion to estimate the new position of visual features. This technique allows the system to lock onto these featuresas the camera moves. The system also tries to discard features that are occluded. Our targets are plants with a lot of thin structures and occlusions, a very challenging scenario for any computer vision algorithm.

Generating Optimal Samples for VLSI Floorplanning Benchmarks

Date: 2003
Collaborators: Joseph Shinnerl, Michail Romesis, Jason Cong

Floorplanning is a very important stage in the design of integrated circuits. One important constraint is to maximize the utilization of the available area on a silicon wafer. The second constraint is to minimize the overall wire length that connects the various components in the design. The goal of this project was to generate optimal or close-to-optimal arrangements of rectangular components as a means for validating the quality of a given Floorplanning algorithm. There are two approaches to do this: top-down and bottom-up. We researched both approaches. The bottom-up technique that we devised was very promising and had the added advantage of optimizing wire length, however proving the optimality of this technique was extremely difficult. Therefore we ended up concentrating on the top-down technique for a designated publication deadline.