Book chapter in Digital Imaging for Cultural Heritage Preservation: Analysis, Restoration, and Reconstruction of Ancient Artworks, CRC Press, July 28, 2011

Chapter Title: Digitizing the Parthenon: Estimating Surface Reflectance under Measured Natural Illumination, P. Debevec, C. Tchou, A. Gardner, T. Hawkins, C. Poullis, J. Stumpfel, A. Jones, N. Yun, P. Einarsson, T. Lundgren, M. Fajardo, P. Martinez
Editors: Filippo Stanco; Sebastiano Battiato, University of Catania, Italy; Giovanni Gallo, University of Catania, Italy

Book:

3D Reconstruction of Urban Areas
IEEE 3D Imaging, Modeling, Processing, Visualization, Transmission 2011 (3DIM/3DPVT in cooperation with Eurographics), Hangzhou, China

Abstract: Virtual representations of real world areas are increasingly being employed in a variety of different applications such as urban planning, personnel training, simulations, etc. Despite the increasing demand for such realistic 3D representations, it still remains a very hard and often manual process. In this paper, we address the problem of creating photorealistic 3D scene models for large-scale areas and present a complete system.

The proposed system comprises of two main components: (1) A reconstruction pipeline which employs a fully automatic technique for extracting and producing high-fidelity geometric models directly from Light Detection and Ranging (LiDAR) data and (2) A flexible texture blending technique for generating high-quality photorealistic textures by fusing information from multiple optical sensor resources. The result is a photorealistic 3D representation of large-scale areas(city-size) of the real-world.

Paper:

Delineation and Geometric Modeling of Road Networks
ISPRS Journal of Photogrammetry and Remote Sensing, Elsevier, 2009

Abstract: In this work we focus on the automatic and reliable detection and extraction of transportation networks from remote sensor data including aerial photographs, satellite images, and LiDAR. We present an integrated solution that merges the strengths of perceptual grouping theory(Gabor filters, tensor voting) and segmentation(global optimization by graph-cuts), under a unified framework to address the challenging problem of automated feature detection, classifcation and extraction.

The proposed approach leverages the multi-scale, multiorientation capabilities of Gabor filters for the inference of geospatial features, the effective and robust handling of noisy, incomplete data of tensor voting for the feature classification and the fast and efficient optimization of graph cuts for the segmentation and labeling of road features.

Paper:

Automatic Reconstruction of Cities from Remote Sensor Data
IEEE Computer Vision and Pattern Recognition, 2009, Miami Beach, Florida

Abstract: In this paper, we address the complex problem of rapid modeling of large-scale areas and present a novel approach for the automatic reconstruction of cities from remote sensor data. The goal in this work is to automatically create lightweight, watertight polygonal 3D models from LiDAR data(Light Detection and Ranging) captured by an airborne scanner. This is achieved in three steps: preprocessing, segmentation and modeling, as shown in Figure 1.

Our main technical contributions in this paper are: (i) a novel, robust, automatic segmentation technique based on the statistical analysis of the geometric properties of the data, which makes no particular assumptions about the input data, thus having no data dependencies, and (ii) an efficient and automatic modeling pipeline for the reconstruction of large-scale areas containing several thousands of buildings.

We have extensively tested the proposed approach with several city-size datasets including downtown Baltimore, downtown Denver, the city of Atlanta, downtown Oakland, and we present and evaluate the experimental results.

Paper:

Automatic Creation of Massive Virtual Cities
IEEE Virtual Reality 2009, Lafayette, Louisiana

Abstract: This research effort focuses on the historically-difficult problem of creating large-scale (city size) scene models from sensor data, including rapid extraction and modeling of geometry models.

The solution to this problem is sought in the development of a novel modeling system with a fully automatic technique for the extraction of polygonal 3D models from LiDAR (Light Detection And Ranging) data. The result is an accurate 3D model representation of the real-world as shown in Figure 1. We present and evaluate experimental results of our approach for the automatic reconstruction of large U.S. cities.

Paper:

Photorealistic Large-scale Urban City Model Reconstruction
IEEE Transactions on Visualization and Computer Graphics 2009

Abstract: The rapid and efficient creation of virtual environments has become a crucial part of virtual reality applications. In particular, civil and defense applications often require and employ detailed models of operations areas for training, simulations of different scenarios, planning for natural or man-made events, monitoring, surveillance, games and films. A realistic representation of the large-scale environments is therefore imperative for the success of such applications since it increases the immersive experience of its users and helps reduce the difference between physical and virtual reality. However, the task of creating such large-scale virtual environments still remains a time-consuming and manual work.

In this work we propose a novel method for the rapid reconstruction of photorealistic large-scale virtual environments. First, a novel, extendible, parameterized geometric primitive is presented for the automatic building identification and reconstruction of building structures. In addition, buildings with complex roofs containing complex linear and non-linear surfaces are reconstructed interactively using a linear polygonal and a nonlinear primitive, respectively. Secondly, we present a rendering pipeline for the composition of photorealistic textures which unlike existing techniques it can recover missing or occluded texture information by integrating multiple information captured from different optical sensors (ground, aerial and satellite).

Journal:   Video:   Video:

Rapid Creation of Large-scale Photorealistic Virtual Environments
IEEE Virtual Reality 2008, Reno, Nevada

Abstract: The rapid and efficient creation of virtual environments has become a crucial part of virtual reality applications. In particular, civil and defense applications often require and employ detailed models of operations areas for training, simulations of different scenarios, planning for natural or man-made events, monitoring, surveillance, games and films. A realistic representation of the large-scale environments is therefore imperative for the success of such applications since it increases the immersive experience of its users and helps reduce the difference between physical and virtual reality. However, the task of creating such large-scale virtual environments still remains a time-consuming and manual work.

In this work we propose a novel method for the rapid reconstruction of photorealistic large-scale virtual environments. First, a novel parameterized geometric primitive is presented for the automatic building detection, identification and reconstruction of building structures. In addition, buildings with complex roofs containing non-linear surfaces are reconstructed interactively using a nonlinear primitive. Secondly, we present a rendering pipeline for the composition of photorealistic textures which unlike existing techniques it can recover missing or occluded texture information by integrating multiple information captured from different optical sensors (ground, aerial and satellite).

Paper:

A Vision-based System for Automatic Detection and Extraction of Road Networks
IEEE Workshop on Applications of Computer Vision 2008, Copper Mountain, Colorado

Abstract: In this paper we present a novel vision-based system for automatic detection and extraction of complex road networks from various sensor resources such as aerial photographs, satellite images, and LiDAR. Uniquely, the proposed system is an integrated solution that merges the power of perceptual grouping theory(gabor filtering, tensor voting) and optimized segmentation techniques(global optimization using graph-cuts) into a unified framework to address the challenging problems of geospatial feature detection and classification.

Firstly, the local presicion of the gabor filters is combined with the global context of the tensor voting to produce accurate classification of the geospatial features. In addition, the tensorial representation used for the encoding of the data eliminates the need for any thresholds, therefore removing any data dependencies.

Secondly, a novel orientation-based segmentation is presented which incorporates the classification of the perceptual grouping, and results in segmentations with better defined boundaries and continuous linear segments.

Finally, a set of gaussian-based filters are applied to automatically extract centerline information (magnitude, width and orientation). This information is then used for creating road segments and then transforming them to their polygonal representations.

Paper:   Video:

Linear Feature Extraction Using Perceptual Grouping And Graph-Cuts
ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems 2007,Seattle, Washington

Abstract: In this paper we present a novel system for the detection and extraction of road map information from high-resolution satellite imagery.

Uniquely, the proposed system is an integrated solution that merges the power of perceptual grouping theory (gabor filtering, tensor voting) and segmentation (graph-cuts) into a unified framework to address the problems of road feature detection and classification. Local orientation information is derived using a bank of gabor filters and is refined using tensor voting. A segmentation method based on global optimization by graph-cuts is developed for segmenting foreground(road pixels) and background objects while preserving oriented boundaries. Road centerlines are detected using pairs of gaussian-based filters and road network vector maps are finally extracted using a tracking algorithm.

The proposed system works with a single or multiple images, and any available elevation information. User interaction is limited and is performed at the begining of the system execution. User intervention is allowed at any stage of the process to refine or edit the automatically generated results.

Paper:   Poster:   Video:

Generating High-Resolution Textures For 3D Virtual Environments using VITM
IEEE International Conference on Multimedia and Expo 2007,Beijing,China

Abstract: Image based modeling and rendering techniques have become increasingly popular for creating and visualizing 3D models from a set of images. Typically, these techniques depend on view-dependent texture mapping to render the textured 3D models in which the texture of novel views is synthesized at runtime according to different view-points. This is computationaly expensive and limits their application in domains where efficient computations are required, such as games and virtual reality. In this paper we present an offline technique for creating view-independent texture atlases for 3D models, given a set of registered images. The best texture map resolution is computed by considering the areas of the projected polygons in the images. Texture maps are generated by a weighted composition of all available image information in the scene.Assuming that all surfaces of the model are exhibiting Lambertian reflectance properties, ray-tracing is then employed, for creating the view-independent texture maps. Finally, all the generated texture maps are packed into texture atlases. The result is a 3D model with an associated view independent texture atlas which can be used efficiently in any application without any knowledge of camera pose information.

Paper:   Poster:

Automatic Building Extraction from LiDAR data
American Society for Photogrammetry and Remote Sensing 2007, Tampa, Florida

Presentation:

Automatic Extraction of Linear Features from LiDAR and Imagery
American Society for Photogrammetry and Remote Sensing 2007, Tampa, Florida

Presentation:

Making "The Parthenon": Digitization and Reunification of The Parthenon and its Sculptures
Foundation of the Hellenic World 2005, Athens, Greece

Abstract: In this paper we overview the technology and production processes used to create “The Parthenon”, a short computer animation that visually reunites the Parthenon and its sculptural decorations, separated since the beginning of the 19th century. The film uses a variety of technologies including 3D laser scanning, structured light scanning, photometric stereo, inverse global illumination, photogrammetric modeling, imagebased rendering, BRDF measurement, and monte-carlo global illumination to create the imagery used in the film.

Paper:

Photogrammetric modeling and image-based rendering for rapid virtual environment creation
24th Army Science Conference 2004, Orlando, Florida

Abstract: For realistic simulations, architecture is one of the most important elements to model and render photorealistically. Current techniques of converting architectural plans or survey data to CAD models are labor intensive, and methods for rendering such models are generally not photorealistic. In this work, we present a new approach for modeling and rendering existing architectural scenes from a sparse set of still photographs. For modeling, we use photogrammetric modeling techniques to recover a the geometric representation of the architecture. The photogrammetric modeling approach presented in this paper is effective, robust and powerful because it fully exploits structural symmetries and constraints which are characteristic of architectural scenes. For rendering, we use view-dependent texture mapping, a method for compositing multiple images of a scene to create renderings from novel views. Lastly, we present a software package, named Fac¸ade, which uses the techniques described to recover the geometry and appearance of architectural scenes directly from a sparse set of photographs.

Software: Facade v2.0, v3.0, v3.0.1, v3.1. An extention of the original SGI software package, Facade v1.5 by Paul Debevec, designed for Windows.

Paper:

Short film “The Parthenon”
Electronic Theater, ACM SIGGRAPH(Special Interest Group on Graphics and Interactive Techniques) 2004

More information can be found here.

Estimating Surface Reflectance Properties of a Complex Scene under Captured Natural Illumination
USC, Institute of Creative Technologies Technical Report 2004/International Journal of Architectural Computing, 2007

Abstract: We present a process for estimating spatially-varying surface reflectance of a complex scene observed under natural illumination conditions. The process uses a laser-scanned model of the scene’s geometry, a set of digital images viewing the scene’s surfaces under a variety of natural illumination conditions, and a set of corresponding measurements of the scene’s incident illumination in each photograph. The process then employs an iterative inverse global illumination technique to compute surface colors for the scene which, when rendered under the recorded illumination conditions, best reproduce the scene’s appearance in the photographs. In our process we measure BRDFs of representative surfaces in the scene to better model the non-Lambertian surface reflectance. Our process uses a novel lighting measurement apparatus to record the full dynamic range of both sunlit and cloudy natural illumination conditions. We employ Monte-Carlo global illumination, multiresolution geometry, and a texture atlas system to perform inverse global illumination on the scene. The result is a lighting-independent model of the scene that can be re-illuminated under any form of lighting. We demonstrate the process on a real-world archaeological site, showing that the technique can produce novel illumination renderings consistent with real photographs as well as reflectance properties that are consistent with ground-truth reflectance measurements.

Technical report:

3D Animation for a Segmented Space Reflector Telescope
World Scientific and Engineering Academy and Society Conference SCI 2003

Abstract: This paper presents the design and implementation of a 3-D animation for a large space segmented telescope test-bed. The test-bed was developed to provide a platform for experiments related to the control of precision segmented reflectors. In this paper, the physical model of the segmented telescope was imported into our 3-D animation software architecture. Algorithms based on computational geometry were developed to animate the movements of the panel segments of the telescope. In addition, a modulization of the animation system was proposed to facilitate the development and the maintenance of the animation system. This 3-D animation software architecture provides a visualization of control implementation for the segmented telescope test-bed.

Paper: