We present a new meshing algorithm called guided and augmented meshing, GAMesh, which uses a mesh prior to generate a surface for the output points of a point network. By projecting the output points onto this prior and simplifying the resulting mesh, GAMesh ensures a surface with the same topology as the mesh prior but whose geometric fidelity is controlled by the point network. This makes GAMesh independent of both the density and distribution of the output points, a common artifact in traditional surface reconstruction algorithms. We show that such a separation of geometry from topology can have several advantages especially in single-view shape prediction, fair evaluation of point networks and reconstructing surfaces for networks which output sparse point clouds. We further show that by training point networks with GAMesh, we can directly optimize the vertex positions to generate adaptive meshes with arbitrary topologies.
Distortions due to perspective projection is often described under the umbrella term of foreshortening in computer graphics and are treated the same way. However, a large body of literature from artists, perceptual psychologists and perception scientists have shown that the perception of these distortions is different in different situations. While the distortions themselves depend on both the depth and the orientation of the object with respect to the camera image plane, the perception of these distortions depends on other depth cues present in the image. In the absence of any depth cue or prior knowledge about the objects in the scene, the visual system finds it hard to correct the foreshortening automatically and such images need user input and external algorithmic distortion correction.In this paper, we claim that the shape distortion is more perceptible than area distortion, and quantify such perceived foreshortening as the non-uniformity across the image, of the ratio e of the differential areas of an object in the scene and its projection. We also categorize foreshortening into uniform and non-uniform foreshortening. Uniform foreshortening is perceived by our visual system as a distortion, even if e is uniform across the image, only when comparative objects of known sizes are present in the image. Non-uniform foreshortening is perceived when there is no other depth cue in the scene that can help the brain to correct for the distortion. We present a unified solution to correct these distortions in one or more non-occluded foreground objects by applying object-specific segmentation and affine transformation of the segmented camera image plane. Our method also ensures that the background undergoes minimal distortion and preserves background features during this process. This is achieved efficiently by solving Laplace's equations with Dirichlet boundary conditions, assisted by a simple and intuitive user interface.
Performance of interactive graphics walkthrough systems depends on the time taken to fetch the required data from the secondary storage to main memory. It has been earlier established that a large fraction of this fetch time is spent on seeking the data on the hard disk. In order to reduce this seek time, redundant data storage has been proposed in the literature, but the redundancy factors of those layouts are prohibitively high. In this paper, we develop a cost model for the seek time of a layout. Based on this cost model, we propose an elegant algorithm that computes a redundant data layout with the redundancy factor that is within the user specified bounds, while maximizing the performance of the system. Unlike most existing methods, our data layout method can work with models with textures. The interactive rendering speed of the walkthrough system was improved by a factor of 2-4 by using our data layout method when compared to existing methods with or without redundancy.
Vibrant lighting of relief maps via high-resolution dynamic projected imagery can be a powerful tool for simulation, augmented reality, and visualization, enabling several scientific, educational and entertainment applications. This is usually achieved via multiple projectors lighting the relief map, which are then sensed by multiple cameras for geometric and color calibration. However, cumbersome semi-automatic calibration techniques have limited the applicability of such systems.
Many works focus on multi-spectral capture and analysis, but multi-spectral display still remains a challenge. Most prior works on multi-primary displays use ad-hoc narrow band primaries that assure a larger color gamut, but cannot assure a good spectral reproduction. Content-dependent spectral analysis is the only way to produce good spectral reproduction, but cannot be applied to general data sets. Wide primaries are better suited for assuring good spectral reproduction due to greater coverage of the spectral range, but have not been explored much. In this paper we explore the use of wide band primaries for accurate spectral reproduction for the first time and present the first content-independent multi-spectral display achieved using superimposed projections with modified wide band primaries. We present a content-independent primary selection method that selects a small set of n primaries from a large set of m candidate primaries where m > n. Our primary selection method chooses primaries with complete coverage of the range of visible wavelength (for good spectral reproduction accuracy), low interdependency (to limit the primaries to a small number) and higher light throughput (for higher light efficiency). Once the primaries are selected, the input values of the different primary channels to generate a desired spectrum are computed using an optimization method that minimizes spectral mismatch while maximizing visual quality. We implement a real prototype of multi-spectral display consisting of 9-primaries using three modified conventional 3-primary projectors, and compare it with a conventional display to demonstrate its superior performance. Experiments show our display is capable of providing large gamut assuring a good visual appearance while displaying any multi-spectral images at a high spectral accuracy.
Geometric trees are graphs with no cycles in which the nodes have spatial co-ordinates and the edges are geometric curves. Many physical systems can be represented effectively using geometric trees, e.g. river beds, animal neurons, respiratory tracks of mammals etc. As these systems undergo structural metamorphosis, temporally or under the effect of some external stimulus, the underlying tree structures also change. Given two snapshots of structurally morphed trees, an algorithm for comparing them based on geometric and topological tree features is presented. Such comparison provides a wealth of information for interpreting the metamorphosis.
With increasing speed of graphics rendering, the bottleneck in walkthrough applications has shifted to data transfer from secondary storage device to main memory. While techniques designed to reduce the data transfer volume and amortize the transfer cost are well-studied, the disk seek time, which is one of the most important components of the total rendering cost is not reduced explicitly. In this work, we propose an orthogonal approach to address the disk seek time bottleneck, namely single-seek data layouts. This is a solution in one end of the spectrum of solutions that guarantee an upper bound on the number of disk seeks. Using this approach, we can reduce the number of disk seeks required to load the data for any viewpoint in the scene to no more than one. We achieve this single seek layout using data redundancy. We provide a theoretical proof on the upper-bound of this redundancy factor, and analyze its trade-off with the rendering performance through an implementation that uses this data layout for walkthrough applications of datasets with hundreds of millions of triangles.
When a disk in 2D is stretched arbitrarily with possible self-overlaps, without twisting it, its boundary forms a complex curve known as a self-overlapping curve. The mapping between the disk and its deformed self, also called an immersion of the disk, is useful in many applications like shape morphing and curve interpretation. Given a self-overlapping curve, an algorithm for computing its immersion is presented, which has an average time complexity quadratic in the number of points on the curve.
In this paper, we present a simple definition for, and a method to find, cavities and protrusions of a 2D polygon. Using these, we fit conformal polygons with each other, which is similar to a jigsaw puzzle and in a general case is NP-hard to solve. We first build a hierarchy of cavities and protrusions for each polygon and use this hierarchy to check for matches between these geometric features of two polygons. This data structure allows for early rejection of mismatches and thus speeds up the fitting process. We show using many examples, that most of the common configurations in exact polygon fitting can be handled by this algorithm in polynomial time. In case of exact, yet non-unique matches, this algorithm will solve the problem in exponential time.
Color is one of the most common ways to convey information in visualization applications. Color vision deficiency (CVD) affects approximately 200 million individuals worldwide and considerably degrades their performance in understanding such contents by creating red-green or blue-yellow ambiguities. While several content-specific methods have been proposed to resolve these ambiguities, they cannot achieve this effectively in many situations for contents with a large variety of colors. More importantly, they cannot facilitate color identification. We propose a technique for using patterns to encode color information for individuals with CVD, in particular for dichromats. We present the first content-independent method to overlay patterns on colored visualization contents that not only minimizes ambiguities but also allows color identification. Further, since overlaying patterns does not compromise the underlying original colors, it does not hamper the perception of normal trichromats. We validated our method with two user studies: one including 11 subjects with CVD and 19 normal trichromats, and focused on images that use colors to represent multiple categories; and another one including 16 subjects with CVD and 22 normal trichromats, which considered a broader set of images. Our results show that overlaying patterns significantly improves the performance of dichromats in several color-based visualization tasks, making their performance almost similar to normal trichromats'. More interestingly, the patterns augment color information in a positive manner, allowing normal trichromats to perform with greater accuracy.
We present a system that superimposes multiple projections onto an object of arbitrary shape and color to produce high-resolution appearance changes. Our system produces appearances at an improved resolution compared to prior works and can change appearances at near interactive rates. Three main components are central to our system. First, the problem of computing compensation images is formulated as a constrained optimization which yields high-resolution appearances. Second, decomposition of the target appearance into base and scale images enables fast swapping of appearances on the object by requiring the constrained optimization to be computed only once per object. Finally, to make high-quality appearance edits practical, an elliptical Gaussian is used to model projector pixels and their interaction between projectors. To the best of our knowledge, we build the first system that achieves high-resolution and high-quality appearance edits using multiple superimposed projectors on complex nonplanar colored objects. We demonstrate several appearance edits including specular lighting, subsurface scattering, inter-reflections, and color, texture, and geometry changes on objects with different shapes and colors.
Remyelination therapy is a state-of-the-art technique for treating spinal cord injury (SCI). Demyelination—the loss of myelin sheath that insulates axons, is a prominent feature in many neurological disorders resulting in SCI. This lost myelin sheath can be replaced by remyelination. In this paper, we propose an algorithm for efficient automated cell classification and visualization to analyze the progress of remyelination therapy in SCI. Our method takes as input the images of the cells and outputs a density map of the therapeutically important oligodendrocyte-remyelinated axons (OR-axons) which is used for efficacy analysis of the therapy. Our method starts with detecting cell boundaries using a robust, shape-independent algorithm based on iso-contour analysis of the image at progressively increasing intensity levels. The detected boundaries of spatially clustered cells are then separated using the Delaunay triangulation based contour separation method. Finally, the OR-axons are identified and a density map is generated for efficacy analysis of the therapy. Our efficient automated cell classification and visualization of remyelination analysis significantly reduces error due to human subjectivity. We validate the accuracy of our results by extensive cross-verification by the domain experts.
In this paper, we present the first method for the geometric autocalibration of multiple projectors on a set of CAVE-like immersive display surfaces including truncated domes and 4 or 5-wall CAVEs (three side walls, floor, and/or ceiling). All such surfaces can be categorized as swept surfaces and multiple projectors can be registered on them using a single uncalibrated camera without using any physical markers on the surface. Our method can also handle nonlinear distortion in the projectors, common in compact setups where a short throw lens is mounted on each projector. Further, when the whole swept surface is not visible from a single camera view, we can register the projectors using multiple pan and tilted views of the same camera. Thus, our method scales well with different size and resolution of the display. Since we recover the 3D shape of the display, we can achieve registration that is correct from any arbitrary viewpoint appropriate for head-tracked single-user virtual reality systems. We can also achieve wallpapered registration, more appropriate for multiuser collaborative explorations. Though much more immersive than common surfaces like planes and cylinders, general swept surfaces are used today only for niche display environments. Even the more popular 4 or 5-wall CAVE is treated as a piecewise planar surface for calibration purposes and hence projectors are not allowed to be overlapped across the corners. Our method opens up the possibility of using such swept surfaces to create more immersive VR systems without compromising the simplicity of having a completely automatic calibration technique. Such calibration allows completely arbitrary positioning of the projectors in a 5-wall CAVE, without respecting the corners.
We define geometric trees as graphs with no cycles in which nodes have spatial co-ordinates and edges are geometric curves. Temporal morphing of geometric trees through changing positions and geometry of the nodes and edges are common in many applications, physical and simulated data. Examples include geological structures such as changing patterns of river beds and biological structures such as respiratory tracks and deforming patterns of neurons. Given two snapshots of such trees at different times, or two trees representing similar data sets, or trees generated by two different methods, comparison between these trees will provide a wealth of information for interpreting the data or the methods that produced that data. We propose an algorithm to compare geometric trees by detecting feature similarities between the trees, wherein the features are geometric as well as topological.
Cellular biology deals with studying the behavior of cells. Current time-lapse imaging microscopes help us capture the progress of experiments at intervals that allow for understanding of the dynamic and kinematic behavior of the cells. On the other hand, these devices generate such massive amounts of data (250GB of data per experiment) that manual sieving of data to identify interesting patterns becomes virtually impossible. In this paper we propose an end-to-end system to analyze time-lapse images of the cultures of human neural stem cells (hNSC), that includes an image processing system to analyze the images to extract all the relevant geometric and statistical features within and between images, a database management system to manage and handle queries on the data, a visual analytic system to navigate through the data, and a visual query system to explore different relationships and correlations between the parameters. In each stage of the pipeline we make novel algorithmic and conceptual contributions, and the entire system design is motivated by many different yet unanswered exploratory questions pursued by our neurobiologist collaborators. With a few examples we show how such abstract biological queries can be analyzed and answered by our system.
Automated segmentation and tracking of cells in actively developing tissues can provide high-throughput and quantitative spatiotemporal measurements of a range of cell behaviors; cell expansion and cell-division kinetics leading to a better understanding of the underlying dynamics of morphogenesis. Here, we have studied the problem of constructing cell lineages in time-lapse volumetric image stacks obtained using Confocal Laser Scanning Microscopy (CLSM). The novel contribution of the work lies in its ability to segment and track cells in densely packed tissue, the shoot apical meristem (SAM), through the use of a close-loop, adaptive segmentation, and tracking approach. The tracking output acts as an indicator of the quality of segmentation and, in turn, the segmentation can be improved to obtain better tracking results. We construct an optimization function that minimizes the segmentation error, which is, in turn, estimated from the tracking results. This adaptive approach significantly improves both tracking and segmentation when compared to an open loop framework in which segmentation and tracking modules operate separately.
Analysis of Confocal Laser Scanning Microscopy (CLSM) images is gaining popularity in developmental biology for understanding growth dynamics. The automated analysis of such images is highly desirable for efficiency and accuracy. The first step in this process is segmentation and tracking leading to computation of cell lineages. In this paper, we present efficient, accurate, and robust segmentation and tracking algorithms for cells and detection of cell divisions in a 4D spatio-temporal image stack of a growing plant meristem. We show how to optimally choose the parameters in the watershed algorithm for high quality segmentation results. This yields high quality tracking results using cell correspondence evaluation functions. We show segmentation and tracking results on Confocal laser scanning microscopy data captured for 72 hours at every 3 hour intervals. Compared to recent results in this area, the proposed algorithms provide significantly longer cell lineages and more comprehensive identification of cell divisions.
The process of generating a 3D model from a set of 2D planar curves is complex due to the existence of many solutions. In this paper we consider a self-intersecting planar closed loop curve, and determine the 3D layered surface P with the curve as its boundary. Specifically, we are interested in a particular class of closed loop curves in 2D with multiple self-crossings which bound a surface homeomorphic to a topological disk. Given such a self-crossing closed loop curve in 2D, we find the deformation of the topological disk whose boundary is the given loop. Further, we find the surface in 3D whose orthographic projection is the computed deformed disk, thus assigning 3D coordinates for the points in the self-crossing loop and its interior space. We also make theoretical observations as to when, given a topological disk in 2D, the computed 3D surface will self-intersect.
Given the time lapse images of human Neuro Stem Cells (hNSC) marked by fluorescent proteins, that are obtained from a confocal laser microscope, we present algorithms to identify, segment, track, and estimate statistical parameters of the cells. The structure of these cells are quite complex and irregular, which makes segmentation and tracking even more challenging. We use a novel combination of Difference of Gaussians and a variant of the Watershed algorithm to segment cells accurately. Our tracking algorithm can identify not only the temporal path of the cells but also events like cell divisions and deaths. Our system is robust, efficient, completely automatic, and removes many drawbacks faced by earlier solutions. We also propose the first geometric algorithm that uses Delaunay triangulation, to find the number of the branches of the cells, which is an important biological feature.
Large scale and structurally complex volume datasets from high-resolution 3D imaging devices or computational simulations pose a number of technical challenges for interactive visual analysis. In this paper, we present the first integration of a multiscale volume representation based on tensor approximation within a GPU-accelerated out-of-core multiresolution rendering framework. Specific contributions include (a) a hierarchical brick-tensor decomposition approach for pre-processing large volume data, (b) a GPU accelerated tensor reconstruction implementation exploiting CUDA capabilities, and (c) an effective tensor-specific quantization strategy for reducing data transfer bandwidth and out-of-core memory footprint. Our multiscale representation allows for the extraction, analysis and display of structural features at variable spatial scales, while adaptive level-of-detail rendering methods make it possible to interactively explore large datasets within a constrained memory footprint. The quality and performance of our prototype system is evaluated on large structurally complex datasets, including gigabyte-sized micro-tomographic volumes.
projection-based appearances are used in a variety of computer graphics applications to impart different appearances onto physical surfaces using digitally controlled projector light. To achieve a compliant appearance, all points on the physical surface must be altered to the colours of the desired target appearance; otherwise, an incompliant appearance results in a misleading visualization. Previous systems typically assume to operate with compliant appearances or restrict themselves to the simpler case of white surfaces. To achieve compliancy, one may change the physical surface's albedo, increase the amount of projector light radiance available or modify the target appearance's colours. This paper presents an approach to modify a target appearance to achieve compliant appearance editing without altering the physical surface or the projector setup. Our system minimally alters the target appearance's colours while maintaining cues important for perceptual similarity (e.g. colour constancy). First, we discuss how to measure colour compliancy. Next, we describe our approach to partition the physical surface into patches based on the surface's colours and the target appearance's colours. Finally, we describe our appearance optimization process, which computes a compliant appearance that is as perceptually similar as possible to the target appearance's colours. We perform several real-world projection-based appearances and compare our results to naïve approaches, which either ignore compliancy or simply reduce the appearance's overall brightness.
We present a camera with switchable primaries using shiftable layers of color filter arrays (CFAs). By layering a pair of CMY CFAs in this novel manner we can switch between multiple sets of color primaries (namely RGB, CMY and RGBCY) in the same camera. In contrast to fixed color primaries (e.g. RGB or CMY), which cannot provide optimal image quality for all scene conditions, our camera with switchable primaries provides optimal color fidelity and signal to noise ratio for multiple scene conditions. Next, we show that the same concept can be used to layer two RGB CFAs to design a camera with switchable low dynamic range (LDR) and high dynamic range (HDR) modes. Further, we show that such layering can be generalized as a constrained satisfaction problem (CSP) allowing to constrain a large number of parameters (e.g. different operational modes, amount and direction of the shifts, placement of the primaries in the CFA) to provide an optimal solution. We investigate practical design options for shiftable layering of the CFAs. We demonstrate these by building prototype cameras for both switchable primaries and switchable LDR/HDR modes. To the best of our knowledge, we present, for the first time, the concept of shiftable layers of CFAs that provides a new degree of freedom in photography where multiple operational modes are available to the user in a single camera for optimizing the picture quality based on the nature of the scene geometry, color and illumination.
In this paper we present a novel technique for easily calibrating multiple casually aligned projectors on spherical domes using a single uncalibrated camera. Using the prior knowledge of the display surface being a dome, we can estimate the camera intrinsic and extrinsic parameters and the projector to display surface correspondences automatically using a set of images. These images include the image of the dome itself and a projected pattern from each projector. Using these correspondences we can register images from the multiple projectors on the dome. Further, we can register displays which are not entirely visible in a single camera view using multiple pan and tilted views of an uncalibrated camera making our method suitable for displays of different size and resolution. We can register images from any arbitrary viewpoint making it appropriate for a single head-tracked user in a 3D visualization system. Also, we can use several cartographic mapping techniques to register images in a manner that is appropriate for multi-user visualization. Domes are known to produce a tremendous sense of immersion and presence in visualization systems. Yet, till date, there exists no easy way to register multiple projectors on a dome to create a high-resolution realistic visualizations. To the best of our knowledge, this is the first method that can achieve accurate geometric registration of multiple projectors on a dome simply and automatically using a single uncalibrated camera.
Solid state drives (SSDs) are emerging as an alternative storage medium to HDDs. SSDs have performance characteristics (e.g., fast random reads) that are very different from those of HDDs. Because of the high performance of SSDs, there are increasingly more research efforts to redesign the established techniques that are optimized for HDDs, to work well with SSDs. In this paper we focus on computing cache-coherent layouts of large-scale models for SSDs. It has been demonstrated that cache-oblivious layouts perform well for various applications running on HDDs. However, computing cache-oblivious layouts for large-models is known to be very expensive. Also these layouts cannot be maintained efficiently for dynamically changing models. Utilizing the properties of SSDs we propose an efficient layout computation method that produces a page-based cache-aware layout for SSDs. We show that the performance of our layout can be maintained under dynamic changes on the model and is similar to the cache-oblivious layout optimized for static models. We demonstrate the benefits of our method for large-scale walkthrough scene editing and rendering, and collision detection.
In this paper, we present a novel technique to calibrate multiple casually aligned projectors on fiducial-free piecewise smooth vertically extruded surfaces using a single camera. Such surfaces include cylindrical displays and CAVEs, common in immersive virtual reality systems. We impose two priors to the display surface. We assume the surface is a piecewise smooth vertically extruded surface for which the aspect ratio of the rectangle formed by the four corners of the surface is known and the boundary is visible and segmentable. Using these priors, we can estimate the display's 3D geometry and camera extrinsic parameters using a nonlinear optimization technique from a single image without any explicit display to camera correspondences. Using the estimated camera and display properties, the intrinsic and extrinsic parameters of each projector are recovered using a single projected pattern seen by the camera. This in turn is used to register the images on the display from any arbitrary viewpoint making it appropriate for virtual reality systems. The fast convergence and robustness of this method is achieved via a novel dimension reduction technique for camera parameter estimation and a novel deterministic technique for projector property estimation. This simplicity, efficiency, and robustness of our method enable several coveted features for nonplanar projection-based displays. First, it allows fast recalibration in the face of projector, display or camera movements and even change in display shape. Second, this opens up, for the first time, the possibility of allowing multiple projectors to overlap on the corners of the CAVE-a popular immersive VR display system. Finally, this opens up the possibility of easily deploying multiprojector displays on aesthetic novel shapes for edutainment and digital signage applications
Ultra-portable projectors, called pico projectors, are now being embedded in mobile devices like cell-phones. Such mobile devices are projected to be the primary device to be used by younger people for ubiquitous sharing of all possible media initiating novel social interaction paradigms. Yet, the pico-projectors offer a much lower resolution and brightness than a standard projector. However, images displayed from multiple such mobile devices can be tiled to create a dramatically improved display in both brightness and resolution. This will allow multiple users to view and share media at a much higher quality. In this paper, we present a camera-based video synchronization algorithm that allows a federation of projection-enabled mobile devices to collaboratively present a synchronized video stream, though only a smaller part of the video comes from each device. Since, the synchronization does not use any wireless network infrastructure, it is independent of network congestion and connectivity. We combined our method with existing distributed registration techniques to demonstrate a synchronized video stream for a federation of four projectors arranged in a 2 × 2 array. To the best of our knowledge, this is the first time that a camera-based technique has been used to mitigate network uncertainties to achieve accurate video synchronization across multiple devices.
We present the first distributed paradigm for multiple users to interact simultaneously with large tiled rear projection display walls. Unlike earlier works, our paradigm allows easy scalability across different applications, interaction modalities, displays and users. The novelty of the design lies in its distributed nature allowing well-compartmented, application independent, and application specific modules. This enables adapting to different 2D applications and interaction modalities easily by changing a few application specific modules. We demonstrate four challenging 2D applications on a nine projector display to demonstrate the application scalability of our method: map visualization, virtual graffiti, virtual bulletin board and an emergency management system. We demonstrate the scalability of our method to multiple interaction modalities by showing both gesture-based and laser-based user interfaces. Finally, we improve earlier distributed methods to register multiple projectors. Previous works need multiple patterns to identify the neighbors, the configuration of the display and the registration across multiple projectors in logarithmic time with respect to the number of projectors in the display. We propose a new approach that achieves this using a single pattern based on specially augmented QR codes in constant time. Further, previous distributed registration algorithms are prone to large misregistrations. We propose a novel radially cascading geometric registration technique that yields significantly better accuracy. Thus, our improvements allow a significantly more efficient and accurate technique for distributed self-registration of multi-projector display walls.
Many visualization applications benefit from displaying content on real-world objects rather than on a traditional display (e.g., a monitor). This type of visualization display is achieved by projecting precisely controlled illumination from multiple projectors onto the real-world colored objects. For such a task, the placement of the projectors is critical in assuring that the desired visualization is possible. Using ad hoc projector placement may cause some appearances to suffer from color shifting due to insufficient projector light radiance being exposed onto the physical surface. This leads to an incorrect appearance and ultimately to a false and potentially misleading visualization. In this paper, we present a framework to discover the optimal position and orientation of the projectors for such projection-based visualization displays. An optimal projector placement should be able to achieve the desired visualization with minimal projector light radiance. When determining optimal projector placement, object visibility, surface reflectance properties, and projector-surface distance and orientation need to be considered. We first formalize a theory for appearance editing image formation and construct a constrained linear system of equations that express when a desired novel appearance or visualization is possible given a geometric and surface reflectance model of the physical surface. Then, we show how to apply this constrained system in an adaptive search to efficiently discover the optimal projector placement which achieves the desired appearance. Constraints can be imposed on the maximum radiance allowed by the projectors and the projectors' placement to support specific goals of various visualization applications. We perform several real-world and simulated appearance edits and visualizations to demonstrate the improvement obtained by our discovered projector placement over ad hoc projector placement.
In this paper, we present the first method to geometrically register multiple projectors on a swept surface (e.g. a truncated dome) using a single uncalibrated camera without using any physical markers on the surface. Our method can even handle non-linear distortion in projectors common in compact setups where a short throw lens is mounted on each projector. Further, when the whole swept surface is not visible from a single camera view, we can register the projectors using multiple pan and tilted views of the same camera. Thus, our method scales well with different size and resolution of the display. Since we recover the 3D shape of the display, we can achieve registration that is correct from any arbitrary viewpoint appropriate for head-tracked single-user virtual reality systems. We can also achieve wallpapered registration more appropriate for multi-user collaborative explorations. Our method achieves sub-pixel accuracy and the image correction required to achieve the registration runs in real-time on the GPU. Swept surfaces are much more immersive than popular display shapes like planes, cylinders and CAVES. Our method opens up the possibility of using such immersive swept surfaces to create more immersive VR systems without compromising the simplicity of having a completely automated registration technique.
A color transfer function describes the relationship between the input and the output colors of a device. Computing this function is difficult when devices do not follow traditionally coveted properties like channel independency or color constancy, as is the case with most commodity capture and display devices (like projectors, camerass and printers). In this paper we present a novel representation for the color transfer function of any device, using higher-dimensional Bézier patches, that does not rely on any restrictive assumptions and hence can handle devices that do not behave in an ideal manner. Using this representation and a novel reparametrization technique, we design a color transformation method that is more accurate and free of local artifacts compared to existing color transformation methods. We demonstrate this method’s generality by using it for color management on a variety of input and output devices. Our method shows significant improvement in the appearance of seamlessness when used in the particularly demanding application of color matching across multi-projector displays or multi-camera systems. Finally we demonstrate that our color transformation method can be performed efficiently using a real-time GPU implementation.
In this paper we present a novel technique to calibrate multiple casually aligned projectors on a fiducial-free cylindrical curved surface using a single camera. We impose two priors to the cylindrical display: (a) cylinder is a vertically extruded surface; and (b) the aspect ratio of the rectangle formed by the four corners of the screen is known. Using these priors, we can estimate the display's 3D surface geometry and camera extrinsic parameters using a single image without any explicit display to camera correspondences. Using the estimated camera and display properties, we design a novel deterministic algorithm to recover the intrinsic and extrinsic parameters of each projector using a single projected pattern seen by the camera which is then used to register the images on the display from any arbitrary viewpoint making it appropriate for virtual reality systems. Finally, our method can be extended easily to handle sharp corners - making it suitable for the common CAVE like VR setup. To the best of our knowledge, this is the first method that can achieve accurate geometric auto-calibration of multiple projectors on a cylindrical display without performing an extensive stereo reconstruction.
Recent work have shown that it is possible to register multiple projectors on non-planar surfaces using a single uncalibrated camera instead of a calibrated stereo pair when dealing with a special class of non-planar surfaces, vertically extruded surfaces. However, this requires the camera view to contain the entire display surface. This is often an impossible scenario for large displays, especially common in visualization, edutainment, training and simulation applications. In this paper we present a new method that can achieve an accurate geometric registration even when the field-of-view of the uncalibrated camera can cover only a part of the vertically extruded display at a time. We pan and tilt the camera from a single point and employ a multi-view approach to register the projectors on the display. This allows the method to scale easily both in terms of camera resolution and display size. To the best of our knowledge, our method is the first to achieve a scalable multi-view geometric registration of large vertically extruded displays with a single uncalibrated camera. This method can also handle a different situation of having multiple similarly oriented cameras in different locations, if the camera focal length is known.
A feature-oriented generic progressive lossless mesh coder (FOLProM) is proposed to encode triangular meshes with arbitrarily complex geometry and topology. In this work, a sequence of levels of detail (LODs) are generated through iterative vertex set split and bounding volume subdivision. The incremental geometry and connectivity updates associated with each vertex set split and/or bounding volume subdivision are entropy coded. Due to the visual importance of sharp geometric features, the whole geometry coding process is optimized for a better presentation of geometric features, especially at low coding bitrates. Feature-oriented optimization in FOLProM is performed in hierarchy control and adaptive quantization. Efficient coordinate representation and prediction schemes are employed to reduce the entropy of data significantly. Furthermore, a simple yet efficient connectivity coding scheme is proposed. It is shown that FOLProM offers a significant rate-distortion (R-D) gain over the prior art, which is especially obvious at low bitrates.
Demyelination- the loss of myelin sheath that insulates axons, is a prominent feature in many neurological disorders resulting in spinal cord injury (SCI). The lost myelin sheath can be replaced by remyelination, used in SCI treatment. In this paper, we propose an algorithm for efficient automated analysis of remyelination therapy. We use a robust, shape-independent algorithm based on iso-contour analysis of the image at progressively increasing intensity levels for detecting cell boundaries. The detected boundaries of spatially clustered cells are then separated using Delaunay triangulation based contour separation method. The therapeutically important oligodendrocyte-remyelinated axons (OR-axons) are identified and a density map is generated for efficacy analysis of the therapy. Our efficient automated remyelination analysis significantly reduces error due to human subjectivity. We corroborate the accuracy of our results by extensive cross-verification by the domain experts.
In this paper we propose a compact device-independent representation of the photometric properties of a camera, especially the vignetting effect. This representation can be computed by recovering the photometric parameters at sparsely sampled device configurations(defined by aperture, focal length and so on). More importantly, this can then be used for accurate prediction of the photometric parameters at other new device configurations where they have not been measured.
Emerging next generation digital light projectors are using multiple LED/laser sources instead of one white lamp. This results in a color gamut much larger than any existing display or capture device. Though advantageous in theory, when used to display contents captured/processed at a smaller gamut, a large gamut expansion results in hue-shift artifacts. We present a hardware-assisted 3D gamut reshaping method that handles the gamut expansion in LED based DLP displays by hierarchical temporal multiplexing of the multiple primaries. This, in turn, results in a color emulation technique by which projectors with such large gamuts can also achieve a standard color gamut and white point – the two most important color properties in terms of display quality, with an additional advantage of increased brightness and dynamic range. The same method can also be used for color balancing across multiple projectors that are often used to create large-scale high resolution displays.
Given a data layout of a large walkthrough scene, we present a novel and simple spatial hierarchy on the disk-pages of the layout that has notable advantages over a conventional spatial hierarchy on the scene geometry. Assume that each disk-page consists of a set of triangles whose bounding boxes are computed. A spatial hierarchy of the walkthrough space is constructed, not with the given scene, but with the bounding boxes of disk-pages. The leaf nodes of the spatial-hierarchy refer directly to the page numbers of the pages of the bounding box it contains. We call this hierarchy on the pages as the disk-page hierarchy. We also propose a self-contained diskpage format that would suit this data structure well. Further, we present a new cache-oblivious graph-based data layout algorithm called the 2-factor layout that would preserve the proximity and orientation properties of the primitives in the layout. Walkthrough experiments have been conducted on a city scene consisting of over 110M triangles. Our system renders this scene on a laptop within a one pixel projection error at over 20 fps with simple texture substitution based simplification of distant objects, and with no explicit data/cache management.
Imagine a world in which people can use their handheld mobile devices to project and transfer content. Everyone can join the collaboration by simply bringing their mobile devices close to each other. People can grab data from each others’ devices with simple hand gestures. Now imagine a large display created by tiling multiple displays where multiple users can interact with a large dynamically changing data set in a collocated, collaborative setting and the displays will take care of the data transfer and handling functions in a way that is transparent to the users. In this paper we present a novel data-handling display which works as not only a display device but also as an interaction and data transfer module. We propose simple gesture based solutions to transfer information between these data-handling modules. We achieve high scalability by presenting a fully distributed architecture in which each device is responsible for its own data and also communicates and collaborates with other devices. We also show the usefulness of our work in visualizing large datasets and at the same time allowing multiple users to interact with the data.
We propose a robust, feature preserving and user-steerable mesh sampling algorithm, based on the one-to-many mapping of a regular sampling of the Gaussian sphere onto a given manifold surface. Most of the operations are local and no global information is maintained. For this reason, our algorithm is amenable to a parallel or streaming implementation, and is most suitable in situations when it is not possible to hold all the input data in memory at the same time. Using -nets, we analyze the sampling method and propose solutions to avoid shortcomings inherent to all localized sampling methods. Further, as a byproduct of our sampling algorithm, a shape approximation is produced. Finally, we demonstrate a streaming implementation that handles large meshes with a small memory footprint.
We present a graph algorithm to find fundamental cycles aligned with the principal curvature directions of a surface. Specifically, we use the tree-cotree decomposition of graphs embedded in manifolds, guided with edge weights, in order to produce these cycles. Our algorithm is very quick compared to existing methods, with a worst case running time of O(nlogn+gn) where n is the number of faces and g is the surface genus. Further, its flexibility to accommodate different weighting functions and to handle boundaries may be used to produce cycles suitable for a variety of applications and models.
Shape simplification and re-sampling of underlying point-sampled surfaces under userdefined error bounds is an important and challenging issue. Based on the regular triangulation of the Gaussian sphere and the surface normals mapping onto the Gaussian sphere, a Gaussian sphere based re-sampling scheme is presented that generates a non-uniformly curvature-aware simplification of the given point-sampled model. Owing to the theoretical analysis of shape isophotic error metric for did that Gaussian sphere based sampling, the proposed simplification scheme provides a convenient way to control the re-sampling results under a user-specified error metric bound. The novel algorithm has been implemented and demonstrated on several examples.
In this paper, we present the first algorithm to geometrically register multiple projectors in a view-independent manner (i.e. wallpapered) on a common type of curved surface, vertically extruded surface, using an uncalibrated camera without attaching any obtrusive markers to the display screen. Further, it can also tolerate large non-linear geometric distortions in the projectors as is common when mounting short throw lenses to allow a compact set-up. Our registration achieves sub-pixel accuracy on a large number of different vertically extruded surfaces and the image correction to achieve this registration can be run in real time on the GPU. This simple markerless registration has the potential to have a large impact on easy set-up and maintenance of large curved multi-projector displays, common for visualization, edutainment, training and simulation applications.
We introduce a new color transformation representation based on higher-dimensional Bezier to maintain color consistency across non-linear devices (like commodity printers and displays) that yields more accurate and artifact-free results than existing methods. It is amenable to hardware implementation making it suitable for next generation commodity devices.
We introduce a new color transformation representation based on higher-dimensional Bezier fitting to maintain color consistency across non-linear devices (like commodity printers and displays). Our representation yields more accurate and artifact-free results than existing methods, and is amenable to hardware implementation, thus making it suitable for next generation commodity devices.
Imagine a world in which people can use their handheld mobile devices to project and transfer content. Everyone can join the collaboration by simply bringing their mobile devices close to each other. People can grab data from each others’ devices with simple hand gestures. Now imagine a large display created by tiling multiple displays where multiple users can interact with a large dynamically changing data set in a collocated, collaborative setting and the displays will take care of the data transfer and handling functions in a way that is transparent to the users. In this paper we present a novel data-handling display which works as not only a display device but also as an interaction and data transfer module. We propose simple gesture based solutions to transfer information between these data-handling modules. We achieve high scalability by presenting a fully distributed architecture in which each device is responsible for its own data and also communicates and collaborates with other devices. We also show the usefulness of our work in visualizing large datasets and at the same time allowing multiple users to interact with the data.
In this paper, we will describe the creation of shared visualization spaces which provides a natural means of sharing, interacting and working with the collection of media artifacts, which may not exist in totality in any one physical environment. The idea is to create a virtual collaborative space which takes inputs from multiple physical spaces. Each user environment has a projectorcamera system. The projector shows the Shared Visualization Space in each user’s environment and each user can make changes to it using natural hand gestures which are being captured by a camera.
In this paper, we propose a generic point cloud encoder that provides a unified framework for compressing different attributes of point samples corresponding to 3D objects with arbitrary topology. In the proposed scheme, the coding process is led by an iterative octree cell subdivision of the object space. At each level of subdivision, positions of point samples are approximated by the geometry centers of all treefront cells while normals and colors are approximated by their statistical average within each of tree-front cells. With this framework, we employ attribute-dependent encoding techniques to exploit different characteristics of various attributes. All of these have led to significant improvement in the rate-distortion (R-D) performance and a computational advantage over the state of the art. Furthermore, given sufficient levels of octree expansion, normal space partitioning and resolution of color quantization, the proposed point cloud encoder can be potentially used for lossless coding of 3D point clouds.
Study of contrast sensitivity of the human eye shows that our suprathreshold contrast sensitivity follows the Weber Law and, hence, increases proportionally with the increase in the mean local luminance. In this paper, we effectively apply this fact to design a contrast-enhancement method for images that improves the local image contrast by controlling the local image gradient with a single parameter. Unlike previous methods, we achieve this without explicit segmentation of the image, either in the spatial (multiscale) or frequency (multiresolution) domain. We pose the contrast enhancement as an optimization problem that maximizes the average local contrast of an image strictly constrained by a perceptual constraint derived directly from the Weber Law. We then propose a greedy heuristic, controlled by a single parameter, to approximate this optimization problem.
Multi-projector displays today are automatically registered, both geometrically and photometrically, using cameras. Existing registration techniques assume pre-calibrated projectors and cameras that are devoid of imperfections such as lens distortion. In practice, however, these devices are usually imperfect and uncalibrated. Registration of each of these devices is often more challenging than the multi-projector display registration itself. To make tiled projection-based displays accessible to a layman user we should allow the use of uncalibrated inexpensive devices that are prone to imperfections. In this paper, we make two important advances in this direction. First, we present a new geometric registration technique that can achieve geometric alignment in the presence of severe projector lens distortion using a relatively inexpensive low-resolution camera. This is achieved via a closed-form model that relates the projectors to cameras, in planar multi-projector displays, using rational Bezier patches. This enables us to geometrically calibrate a 3000×2500 resolution planar multi-projector display made of 3×3 array of nine severely distorted projectors using a low resolution (640 × 480) VGA camera. Second, we present a photometric self-calibration technique for a projector-camera pair. This allows us to photometrically calibrate the same display made of nine projectors using a photometrically uncalibrated camera. To the best of our knowledge, this is the first work that allows geometrically imperfect projectors and photometrically uncalibrated cameras in calibrating multi-projector displays.
Sampling of 3D meshes is at the foundation of any surface simplification technique. In this paper, we use the recent results on quantization and surface approximation theory to propose a simple, robust, linear time, output sensitive algorithm for sampling meshes with the purpose of surface approximation. Our algorithm is based on the mapping of regular sampling and triangulation of the Gaussian sphere onto a manifold surface. An interesting aspect of our algorithm is that we do not explicitly measure, minimize, or prioritize any error to simplify and do not explicitly cluster the faces to find proxies, but still achieve bounded error approximation of the shape
Tiled displays provide high resolution and large scale simultaneously. Projectors can project on any available surface. Thus, it is possible to create a large high-resolution display by simply tiling multiple projectors on any available regular surface. The tremendous advancement in projection technology has made projectors portable and affordable. One can envision displays made of multiple such projectors that can be packed in one's car trunk, carried from one location to another, deployed at each location easily to create a seamless high-resolution display, and, finally, dismantled in minutes to be taken to the next location — essentially a pack-and-go display. Several challenges must be overcome in order to realize such pack-and-go displays. These include allowing for imperfect uncalibrated devices, uneven non-diffused display surfaces, and a layman user via complete automation in deployment that requires no user invention. We described the advances we have made in addressing these challenges for the most common case of planar display surfaces. First, we present a technique to allow imperfect projectors. Next, we present a technique to allow a photometrically uncalibrated camera. Finally, we present a novel distributed architecture that renders critical display capabilities such as self-calibration, scalability, and reconfigurability without any user intervention. These advances are important milestones towards the development of easy-to-use multi-projector displays that can be deployed anywhere and by anyone.
Multi-projector displays provide a large-scale and a high-resolution view at the same time. Today, projectors are commodity products, making such displays are affordable, but several challenges must be overcome in order to make this technology available to everyday users. We present several advances in the area of multiprojector displays. These employ projectors with enhanced capabilities, methodologies for efficiently and inexpensively calibrating and maintaining displays composed of these units, and the ability to handle imperfect and uncalibrated devices when performing geometric and photometric registration of imagery across multiple projectors. These are the first steps in realizing the goal of truly ubiquitous pixels, where users can be completely oblivious to the type and number of devices providing the display.
In this paper, we present a method for photometric selfcalibration of a projector-camera system. In addition to the input transfer functions (commonly called gamma functions), we also reconstruct the spatial intensity fall-off from the center to fringe (commonly called the vignetting effect) for both the projector and camera. Projector-camera systems are becoming more popular in a large number of applications like scene capture, 3D reconstruction, and calibrating multi-projector displays. Our method enables the use of photometrically uncalibrated projectors and cameras in all such applications.
This paper presents seamless tiled displays via completely distributed network of projector-camera systems that calibrates itself without any user intervention. This makes projection-based tiled displays very easy to deploy and maintain. The decentralized calibration methodology presented to achieve this also enables advanced capabilities like scalability, reconfigurability and fault-tolerance.
Centralized techniques have been used until now when automatically calibrating (both geometrically and photometrically) large high-resolution displays created by tiling multiple projectors in a 2D array. A centralized server managed all the projectors and also the camera(s) used to calibrate the display. In this paper, we propose an asynchronous distributed calibration methodology via a display unit called the plug-and-play projector (PPP). The PPP consists of a projector, camera, computation and communication unit, thus creating a self-sufficient module that enables an asynchronous distributed architecture for multi-projector displays. We present a single-program-multiple-data (SPMD) calibration algorithm that runs on each PPP and achieves truly scalable and reconfigurable displays without any input from the user and instruments novel capabilities like adding/removing PPPs from the display dynamically, detecting faults, and reshaping the display to a reasonable rectangular shape to react to the addition/removal/faults. To the best of our knowledge, this is the first attempt to realize a completely asynchronous and distributed calibration architecture and methodology for multi-projector displays.
Study of contrast sensitivity of the human eye shows that our contrast discrimination sensitivity follows the weber law for suprathreshold levels. In this paper, we apply this fact effectively to design a contrast enhancement method for images that improves the local image contrast by controlling the local image gradient. Unlike previous methods, we achieve this without segmenting the image either in the spatial (multiscale) or frequency (multi-resolution) domain. We pose contrast enhancement as an optimization problem that maximizes the average local contrast of an image. The optimization formulation includes a perceptual constraint derived directly from human suprathreshold contrast sensitivity function. Then, we propose a greedy heuristic, controlled by a single parameter, to approximate this optimization problem. The results generated by our method is superior to existing techniques showing none of the common artifacts of contrast enhancements like halos, hue shift, and intensity burn-outs.
Representing a triangulated two manifold using a single triangle strip is an NP-complete problem. By introducing a few Steiner vertices, recent works find such a single-strip, and hence a linear ordering of edge-connected triangles of the entire triangulation. In this paper, we extend previous results [10] that exploit this linear ordering in ef- ficient triangle-strip management for high-performance rendering. We present new algorithms to generate single-strip representations that follow different user defined constraints or preferences in the form of edge weights. These functional constraints are application dependent; For example, normalbased constraints can be used for efficient rendering after visibility culling, or spatial constraints for highly coherent vertex-caching. We highlight the flexibility of this approach by generating single-strips with preferences as arbitrary as the orientation of the edges. We also present a hierarchical single-strip management strategy for high-performance interactive 3D rendering.
The single triangle-strip loop generation algorithm on a triangulated two-manifold presented by Gopi and Eppstein [4] is based on the guaranteed existence of a perfect matching in its dual graph. But such a perfect matching is not guaranteed in the dual graph of triangulated manifolds with boundaries. In this paper, we present algorithms that suitably modify the results of the dual graph matching to generate a single strip loop on manifolds with boundaries. Further, the algorithm presented in [4] can produce only strip loops and not linear strips. We present an algorithm that does topological surgery to construct linear strips with user-specified start and end triangles on manifolds with or without boundaries. The main contributions of this paper include graph algorithms to handle unmatched triangles, reduction of the number of Steiner vertices introduced to create strip loops, and finally a novel method to generate single linear strip with arbitrary start and end triangles.
We present a novel cost function to prioritize points and subsample a point set based on the dominant geometric features and local sampling density of the model. This cost function is easy to compute and at the same time provides rich feedback in the form of redundancy and non-uniformity in the sampling. We use this cost function to simplify the given point set and thus reduce the CSRBF (Compactly Supported Radial Basis Function) coefficients of the surface fit overthis pointset. Further compression of CSRBF data set is effected by employing lossy encoding techniques on the geometry of the simplified model, namely the positions and normal vectors, and lossless encoding on the CSRBF coefficients. Results on the quality of subsampling and our compression algorithms are provided. The major advantages of our method include highly efficient subsampling using carefully designed, effective, and easy compute cost function, in addition to a very high PSNR (Peak Signal to Noise Ratio) of our compression technique relative to other known point set subsampling techniques.
We propose a generic point cloud encoder that compresses geometry data including positions and normals of point samples corresponding to 3D objects with arbitrary topology. In this work, the coding process is led by an iterative octree cell subdivision of the object space. At each level of subdivision, positions of point samples are approximated by the geometry centers of all tree-front cells while normals are approximated by their statistical average within each of the tree-front cells. With this framework, we employ attribute-dependent encoding techniques to exploit different characteristics of various attributes. As a result, significant improvement in the rate-distortion (R-D) performance has been obtained with respect to the prior art. Furthermore, the proposed point cloud encoder can be potentially used for lossless geometry coding of 3D point clouds, given sufficient levels of octree expansion and normal space partitioning.
In order to achieve seamless imagery in a planar multiprojector display, geometric distortions and misalignment of images within and across projectors have to be removed. Camera-based calibration methods are popular for achieving this in an automated fashion. Previous methods for geometric calibration fall into two categories: (a) Methods that model the geometric function relating the projectors to cameras using simple linear models, like homography, to calibrate the display. These models assume perfect linear devices and cannot address projector non-linearities, like lens distortions, which are common in most commodity projectors. (b) Methods that use piecewise linear approximations to model the relationship between projectors and cameras. These require a dense sampling of the function space to achieve good calibration. In this paper, we present a new closed-form model that relates projectors to cameras in planar multi-projector displays, using rational Bezier patches. This model overcomes the shortcomings of the previous methods by allowing for projectors with significant lens distortion. It can be further used to develop an efficient and accurate geometric calibration method with a sparse sampling of the function.
Arguably, the most vexing problem remaining for multi-projector displays is that of photometric (brightness) seamlessness within and across different projectors. Researchers have strived for strict photometric uniformity that achieves identical response at every pixel of the display. However, this goal typically results in displays with severely compressed dynamic range and poor image quality. In this paper, we show that strict photometric uniformity is not a requirement for achieving photometric seamlessness. We introduce a general goal for photometric seamlessness by defining it as an optimization problem balancing perceptual uniformity with display quality. Based on this goal, we present a new method to achieve perceptually seamless high quality displays. We first derive a model that describes the photometric response of projection-based displays. Then we estimate the model parameters and modify them using perception-driven criteria. Finally, we use the graphics hardware to reproject the image computed using the modified model parameters by manipulating only the projector inputs at interactive rates. Our method has been successfully demonstrated on three different practical display systems at Argonne National Laboratory, made of 2 × 2 array of four projectors, 2 × 3 array of six projectors and 3 × 5 array of fifteen projectors. Our approach is efficient, automatic and scalable – requiring only a digital camera and a photometer. To the best of our knowledge, this is the first approach and system that addresses the photometric variation problem from a perceptual stand point and generates truly seamless displays with high dynamic range.
The concept of tiled displays can be successful only if such displays are made to look like a single display perceptually. The two issues that need to be solved to achieve this goal are geometric correction and color seamlessness of images spanning across tiles. Geometric correction algorithms borrow pin-hole camera models to model projector display geometry. In this paper, we introduce an abstract modeling function that describes the color seen by a viewer when displayed by a display device. Though this function can be used to model color displayed by any common display device, in this paper, we use it to model color in multi-projector display systems. We use the model to explain the reasons for different types of color variations in a multi-projector display, to compare different color correction algorithms, and to derive such algorithms directly from the model.
Multi-projector, large-scale displays are used in scientific visualization, virtual reality and other visually intensive applications. In recent years, a number of camera-based computer vision techniques have been proposed to register the geometry and color of tiled projectionbased display. These automated techniques use cameras to “calibrate” display geometry and photometry, computing per-projector corrective warps and intensity corrections that are necessary to produce seamless imagery across projector mosaics. These techniques replace the traditional labor-intensive manual alignment and maintenance steps, making such displays cost-effective, flexible, and accessible. In this paper, we present a survey of different camerabased geometric and photometric registration techniques reported in the literature to date. We discuss several techniques that have been proposed and demonstrated, each addressing particular display configurations and modes of operation. We overview each of these approaches and discuss their advantages and disadvantages. We examine techniques that address registration on both planar (video walls) and arbitrary display surfaces and photometric correction for different kinds of display surfaces. We conclude with a discussion of the remaining challenges and research opportunities for multi-projector displays.
Study of contrast sensitivity of the human eye shows that we are more sensitive to brightness differences at low intensity levels than at high intensity levels. We apply this fact effectively to achieve brightness seamlessness in multi-projector displays. Multi-displays, popularly made of a rectangular array of partially overlapping projectors, show severe spatial variation in brightness. Existing methods achieve brightness uniformity across the display by matching the brightness response of every pixel to the pixel with the most limited contrast leading to severe compression in the contrast of the display. In this paper, we propose a method that allows a constrained variation in brightness guided by the human contrast sensitivity function such that it is imperceptible to the human eye and this achieves seamless multidisplay. At the same time, such a constrained smoothing of the brightness leads to dramatic improvement in the contrast of the display making it practically usable.
An interesting issue of contemplation amongst researchers working on multi-projector displays is whether spatial superresolution can be achieved by overlapping images from multiple projectors. This paper presents a thorough theoretical analysis to answer this question using signal processing and perturbation theory. Our analysis is supported by results from a simulated overlapping projector display. This analysis shows that achieving spatial super-resolution using overlapping projectors is practically infeasible.
In order to find a 2-factor of a graph, there exist O(n 1.5 ) deterministic algorithm [7] and O(n 3 ) randomized algorithm [14]. In this paper, we propose novel O(n log3 n log log n) algorithms to find a 2-factor, if one exists, of a graph in which all n vertices have degree four or less. Such graphs are actually dual graphs of quadrilateral and tetrahedral meshes. A 2-factor of such graphs implicitly defines a linear ordering of the mesh primitives in the form of strips. Further, by introducing a few additional primitives, we reduce the number of tetrahedral strips to represent the entire tetrahedral mesh, and represent the entire quad-surface using a single quad-strip.
In this paper we explore the algorithmic space in which stripification, simplification and geometric compression of triangulated 2-manifolds overlap. Edge-collase/uncollapse based geometric simplification algorithms develop a hierarchy of collapses such that during uncollapse the reverse order has to be maintained. We show that restricting the simplification and refinement operations only to, what we call, the collapsible edges creates hierarchyless simplification in which the operations on one edge can be performed independent of those on another. Although only a restricted set of edges is used for simplification operations, we prove topological results to show that, with minor retriangulation, any triangulated 2-manifold can be reduced to either a single vertex or a single edge using the hierarchyless simplification, resulting in extreme simplification. The set of collapsible edges helps us analyze and relate the similarities between simplification, stripification and geometric compression algorithms. We show that the maximal set of collapsible edges implicitly describes a triangle strip representation of the original model. Further, these strips can be effortlessly maintained on multiresolution models obtained through any sequence of hierarchyless simplifications on these collapsible edges. Due to natural relationship between stripification and geometric compression, these multi-resolution models can also be efficiently compressed using traditional compression algorithms. We present algorithms to find the maximal set of collapsible edges and reorganization of these edges to get minimum number of connected components of these edges. An order-independent simplification and refinement of these edges is achieved by our novel data structure and we show the results of our implementation of view-dependent, dynamic, hierarchyless simplification. We maintain a single triangle strip across all multi-resolution models created by the view-dependent simplification process. We present a new algorithm to compress the models using the triangle strips implicitly defined by the collapsible edges.
Representing a triangulated two manifold using a single triangle strip is an NP-complete problem. By introducing a few Steiner vertices, recent works find such a single-strip and hence a linear ordering of edge-connected triangles of the entire triangulation. In this paper, we highlight and exploit this linear order in effi- cient triangle-strip management for high-performance rendering. We present new algorithms to generate weighted single-strip representations that respect different constraint-based clustering of triangles. These functional constraints can be application dependent; for example, normal-based constraints for efficient visibility culling or spatial constraints for highly coherent vertex-caching. We also present a hierarchical single-strip-management strategy for highperformance interactive 3D rendering.
This work addresses the issue of generating free-form surfaces using a 2D sketch interface. As the first step in this process, we develop a methodology to sketch 3D space curves from 2D sketches. Since the inverse projection from 2D sketches to 3D curve or surface is a one to many function, there is no unique solution. Hence we propose to interpret the given 2D curve to be the projection of the 3D curve that has minimum curvature among all the candidates in 3D. We present an algorithm to efficiently find a close approximation of this minimum curvature 3D space curve. In the second step, this network of curves along with the boundary information are given to the surface fitting method to generate free-form surfaces.
Space curve sketching using 2D user interface is a challenging task and forms the foundation for almost all sketch based modeling systems. Since the inverse projection from 2D to 3D is a one to many function, there is no unique solution. In this paper, we propose to interpret the given 2D curve to be the projection of the 3D curve that has minimum curvature among all the candidates in 3D. We present an elegant algorithm to efficiently find a close approximation of this minimum curvature 3D space curve. We also compare with many other 2D to 3D curve generation techniques to show that the curve generated by our method is the one of the most “intuitive” and easy to compute. Fixing the space curve to be the minimum curvature curve does not restrict the user from getting high-curvature space curve, if desired. We analyze the complete behavior of our space curve generation algorithm in order to provide simple sketching rules to achieve the curve that the user wants, even the high-curvature space curve, with no change in our algorithm.
In this paper, we propose a method for obtaining a textured billboards representation of a static scene, given a sequence of calibrated video images. Each billboard is a textured and partially transparent plane into which the input images are mapped using perspective projection. Binning using Hough transform is used to find the position of the billboards, and optic flow measures are used to determine their textures. Since these billboards are correct only from specific view-points, view-dependent rendering is used to choose and display appropriate billboards to reproduce the input.
Multi-projector displays exhibit severe luminance variation. Matching luminance at every pixel to the pixel with most limited luminance range leads to severe degradation in the dynamic range of the display, rendering it practically useless. We present a new optimization technique to manage the luminance in a smooth constraint manner to maintain a perceptual seamlessness while improving the dynamic range of the display dramatically.
We present a technique that achieves local contrast enhancement by representing it as an optimization problem. For this, we first introduce a scalar objective function that estimates the average local contrast of the image; to achieve the contrast enhancement, we seek to maximize this objective function subject to strict constraints on the local gradients and the color range of the image. The former constraint controls the amount of contrast enhancement achieved while the latter prevents over or under saturation of the colors as a result of the enhancement. We propose a greedy iterative algorithm, controlled by a single parameter, to solve this optimization problem. Thus, our contrast enhancement is achieved without explicitly segmenting the image either in the spatial (multi-scale) or frequency (multi-resolution) domain. We demonstrate our method on both gray and color images and compare it with other existing global and local contrast enhancement techniques.
Large-area displays made up of several projectors show significant variation in color. In this paper, we identify different projector parameters that cause the color variation and study their effects on the luminance and chrominance characteristics of the display. This work leads to the realization that luminance varies significantly within and across projectors while chrominance variation is relatively small, especially across projectors of same model. To address this situation, we present a method to achieve luminance matching across all pixels of a multiprojector display that results in photometrically uniform displays. We use a camera as measurement device for this purpose. Our method comprises a one-time calibration step that generates a per channel per projector luminance attenuation map (LAM), which is then used to correct any image projected on the display at interactive rates on commodity graphics hardware. To the best of our knowledge, this is the first effort to match luminance across all the pixels of a multiprojector display.
Triangle strips have been widely used for efficient rendering. It is NP-complete to test whether a given triangulated model can be represented as a single triangle strip, so many heuristics have been proposed to partition models into few long strips. In this paper, we present a new algorithm for creating a single triangle loop or strip from a triangulated model. Our method applies a dual graph matching algorithm to partition the mesh into cycles, and then merges pairs of cycles by splitting adjacent triangles when necessary. New vertices are introduced at midpoints of edges and the new triangles thus formed are coplanar with their parent triangles, hence the visual fidelity of the geometry is not changed. We prove that the increase in the number of triangles due to this splitting is 50% in the worst case, however for all models we tested the increase was less than 2%. We also prove tight bounds on the number of triangles needed for a single-strip representation of a model with holes on its boundary. Our strips can be used not only for efficient rendering, but also for other applications including the generation of space filling curves on a manifold of any arbitrary topology.
In this paper we introduce a method to represent a given triangular model using a single triangle strip. Since this problem is NP-complete, we break the limitation by splitting adjacent triangles when necessary. The common edge is split at the mid-point, and the newly formed triangles are coplanar with their parent triangles. Hence the resulting geometry of the model is visually and topologically identical to the original triangular model. Our method can develop any edge-connected oriented 2-manifold of arbitrary topology, with or without boundary, into a single strip. Our stripification method can be controlled to start and end at triangles incident on specific vertices. Further, an acyclic set of edges of the input model can be marked as “constraint edges” and our method can generate a single strip that does not cross over these edges, but still cover the whole model
An important problem in achieving seamless multi-projector the camera. Hence. comparing the image captured by the camera with the input digital image does not make sense. - .. displays is the photometric (luminance) variation across the display. Currently, evaluation of the improvement in the quality of the imagery achieved by applying different photometric correction methods and a comparison between them is done by a subjective human user. In this paper, we present a camera-based method for relative evaluation of the luminance properties of the imagery on multi-projector displays, both before correction and after correction using different compensation techniques. To the best of our knowledge. this is the first camera-based method designed to evaluate the different photometric correction methods on a multiprojector display. We show that using our simple and aulomatic evaluation scheme the optimal photometric compensation parametcr for a multi-projcctor display can be identified, which is not possible by subjective evaluation.
Photometric variation in multi-projector displays is arguably the most vexing problem that needs to be addressed to achieve seamless tiled multi-projector displays. In this paper, we present a scalable real-time solution to correct the spatial photometric variation in multi-projector displays. A digital camera is used to capture the intensity variation across the display in a scalable fashion. This information is then used to generate a per-pixel attenuation and offset map for each projector. The former achieves intensity scaling while the later compensates for differing black levels at different regions of the display. These maps are then used to modify the input to the projectors to correct the photometric variation. The contributions of our photometric capture and the correction method are the follwing. First, despite the limited resolution of the digital camera, this method is scalable to very high resolution displays. Second, unlike previous methods, this does not require an explicit knowledge of the location of the boundaries of the projectors for generating the attenuation and offset maps. Third, this compensates for the varying black level of the display. Finally, the correction is implemented in real time for OpenGL applications using the distributed rendering framework of Chromium
Transparency in 3D graphics has traditionally been created by ordering the transparent objects from back-to-front with respect to the viewpoint, and rendering the opaque objects first and then the transparent objects in the prescribed order. This has three major disadvantages: need for splitting intersecting polygons, repeated ordering for varying view points, and finally, incorrect transparency at the regions near silhouettes. The first two problems are eliminated by order independent transparency rendering techniques. The goal of this paper is to eliminate the third disadvantage also. Silhouettes look more opaque than the rest of the regions of the model. We call this silhouette-opaque transparency rendering. We use the alpha value as a probabilistic measure, similar to other order independent methods. We differ from the traditional methods by using this probabilistic measure in object space rather than in image space to render the transparency in silhouettes correctly. We call our technique to achieve silhouette-opacity as object-space screen-door transparency.
We introduce PixelFlex2, our newest scalable wall-sized, multi-projector display system. For it, we had to solve most of the difficult problems left open by its predecessor, PixelFlex, a proof-of-concept demonstration driven by a large, multi-headed SGI graphics system. PixelFlex2 retains the achievements of PixelFlex (high-performance through single-pass rendering, single-pixel accuracy for geometric blending with only casual placement of projectors), while adding a) higher performance and scalability with a Linux PC-cluster, b) application support with either the distributed-rendering framework of Chromium or a performance-oriented, parallel-process framework supported by a proprietary API, c) improved geometric calibration by using a corner finder for feature detection, and d) photometric calibration with a single conventional camera using high dynamic range imaging techniques rather than an expensive photometer.
Large-area multi-projector displays show significant spatial variation in color, both within a single projector’s field of view and across different projectors. Recent research in this area has shown that the color variation is primarily due to luminance variation. Luminance varies within a single projector’s field of view, across different brands of projectors and with the variation in projector parameters. Luminance variation is also introduced by overlap between adjacent projectors. On the other hand, chrominance remains constant throughout a projector’s field of view and varies little with the change in projector parameters, especially for projectors of the same brand. Hence, matching luminance response of all the pixels of a multi-projector display should help us to achieve photometric uniformity. In this paper, we present a method to do a per channel per pixel luminance matching. Our method consists of a onetime calibration procedure when a luminance attenuation map (LAM) is generated. This LAM is then used to correct any image to achieve photometric uniformity. In the onetime calibration step, we first use a camera to measure the per channel luminance response of a multi-projector display and find the pixel with the most “limited” luminance response. Then, for each projector, we generate a per channel LAM that assigns a weight to every pixel of the projector to scale the luminance response of that pixel to match with the most limited response. This LAM is then used to attenuate any image projected by the projector. This method can be extended to do the image correction in real time on traditional graphics pipeline by using alpha blending and color look-up-tables. To the best of our knowledge, this is the first effort to match luminance across all the pixels of a multi-projector display. Our results show that luminance matching can indeed achieve photometric uniformity.
Large area multi-projector displays are becoming increasingly popular for scientific visualization and virtual reality applications. Though most of the geometric calibration issues for such displays have been addressed in the past, the color variation across the display is still not addressed in a comprehensive fashion. The two major components of this color variation are the color non-uniformity within a single projector’s field of view termed as intra-projector variation and the variation across different projectors termed as inter-projector variations.
Tiled displays systems built by combining the images from arrays of projectors can provide a huge number of pixel elements to applications needing to visually represent a large amount of information, including a range of scientific visualization applications and high-fidelity collaborative environments. With commodity projectors and COTS rendering clusters, these large-area displays have the potential to provide very high performance and high resolution at low per pixel cost. But it is difficult to create the illusion of a unified seamless display for a variety of reasons. These include small misalignments of the projectors, optical distortion of the individual projector images because of imperfections in the lenses, intensity variation from projector to projector and across the field of each single projector, and color differences between projectors. Argonne has been researching methods for addressing these limitations, in particular, geometry and color. These methods are briefly discussed below. Both methods produce correction values that can be applied to the projected images to create a single seamless image. For these techniques to become truly useful, however, they must be integrated into existing infrastructure so that all applications and users can benefit from them. Such integration is the subject of the rest of this paper.
In this paper, we present simple rendering techniques implemented using traditional graphics hardware to achieve the effects of charcoal drawing. The effects include characteristics of charcoal drawings like broad grainy strokes and smooth tonal variations that are achieved by smudging the charcoal by hand. Further, we also generate the closure effect that is used by artists at times to avoid hard silhouette edges. All these effects are achieved using contrast enhancement operators on textures and/or colors of the 3D model. Our contribution lies in unifying the methods to achieve these effects under the common framework of contrast enhancement operators. Further, since the effects have been implemented using traditional graphics hardware, a single rendering pass is sufficient to create different effects. Hence, we can render highly complex models with large number of triangles at interactive rates. Thus, our method is especially suited for applications like scientific visualization and preliminary sketches/animations.
We present a fast and memory efficient algorithm that generates a manifold triangular mesh ✂ with or without boundary passing through a set of unorganized points ✄✆☎✞✝✠✟ with no other additional information. Nothing is assumed about the geometry or topology of the sampled manifold model, except for its reasonable smoothness. The speed of our algorithm is derived from a projection-based approach we use to determine the incident faces on a point. Our algorithm has successfully reconstructed the surfaces of unorganized point clouds of sizes varying from 10,000 to 100,000 in about 3–30 seconds on a 250 MHz, R10000 SGI Onyx2. Our technique can be specialized for different kinds of input and applications. For example, our algorithm can be specialized to handle data from height fields like terrain and range scan, even in the presence of noise. We have successfully generated meshes for range scan data of size 900,000 points in less than 40 seconds.
Surface reconstruction algorithms can ensure correctness of the reconstructed surface by imposing conditions on sampling and by assuming that the given point set satisfies the prescribed conditions. Traditionally, such sampling conditions impose minimum required sampling density for correct reconstruction. Let us assume that no additional information is provided to the reconstruction algorithm other than the input set of points. Under this assumption, we show that imposing minimum required sampling density is not sufficient to ensure correct reconstruction of surfaces with boundaries. In other words, to prove the correctness of the reconstructed surface, either the sampling conditions have to be strengthened or more information has to be provided to the reconstruction algorithm. Further, we prove that the strengthened sampling condition for sampling surfaces with boundaries has a unique property called non-monotonicity. We analyze the applicability of this framework to various algorithms like medial axis estimation algorithms, curve reconstruction algorithms, and reconstruction of surfaces from noisy point sets.
In this paper, we present simple rendering techniques implemented using traditional graphics hardware to achieve the effects of charcoal drawing. The effects include characteristics of charcoal drawings like broad grainy strokes and smooth tonal variations that are achieved by smudging the charcoal by hand. Further, we also generate the closure effect that is used by artists at times to avoid hard silhouette edges. All these effects are achieved using contrast enhancement operators on textures and/or colors of the 3D model. Our contribution lies in unifying the methods to achieve these effects under the common framework of contrast enhancement operators. Further, since the effects have been implemented using traditional graphics hardware, a single rendering pass is sufficient to create different effects. Hence, we can render highly complex models with large number of triangles at interactive rates. Thus, our method is especially suited for applications like scientific visualization and preliminary sketches/animations.
We present some ideas and demonstrations for a hybrid projectorbased rendering and display technique we call Computer Graphics Optique. Instead of partially overlapping projected images to achieve a wide-area display, we completely overlap projected images on top of each other to achieve the addition of light and color in an “optical composition buffer.” The idea is to use the optical composition to replace some analytical computation, to increase rendering speed, gain flexibility, intensity range, and intensity resolution. In addition one can make use of electronic and optical projector controls such as focus augmented with the optical superposition to achieve effects that are otherwise computationally expensive. We believe that this technique offers the possibility of a new paradigm for combined rendering and projector-based display.
Surface sampling and reconstruction are used in modeling objects in graphics and digital archiving of mechanical parts in Computer Aided Design and Manufacturing (CAD/CAM). Sampling involves collecting 3D points from the surface. Using these point samples, a reconstruction process rebuilds a surface that is topologically equivalent and geometrically close to the original surface. Conditions are imposed on sampling to ensure correct reconstruction. For a special case of manifolds, there are theoretically sound algorithms for sampling and reconstruction. The sampling conditions for such algorithms impose a minimum required sampling density (maximum distance between close samples) to ensure correct reconstruction. In this dissertation, I study the sampling and reconstruction of manifolds with boundaries. For this class of surfaces, I show that the conditions on minimum required sampling density are not sufficient to ensure correct reconstruction if only the point samples are given as input to the reconstruction process. Additional information like the smallest boundary size in a model, though sufficient to ensure correct reconstruction, imposes uniform sampling density throughout the model. In this dissertation, I propose a novel way to use the variation in the sampling density across the surface to encode the presence of a boundary. A sampling condition is proposed based on this approach, and the reconstruction process requires no additional information other than the input set of sample points. The reconstruction algorithm presented in this dissertation for reconstructing manifolds with or without boundaries is shown to be correct, efficient, and easy to implement.
Large area tiled displays are gaining popularity for use in collaborative immersive virtual environments and scientific visualization. While recent work has addressed the issues of geometric registration, rendering architectures, and human interfaces, there has been relatively little work on photometric calibration in general, and photometric non-uniformity in particular. For example, as a result of differences in the photometric characteristics of projectors, the color and intensity of a large area display varies from place to place. Further, the imagery typically appears brighter at the regions of overlap between adjacent projectors. In this paper we analyze and classify the causes of photometric non-uniformity in a tiled display. We then propose a methodology for determining corrections designed to achieve uniformity, that can correct for the photometric variations across a tiled projector display in real time using the per channel color look-up-tables (LUT) available in all image generators.
We present a fast, memory efficient algorithm that generates a manifold triangular mesh S passing through a set of unorganized points P R3. Nothing is assumed about the geometry, topology or presence of boundaries in the data set except that P is sampled from a real manifold surface. The speed of our algorithm is derived from a projection-based approach we use to determine the incident faces on a point. We define our sampling criteria to sample the surface and guarantee a topologically correct mesh after surface reconstruction for such a sampled surface. We also present a new algorithm to find the normal at a vertex, when the surface is sampled according our given criteria. We also present results of our surface reconstruction using our algorithm on unorganized point clouds of various models
This paper presents a new algorithm for immersive teleconferencing which addresses the problem of registering and blending multiple images together to create a single seamless panorama In the immersive teleconference paradigm one frame of the teleconference is a panorama that is constructed from a compound image sensing device These frames are rendered at the remote site on a projection surface that surrounds the user creating an immersive feeling of presence and participation in the teleconference Our algorithm efficiently creates panoramic frames for a teleconference session that are both geometrically registered and intensity blended We demonstrate a prototype that is able to capture images from a compoundimage sensor register them into a seamless panoramic frame and render those panoramic frames on a projection surface at frames per second Intro
We present a new approach for simplifying models composed of rational spline patches. Given an input model, the algorithm computes a new approximation of the model in terms of cubic triangular Bezier patches. It performs a series of geometric operations, consisting of patch merging and swapping diagonals, and makes use of patch connectivity information to generate C-LODs (curved levelsof-detail). Each C-LOD is represented using cubic triangular Bezier patches. The C-LOD's provide a compact representation for storing the model. We also present techniques to quantify the error introduced by our algorithm. Given the C-LODs, the tessellation algorithms can generate the spline model's polygonal approximations using static and dynamic tessellation schemes. The simplification algorithm has been implemented and we highlight its performance on a number of models.
In this paper, we present an efficient algorithm for contact determination between spline models. We make use of a new hierarchy, called ShellTree, that comprises ofspherical shells and oriented bounding boxes. Each spherical shell corresponds to a portion of the volume between two concentric spheres. Given large spline models, our algorithm decomposes each surface into Bézier patches as part of pre-processing. At runtime it dynamically computes a tight fitting axis-aligned bounding box across each Bézier patch and efficiently checks all such boxes for overlap. Using off-line and on-line techniques for tree construction, our algorithm computes ShellTrees for Bézier patches and performs fast overlap tests between them to detect collisions. The overall approach can trade off runtime performance for reduced memory requirements. We have implemented the algorithm and tested it on large models, each composed of hundred of patches. Its performance varies with the configurations of the objects. For many complex models composed of hundreds of patches, it can accurately compute the contacts in a few milliseconds.
We present a new approach for simplifying models composed of polygons or spline patches. Given an input model, the algorithm computes a new representation of the model in terms of triangular Bezier ´ patches. It performs a series of geometric operations, consisting of patch merging and swapping diagonals, and makes use of patch connectivity information to generate C-LODs (curved levelsof-detail). EachC-LOD is represented using cubic triangular Bezier ´ patches. The C-LOD's provide a compact representation for storing the model. The algorithm tries to minimize the surface deviation error and maintains continuity at patch boundaries. Given the CLOD's, the algorithm can generate their polygonal approximations using static and dynamic tessellation schemes. It has been implemented and we highlight its performance on a number of polygonal and spline models.