Intensity Control of Projectors in Parallel – A Doorway to an Augmented Reality Future

A challenge to adopting augmented reality (AR) in wider applications is working with dynamic objects, owing to a delay between their movement and the projection of light onto their new position. But, Tokyo Tech scientists may have a workaround. They have developed a method that uses multiple projectors while reducing delay time. Their method could open the door to a future driven by AR, helping us live increasingly technology-centered lives.

Technological advancements continue to redesign the way we interact with digital media, the world around us, and each other. Augmented reality (AR), which uses technology to alter the perception of objects in the real world, is unlocking unprecedented landscapes in entertainment, advertising, education, and across many other industries. The use of multiple projectors plays an important role in expanding the usage of AR, alongside a technique called projection mapping. However, an obstacle to the widespread adoption of AR is the application of this method to moving, or "dynamic," targets without the loss of immersion in the AR space.

This technique, known as dynamic projection mapping, relies on a blend of cameras and projectors that visually detect target surfaces and project onto them, respectively. A critical aspect is the need for high speed in information transfer and low "latency," or delay between detection and projection. Any latency leads to a misalignment of the projector, which effects our perception, and reduces the effectiveness of the AR space.

Other issues like changes in shadowing and target overlap are solved easily by using multiple projectors. However, the addition of new projectors correspondingly drives up the latency. This is a result of the need to calculate the intensity at every pixel simultaneously for every frame of a moving scene. Simply put, more projectors lead to longer and more complex calculations. The latency is a massive hurdle towards AR taking a true foothold in broader applications across society.

Thankfully, a team of scientists at Tokyo Institute of Technology (Tokyo Tech), led by Associate Professor Yoshihiro Watanabe, might just have the necessary solution. They have developed a novel method to calculate the intensity of each pixel on a target in parallel, reducing the need for a single large optimization calculation. Their method relies on the principle that if pixels are small enough, they can be evaluated independently. While based on an approximation, their results, published in IEEE Transactions on Visualization and Computer Graphics, suggest that they could achieve the same quality of images as conventional, more computationally expensive methods, while drastically increasing the mapping speed and thereby reducing the latency.

1. The weight, or importance, of a given pixel is used to control the projector intensity on each pixel. The weights are assigned depending on the area of the surface that is illuminated. Pixels with a smaller projection area on the surface are controlled to have larger weight, whereas pixels with a larger area are controlled to have smaller weight.

"Another advantage of our proposed method is, as there is no longer need for a single global calculation, it allows the use of multiple rendering computers connected through a network, each only controlling a single projector," explains Dr. Watanabe. "Such a network system is easily customizable to incorporate more projectors, without major sacrifices to the latency."

This new method can allow large spaces with many projectors for efficient dynamic projection mapping, taking us a step closer to broader AR applications, as Dr. Watanabe describes: "The presented high-speed multi-projection is expected to be a major part of important base technologies that will advance spatial AR to derive more practical uses in our daily life."

We are excited to see how this technology evolves and how it illuminates the path towards an AR-centric future!

• Reference

Authors :	Takashi Nomoto1, Wanlong Li1, Hao-Lun Peng1, and Yoshihiro Watanabe1
Title of original paper :	Dynamic Multi-projection Mapping Based on Parallel Intensity Control
Journal :	IEEE Transactions on Visualization and Computer Graphics
Conference :	IEEE VR Conference
DOI :	10.1109/TVCG.2022.3150488
Affiliations :	1 Tokyo Institute of Technology