How Augmented Reality Can Make Objects Seem Heavier

A Japanese research team has demonstrated that augmented reality (AR) can influence a person's weight perception. AR technology superimposes computer-generated images (CGI) onto real-world scenes.

The team focused on studying the size-weight illusion (SWI). This is a phenomenon where individuals perceive smaller objects as heavier when lifting objects of equal weight. Their findings were published in the Journal of Digital Life, a renowned electronic journal for academic papers in the digital field.

The team was led by Professor Takahiro Higuchi of Tokyo Metropolitan University. Joining him from the university were Assistant Professor Ryo Watanabe and students Koki Yokoyama and Takuma Umemori. Sony Group Corporation researchers Junki Inoue (also a visiting researcher at the university), Tsuyoshi Ishikawa, and Takanobu Omata were also part of the team.

The year 2016 was considered the "year of virtual reality." This is when companies like Sony Interactive Entertainment and Facebook (now Meta) introduced VR devices to the market. Although global shipments experienced a temporary dip, the domain of software and applications, including AR, has been steadily expanding.

In February 2023, Sony released PlayStation VR2. In June, Meta announced the upcoming release of Meta Quest 3. While major players were launching new VR devices, Apple had been hesitant to enter the market. This changed when it unveiled "Apple Vision Pro," set to be released in 2024. These developments have revitalized the VR, AR, and mixed reality (MR) industries.

The Size-Weight Illusion

Researchers have been exploring how technologies with significant impact on vision, like AR, can be used to "trick the brain." Humans primarily rely on visual input when receiving sensory information. In one study, AR was used to make cookies appear larger. As a result of this, participants felt more full when eating them.

The research team led by Professor Higuchi hypothesized that AR technology could influence weight perception. Therefore, the experiment focused on a phenomenon called the "size-weight illusion."

"We have a preconception that larger objects tend to be heavier and smaller objects tend to be lighter. Using this, the brain makes predictions about an object's weight," Professor Higuchi explains.

"Consequently, when holding objects of different sizes but with the same weight, the anticipation can lead to a misalignment between our expectations and actual sensations. This makes smaller objects feel heavier."

The Experiment

Participants wore head-mounted displays with cameras, allowing them to observe their surroundings through real-time video. Instead of seeing the real object, they saw virtual images that were superimposed on them. The real object that was placed on a table in front of them was either a cube (6 cm) or a bigger cuboid (6 cm × 9 cm × 6 cm). The virtual object was also either a cube (7cm) or a bigger cuboid (7 cm x 10.5 cm x 7 cm) but with different dimensions. Participants could move the objects and "view" them from different perspectives, thanks to the sensors on the head-mounted display.

The researchers performed dummy trials between the main trials, using virtual objects of various sizes. This was done to prevent the participants from learning the objects' weight beforehand. In the main trials, the participants were instructed to lift either the cube or the cuboid using a vertical handle. Unbeknownst to the participants, both objects had an identical weight of 210 grams (0.5 pounds).

Small Objects Perceived as Heavier

Professor Higuchi hypothesized, "If [visual] information about the size of the CGI affects weight prediction, then SWI will occur based on the size of the superimposed CGI. This would apply regardless of the shape or size of the object being lifted."

To test this hypothesis, the team tested three combinations.

(1) A small virtual object and a small real object
(2) A large virtual object and a small real object
(3) A large virtual object and a large real object

After analyzing data from 22 participants, they observed that participants tended to perceive smaller virtual objects (1) as heavier than larger virtual objects (2 and 3).

The team concluded that SWI occurred when they changed the size of the virtual object, irrespective of the size of the real object. This demonstrated that it was "possible to manipulate a person's weight perception by changing the visual features of a virtual object in an AR environment."

Enhancing Gameplay

Similar experiments have been conducted overseas using semi-transparent virtual objects. In these cases, changing the size of virtual objects in an AR environment did not change weight perception.

Regarding this discrepancy, the research team explains that "to manipulate weight perception, the participants need to see the CGI alone." Weight perception couldn't be manipulated using semi-transparent virtual images because the participants received conflicting visual information.

Additionally, changing the size of the virtual object right before or after the participants lifted the object did not cause a statistically significant illusion. The same was observed when the participants' vision was blocked by showing a black screen, preventing them from seeing their surroundings while lifting the objects.

Professor Higuchi believes that the study's results could have promising applications in the entertainment industry, especially in AR games and experiential entertainment. These include AR games in which players use controllers as weapons or tools. Adjusting the virtual objects superimposed on the controller to match the scene could make gameplay more realistic and immersive.