NXT _ 5HM09
Shanghai Museum of Glass NXT is an immersive hyperrealistic realtime-3D experience for education and cultural entertainment. Shanghai Museum of Glass in the real world is a renowned cultural destination, offering over 20,000 square meters of exhibition space and featuring more than 6,000 exhibits and interactive installations across art, design, science, and technology.
NXT is the initial move to digitalize the entire museum collection, laying the groundwork for a sustained commitment to delivering state-of-the-art visitor experiences in the forthcoming digital era. It merges the physical and virtual environments, empowers visitors to interact with art, and fosters social connections. All these elements together curate a deeply introspective and participatory cultural journey, elevating museum experiences to the next level.
Accessible through designated stations at the physical museum, or via VR headsets with the portable master station, NXT offers visitors an unhindered virtual environment to explore and engage with exhibits at their own pace. Art objects are recreated with precision and intricacy. Interactive installations are digitally adapted with a fantastical touch. Historical photographs or events are manifested as live scenes.
In the physical museum, visitors observe the intricate details, vibrant colors, and captivating reflections of glass objects. The digital museum offers dynamic, interactive experiences that extend the storytelling possibilities and the reach of the art collection.
Beyond displaying its art, NXT’s curation highlights the unexpected connections that exist within the diverse collection, and prompts visitors to contemplate broader themes related to the future, nature, and social topics.
Another commitment of NXT is to foster social connections. Visitors in the real world can observe others navigating the virtual tour on display screens, while in the virtual realm, each visitor is represented by a digital avatar, allowing for interaction and a social experience. Furthermore, NXT is populated with glass-figure NPCs, who share their “Transparent Thoughts” with the visitors when approached, creating a sense of community and shared discovery.
The digital interface offers a dynamic visitor experience, while the tangible museum offers a steadfast environment for its art collection. This dual approach minimizes the need for frequent renovations, thus optimizing resource utilization and reducing environmental impact.
Type:
Realtime-3D Museum
3D-Engine:
Unreal Engine
Extent:
3,925 m2 (Phase 1)
Year:
2023
Idea, Curation and Design:
Tilman Thürmer
Game Design:
David Keohane
Character Design:
Siyi Liu
Yusi Li
Level Design:
David Keohane
Sound Design:
David Keohane
Modelling:
Tommy Jiang
Yusi Li
Siyi Liu
Yeili Wu
Ding Xu
Game Development:
David Keohane
Programming:
David Keohane
Game Testing:
Cathal Keohane, Leo Keohane, Tanita Thürmer, Chen Ying
Content Development:
Tilman Thürmer
Yichun Chen
Yusi Li
Content Development Support:
ChatGPT-4
Voiceover AI:
Eleven Labs
Genny
SoVoiceover Design Managemnent:
Bon Wen
Communication:
Jiali Xuan
Original In-Game Footage
Overview Phase 1 (5HM09)
SHMOG NXT aims to deliver a visitor experience that blends and blurs the boundary of the real and the virtual worlds. The digital spaces are envisioned as integral components of the museum, equal in importance to the physical halls, to achieve a “Phygital Balance”.
In the physical museum, visitors can experience firsthand the process of live glass making, feel the heat from the kilns, and marvel at the intricate details, vibrant colors, and captivating reflections of physical glass objects – aspects that cannot be fully replicated in the digital world.
Simultaneously, the digital museum offers dynamic, interactive experiences that extend the storytelling possibilities and the reach of the art collection. Its low-threshold, content-rich environments provide a flexibility that allows for fast, environmentally friendly changes and updates.
In essence, the dual strategy aims to create a harmonious balance between the physical and digital spaces, where each enhances the other, providing a comprehensive, immersive experience for all visitors.
The Shanghai Museum of Glass houses a varied art collection that includes contemporary glass art, conceptual design experiments, hands-on material science displays, ancient artifacts, and insightful dialogues on social and ethical issues.
NXT evaluate each category and devises strategies for their optimal digital integration.
Art objects are recreated with precision and intricacy. Interactive installations are digitally adapted with a more fantastical touch. Historical photographs or events are manifested as live scenes, some of them made interactive for visitors to travel back in time to experience firsthand.
NXT features glass-figure NPCs, some representing museum staff while the rest depict fellow visitors. Each character has a unique personality and is linked with a backstory.
Upon approaching these glass figures, visitors of NXT will hear “Transparent Thoughts”, which are random reflections from the NPCs. These thoughts could be about the museum, a particular exhibit, or even personal issues.
The NPCs and their “Transparent Thoughts” create a sense of community and shared discovery. The goal is for visitors to explore exhibits and content not just as isolated individuals, but as part of a vibrant, interconnected community.
1. “I think, I need to do more research on the history of glass and its impact on art and culture. Hmm, the museum restaurant is not open yet, duh!” Ruina, 32, documentary filmmaker
2. “I need to remember that there’s more to life than just getting into a good university. Why does society value success over happiness? Maybe I should start valuing my mental health over good grades.” Feifei, 15, high school student
3. A glass-figure museum staff NPC is adjusting an audio-visual device.
4. “I wonder if they have any glass pieces that match my outfit? I hope my followers appreciate the effort I put into finding cool places to visit.” Yanni, 36, social media influencer
5. A bilingual cat speaking to a walkie-talkie in Shanghai dialect and Gaelic.
6. “I feel like a kid in a candy store…except it’s a glass museum and I’m an adult with a job.” Lidan, 36, museum staff greeting visitors behind reception desk.
7. Two glass-figure NPCs appreciating the intricate details of glass trees in the main hall atrium.
8. The glass-figure NPC “Astronaut” is break dancing.
9. “I wish there was a formula for success, like the chemical composition of glass. But maybe success is more of an art than a science.” Feifei, 15, high school student
All images are unretouched and show original in-game footage.
The digital world of SHMOG NXT is based on the real physical environment of the Shanghai Museum of Glass.
All elements in the virtual environment, including buildings, interiors, displays, and exhibits, have been meticulously re-created based on the physical layout and design of the real Shanghai Museum of Glass to provide visitors with a fully immersive and realistic 3D experience.
SHMOG NXT uses Unreal Engine 5.3, The world’s most advanced real-time 3D creation tool for photorealistic visuals and immersive experiences.
SHMOG NXT simulates the exploration of a photorealistic museum environment from the perspective of the visitor in real time.
For SHMOG NXT, 3D experts have meticulously created digital versions of every piece in the Shanghai Museum of Glass’ permanent collection. Due to the reflective nature of glass, each piece must be individually hand-modeled as 3D scanning is not a viable option.
SHMOG NXT uses Unreal Engine’s Lumen system to control light in the museum. Lumen is a powerful global illumination and reflections system that provides dynamic light control, infinite diffuse interreflection bounces, and large scale indirect specular reflections. This makes it ideal for glass objects, which require precise light control to highlight their transparent properties and details. By using Lumen, SHMOG NXT is able to create a highly realistic and visually stunning digital representation of the glass exhibits in the museum.
Procedural audio design metasound in Unreal Engine enhances the immersive experience in SHMOG NXT by providing precise and dynamic control over sound sources. The system generates audio DSP graphs to manage all aspects of audio rendering, creating next-generation, procedural audio experiences that bring exhibits to life.
The Blueprint Visual Scripting system in Unreal Engine is a comprehensive gameplay scripting solution that utilizes a node-based interface to create gameplay elements directly within Unreal Editor.
This system offers unparalleled flexibility and power, allowing designers to access a vast array of concepts and tools typically only available to programmers. Additionally, the Blueprint-specific markup available in Unreal Engine’s C++ implementation empowers programmers to create baseline systems that can be expanded upon by designers.
The example on the right, shows C++ coding and Blueprint scripting for SHMOG NXT’s ‘H0U53 0F R3FL3C710N5’.
