Located in Yuci, Shanxi Province, the Chang Family Manor was first built during the Qianlong and Jiaqing reigns of the Qing Dynasty. With a well-organized layout and rigorous structure, it covers over 120,000 square meters. As the largest merchant courtyard of the Shanxi merchants and the most extensive manor-style architectural complex in China, it fully embodies the grandeur and dignity of a Confucian merchant household.
How to introduce this historically rich manor to the public in a more vivid and engaging way through new interactive technologies became the core focus of the Erudite project team.
Unlike traditional digital guides or basic scene reconstructions, the Erudite project centered around AR (Augmented Reality) technology, integrating digitized artifacts with original storytelling to create a unique interactive experience. Through immersive narratives and interactive mechanisms, users can explore artifacts, historical stories behind them, and Shanxi merchant culture from the Ming and Qing dynasties—all from a first-person perspective without setting foot in the Chang Manor. With cultural heritage as its core, the project seeks to modernize its presentation, aiming to demonstrate the value of heritage through innovative storytelling and interaction design.
The project originated from Jingyang and his teammates’ exploration of AR technology during a classroom exercise. At the time, he was enrolled in an innovative theory course taught by Liqi Ren, the Associate Director for Innovation and Entrepreneurship, where students were encouraged to propose creative ideas and collaborate to turn them into practical outcomes. For their final project, Jingyang and two teammates developed an AR reader prototype, using a chapter from “Harry Potter and the Philosopher’s Stone” as an experimental subject. Readers could scan the text with their phone cameras, and visual elements described in the passage—such as the Philosopher’s Stone, broomsticks, or the castle map—would appear on screen, making the reading experience more interactive and visually engaging. Encouraged by Ren’s recognition and support, the team decided to develop this classroom project further and apply for 2022 College Student Innovation and Entrepreneurship Training Program (DaChuang).
Based on her understanding of interaction design in the museum and cultural heritage field, Ren suggested the team consider using cultural heritage as the project’s focus. She noted that Shanxi province is rich in historical sites, and that AR technology could offer new possibilities for digital preservation and storytelling. Taking her advice, the team chose to focus on integrating AR with heritage, and successfully applied for the Dachuang program.
They conducted extensive research: collecting case studies of AR, VR, and XR applications in museums, referencing academic papers, reviewing news articles, and analyzing user feedback from social media platforms. After multiple rounds of research and brainstorming, the team selected Chang Family Manor as their subject and began designing a customized AR digital guide system.
In the initial design, the team envisioned combining GPS positioning with AR, allowing users to scan real-world scenes during an on-site visit to view overlaid digital content. As Jingyang explained:
“We wanted to create a concept of layered time and space—when you walk through the present-day Chang Manor, your phone screen reveals how that spot looked in the past. Characters from historical stories might appear to guide you to specific locations, and as you walk there in real life, you’d uncover more narrative content we created.”
The team developed several prototypes to test the system’s guiding functions. However, due to pandemic restrictions delaying their fieldwork, they first applied the concept to the DKU campus, developing a real-world AR navigation system to use as a design reference for the Chang Manor version. This campus guide was able to lead users along a preset path using their real-time location, introducing buildings and their functions along the way.
It is a fully functional version of the guide. According to their original plan, they only needed to complete field data collection at Chang Manor and reuse the same technical framework to launch the final version. But after visiting Chang Manor and other cultural institutions, the team realized that AR-based digital guides were already widely used—and often underwhelming in execution.
Many museums had purchased AR headsets, but visitors typically saw only a few short, non-interactive videos poorly aligned with the physical environment.
While such solutions were valid given budget constraints and technical accessibility, the Erudite team wasn’t satisfied. As an innovation-focused group, they wanted more than just static overlays—they aimed for immersive scene reconstruction and meaningful interactivity.
Driven by this mindset, the team reassessed their direction, deciding to move beyond site-specific digital guides and instead develop an interactive storytelling game based on cultural heritage. This would allow users to engage with the experience anywhere, without being tied to a physical location.
They wrote an original mystery story, integrated it with AR technology, and designed a multi-branch interactive narrative game. Players could scan specific AR markers with their phones to trigger scenes and storylines, making choices at key points to influence how the story unfolds.
A demo was presented at DKU’s Performance Café, where the team arranged several small AR markers. Scanning them triggered AR content that brought Chang Manor’s artifacts and environments to life on-screen. All 3D models were digitized from the manor and optimized for mobile performance.
Many students and faculty offered positive feedback, and Jingyang reflected: “Of course, it’s nothing like a commercial game, but I think our interaction logic is fun enough.”
Looking back, Jingyang said that all the core team members learned a lot throughout the process. From initial research, narrowing the focus, selecting a theme, to enduring rounds of development work to make their vision real—the project journey involved disagreements, adaptation, and eventually true collaboration.
For him, the true value of the Erudite project lies in the research and practice process itself.
Although Erudite has come to a close, Jingyang and his teammates are far from done: “What we’ve done may just be a form—but what we’re really pursuing is the deeper research question behind it.”
