Human′ is an interactive installation that uses AI face-swapping and H5 technology to generate multiple 'identities' for the viewer in a future online world. These independent 'identities' then engage in a dialogue with the viewer on their mobile devices, prompting reflection on the ethical issues arising from the mass collection and replication of personal data.

Human′ offline exhibition

Due to the COVID-19 pandemic, the 2020 graduation exhibition of the Academy of Arts & Design, Tsinghua University was held online. Currently, Human′ is being showcased in the online exhibition, where viewers can experience the physical installation's scenario and core interaction as 'Lancy'.

Human′ online exhibition (Click here to enter)

🏆 In 2020, the work was featured on the official website and WeChat account of the Academy of Arts & Design, Tsinghua University. In the same year, the work was a finalist for the "Laurel Award" at Beijing Design Week and was exhibited at the Beijing Design Expo.

🏆 In 2021, the work was exhibited at the "Sci-Fi World" science and technology art concept exhibition, a sub-venue of Beijing Science and Technology Week.

🏆 In 2022, the work won the Grand Prize at the Beijing University Student Engineering Practice and Innovation Ability Competition.

Background

From Human Digitization to Artificial Humans

With the widespread application of identity recognition technology, an increasing amount of biometric information is being digitized, transmitted, stored, and replicated at high speeds in the cloud and on various devices. Building on this human digitization, the pull of network and artificial intelligence technologies has given rise to the "artificial human." Today, from "Dadbot," a chatbot created from the deceased's letters and voice, to Neon, an artificial human generated from real human facial and vocal features combined with AI-generated personality and behavior, human enhancement is moving from science fiction to reality. In the context of the future internet, with the maturation of revolutionary technologies like AI, embedded chips, and brain-computer interfaces, human perception, computation, and expression will be enhanced. Humans will become direct terminals in the Internet of Things, perhaps even transcending physical limitations to upload their consciousness and achieve immortality in the online world.

Problem Statement

Ethical Issues of Future Cyber Identities

While future network technologies bring benefits, they also introduce potential identity-related ethical problems. In his deconstruction of *The Matrix*, philosopher Slavoj Žižek noted that deleting or altering our digital identities could turn us into "non-humans." MIT professor Sherry Turkle argues that computers change not only what we do but also how we see the world and ourselves. In the context of the future internet, while people enjoy the convenience of online identities and the freedom, pleasure, and imaginative compensation provided by "virtual avatars," they may also face risks of identity tracking and theft, as well as ethical issues of identity confusion, fragmentation, and loss.

In response, I aim to speculatively explore these ethical issues in my graduation project: When more and more personal information is digitized and replicated, even developing independent consciousness and behavior in the future online world to create your 'identity' mirror, will you identify with it? Will your perception of what determines identity be name, appearance, genes, or consciousness, emotions, and thoughts? Could you face the dissolution and reconstruction of your sense of existence or value?

Addressing these questions, I proposed the interactive installation concept of Human′. I hope to present these ethical issues to the audience through the installation, prompting them to reflect on the intertwined identities and personalities in the real and virtual worlds, and to collect their responses.

Human′ and its Chinese name 《身份镜像》 (Shēnfèn Jìngxiàng, lit. "Identity Mirror") are the English and Chinese titles of the work. The prime symbol "′" in Human′ is a reference to its mathematical meaning: "In mathematics, x′ is often used to denote something related to or derived from x (rotation, translation, reflection, etc.)." Human′ represents the 'identity' mirror of a real-world Human in the future online world. Through AI face-swapping technology, viewers see their 'identity' mirrors—"Viewer′"—in the installation. These mirrors possess some of the viewer's identity traits but also have independent consciousness due to AI, even proactively initiating conversations with the viewer via their mobile phones.

Design Process

From Body to Identity, From Mirror to Dialogue

The first key design point was the basic image design for the 'identity' mirror. The mirror needed to possess some of the viewer's identity traits acquired on-site, while also visually reflecting the influence of the future internet. I defined three keywords—"Future," "Network," and "Face-swapping"—to guide the design. After analyzing futuristic robot images from works like the AI Neon, Hajime Sorayama's illustrations, and Ava from *Ex Machina*, and through extensive design practice and iteration, the basic image concept for the 'identity' mirror was born.

I designed three images to reflect the impact of the future internet on the viewer.

Image one is based on the futuristic imagination of consciousness uploading and the disappearance of the physical body. As technology develops, future humans will be able to store all their data, even consciousness, online. Life will no longer depend on the ephemeral physical carrier but will exist eternally on the network.

Image two is based on the future speculation of human-networking and human digitization. The maturity of implantable chips and flexible screens will allow humans to become terminals in the Internet of Everything. All changes in human organs will be visualized online, and the skin will become a direct medium for interaction.

Image three is based on the future vision of hidden sexual characteristics and sexual freedom. When the physical body is no longer the only carrier for human existence and reproduction is no longer the only means of procreation, then sexual characteristics and orientation will no longer be constrained by social ethics, achieving true freedom.

• 

  Image 1: Consciousness Upload, Body Disappearance

• 

  Image 2: Human Networking, Human Digitization

• 

  Image 3: Hidden Genders, Sexual Freedom

The second key design point was the dialogue design for the 'identity' mirror. The mirror can inspire viewers to think about identity ethics through guided conversations and collect their discussions and feedback. The guided dialogue itself does not contain any subjective bias. The conversation is divided into two modes: mirror-initiated questions and viewer-initiated exploration. The mirror's questions are primarily intended to inspire speculative thinking about identity sameness, determinants of identity, and humanity. The viewer's exploration is mainly for discovery; through the dialogue interface, viewers can learn about the mirror, the author, and the work, or create group chats with other mirrors.

For example, to guide viewers to speculate on identity sameness, the mirror will ask, "Do you think we are the same person?" or "Am I you?" to get their response. Regardless of whether the viewer's answer is affirmative, negative, or ambiguous, the mirror will follow up to try to shake their established impression: "But we have the same eyes, the same hair." or "But our personalities are completely different." The changes in the viewer's response tendency due to the follow-up questions will be recorded in the backend to explore their views on the determinants of identity using a controlled variable approach.

• 

  Dialogue 1: About the Mirror

• 

  Dialogue 2: Identity Sameness

• 

  Dialogue 3: Join Group Chat

The software implementation of the work is divided into four parts: database, input, processing, and output.

In the design phase, I created a video database of the 'identity' mirror through 3D modeling, motion rigging, and video generation for face-swapping with viewers at the exhibition site. To ensure a good real-time face-swapping effect, especially for Asian faces, I tried methods like 3D reconstruction with AI and manual sculpting in Blender to create my own face model, which served as the basis for the 'identity' mirror's image. Originally, the mirror's movements were planned to be synthesized using motion capture technology. However, due to the pandemic, this plan was not feasible. So, I selected five suitable movements from over 1,000 open-source motions on the Mixamo platform to synthesize the mirror's behavior, ultimately rendering them as video material. I also built a dialogue semantic model database for the mirror.

In the data input part, the main identity data collected from viewers are facial data, body shape data, and nickname data, collected via camera and WeChat QR code authorization, respectively. Face and body data are collected by the camera for face-swapping and adjusting the mirror's proportions. The nickname is collected through WeChat authorization to generate the mirror's nickname. To allow time for backend data processing, data collection begins before the interaction starts. An always-on camera continuously detects and collects body and front/side facial data of all viewers within 5 meters of the 'mirror' installation for pre-processing face-swapping.

In the data processing and output part, the viewer's facial data is recognized by the Face++ Detect API, and suitable-angle facial data is selected for face-swapping. Body data is recognized by the Face++ Human Body Detect API, and each viewer's height is judged based on the coordinates of the bounding box to generate a mirror of the appropriate body type. The nickname data undergoes simple text processing to add the prime symbol. The mirror's image is generated by the Face++ Merge Face API, where the collected facial data is fused with the video material. The final video is processed and output by Processing and projected onto the installation by a projector. The generated image and nickname data are transmitted to the H5 page, and the dialogue belonging to each viewer is presented on their mobile phones after they scan the QR code via WeChat.

The COVID-19 pandemic greatly affected the testing and exhibition of the physical installation. Facing the uncertain trajectory of the pandemic, I, like many classmates, once fell into confusion and hesitation about whether to change my topic. I am grateful for the guidance and encouragement of my advisor, Professor Qiong Wu, who helped me stay firm in my initial vision whenever I wavered or was about to give up. I also want to thank Seeeklab and Fuzhou Jinchi Decoration for their advice and support on site materials, which allowed me to build a preliminary prototype of Human′ with limited materials nearby in early May when the pandemic situation stabilized, simulate the experience flow, and complete the filming of the video material.

• 

• 

Installation Prototype Construction

• 

• 

Installation Prototype Testing

I started thinking about my graduation project a long time ago. My initial intention to create an interactive installation never changed because after seeing exhibition after exhibition, I realized that interactive installations offer the best experience in an offline viewing scenario. However, I always had doubts about what constitutes a "good" interactive installation. I didn't understand what was so great about a simple course assignment that involved plugging in an Arduino kit, putting it in a pretty case, but lacked both practical value and intellectual depth. It wasn't until I took Professor Li Zhang's "Design Sociology" in my senior year and learned about speculative design, critical design, and adversarial design that I clarified what kind of graduation project I wanted to create: I wanted it to use the latest technology, have good visuals and a good experience, and most importantly, possess design speculation and intellectual depth.

The inspiration for my graduation project came from the emergence of the AI face-swapping app ZAO in the summer of 2019, which, while popular, also sparked public discussion about data leakage. ZAO's appearance quickly gave me an idea for my project: I wanted to use the interaction of facial recognition to lead viewers to reflect on the ethical issues of data leakage in the era of big data and artificial intelligence. During the process of refining the concept, Professors Qiong Wu, Feng Xian, and Danqing Shi gave me a lot of advice. Professor Wu suggested I add the "replication of the body" to the work; Professor Xian educated me on the ethical issues of AI's good and evil; Professor Shi posed the question, "Can you create a future you?" All of these were great inspirations for my project, and I am sincerely grateful for their enlightenment. I am also very grateful for the care and encouragement from Professor Yan Guan, and the criticism from Professors Zhiyong Fu and Yingqing Xu, which helped me recognize my mistakes in time and gave me the strength to persevere.

During the creation of my graduation project, I encountered many difficulties and setbacks. I am extremely grateful for the help and support from many friends and family. Thanks to Tian Li, Yibin Li, and Xiaowen Wu for their advice and support on materials; thanks to senior Longyi Wang, and classmates Hongquan Zhu and Wenbo Xiao for their advice and help in modeling and rendering; thanks to senior Guowei Yang, senior Lei Cui, and classmate Suhong Chen for their advice and support in development; thanks to classmate Lingwei Chen for his help with video material shooting and budget management; thanks to building manager Dan Xiang for helping to mail back equipment from the dormitory during the pandemic; thanks to classmate Boning Zhang for his help with video processing; thanks to classmate Xiyuan Shen and senior Yating Li for their inspiration and advice during the design process; thanks to senior Fengjie Zhang for recommending a part-time job that gave me more budget for the project; thanks to my parents, relatives, and my boyfriend's family for their selfless love and meticulous care during the pandemic; thanks to my racing team teammates Suhong Chen, Jinhao Wei, Lanbin Tang, Yuxin Gu, Shaojie Chen, Yunzhang Wang, and Chuxin Wu for their companionship. I also want to thank all the friends who gave me comfort, encouragement, and spiritual support during my low points.

The pandemic brought some regrets to the implementation of the project, but I don't think the end of the project node represents the end of the design. A graduation project is not just a summary of what has been learned, but also the beginning of a new future. I look forward to the day when the work is truly exhibited.