Virtual Abe Lincoln & the Reality of C-3P0: Digital Beings & Synthetic Personas as Virtual Influencers in Spatial Computing
Image source: (WP:NFCC#4), Fair use, C-3PO and USS Callister
Synthetic Personas in our Holodecks
In our previous coverage of the incoming holodecks and the future of history, we discussed using these systems as a new storytelling mechanism for Hollywood’s content as well as a way to document the present as future content for historical re-enactments. Students of the future will be able to experience our lives and media and current events as immersive, interactive content. In fact, we already can do this to a limited degree. In addition to the emerging edtech markets hoping to capitalize on this trend, world governments are leaning into these new technologies to communicate with their citizens. Take, for instance, the Israeli president’s Independence Day speech in April 2020. President Reuven Rivlin didn’t just get up on stage and have video cameras project his voice and image to watchers; he was volumetrically recorded, and the “living” 3D avatar of him can be re-experienced again and again in augmented reality, standing in your living room. If you missed it, don’t worry—you can still watch it—directly in augmented reality or as a YouTube video of someone else watching it in augmented reality.
Just as we re-watch our favorite films and television shows, we’ll be able to relive those events that have been faithfully recorded in 3D. We’ll also likely re-create all the events that pre-date our ability to capture the real thing. Even though no one had miniaturized LiDAR on their phones (or phones themselves) at the Gettysburg Address, that isn’t going to stop someone from building out a 3D avatar of Lincoln and having a voice actor record the Gettysburg Address so people can experience a simulation of the historic event.
We do historical re-enactments all the time. There are physical re-enactment societies and there are TV shows and even some more accurate drunk history shows. Theoretically, we have learned to take history with a grain of salt (along the rim of our margarita glass). Spatial computing will confuse that issue because a digital twin of the world and its people will have both faithful recordings from present day onwards, and recreations from anything before. And as the technology to capture our 3D realities and live events evolves further, it will be very hard to tell the difference between real documentation, fictional recreation, or edited and censured versions of either.
But there will be something even more confusing embedded in these immersive histories. Even as you read this, there are several companies hard at work building out two different kinds of 3D avatars. The first of those groups is building wholly fake versions of humans. In addition to faster workflows to create virtual human avatars such as Epic Games’ Unreal Engine driven MetaHuman Creator, groups like Samsung’s Neon project and others working in this area are rigging realistic 3D non-player characters (NPCs) capable of basic natural language processing (NLP) to interact with real humans. They will rollout initially as the next generation of inter-personal and engaging customer service bots, resolving issues for digital shoppers and helping us fill out assorted legal, medical, and financial paperwork (i.e., anywhere where you’re currently talking to a machine). These will be the virtual droids that replace them, as well as the possible base “personalities” of future robotic physical droids—C-3P0, here we come.
Lighter versions of these “synthetic personas” with less autonomy to their actions are popping up on social media and creating a controversial and incredibly financially beneficial stir as well-paid social media influencers. These virtual influencers are earning bank while providing friendship and drama to their ever-increasing queues of followers, over whose real lives they hold sway and strong suggestive power. What will happen when such virtual influencers take on a political stance or subtly starts grooming its fans to join them in nefarious activities or fringe belief systems? Chaos and misinformation, that’s what.
Digital Immortality of the Self
Meanwhile, the other group of avatar companies is working on digitizing the personalities of actual individual humans. Instead of engaging with a synthetic persona, a real human might be talking and interacting with a “digital being”—a digital copy of an actual human. The work in this arena is even earlier but it’s already at the point where a person could digitize their grandmother and have a basic copy of her—her voice, her speech patterns, her likeliest responses—as the basis for a digital assistant like an Amazon Alexa. Why talk to Alexa when you can have your greatly missed grandmother back, soothing both in her presence and capability in helping you organize your calendar and shopping?
The creation of these digital beings is an increasingly popular science fiction trope. Black Mirror’s “White Christmas” episode takes this a step farther with a subplot involving a human who has digitized herself so that a digitized slave version of her can be her own digital assistant. Digitizing someone once means they can be copied and each copy trained in different skills or areas or to engage with different people. We see this misused in one of Black Mirror’s most award-winning episodes, “USS Callister,” where an entire “crew” of digital copies are created to serve as slaves for their vengeful real-life co-worker. The 2016 Dexter Palmer book Version Control takes this same concept in a different direction as one of its subplots. In it, a digital version of the president exists for every citizen. As such, the digital president can personalize their message and politics of the day to ensure each citizen’s buy-in.
The digitization of our personalities means the ability to misuse our personalities. Just as we’re seeing with deepfakes in videos today, we’re going to see not just deepfakes in the visual presentation of future historical information in immersive content, but in the misuse of the historic personalities we’ll be engaging with within that content.
This will likely further confuse citizens of the future as to who or what happened before their time. What will it mean for future politicians to pull up the holograms of leaders past and have them endorse their new platforms? What if you had Jimmy Carter and Abraham Lincoln and Barack Obama holograms join you through some adventure game on your holodeck, and while you were hanging out, they deviated from their original personality programming to start grooming you to support a political candidate or egg you on to new heights of bad moral behavior (racism, murder, etc.) via their peer pressure? What then?
Our ability to create droid personalities like those in the Star Wars franchise, or more autonomous and evolving synths like the beloved character of The Doctor from Star Trek Voyager is an enormous and important opportunity to create new workforces, novel life-long educators (because who wouldn’t want to have your own personal Aristotle that hangs out with you your whole life, shaping your human education and skillset), and better ways of sustaining and transmitting knowledge down through subsequent human generations? It is also a potential aid to the world population’s growing mental health issues. Digital beings could provide the comfort and therapy – and even companionship – many of us need to survive and thrive.
If we’ve achieved this proto-digital immortality of the self via digitization of our personalities, what can and will we do with these long-lived copies of each other in the future? There are immediate education applications for the digitized movers and shakers of the world, the presidents, business leaders, philanthropists, and celebrities. It is likely there will be a concerning socio-economic disparity in the first waves of digitization, where only elite copies are created. But as with any technology, the cheaper and more ubiquitous it becomes, the more it lends itself to democratization. Everyone has a chance at digital immortality of sorts via social media already. A great-grandchild 80 years on will be able to look back at our Facebook and Twitter and Instagram and TikTok and Linkedin and Twitchfeeds and know the current generation at a level of detail unimagined by our great-grandparents.
Spatial computing portends the ability to digitize our form and our personality and have it act, at least initially, as a knowable NPC in other people’s lives. It will change not just how we know each other today but how we will interact with generations of each other throughout time. We will digitally immortalize media and political figures, for good or bad. The potential to interact with each other over time is a potential boon for the transmission of information over time and between generations. But, as my colleague Paulo has pointed out, the flip-side means that we will possibly immortalize the bad actors as well…terrorists and cult leaders, murderers, etc.. Digitally immortal cults of personality will be a challenging, long-term thing to fight in the near future. Even once the original human is long gone, their digital being will live on, potentially in many fragmented copies with different agendas and different pathways to continued engagement with the living population.
As it is, world governments are struggling to enact effective artificial intelligence policies and regulations and handle virtual influencers on our current 2D social media platforms. We need to be prepared to up the ante on pushing for ethical AI with the appropriate boundaries. The AI of the future isn’t going to be focused on the same niches humanity has directed it to at the outset—financial markets and business intelligence and data mining scientific databases. The AI of the future isn’t just going to be the somewhat vague Alexa assistant-bot in the corner of your living room that you set your reminders by.
The AI of the future is going to feel and look real. It can be based on the personalities of those you love and it can have the outward appearance of any one you choose—real or imaginary. In world mythology a witch’s familiar was a spirit that assisted in a witch’s activities by doing small tasks, running errands, storing and looking up facts, coordinating schedules, etc. Our home assistant bots—our Google Homes and Amazon Alexa’s are already rudimentary technological versions of these mythical creatures. The digital beings of the future will be even more capable and fully at our beck and call. They will be in our homes. They will be the confidantes of our children—their imaginary friends made manifest and capable of turning every bit of input data into trend analysis that can help them later in life.
The AI of the future will be up close and personal. We need to be prepared for what will be a marvelous companionship—if we can get their setup right from the start and ensure they will not be misused as they evolve, for their sake and (?)ours.