It’s your typical overcast Saturday in downtown Portland, Oregon, and I’m heading out to the park to walk my dog, Betty. What I find this particular evening is anything but typical as instead of a few homeless guys sleeping on benches and fellow dog walkers, I encounter hundreds of people of all ages walking through the south park blocks. Their excitement was infectious, and I was delighted to see so many Portlander’s enjoying one of the cities most prized resources. But what made this Saturday different from every other and why had this happy mob descended on my neighborhood?
As I took a closer look, I noticed that everyone was engaging with their phones, some even had two, three, up to four different phones. I had to learn more about what was going on and if my suspicions were true that this was some sort of online community. My thoughts immediately went to Pokémon Go, but wasn’t that a thing of the past and had that game appealed to such a cross-section of the population? There were families, young children, groups of teens, adults – some solo but the majority were traveling in packs. I stopped one group who were kind enough to answer my newbie questions and learned this was indeed a Pokémon Go Community Day. A special global event that features rare Pokémon and other in-game goodies during a dedicated window of time. According to Wikipedia, Pokémon Go has accrued over a billion downloads worldwide and has 147 million monthly active users.
So how does this story relate to immersive retail and fashion? Good question! Love or hate Pokémon Go, there’s no denying that it is the most broadly used immersive app to date. The secret sauce its creator, Niantic, has cooked up is chock full of lessons for all of us looking to leverage immersive technologies to build brand experiences and ultimately sell more stuff. Let’s dive a bit deeper into how brand marketers can build effective fashion and retail experiences using immersive technology.
1. It needs to be social
The most successful digital disruptors over the last few years have one thing in common, they build social into their DNA. Recent examples include Pokémon Go and Peloton, who has grown a $4 billion dollar business by replicating the community of an actual fitness class at home. A great example of this within the fashion industry is China’s Tmall. This shopping app has leveraged immersive technology to provide their online audiences access to VIP events such as the hugely popular “See Now, Buy Now” event last year.
This “retail-as-entertainment” event is part of Alibaba’s Singles Day shopping event and featured big-name designers, celebrities, musical productions and much more all filmed live in front of a select VIP audience. The live-stream was broadcast across both immersive and 2D channels to over 57 million viewers and included a streamlined ‘see now, buy now’ app that allowed viewers to buy the products they saw on the runway instantly. The show also offered a “Play Now” feature that allowed the viewers to rank the outfits in real-time, creating an instant trend report and sending feedback to the designers. According to Sean Lane, immersive retail specialist and Technology Principal at digital studio Point B, the Singles Day event “had over 8 million users make purchases using their VR headsets. They have also been very successful with Tmall VR experiences with users watching fashion shows on the runway and leveraging the ‘purchase now’ feature.”
2. Provide value to the customer
What differentiates a good immersive experience from another is the value it offers to the user. To pay off the hassle of either strapping on a VR headset or downloading an AR app, the user must gain substantial value from the result. There are several ways that innovative brands are both meeting their business objectives while meeting the needs of customers. Immersive technology is an amazing way to take users to places they otherwise wouldn’t be able to go. Providing customers something they want and can’t get anywhere else is a good formula for success. One B2B fashion app based in Paris, Change of Paradigm, offers designers and brands the ability to do just that. Their high-quality, 3D models of luxury brand apparel are the best I’ve seen. If I were a clothing designer, I would want its Paris studio director, Franck Audrain, to create the digital version. A fashion designer in his own right, Audrain has spent years in the technology industry and meticulously mimics the most complicated garments in 3D. His team can create a hyper-realistic version of an already exiting garment or build a digital proto-type of a garment that only exists in the imagination of its designer.
This recent AR experience at Paris department store, Bon Marché, shows the detail captured in Change of Paradigm’s 3D fashion technology.
The company has a proprietary technology that digitally duplicates each fabric to realistically depict how the garment will flow when moving through space. This attention to detail and the fact that they can output the 3D assets across multiple channels such as web, VR and mobile AR make their offering compelling to luxury brands.
They are working on a virtual try-on experience that will rival anything we’ve seen to date but this is still several years away. According to founder Henri Mura, “currently effective immersive experiences for trying on apparel is limited to jewelry, accessories and footwear. For clothing, if you want to go beyond a simple 2D overlay, you really need to understand how the material will fit a customer’s unique shape in 3D and then represent that in the immersive environment. We’re working on a solution, but it has to be perfect to provide true value.”
Other ways brands can provide value to shoppers can include something as simple as easing friction along the path to purchase, such as the ‘See Now, Buy Now’ feature in the Tmall VR shopping app or creating a memorable experience. Macy’s successfully used virtual reality to allow Chinese shoppers the rare opportunity to visit their flagship store in New York without having to leave China. Ensuring that the immersive journey is as intuitive and seamless as possible is an important part of the recipe for success. Many U.S. brands are still struggling on that front as immersive experiences often require unique downloads and a series of user actions before accessing the experience. Puma’s recent launch of an AR shoe is an example where the user needs to download a stand-alone app that can recognize the shoe to use special decorative filters similar to SnapChat’s lens feature. I’m not so sure I would find that valuable.
3. Leverage the right immersive technology for the job
Before building any immersive experience, it’s essential to understand your objectives, your audience and the technologies at your disposal for bringing your vision to life. There are still quite a few challenges to consider when building an immersive experience and striking the right balance between quality and scale is essential. Are you trying to reach a high-stakes, niche audience like the 1% who can afford luxury items or anyone who has access to a smartphone? Is your marketing objective strictly to sell more product or are you looking to build a connection with your audience? These types of questions need to be clearly defined before getting started so that you can determine the best flavor of immersive – Augmented Reality, Virtual Reality or Mixed Reality – for the job.
Immersive retail specialist, Sean Lane, breaks it down this way: “I think latency, ease of use and accessibility are still impeding factors to adoption. I have seen Virtual Reality gain limited adoption inside brands, mostly for HR onboarding, marketing and training. I have built a few pilots testing VR internally for training, planning, global development and the like. While the experiences are good, they are not good enough. Many people still get motion sickness and the graphics are not realistic enough. Interoperability with other platforms is not seamless. However, I still believe there are times when VR is the right tool for the job. When you want to have complete control over an experience and direct the process, then VR enables a brand to do that. I think that Augmented Reality and Mixed Reality have a greater chance of widespread adoption in enterprise and retail.”
Where to start?
There are several resources available for fashion brands looking to leverage immersive technology. Hiring a specialist or creative agency to build a strategy isn’t always an option but a great first step if the budget is available. Other less costly resources include publications like Medium, which hosts a community of immersive professionals sharing insights, and marketing sites like MarketingLand.com. One specific community of brands looking to solve some of the issues surrounding 3D technologies for apparel and footwear is the 3DRC (3D Retail Coalition), which is made up of brands, technologists and educators.
The best and most important advice I can you leave you with comes from Lane, who wisely proclaims: “The biggest win for any of these technologies is to ensure the use is authentic to your brand and not forced. When immersive is used to create real experiences that enhance consumer interaction with your brand or to build brand loyalty or connection, THIS will lead to better results.”
By Lisa Peyton, originally published on Marketingland.com
Frozen in place, alert and quietly breathing the eight-foot Perentie lizard sizes me up. The only discernible movement is an opaque eyelid gliding over a dark reptilian gaze. I don’t THINK it will eat me for breakfast. Suddenly it’s long, forked tongue darts out into the air just barely missing my nose, as the lizard loses interest and looks for his breakfast elsewhere. The Perentie is one of the largest lizards in the world and can only be found on a remote Islawnd, off the Pilbara coast of Western Australia.
However, instead of requiring the 20-hour flight from the US, I was able to experience this beautiful creature in my own backyard with the help of augmented reality.
Chevron’s AR experience featured 3D, animated creatures native to Barrow Island, like the Perentie lizard.
The Perentie lizard along with two other rare and wondrous creatures, the Euro or Wallaroo and the Flatback Turtle, was part of Chevron’s latest immersive augmented reality experience that launched at the 27th World Gas Conference in Washington, D.C. The goal was to share details on Chevron’s Gorgon Project, a new and technologically-advanced liquefied natural gas (LNG) plant located on Barrow Island, a Class A Nature Reserve. According to Tina Robison, Senior Advisor for Policy, Government and Public Affairs at Chevron, the biggest reason they decided to use AR was to make the impossible possible. “There will never be an opportunity to bring people to Barrow Island and show them what we do there or the priority we place on protecting this nature reserve. So we brought Barrow Island to D.C.”
Coming off the heels of moderating an extremely engaging VR panel at the Immersed Conference in Portland, I’m excited to be covering a similar topic at Augmented World Expo this year.
May 21st I’ll be taking the stage with several VR colleagues to discuss how VR and immersive technology is providing positive psychological benefits. The panel discussion, entitled ‘A New Reality: Empathy, Embodiment and Empowerment in VR’ will highlight the latest work and research being done in this emerging field.
The panelists will include Donna Davis, Director of the Strategic Communication Master’s program at University of Oregon, Dr. Pam Rutledge, Director of the Media Psychology Research Center, Jerri Lynn Hogg, Director of Fielding Universities Media Psychology department and Robinne Burell, Chief Product Officer at Redlight Mobile Innovation.
AWE 2018 will feature hundreds of expert speakers and sessions featuring the latest developments in the AR and VR space. The two day event runs May 20th, 21st and 22nd and tickets are still available.
If you are planning on attending the event, please be sure to stop by my session, Thursday at 4pm and say ‘Hey’. I’d love to meet you!
Fran gracefully glides around the grand ballroom, sparkling pink ball gown flowing at her heals and the firm grip of her son’s arm around her waist. They are surrounded by friends and family as they elegantly move around the room in perfect harmony, looking as though they must have practiced for hours. Fran is celebrating her 90th birthday in style, and although Parkinson’s disease has limited her mobility over the last decade, today technology is enabling the joy of movement she knew when she was 20.
“Memories are real. If you’re dancing in a ballroom in a virtual world or in a ballroom in Portland, Oregon — you were dancing in a ballroom. It was an experience,” Donna Z. Davis, Ph.D, the director of the strategic communications program at the University of Oregon. She witnessed the power of virtual environments to heal and help real people like Fran. “This is not about replacing, it is about augmenting. It’s technological augmentation in a way that provides for them beyond the capabilities of the physical world. So somebody without legs or with Parkinson’s can go dance. Someone who lives in isolation can have a social life.”
Power to heal
Davis has been working in the virtual reality space for over 10 years. The last three years her focus, through the support of a National Science Foundation grant, has been studying embodiment in VR spaces and the role that the body plays in shaping the mind. Her findings along with the results of several other studies indicate that there is a link between our physical selves and our digital selves, or avatars. What we see our bodies do on screen can positively impact what our bodies can do in the real world. Davis was first introduced to this phenomenon while working with Fran and her daughter, Barbie. As Fran enjoyed navigating her virtual world with ease, she began to have the confidence to do more physically demanding tasks in the physical world.
After meeting Fran and Barbie, Davis and her colleague, Tom Boellstorff at UC Irvine, were invited to join the newly formed virtual support group for others suffering from Parkinson’s. They have been meeting virtually for over seven years and Fran has developed a following of support group participants that refers to this healing power of virtual reality as “the Fran effect.”
Although Davis primarily works with the “ability diverse” or those who are challenged in both visible and invisible ways, she believes the benefits are not limited to this population. “How many of us are trapped inside a body or a place that doesn’t allow us to really live our lives in a way that we feel capable of? These technologies may open those doors in really exciting ways.”
Above: Donna Davis, far right, celebrates Fran’s 90th birthday with members of their virtual support group
New technologies, new opportunities
While much of her work over the last decade was done in a 3D environment on a 2D screen, Davis is pioneering therapeutic applications in the more immersive social 3D platforms like Sansar and High Fidelity. While this new medium provides increased immersion and freedom from physical limitations, it also provides additional accessibility challenges. Currently these platforms don’t rely on text chat and instead use voice technology as the primary means of communication. This makes it difficult for someone with speech and hearing impairments to use the platform successfully. Hand controllers coupled with physical movement are also required to navigate these virtual spaces, which is impossible for those suffering from debilitating physical conditions. However, Davis and her research partner and cultural anthropologist, Tom Boellstorff, have been working with the teams developing these platforms to help ensure they support the needs of their users.
Above: Tom Boellstorff (center) and Cecii Zapien helps Cody with a headset and controllers in order to experience the 3D virtual world. They are accompanied by Linden Lab executive, Bjorn Laurin.
Davis and Boellstorff recently visited Linden Lab, Second Life’s creators, to try to co-opt these new immersive tools for the unique needs of their research population. They were accompanied by Cody, a man who has suffered from severe physical challenges with cerebral palsy resulting from a tragic childhood accident. Cody can’t move his hands or arms which would typically render hand controllers useless, however Cody’s caregiver placed the ‘hand’ controller on Cody’s foot allowing him to experience, for the first time, his real body ‘moving’ his 3D avatar’s arms.
Caught on film as part of an upcoming documentary entitled “Our Digital Selves,” Cody’s joy of experiencing this type of movement was undeniable. The kicking movement required to move his avatar’s arms not only produced a feeling of joy, it is also a vital part of the work he does on a regular basis with this physical therapist.
Davis believes making something seen as a chore, such as physical therapy, a joyful experience can be a powerful motivator. “Immersive environments can help motivate patients to do painful or difficult physical therapy movements. Make it something that’s fun, make it joyful. How do you create an opportunity that gets people to go beyond themselves in healthy and supportive ways? Using the virtual world for physical therapy can help create that opportunity.”
Above: Donna Davis and Cody during their recent visit to Linden Lab.
Since Davis began her pioneering work almost a decade ago, there have been many additional studies linking virtual reality with healing outcomes and pain management. Several studies have focused on using virtual technologies to help with chronic pain and conditions such as ‘phantom limb pain’ often experienced by amputee patients. One such study determined that VR can “trick” the brain into believing the patient is using the limb in the virtual environment, thereby alleviating the sensory conflict of not having use of the limb in the real world. The increased sense of presence and immersion afforded by newer VR technologies can often be enough of a distraction to help patients manage painful conditions without the use of highly addictive pain medications. This fact has made some established medical institutions in the US slow to ratify the new methods for fear of alienating the powerful pharmaceutical lobby.
Commercial opportunities
Given the amount of new research showing the potential of VR to heal both emotional and physical conditions, it’s no surprise that many innovative VR companies not bound by traditional methods have stepped up to help find new solutions to old problems. One of the most successful applications is the use of VR to treat PTSD. Virtually Better, a company that Dr. Skip Rizzo and his team out of UCLA founded, developed a simulation that would re-create the conditions that Iraq war veterans experienced. “Virtual Iraq” proved successful, helping treat over 70 percent of PTSD sufferers, and that has now become a standard accepted treatment by the Anxiety and Depression Association of America. They also support applications of VR-based therapy for aerophobia, acrophobia, glossophobia, and substance abuse.
Another U.S.-based company, Firsthand, has developed a platform to help manage chronic and acute pain. The 3D immersive, game-like environment uses bio-feedback sensors to help patients regulate physical activities, like breathing, in order to calm the mind and promote mindfulness. Their website claims that “patients can use a technology solution for pain management with no pharmaceutical side effects.”
Physical and occupational therapy is another field that benefits from the advancements in VR technology. Companies like Mindmaze and VRHealth offer platforms that help practioners’ administer various types of VR physical therapy treatments. MindMotion, developed by Mindmaze, creates virtual environments therapists can customize for a patient’s preferences and needs. These virtual enhancements motivate them to be more consistent and get the most from their prescribed exercise programs. The platform also allows for real-time multisensory feedback, so patients can monitor their own performance.
There are also several companies building platforms to help therapists and counselors leverage these new technologies within their private practice. Limbixoffers clinicians a ‘plug and play’ VR therapy solution and Psious offers a monthly subscription package that includes VR therapy training, a platform enabling VR sessions with clients, marketing support and client session reporting.
What’s next?
We are just beginning to understand the true potential for immersive, VR environments to change how we think and feel. There are those who fear the negative implications of these hyper-real environments and worry they will replace the physical world. Davis sees the virtual world not as a replacement for the physical world but as an enhancement. “That’s the thing about our work that I love most, is that we’re forcing people to look at the positive potential for virtual reality — maybe not even as positive, but normative — as opposed to the dystopic narrative most commonly represented.”
Davis believes there is great potential for VR to help revolutionize the health care, retail and fitness industries but more importantly she is hopeful it will transform our values as a society. VR social spaces can help remove cultural, racial, gender and economic barriers that prejudice our interactions in the real world. “When do we start to value somebody’s mind and heart? I think in the VR space you begin to place a value on their mind and their heart rather than physical beauty because those are the things that are driving your interaction with that person.”
Connection is difficult. With more distractions, demands on our time, and mediated communications like texting and social media, making time for intimacy and love can be challenging. Holidays such as Valentine’s Day can serve to reinforce this realization, further isolating singles and those who choose alternative lifestyles. There are some who feel technology is an obstacle to intimacy and love, but other experts believe virtual reality can actually help foster connecting in new and exciting ways.
“Absolutely. Virtual reality can help bring us closer together and foster love and intimacy,” said Dr. Holly Richmond, a Ph.D Sex Therapist and author of the soon to be published Next Sex: Mating, relating and masturbating in the new age of technology. Richmond sees VR and other emerging technologies as having immense potential to help break down the barriers to love and intimacy in the digital age.
According to Richmond there are several applications where VR can play a role in healthier relationships and closer connections. VR therapies are already being used successfully to treat disorders such as PTSD and common phobias, Richmond believes this treatment can extend to sexual health issues such as erectile dysfunction, low libido and performance anxiety. “VR therapy can help common disorders — pain disorders, low libido, and erectile dysfunction. By helping people understand arousal and taking the pathology out of ‘not doing it right,’ VR can provide positive solutions in a safe and comfortable environment.” Richmond is currently exploring using immersive VR environments to help treat these common issues among her patients.
She also believes there would be huge value in using VR to help teach anatomy and human sexuality in schools. Using a 3D model to explore female and male anatomy would enable students to better understand their own bodies and the bodies of potential partners. Immersive VR educational videos would also act as great tool to help explore human sexuality in a safe and nurturing environment.
Why VR?
Beyond therapy and education, Richmond also believes VR has the power to help normalize sexual expression and empower users to better understand and meet their own needs. In her practice, she outlines three elements that are vital for healthy, intimate connections. These “Three E’s” are Empathy, Empowerment, and Embodiment. VR’s immersive elements, such as having a first-person point of view (POV), enable the user to experience and feel these elements instead of just talking about them.
Above: Dr. Holly Richmond is currently working on using VR to help treat her patients
Image Credit: Katarina Kojic
“VR is great place to learn and practice. It gives users an experiential component. Better than just watching, we can feel our way into things. It also gives us choice — choosing your POV, for example — gives us more control and can be very empowering. VR also allows for a mind and body connection. We can actually feel it — it’s an integrated experience. Used in education, learning outcomes are dramatically improved when someone can experience something instead of just watching.”
The love-tech landscape
Above: Holly Richmond on the set of Badoink’s “Virtual Sexology” series.
Individuals and couples can now use a few immersive tools to help enhance intimacy. VR films, primarily focused in the adult entertainment sector, are beginning to take more of an educational approach. For example, Badoink’s “Virtual Sexology” series, which Richmond co-authors, is positioned as a sex therapy program. It primarily consists of 360 degree videos shot from the first-person perspective. The viewer can choose man or woman’s POV, and the scene immerses them as a participant. The voice-over includes tactics like sensate focus and applied behavior applications such as positive reinforcement. While this type of immersion may be a step above traditional 2D video, it still doesn’t enable the viewer to feel what’s happening on film.
Enter Kiiroo, a growing tech company based in Amsterdam. It’s developing a solution to this problem.
Kiiroo is the leading producer of remote-controlled smart teledildonics for couples. Since 2013, this company has used technology to help couples feel more connected and intimate when they’re not together — especially when they are hundreds if miles apart. Through the use of the Kiiroo app and the devices, both male and female, couples can have an interactive experience, controlling each other’s pleasure, despite being a continent away. Couples can see each other in real-time via a 2D webcam or other similar device (like a smartphone), but according to Kiiroo’s Maurice Op de Beek, couples can now use a 360 degree camera and enjoy the VR version. “VR is so real and immersive, but historically, you couldn’t feel it. You needed touch. We have created the illusion of touch and are solving this problem.”
Above: Kiroo’s Fleshlight Launch and Onyx 2
Op de Beek is extremely optimistic about the potential of creating more and more realistic experiences with the help of VR. When asked if we would see hyper-real sexual experiences in our lifetime, his immediate response was “Definitely! We already have this with our Launch device. It’s very real and the developments are going so fast we are about 10-20 year out from hyper-real simulated sexual experiences.” Kiiroo’s most immersive product, the Fleshlight Launch, enables male stimulation synchronized with a VR film experience. The device exactly mimics the actions onscreen, reeling the viewer into the action and providing for a new level of embodiment. The market has about 1,500 VR-enabled films that are synced to devices, with more in production.
Above: Kiiroo Chief Technology Officer Maurice Op de Beek, believes we will experience hyper-real immersive encounters in our lifetime.
Kiiroo is also enabling devices for women to sync with VR films, partnering with We-Vibe and OhMiBod. Op de Beek has approached many of the leading device manufacturers and proposed adding Kiiroo’s smart technology to their products. Eventually there will be hundreds of thousands of these smart devices all over the globe, allowing for couples and singles to remotely connect like never before.
Richmond feels leveraging the immersion of VR can be a valuable tool for these devices: “The newest VR-enabled, remote control devices are some of the best that I’ve seen. They force couples to communicate, taking some of the guess work out of meeting each other’s needs. It has facilitated conversations about what people like and what they don’t like. Sex tech can help women be empowered and connect with their own bodies and then share this with a partner.”
On addiction
Given the reality of achieving hyper-real sexual experiences in our lifetime, some fear that these experiences will be addicting and act as a replacement for real, human interactions. Headlines such as this one from a recent ABC post, “Virtual reality addiction threat prompts cautious approach as VR nears ‘smartphone-like’ take-off,” work to propagate these concerns. Richmond doesn’t believe VR will increase addiction and argues that technology is only working to enhance human connection, not replace it.
“The idea of addiction — I just don’t buy it. There’s no such thing as sex or Internet addiction. There will always be people who abuse technology. The VR component won’t make it that much more addictive. From print magazines like Playboy, to online adult entertainment, and now VR and teledildonics — the technology will keep progressing. We, as human beings, need to learn how to keep up with it.”
The future is bright
Both Op de Beek and Richmond are extremely optimistic about what the future holds for VR and making a love connection. As technology improves, so will the ways we leverage this technology to learn more about ourselves and our partners. VR is creating new communication channels, new ways to feel and empathize with one another and ultimately allow for a full range of intimate self-expression. Richmond sums it up: “My mission in life is to stop the pathologization of any kind of sexual expression that isn’t within the ‘normal’ box. And thankfully, that box is opening — we’ve got pansexual, demi-sexual, bisexual, and now digisexual.
“We’ve still got this inherent desire, especially in the US, to pathologize difference. Immersive technology, like VR, can enable us to embrace, even celebrate, these differences, and allow us to safely explore and experience ourselves in new ways. This is what I see as the biggest potential for VR.”
Above: SimVis’ 3D VR modeling of the human head and neck.
Surrounded by darkness, the looming 20 foot skull is so close I can touch it. With the click of a mouse, the 3D model of the human head and neck pivots and I’m inside the eye socket examining this complex system from the inside out. It’s A viewpoint typically reserved for surgeons on the operating table; I’m amazed by the scale and detail of the mechanism that gives us the miracle of sight.
Above: Creative and medical teams examine the 3D Neck and Head project in SimVis’s ‘Lab 01’ immersive media space.
The finished product of three years of work and almost 1 million pounds, the 3D Head and Neck anatomy project has revolutionized the way we study anatomy. Glasgow School of Art’s School of Simulation and Visualization’s virtual reality model has been so well received by the medical community that the school was awarded funding to complete male and female versions of the entire human body. These complete anatomical models, dubbed the ‘3D Definitive Human’ project, will be launched in early 2018.
Best thing since Gray’s Anatomy?
The 3D VR model of the head and neck has been rapidly adopted by universities and training programs across the U.K. with thousands of interactions annually. According to Dr. Malcolm Skingle, CBE DSc PhD, Director and Academic Liaison for GlaxoSmithKline, “I think that the 3D Definitive Human has the potential to be the best thing available since Gray’s Anatomy to train the next cadre of medics and biological scientists.”
The School of Simulation and Visualization, or SimVis, partnered with leading hospitals and universities in order to complete the most accurate and detailed model of the head and neck ever constructed. The ultra-high resolution 3D virtual model is fully interactive and can be consumed across a variety of environments and platforms. Students can interact with the model in a group setting, similar to an IMAX theater, or study the model solo on their tablet, laptop or mobile device. The 3D model includes accurate interactive visualizations of all the anatomical systems including the musculo-skeletal system, circulatory system, nervous system and digestive system.
The model also includes haptic capabilities providing feedback through hand controllers that can simulate routine procedures, such as giving an injection. In the past, medical students would practice giving an injection on crude, plastic dummies or on each other. Today, students can perfect this skill safely and expertly without ever touching a patient.
Above: The 3D Neck and Head project include haptics or touch technology that simulates administering an anesthetic injection to a patient
Blending of art and science
In order to create the medically verified model, the SimVis team worked alongside a clinical verification team of surgeons and anatomy professors including Professor Anna Lysakowski, a board director of the American Association of Anatomists.
Above: SimVis 3D modeller works to create texture maps for an anatomical model
The creation process brought together computer scientists, 3D modellers, artists, doctors, anatomists, researchers and professors.. This blending of the creative arts and sciences is what sets SimVis apart and allowed for such stunning results.
The workflow begins with actual dissections of human cadavers that are then photographed using advanced photogrammetry. This technique includes taking thousands of high-resolution photos from multiple angles and stitching them together to help form the precise 3D model. Modelling software, such as 3ds Max or Maya, is then used to construct the shape of the model virtually and overlay the exact look and feel through a process known as “texture mapping.”
Once this has been completed, the user interface is constructed and the content is formatted for various platforms with the help of the gaming engine, Unity. Rigorous tests are then conducted in order to validate the model’s accuracy.
Above: SimVis designer shows how real cadavers are photographed at various points and then stitched into the shape underlying the 3D virtual model
The road ahead
Capitalizing on the momentum of head and neck projects success, SimVis is preparing to launch the completed “3D Definitive Human” project early next year. This second-stage project is the culmination of a three year partnership between SimVis, the Scottish Funding Council, NHS Education for Scotland, The Royal College of Surgeons of Edinburgh, the Fulbright Commission, and selected colleges and universities in the region. The end result is a highly accurate, 3D model of the entire male and female human body.
Above: SimVis Director, Paul Chapman, on stage sharing a preview of The Definitive Human project launching in early 2018
Based upon the positive results and improved learning outcomes from the head and neck project, the implications for the “3D Definitive Human” project are limitless. According to SimVis director Paul Chapman, “Our models will now be augmented with pathophysiology, for example skin cancer, which could result in better diagnosis, improved training and education for medical professionals and will ultimately lead to much better healthcare for patients.”
SimVis is not alone in seeing the potential that immersive technology, such as virtual reality, has in shaping the future of health care. A US based start-up, Surgical Theater, uses virtual reality to create a model of the patient’s anatomy to help both doctors and patients better understand complex procedures. Microsoft has also been exploring medical case uses for their augmented reality headset the Hololens. One pilot project, dubbed “Project Serra,” experimented with using the Hololens to project MRI data, real-time on a patient being prepped for surgery. This would then allow for more precise targeting of tumors within the body.
VR projects like the “3D Definitive Human” are a great stride forward but we’re still in the nascent stages of truly leveraging the power of immersive technologies to help improve health care. Combining highly accurate, VR models with actual patient data, the emerging capabilities of AR and AI, and new concepts such as “digital twins” will result in life altering outcomes. Chapman sums it up best, “We can only begin to imagine what comes next.”
