Next week, I’m thrilled to be attending the Immersed Conference in Portland, Oregon. The conference is a two day event focused on immersive technologies like VR and AR and how they are impacting the world around us.
I’ll be moderating a panel discussion on empathy, empowerment and embodiment in VR featuring my colleagues and friends Donna Davis, Tawny Schlieski and Jerri Lynn Hogg. You can learn more about all the conference sessions, buy tickets and join the community on the Design Reality website.
Stephanie Mendoza, an immersive media artist and enthusiast will also be participating in the conference. She will be teaching a workshop on WebVR and PIMG will be demoing on the exhibition floor along with other Portland VR/AR startups. She was kind enough to answer a few questions on her experience with VR, Portland and becoming a dragon!
1) How long have you been in Portland? Why did you decided to stay here?
I’m coming up on 4 years now, I arrived in May 2015 to take some front end web development courses over the month. Then I got a job in Seattle- didn’t like Seattle very much so I went to Canada for a month and a half to visit old friends. I’d also made enough money selling drawings at an art auction to visit Portland again before going home (also my car was there). The intention was not to stay, but to spend two weeks with the friends I’d made and then move back to Texas.. well my student loans kicked in and automatically took money I didn’t have, leaving me with a balance of
(-$600) and no gasoline to get back. I would have been homeless, had I not been taken under the wing of a Cyborg Anthropologist I got nerdy with at a Hackerspace in NE called CtrlH. That really started it all.
2) Describe the creative and emerging tech scene in Portland? Could it be recreated elsewhere?
Portland is pretty special, though I’m not sure if I’ve traveled enough to know exactly how unique it is. The city is easy to navigate, creative and there are many programs that support practicing artists. I can meet people from almost every socioeconomic/cultural background, and not be restricted by notions of cultural bias, so long as a common interest is pursued. This place is incredibly community oriented, it seems that each group has about one or two degrees of separation between its members, that tend to just get together en mass, and hack at whatever they are doing.
3) What latest tech trends excite you the most?
GDPR. I’ve been an advocate for data protection since I was in 8th grade and social media was taking off. Specifically, what gets me excited is GDPR (General Data Protection Regulation) in Europe which comes into effect on May 25th- most importantly, it states that 1) Right of Access: Users have full access to all of the data a company has on them as well as how it is being processed, and 2) Right to Erasure: Users have a right to have that data removed. Failure to comply could lead to sanctions as high as 4% of annual worldwide turnover, so it’s not toothless.
Additionally, people are finally listening to me about dumping Facebook, and caring about how companies treat their user’s data. I strongly believe that this is the most important trend to evangelize in our modern tech climate.
4) How did you get involved with social VR? Which social VR platforms do you hang out in and why?
I started in Anyland, a social VR sandbox where anyone could make anything with unlimited resources (well, until their GPU choked on the particle effects). Anyland was and is still an incredibly powerful tool for rapid prototyping of ideas that just can’t exist yet in tangible space, but who’s presence we can prepare for in a simulation.
I’m in VR Chat a lot now too. I got sucked in to watching a bunch of Ugandan Knuckles videos over my VR deprived trip to visit family in Mexico, and spent months making highly detailed avatars (also converting my Anyland one). I still think I prefer the open and immersed creativity of Anyland, but I’ve been spending much of this year exploring VR Chat.
5) What do you see as the BIGGEST potential for VR, AR and immersive technology?
VR is an incredible tool for exposure therapy and it shields us from own own biases about identity when avatars come in to play, exposing something deeper. One of the first things that happened when I started developing for VR in Unity was loosing my fear of heights, which was pretty bad at the time and hasn’t returned since. I did the same for my stage fright, where I simulated the old New Relic office and practiced in there with my slides in front of a fake audience of zombies, tigers, and men in blue suits. This all gets amplified when it involves social learning and observation.
6) Do you feel large corporations such as Intel and Nike have played a role in the technology innovations happening in Portland? What role should corporations play in promoting innovation?
Bike town provides a good model of accessible equipment. Also Intel sponsors a lot of events like hackathons and small conferences/meetups (that feed us starving tech artists, many of whom us won’t admit to barely having enough to feed ourselves), so keep doing that!
7) Do you feel immersive communities and social VR can help empower and heal people or do you feel they will have more of a negative impact?
Both. Back in Anyland the crowd was mostly west coast, with a good amount of midwestern conservatives, and a few Europeans and Asians sprinkled into the mix. It was a small but tightly knit community. We actually managed to build bridges between the two wings of American ideologies (liberal redditors, and 4chan trump trolls) using the unlimited creativity Anyland provided us. We could coexist extremely well in a simulated utopia, void of any ‘economy’ other than a gifting one based on creative capitol. It sounds unrealistic, but also not impossible. I think this kind of thing can work wonders helping people get out of the mindset that everyone has to have a job, in a rapidly approaching post-work world.
That being said social VR can be a serious trigger for those with preexisting conditions who have PTSD from combat or assault. People in these spaces can be ruthless, but there are definitely ways to deal with them as an individual without getting the mods or the company involved. Trolling in real-time is a double edged sword and often the worst offenders are terrible at comebacks, which in my experience changes the game. Most of the other users will also have your back, you just have to have some thick skin, which comes with experience. It can be harrowing for a first timer.
8) How has spending time in social VR communities impacted you pesonally?
I have finally realized that yes, I can grow up to become a dragon; nine year old me is very pleased. I feel like anything is possible now, and I can test new inventions people come up with, before they manifest in ‘reality’. I wind up having these extreme experiences that can only be described as if they were a dream, since often it involves shapeshifting, and strange worlds/contraptions that exist as pure imagination manifested by data. Also shapeshifting was always my go-to superpower so thats a huge incentive drawing me back to the territory.
9) Currently VR is really only accesible by a small number of users – when do you think VR will hit mass adoption and why?
Probably once the next generation of headsets comes out, just like any other console or ‘gaming’ hardware. Go to goodwill and you can see the stacks of unwanted Wii’s and Xboxes, and on the shelf below them, every variety of mobile VR headsets.
10) Which local start ups or local innovators do you feel have the most potential? Why?
So I can’t answer this question in the context of financial potential, because the places I see the most creativity come out of, and have the greatest community impact, are also the places that earn the least and are burdened the most- especially during this current administration which has dramatically cut funding for the arts nationally. Take Open Signal and Enthusiasm Collective- they provide a vital service to the Art and Tech community, and without places like that many VR devs/artists would not have had opportunities to get started.
I have high hopes for our local hackerspaces as well, I want to give a shout out to CtrlH for the amazing things they’ve done, providing equipment for local creators, and bringing together a variety of interesting meetups- Dorkbot (robotics) and Exploit(information security) come to mind, and they have even supported Portland Immersive Media Group(VR) when we needed a venue.
11) How did you get involved with the Design Reality community? What will your upcoming presentation at the Immersed Conference cover? Who do you feel should attend this conference and why?
Luck. I met Joshua at Kent Bye’s house when Kent had a pre-release of Google Earth VR. A few weeks later I got an email about the new VR meet up, noticed that he was charging for tickets, and promptly wrote a concerned letter. I knew that charging ANY amount would immediately discourage most of the artists that I knew were currently participating in VR from attending, since even a small $5 fee is detrimental as costs add up when you include parking/MAX fares, and that resulting $10-15 is a grocery budget for the week. Yes we are that economically challenged. I worried that this would result in further homogenization of the group, creating the beginnings of an elitist VR scene in Portland. Fortunately that didn’t happen, and Joshua completely understood what I was talking about. He is still charging for tickets, but there are both volunteer and scholarship opportunities for those of us who wouldn’t be able to participate otherwise.
I feel like this conference is for anyone who wants to get the full scope of what Portland VR has to offer. I will be teaching a workshop on WebVR and PIMG will be demoing on the exhibition floor along with other Portland VR/AR startups.
Connection is difficult. With more distractions, demands on our time, and mediated communications like texting and social media, making time for intimacy and love can be challenging. Holidays such as Valentine’s Day can serve to reinforce this realization, further isolating singles and those who choose alternative lifestyles. There are some who feel technology is an obstacle to intimacy and love, but other experts believe virtual reality can actually help foster connecting in new and exciting ways.
“Absolutely. Virtual reality can help bring us closer together and foster love and intimacy,” said Dr. Holly Richmond, a Ph.D Sex Therapist and author of the soon to be published Next Sex: Mating, relating and masturbating in the new age of technology. Richmond sees VR and other emerging technologies as having immense potential to help break down the barriers to love and intimacy in the digital age.
According to Richmond there are several applications where VR can play a role in healthier relationships and closer connections. VR therapies are already being used successfully to treat disorders such as PTSD and common phobias, Richmond believes this treatment can extend to sexual health issues such as erectile dysfunction, low libido and performance anxiety. “VR therapy can help common disorders — pain disorders, low libido, and erectile dysfunction. By helping people understand arousal and taking the pathology out of ‘not doing it right,’ VR can provide positive solutions in a safe and comfortable environment.” Richmond is currently exploring using immersive VR environments to help treat these common issues among her patients.
She also believes there would be huge value in using VR to help teach anatomy and human sexuality in schools. Using a 3D model to explore female and male anatomy would enable students to better understand their own bodies and the bodies of potential partners. Immersive VR educational videos would also act as great tool to help explore human sexuality in a safe and nurturing environment.
Beyond therapy and education, Richmond also believes VR has the power to help normalize sexual expression and empower users to better understand and meet their own needs. In her practice, she outlines three elements that are vital for healthy, intimate connections. These “Three E’s” are Empathy, Empowerment, and Embodiment. VR’s immersive elements, such as having a first-person point of view (POV), enable the user to experience and feel these elements instead of just talking about them.
Above: Dr. Holly Richmond is currently working on using VR to help treat her patients
Image Credit: Katarina Kojic
“VR is great place to learn and practice. It gives users an experiential component. Better than just watching, we can feel our way into things. It also gives us choice — choosing your POV, for example — gives us more control and can be very empowering. VR also allows for a mind and body connection. We can actually feel it — it’s an integrated experience. Used in education, learning outcomes are dramatically improved when someone can experience something instead of just watching.”
The love-tech landscape
Above: Holly Richmond on the set of Badoink’s “Virtual Sexology” series.
Individuals and couples can now use a few immersive tools to help enhance intimacy. VR films, primarily focused in the adult entertainment sector, are beginning to take more of an educational approach. For example, Badoink’s “Virtual Sexology” series, which Richmond co-authors, is positioned as a sex therapy program. It primarily consists of 360 degree videos shot from the first-person perspective. The viewer can choose man or woman’s POV, and the scene immerses them as a participant. The voice-over includes tactics like sensate focus and applied behavior applications such as positive reinforcement. While this type of immersion may be a step above traditional 2D video, it still doesn’t enable the viewer to feel what’s happening on film.
Enter Kiiroo, a growing tech company based in Amsterdam. It’s developing a solution to this problem.
Kiiroo is the leading producer of remote-controlled smart teledildonics for couples. Since 2013, this company has used technology to help couples feel more connected and intimate when they’re not together — especially when they are hundreds if miles apart. Through the use of the Kiiroo app and the devices, both male and female, couples can have an interactive experience, controlling each other’s pleasure, despite being a continent away. Couples can see each other in real-time via a 2D webcam or other similar device (like a smartphone), but according to Kiiroo’s Maurice Op de Beek, couples can now use a 360 degree camera and enjoy the VR version. “VR is so real and immersive, but historically, you couldn’t feel it. You needed touch. We have created the illusion of touch and are solving this problem.”
Above: Kiroo’s Fleshlight Launch and Onyx 2
Op de Beek is extremely optimistic about the potential of creating more and more realistic experiences with the help of VR. When asked if we would see hyper-real sexual experiences in our lifetime, his immediate response was “Definitely! We already have this with our Launch device. It’s very real and the developments are going so fast we are about 10-20 year out from hyper-real simulated sexual experiences.” Kiiroo’s most immersive product, the Fleshlight Launch, enables male stimulation synchronized with a VR film experience. The device exactly mimics the actions onscreen, reeling the viewer into the action and providing for a new level of embodiment. The market has about 1,500 VR-enabled films that are synced to devices, with more in production.
Above: Kiiroo Chief Technology Officer Maurice Op de Beek, believes we will experience hyper-real immersive encounters in our lifetime.
Kiiroo is also enabling devices for women to sync with VR films, partnering with We-Vibe and OhMiBod. Op de Beek has approached many of the leading device manufacturers and proposed adding Kiiroo’s smart technology to their products. Eventually there will be hundreds of thousands of these smart devices all over the globe, allowing for couples and singles to remotely connect like never before.
Richmond feels leveraging the immersion of VR can be a valuable tool for these devices: “The newest VR-enabled, remote control devices are some of the best that I’ve seen. They force couples to communicate, taking some of the guess work out of meeting each other’s needs. It has facilitated conversations about what people like and what they don’t like. Sex tech can help women be empowered and connect with their own bodies and then share this with a partner.”
Given the reality of achieving hyper-real sexual experiences in our lifetime, some fear that these experiences will be addicting and act as a replacement for real, human interactions. Headlines such as this one from a recent ABC post, “Virtual reality addiction threat prompts cautious approach as VR nears ‘smartphone-like’ take-off,” work to propagate these concerns. Richmond doesn’t believe VR will increase addiction and argues that technology is only working to enhance human connection, not replace it.
“The idea of addiction — I just don’t buy it. There’s no such thing as sex or Internet addiction. There will always be people who abuse technology. The VR component won’t make it that much more addictive. From print magazines like Playboy, to online adult entertainment, and now VR and teledildonics — the technology will keep progressing. We, as human beings, need to learn how to keep up with it.”
The future is bright
Both Op de Beek and Richmond are extremely optimistic about what the future holds for VR and making a love connection. As technology improves, so will the ways we leverage this technology to learn more about ourselves and our partners. VR is creating new communication channels, new ways to feel and empathize with one another and ultimately allow for a full range of intimate self-expression. Richmond sums it up: “My mission in life is to stop the pathologization of any kind of sexual expression that isn’t within the ‘normal’ box. And thankfully, that box is opening — we’ve got pansexual, demi-sexual, bisexual, and now digisexual.
“We’ve still got this inherent desire, especially in the US, to pathologize difference. Immersive technology, like VR, can enable us to embrace, even celebrate, these differences, and allow us to safely explore and experience ourselves in new ways. This is what I see as the biggest potential for VR.”
Above: SimVis’ 3D VR modeling of the human head and neck.
Surrounded by darkness, the looming 20 foot skull is so close I can touch it. With the click of a mouse, the 3D model of the human head and neck pivots and I’m inside the eye socket examining this complex system from the inside out. It’s A viewpoint typically reserved for surgeons on the operating table; I’m amazed by the scale and detail of the mechanism that gives us the miracle of sight.
Above: Creative and medical teams examine the 3D Neck and Head project in SimVis’s ‘Lab 01’ immersive media space.
The finished product of three years of work and almost 1 million pounds, the 3D Head and Neck anatomy project has revolutionized the way we study anatomy. Glasgow School of Art’s School of Simulation and Visualization’s virtual reality model has been so well received by the medical community that the school was awarded funding to complete male and female versions of the entire human body. These complete anatomical models, dubbed the ‘3D Definitive Human’ project, will be launched in early 2018.
Best thing since Gray’s Anatomy?
The 3D VR model of the head and neck has been rapidly adopted by universities and training programs across the U.K. with thousands of interactions annually. According to Dr. Malcolm Skingle, CBE DSc PhD, Director and Academic Liaison for GlaxoSmithKline, “I think that the 3D Definitive Human has the potential to be the best thing available since Gray’s Anatomy to train the next cadre of medics and biological scientists.”
The School of Simulation and Visualization, or SimVis, partnered with leading hospitals and universities in order to complete the most accurate and detailed model of the head and neck ever constructed. The ultra-high resolution 3D virtual model is fully interactive and can be consumed across a variety of environments and platforms. Students can interact with the model in a group setting, similar to an IMAX theater, or study the model solo on their tablet, laptop or mobile device. The 3D model includes accurate interactive visualizations of all the anatomical systems including the musculo-skeletal system, circulatory system, nervous system and digestive system.
The model also includes haptic capabilities providing feedback through hand controllers that can simulate routine procedures, such as giving an injection. In the past, medical students would practice giving an injection on crude, plastic dummies or on each other. Today, students can perfect this skill safely and expertly without ever touching a patient.
Above: The 3D Neck and Head project include haptics or touch technology that simulates administering an anesthetic injection to a patient
Blending of art and science
In order to create the medically verified model, the SimVis team worked alongside a clinical verification team of surgeons and anatomy professors including Professor Anna Lysakowski, a board director of the American Association of Anatomists.
Above: SimVis 3D modeller works to create texture maps for an anatomical model
The creation process brought together computer scientists, 3D modellers, artists, doctors, anatomists, researchers and professors.. This blending of the creative arts and sciences is what sets SimVis apart and allowed for such stunning results.
The workflow begins with actual dissections of human cadavers that are then photographed using advanced photogrammetry. This technique includes taking thousands of high-resolution photos from multiple angles and stitching them together to help form the precise 3D model. Modelling software, such as 3ds Max or Maya, is then used to construct the shape of the model virtually and overlay the exact look and feel through a process known as “texture mapping.”
Once this has been completed, the user interface is constructed and the content is formatted for various platforms with the help of the gaming engine, Unity. Rigorous tests are then conducted in order to validate the model’s accuracy.
Above: SimVis designer shows how real cadavers are photographed at various points and then stitched into the shape underlying the 3D virtual model
The road ahead
Capitalizing on the momentum of head and neck projects success, SimVis is preparing to launch the completed “3D Definitive Human” project early next year. This second-stage project is the culmination of a three year partnership between SimVis, the Scottish Funding Council, NHS Education for Scotland, The Royal College of Surgeons of Edinburgh, the Fulbright Commission, and selected colleges and universities in the region. The end result is a highly accurate, 3D model of the entire male and female human body.
Above: SimVis Director, Paul Chapman, on stage sharing a preview of The Definitive Human project launching in early 2018
Based upon the positive results and improved learning outcomes from the head and neck project, the implications for the “3D Definitive Human” project are limitless. According to SimVis director Paul Chapman, “Our models will now be augmented with pathophysiology, for example skin cancer, which could result in better diagnosis, improved training and education for medical professionals and will ultimately lead to much better healthcare for patients.”
SimVis is not alone in seeing the potential that immersive technology, such as virtual reality, has in shaping the future of health care. A US based start-up, Surgical Theater, uses virtual reality to create a model of the patient’s anatomy to help both doctors and patients better understand complex procedures. Microsoft has also been exploring medical case uses for their augmented reality headset the Hololens. One pilot project, dubbed “Project Serra,” experimented with using the Hololens to project MRI data, real-time on a patient being prepped for surgery. This would then allow for more precise targeting of tumors within the body.
VR projects like the “3D Definitive Human” are a great stride forward but we’re still in the nascent stages of truly leveraging the power of immersive technologies to help improve health care. Combining highly accurate, VR models with actual patient data, the emerging capabilities of AR and AI, and new concepts such as “digital twins” will result in life altering outcomes. Chapman sums it up best, “We can only begin to imagine what comes next.”
What causes immersion? As the digital revolution rages on, technology has been both blamed and lauded for capturing more and more of our attention. We have been given the tools to tell stories in new and exciting ways, access to data that has never been available before and the ability to connect to a fully realized digital universe. As digitally connected citizens, we are constantly bombarded with incoming messages and have become adept at tuning out endless amounts of ‘noise’. So what then CAN capture and keep our attention? An array of augmented reality and virtual reality applications or ‘immersive media’ experiences are being developed across various sectors and engage users like never before. The immersion grid has been developed to help compare and contrast these applications and predict how likely they are to be truly immersive. (more…)
Advances in digital technology and communications are radically changing the way we live. Can they change the way we learn? YES!
One of the areas digital technology has the potential to make the biggest impact is education. I’ve been a teacher for over a decade and currently teach both online and offline courses. I’m excited about the paradigm shift that is happening in education, however there is is still a VERY wide gap between what is possible and the current state of classrooms. Today’s students aren’t engaged by the old teaching methods and aren’t gaining the skills required to succeed in a digital world. This is where immersive technologies, such as Augmented Reality (AR) and virtual environments can help. (more…)
User generated content in Second Life houses MANY representations of big brands, such as Converse, created by brand advocates.
In the final chapters of his book, Convergence Culture, Jenkins (2006) outlines the characteristics of a paradigm shift to a ‘participatory culture’. He cites many examples of the tensions that are arising between the old mass media and the new media revolution. The digital tools that have allowed the average consumer to find a voice online and create user-generated content are also being exploited by larger brands, attempting to mimic this grassroots phenomenon.
As I read Jenkins (2006) observations, I was struck by what may have been an unintentional experiment testing this new paradigm. Many of the issues raised in the text, reminded me of events that had transpired years before in a virtual world called Second Life. SL, as it’s referred to by insiders, was started in 2003 and is COMPLETELY crafted from user-generated content. The platform gained popularity briefly but never quite made its way onto the list of mainstream social networking platforms.
SL had several obstacles keeping it from going mainstream, ranging from technical issues and a steep learning curve to adult and x-rated content. The latter, coupled with the old media’s focus on what they deemed as unacceptable content worked to marginalize the entire SL community. Jenkin’s (2006) sums it up this way, “Old media still defines which forms of cultural expression are mainstream through its ability to amplify the impact of some user-generated content while labeling other submissions out of bounds.” Articles like the recent post, Second Life: what went wrong?, focused on the platforms adult content getting into the hands of children. The author calls SL ‘a disturbing hive of graphic sexual content, extreme rape fantasies, and vendors selling webcam sex…children were inhabiting some of the most hardcore locations’. This may be true but children are accessing porn on the Internet everyday and I don’t hear a call for turning ‘off’ the Internet. Just like the web, SL houses a myriad of communities OTHER than those that are adult-themed.
My friend and old SL resident, Draxtor Despres, has made it his mission to tell the stories that mainstream media has chosen to overlook. An amazing example of user-generated content, his YouTube channel houses dozens of expertly crafted videos telling a very different SL story. There’s the video about an elderly women with Parkinson’s who is finding renewed vigor thanks to her time in SL. The video outlines how mirroring what her avatar does on the screen actually strengthens her neural pathways in real life and allows her better mobility.
Another story outlines the flourishing fashion industry in SL and explores the relationship between avatar and real-life.
Better Way to Learn?
Back in its heyday, SL was seen as a promising alternative for virtual learning. Many institutions created SL islands and universities. Educators worked to provide online students with a more engaging environment. Today many of the university campuses have disappeared and online learning has moved in a different direction.
Educators my have overlooked the fact that learning happens in a very different way in SL. Instead of learning in virtual classrooms, SL residents learn by interacting with the virtual environment. Jenkin’s (2006) references “scaffolding” as a means to help students try out new skills. He describes it this way, “In the classroom scaffolding is provided by the teacher. In a participatory culture, the entire community takes on some responsibility for helping newbies find their way.” This is VERY much a part of SL culture. The older, more experienced SL residents take it as their duty to help ‘newbs’ maneuver the steep-learning curve that is required to truly enjoy SL. The learning happens by attempting to navigate and participate in the community.
Brands in SL?
As SL was becoming more popular and recruiting more die-hard users, brands decided they wanted to market to this extremely desirable demographic. Several corporations, including IBM, coca-cola and KLM, built an in-world presence. In very much the same way consumers have rejected ‘astroturf’ or FAKE grassroots media, SL residents rejected big brands attempting to hijack their virtual community. Corporations attempted to use this new media platform to sing the same old song – buy our products. They built virtual cars, virtual clothes, virtual jewelry, virtual beverages and expected residents to assume some sort of brand affinity. Instead of speaking the language of the residents and giving them the power to assume an active role in their brands, companies attempted to control the message. This was a HUGE failure. IBM, who once held global company meetings in SL has left all but left the space. Companies have pulled the plug on SL but instead of gaining some self-awareness of how they could have better engaged this community, they simply deemed all of SL a huge waste of time.
The branding that survives in SL today are the items that have been built organically by SL residents and NOT corporations. You can still find Converse sneakers and virtual Starbucks coffee shops. Are these brands aware that their brand has been co-opted by virtual avatars? Probably not. Years ago these brands may have tried to shut down this type of content. What would they do today?
Looking at these virtual content creators through the lens of today’s marketers, we see a group of brand advocates replicating items they love in SL. These are the consumers of marketers dreams, referred to as the ‘loyals’ in Jenkins (2006) writings. Marketing luminaries, such as AdAge editor, Scott Donaton, realized years ago that attempting to control the narrative would have negative effects on the brand. Second Life has proven this to be true.