With focus on fact-based, journalistic stories and the most seamless, mobile and cost-efficient technologies available at the time.
What you can expect
In this article I’m going to share with interested readers and fellow educators my key learnings from four years of teaching immersive 360/VR journalistic storytelling in a hands-on training and learning-together approach. Throughout we used the most seamless, mobile and cost-efficient technologies available at the time.
While I taught this course several times for postgraduate students in media business, communication, PR and marketing who joined my class without prior knowledge or experience in journalism and storytelling, the 8-steps-outline can also be applied to the training of young j-students who likely would participate with a deeper affinity and profound experience in story research and multimedia storytelling methods.
First, I’ll pin down the 8-steps which I designed within the limits of one term (at a German University of Applied Sciences), organised in 2 hours in class per week and with overall 5 ECTS workload — equals 150 hours — for each participant. The course comprises sixteen weeks, composed of twelve sessions of hands-on practising and knowledge-sharing with input provided by top experts during the lecture period and followed by — as agreed on with the participants — four weeks of self-organised 360/VR production and distribution during the term-break period reserved for exams and homework. At all steps regular feedback cycles ought to be enabled.
Secondly, I’m going to explain the major challenges for teaching 360/VR storytelling to newbies and how I approached them in tailoring this class and constantly shaping methods and input during the run of this class, particularly during the first run.
Thirdly, I’ll wrap-up with sharing a short list of best practice examples of “360/VR journalistic stories” including a few “responsive VR journalistic stories” published since 2015 by experienced journalists, either independent or working for legacy newsrooms, experimenting with the new immersive 360/VR storytelling format. The selected works complement each other by adding specific criteria which play a key role for 360/VR storytelling and for each I will quickly explaining why I’d show them learners.
Let’s do this!
8-steps outline
How to teach “360/VR journalistic storytelling” — with the most seamless, mobile and cost-efficient technologies available at the time.
Lecture period
(About twelve hands-on training sessions á 90 min. with shared-knowledge input by international top educators and industry leaders.)
1. Experience 360/VR stories and responsive VR stories on both mobile and head-mounted displays.
2. Understand the history and current state of mixed technologies and how to use them for journalistic immersive storytelling, with focus on the most seamless, mobile and cost-efficient devices and methods available at the time.
3. Practice shooting and editing immersive visuals and videos, with focus on the most seamless, mobile and cost-efficient methods available at the time. Easy to use mobile camera equipment with 4k, live-streaming, in-app stitching: Insta360 One, Insta360 One X, Ricoh Theta S, Samsung360.
4. Understand the specific criteria for spatial storytelling and what makes for a valuable 360/ VR story. Then find a theme — do the research — plan, anticipate and draft the story. Students with an advanced understanding can possibly plan for and produce a story along with 2D or/and 3D multimedia hot spots embedded in the primary 360/VR story.
5. Prepare the shooting — interviews — location(s) — equipment — production plan, aiming for distribution on mobile, desktop and head-mounted displays.
Term-break
(4 weeks in a row, self-organised in small teams of diversely skilled students.)
6. Shootings.
7. Postproduction, with focus on the most seamless, mobile and cost-efficient methods available at the time. Easy to work with postproduction platforms for 360/VR stories and embedding 2D or/and 3D multimedia stories, are: Desktop-based: VIAR360 (with trial period, premium only) — Fader (free). Mobile-based: LumaFusion (premium only). Desktop-based with high-end workflows: Adobe Premiere — Final Cut Pro.
8. Distribution: Compatible on mobile, desktop and head-mounted displays. We use Youtube for the primary distribution.
All learning outcomes are projected onto each learner in a two-folded curve: With short individual training assignments and teamwork in small groups of five or six diversely skilled students, working together on one long 360/VR story.
At the end of step 5 every participant has individually produced a number of short 360/VR experiences and multimedia stories from scratch. At the end of step 8 every participant has in a team produced one longer 360/VR journalistic story.
Everyone writes individually a “learning diary”, like a production logbook, reflecting each step of the process, describing the tools used, reflecting insights into the nature of the technology and the new knowledge gained, learning from failures and explaining workarounds and solutions. “Learning diaries” are supposed to be individually designed, mirroring the individual creativity brought into this project.
These logbooks make the experience stick for the creator and provide shareable treasure troves for the next student batch and one way to make innovative methods iterable. Of course, logbook creators have to agree that their works are shared with newbies, which all of my students whom I asked generously did.
If marking is obligatory, the logbook may receive an individual mark and the final 360/VR story a team mark, which both add up to the final mark.
Why teaching 360/VR storytelling matters
It all started for me in the summer of 2015. In conversations with tens of students I had become aware that none of my young German students in the age of 18 to 23 years — many of whom would call themselves “digital natives” — had put on a XR headset — neither low-end like a cardboard nor high-end like a Samsung Gear, Oculus Rift or HTC Vive. What shocked me that also none of them had experienced immersive 360/VR stories on mobile or tablet. And what alarmed me even more, most didn’t even seem to bother about trying it as in their internships they experienced equally either conservative or outright tech-adverse newsrooms, comms teams and marketers; what holds still true for the media industry in Germany, though not for the retail or tech-driven industries like car manufacturers which are offensively working with XR technology in both their production flows and their customer communication.
However, our approach at the “Now Age Storytelling” Institute is just the opposite, since we’re driven to promote creativity, encourage continuous learning and to spend generously on being entrepreneurial minds outside of the “old” system, which in so many aspects has failed the generations of today and of the near future.
Me, coming from a long career in public broadcast long-form journalism, where I began at a time when we still worked analog tapes, and engineers, editorial and artistic colleagues had to rely on open mind-sets for finding innovative solutions for either special effects or unforeseeable technical glitches and challenges on the fly. And believe me, an open-mind-set and intuitive digital understanding is not age-dependent.
Despite the fact that members of my generation don’t count as born “digital natives”, we are, however, “digital naturals”, time-lines of media evolution, turning our experience into an advantage for Millennials and everyone and every profession we work with on innovating media. We’re passionate about and experienced in marrying tech and content. Our mission is to developing digital skills for all in order to lead people to better informed decisions for the future of our open societies and the well-being of our planet.
Use the natural language of experience
As technology changes how we define communication and storytelling, we find ourselves as test subjects within different experiences. Having been following the evolving media technology on a daily basis since the Eighties, I am convinced that everyone is affected by the fast evolving digital ecosystem. Hence, everyone needs to develop at least a basic understanding of digital communication, upskill with digital skills like verifying information and telling stories on the go and learn how to confidently navigate in and personally and professionally grow in the digital ecosystem. With digital we’ve entered the era of experiential communication where illiterates become multimedia literates and multiplayers of shared immersive experiences.
In other words we’ve entered the “Now Age” as Devadas Rajaram and I call it. In the Now Age we have entered a stage where we use spatial computing in all our devices, basically producing an immersive, 3D-layered interactive semiotic system in real-time which allows us to produce and experience 3D information built and rendered in different levels of complexity from anywhere at anytime.
“Reading is a skill that must be learned, but experiencing is just hard-wired into our brains. How cool is it that we as journalists can now finally inform our audiences using the natural language of physical experience?”, asks journalist and innovator Dan Pacheco in a recent conversation with the Global Editorial Network, and answers, “That’s really what the XR revolution is about.”
Currently we have the sophisticated choice between either working with low-end or with high-end XR technologies and both offer certain advantages and come with limitations.
On the one hand, seamless, affordable and mobile technologies enable us to reach out to a large, diverse, inclusive audience including minority groups and communities which are deprived of any privilege and access to resources. Using the same, affordable tools like our diverse audience raises an understanding which goes both directions. Users recognise the methods how the story was produced and can seamlessly use it; while producers understand the use-cases of their users and can produce with much more seamless, accessible and responsive methods.
On the other hand, cutting-edge and costly high-end technologies are often (still sort of) stationary and enable us to only reach out to the small group of first-movers, innovators and, of course, the economically privileged. Content and stories are much more polished, time-consuming and produced by expert teams who produce content and stories which can only be experienced with specific equipment.
In my approach the first option is much more appealing because with low-end technologies we truly harvest the full revolutionary potential of spatial computing and mobile technologies to reach out to potentially everyone and empower people and communities who were/are never included in the era of legacy mass media.
In the entire range of immersive communication and storytelling, mobile 360/VR is one of the most basic experiences which by now can be easily produced and experienced with smartphones and near-smartphones. With these powerful tiny publishing houses in 2019 almost everyone aged over 15 has in her or his pocket, as futurist Benedict Evans points out.
Climb up the ladder of challenges
All of the previous weighing in my mind, by summer 2015 I focused my special interest on how to introduce 360/VR journalistic storytelling to young media students who would soon enter the industry as junior media professionals. So, I developed and started soon after to teach 360/VR journalistic storytelling for postgraduates in an elective class. The seminar was named “Young Lab: Innovationtelling”, taking in postgraduate students who mostly come with a first degree from top ranked academic universities without having learned any practical (experiential) workflows and applicable methods.
My long-term plan was to start with 360/VR mobile storytelling and over time extend the course content to experimenting with other evolving technologies like voice assistants, bots, watches, wearables, drones. As we would always look into the potential for storytellers, our overall goal would always have to be: Find a story and a story angle which is fact-based and journalistically relevant and at the same time worth to be explored with the means of this new technology; which themes, which location, which production workflows would be useful and which not.
On top of this I was determined to add the learning goal — set for me as well as for the student-participants — to always experiment and develop stories with the most seamless and mobile XR available. For a couple of reasons: Using cost-efficient, affordable and seamless technology would mirror the experience of most users worldwide and open our experience beyond privileged users as well as make our workflow fast and seamless as possible as well. In this way we would accomplish both connect possibly to the largest and most diverse group of users worldwide and vice versa enable users to seamlessly experience our stories.
My agile concept turned out to become challenged by a four-factors-disposition deep-rooted in the mind-set of most of the participating German postgraduate students, who, as highlighted before, joined without prior experience in journalism or storytelling. I learned only after the first “Young Lab” had started what that concrete meant:
1. adoring big equipment and large movie settings.
2. taking reenactment for storytelling.
3. inexperienced in going out and speaking to people or interviewing people.
4. underestimating the value of audio and the engaging and immersive nature of voices.
I didn’t lightly say before that student-participants and I would share the most profound learning goals. It long had become clear to me that we had moved into a life-long-learning environment. And, in the case of digital media and now age storytelling, I was also aware of the resistance against digital transformation and the slow pace of adapting to digital methods in German media organisations and media education. So, I decided to address the four-factors-disposition by catering for intensive knowledge sharing with German and international experts and starting a lively conversation about 360/VR storytelling.
Remember, we’re talking about teaching an innovative subject like 360/VR storytelling which comes by its nature without a book of rules, making it crucial to collect comprehensive knowledge which can be taught and shared in a systematic and repeatable way. Also, keeping an open-mind for trial and error as well as working in English or other foreign languages are mandatory for educators and students. After I identified the four-factors on part of my students, I then created four corresponding knowledge and competence areas, we would cover in the course right away from the start:
1. Understanding the theoretical state of mixed technologies — methods, devices, headsets -, with focus on storytelling.
2. Experiencing immersive XR stories with the use of mobile and head-mounted displays.
3. Develop an understanding of: a- How to find and build a journalistic story. b- How to find a spatial journalistic theme and build a journalistic immersive, audio-visual, multimedia, interactive 360/VR story with mobile technology.
4. How to produce and distribute an audio-visual 360/VR story with mobile technology.
Since the curriculum management at my university didn’t allow us to learn in more flexible units and intensive workshops, we would have to meet weekly for 90 minutes only. Working within this logistical frame, I organised successive input in all four knowledge areas during the first weeks of a term. We participated in knowledge sharing sessions and started in parallel every fortnight a short producing cycle of 360/VR content, experimenting with different methods, apps and workflows, beginning with 360 photo storytelling with smartphones.
Get international experts on board (online and offline)
In 2015 I already had reached out to innovative international educators and entrepreneurs, namely in Germany, in Denmark, in India, in the US, in the UK and Sweden and successfully created a productive synergy of getting the pulse of the rapidly expanding XR universe. We started an agile cross border strategy of versatile offline and at times online knowledge sharing and project-based collaboration which soon transformed into a sustainable transdisciplinary long-term network with senior experts from the industry and educational sector as well as students and junior experts.
This international network of “Yay”-sayers belongs to day to the best and most encouraging experience in my career as well as for the dozens of students who became an active part of it until now. The beauty of it is also that students who continuously experiment and work with XR and connect with international experts in that field soon belong to the 10% of international XR experts in the news and media industry.
During the next four terms in a row we benefited from a core network of senior experts who passionately shared their knowledge with us, all eager to empower more young media students with cutting-edge knowledge, plant the seed of immersive media skills and connect with young minds and fresh habits.
Maximilian Schmierer, the young CEO and profoundly experienced XR pioneer at “b.ReX” Manufactory for Digital Reality would regularly invite our whole class over to his company in Stuttgart West to present an ever so inspiring introduction into the state of the industry, introduce us to the latest methods and devices and then lead us to a real XR parcours where everyone experienced stories through state-of-the-art XR devices like Hololens or HTC Vive. For my students this quality time always was kind of a mind-blowing eye-opener and kickstarting their interest in new ways of immersive storytelling.
For more theory and knowledge sharing about the history of immersive content, XR technologies and how to apply them to storytelling Christopher Caldwell’s talent to make rocket science easy and pleasant to understand was an unbeatable inspiration. The young creative director and renowned expert for all 360/VR and A.I. at “b.ReX” Manufactory for Digital Reality took us frequently on a nerdy journey split into the history of stereoscopic methods and the concrete high-end workflow how to produce 360/VR stories with the use of camera-rigs and post-production tools provided by “Adobe” and “Unity”. Although, as mentioned before, in the “Young Lab” we’d work with low-end tools and mobile methods, we wanted to empower the participants with the comprehension of the entire spectrum of 360/VR story production.
For deepening the understanding of 360/VR innovation applied to journalistic storytelling Devadas Rajaram, new media educator and journalism innovator at the Asian College of Journalism in Chennai, collaborated with us. He shared with us newly tested immersive journalistic methods, smart agile workflows and best practice examples, partly through virtual hangouts, partly as a regular visitor to the “Young Lab” and other classes he taught at Stuttgart Media University.
Experienced in long-form 360/VR and responsive VR narratives Michael Geidel engaged in a lively offline and online collaboration with our “Young Lab”. The pioneer XR producer and currently CEO at Actrio Studio, based in Leipzig and Potsdam, introduced us to his genuine systematic “Pizza”-approach how to anticipate, plan, draft and shoot a 360/VR story.
Journalism innovator Dan Pacheco from the S.I. Newhouse School for Public Communication at the Syracuse University in the US, belongs to the most approachable and inspiring colleagues in this field who also joined us, once for a one-day-workshop on high-end methods like volumetric visuals and the use of the game-engine “Unity” for VR storytelling.
Swedish educator and sonification genius Niklas Roennberg shared with us a wonderful presentation on evolving spatial audio, nurturing the necessary basic understanding in this powerful area, really, a centerpiece for immersive storytelling.
The Youtube channel “Journalism 360” by pioneering 360/VR expert Sarah Hill provided at an early stage lots of insights into applicable 360/VR equipment and journalistic storytelling, useful tips which are still relevant and online available.
And the incredibly inspiring team of international researchers at the non-profit Max-Planck-Institute for Intelligent Systems (MPI IS) in Tuebingen, in particular junior researcher and PhD candidate Matthias Hohmann, introduced us to the world of “Locked-in” patients and the latest A.I.-based research into how to communicate with them. “Locked-in” patients lose over time their muscle power while brain power, thinking and memory remain intact; world famous physicist Stephen Hawking suffered from this disease.
Immersing users into the world as viewed by a “Locked-in” patient was my example of an ideal story to be told with the use of spatial 360/VR-technology, and we introduced this story setting to the first two student batches. Although the students never shared our passion for this story and hence it was neither produced nor told in any of my classes, it gave everybody a clear idea about the potential of both spatial 360/VR-storytelling and A.I. super powers at our hands and in our pockets in the Now Age. In my "Innovationtelling" approach it’s important that journalists tell stories about these technologies, how they positively impact all our lives as well as the downsides and to spread the knowledge.
Following the collective approach to gather the most complete, most updated, and most comprehensive knowledge about mixed technologies and 360/VR storytelling in the shortest possible time by connecting and collaborating with the industry and senior experts, eventually the student-participants were challenged to identify and find an appropriate story to which a 360/VR spatial approach would add value for users. I developed the “Innovationtelling” method and following this approach it’s the responsibility of journalists to tell stories about these technologies, how they positively impact all our lives as well as discussing the downsides, and to spread the knowledge.
While I had prepared a network for my students to easily research and work on the story of a “Locked-in Patient” which could be told with a 360 camera and production workflow, immersing users into the head of a patient, my students didn’t share my interest in this story and decided to find their own story points and angles. Despite the limits of time and experience, I also believe, finding and defining the story is a very important step in the whole process and I would never suggest a story to students again. To support my students in the difficile mission to find a journalistic story worth to be told by the use of 360/VR methods, each term I introduced them to a collection of best practice examples available at that time. Here I list the most important ones, each highlighting a specific feature and production angle genuinely related to 360/VR journalistic storytelling.
Best practice examples of 360/VR stories and responsive VR stories
1. Show students the immersive power of a true story and the lasting impact of original voices - “Kiya” is a responsive VR “literary documentary” by Nonny de la Peña - the “godmother” of VR journalism - and her team at “Emblematic Media” in cooperation wit “AlJazeera” in 2015. It reconstructs the scene of two sisters trying in vain to save a third from being shot by her ex-boyfriend. The events were recreated by actors wearing motion capture suits, and the environment was based on location and crime scene photographs.
2. Immerse students into a world which is hard to access and hard to understand as well as underreported. Users will develop a more empathetic perspective by experiencing this strict journalistic responsive VR doc “Solitary Confinement” reconstructing a US solitary confinement prison cell by “The Guardian” in 2016.
3. Let students experience an autobiography at places where it happens. Kenny tells his life story about the time he spent in solitary confinement in a US prison and what happened when he got out in "After Solitary” by “Emblematic Media” in cooperation with “PBS Frontline” in 2017. First time a story is produced with volumetric video methods.
4. Students can experience the radical change of perspective enabled by a spatially presented story in Robots and humans team up at Amazon warehouses which is part of the year-long “Daily 360” productions by the “New York Times” in 2017. The immersive 360/VR video report is told by workers and viewed from the perspective of boxes.
5. Immerse students in the space of a refugee camp in Lebanon, with embedded hot spots presenting multimedia stories of refugees telling their own stories in this 360/VR journalistic reportage "Four Walls" produced by pioneering journalist Rashida Jones in cooperation with the “International Red Cross” (IRC) in 2017 and there’s a same-named app available.
6. Learn more about the introspective power of a 360/VR camera and immerse students into the head of an autistic girl in the UK who experiences a birthday party in her home. This extraordinary intense and well researched story is titled “The Party”, produced by “The Guardian” in 2017.
7. Meet my absolute favourite VR story for learning the immersive power of audio storytelling: “Make noise, Raise your voice” by the “BBC VR Hub” in 2018 - hands-free accessible through the “Oculus Go”. Let students interact with their own voice and impact the intensity of this artfully produced story of the suffragettes and how a few women started one of the most important movements in history, using original archive material.
8. Last, not least, I proudly suggest to immerse students in the 360/VR story of Katharina, a transgender woman in Stuttgart, Germany, told by herself and my students in “Relabeled” published under the umbrella of the “Young Lab” in 2018. The story is an example how we can empower minority voices when we’re hitting the ground, talking to people, helping them to tell their stories — with the use of affordable and for all accessible mobile technologies.
--------------------
For newsrooms, comms teams and organisations looking to get into multimedia storytelling and product development with immersive technologies focusing on seamless and cost-efficient methods our Now Age Storytelling Institute offers comprehensive support in hands-on trainings and strategic deployment.
Find out what we think will happen in multimedia storytelling with emerging technologies in 2019 and the near future and follow @devadasrajaram @memplexx @nowstorytelling on Twitter.