While all of the multimedia stories listed on this blog integrate different forms of media in some way, there is sometimes one form that drives forward the narrative with other forms used to support it. This allows for a more cohesive narrative where the user can focus their attention on one form for the story while choosing to pay attention to other forms according to their preference.
Welcome To Pine Point includes just about every media you can think of – photos, videos, audio clips, graphics, animations, and music – all integrated together about as seamlessly as possible. But the basic story that makes use of all of these things is told using text. The words push the story forward and provide the context for all of the other elements. Text describing the creation of the town – “Pine Point has none of the organic growth of most towns … Instead it was Economics 101” – is paired with animation of the town coming together. An animated bird caught in a rain storm is paired with text talking about bad times and the closing of an entire town. A photo slideshow of the area comes with the explanation “Pine Point is surrounded by dozens of otherworldly pits…” This doesn’t mean that there is an over reliance on text. Indeed, often only a few choice sentences are used on a page. But the text serves to tie together the story, an important function in such a rich multimedia experience.
For Want Of Water uses five videos to drive the story. The videos use a relatively traditional documentary style. But while they play, other forms of media react around them. A Bing map shows the exact location being discussed in the video. An info box that gives extra information about certain details mentioned in the video. There is the occasional extra video to watch in the info box that pauses the main video when played. There are pop-up text bios of onscreen interviewees when they talking. All of these extras can easily be ignored by the user without taking much away from the content of the main videos. Here the multimedia is optional. Other stories like Talking To The Taliban, A Year At War, and Behind the Veil similarly lead with videos in this way.
Audio drives the story in This Land, which builds on the story of journalist Dianne Whelan’s journey to Alert as told by narrator Karin Konoval. While the user listens, they can flip through images and videos from the trip, read about the travel conditions, and view a map of Whelan’s progress. But the story is told by the audio. Indeed, the straight audio story is available for download without any extras at all.
And multimedia stories with one foot firmly planted in traditional print techniques like Our Choice and A Matter of Life & Death put the focus on the text with other elements like photos, videos, text pop ups, and maps used as extras.
The use of a lead media form is particularly useful for linear narratives that seek to tell a single story from start to finish. The lead media prompts the user to continue on with the story instead of getting lost in the multimedia.
While a single lead media keeps things simple, different forms of media can be efficiently used together to similarly keep things moving. Days With My Father, the simplest story listed on this blog, balances photos and text in a way that utilizes the strength of both. And Hope: Living & Loving With HIV In Jamaica effectively combines the text of poems with audio recordings of the author reading that text (a combination that works particularly well for poetry) while linking to secondary video interviews of the people the poems were written about.
In 2007, poet and writer Kwame Dawes made five trips to Jamacia to learn about the impact of HIV and AIDS there. He talked to both people with HIV and people in the medical community fighting the disease. Based on the stories he collected, Dawes wrote 22 poems, which he calls “anthems of hope and alarms of warning.” Hope: Living & Loving With HIV In Jamaica is a multimedia story that combines those poems with video, audio, images, and music.
How It Works
The 22 poems are the focus of Hope: Living & Loving. They are presented both as text and as audio. Dawes himself recites the poems. For nine of the poems, there is also an audio recording of a musical version with the words as lyrics. For four “featured” poems, there is a slideshow version, where each line of poetry is paired with a photo. The slideshow plays in step with Dawes’ recital of the poem and the text appears, line by line, like photo captions. So when the user hears the line “the bodies broken, placid as saints, hobble along the tiled corridors, from room to room,” they see a photo of a man, bent over, walking down the hallway of a hospice.
Most of the poems are linked to profiles of the interview subjects who inspired it. These profiles include one to four short interview clips that deal with particular topics, such as “on accepting those with HIV” and “on sex, sin, and death.”
There are also two longer “infocus” videos that discuss the stigma surrounding HIV in Jamaica and what it is like to live with the disease. These look at the bigger issues involved, providing some context for the poems and the people interviewed
Finally, there is an image gallery of related photos. These photos also appear in the background when other content appears on the screen.
What Works, What Doesn’t
Without judging the quality of the poems themselves, something I am certainly unqualified to do, I can say that the poetry really benefits from the multimedia treatment. It’s an oral form as much as it is a written one. When it is read out loud (by the poet himself no less), the rhythm of the lines come through.
This use of audio doesn’t particularly need a visual accompaniment, but when there is one for the four featured poems, it is generally well done. The images are subtle and don’t compete with the words for the user’s attention. It’s always important for the supporting media form not to compete with the lead form.
The supporting videos that give context to the poetry are all well used. The infocus videos use a variety of footage to talk about two particularly important subjects. The briefer interview clips give the user a personal connection to the various subjects. When someone talks on camera about how the felt when they first found out they were HIV positive, it can’t help but be emotional. And by linking the poems directly to the particular subjects who inspired them, the two forms are connected.
How a multimedia story comes together and is experienced by the user depends in large part on how the creator chooses to package it. While all of the multimedia stories listed on this blog are digital creations, they aren’t all built the same way.
The majority of the stories listed here were created using Flash. Without getting into the more technical aspects of the program, I want to point out some of the advantages Flash-based stories have.
One, they look a lot better than traditional websites. Consider the difference between Waterlife (or any of the NFB projects) and Project Jacmel. Aesthetically, it’s night and day (although to be fair to Project Jacmel, a beautiful presentation wasn’t really the point).
Two, they are easier to navigate. Utilitarian hyperlinks work well enough to connect different websites together, but there are better ways to move around a single story. In Out My Window, links to individual stories are embedded in a 360° collage of images from an apartment. When the user rolls over a hot spot with the cursor, the image transforms, alerting the user of the link. In How Much Is Left? links are embedded in an interactive timeline. In Beyond The Stoop, the user clicks on the image of an apartment building to listen to the story of the individual who lives there. In this way, the navigation is part of the design itself. Yes, stories created with Flash also have more traditional menu navigation. The above three examples all use menus in addition to the hotspot navigation. But they aren’t limited to it.
Even without using this kind of hotspot navigation, navigating through a Flash site can be smoother than just clicking on a hyperlink. Days With My Father uses a very simple form of page navigation. But just sliding between pages creates a more unified experience than jumping directly from one page to another.
Three, Flash sites are more immersive. A website can be a distracting thing, with links to a multitude of other content vying for the user’s attention. Using Flash can help focus the user’s attention on the story itself. Indeed, many stories created with Flash have a full-screen option to remove the browser altogether. The Big Issue doesn’t just immerse the user, it tries to make them a part of the story using a “choose-your-own-adventure” style of navigation. And while I’m not sure it is entirely successful in pulling this off, just attempting to do so would be impossible on a traditional website.
Four, Flash allows different forms of media to be integrated more seamlessly. In Welcome To Pine Point, a single page can contain video, photos, animation, and text (plus the music playing in the background). Every element is designed to fit together by carefully matching the look of each one. In Soul-Patron, an animated stuffed animal wanders through video footage and provides information to the user through speech bubbles.
Of course, traditional websites have their advantages. Those hyperlinks are good for some things. Commenting, linking, and sharing – some of the real benefits of the internet – are all easier to do in HTML. And adding piecemeal content would be difficult to do in Flash, although as Love Letters To The Future demonstrates, is not impossible.
But the days of Flash dominating multimedia storytelling may well be numbered. Apps created for tablets like the iPad offer all of the advantages listed above and then add a couple more.
First, the shape and size of the iPad makes it an ideal computer for content consumption. If Welcome To Pine Point was designed to be a “liquid book,” wouldn’t it be better to read it like a book? The immersive experience Flash allows for is somewhat belayed by the fact that that experience is being filtered through a laptop or desktop computer, machines that generally do not favour long-term reading.
Second, the touch screen of the tablet allows for direct interaction with content without the need for a cursor. This is especially useful for interactive multimedia content. Videos can be triggered with a tap. Images can be blown up to full screen with a flick. Pages can be turned with a swipe. The excellent interactive infographics in Our Choice benefit from this hands on approach.
And as more and more traditional publications create apps that include various forms of media to complement their print editions, tablet computers may very well become the medium of choice for multimedia storytelling.
Many multimedia stories take advantage of the digital space to include multiple entry points and a structure that allows the user to navigate wherever they want without disrupting the narrative. New media theorist Lev Manovich calls these “hyper-narratives” where the user is free to follow multiple trajectories through a database of connected elements. I’ll call them non-linear narratives. There’s still an overall leaning, but it’s kept loose. It’s up to the user to connect the pieces together to uncover it. As Katerina Cizek says of her non-linear interactive documentary Out My Window, “Small tales that, as they add up, create a collage of meaning, of experience. Together, subtly, gently, the stories accumulate into epic narratives.”
Indeed, Out My Window is a great example of a non-linear narrative. There are 13 different apartments from different parts of the world, each of which offer multiple stories. But common themes of home and community emerge. Someone from Johannesburg talking about violence in their neighbourhood is echoed by a Chicagoan. Someone from Prague talking about urban renewal after a period of conflict is echoed by a Beirutian.
Similarly, two New York Times stories connect multiple stories from a specific geographical area to get at a particular time and place. Anticipation On A City Block and Beyond The Stoop both interview different people from a single city block to talk about community, the former in the context of President Obama’s inauguration, the latter more generally about New York living. None of the individual stories are particularly detailed or deep, but they are personal, and taken together, give the user at least a small sense of the varied life on the block. The user is left to consider how these different people live together, even though these relationships are not made explicit.
Such non-linear narratives also have the advantage of allowing the user to consume small amounts of content at a time, a form of consumption well suited to the Internet. And should the user only have the time or inclination to look at part of the content available, they are free to do so without feeling like the story is incomplete.
However, by spreading meaning among many smaller stories, they can have more difficulty generating the kind of impact a more unified narrative can have.
For example, the story Waterlife communicates a small amount of information about 23 different topics that cover everything from fishing to pollution to dredging. The user is never given more than a cursory review of any one issue. Should they want to know more, they have to click on outside links and leave Waterlife, which interrupts the narrative. There are no characters for the user to connect to or devices to tie the different pieces together beyond the navigation and common topic. This makes for a rather impersonal experience. In contrast, Out My Window, Anticipation On A City Block, and Beyond The Stoop are able to maintain more unity by focusing on shared human experience as much as topic.
Multimedia stories can also be more traditional linear narratives with an obvious beginning, a middle, and an end.
This Land follows journalist Dianne Whelan as she travels more than 2,000 kilometres by snowmobile across Canada’s arctic with a group of soldiers. It is divided up into 12 chapters, one for each day of the journey. Everyday Whelan travels further and learns (as does the user) something new about life and survival in the arctic. Eventually, she reaches her final destination, the safety of the military base at Alert. And while the ending is somewhat anticlimactic (everyone is asleep when they get to Alert), the journey allows the user to connect with Whelan in a way that wouldn’t be possible in a non-linear narrative.
Another NFB story, Welcome To Pine Point, was designed as a liquid book (although less book-like than other stories like Our Choice). There are pages divided into chapters that the user flips through one after the other. Details about the town and the people who used to live there are filled as the story progresses. There is never too much information at once. The user is drawn in slowly, making for a stronger connection to the characters. This type of storytelling demands that the creator remains in control of the narrative.
Done well, both non-linear or linear multimedia narratives can be very compelling. While the digital space certainly allows for more non-traditional forms of storytelling, it certainly doesn’t mean the end of more traditional techniques. Indeed, as stories like Welcome to Pine Point and Our Choice demonstrate, the basic book form can find new life online.
Not all multimedia stories come into beings as a single, unified package. Some come together bit by bit, one piece at a time. Such narratives are necessarily non-linear to some extent because the ending hasn’t even happened when the story begins and interesting bits of content that come up in the middle don’t necessarily fit neatly into a larger narrative.
This kind of content creation has a number of advantages. One, it allows the user to explore a topic in relative real time, as it is actually happening on the ground. Should a situation be ongoing, a war or a disaster for example, it might make sense to do this.
Consider the Globe and Mail story Project Jacmel, which brings together a number of stories about the rebuilding of a city in Haiti after the earthquake there. The story follows seven subjects over time, from the immediate aftermath of the disaster, through the rebuild, to the future. Updates were posted fairly regularly to give readers a chance to find out what was happening to the characters they had been introduced to. There were even individual RSS feeds for each subject to keep users in the loop. A Project Jacmel blog provided even more up-to-date information, although in smaller bites. A year after the earthquake, the Globe published more content taking a closer look at Haiti one year on, tying together some of the threads from the previous coverage. By publishing content in this way, the Globe gave the reader a sense of immediacy and connection that wouldn’t have been the same had they put together Project Jacmel together as a single package a year after the earthquake. The timeframe helped the storytelling. And because it was ongoing, it gave users a chance to become involved. Included in Project Jacmel were articles about how to help the people of Haiti.
Similarly, the New York Times story A Year At War looks at the U.S. led war in Afghanistan through a single battalion. It begins with deployment and ends with the battalion coming home. Along the way, the user is introduced to numerous soldiers through interviews, sees real life combat, and gets some perspective on some of the challenges facing soldiers in Afghanistan. It would have been a lot of content to take in had it been delivered all at once. But doled out in pieces, A Year At War leaves room for small details (such as seven short videos that cover smaller topics like the death of a soldier and the war dogs the battalion adopted and nineteen even shorter video “moments” that offer brief glimpses into life in Afghanistan). Comparatively, Ian Fisher: American Solider, which covers similar ground although in a different war, and was published in three parts over three consecutive days, is much more focused, covering the experiences of a single soldier. Ian Fisher: American Solider delves much deeper into that one story than A Year At War does for any single solider, but A Year At War manages to wider scope that the other can only hint at.
A second advantage of piecemeal content creation is that it fits well with the newspaper publishing schedule. Newspapers are one of the main sources of multimedia stories. They mostly publish smaller articles daily, which, taken together, add up to coverage of a given topic. When an online shell is created to collect these stories together, this coverage can be better navigated by the user and the various narrative threads can be more easily pulled together. This also makes it easier to repurpose content originally created for the newspaper. Many of the articles that make up Project Jacmel were also published in the print version of the paper.
A third advantage is that it allows for more user-generated content. When a site is live and constantly growing, it makes it easier to invite the user to submit additional content instead of just commenting on what has already been made. A Year At War includes a section of content submitted by the soldiers themselves. Another piecemeal site about war, Assignment Afghanistan, has a similar section. And other sites like Test Tube and Love Letters To The Future are almost entirely based on user-generated content. If they didn’t grow over time, there wouldn’t be much of a point to them. Why submit an activity to Test Tube if not to compare your answer to what other people have said?
The trick with piecemeal content creation is to maintain easy navigation, even as new content is being added regularly. Returning users need to find it easy to view new content without having to sift through things they have already seen. And ideally some form of overall narrative needs to be imposed, something that can be difficult when the content has only a topic in common. Some stories do this better than others. Because it has its own site A Year At War is easier to navigate than Project Jacmel, which is spread around the Globe website. But Project Jacmel because it organizes stories according to subject and time, does a better job of connecting topics than Assignment Afghanistan, which makes little attempt to connect different stories together beyond their common topic.
When a story uses a short embedded audio recording of a subject or expert speaking. These clips play on their own instead of being part of a video. They are either activated by the user or play automatically when the user navigates to the page. Embedded audio clips are often paired with photography to give the user something to look at while they listen.
When sentences or words from an audio recording or video appear on the screen as text. These captions most often appear in photo videos where they are less likely to interfere with the image. They are usually used to emphasize a key point or particularly poignant line, like a pull quote in a magazine story.