Archive | Immersive Technology RSS feed for this section

GDC 2014: The Pursuit of Loneliness

26 Mar

Last week, I attended the 2014 Game Developers Conference in San Francisco — it’s pretty hard to find a group of companies and people that is so unapologetically hell-bent on mind-melting. I won’t be providing yet another rehash of the headline-making announcements from Sony and Oculus, but I am excited to share some of the other experiences that caught my attention.

Before we get started, about a year ago, I had a lot of good things to say about my first experience wearing Oculus Rift. I’m still a big fan, but it was interesting to see the number of different ways that exhibitors incorporated it into their demos at GDC. Instead of being the star attraction, it was often a complement to some other amazing technology that the company was showing off. Not only is the experience of total audiovisual isolation/replacement evolving into a mainstay, but additional technology is being incorporated to allow one to remain in the experience longer and deeper.

OK, enough reflection. Onward into the bold, bright, and lonely future!

IMG_0682

Also effective as sunglasses

Sulon: I think the business plan for this company is, “Let’s take everything and then do that.” The Cortex (the crazy thing on my head in the picture)  uses an Oculus Rift to provide both Virtual Reality and Augmented Reality in conjunction with spatial scanners, handheld game controllers, wireless technology, a video camera, an Android tablet, and who knows what else. What does this mean? The camera provides a video feed of your environment that is displayed within the OR, but it is “augmented” with virtual objects that you could shoot at using the controllers. One of the augmentations looked like a teleporter beam and when I walked into it, the entire environment changed to virtual reality. However, in both cases, the sensors incorporated the physical walls of the room into the digital environment. In VR mode, it was possible to physically walk through the real room, experienced as a purely digital room, and proximity sensors would beep to let you know if you were about to walk into a wall. The Cortex is an ambitious mashup of cutting-edge technology and if they can work out the kinks, the whole might be greater than the sum of its parts.

Condition One Camera

Be careful where you point that thing!

Condition One: This camera was quite a spectacle on its own. It shoots in 360 degrees simultaneously in high resolution and at a high frame rate. At the booth, I used Oculus Rift to watch a short movie that was shot with the rig. Actually, it was three unrelated shorts that provided a good showcase of different styles and content. This was probably one of the only booths at GDC that featured an OR in a non-interactive context. However, part of the allure of VR is the experience of being instantly transported to another world. In a gaming context, I often forget what I’m supposed to be doing because I enjoy luxuriating in the visually rich, all-encompassing surroundings. It was a nice change to be able to relax and observe and not worry about being shot at or accomplishing something. It was also refreshing to see live action content as opposed to CG. Although these shorts had a non-narrative, verite style to them, I could see how this type of production could also be effective for a sophisticated and complex narrative piece.

Not me on Virtuix Omni

Not me rigged with PrioVR

Virtuix & PrioVR: I’m lumping these companies together because they fall into the category of, “Wow, this looks super cool, but it’s taking too long to cycle people through the demo so I’m going to pass.” Both companies feature technologies that complement the Oculus Rift with a sense of full-body immersion. The Virtuix Omni allows you to physically pace through a virtual environment using an array of proximity sensors embedded in a treadmill that parse data generated by special shoes that you need to wear. Seriously. PrioVR uses inertial sensors attached to various parts of your body to provide full body tracking. If any of this sounds remotely interesting and you were alive during or heard of the 1980s, I highly recommend Ready Player One. This book seems incredibly prescient except that many of the things that it anticipates happening 30 years from now happened last week. And it was written two years ago. Oops.

IDNA: In their own words, “IDNA is a new kind of storytelling experience at the crossing between an animation film, a choose-your-own-adventure book, and a video game. Each scene of the story is designed in 360 degrees; it’s up to you to decide where to look, simply by moving your device around you. The narrative is never twice the same, as your focus will seamlessly influence the course of the story.” Basically, it’s the narrative experience I was jonesing for in my blurb about Condition One. Talk about wish fulfillment. Recognizing that not many people have an Oculus Rift, they developed a “poor-man’s VR” version using an iPad. Holding the iPad up and rotating in a circle provides a decent emulation of wearing a HMD and turning your head. At the worst, you might be mistaken for one of those people who uses their iPad as a camera. I know it sounds like I’m poking fun at them, but I’m not; I think it’s a great idea, and a clever and effective workaround for a technology that is still very much in transition. Even better, this is basically the product of a handful of smart folks from Switzerland who are doing something creative. It’s incredibly annoying when Google/Motorola do essentially the same thing, restrict it to a phone that nobody wants, and make it sound like it’s the second coming of bread.

There was more to GDC than VR, and I was glad to see it. After spending about ten continuous minutes in one of these demos, I found it very difficult to reintegrate with my surroundings. I was trying to have an intelligent conversation with one of the representatives at the booth, but the part of my brain that was responsible for speech and words had come totally unhinged. Welcome to your future.

And now for some more than honorable mentions of people who are just doing fun, cool, stuff for its own sake.

A group of people playing together. No, this wasn’t the history exhibit.

Hot Shots: This game won the All In One Platform Award in the Intel App Innovation Contest. There’s no immersion and no storytelling, but I’m mentioning it for a couple reasons. 1) The developer was hosting the demo, and he was very excited to be there. In his own words, winning this contest created a massive change in his life and he was thrilled. It was contagious to see someone so excited and optimistic and talented. 2) His game was fun. It supported up to nine people playing it at once around a fairly big touchscreen. I appreciated that it’s too epic to play on a phone or a tablet, not really playable on a wall-mounted TV screen, and basically designed to bring people together to have fun. Total chaos. I was horrible at it. Still had a great time. Hope this is the first of many wins for this team.

Sorry, no points for urinating in the classroom

Tenya Wanya Teens: This game was in the alternative control exhibit. It’s a self-described “party game” for two players who each have about 16 buttons. The gameplay is simple. Press the colored button to trigger the action that’s appropriate for the situation. Actions include urinating, farting, kicking, punching, confessing undying love, etc. There’s just one wrinkle: the colors of the buttons keep changing, so not all actions are appropriate for every situation. The game lasted about five minutes and we laughed the whole way through. Unexpected, random, zany fun.

Goat Simulator: Thinking about the fact that people got together to make this has become my new happy place. Follow the link, watch the trailer. This is why we have games. Thank you Coffee Stain Studios.

Advertisements

Test Track 2.0 = Attractions 2.0

7 Jan

I’m just going to come right out and say it. I’m really just not into cars that much. Sure, I went through my pre-adolescent fixations with Hot Wheels and The Blue Flame, but as I got older, I grew to see a car as just another option that could get me from point A to point G. That said, I’m not completely immune to their charms; I could probably recite a full traffic school curriculum on account of an inexplicable magnetism for speeding tickets. However, even in a lottery fantasy scenario, a fancy car would still rate pretty low on the list.

Maybe it’s just real cars that I don’t like

Finding myself at Epcot, the day before IAAPA (which is another story entirely,) I surveyed my options and approached Test Track with an academic indifference, knowing that I would ultimately try to experience everything the park had to offer. I understood that the show had been redesigned within the past year, but I had not experienced its previous incarnation, so I really had no idea what to expect.

The first thing that got my attention was that the queue temporarily deposits you into a room filled with touchscreen kiosks. I picked up an RFID card, tagged into the kiosk, and found myself designing a car by balancing four fundamental characteristics: Capability, Efficiency, Responsiveness, and Power. OK, design is a bit of an exaggeration; from a technical standpoint, it’s more of a sophisticated menu system that allows you to select from a number of pre-designed models based on your personal preferences. Still, this type of interactivity was a first for me in a queue.

Sammy approves of Test Track

Before boarding the ride itself, I tagged my RFID card and “uploaded” my designed vehicle into the ride vehicle. The narrative of the ride is that you are testing your vehicle on a specialized track. Each phase of the ride tests a different characteristic, and features a monitor that ranks the uploaded vehicles, which makes the experience feel very personalized in an unexpected way. The ride itself was beautifully designed with a streamlined aesthetic that felt convincingly contemporary and even a bit futuristic, which is no small feat in the context of the dynamic technological wonderland of daily life. The Power test is essentially a speed trial that hits a speed of 65 MPH, making Test Track one of the fastest rides in the Disney universe. As the vehicle approached the unloading area, I thought the ride had a little bit of everything including high-speed thrills, and I felt pleasantly surprised, wondering what I would ride next. Little did I suspect that the experience was far from over.

Not a popular option in the 9-13 demographic

As I exited the vehicle, the first thing I noticed was that the younger riders, especially the boys, were FREAKING OUT! In a good way! Kids were tugging at their parents, begging to ride again before their feet had even hit the platform, and by the look on the parents’ faces, I could tell that this was not the first such request of the day. Boys were swaggering about how their car dominated the Power test (although low marks on the Efficiency test were conveniently ignored.) There was a buzz of infectious and uplifting energy that felt very special. Test Track is not a superlative thrill ride, doesn’t feature any fictional characters, and is not necessarily an icon of themed design, but these young riders showed a level of pride, ownership, and excitement that would be a high point in any designer’s career. I sought the exit so that I could get back in line and check it out again, but that turned out to be not as easy as I would have expected.

Play at your own risk

The first post-show area features a giant screen where I tagged my RFID card and saw my vehicle’s aggregate score compared to those of the other riders around me, along with a Daily Best. I noted with some disappointment that I was about 15 points shy of the Daily Best. The irrationally competitive side of me that mercilessly crushes anonymous opponents in in-flight, seat-embedded trivia games wanted the most Efficient, Capable, Powerful, Responsive car in all of Epcot on that November day. It had to be buried somewhere in that design kiosk and I was going to find it! I continued toward the exit with renewed intent.

And entered another room with touchscreen kiosks that I almost blew right past, but it was early in the day and I couldn’t resist those screens. At these kiosks, I designed a commercial by mashing up a narrator, music style, setting, and vehicle attribute into a short animation featuring my car, which I emailed straight from the kiosk. Mine was a cowboy talking about how powerful my car was while jumping around on the moon with a disco soundtrack. It was so much fun, I did it twice before heading toward the exit.

And found myself in yet another area with even more kiosks that allowed me to customize the look and performance of my car to a much greater degree than the queue kiosks. I fell into another rabbit hole doing this for a while, uploaded my design to my card, and brought it to another interface where I uploaded it to a virtual simulator and drove it around a video track. At this point, I started to realize that there was a possibility that I might spend all day at Test Track and I hadn’t even set foot inside the World Showcase yet.

Weird

I segued into another area that was essentially a giant showroom for Chevrolet, the sponsor of the ride. I spent the least amount of time here since, you know, I’m not a car guy. However, I couldn’t help noticing that people were actually stopping to take pictures in front of the cars and check them out. Personal preferences aside, GM is a perfectly capable industry leader when it comes to making automobiles. However, GM cars do not have the fantasy appeal of  Ferrari or Porsche or Lamborghini, but at this point in the experience, I was starting to feel pretty good about Chevy. Corporate sponsorship of attractions goes all the way back to Walt, but I cannot remember a more symbiotic and positive example. Finally, after all this, the exit through retail felt gentle and inoffensive instead of the harsh return to semi-reality that it can sometimes be.

Leaked concept art for Test Track 3.0

To my surprise, not only did I like Test Track, but I loved it, and my enthusiasm extended above and beyond the ride itself. While many attractions provide a “best-case scenario” approach to the decidedly mundane and obligatory aspects of waiting in line and exiting, Test Track engages and involves you at every turn. From an experience design standpoint, there is not a single wasted opportunity to entertain and delight the guest. In fact, the ride itself is less of an inflection point in a humdrum routine, and feels like one of the many components of the overall experience, albeit one that is markedly visceral.

In the past year, I had the privilege of experiencing some of the most outstanding attractions in the themed entertainment world, including mind-blowing experiences at Universal Studios Hollywood, Islands of Adventure, and Cars Land at DCA. While those parks feature unforgettable, world-class, game-changing achievements, there is something about Test Track that announces the true future potential of experience design. I am a strong believer in the potential of interactive technology to take immersive experiences to a new level, but as designers, we must always be mindful of not letting the tail wag the dog. In my opinion, Test Track not only strikes a harmonious balance between technological innovation and thoughtful storytelling, but does so in a bold and fearless fashion that has truly raised the bar for the industry.

I never did figure out how to design the best car of the day, but my happy memory of the experience made me start counting the days until my own kids will be old enough to ride it with me and take a crack at it.

2014 Digital Media/Entertainment Trends

17 Nov

notliketheothersI was recently asked to research and report on the digital media trends that would have the most impact on the entertainment world in 2014, and I wanted to share my findings.

NOTE: If you are new to this blog, please note that this is not the ordinary fare. Will return with more posts about mind-blowing experiences and technology, once I return from IAAPA.

First the list, in case you’re a “just the facts” kind of person:

1. 2nd Generation “Smart TVs” Will Establish a New Platform Paradigm
2. Cross-Platform OS Convergence on Non-PC Devices
3. Entertainment Platform Convergence
4. Biometric Gaming Leads Innovation of Quantified Self
5. Internet of Things Creates New Opportunities for Location Based Entertainment
6. Google Glass Will NOT Take Off in 2014
7. Augmented Reality Will Not Be Mainstream Without HUD and Killer Apps

And here is some brief discussion on each:

1. 2nd Generation “Smart TVs” Will Establish a New Platform Paradigm

smarttvThe first generation of Smart TVs got a bad rap for either being difficult/nonintuitive to use or providing a lot of un-TV-like functionality that consumers didn’t care for. They also suffered from performance/processing issues related to hardware & bandwidth limitations. Some critics went so far as to write off the Smart TV as a failure.

The next generation of Smart TVs should be more intuitive to use and will be designed to integrate with and complement tablets as a second screen. Generally speaking, a tighter integration between TV, internet, and peripheral devices will be a catalyst for derivative trends in content creation & consumption.

Despite modest market forecasts, some retailers suggest that they will only be selling Smart TVs.

An entry by Apple into this space may have a “Rising Tide Lifts All Ships” effect. See Trend 2 for further thoughts on this.

References

http://www.trustedreviews.com/news/john-lewis-we-will-only-sell-smart-tvs-in-2014

http://www.theguardian.com/media-network/media-network-blog/2013/mar/11/beyond-apps-future-smart-tv

2. Cross-Platform OS Convergence on Non-PC Devices

While all eyes will likely be on Apple’s anticipated 2014 release of an actual Smart TV (not to be confused with their “Apple TV” set-top box), there are other players to track. Samsung currently enjoys a dominant position in the mobile market with their Android devices. However, they have announced plans to incorporate Tizen, an alternate open-source operating system, into phones and eventually TVs. On a smaller scale, Amazon expects to launch a set-top box (not an actual TV) to compete with the Apple TV set-top box.

For Apple and Samsung, these moves suggest that customers will have an optimized experience when they consolidate their media platforms in a single operating environment. For instance: iPhone + iPad + Apple Smart TV (iOS) or Samsung phone/tablet + Samsung Smart TV (Tizen), etc. Amazon’s popular Kindle Fire is built on the Android platform, which shows that rather than reinventing the wheel from a hardware standpoint, they will probably continue to deploy solutions on popular open architectures, and focus on strategies and relationships that leverage content.

What this means for entertainment is uncertain, but this type of convergence will give these companies extraordinary market leverage with respect to both direct competitors in the hardware space as well as content providers.

References

http://appleinsider.com/articles/13/10/29/amazons-anticipated-apple-tv-competitor-not-expected-to-launch-until-2014

http://www.bloomberg.com/news/2013-10-22/apple-preparing-65-inch-tv-for-release-next-year-analyst-says.html

http://www.businesskorea.co.kr/article/2143/tizen-phone-tv-samsung-reveal-smartphone-smart-tv-based-tizen-os-2014

3. Entertainment Platform Convergence

6a00d8341c858253ef00e55215cad98834-640wiSony’s PlayStation 4 and Microsoft’s Xbox One will both contain features that point to a convergence of different types of home entertainment on a single platform, including gaming, social media, TV, internet, etc. The lines between what are traditionally seen as separate, non-integrated forms of entertainment/media will start to become blurred, and this will pave the way for the development of “hybrid” experiences.

References

http://www.forbes.com/sites/jaysondemers/2013/10/31/the-top-7-technology-trends-that-will-dominate-2014/2/

4. Biometric Gaming Leads Innovation of Quantified Self

indexThe Quantified Self movement is primarily associated with self-help and health-oriented applications. However, as the underlying technology develops, it can be integrated as a unique component of entertainment, media, and marketing applications. Biometric sensors will be integrated into next-generation gaming platforms in an effort to provide a richer experience. The gaming industry, with its dynamic, competitive environment, combined with its commitment to large-scale R&D, is well-positioned to lead this development. Precedent strongly suggests that other industries can expect to benefit from a trickle-down effect of these innovations.

“Kinect 2.0 goes a step further by being able to determine where in the room a player is looking, what their facial expressions are to determine their mood, and even pick up the player’s heart rate based on tiny fluctuations in their skin color. All this information can be used by game developers to deliver tailored experiences that create more immersive and engaging gameplay.”

References

http://www.biometricupdate.com/201310/obje-acquires-novalon-games-in-push-for-biometric-gaming-revolution

http://www.psfk.com/2013/09/entertainment-trends-nbc-curve-report.html

5. Internet of Things Creates New Opportunities for Location Based Entertainment

worlds-first-flying-car-07With many trends focused on enhancing the home/mobile experience, LBE can start to feel dated. However, the paradigm of the “Internet of Things” provides a number of opportunities to enhance & streamline these types of experiences. Walt Disney World’s MagicBands is already old news, and despite its massive development and deployment cost, it really only scratches the surface of what’s possible. In the referenced video, Intel even uses a theme park as a complex model to show what IoT can enable. The focus continues to be on operational aspects, but extending these techniques and infrastructure to more entertainment-oriented applications is a natural next step.

References

http://www.youtube.com/watch?v=8pkhL432GNg

6. Google Glass Will NOT Take Off in 2014

3dglassesDespite all the hype, Google Glass will be more of a long-term play. Glass will continue to play a central role in the overall future of wearable technology, but it may not be the first device that people gravitate toward in this sector.

I think that Google overplayed its hand by introducing a photo/video capture capability in its pre-release product. This functionality has been a lightning rod for privacy concerns, and has possibly generated more negative controversy than positive anticipation for the product as a whole. Moreover, although they couldn’t have anticipated it, their timing couldn’t have been worse with the general climate of privacy-related issues in the news on a daily basis. The Google Glass mantra is to provide users with technology that is “there when you want it, and out of the way when you don’t.” This is achieved through a rudimentary yet effective integration with their “Google Now” service, and this in itself has the potential to radically shift the way people consume information on the move. That said, I appreciate their “go big or go home” approach to rolling out this product, and they clearly have the resources to back the long play.

References

http://marketingland.com/180-days-with-google-glass-hits-misses-what-marketers-need-to-know-64289

7. Augmented Reality Will Not Be Mainstream Without HUD and Killer Apps

grandma-ocularGoogle Glass is not AR. I’m 100% sure Google would agree with me on this. The AR landscape still lacks an effective, consumer-grade Heads-Up Display (HUD) and applications that really matter. I do not dispute that anyone who experiences AR for the first time, even in its current, clunky handheld-device modality, will feel awe, delight, and even a sense of magic. However, until it can be more seamlessly integrated into everyday life on both a hardware and software level, it will will continue to have limited penetration as a novelty or will be most effective in specialized, controlled use-cases.

References

http://www.thorntech.com/2013/04/augmented-reality-will-wearable-hardware-bring-ar-to-the-mainstream/

Coffee, Beer, Zombies, and the Future of Immersive Entertainment

30 Oct

Unfortunately, it couldn’t be all play, all the time in Atlanta. We had a full day of work ahead of us to set the record straight on a few items.

Does Octane deserve to be considered one of the nation’s best coffee establishments? Yes. Especially when paired with a popover from the adjoining Little Tart Bakeshop in the Grant Park location.

Will Rosebud provide a superlative experience for the discerning brunch enthusiast? We started with fried cheese grits with smoked cheddar and pepper jelly. Need I go on?

Can Porter Beer Bar live up to its reputation as one of Atlanta’s best destinations for enjoying a wide variety of amazing beer? Early results are extremely promising, but from a due diligence standpoint, we feel that it would be imprudent to end our investigation prematurely.

Atlanta is a flexible concept

“Atlanta” is a flexible concept

Refreshed and recharged, and in gratitude to Atlanta’s gracious hospitality, we thought it was only right that we do our part to assist them with their zombie problem. Atlanta Zombie Apocalypse promises an interesting variant on the standard haunted attraction. Unlike NetherWorld, which was incongruously located a few blocks from suburban strip malls, AZA is convincingly nestled in the woods off a deserted stretch of highway. The attraction is built on and around the grounds of a deserted motel and it didn’t take much effort to believe that zombies might be nearby.

For those who don’t frequent this type of entertainment, the haunted attraction industry is all grown up and embraces more than what we endearingly referred to as a “haunted house” back in the day. There are a growing number of variants on the theme of scaring people for fun and profit, including haunted mazes, haunted trails, scare zones, haunted hayrides, etc. Many attractions aggregate and even mashup several of these styles to make sure that you don’t get too comfortable. AZA provides three attractions which can be generally described as 1) haunted house/haunted trail mashup, 2) haunted maze + paintball and 3) haunted house/immersive theatre.

cdc-zombie-posterOut of respect for the organizers, I won’t give up any spoilers, but I will share some general impressions. Admission can be purchased on a per attraction basis. I was here from California and was in the middle of nowhere, GA. I was going to all three.

Although the attractions can be visited on a standalone basis, some of the employees had specific ideas on the order in which to experience them if you were seeing more than one. They were essentially imposing a meta-level of story and emotional engagement that hadn’t actually been accounted for in the overall design of the attraction. The best part is: different people had different opinions on the ideal order, and they were pretty passionate about their individual assessments. It was like having a personal team of horror sommeliers. More importantly, it was unscripted proof that the tendency to find story in our lives is central to the human experience. It also echoed my thoughts on the very personalized reactions that one can expect from immersive experiences.

Unlike many haunted attractions, which are essentially self-directed (i.e., here’s the entrance, proceed to the exit,) each of these attractions had a guided component. While the most memorable features of many attractions are the live performers, the level of interactivity is typically low and unidirectional: they scare, you scream, keep moving. The guides and performers at AZA added a sense of theatricality and interactivity which enhanced the the overall immersive quality of the experience. The first attraction we chose was called “The Curse,” and it took advantage of the ample, wild surroundings by using parts of the neighboring forest as the set. There was a backstory and a mystery to the attraction, and we were recruited as investigators and addressed directly by the guides and performers. Although I saw room for improvement, I was certainly entertained.

All in a day's work

The blogger in the line of duty

For me, the main attraction, and the reason I found myself in Conley, GA deferring my important research of Atlanta’s brewpubs, was the “Zombie Shoot.” We were provided with a semi-automatic AirSoft gun (essentially paintball, but using specialized BBs instead of paintballs) and protective headgear, and were told, “Aim for the head or the chest. Keep firing until they go down.” Pretty much the exact opposite of your standard preshow advisement, “Keep your arms and legs inside the vehicle at all times.” Also, in case it’s not totally clear, you’re firing at real people, I mean zombies, not cardboard cut-outs or animatronics. Did it work? Mostly. Was it fun? Definitely.

Maybe King Lear actually IS the future of haunted attractions

Maybe Shakespeare actually IS the future of haunted attractions

The final attraction was called “?” and I think it’s telling that it might have been my favorite even though on its surface, it seemed to be a standard haunted house. Why? Story. It was the most theatrical of the three and featured the most interesting and inventive story, while still remaining focused on trying to scare you out of the building. Granted, we’re not talking Shakespeare, but it was a refreshing and successful variant on what could have been an uninspired production.

The production value in these attractions was not award-winning, and the storytelling was not expert, but I’m still glad I checked them out. It was all produced with a lot of heart, and everybody from the people selling the tickets to the performers and guides and zombies seemed to be having a good time and wanted to make sure that we were too. By providing three very different, hybridized attractions, it was clear that the creators were willing to take some risks and try out some new ideas, which will always impress me more than playing it safe.

Crossing the line from a one-way performance to an interactive experience can immediately raise many expectations that are potentially difficult to meet. The biggest challenge becomes the reconciliation between the audience’s sense of agency and the practical and aesthetic parameters of structured entertainment. It’s a tricky balancing act between living an experience and playing along. To choose an extreme yet practical example from this show, being provided with a gun and an opportunity to fend off assailants is real and pulls you right into the experience. However, remaining cognizant of where you can and can’t shoot can temporarily pull you out. It trades the steady equilibrium of an evenly immersive but passive experience for one which attempts to balance instances of deep immersion within a more structured rules-based framework. The resulting dynamics suggest a hybrid with gaming, sports, and other experiences that would otherwise seem at odds with what (in this case) is essentially theater. Personally, I think that finding a satisfying balance in these intersecting sensibilities is a challenge worth taking on, and that we’re hopefully just scratching the surface of the types of experiences we can expect to enjoy in the future.

Breaking Bad: Need a Third Eye For My Second Screen

20 Sep
I Heart Breaking Bad

I Heart Breaking Bad

I finally got a chance to experience Story Sync for AMC’s Breaking Bad.

DISCLAIMER: I’m a big fan of Breaking Bad; it’s one of the few shows I can remember watching from inception instead of telebinging my way into a state of catatonia.

RANT ALERT: I have almost nothing good to say about this, so if you’re generally having a nice day, and would rather not read about someone complaining about television, then feel free to check in later when I’m in a better mood.

Quick! Breaking Bad is about to start!

Quick! Breaking Bad is about to start!

Story Sync is a “second screen” experience, which refers to the use of an additional device (e.g., smartphone, tablet) while consuming a primary source of content (e.g., TV). Despite the high-tech sound of it, Story Sync actually requires you to recalibrate your habits by a few decades. That’s right, prepare to depart the world of DVR, Hulu, VOD, Netflix, iTunes, and any other means that you regularly use to consume content at pretty much ANY OTHER TIME than the exact moment in time when it originally aired, because Story Sync only really works as designed during the show’s original broadcast. Aside from the Super Bowl, I can’t remember the last time I watched a TV show during its scheduled airtime. At least the Super Bowl is a live event, broadcast once annually, with an unknowable outcome that will be spoiled within about five minutes of your engaging the outside world. On the other hand, it makes no sense for me to drop everything I’m doing in my life to watch a pre-recorded, serialized, dramatic show, but that is what AMC requires if you want to experience Story Sync.

AMC Executive Headquarters

AMC HQ

To get started, you must download the AMC Mobile app to your supported device of choice and keep it handy as you enjoy one of the best shows on television. As the show progresses, your device will play an alert sound whenever it has new synchronized content for you to consume, and the interface also provides a running countdown to when the next tantalizing update will be served. The second screen content is in the form of “cards” which contain a unit of content which is typically one of the following:

– relevant still photo from the current or a past episode

– relevant video clip from a past episode

– interactive multiple choice trivia question

– interactive poll

– advertisement; yes, advertisements.

imagesLet me just cut to the chase. Breaking Bad is an intense show. It has successfully navigated the murky waters of long-format television to create an epic story that has momentum and feels real without getting mired in its own intricacies. On the whole, the aggregate experience can be as visceral and emotionally satisfying as watching a great movie. The level of technical artistry from the writing to the cinematography to the editing, sound, music, etc. is exemplary and all of it is brought home by incredible performances by a very talented and well-cast collection of performers. [PING] Hold on, it’s Story Sync, be right back, I need to answer a trivia question. Dammit! Oh, that’s the right answer? Right, now I remember. OK, where was I? Without giving anything away, characters do unexpected things and the plot never fails to enthrall over literally dozens of hours. Particularly in these final episodes, there are moments of great pathos and emotional catharsis as [PING] Let’s see here, a poll: how would I rate Walt’s ethical decision here on a scale of 1-5. Hmmm. Well, could be a three, but I’m leaning toward two. Two? Three? Umm. OK, two. Cool, 31% of Story Sync’d America agrees with me; I feel so . . . connected.  Even the arguably relevant content which is designed to refresh your memory about a pertinent detail from a past episode is usually an inappropriately timed distraction.

You’re probably wondering why I don’t just pause the show so I can focus my attention on the poll and not miss what’s happening on the “first screen.” Well, this is a “live TV” experience, and pausing would cause me to lose my sync, not to mention run the risk that my second screen updates might actually end up SPOILING something I haven’t seen on the first screen yet. Also, while you can technically experience Story Sync on a non-live version of the show, it is provided as an unsynchronized “archive” version of all of the second screen content. This means that spoiler information for the entire show that you are about to watch is just a swipe away.

ya-voteBefore you write me off as a Luddite, (actually a pretty audacious proposition if you’ve read any of my other posts) I admit that the concept of second screen might have its place. I’m an unabashedly huge fan of Project Runway, which has a second screen feature that allows you to vote about whether you agree with the judges or whether you think a particular contestant is headed for disaster. Chances are, you’re probably already thinking or talking about the exact question and it takes about a second to form a quick opinion and cast a vote. The results are shown on the air in a corner of the screen in real-time and then they disappear. This works because Project Runway is a reality show and a competition. There is no disbelief to suspend and the second screen experience provides a brief, intermittent outlet to your inner judge with instant gratification. This type of entertainment has a familiar and expected structure (setup, competition, drama, judging, victory/loss, reflection) that is simply more compatible with a second screen experience that can enhance those inherent emotional beats.

The opposite is true with a dramatic show. The level of impact of your emotional experience with the created world of the show is directly correlated to your engagement and attention. Every detail that is seen and heard on-screen is deliberately designed to deepen and retain your immersion in the story as it unfolds. But it’s a two-way street. While I can see that the idea behind Story Sync was to encourage a deeper level of engagement, unfortunately, it seems to have the opposite effect. Moreover, even the smallest amount of analysis or audience-testing would have yielded the same conclusion before it saw the light of day, or at least the dim lighting of your living room.

Walter White disapproves of Story Sync

Heisenberg votes “No” on Story Sync

Plain and simple, Story Sync represents an extraordinary lapse of judgment on the part of the decision-makers who thought this would be a good idea for Breaking Bad. When this show premiered more than five years ago, I’m pretty sure that the creators did not intend for the attention of their audience to be dissipated into a Chinese water torture of inconsequential irritations and distractions. I’m sure that there must be a creative and compelling way to keep fans immersed and engaged with this world, but I’m also sure that this isn’t it.

Choose Your Own Adventure: 21st Century Edition

13 Sep
Where's the Spoiler Alert?

Spoiler Alert!

I have many fond memories of the Choose Your Own Adventure series of books. I whittled away many hours of post-recess silent reading wandering through myriad outcomes in my imagination. I always wondered why that experience had to be limited to the world of books, and I thought it would be so cool if that sensibility could be woven into movies. Well, as it turns out it’s happening . . . . sort of.

When I was looking for apps to purchase for my NeuroSky MindWave EEG headset, I couldn’t resist Paranormal Mynd: Exorcism from MyndPlay. Before I go any further, if you aren’t familiar with what’s currently happening with consumer-grade EEG technology, a brief background will be helpful. Electroencephalography (EEG) is the detection and recording of the electrical activity generated by the brain. For this information to be useful in a medical context, it is not uncommon for a recording device, typically worn as some form of headgear, to have anywhere from 16 to 256 electrodes. This grade of equipment can be expensive as well as generally impractical for casual/recreational use.

Medical grade data and great for metal night at the karaoke bar

Medical grade data and great for metal night at the karaoke bar

In the past few years, NeuroSky and other companies, including Emotiv and InteraXon, have developed consumer-grade devices that have 1-4 electrodes and range in price from $80 – $300. The tradeoff pertains to the quantity and quality of the data collected. NeuroSky’s MindWave headset uses a single dry electrode, but can be worn and activated in less than one minute. Their proprietary algorithms currently interpret one’s brain activity as variable levels of FOCUS and RELAXATION. From an application development standpoint, this allows for the conditional execution of routines based on either or both of those levels as needed.

Possession or just caffeine withdrawal?

Possession or just caffeine withdrawal?

Paranormal Mynd: Exorcism is essentially a movie that contains “decision points” which launch different scenes based on the outcome of some period of EEG analysis. The story is told primarily from a first person point of view. As the viewer, you are an exorcist, and the decision points require you to maintain a threshold level of focus for a period of time in order to successfully exercise your powers. In this title, there are only two decision points, and the entire experience takes less than ten minutes.

On one hand, It’s easy to dismiss this as something that I paid a buck for that entertained me for fifteen minutes. However, I found it unique enough to also see it as a proof-of-concept that raises some interesting questions about the potential for this style of storytelling.

The plot goes something like this . . .

The plot goes something like this . . .

Even more than the old Choose Your Own Adventure books, there is a game aspect to it that makes it more than just a story. While some of those books had multiple satisfying outcomes, others seemed designed to encourage you to exercise your judgment to find a single outcome that felt like “winning.” The MyndPlay title adds a new dimension by requiring some level of “skill” to trigger a desired outcome. In other words, your decision to access a certain outcome is separate and different from your ability to access it, which certainly amplifies the game quality of the experience. While a good game often has a great story component to it, I still make a distinction between when I’m in the mood to “play a game” vs. “enjoy a story.” This experience essentially combines the two, and it’s still an open question as to whether it works for me.

Other questions that I had:

Do I want to see all the outcomes? Even the ones where I don’t win? Do I want to try harder to see a different outcome? How does this meta-level of engagement change my emotional relationship to the story? Does it matter?

This title employed classic horror tropes with very little subtlety. Girl is possessed. Bad things happen if she stays that way. You need to save the day. Could this format be effective in the telling of more sophisticated stories?

Co-author of this post

Co-author of this post

Do the nuances of the decision mechanism enhance or detract from the experience of the story? For instance, I actually found it very easy to achieve threshold levels of focus, because my attention was riveted on the possessed character on the screen. In fact, I found I had to try harder to trigger a threshold of non-focus just to explore the alternate outcome. What if the threshold tests were structured in counterpoint to the content, e.g., what if I had to trigger a threshold of relaxation while watching the same intense scene? Would that make it too much of a game?

Basically, if this is an intermediate step in the exploration of a new style of storytelling, where do we go from here?

Neurogaming Conference 2013: Expo Highlights Part 2

16 Aug
'Nuff Said

‘Nuff Said

In my previous post, I talked about some of the highlights from the 2013 Neurogaming Conference and Expo. Did I mention that the Expo was in a nightclub? So basically on a pleasant weekday in early May, while you were probably engaged in performing or avoiding something at work, I was hanging out in a nightclub making drones fly around with my brain. Man, I’ve always wanted to say that!

Anyway, where was I? Ah yes, the bar had just opened. My go-to industry event cocktail tends to be a Gin & Tonic. It tastes great in a plastic cup and it’s almost impossible to screw up. So drink in hand, I continued my explorations.

Tactical Haptics: These guys are developing a technology called Reactive Grip which is partially based on technologies developed out of the University of Utah. I’m not a gamer, so please feel free to correct me if I get some of this wrong. In a sense, they are extending the general paradigm of a motion-based game controller such as the Razer Hydra by Sixense by adding haptic feedback. Sounds straightforward enough, but it’s not. It was one of my favorite demos at the Expo, and I’ve been trying to figure out why. Basically, my limited experience with haptic feedback is that it’s like the 3D of the gaming world. With rare exception, studios are churning out 3D movies because they feel like they’re supposed to, and the end result is an obligatory, thoughtlessly-executed assault that is ultimately distracting. Haptic feedback can be the same way: I’m firing a big gun or driving a big car, my hands are buzzing, OK, I get it, tell me something I don’t know. I don’t feel that much more involved in the experience just because I’m feeling the exact same buzzing sensation too many times at totally predictable points.

Michael Buffer doesn't need Tactical Haptics

Michael Buffer doesn’t need Tactical Haptics

Tactical Haptics takes it much further. First of all, the haptic feedback in Reactive Grip is palm-facing so it’s accessing what is probably a more vulnerable and sensitive part of your hand. As a result, it lends itself to enhancing the types of actions that you would perform while gripping something with your palm and fingers. I have found a disconnect between the way you hold a standard haptic game controller and the object it intends to emulate, e.g., steering wheel, gun, etc. However, the form factor of Reactive Grip maps pretty closely to how you would hold a sword, for instance. In fact, one of their demos featured a virtual on-screen mannequin that you could hack with a sword. The continuity of what you see on the screen with what you feel in your hand is unlike anything I’d ever experienced. Even the level of feedback was regulated based on whether you were “touching” the mannequin or attempting to cut through it. The effect is uncanny, which is usually a good indication that you’re on to something big. Rather than providing a token enhancement, this technology really pulls you into the experience by engaging a critical sense in a well-considered way. These guys are looking to launch a kickstarter and they’re hustling on the road, making appearances at Meetups in the Bay Area. Worth checking out!

NextGen Interactions: I had been wanting to check this out for a while, because it allowed hands-on (or would that be heads-on) experience with the Oculus Rift virtual reality headset. One of the guys in line mentioned that there had been “a line to get into the line” to try Oculus Rift at the recent Game Developers Conference, so waiting for a few minutes seemed like a good deal. Jason, the founder of NextGen Interactions, was allowing visitors to experience a prototype of a game that he was developing that featured the headset. At this point, it’s hard to say anything about experiencing Oculus Rift for the first time that hasn’t been said before, but the general consensus, with which I agree, can be accurately summed up as HOLY $#!+!!!

Better give the Oculus Rift a break

Better give the Oculus Rift a break

It’s real easy. You sit down in the chair, put on the goggles, and the world that you know disappears. Let me say that again. The world disappears. The first thing I did when I put on the goggles was look up and then look behind me. Yup, looking up at a tall building or back at an empty landscape. Total 360 degree immersion. Jason incorporated Razer Hydra as the controller which makes for a very intuitive way to move around, and once you start moving, it is impossible to continue believing that your chair is not actually moving. Actually, you basically forget all about the chair. Jason patiently walked me through the level he designed, where he had embedded some features that I appreciated including puzzles that required you to interact directly with your environment like picking up objects, stacking them, etc., which you literally do with your hands, thanks to the Hydra. As a testament to the level of immersion though, I found it very difficult to focus on what he was saying, because the whole concept of his voice coming in from some nightclub in San Francisco was totally alien to what was “real” for me, which was a post-apocalyptic landscape that I was intent to explore. After about ten minutes of this, it was time to move on, and I actually found it somewhat disappointing to return to the real world. Jason was meticulous about collecting feedback, and apparently my enjoyment of the Gin & Tonic made me a subject of interest with regard to the potential for motion sickness. I admit to a slight feeling of queasiness which I think I would have felt even without the drink. Hopefully, he had a control group. Oculus Rift did not invent virtual reality, but they seem to have figured out a way to engineer it in a way that will be consumer-friendly (i.e., it won’t cost thousands of dollars.) With developers like NextGen Interactions building content for the platform, it shouldn’t take long to catch on.

Coming Soon: Isaac Asimov posthumously sighs at what I’m getting out of the Foundation Trilogy.

Haunting

A Community for Immersive Theater & Horror

This Week in Laundry

Immersive Storytelling Technology

Theme Park University

Stories on Themed Entertainment

Theme Park Insider

Immersive Storytelling Technology

InPark Magazine

Serving the themed entertainment community

imho

Sharing What I've Learned...of Creating Experience with Deep, Emotional Connection