2018 Research Talk Abstracts

Research Presentations Program Chair: Melanie Dickinson | @meldckn

Session 1: Game Studies

Session Chair: Noah Wardrip-Fruin

Grimes' Fairy Tales: A 1960s Story Generator

James Ryan | PhD, Computational Media | jamesryan.world | @xfoml

I provide the first extensive account of an unknown computer story generator that was developed by linguist Joseph E. Grimes in the early 1960s. Story generation is the application area of artificial intelligence in which computers are made to tell stories. A pioneering system, Grimes's was the first to take a grammar-based approach and the first to operationalize Propp’s famous model. This is the initial report of a larger project that aims to reformulate the prevailing history of story generation in light of new findings I have made pertaining to several forgotten early projects. My study here has been made possible by personal communication with the system’s creator, Grimes, and excavation of three obscure contemporaneous sources. While the accepted knowledge in our field is that the earliest story generator was Sheldon Klein’s automatic novel writer, first reported in 1971, I show that Grimes’s system and two others preceded it. In doing this, I reveal a new earliest known system. With this paper, and follow-ups to it that are in progress, I aim to provide a new account of the area of story generation that will lend the community insight as to where it came from and where it should go next. I hope others will join me in this mission. Read the full paper here. 

The Hidden History of Bandai Board Games, 1980–90

Nathan Altice | Professor, Computational Media | metopal.com | @circuitlions

If you played NES in the 80s, watched Power Rangers in the 90s, or dipped into the world of Japanese toys in the past five decades, you’ve likely heard of Bandai. Established in 1950, the toymaker managed to insinuate their name into the American consciousness like many other Japanese companies of the time—via toys, television, and videogames. But despite Bandai’s decades of international success, a major part of their history is almost completely unknown outside Japan.

Between 1980 and 1990, Bandai released nearly 300 board games under the Joy Family, Party Joy, and affiliated spin-off series. And like their toys and models, these games would feature an impressive breadth of original and licensed characters, from horror stalwarts like Dracula to an up-and-coming pair of action-plumbing brothers named Mario and Luigi. In the U.S., however, Bandai’s board game oeuvre is known only by a few collectors and a handful of entries on Board Game Geek. These omissions overlook not only a landmark in Japan’s board game history, but an important bellwether for changing tastes in popular culture and media during the 1980s.

Based on extensive archival research and translation, this talk introduces a vital moment in Japanese game history via Bandai’s extensive board game output and connects that moment to centuries-old cultures of folklore, art, and play in Japan.

Why Critical Game Studies Matters

Soraya Murray | Professor, Film & Digital Media | sorayamurray.com | @sorayamurray

As quintessential visual culture forms in the twenty-first century, mainstream games both mirror and constitute larger societal fears, hopes and dreams, and even address complex struggles for recognition. Professor Soraya Murray will discuss her new book, On Video Games: The Visual Politics of Race, Gender and Space (IB Tauris, 2018), which critically explores blockbusters like The Last of Us, Metal Gear Solid, Spec Ops: The Line, Tomb Raider and Assassin’s Creed to show how they are deeply entangled with American ideological positions and contemporary political, cultural and economic conflicts. Murray provides novel theoretical tools for discussing games as culture. In this talk, she’ll discuss why these approaches matter, and what is at stake in critical game studies for players and designers alike. Demonstrating that playable media are at the frontline of power relations, she reimagines how we see them – and more importantly how we understand them. 

Don’t Judge a Book by its Cover: Content-based Retrieval for Videogames

Adam Smith | Professor, Computational Media 

This talk describes an effort at UC Santa Cruz to build a new kind of search engine: one that understands the moment-by-moment play inside of a game rather than trusting superficial metadata (title, category, or tag) for the game as whole. We start with data from expert speedruns and student volunteer players then explore more moments with newly developed automatic exploration algorithms. Using deep learning, we map all of these moments into an embedding space where similar moments are usefully organized. Given a screenshot or other form of query, we can retrieve and rank the most related moments we’ve seen so far. We’ll describe how our system can recognize a specific boss battle from a single blurry screenshot. So far, we have focused our attention on a collection of 750 Super Nintendo Entertainment System games (already much more complex than the 8-bit Atari games used in contemporary deep reinforcement learning research), but we have our sights set on tackling recent Android games next.

Session 2: Game Design

Session Chair: Susana Ruiz

The Poetics of Games or: Another lens for thinking about the expressive potential of digital/computational/interactive media

Jordan Magnuson | MFA, Digital Arts & New Media

The history of game studies has been and continues to be defined in many ways by the debate between the ludologists and the narratologists. Of course, in the last few years it has been shown time and again that this so-called debate was built mostly on straw, and that videogames can both be games and tell stories. But this state of affairs is precisely the problem, has always been the problem: game studies is stuck on a terribly limiting spectrum with “narrative” on one side, and “gameplay” on the other. It is not enough to smile knowingly and say that games can be “both and”: the problem is the limiting spectrum itself. What we need for games now are other points of reference entirely, outside of narratology and ludology, to help break down this limiting dichotomy. In this talk I will present my research on one such point of reference—lyric poetry—and hint at how it might be used as a helpful starting point for expanding our possibility space when it comes to understanding and creating games.​

Reparative Game Design: Emotion, Queerness, Disability, and Environmentalism

Kara Stone | PhD, Film & Digital Media | karastonesite.com | @karaastone

This talk will discuss ways in which game design can be a form of research creation on the affinities of queerness, disability, and environmental justice. If we view games as emotionally provocative works of art, what role can they play in affective communication, reparation, and healing? My research augments queer theorist Eve Sedgewick’s notion of reparative reading to form a reparative art practice, one that is inclusive of the paranoia, critical, difficult, and bad feelings that are a part of queer and debilitated life. I create tentacles (à la Haraway) between queer time, crip time (the relationship between temporality and disability) and deep time (imagining geologic time as it stretches beyond the human). I will discuss my practice of reparative game design I have been experimenting with through a handful of games I’ve designed, including Medication Meditation, Ritual of the Moon, the earth is a better person than me, and Humaning.   

An Exploration of Interactive Paradox

Cyril Focht | MS, Computational Media

Paradox can occur in a variety of forms from logical and literary to visual and auditory, but what does paradox look like in an interactive form? Paradoxical approaches to puzzle design are growing in popularity, and as such are opening up the space of paradox which is interactive in nature. In this presentation I explore the idea of interactive paradox by discussing examples of games where paradox is found. This discussion will include similarities and distinctions between these forms of paradox, which could be used to form a set of classifications, such as those based on spatial interpretation, forms of figure-ground ambiguity, and limitations of a player's mental model of the game system. Following from these examples I propose distinctions between interactive paradox and paradoxes which are not interactive in nature, as well as between various forms of interactive paradox. 

Talin - A Better Way to Make Tutorials

Batu Aytemiz | PhD, Computational Media

Most tutorials in video games do not factor the skill level of the player when deciding what information to show. This makes many tutorials either boring for very experienced players or not specific enough for player who are new to the genre. Furthermore, due to the tutorials mostly being frontloaded in one section, it makes it difficult to for players returning to the game after a hiatus to remember what to do. With the Talin framework we make it possible to create a mastery model of an individual player's skill levels by operationalizing Dan Cook’s skill atom theory. We propose that using this mastery model opens up a new design space when it comes to designing tutorials. We show an example tutorial implementation using the Talin framework which result in the player receiving information only when they need it, whenever they need it.

Monster Carlo: Machine playtesting for Unity games

Oleksandra Keehl | PhD, Computational Media

Human play-testing of in-development games is important, but it's time consuming and slow. For a quick and cheap feedback along the way, machine testing can be used. In this talk I'll introduce Monster Carlo, a design tool for Unity programmers. I will explain how it uses Monte Carlo Tree Search (and what that is) in order to automatically play games and collect data to help the developer make design decisions about their game. For example, you may want to see how your game changes if you make your character stronger, but make the power-ups less frequent. Perhaps you want to try many different settings for each of those values. Testing each version with human testers would take a very long time. Instead, you can set up experiments with each variation of the values and have Monster Carlo play all versions of the game thousands of times. You can then compare average scores achieved and decide which results best fit your vision. Then you can save your human playtesters for the most promising variants of your game. Finally, I will demonstrate how I used Monster Carlo while developing my own game, "It's Alive!" and what I learned in the process. 

SpokeIt: A Mobile Speech Therapy Game

Jared Duval | PhD, Computational Media

SpokeIt is a serious game for health co-created with medical professionals, speech language pathologists, developmental psychologists, game developers, artists, and players. SpokeIt is the first mobile speech therapy game to employ an offline critical speech recognition system capable of distinguishing between correct and incorrect speech. SpokeIt is meant to augment at-home speech therapy for jaded individuals who typically only practice with a speech therapist. The iPad based game is capable of giving real time feedback to players about how well they are articulating, keep track of speech progress over time, record how often and long players practice, generate reports for speech language pathologists, and dynamically adjust curriculums based on speech goals set by medical professionals. SpokeIt not only listens to speech, but can demonstrate correct speech with our custom lip sync animations that synchronize with audio as in-game characters speak. Users with articulation disorders have participated in co-creation sessions and user studies, which have yielded an experience that is enjoyable and effective. We are gearing up for the beta release of SpokeIt and look forward to helping people achieve their speech goals. 

Session 3: Generative Methods

Session Chair: Jim Whitehead

Surprise! The Joy of Generative Art

Kate Compton | PhD, Computer Science

Generative algorithms can spin simple interactions and data into complex tapestries of color and animation. Kate Compton shows a variety of tools from her dissertation work which can be used to design and build such systems. Generominos is a Kickstarted deck of cards for inspiring and educating students about generative design. Tracery and Bottery are two bot-making tools for casual users, and Tracery currently runs at least 10,000 twitterbots.

Procedural Content Generation with WaveFunctionCollapse

Isaac Karth | PhD, Computational Media

In this talk, I'll introduce WaveFunctionCollapse, a procedural content generation algorithm invented in 2016 by Maxim Gumin and already in use for generating dungeon levels, looping animations, islands, and poetry. It's particularly strong at solving difficult layout problems that have potentially mutually exclusive elements, such as drawing complicated tangled knots or other constraints that make it hard to tile randomly but straightforward for WaveFunctionCollapse to solve. I'll explain how how it can generate a wide range of results from a single example image, getting great results without needing a lot of data. While inspired by earlier research into texture synthesis, it combines several strands into a more effective whole. I'll show you how it works, why it is a form of constraint solving, how the probabilistic constraint solving model leads to more expressive results, and the process we used to take advantage of its constraint solving properties and applied an answer set solver to add new constraints. 

Proceduralizing Character Development with Parametrized Storylets

Max Kreminski | PhD, Computational Media | mkremins.github.io | @maxkreminski

One recurring challenge in storytelling involves the difficulty of making the audience care about what happens to your characters. In narrative-focused games that make use of procedural content generation to generate characters, this challenge becomes even more acute. Traditional storytelling makes use of character development to generate ongoing audience interest in characters, but hand-authoring a unique character arc for each distinct character in a fully procedurally generated storyworld is often out of the question due to the sheer number of characters that might appear. The challenge, then, is to somehow generate compelling character arcs on the fly, while still limiting the degree of overlap between arcs enough to ensure that the audience will perceive each individual character’s arc as distinct.

In this talk, I’ll discuss my initial approach to solving this problem in Starfreighter, a work-in-progress experimental narrative game set in a procedurally generated universe. In particular, I’ll talk about how I attempt to proceduralize a variety of storytelling techniques borrowed from episodic or serialized fiction: character focus, relationship focus, hidden backstories, foreshadowing, and callbacks. On the implementation side, I’ll also discuss how proceduralizing these storytelling techniques is made easier by the decomposition of high-level character arcs into parametrized storylets: structured, reusable units of narrative content, each of which can potentially play a role in a wide variety of unique character arcs.

Generative Soundtracks for Oral Storytelling

Lucas N. Ferreira | PhD, Computer Science

The ability to understand and express emotions is fundamental for creative agents which generate art pieces that attempt to evoke specific emotions in humans. Modeling and manipulating a piece computationally in order to evoke specific emotions in humans is a difficult problem because emotions are subjective. Thus, there is no direct metric to evaluate how much a generated piece will affect the emotion of a person who is exposed to that piece. This project explores this problem in the context of soundtrack generation for oral storytelling. The goal is to augment oral storytelling experiences by generating orchestral soundtracks that match the emotion of the story in real-time. To accomplish this goal, we propose a creative agent called Bardo that analyzes the speech of story tellers in order to recognize the current emotion of the story. Bardo uses state space search algorithms to develop musical motifs to be combined into musical phrases that maximize emotion appropriateness to the current narrative context. We plan to evaluate Bardo with user studies in the domain of tabletop role-playing games. In this type of game, players typically augment their experiences with music by manually selecting songs from a playlist. This manual approach is a good baseline for analyzing the performance of Bardo. Preliminary results with background music selection showed that generative soundtracks have the potential to be better than manual song selections.