Control Issues (or Mimesis vs. Semiosis)

I was talking to my class about game controllers a week or so ago, largely in the context of the “physical” movement we’ve seen with the Wii, Move, and now the release of the Kinect. All of these introduce the idea of playing games by waving your arms (and legs) around, generally speaking creating interaction through mimesis, with the avatar on screen doing as you do. Good stuff.

In the class, I contrasted this kind of mimetic, physical interaction with the kinds of controls that most gamers are very familiar with: buttons, joysticks, and mice. We explored various important differences, and one that sticks with me is that, while there are some low level spatial (and other) metaphors that make sense of traditional controllers, there’s generally a pretty arbitrary relationship. As such, with our XBox controller in hand, say, we’re largely operating at a semiotic, rather than mimetic, level – we hit abstract symbols to make things happen on screen. Press Y to steel a car, pull RT to accelerate away. It’s this semi-arbitrary relationship that scares the crap out of beginners.

Because of the topic of the class, I’d had a temptation to wax lyrical about how great these new mimetic, physical interfaces are. You swing your arm – your avatar swings a racket! You do a starjump and the tiger copies you! It’s so natural! But this all made me think more about distinctions between the two forms some more, particularly in terms of what we might call “player space” and “game space” – player space being the physical reality we’re in as we play, and game space being the virtual one.

Thus, traditional games essential drag play into the game space. We sit on the couch, barely moving but for our fingers pressing coloured buttons, watching a virtual environment on the screen where the game is actually being played. What we do in reality is both essential (if we don’t move our fingers, nothing happens), and trivial (we’re not doing much). What these newer mimetic games have done is drag play back out toward the player space, the physical reality we exist in as we interact with the game. When we swing our arm to swing the virtual tennis racket, we’re performing the actual action which is translated into the virtual world. The fun of the game and the performance of play takes place just as much, if not more, in our present reality as in the representations. When we play in social settings, the performative aspects of this can begin to completely overwhelm the virtual setting in favour of the physical.

All of which is a positive outcome, it’s fairly clear – people obviously love it, and moving play and playfulness more toward the real world is a nice thing. Systems like the Wii and Kinect, perhaps more than anything else, give people permission to move in playful and often absurd ways, to perform. When we were kids, perhaps, we’d move as we wished merely at the whim of imagination, but these days, as adults, we tend to need contextualisation for fear of looking like an ass. And now we have it.

On the other hand, it’s also clear that a major part of the beauty of “classical” or “semiotic” control is that it does draw the play into the virtual world, critically that it takes us with it. Where the mimetic-kinetic actions in a Wii game make us more away of ourselves and our environment (both social and physical), the sheer abstractness and minimalism of an XBox controller or a keyboard and mouse invites us to forget about reality – it’s simply not relevant to our play. Unless the controller runs out of batteries or the mouse gets unplugged, we’ve little or no need to think about our physical existence as we play, and can thus focus almost entirely on our virtual existence. Without wanting to enter into debate on the term, this kind of semiotic control is excellent for what tends to get called “immersion”.

So, whether you’re failing around madly and to the delight of your peers as you try to punch their lights out in Wii Boxing, or dead-faced and barely moving while involved in saving humanity in Mass Effect 2, I say: enjoy it. Both extremes, and the inevitable continuum between them, are a hell of a lot of fun and facilitate different and highly valuable experiences.

6 November 2010
← next words previous words →