New Project: v r 3

Screen Shot 2017-01-30 at 13.29.50

Have started off on a new project post-SNAKISMS. I’m calling it v r 3 after a relatively agonising process of coming up with a name for it. The core idea is that it will be a kind of gallery/museum space exhibiting… virtual stuff. I started off calling it the “New Scene Gallery”, but never felt comfortable with that. After deciding to use the Marfa buildings from v r 2 as the gallery space, it started to seem sensible to just acknowledge that this is another step along in the v r series and name as as such.

This is a shame in the sense that it loses, I think, some of the sense of a “permanent” virtual space online for exhibitions, but it frees me up to do things differently for each “show” I suppose, from different landscapes to different buildings to… whatever. So v r 3 it is. The gallery/museum space itself will have no name I think.

The first exhibition (the one in v r 3 specifically) is going to be of water. I’ve been really interested in water technology in the Unity engine since a student waxed lyrical about how beautiful the “Unity Pro Water” is and demonstrated as much in a particular scene that was dark with neon lights reflecting in the water. It really is impressive water, and the fact that one can be impressed by water interests me. The fact it’s call “pro water” is also quite hilarious. There’s a lot going on, in short, around this idea of water as technology, water as something beautiful, perhaps water as something to be looked at (rather than incidental or scene-setting). So that’s what this exhibition is going to be about.

The current idea is to exhibit both Unity’s “professional water” and third-party waters from the Unity Asset Store, which is a whole other strange concept. Third-party water, water you might pay for, water with different aesthetic goals and implementations. It’s great.

So that’s what I’m up to for the moment. Hopefully it won’t take me too long to put together, but everything always takes too long, so this will too.

So that’s why curators have technicians

Screen Shot 2017-02-02 at 14.17.49

Work continues on v r 3, my exhibition of water. Yesterday I started actually messing around in Unity itself putting water all over the place and attempting to have it behave itself.

I’ve written about this sort of thing before, but I’m always really interested in the strange parallels that come up when you’re trying to do some sort of “job” in a game creation context. Whether it’s setting up the staging of a Eurovision show (Epic Sax Game) or doing the interior design for a non-existent institute (Digital Marina Abramovic Institute), there are always these situations where you’re trying to accomplish a task in digital space that is most often done in physical space. Setting up an exhibition of water is another example of this experience.

The most interesting thing here, of course, is the ways in which the digital impinges on the task, making some things much easier, some things harder, and some things just strange. So you can organise a gallery space by just cutting and pasting a file from a previous project, but in the same breath you might find out that light somehow shines through one of its walls which you need to fix. There have been various oddities like this that have cropped up so far in a single day of working on v r 3. Here are two.

You can’t lean over. In designing the plinth to hold the water I wanted it to be as similar as possible to the cubes in v r 2 as a kind of continuation and consistency. But in thinking about labelling and the ability to actually look at the water itself, it occurs to me that a really typical movement people perform in galleries is leaning over something that is lower down in space. The typical avatar always stands completely upright, just swivelling their neck around or, at best, doing a really weird crouch in which they lower themselves straight down in some kind of creepily smooth squat before scooting around the landscape like that. There’s no really naturalistic way to look at something from other postures. And this kind of rigid body (physics joke for you fans out there) has implications for the kind of display technology you then end up using. Maybe the plinth has to be higher so that the weirdo viewing the show is able to look clearly. Maybe the audience member has to be fitted with terrifying telescoping eyes so they can “zoom in” on things they can’t lean toward. Maybe the didact with the information about the particular piece they’re looking at has to be angled at 45 degrees to be legible, or has to be in extra large type. Etc. Consequences.

Water isn’t water. The water I’m displaying in v r 3 is obviously not “really” water, right? It’s not just a physical substance that behaves according to the laws of physics. Now, if I were having this show in reality, there would be all kinds of problems around displaying water, like the risk of it spilling or evaporating, of somebody trying to drink it or splash it, or of it leaking through the materials of the plinth, say. None of those issues are issues in a digital space, though – in those ways digital water is very neat and tidy and resistant to interaction. However, it doesn’t play nice in other ways. You can’t, for instance, take for granted that the world will be reflected correctly from the surface of the water (should you choose to enable reflections, which is a decision you can make). Instead, I fought for a couple of hours with the phenomenon of looking down into a pool of water only to see various bits of the reflective world pop in and out of existence. This does not happen, as far as I know, with real physical water. There’s lots of stuff like this. At one point I had multiple pools of water and realised they were all co-dependent – when I changed the colour of one (again, easy to do, no dye required) they all sympathetically changed colour. Not what I wanted. And on and on it goes. Currently I’m dealing with an issue where if you stand a really specific distance from the water and look into it, it refracts the world around it in a kind of scary infinite-looking regression of swirling shapes. A bit like a demon is about to swim up out of it and claim your soul. Not what I want.

So, once again I’m brushing up against all of these oddities of virtual spaces and virtual objects and virtual liquids. On the one hand they’re often very clean and simple to deal with or move around or remove or add, but on another hand they sometimes all too much like what they are, assemblages of code and assets that may or may not reflect reality.

Splish splash.

My digital water’s like a swamp, full of bugs

Screen Shot 2017-02-02 at 14.43.28

Oh hi, I’m still working away on v r 3 at the times when I can do that, and it’s been quite the ride of highs and lows. Sometimes I’m really, really pleased with the overall idea and how I think it will look and feel when it’s done (and yes, I know this feeling may be limited to me and like two other people), and other times I’m feeling really, really down.

Because it turns out to be inanely difficult to put water in a bunch of plinths in a virtual gallery. It sounds easy, but it ain’t. As I’ve said before, a lot of it is fairly easy in terms of the basic physics of putting objects in a virtual space etc., in ways that would be hard to do in regular old reality. But it’s the weird technicalities that are proving remarkably resistant to my “vision”.

As I work with Unity’s free “professional water” more and more I’m getting considerably deeper into thinking about what it is and how it works, because I just keep running into incredibly frustrating mistakes and, perhaps, bugs. It’s really interesting to me that there’s this process of needing to understand what’s going on at lower and lower levels of this piece of technology (and, of course, that the water is a technology in the first place). It’s not enough to just grab a “prefab” (a pre-build example of water that comes with the software) and drop it into the gallery, because I have a bunch of constraints. The chief of which is needing multiple instances of water in my environment, rather than just the usual single body of water (like a lake or whatever), and importantly that those multiple instances explicitly need to have different parameters set on them (reflective, refractive, or not, etc.).

Because of my requirements, which make total sense to me, I’m having a lot of difficulty. It turns out, at least to this point of my knowledge of Unity’s water, that the water doesn’t seem to “want” to exist in multiple configurations in the same space. So if I make some water that’s only reflective, and some other water that’s both reflective and refractive, they will enter into some sort of weird “possible universes” situation where, depending on how you look at one of them, it will flip between being reflective or refractive and back again. Not the ideal behaviour for water. I’ve now reached the point of going all the way down to reading the underlying code that defines the water itself to try to figure this out, without too much luck as yet.

A weird addition to all this is that of course I could at some point choose to exhibit these weird behaviours as part of this “show” of water, or even a separate show of things that don’t work the way they’re meant to? On and on it goes…

Water as Time Dilator

Screen Shot 2017-02-10 at 18.11.43

Just some brief words this evening since I’m kind of weary and forgot to write anything while I was more coherent earlier in the day. Still working on v r 3, still running into frustrating but simultaneously interesting technical issues.

So I have finally, finally managed to put different kinds of Unity professional water into all my plinths. This included editing the water script itself and doing a little big of shader debugging, neither of which I envisaged needing to touch when I started the project. However, it now turns out that if you have all (or many) of the plinths in your visual field while playing the game (at least on the MacBook Air that I do everything on), the framerate drops significantly and everything gets a bit dodging-a-bullet-in-the-Matrix.

Clearly the work of rendering all these different planes of water at the same time is kind of killing the performance, which I guess is fair enough? I wouldn’t know. I think there’s already something strange and intriguing there, right? That water, ostensibly this innocuous physical substance can slow time itself is pretty remarkable. Game engines just do the darndest things. So water turns out to be, in some sense, expensive for the graphics rendering of the game, especially when you have the game rendering 24 different planes of different water, animating them, presumably performing reflection and refraction calculations etc. etc. Perhaps that’s just more than one should expect. So the water does time dilation. Fine.

Some investigation showed that even water that isn’t actually visible to the camera is still being rendered, which is a pain. It would be nice if the computer didn’t have to grind away on all the computation required to display water I can’t see. This led me down the rabbit hole of looking at occlusion in Unity thanks to Ben Gattet, and we managed to get these cool visualisations of what the avatar (camera) can see and what’s being rendered and so on. Water is definitely being rendered even when it’s technically not visible. And this is apparently because the things hiding it (e.g. the sides of the plinths) aren’t big enough to be considered for “occlusion culling” (the process by which the game engine decides whether or not something is hidden from view and thus doesn’t bother rendering it at all).

I tried a few tricks in this world, notably trying to define a much smaller definition of what could count as occluding something else, but that started to crash my computer and generate insanely large files storing all the data, so it doesn’t look like that’s a winner. In my daydreams I’ve been thinking about writing a custom script to hide all water a certain distance from the avatar, but I don’t know if that would work or just look horrible.

Which all leads to a question about the overall exhibition of water itself. If the idea is to demonstrate Unity’s professional water set up with different parameters, and if the act of doing so causes the game to slow down, am I right in trying to “fix” that problem, or is that actually part of how the water is experienced? Just as much as some water is see through and some is opaque and some is red and some is pixellated, isn’t it also the case that, as an overall exhibition, the water just happens to slow down time? Maybe that’s just a “physical” property of water in this setting. It definitely has the benefit of drawing attention to the technical status of the water, for one thing, which is kind of the core idea behind the show. Nothing says “this water is constructed from code” better than “the water code is not performing well right now”, right?

Wheels within wheels. Water within water. Though perhaps not the later, might crash the world.

Third-Party Water

Screen Shot 2017-02-16 at 13.23.14

Having more or less conquered Unity’s water (touch “wood” ha ha) I’ve moved on to populating the other building in v r 3 with “third party water” – that is, water I’m obtaining from Unity’s Asset Store. I’ve only installed four of them so far, but already there are interesting elements surfacing (ha ha?):

Broken water. Not unlike Unity’s water, I’m definitely running into problems with the third party water here and there, and it again reveals assumptions being made about the purpose/nature of water in videogames. One of the water’s I’m using comes with a prefab (readymade version) that is scaled to be enormous (100×100 units) which, when you place it in the world, makes it more like a reasonably large lake. Naturally that’s not the size I want in my game – I want a 0.1×0.1 scaled bit of water (1000 times smaller!). But when I scaled the water down to my size, it turns out to “break” it in the sense that all the default parameters are tuned to the water being large. It’s thus been an ongoing battle to work out how to tune the parameters to allow for the concept of a small amount of water. This water “wants” to be a lake, and I want it to be a puddle.

The people’s water. Now that I have water from multiple creators (Unity, plus each of the creators of the third-party water), you really start to get this sense I’d wanted of comparing water to water, and notably thinking of the water as something created by different people. Recontextualised as something to look at, and labelled with the creator’s name (like CruduxCruo above), the person behind the water comes more into focus and the water becomes, even more, something constructed.

Purposeful water. While the main Unity water is clearly trying more or less to look like “real water” of different qualities, it’s already the case that the third-party waters demonstrate different ideas about what water is for in a videogame environment. Notably, I have some cartoon-style water alongside some realistic water alongside some water that defaults to looking like surf rushing across sand. So we have not just the aesthetics of parameters tweaks to realism (in the Unity building), but also larger questions of how to represent water graphically.

The price of water. Most of the water on the Unity Asset Store costs money. Naturally this leads to questions of whether the water you’re looking at in the game was worth the cost (I’ve added the cost to each of the didacts). This naturally interacts with the purpose idea above. Your initial reaction might be that more expensive water should look more realistic, perhaps, but there might be other considerations that will come up through the exhibition. (There are also hidden considerations like how “easy to use” the water is, how many parameters it offers, etc.) This is also shown in a really nice way because the one free water I’m displaying also has a paid version, so there’s a direct comparison there of what added value you get when you use the paid version.

There’s bound to be more as I proceed, but it’s kind of amazing to me just how generative this stuff is in terms of really thinking about how videogames are constructed, how game engines work, and so on. I’m quite pleased with it.