Numbers in black boxes

I sometimes feel like I’m, in essence, a bad coder. Partially because of the impostor syndrome I (hopefully) have, but mostly because code to me is just means to an end. Not that I don’t end up enjoying reading about some minute implementation detail every now and then, but it’s usually related to something more practical than theoretical. I guess that being interested in everything leaves me no room for being deeply interested in something enough To Be the Best ™. Jack of all trades, master of none, as they say - although I don’t think the opposite is nowhere near as interesting.

Probably because of that, besides game development, I love any kind of creative computing. That’s a nice way to say “I love hacking useless stuff together with Python”- like NosePicker, which was a PointerPointer-inspired experiment that I made for an Information Retrieval class just so I had an excuse to use OpenCV’s face detection. The most creative part about that project was figuring out an actual use for finding out people’s faces in images so I could justify it with something other than “I love hacking useless stuff together with Python”: as far as the record goes (and the paper I just found about that project), I devised a way to make it easier to decrease the time needed to analyze security camera footage by marking frames with people in it in certain areas. Even I bought that by the end of my presentation!

In all these years of gamedev and creative computing, the best concept I’ve internalized is that sometimes it’s better to think of things in a relative rather than in an absolute manner. Sometimes, all you need is analyzing the kind of numbers that you can extract from something, and then figuring out the black box you have to build to make those numbers mean something. This lets you, most of the time, free yourself from creative constraints, which is always nice on an initial phase. Don’t get me wrong: this kind of approach is highly experimental, therefore, I’m by no means advocating “barely think of something and just run with the ball until it’s done”, because God knows how many times I had to (unwillingly) deal with the consequences of that kind of thinking. But my point is: if you have something that gives you numbers, you can turn them into something cool, even if you’re completely faking everything.

That video is another good example, that came from a class where we had to do a project using sensors. I’m a sucker for animatronics and Jim Henson since I was a kid, so I got inspired to do a sock-puppet controller for virtual characters based on Waldo. I just had a bunch of sensors and the final idea, so all I needed was a black box. For the mouth, I ended up using a magnetic sensor: I had a magnet on the upper part, and the sensor in the lower, so smaller readings would open the mouth, and bigger ones close it. That kind of lateral thinking would never happen if I spent my time searching for a proper sensor instead of just embracing kludging analog readings into something else via multiplying magic numbers here and there.

There’s also a ton of that in OctoRhythm, the game I released for Leap Motion. Usually I got questions from other developers about how did I filter stuff, only to be answered with “oh, I just average some stuff and cut out values under a threshold” (which is, by definition, a filter, but the result made it look like something way more complex, apparently).

So how does that relate to Aracnia? When my advisor said “how will you analyze the data you get from the biofeedback sensors?” I remembered the relativeness of numbers in black boxes and didn’t panic - I’d just compare the results with each other. There are a ton of things to consider though: I’d be using sweat detection, which could vary depending on the humidity and temperature, for example. That’s one of the reasons that made me design the 4-phase test (as described on this post). So the answer to “what the hell is a reading of 50 in GSR?!?” can be “who the hell cares! It’s less than 90!” and scientifically sound at the same time.

Fortunately, GSR has a typical stress-response shape that makes “who the hell cares” be even more sound.

Benchmarking, evaluating & sensors

Subjective tests

As this is a masters dissertation, it needs to be scientifically tested. That’s when we hit a wall: the description of fear is subjective. Both to the person who feels it and to a person that may measure it by observing reactions.

To aid that, there are standardized questionnaires to try and somehow score the phobia level of a person. The problem is: it’s kind of an “open” result; you have a score, but where is the line drawn? The best you can do is compare between people, or between two results from the same person.

That’s a very basic way of validating treatment: test a person (which I’ll refer to as benchmarking just because itsounds like a total proper term for that), perform whatever process you’d like to, then test the person again and proceed to compare results (validation).

In my project, I’ll be using the Fear of Spider Questionnaire (FSQ) and the Spider Phobia Questionnaire (SPQ). The SPQ contains 31 questions, with “yes” or “no” as possible answers. The FSQ was designed to complement the SPQ and contains 18 questions with answers graded from 1 (“completely disagree”) to 5 (“completely agree”). With that, I’d have a scientifically validated set of subjective tests with associated scores to verify if interacting with the game would have any effect with the subjects (which I’ll call “treatment” from now on, just to use a more generic term).

Do I think the answers are prone to change, thus changing the final score in a considerable amount? I’m not sure. But I still think the overall phobia will diminish.

So how can I prove that? Enter bio sensors.

Biofeedback

It’s obvious that our body responds to even slight forms of stress. Some of them might be visible to a trained eye, but most of them are invisible. Unless you wire people up - hence polygraphs and such.

So, it’s obviously stressful having to interact with your phobia, on different levels. Some people can look at pictures of spiders, but wouldn’t get near one. Other people can barely look. In my girlfriend’s case, barely glimpsing at a spider leg makes her jump from her seat. Obviously follows sweat and heart rate increase - it’s her body's fight-or-flight response.

Hence, two of the easiest ways to detect stress are changes on heart rate and skin conductibility, so those are the sensors I decided to use. The heart rate has quick response, but a slow “fade” to normal rates, while the skin conductibility (aka Galvanic Skin Response, or GSR) tends to back to normal faster, so they can be kind of complimentary.

As someone interested in hacking stuff together, but with (almost) absolutely no formation in electronics, the Arduino was my obvious choice. The second reason for that was the fact that, even though the sensors are cheap and easy (electronics-wise), their commercial counterparts are expensive as hell.

Objective tests

So how do I get objective results out of a subjective situation? You need a bunch of sensors, an arachnophobic test subject and a simple exposure test. The test I’ve come up with consists of displaying several pictures to the subject, in 4 phases.

Phase 1:

In the first phase, you’re normalizing their readings: sweat and heart rate can vary depending on the situation, the place you’re in etc., so you need to get the subject’s current “standard” reading. We display 15 random neutral pictures (no spiders here), for 10 seconds each.

Phase 2:

We can test out the person’s exposure to some pictures that LOOK like spiders, but aren’t - bugs, beetles etc. That MIGHT set their phobia off, so you may want to warn them specifically that there aren’t any spider pictures yet (I’m still undecided about that and adding insect photos on phase 1). Therefore, we display 20 photos in quick 1-second flashes.

Phase 3:

If it wasn’t kind of terrible on my part, I’d call this phase the “scare-test”. You want to test out how the person’s organism behaves with a quick glance of spiders. It’s the same as before (20, quick-flashed pictures), but with some of them being spider pictures. I’ve set the test to display these spider pictures way smaller than all the others to reduce the stress level that the person is exposed to (see?! I have ethics! Or at least a heart.)

Phase 4:

The stress test: it’s the same as Phase 1 (15 pictures, 10 second exposure) but with spider pictures amidst them. That will measure how long a person can stare at a spider picture. As I’m obviously no sadist, there’s a “panic” button to stop the current picture and show the next one. Not to mention the person having the urge to skip the picture is excellent data.

All sensor readings must be recorded, and you have to log all the information from the interface as well (which picture was displayed, for how long, did the person cancel it or not, was it a spider picture etc). Preferably with an unified time stamp so you can plot the sensor data and tag it with all the interface information. You might also want to record it on video to check body movement, looking away etc. (you COULD do that by OpenCV or some other form of computer-aided tracking, but the good ole human eye will spot any undeniable differences way quicker). You may also train your self control by not posting it on YouTube later.

If after that you still have a ~~girlfriend~~ test subject, you can do the treatment and then re-do the test, to compare the before and after results. Try to replicate the original environment as best as you can (to have the standard readings as close as possible to the original), but my suspicion is that probably the variations themselves will be the best shot on measuring how successful the treatment was.

I’m off to try and find some components for sensor building, so hopefully in some time I’ll come back with some electronics-related posts.

Yanko's Experimental Game Joint — Numbers in black boxes

See, that’s what the app is perfect for.

Numbers in black boxes

More you might like

Benchmarking, evaluating & sensors