The Role of Experimentation in Inventing Novel User Interfaces

What is experimentation in software? So last night I read a description of Edison's experimentation process when it came to both inventing the lightbulb and numerous other inventions and developments related to electricity. In one instance, he tried over 500 different materials to identify the best filament to use in the lightbulb itself. In another instance, figuring out how to lay underground wires to transmit electricity to houses. He experimented with many different casings of the electrical wires to maximize insulation while minimizing current loss.

Now he approached these experiments not merely from a theoretical standpoint of what should work in the lab, but from a rigorous try it and find out perspective. Now he was familiar with the theoretical research and would sometimes try something because it was documented in the research. But his core approach was try it and see.

In the software, we have the ability to try many different things, but the thing I'm most curious about is the experimental approach applied to human-computer interaction research, so interfaces. We find that in industry, the companies that succeed the most have a very rapid iteration speed on cranking out new features, new changes to the software, they're basically always looking at the data and then updating the software accordingly until the data indicates that the software is working appropriately. Now there are theories around that as well. If you see books like Hooked, which introduces Fogg's motivation theory, and different things like that, but it's the experimental approach of trying many things out in a real context of use very quickly that yields success.

So what makes a great experimentalist? One, they are familiar with the nitty-gritty of the actual materials and how to work with the materials. They described one experimental physicist as the mind of a theoretician in the hands of a blacksmith. He knew metallurgy, he knew all the properties of metal, but he knew how to work it as well. So a great experimentalist knows his materials, knows how to work with them. A great experimentalist does know the theory, but he is not slave to the theory. I would say trying it in the lab is always considered to be the ultimate way to find out whether something works or not. The theory is always suspect until it's proven in the lab. You don't prove the theory, you prove it in theory land, you prove it in the lab.

The... Oh man. What's also true of great experimentalists is that they are dogged in their pursuit of developing... of solving the problem. They will spend countless iterations working through all the different variables over and over and over and over and over again until they solve it. Another thing that's true is creativity. Experimentalists are creative in thinking outside the box, both in how they solve the problem, like the different ways in which they pursue the experiment, but also the way that they structure the experiment itself so that it gives them the information that they need. Their analysis of the data is also extremely creative and insightful so that they can build a test that quickly validates or invalidates a theory that they have.

And finally, many experimentalists, probably the greatest experimentalists, are commercial in nature in the sense that they want to build something, create something that is of value in the world, and they see what is most valuable. They're not pursuing things completely on a whim. Now, perhaps this is a quality of those who manage experimentalists so that they spend their experimental time working on valuable problems, both to the firm and to humanity, and they'll send things on total larks, following their interests purely.

I think the two phrases used were circumscribed freedom, which was the Bell Labs model, and then a long leash, I think, with... There was something with a long leash as well.

So, experimentation in computing interfaces, we, for one, as Alexander Obenauer pointed out, the feedback loop between thinking of an interface and creating it is too long, and we need to be able to iterate on interfaces. He's got a research project called Gestural Interfaces that he's working on at Nikon Switch, and we need to be able to imagine an interface and quickly put it together and try it out. So, in other words, we need to improve our experimental setup for interface development, even though we have some great tools where software's gotten much, much cheaper to build, it's still too time-consuming to be able to build different interfaces on top. Another part of our experimental interface that we need to figure out is when we have software that is being used by enough people to be able to gather the data about how they're using it, so they have this real context of use that Andy Matuschak writes about. We often... There's cost in switching out the interface on those people who are in that real context of use. So, how do you develop an experiment that has the necessary rigor to understand whether a particular thing is useful or not?

We're probably at a period in computing interfaces, much like the Curies in early quantum physics, where we experiment on ourselves. We have to be the individual that will try something. Alexander Obenhauer has modeled that with his OLLOS project, certainly with Composer we're trying that at DXOS. So, also thinking of a tweet from Geoffrey Litt, where he gave a simple recipe for making a good thing, which I think was from Charlie Munger, which is basically, make lots of bad things really fast.

And the thing that I have realized about myself is that I'm good at making things fast and learning from them, but I'm not good at making many iterations. The place where I need to grow as an experimentalist is doing many iterations and being more dogged and determined.