LiveLabs: initial reflections on building a large-scale mobile behavioral experimentation testbed

We believe that, for successful adoption of novel mobile technologies and applications, it is important to be able to test them under real usage patterns, and with real users. To implement this vision, we present our initial effort in building LiveLabs, a large-scale mobile testbed for in-situ experimentation. LiveLabs is unique in two aspects. First, LiveLabs operates on a scale much larger than most research testbeds it is being deployed in four different public spaces in Singapore (a university campus, a shopping mall, an airport and a leisure resort), and is expected to have a pool of over 30,000 opt-in participants. Second, LiveLabs not only instruments smartphones and the infrastructure to gather deep individual and collective context, but also provides a unique experimentation platform that automates many aspects of behavioral experimentation, such as subject selection and context-triggered delivery of interventions. We briefly describe some of the research challenges associated with building such a large-scale deep-context collection testbed, as well as the current status of LiveLabs. We then share our perspectives on the challenges of setting up and operating such testbeds, with the expectation that our experiences will prove useful to other researchers looking to build similar testbeds elsewhere.