An exploratory design science study on theory testing using crowdsourcing

Abstract

Theory in Information Systems (IS) is very important to the development of the field. Theory building, and theory testing seeks to accumulate knowledge about the relationships between people and technology. Testing theory can be difficult to accomplish, especially when it involves humans, a diversity of methods and sources, multiple experiments, large data sets, and careful tuning of conditions and instruments. Crowdsourcing is a strategy supporting the distribution of activities to crowd workers, which suggests that it may be used to support theory testing.

This exploratory study seeks to analyse the adoption of crowdsourcing in theory testing, and to develop guidance for researchers to instantiate the strategy in their research projects.

The study adopts the design science research paradigm to explore incorporating the crowdsourcing strategy in theory testing, and to evaluate its viability and utility. According to the principles of design science research, the study is structured around the construction of several interconnected IS artefacts: 1) a conceptual framework articulating the main principles of theory testing; 2) a pattern model of theory testing, which codifies existing research approaches to theory testing; and 3) a decision tool, which codifies guidelines for researchers making decisions on which research activities to crowdsource.

In order to build the conceptual framework and pattern model, the study conducts a systematic review of theory testing in the IS domain. Both the conceptual framework and pattern model are then operationalized in the decision tool. The utility of the various artefacts is then assessed with the participation of research practitioners.

This study is relevant because it synthesizes knowledge about theory testing, builds innovative artefacts supporting the adoption of crowdsourcing in theory testing, helps academic researchers understanding the theory testing process, and enables them to adopt crowdsourcing for theory testing.