With the proliferation of data, the increasing availability of rather simple tools to analyze data and an increasing number of people who can use these tools in combination with the availability of low cost publication platforms (e.g. blogs), the potential to democratize certain aspects of scientific processes – such as empirical data analysis – seems tremendeous. This might give rise to the idea that everyone who can use these tools (such as Python), and publish the results from their analysis (e.g. via blog posts) can now participate in knowledge production.
An opportunity for data analysis by the masses: If true, the potential of such a development would be enormous: By increasing the number of people that participate in scientific processes, we could increase the coverage of interesting phenomena to explore, research activity would not be constrained to areas that are funded by large institutional bodies, and in general more research could get done.
At the same time, this would represent an absolut shift in the way science has been operating up til now, as people formerly not part of traditional scientific processes (and not trained in scientific knowledge production) now move into new territory, and participate in new processes. In order to understand this shift, we need to understand the modi operandi of scientific knowledge production in the past.
Different modes of knowledge production: There are many ways to look at scientific knowledge production. A very influential distinction has been made by Gibbons et al, which argue that we have to differentiate between “Mode 1” and “Mode 2” of knowledge production.
M. Gibbons, C. Limoges, and H. Nowotny. The new production of knowledge: the dynamics of science and research in contemporary societies. Sage, 1997.
Mode 1 refers to traditional knowledge production processes, by focusing on hierarchical mechanisms and processes executed by a set of homogenous actors from a common disciplinary background. An example would be the ivory tower view of a university, where a scientist or group of scientists with homogeneous backgrounds work on disciplinary problems. This mode is increasingly being replaced by Mode 2 knowledge production, which is socially distributed, organizationally diverse, application-oriented, and trans-disciplinary [GLN97, NSG03]. An example would be a network of university partners with different disciplinary backgrounds collaborating on an application-oriented problem with other stakeholders from e.g. industry or other public institutions.
Mode 3 knowledge production: The proliferation of data, tools and people able to make use of them might give rise to what I might call Mode 3 knowledge production which could be self organized, context-focused, and driven by individuals not primarily trained in scientific processes. An example would be an interested user (or group of users) of a social network platform that looks at data that might explain some online social network phenomenon that they feel worth exploring. Another might be a group of patients performing self experiments or experiments with n=1 in order to explore the cause of personal symptoms or health concerns. These groups might embed the discussion of their findings into community conversations and social sensemaking processes.
While this idea looks appealing on the surface, there are a number of issues. For example: Mode 1 and mode 2 knowledge production differ in terms of organization, but both follow the scientific method in terms of basic mechanisms and values. It is yet unclear whether an emerging mode 3 would adhere to the scientific method as well. Being able to use analysis tools to look at data does not necessarily mean that whatever kind of analysis follows from that contributes to scientific processes in meaningful ways.
The scientific method: So what is the scientific method, i.e. what are some of the standards, ethics and practices that mode 1 and mode 2 knowledge production follow, which a potential mode 3 knowledge production would have to adopt as well? Answers can be found in the philosophy of science, which has long been thinking about the nature of science and scientific processes. This is an entire field that can not be adquately described here – the Hempel–Oppenheim model would just be one of many examples.
However, typical qualities of scientific processes would include, but are not limited to: the ability to reproduce results including a proper description of methods and means of data collection, sharing of data, the quality of hypotheses (w.r.t. falsifiability, explanatory power, understandability, etc), the relation to state-of-the-art research including proper citations of existing literature, critical reflections about the validity of findings, as well as the quality of interpretations and whether they follow from the data.
Do blog posts follow the scientific method? While there is nothing that prevents research published via blog posts to follow the scientific method, more often than not blog posts – even data-oriented ones – fail to meet these most basic requirements. For example, from a data visualization published via a blog post it does not necessarily become clear where the data is from, how the data has been collected, which methods have been applied, whether the results are reproducable, whether the data used will be shared, how the analysis relates to the state-of-the-art of scientific knowledge or whether there is an agreement that the conclusions presented follow from the data.
This is not surprising. In scientific articles, peer-review is the most common (but certainly not infallible) instrument to check whether submitted research follows the scientific method. In blog posts and similar user-generated media, there are currently no established social or other mechanisms enforcing the scientific method, which often makes their results – while potentially interesting – less useful from a scientific perspective. In addition, it is typically impossible for a researcher to ignore a reviewer’s comment (as an editor will make a decision based on reviewers’ comments whether to publish an article or not), at the same time it is usually easy for a blogger to delete an unwanted comment.
Conclusion: Whether a third mode of knowledge production will ultimately emerge is unclear. While the democratization of data analysis will expand without a doubt, it will depend on the masses of amateurs and bloggers to adopt principles based on the scientific method or the masses of scientists to participate and enforce the scientific method in blog conversations or both. It will probably not depend on the technicalities of the publishing medium – whether blog posts or not.
References:
M. Gibbons, C. Limoges, and H. Nowotny. The new production of knowledge: the dynamics of science and research in contemporary societies. Sage, 1997.
H. Nowotny, P. Scott, and M. Gibbons. Introduction – mode 2’revisited: The new production of knowledge. Minerva, 41(3):179–194, 2003.
I'm 
Intentialicious: Markus Strohmaier's Weblog 