Philosophy of Science Workshop : Experiment and Theory

In medical research, meta-analyses of randomized controlled trials (RCTs) are praised for mitigating the problem of confounding due to the small sample size of individual RCTs. Yet, meta-analyses have also been severely criticized (Ioannidis 2016; Stegenga 2018). An underestimated limitation is that most RCTs suffer from conflicts of interest (Roseman et al. 2011). Unfortunately, current protocols for meta-analyses do not specify how to deal with the problem. “Meta-level” evidence seems to suggest that conflicts of interest raise the probability of biased estimates (Friedman and Richter 2004; Kjaergard and Als-Nielsen 2002). And yet, “first-level” evidence would indicate that the very same trials are prima facie more reliable, in virtue of their better design. Intuitively, the two considerations pull in different directions, namely including vs excluding RCTs with conflicts of interest from meta-analyses. In this paper, we argue that evidence from biased studies may be useful to more accurately calculate effect sizes, conditional on using an appropriate discounting procedure. Since evidence of conflict of interest is a “meta-level” – rather than “first-level” – kind of evidence, we suggest that it be dealt with by using an indirect-evidence model of bias (Welton et al. 2009; Verde et al. 2020).

A debate has recently taken place in the philosophical literature about the specific challenges that arise for assessing the reliability of simulations once the latter have reached a certain level of complexity. It has recently been argued by Lenhard and Winsberg (2010) that for sufficiently complex simulations, the numerical scheme and the model assumptions become entangled in such a way that, when the model disagrees with the data, the sources of the failure become impossible to locate. Lenhard and Winsberg (2010) refer to this challenge as one of `fuzzy modularity’: the procedures of verification and validation supposed to ensure that the computer simulation is free of numerical errors and based on correct modelling assumptions can no longer be separated. The challenge is not only one of identifying the sources of the discrepancy in order to correct them, as it creates a parallel problem when the simulation’s output agrees with the data. In that case, one cannot be sure that the agreement does not stem from corrections or uncertainties in the model cancelling out.

In this talk, I argue that, while the fuzzy modularity of complex simulations does indeed undermine the use of verification and validation, the spectrum of methodologies available to test the reliability of simulations is broader than what the language of verification and validation can capture. Other procedures, that have not been given a precise formulation yet but have nonetheless been used by astrophysicists, succeed in escaping the challenges that a lack of modularity may generate. My aim here is first to flesh out more precisely what one of these methodologies, that I will refer to as that of crucial simulations, amounts to. Second, it is to emphasize the features of this methodology that permit to minimize the holistic challenges associated with fuzzy modularity

Much of the literature on bias focuses on non-epistemic values related to social factors. Discussions of cognitive biases, on the other hand, tend to be focused in domains such as clinical trials or other examples in biomedical literature. Here, we explore how cognitive biases can play a role in investigations of fundamental physics. We explain some of the existing mechanisms put in place to counteract these biases by examining some of the practices that fall under the heading of “blind analysis”, as used in some experiments in high energy physics. We explore three different ways in which we might classify these practices, which we suggest may be useful for different purposes.

In classical mechanics, the state of a physical system tells us everything we need to know about the possible interactions that it can enter into. The quantum analogue of the classical state description is the state vector, but in this talk I will argue that it is not the state vector but its probabilistic generalisation, the density operator, that most fundamentally characterises a system in standard quantum theory. Density operators represent open systems in standard quantum theory, but there is a general framework for describing open systems—the general quantum theory of open systems—that I will argue should be thought of as more fundamental than standard quantum theory. In particular I will argue that no matter how one interprets quantum mechanics, just so long as one takes it to be complete, there are good reasons to think of it as grounded in the general quantum theory of open systems.