Paradigms and Inductive Knowledge

Since I've been on business trips last week and now this, I've had a bit more time than usual for reading, so I've been working through Jim Manzi's Uncontrolled: The Surprising Payoff of Trial-and-Error for Business, Politics, and Society. The book is about the use of controlled tests in business and politics, but the first hundred pages or so is a very interesting discussion of the development of the scientific method and of the Randomized Field Trial.

One of the really interesting things he talks about in this section is the way in which science gets around the Problem of Induction: the fact that one cannot get from an observation that things have happened a certain way in a number of specific instances to an absolute rule.

So, for instance, the law of gravity, for all that we call it a law, has not been absolutely "proved" by extensive experimentation. It could be that if you drop a quarter right now, it will float in the air, or fly upwards, rather than falling to the ground. However, even though the extensive observation of falling objects doesn't prove the law of gravity, we act as if the law of gravity is proved because doing so allows us to make all sorts of useful predictions. Indeed, if a scientist observes something happening which appears to be contrary to gravity, his first reaction would probably be to assume that there is some other factor at play and that gravity is, in fact, still applicable. Assuming Newton's laws becomes a "paradigm".

[Thomas] Kuhn argued that to make practical progress, a group of scientists accepts an underlying set of assumptions about the physical world, along with accepted experimental procedures, supporting hypotheses, and so on. This paradigm helps to create a coherent discipline. The day-to-day work of scientists is to solve intellectual puzzles that fall within the relevant paradigm. Kuhn calls this normal science, or "worker-bee" science. Anomalies -- factual observations that contradict the tenets of the paradigm -- are rejected and either they are held aside as problems to be solved later or the paradigm is modified slightly to accommodate them. [Uncontrolled, page 24]

Of course, the example I used above related to Newton's laws is a perfect example of this as it is a paradigm that held sway for many years but was modified/replaced in the 20th century by Einstein's theory of relativity. Einstein's theory could explain phenomena which Newton's couldn't (no fault of Newton's, they were unobservable with the technology of his time), and so relativity became the new dominant paradigm.

Of course, there's not just one operative paradigm. Paradigms can be nested and there are different sets of paradigms which apply to different fields.

What's fascinating to me about this is how it uses a sort of factional competition to both get around the problem of induction (that we can't derive absolute laws from observations) while getting around the danger of getting locked into bad explanations that would seem to come with saying "let's just assume it's true".

I'd picked up the book because of my interest in test and control experiments in a business or civic setting which Manzi talks about later in the book. (He was one of the founders of Applied Predictive Technologies, a company which produces the Test & Learn software that I use at work.) However, thus far, I've actually found the first third of the book in which Manzi discusses the epistemology of the experimental method to be the most fascinating part of it.