Is the Scientific Method a Part of Science?

Introduction

You probably all remember it, and I can almost guarantee that you were all taught it if you went through any sort of standard American education system (with full recognition for my non-USAian readers). It’s called the Scientific Method.

That thing where you start with a question, form a hypothesis, do an experiment, see if it supports or refutes your hypothesis, iterate, etc. This thing:

Flow Chart showing the Scientific Method

The question is, does anyone outside of Middle and High School science class actually use it?

A Science Fair Question

I recently judged a middle and high school science fair here in Boulder, CO (USA). The difference in what you see between the two, at least at this science fair, is dramatic: High schoolers are doing undergraduate-level (college) work and often-times novel research while middle schoolers are doing things like, “Does recycled paper hold more weight than non-recycled?” High schoolers are presenting their work on colorful posters with data and graphs and ongoing research questions, while middle schoolers have a board labeled with “Hypothesis,” “Method,” “Data,” and “Conclusions.”

I was asked by a member of the public, after I had finished judging, why that was. He wanted to know why the high school students seemed to have forsaken the entire process and methodology of science, not having those steps clearly laid out.

My answer at the time – very spur-of-the-moment because he was stuttering and I had to catch a bus – was that it IS there in the high school work, but it was more implicit than explicit. That often in research, we have an idea of something and then go about gathering data for it and see what happens: It’s more of an exploration into what the data may show rather than setting out on some narrow path.

That was about a month ago, and I haven’t thought much more about it. But, the Wired article today made me think this would be a good topic for a blog post where I could wax philosophical a bit and see where my own thoughts lay.

Field-Specific?

A disclaimer up-front (in-middle?) is that I’m an astronomer (planetary geophysicist?). This might be field-specific. The Wired article even mentions astronomy in its list of obvious cases where the Scientific Method is usually not used:

Look at just about any astronomy “experiment”. Most of the cool things in astronomy are also discovered and then a model is created. So, the question comes second. How do you do a traditional experiment on star formation? I guess you could start with some hydrogen and let it go – right? Well, that might take a while.

That said, I’m sure that other fields have the same issues, and it’s really just a big grey area. What I’m going to talk about, that is. Some fields may be more towards one end of the greyscale than the other.

A Recent Paper I Co-Authored

I recently was a co-author on a paper entitled, “ Distribution of Early, Middle, and Late Noachian cratered surfaces in the Martian highlands: Implications for resurfacing events and processes.” The paper was probably the only professional paper I have ever been an author on that explicitly laid out Hypotheses, tests for those hypotheses, what the conclusion would be depending on the results, then the Data, then the Conclusions. And it was a really good way to write THAT paper. But not necessarily other papers.

A Recent Paper I Wrote

I had a paper that was recently accepted (too recently to supply a link). The paper was about estimating and modeling the ages of the largest craters on Mars. There was an Introduction, Methods, Data, and Conclusions. There was no Hypothesis. It was effectively a, “Here is something we can explore with this database, let’s do it and put these numbers out there and then OTHER people may be able to do something with those numbers (or we can) in future work.” There really was no hypothesis to investigate. Trying to make one up to suit the Scientific Method would have been contrived.

This is also something the Wired article mentions:

… often the results of a scientific study are often presented in the format of the scientific method (even though it might not have been carried out in that way). This makes it seem like just about all research in science follows the scientific method.

This is especially the case in medical journals, but not necessarily elsewhere.

Change the “Scientific Method?”

The Wired article offers this as the “new” method:

New Scientific Method (via Wired)

Here’s the accompanying justification:

There are a lot of key elements, but I think I could boil it down to this: make models of stuff. Really, that is what we do in science. We try to make equations or conceptual ideas or computer programs that can agree with real life and predict future events in real life. That is science.

I will preface this next part by saying I am NOT up-to-date on the latest pedagogy of teaching and I am NOT trained in teaching methods (other than 50+ hours of Graduate Teacher Program certification during grad school plus teaching several classes, including two as instructor of record).

That in mind, I think that this is a good idea in later years of grade school education. In the early years, I think that the methodology of the Scientific Method helps get across the basic idea and concepts of how science works, while later on you can get to how it practically works.

Let me explain with an example: In third grade, I was taught about the planets in the solar system plus the sun, plus there are asteroids, plus there are random comets. In eighth grade, I was taught a bit more astronomy and the solar system was a bit messier, but still we had those nine planets (this was pre-2006) and the sun and comets and asteroids plus moons and rings.

Then you get into undergrad and grad school, and you learn about streaming particles coming from the sun, that we can be thought of as being in the sun’s outer-most atmosphere. You get taught about magnetic fields and plasmas. Zodiacal light. The Kuiper Belt, Oort Cloud, asteroid resonances, water is everywhere and not just on Earth, and all sorts of other complications that get into how things really work.

To me, that’s how I think the scientific method should be taught. You start with the rigid formality early on, and I think that’s important because at that level you are really duplicating things that are already well known (e.g. Hypothesis: A ping pong ball will fall at the same rate as a bowling ball) and you can follow that straight-forward methodology of designing an experiment, collecting data, and confirming or rejecting the hypothesis. Let’s put it bluntly: You don’t do cutting-edge science in middle school.

In high school — in a high school with good science education — you actually do start to learn more about the details of different ideas and concepts and solid answers are no longer necessarily known. You want to find out, so you might design an experiment after seeing something weird, and then gather data to try to figure out what’s going on.

That’s how science usually works in the real world, and I think it’s a natural progression from the basic process, and I still think that basic process is implicit, if not explicit, in how science is usually done.

I just got back from a major science conference two weeks ago, and I sat through several dozen talks and viewed several hundred poster presentations. I honestly can’t remember a single one that was designed like a middle school science fair with those key steps from the Scientific Method.

Of course, another aspect is that if we get rid of it, we can’t make comics like this that show how it’s “really” done (sorry, I forget where I found this):

How the Scientific Method Really Works
(click to embiggen)

Final Thoughts

That said, this has been a ~1400-word essay on what I think about this subject. I don’t expect much to change in the near future, especially since – as the Wired article points out – this is firmly entrenched in the textbooks and in Middle School Science Fair How-To guides.

But, I’m curious as to what you think. Do you think the Scientific Method is useful, useless, or somewhere in-between? Do you think it should be taught and/or used in schools? Do you think it should be used in science fairs? Do you think professional scientists should use it more explicitly more often?

Advertisement

Comic Strip “Get Fuzzy” Tackles Pseudoscience with the “Big Bonk” Theory

Introduction

I listen to a lot of Coast to Coast AM. I know that a good fraction of the guests are quite literally certifiable, but it’s a good distraction from the mundane work day and, well, it can get addicting.

One of the hallmarks of the show, and something that its originator Art Bell would frequently say, is that that they will put ANYONE on to talk about ANYTHING (so long as it’s civil, etc.). This means that you get a lot of crazies. Same with folks who call in (despite the screeners). People who advocate pretty much anything and everything that the “mainstream” does not advocate. And a frequent refrain is that their “theory” is just as good as the mainstream one.

The Parody

Note: All images posted here can be opened in a new window and will be roughly double the posted size for easier reading.

Early in my college career I was introduced to the comic strip Get Fuzzy drawn by Darby Conley, one of the only things to which I’m thankful for my roommate. But that’s a different story. Anyway, the premise behind the strip is a single guy, Rob, living in an apartment with an anthropomorphized talking cat and dog. The cat, Bucky, is fairly insane while the dog, Satchel, is fairly dopy. I think the height of the strip was a few weeks where the cat was suing the ferret next door for knocking out his tooth … on Judge Judy.

Anyway, starting September 20 and going through October 2, barring the Sundays, the artist ran a story where Bucky decides to challenge, in a very Coast to Coast AM -like way, the idea that an asteroid killed the dinosaurs. The September 21, 2010 strip follows:

September 21, 2010, Get Fuzzy Strip

Well, when put like that, the scientific consensus seems laughable. But that’s because you have left out a significant amount of information. But it gets better. In the strip from September 24, 2010, the final line from Satchel is actually one that the Coast to Coast AM host, George Noory, has stated many times to someone’s crazy idea:

September 24, 2010, Get Fuzzy Strip

Continuing:

September 27, 2010, Get Fuzzy Strip

September 29, 2010, Get Fuzzy Strip

September 30, 2010, Get Fuzzy Strip

October 2, 2010, Get Fuzzy Strip

The Moral of the Story: Final Thoughts

Why am I taking the little free time I have these days to make a post about a comic that has little to do with astronomy? Because it has everything to do with how people think. And it has everything to do with how that maybe 1% of the population is willing to elevate any hair-brained notion to the same status as a scientific theory that has theoretical and observational backing and has withstood all attempts at falsifying it. And I do mean all.

This series may be a comic strip, but people really do this kind of thing, and worse, there are people who unquestioningly believe the one lone nut who promotes it!

The next time you read about or hear about or see about someone’s “theory,” pause and think. Don’t just accept it at face value. Is there evidence behind it? Are they giving you all the background information? Are they making things up, or are they “on the level?” In the end, is it something that’s worth your time to investigate further and seek out independent information, or is it just another “Big Bonk?”

Legal note: All images shown here are housed on the original server and I did not download them. All are copy written by Darby Conley and distributed by UFS, Inc. and posted online at comics.com. For brevity, I have not posted all strips in the series but picked the most relevant.

What’s a Theory? Dictionary versus Science

This is a short post so I’m going to dispense with the normal “Intro” and “Final Thoughts” sections. Back in December 2008 in one of my first posts, I talked about what a scientist means by “Theory” because it’s very different from the general public. The post is reasonably well-used with over 2,300 views (averaging somewhere around 5/day) and it gets a fair amount of hits from Google under the “define: [word]” category of searches.

The crux of that post was that the word “theory” in popular use is simply “vague idea of something” as opposed to the use by scientists being, “This has been elevated to the highest level of certainty possible in science, withstanding hundreds or thousands of attempts to disprove it.”

Recently, though, I’ve been seeing some blog posts and some posts on creationist sites that disagree with this, trying to back up a very fallacious idea that “theory” in science means what the general public uses it for. Unfortunately, when I decided to write this post, I could not readily locate an example, and for that I apologize. But I promise you that unless I was having some very realistic dreams across multiple nights, this is not a straw man argument.

However, the arguments that I have read generally go as follows: “Scientists claim that you can’t say ‘Evolution is JUST a Theory’ because ‘theory’ to them means the pinnacle of scientific certainty. However, the [insert definition number] in the [insert your favorite dictionary] says that ‘theory’ means ‘a supposition’ [or similar language]. Therefore, it is perfectly reasonable to say that evolution is just a theory or the Big Bang is just a theory.”

To say that this is a ridiculous argument is an understatement. It’s exactly what the British Chiropractor Association did to Simon Singh recently. Bot for those of you who don’t know that whole story, let’s have a different example: Sally says, “The star Sirius was really bright last night before it set.” Johnny knows that the word “star” can mean both a famous person (as in “movie star”) or a celestial object that is a giant ball of gas that when alive produces energy through fusion. Despite the context, Johnny chooses to think that Sally meant the up-and-coming movie star with the stage name “Sirius.” Admittedly, like many of my examples, this is a bit contrived, but it is pretty much the same thing.

So, in summary, it doesn’t matter what definitions 1-4 say a word means. In science, the word “theory” has a very set definition. Claiming that scientists mean something else when using it and trying to argue that the dictionary is proof of this is simply absurd, and in itself is a straw man argument.

What Is Science, Its Purpose, and Its Method?

Introduction

Following up on my post “Terminology: What Scientists Mean by “Fact,” “Hypothesis,” “Theory,” and “Law”,” as well as a recent planetarium lecture I gave on young-Earth creationism in astronomy, I thought it would be a valuable post to go over specifically what the purpose of science actually is, and how science goes about, well, science.

I need to make three things very clear up-front: First, I am not a philosopher. I have not taken any philosophy classes, nor have I taken a philosophy of science class (though I think I probably should).

Second, even though “science” is an inactive noun – where I use the word “inactive” to mean that it is a process and a mode of thinking – I will be using it throughout this post as an “active” noun, personifying it to actually “do” things. This is how it’s used in popular culture, and I see no real reason to take efforts to not go with the colloquial use in this posting.

Third, this post is going to serve a dual purpose by contrasting the scientific method with the creationist “method” in order to show how science differs in key, important ways.

Dictionary Definitions of Terms

The way the dictionary that Apple kindly provides on their computers defines “science” as: “The intellectual and practical activity encompassing the systematic study of the structure and behavior of the physical and natural world through observation and experiment.” There are three sub-definitions, but that main one emphasizes that “science” is an activity, a study, and one that looks for natural explanations.

My only qualm with this definition is that I would add to it not only what it does or how it operates, but its purpose, as well: “The purpose of science is that once it has provided an explanation for the physical and natural world, it allows one to use that explanation to make predictions.” I know that when I stand on one foot, if I don’t shift my weight to that one foot, I will likely fall if I do not support myself. That is because I have repeated observations that tell me this. Without that predictive power that in the future I will fall if I don’t shift my weight, then all those previous observations are fairly worthless.

In this section, I also want to define “dogma.” Using the dictionary again: “A principle or set of principles laid down by an authority as incontrovertibly true.”

Now, hopefully I’m stating the obvious, but “dogma” and “science” are not equivalent. In fact, I know that I’m not stating the obvious because there are many, many, many people out there who believe that science simply leads to dogmatic facts/ideas/theories, etc. This is not true. And in the rest of this post I will show you why.

A Look at the Creationist “Science” Method

Before I say anything else, I want to emphasize that this is not a straw man argument, an exaggeration, or anything else that may lead to you thinking this is not true. This section is really how many – if not most or all – biblical literalists view science, and this is how they decide what science to incorporate into their worldview.

Ken Ham, the CEO of the “Answers in Genesis” (a young-Earth creationist think-tank in the US, now separate from the Australian group by the same name), has explicitly stated that one must start with the Bible, while others at AiG have stated that even logic and science itself flows from the Bible, for without it, you couldn’t even have the tools that science uses.

Now that that’s out of the way, let’s look at a flow chart:

Flow Chart Showing Faith-Based 'Science'

The above flow chart shows the basic, fundamental process that most biblical literalists use to vet science. They may get an idea, or hear of something. Let’s use a young-Earth creationist mainstay, Earth’s magnetic field. Data shows that Earth’s field has gone through reversals in polarity at many points in the past. The data is clearly out there for anyone to examine, and it is unambiguous that crustal rocks record a flip-flopping magnetic field.

Now, does it fit in the Bible? Creationists such as Kent Hovind say that it does not. The result is that alternating magnetic fields are simply not possible. In fact, to quote him: “That’s simply baloney [that there are magnetic reversals in the rocks]. There are no ‘reversed polarity areas’ unless it’s where rocks flipped over when the fountains of the deep broke open. … This is a lie talking about magnetic ‘reversals.'” (Taken from his Creation Science Evangelism series, DVD 6:1.)

Alternatively, Russell Humphreys, of Answers in Genesis, accepts that there have been magnetic reversals, as he is able to fit it into a reading of the Bible. He explains the field reversals as rapidly taking place during the 40 24-hr days of Noah’s Flood. Hence, because they are able to fit it into the Bible, they accept it as a dogma.

A Look at the Scientific Method

You’ll notice that this flow chart is a tad larger:

Flow Cart Showing the Scientific Method

It starts at the same place, with an idea/observation/etc., which we call a “hypothesis.” As opposed to testing this hypothesis against the Bible, it is tested by performing an experiment. In other words, can the idea that you have accurately predict the outcome of an experiment?

If not, then the idea is rejected. If it did accurately predict the outcome of the experiment, then ideally you will do several more and gather other observational evidence, but effectively you now have created a theory. A theory is when all pieces of evidence support that idea, and NO experiment has refuted it.

The next step of a theory is to use it to predict a future event. This is where my definition of science differs from the dictionary by adding these predictive properties (the bottom half of the flow chart). Without the theory of gravity being able to predict the motions of the planets and moons, the behavior of tides, etc., then what good is it other than to have on paper and look pretty?

So the theory is used to predict a future event. If it predicted it correctly, then you simply rinse and repeat. Much of basic scientific research is really just testing theories. Far from being the “dogma” that many creationists will want you to believe, theories are subjected to tests every day.

In fact, scientists WANT to be the one to do the experiment that the theory predicted a different outcome for. That’s where we follow the “NO” arrow on the flow chart. If the theory can be modified to support the latest evidence, then it is improved, and you go back and continue to test the now-modified theory. An example of this would be the addition of Inflation to the Big Bang model.

However, if the theory cannot be modified to support the latest evidence, then we have a scientific revolution. People remember your name. You get Nobel Prizes. And money. And women (or men). Anyone over the age of 10 knows Einstein’s name and know him to be synonymous with “Relativity” and likely even “E=m·c².” Advertisers wish they could be that efficient.

Final Thoughts – What’s the Point, and Why No Spiritualism/Paranormal Allowed?

The point here is that, well, I’m honestly sick of hearing the anti-“darwinist” crowd claiming that evolution, the speed of light, the Big Bang, and many other scientific theories are just a “materialistic dogma.” They’re not. Plain and simple. Dogma is where you believe something as FACT and it cannot be shown to be false, regardless of any evidence. Theories and the scientific method is a process that requires evidence to support it, and no evidence to the contrary. It requires predictive power.

And that is why spiritualism/religion/supernatural/paranormal beliefs are simply not allowed in science. Sorry, they’re not. Why? Because almost by their very definition, they lack any predictive ability. If you can’t use your hypothesis or theory to predict a future event, then they have just been shown not to work. Yes, the Flying Spaghetti Monster may have created us all by touching us with His noodly appendage. That may be a hypothesis. But you simply can’t test that because He in His Infinite Carbalicious Goodness can just choose not to do it again. Or some vaguely-defined “Intelligent Designer” may have caused the bacterial flagllum to exist or have formed the mammalian eye. But that belief does not present any way of being tested, whereas evolutionary theory does (and has shown the precursors to all of those).

And that’s really the point of science: To use testable ideas to explain the where we came from, and then to predict where we’re going.

Terminology: What Scientists Mean by “Fact,” “Hypothesis,” “Theory,” and “Law”

Introduction

I’ve decided to write this post so that I have something to refer to and don’t have to constantly re-define these words: Fact, Hypothesis, Theory, and Law.

This may seem silly. “Why,” you may ask, “would you have to define such simple little words?” The reason is that the colloquial use of these words by the general public is very different from their usage by scientists. And let’s really just jump to the chase here: Calling something “Just a Theory” shows both the ignorance of Cobb County, Georgia public school administrators and anyone else who tries to use that phrase to belittle a scientific conclusion.

Colloquial Use

To use math expressions, the general use of these words goes in order of importance as: Fact > Law > Theory > Hypothesis.

“Fact” in Everyday Language: A “fact” is something that is true. Whether you like it or not, “facts are stubborn things” (thank you, John Adams … or, “facts are stupid things” courtesy of Ronald Reagan). In general use, a “fact” is the strongest thing that can be said about, well, anything.

“Law” in Everyday Language: In everyday language, a “law” is generally on the same level as a fact. A law is something that is true, that generally explains or answers lots of different things. However, outside of politics, “law” is rarely used unless actually referring to something scientific.

“Theory” in Everyday Language: This is where the supposed insult to scientists comes in when you call something “just a theory.” Outside of scientific circles, a “theory” is more of a supposition. “I have a theory that my cat will meow when it hears someone at the door.” It may or may not be “true,” but it’s a supposition I have that is probably supported by at least some sort of observation. But it’s really “just a theory” and is just as likely to be shown wrong at any given time as it is to be shown right.

“Hypothesis” in Everyday Language: A “hypothesis” is sort of on the same level as a “theory,” if slightly below. To most people, they can be used interchangeably, though most will just resort to “theory” because “hypothesis” is an extra syllable longer and makes you sound like a nerd.

Scientific Use

In science, the order of importance of these is almost reversed: Theory > Law > Hypothesis > Facts. In addition, each term has a specific, well-defined use.

“Fact” in Science: It may surprise you to know that a “fact” is generally used the same way – it is an observation – but it is very specific. For example, if I drop a ball while holding it in the air above a surface, it is a fact that it will fall to the surface. This term is usually not used, however — we resort to “observations.” For example, I observe that when the wind blows, a flag will flutter.

“Hypothesis” in Science: This is an “idea” that is formulated to explain observations (or our “facts”). In the above to examples, I might hypothesize that there is a force that pulls on the ball, counteracted when I’m holding it. Or that the wind exerts a force on the flag that causes it to flutter. The purpose of a hypothesis is to explain one or more observations in a cogent way. A good hypothesis must be testable – it must be able to make predictions about what would happen in similar situations – otherwise a hypothesis can never be verified nor refuted … and it remains “just a hypothesis.” At present, String “Theory” is really just a hypothesis.

“Law” in Science: Laws are a descriptive generalization about how some aspect of the natural world behaves under stated circumstances. For example, Kepler’s Three Laws of Planetary Motion are (1) Planets travel in ellipses with one focus being the Sun, (2) planets sweep out equal area in equal time, and (3) a planet’s period-squared is proportional to its semi-major-axis-cubed. Laws are generally made from many facts/observations and are effectively an “elevated” level from a hypothesis. Another example are the Laws of Thermodynamics. Because a Law is just a description of how something behaves and it does not explain why it behaves that way, it is usually considered to be below the level of a theory.

“Theory” in Science: A theory is really one of the pinnacles of science – what nearly everyone strives to make out of their hypotheses. A hypothesis is elevated to a theory when it has withstood all attempts to falsify it. Experiment after experiment has shown it sufficient to explain all observations that it encompasses. In other words, a “theory” has never been shown to be false, despite – usually – hundreds if not thousands of separate attempts to break it. It explains the observations with one or more mechanisms and, because it provides that mechanism, it is considered to be above the level of a Law. Examples these days are the Theory of Relativity, Quantum Mechanics, the Germ Theory of Disease, and yes, the Theory of Evolution.

I should note that theories are usually conglomerations of several different hypotheses, laws, facts, inferences, and observations. For example, while the Theory of Evolution is a theory, various mechanisms for it are generally still hypotheses, such as Natural Selection (though some may quibble with me over that).

Another good example of a Theory is the Standard Model of Particle Physics. This describes how fundamental particles and forces interact. It is based upon countless experiments and observations and it rests on solid mathematical framework. It has many different laws in its make-up (such as how particles behave, or how forces interact) as well as many observations (such as the mass of the proton, or the energy of a tau neutrino).

A third example was partially mentioned above – Kepler’s Laws of Planetary Motion. Tycho Brahe and Johannas Kepler made many detailed observations of planetary positions over the course of many years. Kepler formed a hypothesis about how planets moved based upon the data. From the hypothesis, he made predictions on where planets would be later on. When these were confirmed, his hypotheses were elevated to laws. Later, Isaac Newton came along and with his Theory of Gravity was able to provide a physics-based framework for why and how those laws worked.

Finally, it should also be noted that nothing in science is “forever.” It is always subject to further tests and observations. In many cases, people really do try to do this since that’s how you make a name for yourself. If you’re the scientist who has verified for the 123,194th time that a ball and a feather fall at the same rate in a vacuum, so what? But, if you’re the scientist that has found evidence that gravity itself is not a force emitted by an object but rather a bending of the fabric of space itself, then, well, you’d be Einstein – a household name.

(I make this note because a common argument you’ll see from creationists is that they say materialists always want to uphold the status quo.)

Final Thoughts

That’s really about all I wanted to do with this post – clarify these terms and what they actually mean in science. I’m not naïve enough to think that now suddenly this’ll clear everything up and no one will ever say something’s “just a theory” again, but at least now I’ve gone through all these terms step-by-step so that I can refer back to them when need-be.

Edited to Add: I think my post on “the final epsilon” is a relevant follow-up to this one. If you’re interested in the concept of how classical mechanics can still be a theory even though it disagrees at some level with the theory of relativity, I recommend reading it.