Even Science Reporters Are Circumventing Scientific Process

I study impact craters (those circle thingies on other planets, moons, asteroids, comets, etc.). A colleague recently pointed out a manuscript to me that demonstrated a new method to do something with craters. (I’m being purposely vague here to protect the situation.) It was an interesting manuscript, but it was submitted to an open archive (arxiv.org) where anyone can submit pretty much anything that seems sciencey. It has not been through the peer-review process.

Peer-review is not perfect. I’ve written about it before on this blog and discussed it on my podcast. But the purpose of peer-review is to weed out stuff that is obviously wrong. Things that may seem good to a general researcher, but to someone else who really knows the field, it clearly has issues. Other purposes of peer-review are to make sure the work is placed in proper context (usually by citing the reviewers’ works, but that’s a separate issue), making sure that the authors of the manuscript have explained themselves well, that their methods make sense, that they have explored alternative interpretations of their data, etc. In other words, do science “right.” Where “right” is in quotes because there is no formal set of rules by which one must play, but there are general guidelines and important pillars which people should uphold.

After it passes peer-review – if it passes peer-review – then it may be accepted by a journal and published. Some stuff that gets through peer-review is great. Some stuff is utter crap because the process isn’t perfect and because we don’t know everything, and the prevailing scientific opinion can shift with new information.

That is upended in today’s cut-throat world of journalism and a desire to be the first to publish about something that seems new and interesting.

I was contacted yesterday by a freelance reporter for the publication New Scientist. I’m not going to say the reporter’s name, but I have no qualms stating the publication. The reporter, coincidentally, wanted me to comment on the manuscript that had been submitted to arxiv.org. I refused. Here is what I wrote:

Thank you for writing. I am generally happy to comment about crater papers, and I would be happy to comment on this manuscript should it be accepted by the peer-review process. My concern at the moment is that the manuscript is only on an open server to which anyone can submit and it has not been vetted by researchers in the field beyond the authors themselves. The authors also used [specifics redacted] which have some significant omissions, and how that affects their results needs to be assessed by people who know all the ins and outs of their methods, which is not me.

I strongly recommend that you refrain from publishing about this work until it has made it through the peer-review process. It is easy to get excited about new techniques, but at the moment, it has not been vetted by other experts in the field, such that I think writing about it now is premature.

The reporter responded that I had a valid concern, he appreciated my advice, and he would discuss it with his editor.

Then just a few minutes ago, I heard from another friend in the field that she had been asked to comment for the story. She is taking a similar approach, which I greatly appreciate.

But this identifies, to me, a significant problem that those in both the scientific community and skeptic community have pointed out for years: Journalists don’t seem to care about vetting the science about which they write. Now, this could be an isolated example of an over-zealous reporter given the “OK” by their editor. Except it’s not. Too often we see articles about work just at the very edge of the field that offers great marvels and promises, only to hear nothing more from it because it was all based on extraordinarily preliminary efforts. Craters aren’t going to affect your daily life. But the issue here is a symptom of a greater problem. And I think that only if scientists and the reading public demand that reporters stop doing this will we see any sort of change.

Forgiveness, or Why I Like Stargate but Not Hoagland, Creationists, Planet Xers, etc.

Introduction

I’ve thought about writing this post for awhile but never got around to it. Now, I’m writing it instead of going to bed.

If you couldn’t figure it out from the title, the purpose of this post is to discuss why I like some television shows and movies that incorporate some bad science and am willing to forgive that versus why I dislike the purveyance of bad science by people such as creationists, UFOlogists, IDers, Planet Xers, or individuals like Richard Hoagland, Nancy Lieder, Maurice Cotterell, or Whitley Strieber — to name a few.

In other words, why I forgive some, but I don’t forgive others.

For Entertainment Purposes Only

We’ve all seen this or heard this line, especially if we read the 2-pt print at the bottom of many websites for, e.g., astrologers. For them, though, it’s to keep themselves legal. For science fiction shows such as Star Trek, Star Wars, Battlestar Galactica, or Stargate, that really is the intent: To entertain. Well, to make money for the network, but to tell an entertaining story.

I think that Gene Roddenberry was right in that, to tell a good story, it usually has to be about humans and the human condition. That was part of his impetus for having Spock in TOS and TAS, Data in TNG, and later directors to have Kes / 7 of 9 and Neelix in VOY, the various non-humans in DS9, and we’ll ignore That-Series-Which-Must-Not-Be-Named. These were the outsiders looking in on and commenting on and reacting to the humans and how they dealt with new situations.

Star Wars is similar: It can really be boiled down to the classic Hero’s Journey and is about humans fighting for freedom and survival. Stargate is similar, as well, having Teal’c as the alien character looking in for SG-1, Teyla on ATL, and then UNI failed for many reasons, but I think the lack of that non-human character looking in contributed.

Is the science perfect? Abso-friggin’-lutely not. I recently (last week) re-watched the original Stargate movie and then first three episodes of SG-1. In the first episode, the scientist character (Sam Carter, played by the amazing actress Amanda Tapping) has a conversation with the archaeologist (Daniel Jackson, played by the actor Michael Shanks:

Sam: According to the expanding universe model, all bodies in the universe are constantly moving apart.

Daniel: So in the thousands of years since the Stargate was built-

Sam: All the coordinates could have changed.

Daniel: But why does it still work between Abydos and Earth?

Sam: Abydos is probably the closest planet in the network to Earth. I mean, the closer they are, the less the difference in relative position due to expansion. The further away, the greater the difference. In a few thousand more years, it won’t work between Earth and Abydos either.

Daniel: Unless you can adjust for the displacement.

Sam: Right. Now with this map as a base, that should be easy. All we have to do is correct for Doppler shift. Then I should be able to arrive at a computer model that will predict the adjustments necessary to get the Gate working again.

Purists might say there’s nothing wrong with that, in the movie they clearly state Abydos is in a distant galaxy. But, in the TV series, and later in this first episode, they clearly state the Stargate system operates within the Milky Way. You have to have much more power and “dial” an extra glyph to get outside the Galaxy.

Ergo, the expanding universe does not apply in anyway. Sam in supposed to be an astrophysicist. An intro astronomy major would know that this line makes no sense, that galaxies are gravitationally bound objects in this epoch of the universe. Stellar drift – which the script writers use later in the series perhaps because they were told expanding universe doesn’t apply – is a very plausible explanation. But, that doesn’t change the fact that I rolled my eyes and shook my head when I heard that line.

And the follow-up of using Doppler shift to correct for it is equally fallacious: If you’re going to a distant galaxy where expansion plays a role, Doppler shift only gets you the radial velocity towards/away from Earth. You still have to know the Hubble Constant – which they didn’t in the 1990s when SG-1 started – to convert that to a distance, and you would need to know the motion across the sky, which you can’t really get for a distant galaxy (though it would be small relatively speaking).

In other words, the science is wrong. But it wasn’t as though the entire plot hinged upon it. It wasn’t as though the producers were trying to tell us that this is what’s really going on in the world (unlike what William Henry may think).

As such, I’m willing to forgive this kind of thing for the broader entertainment value, just like I’m willing to forgive the fact that everyone somehow speaks English all across the galaxy.

Movies I sometimes hold to a higher standard. For example, I saw the new Star Trek: Into Darkness movie a few weeks ago. Towards the end, the Enterprise is in orbit of Earth, but at the distance of the Moon. No engines. From the shot, they are implying that it is orbiting at the same speed as the Moon around Earth In the space of a half hour or so, the ship is plunging through Earth’s atmosphere, sure to crash. Sorry, but no. Being at the distance of the Moon and traveling at the same velocity is a stable orbit. Or, it took the Apollo astronauts three days to get to the Moon, and three days to get back, under powered travel. Not 20 minutes. No way the ship would be plunging through Earth’s atmosphere so soon. And that bothered me. Perhaps because it was a higher-budget endeavor than a weekly TV show. But, I still enjoyed the movie and it didn’t affect my opinion of it overall.

Then the Others

And then there are the ones of whom and of what I spoke in the second paragraph. They make factual mistakes, too. Like Mike Bara talking about how Mars’ orbit is elliptical because of its large distance difference from Earth, or that the surface of Earth is darker than clouds because light takes more time to reach it than clouds when the camera is in space (and oceans are darkest “because the light has to travel all the way to the ocean floor before it is reflected back to the camera.”

But, they try to sell that “science” as reality, and that’s all they’re selling. Sitchen was not creating an alternate world with an alien race that created humans and lived on a planet that swings near Earth every 3600 years and trying to make money with sci-fi. He really thought that is true.

In-so doing, and in perpetuating their own mythologies as real, they in fact do harm. I’ve often stated in my podcast and blog that bad astronomy is much less harmful than things like bad medicine where people really die because they take a homeopathic pill rather than get chemo. Very rare for someone to die because of astronomy pseudoscience.

But, astronomy pseudoscience is where it can start. Someone listens to James McCanney and electric universe stuff and thinks, “Well that’s weird, I’ve never heard about this before from ‘establishment’ scientists, but this guy has degrees, he has a platform, maybe there’s more to this.”

Bad science in any form is like a gateway drug: If you’re credulous about one thing and you don’t go through the critical thinking necessary to understand why it’s wrong, it opens you up to being taken advantage of by pseudoscience that can do a lot more physical harm.

Final Thoughts

I think that’s why I give science fiction shows and movies a free pass when they get the science wrong (in most cases), but I don’t give people like Richard Hoagland a pass: It’s all about intent.

Stargate is meant to entertain and they usually try to get the science right. Richard Hoagland, on the other hand, does not. He tries to sell you books, sell his appearance on TV shows and conferences, and various other ways of making money on perpetuating a misunderstanding of how science is done and the conclusions from its process.

And I think this is a good post to leave you with as I get ready for TAM 2013!

Is the Scientific Method a Part of Science?

Introduction

You probably all remember it, and I can almost guarantee that you were all taught it if you went through any sort of standard American education system (with full recognition for my non-USAian readers). It’s called the Scientific Method.

That thing where you start with a question, form a hypothesis, do an experiment, see if it supports or refutes your hypothesis, iterate, etc. This thing:

Flow Chart showing the Scientific Method

The question is, does anyone outside of Middle and High School science class actually use it?

A Science Fair Question

I recently judged a middle and high school science fair here in Boulder, CO (USA). The difference in what you see between the two, at least at this science fair, is dramatic: High schoolers are doing undergraduate-level (college) work and often-times novel research while middle schoolers are doing things like, “Does recycled paper hold more weight than non-recycled?” High schoolers are presenting their work on colorful posters with data and graphs and ongoing research questions, while middle schoolers have a board labeled with “Hypothesis,” “Method,” “Data,” and “Conclusions.”

I was asked by a member of the public, after I had finished judging, why that was. He wanted to know why the high school students seemed to have forsaken the entire process and methodology of science, not having those steps clearly laid out.

My answer at the time – very spur-of-the-moment because he was stuttering and I had to catch a bus – was that it IS there in the high school work, but it was more implicit than explicit. That often in research, we have an idea of something and then go about gathering data for it and see what happens: It’s more of an exploration into what the data may show rather than setting out on some narrow path.

That was about a month ago, and I haven’t thought much more about it. But, the Wired article today made me think this would be a good topic for a blog post where I could wax philosophical a bit and see where my own thoughts lay.

Field-Specific?

A disclaimer up-front (in-middle?) is that I’m an astronomer (planetary geophysicist?). This might be field-specific. The Wired article even mentions astronomy in its list of obvious cases where the Scientific Method is usually not used:

Look at just about any astronomy “experiment”. Most of the cool things in astronomy are also discovered and then a model is created. So, the question comes second. How do you do a traditional experiment on star formation? I guess you could start with some hydrogen and let it go – right? Well, that might take a while.

That said, I’m sure that other fields have the same issues, and it’s really just a big grey area. What I’m going to talk about, that is. Some fields may be more towards one end of the greyscale than the other.

A Recent Paper I Co-Authored

I recently was a co-author on a paper entitled, “ Distribution of Early, Middle, and Late Noachian cratered surfaces in the Martian highlands: Implications for resurfacing events and processes.” The paper was probably the only professional paper I have ever been an author on that explicitly laid out Hypotheses, tests for those hypotheses, what the conclusion would be depending on the results, then the Data, then the Conclusions. And it was a really good way to write THAT paper. But not necessarily other papers.

A Recent Paper I Wrote

I had a paper that was recently accepted (too recently to supply a link). The paper was about estimating and modeling the ages of the largest craters on Mars. There was an Introduction, Methods, Data, and Conclusions. There was no Hypothesis. It was effectively a, “Here is something we can explore with this database, let’s do it and put these numbers out there and then OTHER people may be able to do something with those numbers (or we can) in future work.” There really was no hypothesis to investigate. Trying to make one up to suit the Scientific Method would have been contrived.

This is also something the Wired article mentions:

… often the results of a scientific study are often presented in the format of the scientific method (even though it might not have been carried out in that way). This makes it seem like just about all research in science follows the scientific method.

This is especially the case in medical journals, but not necessarily elsewhere.

Change the “Scientific Method?”

The Wired article offers this as the “new” method:

New Scientific Method (via Wired)

Here’s the accompanying justification:

There are a lot of key elements, but I think I could boil it down to this: make models of stuff. Really, that is what we do in science. We try to make equations or conceptual ideas or computer programs that can agree with real life and predict future events in real life. That is science.

I will preface this next part by saying I am NOT up-to-date on the latest pedagogy of teaching and I am NOT trained in teaching methods (other than 50+ hours of Graduate Teacher Program certification during grad school plus teaching several classes, including two as instructor of record).

That in mind, I think that this is a good idea in later years of grade school education. In the early years, I think that the methodology of the Scientific Method helps get across the basic idea and concepts of how science works, while later on you can get to how it practically works.

Let me explain with an example: In third grade, I was taught about the planets in the solar system plus the sun, plus there are asteroids, plus there are random comets. In eighth grade, I was taught a bit more astronomy and the solar system was a bit messier, but still we had those nine planets (this was pre-2006) and the sun and comets and asteroids plus moons and rings.

Then you get into undergrad and grad school, and you learn about streaming particles coming from the sun, that we can be thought of as being in the sun’s outer-most atmosphere. You get taught about magnetic fields and plasmas. Zodiacal light. The Kuiper Belt, Oort Cloud, asteroid resonances, water is everywhere and not just on Earth, and all sorts of other complications that get into how things really work.

To me, that’s how I think the scientific method should be taught. You start with the rigid formality early on, and I think that’s important because at that level you are really duplicating things that are already well known (e.g. Hypothesis: A ping pong ball will fall at the same rate as a bowling ball) and you can follow that straight-forward methodology of designing an experiment, collecting data, and confirming or rejecting the hypothesis. Let’s put it bluntly: You don’t do cutting-edge science in middle school.

In high school — in a high school with good science education — you actually do start to learn more about the details of different ideas and concepts and solid answers are no longer necessarily known. You want to find out, so you might design an experiment after seeing something weird, and then gather data to try to figure out what’s going on.

That’s how science usually works in the real world, and I think it’s a natural progression from the basic process, and I still think that basic process is implicit, if not explicit, in how science is usually done.

I just got back from a major science conference two weeks ago, and I sat through several dozen talks and viewed several hundred poster presentations. I honestly can’t remember a single one that was designed like a middle school science fair with those key steps from the Scientific Method.

Of course, another aspect is that if we get rid of it, we can’t make comics like this that show how it’s “really” done (sorry, I forget where I found this):

How the Scientific Method Really Works
(click to embiggen)

Final Thoughts

That said, this has been a ~1400-word essay on what I think about this subject. I don’t expect much to change in the near future, especially since – as the Wired article points out – this is firmly entrenched in the textbooks and in Middle School Science Fair How-To guides.

But, I’m curious as to what you think. Do you think the Scientific Method is useful, useless, or somewhere in-between? Do you think it should be taught and/or used in schools? Do you think it should be used in science fairs? Do you think professional scientists should use it more explicitly more often?

Why I Do What I Do

Introduction

First post back from the Lunar and Planetary Science Conference, probably the second-largest annual gathering of planetary scientists in the world, and largest of those with a non-Earth focus (December AGU being more terrestrial geology). Whilst I was away, The Star Spot podcast posted an interview I did with them a month or so ago. It focuses mainly on different ideas about Planet X, about which I’ve both written and podcasted extensively.

At the end of the interview, I was asked, effectively, why I do what I do. I admit I hadn’t had much sleep before the interview, and I didn’t exactly have my A-game on. And so I may have come off as being somewhat more self-centered than normal. I have been asked this a few times before, like last year when going back-and forth with Mike Bara about that whole lunar ziggurat thing.

So, here’s the self-reflective but hopefully not as self-centered post. And the announcement of me being interviewed.

Being a Better Scientist

Let’s get this one out of the way because it’s what I mostly answered with when interviewed. One of the things needed to be a good scientist is the ability to ask good questions (let’s not get started on the, “There’s no such thing as a bad question!” because there really are). You have to be able to ask those questions and then investigate them. You have to have a high threshold for evidence. In my opinion, a good scientist needs to set a high threshold for the acceptance of new conclusions and needs to think about what may be mitigating factors.

What I mean by this is that you have to be skeptical. At least one commentator to my blog likes to claim that being skeptical is the antithesis of being a good scientist. That particular person couldn’t be more wrong. While Mike Bara has definitely flung more mud at me, the harshest substantive critiques of what I’ve written have always come from reviewers of papers I’ve written.

That’s what we do: When we sit down to review a paper that describes someone’s data and conclusions, we question everything. Does their data make sense in light of what’s been done before? Do they reference what’s been done before? Does their data description match their diagrams? Does the way in which the data were gathered make sense? Are their conclusions supported by the data? Are they reaching in their conclusions beyond what they have evidence for?

And those are just the big-picture questions. Most reviewers will also bluntly tell you that your grammar is bad, that the paper is poorly written, the figures are illegible, and so-on. I once had a reviewer say that my use of a three-word term once in a 10,000-word paper made everyone in the field look stupid.

This bit of a digression gets back to my main point: Scientists are skeptical, whether they self-identify with that term or not. If you cannot learn how to support your conclusions, if you can’t think of holes others might poke in your arguments and pre-emptively fill those holes, and if you can’t deal with people picking apart your work, you’re not going to make it in science.

Every little claim that I look into, every argument by a young-Earth creationist or UFOlogist that I pick apart, helps me hone my own skills in sorting through evidence and figuring out how to back up my own claims better.

Public Outreach

Yes, to you the public, who are not scientists, it is important to convey good science and to NOT convey and anti-convey (is that a term?) bad science. Not just for the broader utopian goals of a more intellectual society that’s better informed, but let’s face it: It also comes down to money. Pretty much all astronomy-related science is supported by government grants. I should not have to compete with someone like Richard Hoagland for a grant to do research when his stuff is clearly pseudoscience. But, to someone who is uninformed and who doesn’t know the tools and methods and background of how science is done and what he’s claiming, Hoagland’s nonsense may seem just as valid as what I do.

Case in point is that the National Institutes of Health have their “Complimentary and Alternative Medicine” division/institute/thing that actually DOES dole out money for studies into things that have been shown by the normal rules of evidence to not help treat nor cure anything. Real doctors and medical researchers have to compete against chiropractors and homeopathists for a dwindling pool of federal funds. And that’s sad.

I hope that by doing what I do, I can help people realize what science is, what good science is, and how to tell it from bad science.

Applicability to Every-Day Life: Critical Thinking

What this really teaches is critical thinking. Let’s say that you didn’t believe me that Planet X wasn’t going to cause a pole shift on December 21, 2012. I went through numerous posts on it and I got many people writing in the comments that we were all going to die. It’s late March 2013, so clearly they were wrong.

But, clearly they at least read some of what I wrote. It’s not always the conclusion that matters. But, what always matters is the process. The process that I try to go through in my blog and podcast when dissecting claims really boils down to critical thinking. No, not thinking critically (as in badly) about something, but thinking about it in detail and analyzing it in all ways possible.

That method of going through a claim in agonizing detail, showing what it would have to be in order to be correct, showing what it would mean for completely unrelated fields and applications (like, if magnetic therapy bracelets worked, you would explode when you go into an MRI), is – more than most other things – what I hope people get from the work I do here.

You probably aren’t going to come up against someone who’s going to make you decide between whether Billy Meier’s dinosaur photos are of real dinosaurs or of a childrens’ book and depending on your answer you stand to lose $1M or something like that. But, let’s say you’re going to invest money in a high-risk venture. You’ll be thrown a bunch of marketing hype. If you have the critical thinking tools and know where to look for the background knowledge, you could save yourself from quite a bit of financial loss. Perpetual motion scams companies do this all the time, trying to bilk rich people who don’t know any better out of their ¢a$h.

Final Thoughts

Skepticism, to me, is a process. It’s not a conclusion, it’s starting point and a process. I use it in my every-day work, and the more I practice it, the better (hopefully) I get.

I also happen to be in a position where I know more than the average person about a narrow topic range. My hope is that by showing where people go wrong in their thinking, I can help others avoid mistakes. People often learn better by understanding how they got the wrong answer than being told the right answer. That’s the goal here: Understanding the critical thinking process to be better equipped to deal with things that might not be so obvious in the future.

Skeptiko Host Alex Tsakiris on Monster Talk / Skepticality, and More on How to Spot Pseudoscience

Introduction

A few weeks ago, I learned that the popular Monster Talk podcast would be interviewing Skeptiko podcast host, Alex Tsakiris. They ended up later posting it instead on their Skepticality podcast feed, and the interview also was episode 153 of Skeptiko; it came out about two weeks ago. The interviewers from Monster Talk are Blake Smith, Ben Radford, and Karen Stollznow (the last of whom I have the pleasure of knowing). Got all that?

If the name Alex Tsakiris sounds familiar but you can’t quite place it and you’re a reader of this blog, you probably recognize it from the two previous posts I’ve written about him on this blog. The first was on the purpose of peer-review in science because Alex (among others) were talking about how peer-review was a flawed process and also that you should release results early without having a study completed.

Fourteen months later, I wrote another post on Alex, this one being rather lengthy: “Skeptiko Host Alex Tsakiris: On the Non-Scientifically Trained Trying to Do/Understand Science.” The post garnered a lot of comments (and I’ll point out that Alex posted in the comments and then never followed-up with me when he said he would … something he accuses skeptics of not doing), and I think it’s one of my best posts, or at least in the top 10% of the ~200 I’ve written so far.

This post should be shorter than that 2554-word one*, despite me being already in the fourth paragraph and still in the Introduction. This post is further commenting on not the actual substance of Alex Tsakiris’ claims, but rather on the style and format and what those reveal about fundamental differences between real scientists and pseudoscientists. I’m going to number the sections with the points I want to make. Note that all timestamps below refer to the Skeptiko version.

*After writing it, it’s come out to 3437 words. So much for the idea it’d be shorter.

Point 1: Establishing a Phenomenon Before Studying It

About 8 minutes into the episode, Karen talks with Alex about psychics, and Alex responds, “If you’re just going to go out and say, as a skeptic, ‘I’m just interested in going and debunking a psychic at a skeptic [sic] fair,’ I’m gonna say, ‘Okay, but is that really what you’re all about?’ Don’t you want to know the underlying scientific question?”

Alex raises an interesting point that, at first glance, seems to make perfect sense. Why belittle and debunk the crazies out there when you could spend your valuable time instead investigating the real phenomenon going on?

The problem with this statement – and with psi in general – is that it is not an established phenomenon that actually happens. Psi is still in the phase where it has yet to be conclusively shown to exist under strictly controlled situations, and it has yet to be shown to be reliable in its predictions/tenants. By this, I mean that psi has yet to be shown to be repeatable by many independent labs and shown to be statistically robust in its findings. I would note the obvious that if it had been shown to be any of these, then it would no longer be psi/alternative, it would be mainstream.

Hence, what the vast majority of skeptics are doing is going out and looking at the very basic question of does the phenomenon exist in the first place? If it were shown to exist, then we should spend our time studying it. Until then, no, we should not waste time trying to figure out how it happens. This really applies to pretty much everything, including UFO cases. In that situation, one has to establish the validity by exploring the claims before one looks at the implications, just like with alleged psychics.

A really simple if contrived example is the following: Say I want to study life on Io, a moon of Jupiter. I propose a $750 million mission that will study the life there with cameras, voice recording, chemical sensors, the works. I would propose to hire linguists to try to figure out what the beings on Io are saying to the probe, and I’d propose to hire biologists to study how they could survive on such a volcanic world. NASA rejects my proposal. Why? Because no one’s shown that life actually exists there yet, so why should they spend the time and money to study something they don’t know is actually there? And, not only that, but Io is so close to Jupiter that it’s bathed in a huge amount of radiation, and it is so volcanically active that it completely resurfaces itself every 50 years, making even the likelihood of life existing there very slim.

Point 2: Appeal to Quantum Mechanics

I’ll admit, I have a visceral reaction whenever I hear a lay person bring up quantum mechanics as evidence for any phenomenon not specifically related to very precisely defined physics. At about 12.5 minutes into the episode, Alex states quite adamantly that materialism (the idea that everything can be explained through material things as opposed to an etherial consciousness being needed) “is undermined by a whole bunch of science starting with quantum mechanics back a hundred years ago … .”

It’s really simply basically practically and all other -ly things untrue. Alex does not understand quantum mechanics. Almost no lay person understands quantum mechanics. The vast majority of scientists don’t understand quantum mechanics. Most physicists don’t understand quantum mechanics, but at least they know to what things quantum mechanics applies. Alex (or anyone) making a broad, sweeping claim such as he did is revealing more their ignorance of science than anything else.

Unless I’m mistaken and he has a degree in physics and would like to show me the math that shows how quantum mechanics proves materialism is wrong. Alex, if you read this, I’d be more than happy to look at your math.

You will need to show where quantum mechanics shows that consciousness – human thoughts – affect mater at the macroscopic level. Or, if you would like to redefine your terms of “consciousness” and “materialism,” then I will reevaluate this statement.

(For more on quantum mechanics and pseudoscience, I recommend reading my post, “Please, Don’t Appeal to Quantum Mechanics to Propagate Your Pseudoscience.”)

Point 3: Appeal to Individual Researchers’ Results Is a Fallacy

A habit of Alex is to relate the results of individual researchers who found the same psi phenomenon many different times in many different locations (as he does just after talking about quantum mechanics, or about 45 minutes into the episode where they all discuss this, or throughout the psychic detective stuff such as at 1:30:30 into the episode). Since I’ve talked about it at length before, I won’t here. Succinctly, this is an argument from authority, plain and simple. What an individual finds is meaningless as far as general scientific acceptance goes. Independent people must be able to replicate the results for it to be established as a phenomenon. The half dozen people that Alex constantly points to does not trump the hundreds of people who have found null results and the vast amount of theory that says it can’t happen (for more on that, see Point 6).

For more on this, I recommend reading my post on “Logical Fallacies: Argument from Authority versus the Scientific Consensus” where I think I talk about this issue quite eloquently.

It’s also relevant here to point out that a researcher may have completely 100% valid and real data, but that two different people could reach very different conclusions. Effectively, the point here, which is quite subtle, is that conclusions are not data. This comes up quite dramatically in this episode about 22.3 minutes in when discussing the “dogs that know” experiment; in fact, my very point is emphasized by Ben Radford at 24 min 05 sec into the episode. For more on this sub-point, I recommend reading my post from last year‘s Point 1.

Point 4: Investigations Relying on Specific Eyewitness Memories Decades After the Fact = Bad

The discussion here starts about 36 minutes into the episode, stops, and resumes briefly about 50 minutes in, and then they go fully into it at 1 hour 13 minutes in*. For background, there is a long history of Alex looking into alleged psychic detectives, and at one point he was interviewing Ben Radford and they agreed to jointly investigate Alex’s best case of this kind of work and then to hash out their findings on his show. This goes back to 2008 (episode 50), but it really came to a head with episode 69 in mid-2009 where they discussed their findings.

Probably not surprisingly, Alex and Ben disagreed on the findings and what the implications were for psychic detectives (Nancy Weber in this case). If you are genuinely interested in this material, I recommend listening to the episodes because there is much more detail in there than I care to discuss in this quickly lengthening post. The basic problems, though, were really two-fold — Ben and Alex were relying on police detectives remembering specific phrases used by the alleged psychic from a case almost 30 years old (from 1982), and they disagreed on what level of detail counted as a “hit” or “miss.”

For example, when Ben talked with the detectives, they had said the psychic told them the guy was “Eastern European” whereas they had separately told Alex that she had told them the guy was “Polish.” Alex counted it as a hit, Ben a miss. I count it as a “who knows?” Another specific one they talk about in this interview is “The South” versus “Florida” with the same different conclusions from each.

To these points, both scientists and skeptics (and hopefully all scientists are appropriately skeptical, as well) I think can learn a lot when looking into this type of material.

First, I personally think that this was a foolish endeavor from the get-go to do with an old case. Effectively every disagreement Ben and Alex had was over the specific phrasing which, unless every single thing the alleged psychic says is recorded, you are never going to know for sure what she said. Human memory simply is not that reliable. That is a known fact and has been for many years (sources 1 and 2, just to name a couple). Ergo, I think the only proper way to investigate this kind of phenomenon where you have disagreements between skeptics and other people is to wait for a new case and then document every single part of it.

Second, one needs to determine a priori what will count as a hit or miss (“hit” being a correct prediction, “miss” being wrong). In the above example, if they had agreed early on that Nancy Weber only needed to get the region of the planet correct, then it would be a hit. If she needed to get the country (first example) or state (second example) correct, it would be a miss under what the detectives told Ben. This latter point is the one that is more relevant in scientific endeavors, as well. Usually this is accomplished through detailed statistics in objective tests, but in qualitative analyses (more relevant in things like psychiatric studies), you have to decide before you give the test what kinds of answers will be counted as what, and then you have to stick with that.

It should be noted that hits vs. misses was not the actual crux of the disagreement, however. It was the level of specificity the psychic claimed (“Polish”) versus what the detectives told Ben they remembered (“Eastern European”), and then the broader picture to how well that information will help solve a case.

I actually encounter the same thing when grading essays. This is one reason why teachers in science classes like multiple-choice questions more than essays (besides the time it takes to grade): It’s much more quantitative to know the answer is (A) as opposed to parsing through an essay looking for a general understanding of the question being asked.

*I’ll warn you that this goes on for about a half hour and it’s somewhat difficult to listen to with all the shouting going on. If you’re scientifically/skeptically minded, listening to this is going to make you want to smack Alex. If you’re psi/alternative minded, listening to this is going to make you want to smack Ben. This is why I try not to get into the specifics of the exact case but rather point out the process and where the process is going wrong here.

Point 5: Confusing Different Causes for a Single Effect

About 41 minutes into the episode and then for several minutes on, the conversation turned to the idea that psychics help with the grieving process. The reaction from me (and then the hosts) was pretty much, “Duh!” As Blake points out just before the 43 minute mark, “How many times did the [psychic] say, ‘Oh gee! That person’s in Hell!'” Thus, probably, not helping the grieving process.

The conversation steered along the lines of the three hosts of Monster Talk trying to point out that yes, the effect of the alleged psychic talking with the grieving person is that the grieving person felt better. But was the cause (a) because the person was actually psychic, or (b) because the person was telling the grieving people what they wanted to hear that their loved one was happy and still with them and they would join them when they died?

Alex obviously is of the former opinion (after pulling out yet another argument from authority that I talked about in Point 3 above). The others are of the latter. But the point I want to pull from this is something that all scientists must take into account: If they see an effect, there could be causes other than or in addition to their own preferred explanation. That’s really what this case that they talk about boils down to.

For example, we want to know how the moon formed. There are many different hypotheses out there including it formed with Earth, it was flung off Earth, it was captured, it was burped out, or a Mars-sized orbit crashed into Earth and threw off material that coalesced into the moon. I may “believe” in the first. Another person may in the last. We both see the same effect (the moon exists and has various properties), but how we got that effect probably only had one cause. Which one is more likely is the question.

Point 6: It’s Up to the Claimant to Provide the Evidence

I know I’ve discussed this before, but I can’t seem to find the post. Anyway, this came up just before the 52 minute mark in the episode, that Alex frequently states it’s up to the debunkers to debunk something, not for the claimant to prove it. (To be fair, in this particular interview, Alex kinda says he never said that at first, he only says it when it’s a paradigm shift kinda thing that’s already shifted … which it so has not in this case. But then he does say it …)

Blake: I think most skeptical people believe that whenever you’re making a claim that you have the burden of proof every time. And it never shifts …

Alex:… And they’re wrong because that’s not how science works. Science works by continually asking hard, tough questions and then trying to resolve those the best you can.

I’m really not sure where Alex gets this first sentence (the second sentence is correct, but it and the first are not mutually exclusive). It’s simply wrong. In no field is this a valid approach except possibly psi from Alex’s point of view. If you make a claim, you have to support it with evidence that will convince people. If I say I can fly, it shouldn’t be up to you to prove I can’t, it should be up to me to prove I can. It’s that simple. And Alex gets this wrong time after time.

This is further evidence (see Point 2 above) that Alex has no actual concept of science and how it works. And before you accuse me of ad hominems, I’m stating this in an objective way from the data — his own statements that have not been quote-mined (go listen to the episodes yourself if you don’t believe me).

But it continues:

Ben: So who does have the burden of proof?

Alex: Everybody has the burden of proof and that’s why we have scientific peer-reviewed journals, the hurdles out there that you have to overcome to establish what’cha know and prove it in the best way you can. It gets back to a topic we kinda beat to death on Skeptiko and that’s this idea that also hear from skeptic [sic], ‘Extraordinary claims require extraordinary proof.’ Well of course that’s complete nonsense when you really break it down because scientifically the whole reason we have science is to overcome these biases and prejudices that we know we have. So you can’t start by saying ‘Well, I know what’s extraordinary in terms of a claim, and I know what would be extraordinary in terms of a proof,’ well that’s counter to the idea of science. The idea of science is it’s a level playing field, everybody has to rise above it by doing good work and by publishing good data.

(Ben Radford corrects Alex on this point about 54.7 minutes into the episode; feel free to listen, but also know that the points he makes are not the ones I do below. Well, maybe a bit around 56 minutes.)

I know I’ve talked about this before, but not in these exact terms. What Alex is talking about – and getting wrong – without actually realizing it is how a hypothesis becomes a theory and the lengths one has to go to to overturn a theory. That’s what this nugget boils down to.

If you’re not familiar with the basic terminology of what a scientist means by a fact, hypothesis, theory, and law, I recommend reading one of my most popular posts that goes into this. The issue at hand is that it is effectively established theory that, say, people cannot psychically communicate with each other (yes, I know science can’t prove a negative and there’s no Theory of Anti-Psi, but go with me on this; it’s why I said “effectively”). Even if it’s not an exact theory, there are others that are supported by all the evidence that show this isn’t possible nor plausible.

Ergo, to overturn all those theories that together indicate psi can’t happen, you have to have enough convincing and unambiguous data to (a) establish your phenomenon and (b) explain ALL the other data that had backed up the previous theories and been interpreted to show psi is not real.

This is summarized as, “Extraordinary claims require extraordinary evidence.” That’s the phrase, not “proof,” which in itself shows yet again that Alex misses some fundamental tenants of science: You can never prove anything 100% in science, you can only continue to gather evidence to support it. “Proof” does not exist, just like “truth,” as far as science is concerned.

Final Thoughts

Well, this post ended up longer than I had initially planned, and it took several hours not the least of which is because I listened to the episode twice and it’s almost two hours long. I hope that through this I’ve been able to illustrate several points that you and everyone needs to watch out for when evaluating claims.

To quickly recap:

1. You need to establish that a phenomenon exists before studying it.
2. Don’t appeal to quantum mechanics unless you actually know what quantum mechanics is.
3. A single or small group of researchers’ results are not convincing, no matter who they are.
4. If you want to study something that supposedly happens every day, don’t choose an example that’s 30 years old.
5. A single effect can have multiple or different causes, including one that you don’t like.
6. The person making the claim has the burden of evidence … always.

In the end, I’ll admit that this was personally hard to listen to in parts. I took issue with Alex constantly refusing to admit certain things like the detectives saying one thing to him and another to Ben and saying Ben was lying about it and that he should say (what he didn’t say) to the detectives’ faces. That was just hard to listen to. Or Alex’s refusal to directly answer some questions in ways that would have made a politician proud. Another point that was hard to listen to but oh so sweet in the end was Alex claiming that Karen had invited him on but Karen said that Alex had invited himself on. Alex insisted that wasn’t true and said Karen was wrong and he had the transcript … and then a few seconds later the transcript was read and Alex clearly had invited himself onto their show.

But, those are my personal and more emotional observations after listening to this. Do those change what we can learn about the scientific process and where pseudoscientists go wrong? No. Alex Tsakiris continues to unwittingly provide excellent examples of how not to do science.

A Follow-Up on How Science Works versus Creationism

Introduction

This is a short follow-up to my last post, “Mistakes in Science Apparently Means Creationism Is True.” In that fine piece of blogging, I talked some about how science is a process where we continually revise our knowledge based upon new observations and discoveries. Contrasted with creationism.

It was therefore apropos that I ran across this article on Ars Technica, “ How a collapsing scientific hypothesis led to a lawsuit and arrest .”

Article

The article in question was written by John Timmer, a faculty at Cornell Medical College. He got his Ph.D. in Molecular and Cell Biology (like my dad!) from University of California, Berkeley (not like my dad). So I’d say he’s reasonably qualified – while avoiding an argument from authority – to write about this topic.

In his article, Dr. Timmer tells the story of a small group of researchers thought they found a retrovirus associated with prostate cancer, and they later even linked it to people with chronic fatigue syndrome. I’m about as qualified to talk about medicine as any other lay person (so not much), but I can gather that this would be pretty darn important. A retroviral link means (a) a good test to see in who this may develop, and (b) a possible cure if we could get rid of that retrovirus. Their work was published in one of the two leading journals in the world, Science.

Then problems developed. I don’t want to take too much away from Dr. Timmer’s article, which I highly recommend reading. But, suffice to say, other people investigated these claims and tried to verify them. Nothing less than the country’s blood supply was actually at stake if their findings bore out. Problem was that no one could replicate them. And the main researcher’s (Judy Mikovits) co-authors started to walk away. Mikovits didn’t, ended up being fired for insubordination when refusing to share cell cultures as required, and then arrested for stealing her lab notebooks and other things.

So, as the title sums, a collapsing hypothesis led to a lawsuit and arrest, but also a good moment to illustrate how science works, especially in contrast with creationism.

How Science Works

Readers of my blog will recognize that I’ve said this before, but it’s important to get across. So I’ll try to shorten it this time. The scientific process requires duplication of findings. It requires testing of claims. It requires questioning and critiquing others’ results. It requires revision.

All of these requirements are how and why the process of science is incremental and self-correcting. Mikovits’ work made it into one of the most prestigious scientific journals in the world. That does not mean everyone believed it nor that it meant it was “true.” Less than five years later, the paper has been retracted and the researcher has been pretty much disgraced in the scientific community and is facing significant legal issues due to misconduct and theft. The study was shown to be wrong. The scientific process is to thank for showing that.

(And now the obligatory “in contrast, creationism … .”) In contrast, creationism generally requires putting your fingers in your ears and shouting, “La-la-la, I can’t hear you!” when something contradicts their favored position. Or, they will accept the latest study whole-heartedly if it fits their paradigm, but not admit it was wrong if later retracted or shown to be wrong or misunderstood. I pointed to Earth’s magnetic field strength last time, this time I’ll choose comets and simply link to my blog (here, or here), or podcast (here).

Final Thoughts

Perhaps the worst part of the story in question is that a whole new subset of medical pseudoscience has cropped up because of Mikovits’ work. Before she came along, no people suffering from these really thought there was a retrovirus cause. Now some do, and “alternative” practitioners offer to test them for the non-existent retrovirus or offer antiretroviral agents as “cures.” Even though it’s now come out that the original study was simply wrong. But, unfortunately, that doesn’t change things once the idea is out there.

Mistakes in Science Apparently Means Creationism Is True

Introduction

I realize I haven’t posted in a few weeks other than for my podcast. Busy, blah blah blah. Anyway …

The post today centers around yet another Institute for Creation Research (ICR) article, this one by Christine Dao, their assistant editor. I’m not quite sure for what she is the assistant editor, but that’s her title. The article is, “Mistakes and Misconduct in Science.” Peter over at “Eye on the ICR” has already talked about some of the details of this latest piece of young-Earth creationism (YEC) propaganda and I suggest you read it if you’re interested in a detailed picking apart of the article. (He gets up crazy-early to write about these things. Or maybe it’s just ’cause he’s on the other side of the planet from me.)

The reason I’m doing a post on it is that I want to talk more about the process of how science works, how it is conducted, the self-correcting nature, and how this is the opposite of creationism.

Crux of the Nature Report

Ms. Dao’s basic premise is a recent Nature report entitled, “Science Publishing: The Trouble with Retractions” (unlike most articles, this one seems free to the public to read, so go ahead and follow the link to read it).

The report (the point of which the ICR article misses) really talks mostly about attitudes among journals and journal editors with regards to retractions and the reasons for retractions. It also mentioned some numbers — mainly that the number of retractions these days is around 0.02% of all papers published. And that in one week alone, around 27,000 papers are published. That means, if we multiply by about 52, over 1.4 million papers are published in scientific journals each year, and 2,800 of those will eventually be retracted.

Fascinating.

Crux of the ICR Article

Ms. Dao seems to make a really big issue of this tiny number. She blows it way out of proportion – as I’ve shown YECs are wont to do – and somehow says that because there are a tiny number of retractions, with a small percentage of those due to fraud, this means that evolution is wrong. I’m serious. Go read her last two paragraphs if you don’t believe me.

Scientific Process

Something that Ms. Dao either does not know or willfully ignores is that scientists know this. We know that not 100% – or 99.98% – of papers published in journals are going to bear the test of time and further research. No scientist thinks that. In my undergraduate education, we would have a weekly seminar in the astronomy department where we read a paper and discussed it and decided whether or not we thought the results were valid.

This is something called critical thinking where you analyze things and decide whether or not they make sense in light of all the other observable evidence. Probably the reason Ms. Dao chooses to ignore this is because YEC does not hold up to critical thinking, but more on that in the next section.

In fact, far from 100% of published papers being “right,” various studies have shown that at least 30-40% of them will later be falsified based upon new data, observations, experiments, and theory. This is not a secret. It’s how science works. It is self-correcting over time because everything is subject to further testing and independent analysis.

Creationism Process

Goddidit.

Creationism Process, Expanded

I’ve used this diagram before in a lengthy post on the scientific method, but it bears repeating here:

Flow Chart Showing Faith-Based 'Science'

The flow chart shows the basic process that most biblical literalists use to vet new information. They may get an idea, or hear of something. Let’s use a young-Earth creationist mainstay, Earth’s magnetic field (previous blog on this, podcast on this). Data shows that Earth’s field has gone through reversals in polarity at many points in the past. The data is clearly out there for anyone to examine, and it is unambiguous that crustal rocks record a flip-flopping magnetic field.

Now, does it fit in the Bible? Creationists such as Kent Hovind say that it does not. The result is that alternating magnetic fields are simply not possible. To quote him: “That’s simply baloney [that there are magnetic reversals in the rocks]. There are no ‘reversed polarity areas’ unless it’s where rocks flipped over when the fountains of the deep broke open. … This is a lie talking about magnetic ‘reversals.'” (Taken from his Creation Science Evangelism series, DVD 6:1.)

Alternatively, Russell Humphreys, of Answers in Genesis, accepts that there have been magnetic reversals, as he is able to fit it into a reading of the Bible. He explains the field reversals as rapidly taking place during the 40 24-hr days of Noah’s Flood. Hence, because they are able to fit it into the Bible, they accept it as a dogma.

Creationism Retractions?

Ms. Dao makes much ado about almost nothing in terms of retractions in scientific literature. Something she doesn’t tell you are how many retractions there have been in the creationist literature. Obviously the Bible has never been retracted, except for those books that aren’t accepted (like the recently discovered Gospel of Judas, or the Book of Ezekial). But I’m talking things like the articles the ICR, AiG, or CMI puts out that I and many others have picked to shreds over the years. Or perhaps their own journals, like the ARJ (Answers Research Journal) or the Journal of Creation. Any retractions?

The most I have ever seen is AiG’s list of “Arguments Creationists Should Avoid, which CMI copied and expanded, or when Brian Thomas of the ICR changed his article based, perhaps, on my criticisms (see the post-script in that blog post).

In fact, far from a retraction, Creation Ministries International still has its article by Andrew Snelling entitled, “Solar Neutrinos – the Critical Shortfall Still Elusive” that discusses the solar neutrino “problem.” (For information on this, see my blog post on it.) The article is still in its online index for their Journal of Creation with no indication that it was retracted. The only thing you get is in the HTML version where the editor notes that the argument was valid for the time but “that the shortfall problem seems to have been solved. Therefore creationists should no longer use this as an ‘age’ argument.” If that’s what passes for a retraction, these guys should run for Congress.

Final Thoughts

Yes, maybe half of papers published in journals now will be found later on to not be valid in the light of further study. This is true. This does not mean that evolution is not true. It means that creationists would do better to understand a topic before attempting to criticize it if they care at all about intellectual honesty. But that’s a different blog post …