Exposing PseudoAstronomy

May 31, 2010

How to Teach Science versus Pseudoscience — Follow-Up


Introduction

Two posts ago, I wrote a post soliciting advice on how to teach science versus pseudoscience to a class of college undergraduates in an astronomy for non-majors introductory course. I received several replies, though apparently some people had problems leaving comments on the blog … to the point that they tried to leave a comment and I didn’t get any sort of notification — WordPress didn’t record it at all.

Not sure what’s going on there, but if that happens here, I’ll give out my e-mail address and you can e-mail me.

What I’ve Done for the Lecture

It was interesting trying to meld disparate advice into a lecture format that seems to sorta kinda make sense. For the moment, I’ve decided to:

  1. Start with the infamous “Rainbow Lady” YouTube video of the woman thinking there’s a government conspiracy to make water coming out of her sprinkler show a rainbow. The purpose is to have the students try to frame the scenario in a scientific way. An observation is shown, how can they go about figuring out what’s going on? I have prompt questions written to myself to try to steer the conversation towards what experiments could they do, is it simpler to explain it through known phenomenon or a government conspiracy, is it possible to disprove a conspiracy, and is it possible to prove that diffraction is the explanation — answers to the last two being “no” which will lead into …
  2. What is a theory? Differentiating between every-day use and scientific use including the “this can never be proven” part.
  3. What is science? Starting out with a quote by Einstein and then outlining the scientific method.
  4. Flowchart of the scientific method. And I have decided that I will be posting my lectures on the course website as PDFs*.
  5. Finish off the “What Is Science?” part with the facts/observations < hypothesis < law < theory hierarchy.
  6. What is pseudoscience? I’ve written two slides on many common parts of pseudoscientific claims/beliefs/ideas/”theories.” End with a much shorter flow chart.
  7. End with two main types of pseudoscience that will be addressed in the course (mainly through the required writing assignment). Those would be (1) Claims that argue against science or an established concept/idea, and (2) Claims that make you go, “WTF did that come from?”

*I know this is a point of contention among many instructors. I was forever against it. I recently heard though from people who actually do do research in astronomy education that their data shows posting lecture notes does not change their class attendance. Also, since lecture will nominally be only half of any given class period, group work and class discussions comprising the rest will mean that just printing out the lectures and not attending class will not get them a good grade. Posting the lectures online will also let them not have to spend time copying down word-for-word what’s on the screen but focus more on the explanation and discussion.

Other Things I Considered

A few weeks ago I heard an amazing caller on the Coast to Coast AM radio show. In the space of 2 minutes, he talked about how Earth’s atmosphere was lost and flash-froze all the animals due to Earth losing all gravity and coupling to the moon’s off-center core and the moon retreating and …. . Needless to say, I cut that clip out and was going to use it as an example and have the class discuss it. However, I’ve now decided that I’m going to use it, but I’m going to use it as a test question later on in the course after we’ve talked about gravity, Earth, the moon, and atmospheres.

Another thing I considered was to have an example from the other infamous YouTube clip of “Dr. Werner” trying to explain how homeopathy works. If you don’t know the clip, I recommend watching about the first 6.5 minutes. It’s precious. But, I decided that even if I could cut it down to the first 3 min 40 sec, it was too far afield for the class and the history majors may feel lost to the finer points that “Steven Hawkings” didn’t come up with string theory and that mass cannot simply be crossed out of E=m*c2 to make E=c2. Oh, and the lecture already has 11 slides and with discussion that’ll probably put me at 20 minutes already.

From another suggestion, I thought I may start with a magic trick to show the importance of careful observation, how your senses can be fooled, how you think what’s going on isn’t actually going on, and the importance then of careful observation and testing. I was actually pretty into magic early on in my life and I have amassed quite a bit of tricks. The one I was going to show has to do with disappearing water into a cup. I’m not going to go further in case I actually do end up doing the trick and someone in my class finds this blog. But, I for now have decided against that because (1) I’m not sure how a group with a median age of 21 will respond, and (2) I can’t think of a good transition between it and the Rainbow Lady or another good place in the lecture to put it.

Another idea I had was to start out with an observation. Someone weighs themselves, gets a weight, takes a shower, dries off, gets a weight that’s 0.5 lbs more, and then weighs themselves a half hour later and gets the same weight as the first time. How would they go about figuring out what’s going on? I decided against that because it’s a minor thing that’s not really on-point and I think the Rainbow Lady can be better-used to accomplish the same goal.

Final Thoughts

Thanks again for all those who replied or tried to reply. I still have about 22 hours before the class, so if anyone has further advice or comments on what I’ve decided to do so far, please let me know. Post in the comments here, and then copy your comment (before submitting!) and if it doesn’t go through, send me an e-mail to the address provided above.

May 29, 2010

Skeptiko Host Alex Tsakiris: On the Non-Scientifically Trained Trying to Do/Understand Science


Preamble

First, let me give one announcement for folks who may read this blog regularly (hi Karl!). This may be my last post for about a month or so. As you may remember from my last post, I will be teaching all next month, June 1 through July 2, and the class is every day for 95 minutes. I have no idea how much free time I may have to do a blog post, and I have some other projects I need to finish up before the end of the month (I’m also a photographer and I had a bride finally get back to me about photos she wants finished).

Introduction

I have posted once before about Skeptiko podcast host Alex Tsakiris in my post about The Importance of Peer-Review in Science. The purpose of that post was to primarily show that peer review is an important part of the scientific process, a claim contrary to what the host of said podcast had claimed.

Now for the official disclaimer on this post: I do not know if Alex is a trained scientist. Based on what he has stated on his podcast, my conclusion is that he is not. What I have read of his background (something like “successful software entrepreneur” or around those lines) supports that conclusion. However, I don’t want to be called out for libel just in case and so that is my disclaimer.

Also, I am not using this post to say whether I think near-death experiences are a materialistic phenomenon or point to a mind-brain duality (mind/consciousness can exist separately from brain). That is NOT the point of this post and I am unqualified to speak with any authority on the subject (something I think Alex needs to admit more often).

Anyway, I just completed listening to the rather long Skeptiko episode #105 on near-death experiences with Skeptics’ Guide to the Universe host Steven Novella Dr. Steven Novella (see Points 2 and 3 below for that “Dr.” point). I want to use that episode to make a few points about how science is done that an (apparently) non-scientifically-trained person will miss. This post is not meant to be a dig/diss against so-called “citizen science,” rather the pitfalls of which non-scientists should be aware when trying to investigate pretty much ANY kind of science.

Point 1: Conclusions Are Not Data

Many times during the episode’s main interview and after the interview in the “follow-up,” Alex would talk about a paper’s conclusions. “The researchers said …” was a frequent refrain, or “In the paper’s conclusions …” or even “The conclusions in the Abstract …” I may be remembering incorrectly, perhaps someone may point that out, but I do not recall any case where Alex instead stated, “The data in this paper objectively show [this], therefore we can conclude [that].”

This is a subtle difference. Those of you who may not be scientifically trained (or listened to Steve’s interview on the episode) may not notice that there is an important (though subtle) difference there. The difference is that the data are what scientists use to make their conclusion. A conclusion may be wrong. It may be right. It may be partially wrong and partially right (as shown later on with more studies … more data). Hopefully, if there was not academic fraud, intellectual dishonesty, nor faulty workmanship (data gathering methods), the actual data itself will NEVER be wrong, just the conclusions from it. In almost any paper — at least in the fields with which I am familiar — the quick one-line conclusions may be what people take away and remember, but it’s the actual data that will outlive that paper and that other researchers will look at when trying to replicate, use in a graduate classroom, or argue against.

I will provide two examples here, both from my own research. The first is from a paper that I just submitted on using small, 10s to 100s meter-sized craters on Mars to determine the chronology of the last episodes of volcanism on the planet. In doing the work, there were only one or two people who had studied it previously, and so they were obviously talked about in my own paper. Many times I reached the same conclusion as they in terms of ages of some of the volcanos, but several times I did not. In those cases, I went back to their data to try to figure out where/why we disagreed. It wasn’t enough just to say, “I got an age of x, she got an age of y, we disagree.” I had to look through and figure out why, and whether we had the same data results and if so why our interpretations differed, or if our actual data differed.

The second example that’s a little better than the first is with a paper I wrote back in 2008 and was finally published in a special edition of the journal Icarus in April 2010 (one of the two main planetary science journals). The paper was on simulations I did of Saturn’s rings in an attempt to determine the minimum mass of the rings (which is not known). My conclusion is that the minimum mass is about 2x the mass inferred from the old Voyager data. That conclusion is what will be used in classrooms, I have already seen used in other peoples’ presentations, and what I say at conferences. However, people who do research on the rings have my paper open to the data sections, and I emphasize the “s” because in the paper, the data sections (plural) span about 1/2 the paper, the methods section spans about 1/3, and the conclusions are closer to 1/6. When I was doing the simulations, I worked from the data sections of previous papers. It’s the data that matters when looking at these things, NOT an individual (set of) author(s).

Finally for this point, I will acknowledge that Alex often repeats something along the lines of, “I just want to go where the data takes us.” However, saying that and then reading a paper’s conclusions are not mutually compatible. Steve pointed that out at least twice during the interview. At one point in the middle, he exclaimed (paraphrasing), “Alex, I don’t care what the authors conclude in that study! I’m looking at their data and I don’t think the data supports their conclusions.”

Point 2: Argument from Authority Is Not Scientific Consensus

In my series that I got about half-way through at the end of last year on logical fallacies, I specifically avoided doing Argument from Authority because I needed to spend more time on it versus the Scientific Consensus. I still intend to do a post on that, but until then, this is the basic run-down: Argument from Authority is the logical fallacy whereby someone effectively states, “Dr. [so-and-so], who has a Ph.D. in this and is well-credentialed and knows what they’re doing, says [this], therefore it’s true/real.”

If any of my readers have listened to Skeptiko, you are very likely familiar with this argument … Alex uses it in practically EVERY episode MULTIPLE times. He will often present someone’s argument as being from a “well-credentialed scientist” or from someone who “knows what they’re doing.” This bugs the — well, this is a PG blog so I’ll just say it bugs me to no end. ALL BECAUSE SOMEONE HAS A PH.D. DOES NOT MEAN THEY KNOW WHAT THEY’RE DOING. ALL BECAUSE SOMEONE HAS DONE RESEARCH AND/OR PUBLISHED DATA DOES NOT MEAN THEIR CONCLUSIONS ARE CORRECT OR THAT THEY GATHERED THEIR DATA CORRECTLY.

Okay, sorry for going all CAPS on you, but that really cannot be said enough. And Alex seems to simply, plainly, and obviously not understand that. It is clear if you listen to practically any episode of his podcast, especially during any of the “psychic dogs” episodes or “global consciousness” ones. It was also used several times in #105, including one where he explicitly stated that a person was well-credentialed and therefore knows what they’re doing.

Now, very briefly, a single argument from someone does not a scientific consensus make. I think that’s an obvious point, and Steve made it several times during the interview that there is no consensus on the issue and individual arguments from authority are just that — arguments from authority and you need to look at their data and methods before deciding for yourself whether you objectively agree with their conclusions.

Edited to Add: I have since written a lengthy post on the argument from authority versus scientific consensus that I highly recommend people read.

Point 3: Going to Amazon, Searching for Books, to Find Interview Guests

Okay, I’ll admit this has little to do with the scientific process on its face, but it illustrates two points. First, that Alex doesn’t seem to understand the purpose/point of scientific literature, and second that fast-tracking the literature and doing science by popular press is one of the worst ways and a way that strikes many “real” scientists as very disingenuous. I’ll explain …

First, I will again reference my post, “The Importance of Peer-Review in Science.” Fairly self-explanatory on the title, and I will now assume that you’re familiar with its arguments. In fact, I just re-read it (and I have since had my own issues fighting with a reviewer on a paper before the journal editor finally just said “enough” and took my side).

To set the stage, Alex claims in the episode:

“Again, my methodology, just so you don’t think I’m stacking the deck, is really simple. I just go to Amazon and I search for anesthesia books and I just start emailing folks until one of them responds.”

As I explained, peer-reviewed papers are picked apart by people who study the same thing as you do and are familiar with other work in the area. A book is not. A book is read by the publishing company’s editor(s) – unless it’s self-published in which case it’s not even read by someone else – and then it’s printed. There is generally absolutely zero peer-review for books, and so Alex going to Amazon.com to find someone who’s “written” on the subject of near-death experiences will not get an accurate sampling. It will get a sampling of people who believe that near-death experiences show mind-brain duality because …

Published books on a fringe “science” topic are done by the people who generally have been wholeheartedly rejected by the scientific community for their methods, their data-gathering techniques, and/or their conclusions not being supported by the data. But they continue to believe (yes, I use the word “believe” here for a reason) that their interpretations/methods/etc. are correct and hence instead of learning from the peer-review process and tightening their methods, trying to bring in other results, and looking at their data in light of everything else that’s been done, they publish a book that simply bypasses the last few steps of the scientific process.

Not to bring in politics, but from a strictly objective point, this is what George W. Bush did with the US’s “missile defense” system. Test after test failed and showed it didn’t work. Rather than going back and trying to fix things and test again, he just decided to build the thing and stop testing.

Point 4: Confusing a Class of Outcomes with a Single Cause

This was more my interpretation of what Alex did in the interview and what Steve pointed out at many times, and it is less generalizable to the scientific process, but it does apply nonetheless.

Say, in cooking, you serve up a pizza. The pizza is the “class of experiences” here that is the same as a class of things that make up the near-death experience (NDE). The toppings of your pizza are the individual experiences of the NDE. Pizzas will usually have cheese, NDEs will usually have a sense of well-being. Pizzas may more rarely have onions, NDEs may more rarely have a white light tunnel associated with them. You get the idea.

Now, from the impression I got, Alex seemed to claim throughout the episode that there was only one way to make a pizza — have an NDE. Steve argued that there were many different ways to make a pizza, and that all those different techniques will in general lead to something that looks like a pizza.

Point 5: Steve’s a Neurologist, Alex Is Not

I need to say before I explain this point that I am NOT trying to say that you need a Ph.D. in the topic to do real science. I do not in ANY WAY mean to imply that science is an elitist thing where only people “in the club” can participate.

That said, I really am amazed by Alex arguing against people who actually have studied the subject for decades. If you are a non-scientist, or even if you are a scientist but have not studied the topic at-hand (like, gee, me talking about near-death experiences while I’m an astrophysicist/geophysicist), then you need to make darn sure that you know what the heck you’re talking about. And you need to be humble enough to, when the actual person who’s studied this says you’ve made a mistake, take that very seriously and look again at what you thought was going on. The probability that you have made a mistake or misunderstood something as opposed to the expert in the field is fairly high.

Again, this is not my attempt to backtrack and myself commit an argument from authority fallacy. However, there is a difference from making an argument from authority fallaciously versus listening to what an authority on the subject says and taking it into account and re-examining your conclusions. It seriously amazes me how much Alex argued against Steve as if Alex were an expert in neurology. It caused him to simply miss many of the points and arguments Steve was making, as evidenced by Steve saying something and then needing to repeat his argument 20 minutes later because Alex had ignored it because Alex has been buoyed by his interviews with previous pro-duality guests.

Final Thoughts

As I’ve stated, the purpose of this post is not to discuss whether NDEs show a mind-brain duality or if it has a purely materialistic explanation. The purpose is to point out that the methods Alex uses are fallacious, and while I know that people have pointed it out to him before, it seems that it has made very little impact upon the way he argues. I believe this is in part due to his need for confirmation bias – he definitely has made his mind up on whether or not psi-type phenomena exists. But I also am fairly sure that it’s because Alex lacks any kind of formal training in science. Because of that, he makes these kinds of mistakes – at least originally – without knowing any better. Now, since it’s been pointed out to him, I think it’s intellectually dishonest to keep making them, but again that’s beyond the purpose of this post.

So, to wrap this all up, non-scientists take heed! Avoid making these kinds of mistakes when you try to do or to understand science yourself. Make sure that you look at the data, not just the conclusions from a paper. Don’t make arguments from authority. Remember that popular books are not the same as peer-reviewed literature. And keep in mind there can be (a) multiple explanations and (b) multiple ways to reach an end point.

May 25, 2010

How to Teach Science versus Pseudoscience?


Introduction

Some people who read this blog may have noticed another paucity of posts lately. I have the usual excuses – finishing up a paper and submitting it, being busy with research – and then unusual excuses – I went on a camping trip to Yellowstone National Park for a week, and I’m prepping to teach a class next month.

It’s the latter that is the reason for this post. I’m teaching my first class ever as Instructor of Record, meaning I have complete control over what’s taught and how. The class is a “summermester” meaning that it is every day for the entire month of June — June 1 through July 2, 11:00-12:35. The class is “General Astronomy: Solar System” for non-majors without a lab component. Checking the roster shows that, at the moment, there are about 25 students signed up, 1 freshman, 3 sophomores, about 15 juniors, and the rest seniors. Most are liberal arts students, but there are some from the sciences.

One of the assignments that I have already written is a course-long writing assignment. It requires the students to look into one of four modern popular astronomy-related pseudosciences — Planet X and 2012, The Apollo Moon Hoax, Is the Universe <6000 Years Old?, and The Hollo Earth "Theory."

Application of Critical Thinking

I will be passing out this assignment the first day of class. I will also be requiring intermediate progress items: (1) At the end of the first week (only 4 days into the class) they need to turn in an outline of their paper that lists the topic, sub-topics they’ll discuss, and at least 3 references they’ll use; (2) at the end of the 3rd week, a rough draft to me; and (3) at the end of the 4th week, a close to final draft that they’ll exchange with someone who’s NOT doing their topic, read over the weekend, and then peer-review on Monday.

As you can see, there is a reasonable emphasis on this paper. I also hope to gear the class towards an Astronomy Cast -inspired “How do we know what we know?” approach, and I plan to bring in pseudoscience topics that are related to homeworks and tests (I’m a fan of Phil Plait’s question of, “How can I state with great confidence that over 95% of violent crimes occur within 1 week of the full or new moon?”).

How to Teach?

The point of this post is to solicit advice from readers: How can I actually write an introductory lecture on this subject? I want the lecture to be no more than 20 minutes, and I would like it to teach science from pseudoscience without (a) getting preachy and (b) WITHOUT examples at the beginning. The last half of the lecture can be examples (good class discussion starters!), but I would really like to introduce the topic without falling into debunking.

I do have a wonderful 2-minute clip from a Coast to Coast AM caller who rambled about Earth’s gravity field collapsing pushing the moon away removing our atmosphere which flash-froze animals that I plan on playing towards the end.

My thoughts so far are showing the scientific method flow chart, or maybe asking the class how they would go about showing something is “true,” contrast that with methods employed by pseudoscience proponents … but those ideas are fairly vague. I start in a week; if people have advice, I’m interested in hearing (reading) it!

The Rubric Theme. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 1,326 other followers