Introduction
Last week, I judged a regional high school and middle school science fair that was held at my university. I’m trying to be fairly general in my post here and not use specific names, places, etc. in order to keep some semblance of privacy for people involved.
With that in mind, this post contains my experience, thoughts, examples, and general judging criteria, questions I asked, and things that I looked for when judging this fair.
Hopefully it will be (a) an interesting read, and/or (b) useful to future science fair judges, and/or (c) useful for students and parents in preparing for their own science fairs.
Getting Ready
First off, this science fair was for high school (grades 9-12, about ages 13-18) and middle school (grades 6-8, about ages 10-14) students, and it is a “regional” fair, effectively city-wide. It is part of a much larger one where from this, we send people on to state, national, and sometimes just fast-track to international competitions.
There are different categories, in this case the ones I remember were Physics, Earth and Space Science, Environmental, Chemistry, Health and Medicine, I think an Engineering one, and then some others that I don’t remember (’cause they weren’t near my table and I didn’t judge them). When signing up as a judge, we could request the main topic we wanted to judge and then were asked to fill in 1-3 others. I requested Earth and Space Science primarily, and Physics as my second. I also checked the box indicating I wanted to be a “head judge,” meaning that within a particular subject, I would be responsible for determining 1st, 2nd, and 3rd places, and I would also confer with the Roaming Judges about best in show and who to send on to state/national/international. Based on who the head judges were last year and that I had judged this fair before, I thought I had a decent shot.
The week before, I got an e-mail saying I’d been placed in Physics and Environmental. I was not a head judge. When I showed up the morning of, I got my packet and found I was in Earth and Space Science (ESS) in the morning along with Environmental, and ESS and Physics in the afternoon. Still not a head judge. It had been changed because they had last-minute judge cancellations.
The Day Of
I went to sit down before we started at the ESS table. I got to talking with a few people there – only one whom I’d met before – and then the head judge for our table – who studies space weather forecasting – introduced herself and told us what her bias was in judging. She liked the projects where the students really seemed to have a passion for it and followed the scientific method, as evidenced by them coming up with the project themselves, and then forming a hypothesis and working towards answering it.
Fair ‘nough. Since she had told us that, I decided I would say what mine were. I said that I really liked it when the students would form a hypothesis and then their experiment showed that they were actually wrong. And then that they decided that, on the basis of their data, their initial hypothesis was incorrect.
(I said that they get brownie points if it’s a common pseudoscience. This isn’t to say that I would judge students more harshly if they started off with a “correct” hypothesis, just that I think it is admirable when they are willing to change their beliefs on the basis of observable data – since on this blog I have illustrated many cases where people will not.)
Then I gave an example from the previous year of a middle schooler who thought that granite was radioactive and emitted poison radon gas. He found out by testing various common building rocks that he was wrong and he changed his mind about the initial hypothesis as a result. I then gave a second example of a high school student last year who thought that magnetic healing was real and that she …
That was where I was interrupted by the head judge. She said, “But magnetic healing does work.”
I looked at her and replied, “I mean stuff like the bracelets …” and that was when she held up her wrist and I saw one on her “… because blood is made with non-ferromagnetic iron.”
While she admitted that it may be a placebo effect, she claimed that she had debilitating arthritis in her wrist and that the magnets in the bracelet really helped her to function. I dropped it and moved on. But that was how my morning started. I figure if I had been in the Health and Medicine category, I would’ve spoken with the organizer at that time.
Brief Overview of the Day
In the morning session, the high schoolers are judged by up to 6 or 7 judges to get them more exposure. In the afternoon, each middle schooler is judged by up to 4 or 5.
Anyway, the next 8 hours of judging were fairly uneventful. There were some pretty good projects, and there were some really bad ones. Since I’ve seen people ask what judges actually look for and not just what we’re told to look for, I thought I’d post some specific examples of good and bad:
Example of a BAD Middle School Project
There was a middle school project where two guys wanted to see how quickly they could turn a water wheel with a hose based upon how far above the wheel the hose was as some sort of analogy for a hydroelectric dam. One of the guys was out sick, so he may have been the smrt one in the group (see what I did there?).
Besides not really being comparable to most of the projects there in terms of science nor skill, they got the “wrong” answer despite having the “right” physics on the poster and not realizing it. What I mean is that they found their wheel spun faster when the hose was closer to it. He said they had issues with the water blowing in the breeze when they held it high up which may have affected it. So I asked how it did not mimic an actual dam, and the guy thought, and then said the height of the water. I asked, “What else?” He didn’t know … I was looking for the idea that real dams aren’t bothered by the wind.
In his talk to me, he had mentioned potential energy, so I asked him what it was, trying to get him to realize the “right” answer for the overall project. He said he didn’t know, he hadn’t been paying attention in class that day. I looked pointedly at a location on his poster, then he looked, and said, “Oh, it’s right there …” he proceeded to read what they had written as to the definition of potential energy, and then just looked at me and said, “Yeah.”
(For those wondering, the higher the water starts, the more potential energy it has to convert to kinetic energy to spin the wheel.)
Example of a BAD High School Project
A bad one I judged at the high school level was a girl who was looking at minor gas components to Earth’s atmosphere and their affects on infrared (IR) absorption -> greenhouse effects. She used basically a blackbox model (she had no idea what went into it, it was a computer program handed to her) and used Earth’s standard atmosphere.
She modeled Earth and the sun as two black bodies. When I asked her to explain black bodies to me, she pointed to a part of her poster that had the equation and basically read the equation to me. I asked her what “I” was (as in the equation is “I = …”). She rambled off the standard definition of radiative energy per square unit per solid angle per blah blah blah. I asked her to tell me what it was in her own words. She didn’t know. I asked if, “Intensity” could be a synonym, and she said, “I guess.”
One of the results she found was that water vapor accounts for around 60% of IR absorption in the atmosphere. I asked her how much water vapor was in the atmosphere in her model. Since a second step was that she varied the amount from the standard atmospheric composition (making, for example, methane 500% of what it really is), she said, “100%.” I clarified: “No, I mean how much of the atmosphere in the standard model is water vapor?” She didn’t know.
I dinged her quite a bit for (a) just spouting off terms and not knowing what they were, and (b) not knowing what went into her model at all, especially in a field (climate modeling / global warming) where some of the major criticisms are what assumptions and parameters go into the models.
Example of a MEDIOCRE Middle School Project
There was a girl who looked at different soil samples around the county in a North-South then East-West line. She did a decent job, found a pattern or two, and was able to apply it to other things like saying that based on her results, if someone wanted to grow a garden she would recommend certain places and not others. Her stuff was good, but just within the scope of the project. There was nothing really brought in from outside, no bigger picture, and she got some terminology wrong, like what “topography” was. She also had told the judge just before me that she did the project in the space of a week before the Fair because her teacher suggested it to her since she had done well in their soils unit of science class.
Example of a Good Middle School Project
A good middle school project I saw was where a guy had built (supposedly himself) a water tank and made wheels out of styrofoam. The wheels all had the same outside diameter, different inside diameters, and then spokes. It was a model for turbulence for bike wheels.
He threw out terms like “Reynolds Number” and when I asked what it meant, he was able to answer it. Then I asked him if the Reynolds Number of water was larger or smaller than molasses. He got it wrong. Then I asked which had a higher viscosity. He got it right.
Moving on, after he had done his models in water, he tried out different tires with different spoke lengths on a bike — apparently he’s a “pro” biker (remember – a middle schooler) and has won some significant races. He found that the benefits from the water model were much more muted, by say 5% increases in speed due to reduced turbulence instead of the 100% he was seeing in the tank. He was able to answer why that may be. And he was able to say that even at 1-2% when you’re talking about a race where he won by 0.04 seconds over 17 minutes, then that really counts.
Things I Asked and Looked For
1. I started out after introducing myself and telling the student to assume I knew nothing about their project and subject and to start from there. In other words, I really wanted to see if they could explain it to a lay person without using the big words, and in their own words. I also ran into a problem last year where some of the high school projects were over my head (that wasn’t an issue this year); this had been my fault, really, since I should have made them explain it to me until I understood it and could tell if they understood it.
2. Whenever they used a term or concept that I thought was above grade-level, I would stop them and ask them to explain it. Sometimes they could, sometimes they couldn’t. I did count it against them if they couldn’t because it meant they were just using buzz words and didn’t know what they actually meant.
3. About half-way through each project, I would pause and repeat back to them a general summary of what they were doing to make sure that I understood it. After all, it wasn’t fair to the student if I thought they were doing something they weren’t, and I thought it helped the student because it showed I was actually paying attention.
4. I asked them what the bigger picture was at the end. How they could apply it to something else, or what it could do for people? Most of them were able to answer that, and I would hope it’s a fairly standard question. There were admittedly some cases where there was no real practical application, such as determining rotation rates for stars, but in those cases I would ask them what the application to the field would be, instead of every-day life.
5. I would always ask if there was anything else they wanted to tell me or thought I should know before I left and said, “Thank you, and good luck.” I thought that was important in case they realized at that time they had left something important out.
6. If I remembered, I asked them how they got the idea for the project and how much help they had. Unfortunately, I didn’t remember to do that every time. But in general, you can tell as a judge how much help they had had.
7. One aspect that I in particular looked for was confidence in the results – as in error bars. Almost no one had error bars, and the one who did I asked how he got them and whether they were believable.
The error bar obsession probably comes from my own research and my professors from undergrad. In fact, I’m presenting some research on age-dating martian volcanos this week at a conference (LPSC XLI) that’s basically ALL about error bars, and it’s what I expect pretty much everyone to ask me about at my poster on Thursday night (if you’re actually interested in this, ask below in the comments).
An example of the importance of this was a high school student who was doing an experiment growing algae, and he found a negative mass on the 3rd day. I asked him why. He said he didn’t know, maybe something about the atmosphere changing. I asked him what his uncertainty was in his measurements, and he said that when all the analysis was done, he would do the detailed fitting analysis and the uncertainty in his fits. I replied, “No, each measurement you made has an inherent uncertainty in it, and you should be displaying that on your graph. For example, you found a mass of negative 0.01 gms. If your uncertainty in that measurement is ±0.1, then you’re fine. If it’s ±0.00001, then you have a problem.”
In another example, there was a student who I thought was the best in his category that I saw in the middle school part. He had looked at earthquake data in our state in order to test an empirical law, and he showed that it was wrong for small earthquakes. He was graphing magnitude versus number of quakes, and was looking at mag. 3-4 and found that it fit the law with a value of 13, but then 1-2 didn’t fit where he found 9 but expected 60. I asked him what the uncertainty was in each point. He didn’t know, but he said that he found only about 10% of what he expected. I told him I realized that, but in his magnitude 3-4 bin, he found it kinda fit the law, but the question was how well it did. In other words, at what magnitude does the law break-down with statistical “certainty.” He didn’t know. Since I was the last judge for him for the day, I told him he needed to look into Poisson statistics (sometimes referred to as “counting” statistics).
Some of the Formal, Official Stuff, and Judging Paperwork
The way I went about the actual judging part was that I left the score sheets alone. I went to each person with a blank sheet of paper and just wrote down notes. I spent about 10-17 minutes with each person (we were told to spend 12 on middle school, 15 on high school), and I did time it. I felt this was important (a) so the students had an equal amount of time with me, and (b) so that there wasn’t a judge waiting because I was taking up all the student’s time (speaking from experience of being that judge waiting …).
When I was done with each student, I went to a table and wrote down comments on the judging form that the students would actually see. I then went onto the next student I was judging.
I waited until I had talked with ALL students in the section before I sat down to actually assign a numerical score (morning was high school, afternoon was middle school). I figured that was the fairest way to do it so that I wasn’t too easy nor too hard at the beginning of the judging period and so that I could get a feel for the general level of the projects and rate them accordingly.
We were told to judge on a 100-pt scale, and judge independent of whether it was individual or team, and whether they worked on their own or in a lab. 30% went to creative ability, 30% to the scientific thought, 15% to thoroughness, 15% to skill, 10% clarity, and then an additional 16 pts to teamwork where the score was then normalized to 100 for teams. If people are interested in more details on what each of these were defined as, ask me in the Comments section.
Final Thoughts
I find judging – the two times I’ve done it – fairly enjoyable. On a jealous note, it’s really amazing what these young adults can do. When I look back at my middle school and high school science projects, there simply was nothing comparable until I specifically chose my “Senior Project” at the end of high school and built a model roller coaster.
Most of the projects at least show some interest in their chosen topic, especially at the high school level. There are always those few where you can tell that the student felt obligated to do something they really don’t care about, but luckily those can be offset by the ones that show thought.
Determining how much help students had is sometimes hard, but it’s important because it can mean the difference between someone winning first place or not even being in the running (something we did last year on a particular tennis racket project). It can be a hard judgement call, but it’s one that I have given my input on and then left it up to the head judge in the topic to decide.
Anyway, to sum up, I hope that this has been interesting and/or informative. If you’re a scientist, I highly recommend judging a science fair, at least once, to get a feel for it and to try to encourage young, potentially future scientists. Those that are actually interested in what they’re doing do enjoy speaking with people who are not their parents/advisor/mentor that understand what they’re doing and its importance.
Exposing PseudoAstronomy 