About Anything

I was disappointed to see the results of this study, but not surprised.

The study, Effectiveness of Reading and Mathematics Software Products Findings From Two Student Cohorts, sponsored by the U.S. Department of Education, reports on student test scores of a second year of use of selected software programs aimed at 1st grade reading, 4th grade reading, 6th grade math, and algebra I. They looked at 10 software products and found that only one had a statistically significant effect. Given that there were 10 chances, that one should also be considered suspect.

Until we really understand the details of human learning we will not be able to build or evaluate effective teaching technology. These broad brushed studies provide such a coarse look at the overall process that we can conclude very little. The study itself ends with a list of caveats that include: “the study preclude direct comparisons of product effects”;  “Because districts and schools volunteered to implement particular products, their characteristics differ and these differences may relate to effectiveness”; “The study design does not rule out the possibility that a product the study finds to be ineffective could be effective if implemented by other districts or schools”.

So why did the Department of Education bother to do it?

It would be much better to spend resources on understanding the learning process with enough rigor to construct educational environments that improve it.

eSchool News,  a monthly print and web newspaper read by K-20 decision-makers, has published a summary of developments that they believe could have a profound effect on educational technology in the schools. They are:

5. How will ‘validated learning’ be enforced among the nation’s colleges and universities?

4. How will new federal and state regulations affect internet safety education in schools?

3. Who will be Kevin Martin’s successor as chairman of the Federal Communications Commission, and how will this change affect telecommunications policy in 2009?

2. Will the transition to digital TV broadcasting next month occur seamlessly–or will schools experience any problems?

1. How will education fare under the Obama administration?

The article is at: Five ed-tech stories to watch for 2009 (registration required to read the entire article.)

Today’s New York Times has an article titled At M.I.T., Large Lectures Are Going the Way of the Blackboard. It summarizes MIT physics department courses that use Technology Enhanced Active Learning, or TEAL. The article reports that, while replacing lectures with TEAL encountered resistance, attendance in classes is up and failure rates have dropped by more than 50 percent.

A slightly more detail article is available at Teal Teaching in MIT Spectrum, Winter 2004.

The approach was spearheaded by John Belcher, who is quoted in the Spectrum article as saying that TEAL students make gains nearly double those of their counterparts in standard classes.

Course materials, photos of the classroom and interactive visualizations are at Visualizing Electricity and Magnetism at MIT.  The content is available for free used for non-profit educational purposes, as long as an acknowledgment is given to the MIT TEAL/Studio Physics Project.

A research report in Science, Why Peer Discussion Improves Student Performance on In-Class Concept Questions (subscription required), reports that clickers used in the classroom (see clickers in the classroom for a review) can help students arrive at conceptual understanding on their own through the process of group discussion and debate.  According to the authors of the study, most instructors report that the percentage of correct answers almost always increases after peer discussion. They go on to say that it has been generally assumed that active engagement of students during discussion with peers, some of whom know the correct answer, leads to increased conceptual understanding, resulting in improved performance after PI.

In this study, students were asked to answer the question a second time after their peer discussion, but also to answer a different, conceptually related question. Neither the correct answer, nor the student responses were shown between questions. The authors conclude that “our results suggest that peer discussion can be effective for understanding difficult concepts even when no one in the group initially knows the correct answer.”

One of their students, quoted in the study says it pretty well: “Often when talking through the questions, the group can figure out the questions without originally knowing the answer, and the answer almost sticks better that way because we talked through it instead of just hearing the answer.”