MATH CONNECTIONS EVALUATION

The evaluation of the Math Connections program was conducted by Research Center for Educational Technology at Kent State University.

The focus of RCET's evaluation was on student learning from the Math Connections lessons.
To that end, RCET researchers worked with the Akron Public Schools Math Curriculum Supervisor to create short pre-post-tests on benchmarks covered by each of the three Math Connections lessons. Pre/post tests consisted of four to six questions modeled after relevant 6th grade Ohio Achievement Test items. Questions on both test had similar structures but differing contexts and quantities. Pre/post tests were distributed to participating teachers by the APS Math Curriculum Supervisor who also collected reporting sheets on pre/post-test scores and returned them to RCET staff. Pre-tests were given before the first video-conferencing session in each lesson and post-tests were given after the lesson sequence was completed.

In addition, RCET researchers attended both the initial teacher orientation session held on October 17, 2006 and the final teacher focus group session on June 6, 2007.

PRE/POST RESULTS

Pre/post test data was limited. Pre/post test data from the first lesson, Chuck Close-Up Close: A Grid Self-Portrait, which tested proportional reasoning, was obtained for four classes of students. The pre/post test on the second lesson, Get Me to the Game on Time, covered time and distance problems. However, only post-test data from a single class was obtained from the second lesson, thus learning from this lesson could not be evaluated. Pre/post-test data from the third lesson, Determining Dimensional Space, concerning geometric reasoning was obtained from two classes. Because there was so little data, data was collapsed across classes and the two lessons evaluated. Paired sample t-tests were used to compare individual student scores on pre- and post-tests.

Post comparisons revealed that students exhibited statistically significant improvement from pretest to posttest, t (147) = 4.058, p < 0.001. The mean for the pretest was 2.51, while the mean for the posttest was 3.03, with a mean difference of 0.527. The overall effect size from pretest to posttest was 0.314, indicating almost a 1/3 standard deviation increase in performance. Although the data collected was limited, results indicate that students learned the targeted math concepts (proportional and geometric reasoning) from the Math Connections lessons.

TEACHER FOCUS GROUP

A wrap-up focus group was held on June 6, 2007 for participating teachers. It was led by Sandy Kreisman from University Circle Interactive Cleveland. Four teachers attended the session and several other written evaluations were obtained via email.

MATH CONNECTIONS EVALUATION

The evaluation of the Math Connections program was conducted by Research Center for Educational Technology at Kent State University.

The focus of RCET's evaluation was on student learning from the Math Connections lessons.
To that end, RCET researchers worked with the Akron Public Schools Math Curriculum Supervisor to create short pre-post-tests on benchmarks covered by each of the three Math Connections lessons. Pre/post tests consisted of four to six questions modeled after relevant 6th grade Ohio Achievement Test items. Questions on both test had similar structures but differing contexts and quantities. Pre/post tests were distributed to participating teachers by the APS Math Curriculum Supervisor who also collected reporting sheets on pre/post-test scores and returned them to RCET staff. Pre-tests were given before the first video-conferencing session in each lesson and post-tests were given after the lesson sequence was completed.

In addition, RCET researchers attended both the initial teacher orientation session held on October 17, 2006 and the final teacher focus group session on June 6, 2007.

Pre-post comparisons revealed that students exhibited statistically significant improvement from pretest to posttest, t (147) = 4.058, p < 0.001. The mean for the pretest was 2.51, while the mean for the posttest was 3.03, with a mean difference of 0.527. The overall effect size from pretest to posttest was 0.314, indicating almost a 1/3 standard deviation increase in performance. Although the data collected was limited, results indicate that students learned the targeted math concepts (proportional and geometric reasoning) from the Math Connections lessons.