I’m conducting an evaluation of WSD Online, our district’s Moodle system. Right now, we use this as our online classroom management tool. We’re just barely piloting fully-online courses, but that’s not what I’m evaluating. Right now more than a few of our teachers use Moodle as a supplement to their in-class teaching. I’ve decided to evaluate four primary areas regarding our use of Moodle:

  1. How student engagement in our classes is impacted when Moodle is used
  2. If the most popular activities our teachers use in Moodle (in ascending order: quizzes, assignments, and forums) actually reduce teacher preparation time
  3. How the usage of Moodle activities impacts attitudes among teachers and students, such as quiz-building, assignments, over the traditional forms of delivery, and
  4. How Moodle affects the academic performance of students

I’m interested in hard data on how the groups perform compared to non-Moodle-using groups. We only have between 20 and 30 teachers actively using Moodle, which isn’t significant enough for gathering districtwide data. However, I can gauge the results on an individual course basis by comparing academic performance of the students in the teacher’s class with the usage of Moodle, to how the students performed before the teacher used Moodle to supplement the learning environment.
With the help of some of my classmates I’ve evaluated a few other web sites, or web-based learning environments to be exact. A rubric designed by Baya’a, Shehade, & Baya’a (2009) was used to assess usability, content, value, and vividness of each site, to produce a criterion-referenced evaluation report. It did help me understand some of the strengths of using a rubric to assess content, when there are identifiable criteria to evaluate. It also helped me recognize this is probably not the way I want to go for the WSD Online evaluation, since any criteria I assign will be fairly arbitrary at this point. The norm-referenced approach will probably be more useful, because I can directly compare our Moodle-using teachers and students to non-Moodlers.
References
Baya’a, N., Shehade, H. M., & Baya’a, A. R. (2009). A rubric for evaluating web-based learning environments. British Journal of Educational Technology, 40(4), 761-763.