Monday, April 24, 2006

Web Design Unit Evaluation Report

Web Design Unit Evaluation Report

The purpose of this unit was a pilot test of offering EDTC 3123 in an online format. The web design aspect of the course is commonly viewed as the most challenging unit for students, therefore making this the greatest common factor for offering this course online. In other words, if they cannot complete this unit adequately online, then it really does not matter if the other units can be done online or not.

The unit was broken into 5 lessons, and also included a discussion board where students were to ask questions and respond to challenges faced in each lesson. To evaluate the effectiveness of this unit, students completed an online unit evaluation form. Additionally, I held an informal discussion with the students about what they felt went well and what obstacles were difficult to cross.

Below is a summary of the responses to each question of the survey:

5= Strongly Agree
4= Somewhat Agree
3= No Opinion
2= Somewhat Disagree
1= Strongly Agree

1. I was able to access the learning materials in the unit.
Average = 4.63

2. I understood what I was supposed to do in each lesson.
Average = 3.74

3. The content of each lesson was easy to follow.
Average = 3.47

4. The video tutorials were helpful to me.
Average = 4.74

5. The video tutorials played at a speed that was easy to follow.
Average = 4.47
6. I understood what I was looking for in the example websites in lesson 1.
Average = 4.16

7. Using the html tutorials website was helpful to understand html code.
Average = 3.37

8. By following the instruction in this unit, I was able to build and publish a basic website with FrontPage.
Average = 4.68
9. I understand what a WebQuest is and how to use it in a lesson.
Average = 4.26

10. I feel like I know enough now about web design that I could continue to build on my classroom website and use it for instructional purposes.
Average = 4.37

Discussion: From these averages, most students felt that the unit was effective for them to learn how to design an instructional website. All questions averaged greater than 3.

On questions 2 and 3, it revealed what was reinforced by student comments offered later in the evaluation form, such as:

- I think you need to be more careful about how the directions are worded. It might make sense to someone who already knows what is going on, but to someone who has no idea, the directions were kind of confusing.
- Wording, and a more clear form of what is expected in our assignments.
- The instruction of homework was a little bit confusing. So I needed more detail instruction for that.

However, I am leery of relying too heavily on the feedback from this group of students to fully decide how effective the instruction was. When asked how much time was spent on each lesson, here was the average hours given:

Lesson 1- 1.47
Lesson 2- 2.32
Lesson 3- 2.16
Lesson 4- 2.56
Lesson 5- 3.44

In reality, this unit would have been given about 2 weeks of class time to complete. The sum of average time spent on all 5 lessons is just under 12 hours, which would be equivalent to time they would have spent in class. However, normally a student would be expected to spend time out of class studying and working on the project, so the students actually spent much less time on this unit than should have been.

Video Tutorials
The students felt very strongly that the videos were crucial. They definitely benefited by the step by step instruction in this format, especially for those who “discovered” the video control buttons (pause, play, fast forward, etc.). I think a revision that could be make is to more explicitly point out how to use the video tutorials, perhaps even create a tutorial for the tutorial.

Lesson 2: The Infamous HTML Quiz
Probably the most problematic point of the entire unit was a great misunderstanding of how the students were to complete the html quiz. This was intended to be a very simple exercise, however many students misinterpreted the instructions and made this a very, very complex task. So much that many reported that they felt if lesson 2 was so beyond their ability that they felt hopeless about the rest of the lessons. This was one of the excuses given for why so many students did not even access lessons 3 through 5 until the day before the entire unit was to be completed. I personally think the students just procrastinated, and when they finally went into the lessons were overwhelmed by what they would have to complete within 1 day.

In light of how the html quiz went, I am actually considering the fact that this lesson could be dropped completely, for several reasons. For one, it would save time in an already very busy course. Also, if the point is for students to be able to recognize basic html, the reality is that since most people use web editing software today, most pages do not contain basic html code in their source code. So, for these reason, I believe that the html lesson could possibly be excluded altogether.

Accessing the WebQuest Template
Another problem that many students faced was that the tutorial I created for obtaining the WebQuest template was using the built-in Windows XP Zipped File Utility. However, when students tried to do this on campus, they were presented with WinZip (trial version). When they did not recognize this, they asked for assistance from the COE tech help desk, and were told anything from, “I guess you’ll have to buy it” to “It’s not working here, you’ll have to go to Math Sciences”. In actuality, the students only had to make a few clicks to get the template into their FrontPage website.

Final Thoughts
I believe that this unit could undergo several of the revisions mentioned above and be retried before ruling out whether this course could work online. When I read student comments such as:

“I think this was the ONE unit that should have been done in class. Most of the other units in this class would have been perfect for an online class but this one was too complicated to do on our own. Maybe if we would have taken one class period to discuss what you wanted and then finished the lesson outside of class that would have been more helpful. “

I still have to consider that the students really did not spend ample time working on the unit. One student even offered the complaint (after the fact, of course) that, “Well, we didn’t sign up for an online class”. So possibly, this course would have to be actually offered online to receive feedback from students who “signed on” for an online experience.

Considering the average scores on the evaluation given above, I still believe that the students viewed this unit as effective in accomplishing the unit object which was that students could create an instructional classroom website.

No comments: