Task 1

Introduction

This semester, I will be teaching a course, “Build Your Own Computer in 30 Days” that is being offered at the Georgia Center for Continuing Education. This is the first time I am teaching a publicly offered course, so it will be quite a learning process all around for me. EDIT 6210, which I am also taking this semester, mandates an instructionally oriented term project for the semester. As such, I have decided to develop for one of the class sessions a database-based decision support system that allows the learners in the class to learn about the various hardware components through the process of being presented various computer components and asking them to match the component appropriately against a list of choices. The student may be asked to either choose the one that’s definitely incompatible or choose the only one that’s compatible with the displayed component. Additionally, there will be a simulation session in which the same database is utilized to allow the student to pick out various components and attempt to build a computer system from the available components in the database. When the student has assembled his computer, the simulation will analyze the compatibility of all the components as well as ensuring all critical components that make up a system have been selected and then tells the student whether he has built a working system or not.

Since I do not have a lot of experience with simulation packages, I decided to review the research materials available on simulations applied in an instructional situation to better understand the issues surrounding the use of simulations to achieve learning objectives for a course. What follows are five papers that focus on simulation in an instructional context.

Also available for download: task1.doc

Article 1

Cameron, B. (2003). The effectiveness of simulation in a hybrid and online networking course. TechTrends, 47, 18-21.

This article compares the learning outcomes of 85 students on the topic of computer networking where the students are divided into two classes. As the author states in his paper:

The purpose of this research was to compare students' performance on simulation-based courses and static graphic representational teaching of the same course content in an online learning environment. (p. 19)

Where the “static graphical representational teaching” method refers to students diagramming network designs out on paper or with Microsoft Visio diagramming tool. The instructor likewise teaches the course by showing various network configurations drawn up in this method. The simulation-based course provided an interactive simulation package that allowed students to design and interactively tinker with various network designs in a commercially supplied software simulation package that allowed the student to obtain instant feedback on their design efforts.

Cameron goes on to show that the group of students utilizing the simulation tool for learning their class materials performed significantly better than their counterparts learning the materials with the alternative, traditional method. His conclusions are largely drawn from the superior test performances throughout the semester exhibited by the students in the simulation-based course over their peers in the traditional course.

I found it frustrating that the author brought up issues regarding general findings from past simulation studies as well as distance learning studies, such as lack of motivation, low student satisfaction, and inability of simulations to be realistic (i.e. are overly simple) (p. 18), yet he failed to go into details on how the simulation tool that was employed specifically addressed these points, yet alone how he measured his findings. The author gripes that:

However, these studies on the effectiveness of simulations in learning environments compared simulations with traditional classroom instruction. Web-based distance learning courses offer a unique challenge to teaching traditionally hands-on courses like network simulations. (p. 18)

If Cameron thought that past studies had particular short-comings in their methodologies for testing the effectiveness of simulations, then one has to wonder why the author chose to conduct his own research in similarly deficient manner. Additionally, the author mentioned “unique challenge” of web-based courses over traditionally taught hands-on courses, but fails to expound upon what exactly these are in his article. Lastly, the author only hints at the level of detail and realism of the simulation package employed and this lack of details makes it very hard to discern how much of the positive improvement over student learning goals was a direct result of the simulation environment itself vs. other factors. For example, the author noted that “Students who used the simulation reported more time spent on course assignments than did the students who used Microsoft Visio (3.5 hours vs. 2 hours)” (p. 19). This lack of clarity about the simulation package utilized also means that this research would be hard to set up and replicate in order to verify results.

I am actively developing a simulation-based application for a class I am teaching this semester, “Build Your Own Computer in 30 Days,” and this article tends to deliver a positive message that such an approach to delivering information will be an effective approach and as such, provides good motivation to continue with the effort of developing the simulation.

Article 2

Mitchell, R. C. (2004). Combining cases and computer simulations in strategic management courses. Journal of Education for Business, 79(4), 198-204.

This paper discusses two approaches, case based and simulation based, to teaching a strategic management course and delves into examining the benefits and drawbacks of both approaches.

This author is a thorough background researcher. He not only reviewed numerous researches prior to drafting his proposal, but also clearly synthesized and cited the various researches leading up to his experiment and leaned on them to argue the merits of taking his particular approach to further investigating cases vs. simulation and performing an experiment that tests the effectiveness of combining both approaches. To wit:

One design, using the traditional case method (TCM), made primary use of comprehensive cases, which students read as homework and discussed in class. The second design, using computer simulation plus cases (CSC), replaced about half of the case work with a computer simulation. (p. 200)

Unlike the first paper reviewed, the author cites the source of the simulation and describes in good detail what the simulation entails. The author goes on to describe the experiment’s methodology and the expected learning outcomes of the strategic management course and how he obtained objective results and limiting researcher bias.

I found it very interesting that Mitchell eliminated one of the main difficult to control variables of many studies that seek to compare two instructional approaches by teaching all of the class sections and utilizing the two different approaches himself rather than relying on multiple professors. Although the author states that he "did not have preferences between the two designs or pre-existing beliefs about their effectiveness" (p. 201), one must wonder how much to trust such statements in a conducted research experiment. Perhaps a better approach would have been to interview several qualified professors, focusing questions on their learning preferences and find one who surveyed to be unbiased to the two approaches and utilize this particular professor to deliver all instructional material, thus allowing the researcher to remain outside the experiment and perhaps more observant as well as objective in analyzing the experiment and outcomes.

Mitchell concluded that of the two approaches, "(with and without a simulation), there is no difference in performance based on the central learning objectives adopted for the courses in this study," and that "neither is a panacea or has general superiority over the other." (p. 203). Although the author found no significant differences in performance utilizing the two approaches to satisfy mainstream learning objectives, he did go on to state that "it is not hard to imagine some objectives and assessment techniques that would have a definite bias toward either case or simulation activities." (p. 203).

To me, this paper clearly shows that simulations probably shouldn't be seen a replacement for existing pedagogical techniques, let alone real-world experience, but rather, is another tool in an instructor's ever expanding choices of tools to be harnessed in achieving learning objectives. The paper also gives me confidence that a lesson plan that incorporates a simulation as I was planning for my upcoming class will likely be effective.

Article 3

Schmidt, S. J. (2003). Active and cooperative learning using web-based simulations. The Journal of Economic Education, 34(2), 151-67.

The author uses a web-based simulation package to teach basic economic principles by simulating various economic factors and letting students make various decisions that drive the simulation as might occur in a real economy. The primary positioning of this paper is that simulations which "requires students to make use of class material, not merely memorize it, and is believed to lead to greater long-term retention of material than traditional lecturing" (p. 152). A second primary benefit of simulations is that they enable collaborative learning which "requires students to work together in small groups, with responsibility for the group's decisions being shared equally among the members of the group...it can also result in students retaining more material than they would with traditional lecture-oriented instruction" (p. 152).

Schmidt has a rather long-winded manner of writing that, while going into great details about many aspects of the experiment, also serves to cloud over the important points he wishes to make in the paper by speaking at length on items that do not really bring any new knowledge to light for the reader. For example, he devoted many, many paragraphs to explaining economic principles to the reader and concepts that would only matter to someone reading the paper to better understand economics. He even expounded at length on the nature of the World Wide Web and even what platform and programming language the simulation was written in and none of this seems apparently useful to the person who's trying to understand the effectiveness of computer based simulation over traditional methods. To give an example, the author writes:

The simulation is composed of four programs running on a UNIX-based Web server. Two are small programs that handle the submission of data by the students, one for transportation improvements, one for production improvements. They are written in C and process the forms filled out by the students on the simulation's Web pages; they simply update the simulation's data files, adding the improvements and decreasing the students' wealth. A third program, written in TCL/TK, redraws the map on the basis of the updated data files whenever the students refresh the map Web page and recalculates the transportation costs matrix. (p. 158)

I also found that the author tended to make a lot of matter-of-fact statements in the paper that seemingly have no research basis or referenced overly generalized summaries contained in footnotes. The paper grossly failed to cite past material directly in general. For example, the author states, without any citation or background references, "The key to the success of any simulation, or indeed any active learning exercise, is to ensure that the exercise teaches and reinforces the instructor's learning goals for the class." (p. 163).

Finally, instead of discussing the outcome of, say the students' retention rate for the material being measurably increased and citing the observed measurements that show this, Schmidt goes on to discuss how much the students enjoyed the simulation exercises and thus it must have been an improvement since they say so!?

Students in the classes that have used the simulation have greatly enjoyed it; nearly all mentioned it favorably on end-of-term evaluations. They reported that their understanding of the principles was increased by applying them in the simulation and that this enhanced their understanding of the actual historical experience. The simulation appears to benefit students throughout the grade distribution. (p. 164)

Indeed, the only charts the author proffers up in the paper are nothing more than the simulation's "game statistics" that are only meaningful to the participants in that particular simulation game and do not offer anything in the way of statistical analysis for achieving learning objectives and certainly do not shed any light whatsoever on a student's fundamental grasp of basic economic principles.

In short, this paper was a disappointment to me as the title clearly named that which I was very interested in constructing for my upcoming class, a web-based simulation exercise, yet it amounted to no more than one author's experiential journey through employing a simulation exercise in his own classrooms and teaching us, the reader, a little bit of economic principles in the process.

Article 4

Dalgarno, B. (2001). Interpretations of constructivism and consequences for computer assisted learning. British Journal of Educational Technology, 32(2), 183-94.

Dalgarno discusses two broad movements driving recent changes in teaching and learning practices. He begins by discussing the demise of the behaviorist view and the rise of cognitive view of learning in the field of psychology. The second development he discusses is "the gradual rejection of the assumption, held by many cognitivists, that there is some objectively correct knowledge representation...[with]...the alternative view, termed constructivist, is that, within a domain of knowledge, there may be a number of individually constructed knowledge representations that are equally valid." (p. 184).

Of particular educational value to me was the discussion of the three different interpretations of constructivism, which "have been labelled by Moshman (1982) as endogenous, exogenous and dialectical." (p. 185). Seeing for the first time that there at least three such distinct interpretations within this camp helped me to understand why there are so many seemingly conflicting views as well as application of the constructivist approach to instruction.

In the author's various treatments of constructivist approaches, he describes the popularity of simulations as follows:

Firstly, simulations (and some microworlds) provide a realistic context in which learners can explore and experiment, with these explorations allowing the learner to construct their own mental model of the environment. Secondly, the interactivity inherent in microworlds (and usually in simulations) allows learners to see immediate results as they create models or try out their theories about the concepts modelled (Rieber, 1992). (p. 187)

The author also goes on to talk about other constructivist based tools that cater to each of the three primary interpretations of the constructivist view such as guided hypermedia, practice modules, computer supported collaborative learning, etc.. This paper is well researched and intelligently cited various works throughout the paper, making good use of the meta-analysis approach to survey the field at large, pinpoint the recent changes in psychological and pedagogical theories that are driving the constructivist approach to teaching.

For me, this paper gives me a better understanding of the role simulations serve and narrow down the underlying theories that are being applied when one employs a simulation tool in an instructional context.

Article 5

Johannessen, L. R. (2000). Using a simulation and literature to teach the Vietnam war. The Social Studies, 91(2), 79-83.

This article drew my attention with its title as I wondered just what sort of simulation was utilized in teaching the history of a war. As it turns out the simulation is a live-action one in which “The simulation requires the construction of an environment in which the participants move and make decisions as if they were operating in the real environment.” (p. 79).

Johannessen is attempting to simulate the environment soldiers constantly dealt with and the stress and fears constantly at the forefront that was caused by their near constant exposure to possible mines and booby traps that put their lives at risk every day. As the author explains about this particular simulation:

Usually at least one curious individual blows up his or her squad because of taking a closer look at those interesting-looking objects. Two or three squads pick the right trail and make it through without an incident. Another two or three pick the heavily mined and boobytrapped trail and do not make it. Other squads get nearly to the end, stop being so careful, and hit an angled mine stretched across a portion of the trail. In this activity, students get a glimpse of the real terror of combat. (p. 80)

It is hard to imagine that such a simulation even begins to approach bringing about the feelings and fears the soldiers experienced, however, the author goes on to discuss his student’s reactions and comments they make about the literature and war they are studying and it seems evident that:

After completing the activity, students are better prepared for dealing with issues they encounter in their reading because they can relate them to their own experiences and perceptions during the activity. After completing the activity, students are better prepared for dealing with issues they encounter in their reading because they can relate them to their own experiences and perceptions during the activity. (p. 81)

However, the paper had a bit too much of an anecdotal feel to it and, indeed, the author seems to be mostly describing his approach to teaching the war throughout the paper, providing a lot of recommendations and resources another teacher (the reader?) may utilize in his/her own classroom. In other words, it was argumentive with the intent to persuade the reader that the author was right, but there were no formal research protocols followed in the paper. While many references were cited to back many of the author’s points, I was unable to determine what supposedly gave this particular paper a research bent. For example, Johannessen quotes several student reactions to the simulation in the follow up activities, and based solely on the strength of these students’ reactions, states that these “students were actively engaged in discussing how and why American soldiers might commit acts of brutality against Vietnamese civilians. The simulation they participated in provided an important step in helping them understand what [the class literature] was trying to tell readers.” (p. 81).

This paper, while quite persuasive in its argument, really didn’t do a whole lot for me in terms of helping me to evaluate the effectiveness and usefulness of simulation based approaches in helping my students to learn new materials. It did however help me to see that simulations are not confined to being computer based models. This realization got me to thinking about the possibility of utilizing old computers in the classroom whereby I would have the students disassemble an old surplus computer completely and put it back together. An exercise like this would directly expose the learners the various components of a computer in their “natural state” and garner some hands on experience with physically working with computers without fear of breaking brand-new components.

This paper has broken me out of the box, so to speak, and coupled with the prior excellent meta-analysis research paper on theories grounding the principles behind what makes a simulation work, I believe I have gained a synergized outlook regarding simulations in spite of this paper’s shortcomings.