Impact of a Simulated Game on Learning and Engagement
By: Nina Sarkar, PhD,
Stephen W. Hammel, JD, Esq, and
Prof. Christina Manzo, MBA, CPA,
Queensborough Community College, CUNY, New York
Abstract
Today’s students are digital natives who have grown up with computer technology and video games. Their constant exposure to the internet and other digital media has shaped the way they receive, process and learn information. Consequently, the traditional lecture and textbook approach to education is not as effective for this generation. We believe that students can benefit from reality based computer simulated games which are incorporated into the course curriculum. Games and computer simulations are no longer just for fun; they can be used as an effective pedagogical tool to enhance learning and foster an engaged learning environment. While there is a general consensus among educators that educational games are an effective method of motivating students, there is a lack of empirical studies relating to the impact of this teaching modality on students’ academic performance and engagement. This paper reports on the effectiveness of using a computer simulated game on student learning and engagement in three different business courses.
Introduction
A new generation of students, born between 1982 and 2000, called the NetGeneration, are now enrolled in higher education, (Oblinger 2004). Studies show that they are technologically connected, demand immediate response, prefer experiential learning and require social interaction, (Rodley 2005; Prensky 2007). Further, researchers have demonstrated that today’s students have a penchant for highly active and participatory experiences inside and outside their classrooms, (Oblinger, 2004). Unlike generations of past students who studied and primarily acquired knowledge in a brick and mortar style classroom through the traditional lecture format, the NetGeneration students are accustomed to operating in a digital environment for social communication, research and information gathering. Furthermore, they may not even have to struggle to analyze situations, as many opinions are easily available online. Since technology allows for greater time and physical travel freedom, the NetGeneration student’s learning is unconstrained by time, place or formal learning structures. These students are accustomed to obtaining instant access to practically all of society’s questions and combined knowledge. Thus, a sense of independence is developed due to their technological connectivity. Through this connectivity the NetGeneration student actually becomes an active participant in the learning process. Simulations and educational games, represent another avenue for educators to leverage the student’s desire to participate in an informal teaching method while simultaneously providing the connection to technology used and desired by the NetGeneration. Educational games can blend well with these common characteristics of the NetGeneration and can lend itself to the student’s learning experience. Therefore, we find that an increasing number of educators are using simulations and gaming as a means of teaching this next generation of students.
In an effort to connect with the NetGeneration and adapt our teaching to their learning style, we incorporated a simulated stock market investing component in our courses. We incorporated the simulated stock market game across three distinct and separate courses taught by three different professors in the Business Department of a community college. This paper reports on how we incorporated the game in our classes and the impact it has on students learning and engagement.
Using Games as a Pedagogical Tool
What happens when we bring games or simulations into the classroom? Typically, the first thing we might expect to see is increased student motivation. Early research on arcade-style games demonstrated that games create intrinsic motivation through fantasy, control, challenge, curiosity, and competition, (Malone 1981; Cordova & Lepper 1996). Furthermore, through engagement, games leverage the players’ desire to develop new skills, participate in new roles and create a desire to better understand the world from a “professional” perspective, (Gee 2005; Shaffer, Halverson, Squire & Gee 2005). Gaming is undeniably a powerful, pervasive method of learning; indeed, most psychologists would agree that through play we test ideas, strategies, develop new skills, and participate in new social roles (Piaget 1962). Game and simulation based learning encourages motivation and student engagement, since the act of participating in games creates a type of emersion and multisensory experience by encouraging students to be present in body as well as in spirit, (Kapp 2012).
Educational games encourage students to explore and experience situations beyond the boundaries of a textbook. Being engaged in gaming not only encourages students to become proactive and exploratory but also require students to engage in decision making skills thereby teaching students to become self-reliant learners, (Taradi, Taradi, Radic & Pokrajac, 2005). As opposed to the more traditional method of the passive act of only reading, games can create an environment of experiential learning by allowing students to gain knowledge through experiencing simulated situations first hand, (Rickard & Oblinger, 2004).
Furthermore, game based learning has a broad appeal to many types of students. For example, we have all experienced those students who possess the ability to engage in meaningful conversation or have the ability to express a new or novel opinion in class, but do not fare well on examinations. Gaming and simulations allow educators the unique opportunity to appeal to those students in a new and different way. Through gaming, educators can, reach those students who possess this non-typical learning style and typically do not respond or perform as well in the traditional teaching environment. Figure 1 illustrates the different advantages of game based learning.
Figure 1
Not everyone favors game based learning. Harp & Mayer (1998) have argued against the positive effects of game based learning. They state that gaming detracts from the primary focus of the learning experience – it creates distractions and draws the students away from meaningful learning.
Despite the controversy, there has been increased interest in gaming over the last decade. Growing numbers of educators have experimented with game based or game enhanced instruction. While there is a general consensus that gamming in the classroom increases student’s interest in the subject matter and may foster motivation and encourage engagement among students, there is very little evidence which indicates that this translates to better student academic performance or even leads to deeper knowledge. Research into the effectiveness of a game based instruction is spread very thinly over a wide range of subject areas, age groups and educational settings. Canon-Bowers (2006) mentions that we are charging headlong into a game based learning without knowing if it works or not. Additionally, Mayo (2007) has argued that there are only a handful of studies that have rigorously measured the learning outcomes of immersive games compared to other teaching methods. There are only a handful of empirical studies available and none discuss the impact on learning through the use of a simulation which involves actual real time data and true market conditions in an academic setting. Therefore, due to the lack of empirical data and the scarcity of systematic investigations the timeliness of our study is underscored.
This study reports on the impact of using a computer based simulation game in three different Business courses over the period of a semester.
Game Description and Simulation Structure
This study was designed and implemented by three professors who teach three separate and distinct courses in the Business Department of a community college: Business Law, Principles of Accounting and Principles of Finance. While the three courses are very distinct in their learning objectives and outcomes, there is a common thread that weaves through all of them. All of these courses incorporate fundamental core business concepts such as corporate profit and losses, risk and return, social justice, ethical/legal situations, time value of money, and investment diversification. One of the common outcomes of the three courses is that students will understand the relationships between today’s global business environment, geo-political tensions and global inequalities and will be able to relate to real life situations.
Based on previous academic assessments over several years, we found that our students do not fully comprehend these concepts of business and investment, nor are they able to relate to these concepts, and consequently are not motivated to look beyond the textbook to further their knowledge. Moreover, as we teach our own courses and complete the required content in our syllabi, we rarely have the opportunity to further explore these concepts in depth. Therefore, in order to engage and motivate our students to develop a stake in their educational experience, we decided to incorporate a stock market simulation in our classes. We hoped, that the game would both engage and educate students across our classes, in an independent way designed to engage outside of the classroom.
In order to test our hypothesis, we designed a quasi-experimental study to attempt to measure student learning and engagement. We asked and focused on three core questions:
- Was there an improvement in knowledge and understanding of investment and financial concepts after students played the simulation?
- Were the students more engaged as a result of participating in the simulation?
- Did the incorporation of the simulation have any impact on student attrition from our classes?
The simulation we use is a free web-based game called How The Market Works. The game simulates the real world experience of investing in the stock or equities market by using actual real time market data. Additionally, the simulation allows students to manage their own simulated portfolio prefunded with $100,000.00 of simulated cash. The game simulation is linked to live data from three separate stock markets in order to make the portfolio management as realistic as possible.
At the beginning of Fall 2015 semester, one of our faculty members set up the separate individual trading groups and then provided the login instructions, credentials and other parameters to the other faculty members and students. Students were then instructed to register for their individual course’s trading group. Each student’s account was funded with simulated dollars in the amount of $100,000.00. Once registered, students had the ability to make their trading elections and began investing in corporations as they chose. The game automatically provides tracking of student trades, portfolio balance, and investment performance based on the results of the student’s investment choices. Since actual market information is used, students were encouraged to use real corporate financial reports, current news, actual economic data and other important indicators to make investment decisions. Each student’s account was set up to allow them to conduct research, place several types of buy and sell orders such as; market, limit and stop loss orders. Additionally, transaction costs were imposed on all transactions to add an additional feel of realism. Students could track their performance throughout the simulation and compare their returns to other participants, thereby creating an additional competitive aspect to the simulation.
Students enrolled in our classes were required to register for the game, but not required to participate or trade. There was no minimum number of trades required or imposed. So if the students chose to play the game, they did so of their own free will. Additionally, the professors suggested that a prudent investment strategy included the ability to research and analyze a company’s performance prior to executing a trade; however again, no specific research was required. We attempted to convey the feeling that the money in the trading account belonged to the student, so ultimately they decided their individual level of comfort in investing. We felt ownership and engagement would ultimately lead students to determine their level of trading activity. Interestingly, students became more knowledgeable about other world events and how political decisions have an impact on the markets. Research was recommended but not mandatory and the students determined how much research to conduct, and how to balance news such as current geo-political and economic developments in their decisions. The simulation lasted for 12 weeks during the semester. At the end of the semester, students were required to present an investment report with an analysis of the performance of their portfolio.
A week prior to registering for the game, students were required to complete a researcher generated pretest questionnaire that tested the domain knowledge prior to interaction with the game. At the end of the 10th week, students were given a posttest with a very similar questionnaire. Our purpose was to determine if students scored higher in the posttest, suggesting that there was an increase in knowledge of the subject matter due to their exposure to experiencing the simulation.
Students also completed a researcher generated survey designed to evaluate their feelings of immersion and engagement with the course. Further we compared retention of students in our classes with the game and without.
Impact of the Game
We used a quasi-experimental pretest/posttest design model. When utilizing these types of designs, the researcher needs to be especially concerned with internal and external validity. Internal validity is the degree to which the experiment makes a difference in the experimental setting and external validity is the degree to which treatment effect can be generalized. We used a non-randomized group, comprised of students who registered in our classes. Participants were not randomly assigned to an experimental group or a control group, rather all the students who registered for our classes were included in the experiment and therefore all the students formed the non-randomized group. There are certain advantages of a non-randomized group, (Dimitrov & Rumrill, 2003). As a result of this group being intact, it does not disturb the research setting. This reduces the reactive effects of the experimental procedure and thereby improves the external validity of the design because the participants are an intact group and there are no time sampling methods that are employed.
The study was conducted in three different Business department courses over a span of one semester. After adjusting for testing anomalies such as students answering questions multiple times, or not answering either the pretest or posttest in its entirety we had 32 paired samples. A researcher generated pretest and posttest questionnaire was administered to the students. Along with demographic questions, there were 17 questions that were designed to test domain knowledge both prior to playing the game (pretest) and after playing the game (posttest). The mean scores on the pretest and posttest were calculated and are presented in Table 1.
As seen in table one, the mean of the correctly answered questions in the pre-test was 25% with a standard deviation of 0.14, and the mean of correctly answered questions in the posttest was 33% with a standard deviation of 0.12. The average posttest score was 8 percentage points higher than the pretest score for the same grouping of 32 students. This is shown in Graph 1 below.
Graph 1. Percent of Questions Answered Correctly: Pretest vs. Posttest (Aggregate)
Table 2 shows the descriptive statistics of the testing difference. The mean gain of .08, with a standard deviation of .15 is significant at 5%. These results suggest that there was a significant improvement in the test scores between the pretest and posttest.
In future studies working to quantify the effectiveness of computer simulations on learning outcomes, design modifications to the study and increased sample sizes could allow for a more robust analysis where the effect of the simulation could be separated from other factors that could affect the study outcome. In addition developing control groups and monitoring the results over a time period of several semesters vs. the various control groups would allow for the other factors that affect learning outcomes to be more easily identified and accounted for over a period of time.
A researcher generated survey was administered to the students at the end of the semester, after they had played the game to capture student’s perception of learning enhancement and engagement with the course. Students were asked how much they agreed (on a 5 point Lickert scale) with various statements regarding the game. There was also an open ended comment section where students could write about their feelings pertaining to the game. Participation in the survey was voluntary, but despite that, the response rate was a high 82%. Of the survey respondents, 54% were female and the mean age was 19.7 years. The post simulation survey revealed that none of the students had ever played a stock market simulation game prior to our classes. 91.3% of the students reported that they or their family did not have any prior experience investing in the stock market. Only 9% of the students were aware of how the market functioned prior to playing the game. 87.6% of the students reported that they thought the stock market was for the “rich” only. This is not surprising because demographically our students come from lower income communities and are usually first generation immigrants. Table 3 details the survey questions pertaining to students’ level of engagement with the course and their perception of learning enhancement.
It is important to note that while the students were required to register for the game, they were not required to play it. Furthermore, the game was not played in the classroom and was not a required part of the course. The instructors encouraged the students to play for their own learning, but it was not mandatory. If the students chose to play the game, they did so outside of the classroom and on their own time. Almost 74% of the students reported that they agree or strongly agree that they looked forward to playing the game, and 89.1% of them reported playing the game at least once a day, even though this was not required of the course. In the comment section of the survey, students reported that it was the desire to improve their portfolio performance that lead them to gain as much knowledge as they could to earn greater returns. This suggests that the students were actively engaged with the game and therefore engaged with the course. The students were motivated to improve the performance of their portfolio and looked beyond the boundaries of the textbook and classroom lectures to do so. They conducted research and became cognizant of geo-political and economic affairs – and all on their own time and outside of the classroom. 82% agreed or strongly agreed that they liked the fact that the game was linked to real market data. This stock market simulation gave the students a sense of realism. This was also an opportunity for the instructors to engage the students in experiential learning. 67% of the students reported that they would consider a career in investing. This observation was particularly interesting when coming from our students who had no previous experience with the stock market. We found that the students were not only engaged and actively participating in the game but had become independent, self-directed and self-reliant learners. We also observed a type of immersion with the game that acted as a knowledge conduit from theoretical aspects of business, finance and accounting to the practical real life experience of actually investing. We believe that through the game we were able to weave together the fundamental concepts of business. In the future, we propose to integrate the game simulation fully with the course content, projects and activities.
In addition, the withdrawal rates were compared between courses that incorporated the game with the same courses without the game. Historically, in these courses, the withdrawal rates range between 10% to 12%. We find that the withdrawal rates in the courses incorporating the game were much lower – 4%. There was less attrition of students from our classes where the game was played.
Conclusion
Today’s students are digital natives, who have grown up with computer and video games. Their constant exposure to the Internet and other digital media has shaped how they receive information and how they learn. As we move away from the Information Age to an Interactive Age, educators are increasingly taking advantage of the different educational games and simulations available for teaching. There are many attributes of games that make them a pedagogically sound learning tool which position is clearly supported by a continually expanding body of research. The use of simulations and game based literature as applied in the educational realm is continually growing.
Games engage and activate prior learning. In some cases games are based on understanding of topics such as business, mathematics, geography, science or history. Participants must use previously acquired and learned information. Thereafter, they must continue to build and learn new facts, in order to move to higher levels of gameplay.
The learning context is also very important to our understanding. In simulations and games students must acquire the necessary skill of how to contextualize the information which they been learned before they are able to successfully apply it to their decisions. Knowing what information or techniques to apply in a particular situation is critical and enables greater success.
Games provide ample real time feedback on the player’s progress. Scoring, reaching different levels and ultimately winning, or losing, provides rich feedback and assessment to the student.
Games require transfer of learning from other venues such as life, school and other experiences. Students through playing start to begin to be able to see the connection and transfer existing learning to a unique situation is part of gameplay.
Games are inherently experiential. Those who play games engage multiple senses. For each action, there is a reaction. Continuous feedback is swift and sometimes difficult to accept, however it provides needed experience. Learning is often by trial and error and hypotheses are tested and users learn from the results.
Games have many attributes of effective learning environments. They support active learning, experiential learning and problem-based learning. Games make it possible to use information in context and are inherently learner-centered and provide immediate feedback. However, there is a lack of systematic statistical study of the efficacy of this teaching methodology.
Our understanding of how people learn has evolved over time. Today we think of learning as “constructed”, an active process in which the learner develops his or her own understanding by connecting facts, experience and practice. This constructivist approach to learning is also accompanied by a growing realization that learning is an act of participation. Therefore, experiential learning is a very important process of learning. Today’s students participate in activities, as did previous generations, however those activities have changed and are now based on a platform of experiences found through technology.
The goal of learning is competence – not just awareness. Competence requires factual knowledge and reason. Facts are more likely to be remembered if they fit into a conceptual framework such as a computer based game simulation can provide. Experiential learning enables people to move beyond rote learning and acquire the competence to use and re-use knowledge in new situations.
College educators, especially in the area of business need to continue to adjust their approach to teaching while continuing to deliver the same core educational concepts central to their courses. When educators are able to connect with students, the students become more responsive and motivated, resulting in an engaged learning environment. It has also been argued that motivation and students engagement are strongly correlated to better learning outcomes, (Rowe, Shores, Mott and Lester, 2011).
Games are no longer just for fun; they offer the potential for not only creating an engaged learning environment, but also help to deepen the knowledge gained. It is possible to use games as a pedagogically sound learning tool. The evidence supporting this position continues to expand, as does the favorable literature base affirming the use of educational games. Our experience with a simulated game that motivates and engages students appears well founded.
References
Annetta, L. A., Minogue, J., Holmes, S. Y., & Cheng, M. (2009). Investigating the impact of video games on high school students’ engagement and learning about genetics. Computers & Education, 53(1), 74-85.
Canon-Bowers, J. (2006). The state of gaming and simulation. Paper presented at In Training 2006 Conference and Expo, Orlando, FL.
Coller, B., Shernoff, D. (2009). Video-based education in mechanical engineering: A look at student engagement. International Journal of Engineering Education, 25(2), 308-317.
Cordova, D. I., & Lepper, M. R. (1996). Intrinsic motivation and the process of learning: Beneficial effects of contextualization, personalization, and choice. Journal of Educational Psychology, 88(4), 715-730.
Dimitrov, D. M., & Rumrill Jr, P. D. (2003). Pretest-posttest designs and measurement of change. Work, 20(2), 159-165.
Gee, J. P. (2005). The new literacy studies: From ‘socially situated’ to the work of the social. Situated Literacies: Reading and Writing in Context, (2), 177-194.
Harp, S. F., & Mayer, R. E. (1998). How seductive details do their damage: A theory of cognitive interest in science learning. Journal of Educational Psychology, 90(3), 414-434.
Kapp, K. M. (2012). The gamification of learning and instruction: Game-based methods and strategies for training and education. San Francisco, CA: Pfeiffer.
Malone, T. W. (1981). What makes things fun to learn? A study of intrinsically motivating computer games. Pipeline, 6(2), 50.
Mayo, M. J. (2007). Games for science and engineering education. Communications of the ACM, 50(7), 30-35.
Oblinger, D. G. (2004). The next generation of educational engagement. Journal of Interactive Media in Education, 2004(1), 10.
Papastergiou, M. (2009). Digital game-based learning in high school computer science education: Impact on educational effectiveness and student motivation. Computers & Education, 52(1), 1-12.
Piaget, J. (1962). Commentary on Vygotsky’s criticisms of language and thought of the child and judgment and reasoning in the child. Critical Assessments, (1), 241-260.
Prensky, M. (2007). How to teach with technology: Keeping both teachers and students comfortable in a era of exponential change. Emerging Technologies for Learning, 2(4), 40-46.
Rickard, W., & Oblinger, D. (2004). Unlocking the potential of gaming technology. Paper presented at The Higher Education Leaders Symposium, Richmond, Washington.
Rodley, C. (2005, October 27). Meeting the demands of the NetGen. Retrieved August 3, 2015, from http://sydney.edu.au/news/84.html?newsstoryid=744.
Rogosa, D., Brandt, D., & Zimowski, M. (1982). A growth curve approach to the measurement of change. Psychological Bulletin, 92(3), 726-748.
Rowe, J. P., Shores, L. R., Mott, B. W., & Lester, J. C. (2011). Integrating learning, problem solving and engagement in narrative centered learning environment. International Journal of Artificial Intelligence in Education, 21, 115-133.
Sabourin, J., & Lester, J. (2014). Affect and engagement in game-based learning environments. IEEE Transactions on Affective Computing, 5(1), 45-56.
Shaffer, D. W., Halverson, R., Squire, K. R., & Gee, J. P. (2005). Video games and the future of learning (2005-4). Wisconsin Center for Education Research (NJ1).
Squire, K. (2005). Changing the game: What happens when video games enter the classroom? Journal of Online Education, 1(6).
Taradi, S., Taradi, M., Radic, K., & Pocrajac, N. (2005). Blending problem-based learning with web technology positively impacts students learning outcomes in acid-based physiology. Journal of Advanced Physiology Education, 29, 35-39.
Wouters, P., Van Nimwegen, C., Van Oostendorp, H., & Van der Spek, E. D. (2013). A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology, 105(2), 249-265.
Zimmerman, D. W., & Williams, R. H. (1982). Gain scores in research can be highly reliable. Journal Educational Measurement, 19(2), 149-154.
Acknowledgement
The authors would like to acknowledge the contribution and assistance of Kevin A. Cuneo-Tomasi, MBA, in the analysis and interpretation of the data.
Trackback from your site.