I had an early exposure to computers because we had an Apple IIe computer before I even enrolled in kindergarten. One of my father’s favorite stories occurred when he was packing up the computer for a move. As he lifted the floppy disk drive, he noticed it rattled. Well, when he turned it upside down, a slew of coins fell out. I promptly explained that I had been using the disk drive as a piggy bank.
When I was later enrolled in elementary school and tasked with memorizing the multiplication table, my father, recognizing the importance of drill and practice for learning facts, created a flashcard program on a computer that was a little more updated than my former piggy bank. Educational technology (and computers!) has come a long, long way since my father’s homegrown flashcard program. Now we have educational games, simulations to manipulate equations and produce graphs on screen, and even intelligent tutoring systems that can diagnose deficiencies in a student’s learning and adapt accordingly.
The phases of learning, simply put, include (a) presenting information, (b) guiding the learner through initial use of the information or skills, (c) practice, typically for fluency, and (d) assessment to determine if instruction was effective[1]. A teacher will usually accomplish these phases in the classroom, but educational technology may substitute or extend any single or combination of these phases. My father’s flashcard program didn’t actually present information or guide initial use of multiplication facts. Instead, the program allowed me to practice for fluency and was a mere substitute for paper-based flashcards with the problem on the front and solution on the back.
We recently completed a meta-analysis examining the effectiveness of educational technology programs on mathematics learning. There have been numerous meta-analyses examining this area of focus (at least 24 published since 2013) but each failed to truly examine the purpose and features of the program. That is, not all educational technology programs are the same. For example, my father’s flashcard program just allowed me to practice my multiplication tables. A more robust program might demonstrate how a learner would calculate 8 x 9 through a worked example (i.e., demonstration), allow for practice, provide immediate and elaborative feedback, and then adapt to the learner’s response by either presenting easier problems or advancing at a quicker pace. Those in the instructional design field have long been advocating for researchers and practitioners to recognize it’s not educational technology that affects learning, it’s the design of the program or instructional message that is of the utmost importance.
Back to our meta-analysis. Once we settled on a set of evaluations of educational technology that met rigorous methodological standards, we explored the educational technology programs that were studied. We documented the timing and type of feedback, the role of the program in terms of core or supplementary, the number of phases supported, whether the program adapted to the learner’s response (e.g., personalized) and so forth. Unsurprisingly, we found that features of educational technology moderated the effects on mathematics learning. For example, programs that employed personalization produced much greater effects on learning than those that progressed via a set sequence. Those programs that supported all phases of the learning process produced greater effects than those that served just one or two. These two factors illustrate the importance of leveraging the affordances of technology. While a textbook author or teacher could create traditional or print-based instruction that adapts to each individual student, this approach would be very time intensive. A program that presents instruction (as a supplement to the teacher or as a standalone) that a student can re-watch at any moment, then guides the learner with initial practice, incorporates mini games for fluency, offers on-demand help and support, and then assesses understanding as in a mastery learning approach is a much more robust (and more effective) approach to the use of technology in the classroom.
Educational technology is a rapidly growing market, likely accelerated with the COVID-19 pandemic that demanded students learn from home in an online setting. But, not all educational technology programs are created equal. We see research studies and op eds in the media asking whether technology is effective for education. Of course it can be, but the broad question is meaningless and potentially very misleading. Educators and researchers need to carefully consider the purpose of the program, how the program integrates with classroom instruction, and the degree to which the program was designed with sound, research-based practices.
[1] Alessi, S. M., & Trollip, S. R. (2001). Multimedia for learning (3rd ed.). Allyn & Bacon.
