Measuring the Wrong Thing: Home Computers and Academics

The New York Times ran an article by Randall Stross, a professor of business over at San Jose State, this summer about research on the impact of home computers on academic achievement in low socio-economic status households. Games and entertainment options were blamed for poor results.

The first paper discussed was by Ofer Malamud over at University of Chicago and Cristian Pop-Eleches at Columbia, who studied low income families in Romania receiving a voucher to assist in purchasing a home PC. The control group were families who applied for the voucher but did not receive it.

In a draft of an article that the Quarterly Journal of Economics will publish early next year, the professors report finding “strong evidence that children in households who won a voucher received significantly lower school grades in math, English and Romanian.” The principal positive effect on the students was improved computer skills.

At that time, most Romanian households were not yet connected to the Internet. But few children whose families obtained computers said they used the machines for homework. What they were used for — daily — was playing games.

Stross next discusses work by Jacob L. Vigdor and Helen F. Ladd over at Duke University, who performed similar research in the United States for the National Bureau of Economic Research. Their study noted after broadband became widely available in North Carolina, math and reading scores plummeted in low SES homes.

The Duke paper reports that the negative effect on test scores was not universal, but was largely confined to lower-income households, in which, the authors hypothesized, parental supervision might be spottier, giving students greater opportunity to use the computer for entertainment unrelated to homework and reducing the amount of time spent studying.

The North Carolina study suggests the disconcerting possibility that home computers and Internet access have such a negative effect only on some groups and end up widening achievement gaps between socioeconomic groups. The expansion of broadband service was associated with a pronounced drop in test scores for black students in both reading and math, but no effect on the math scores and little on the reading scores of other students. In the report, the authors do not speculate about what caused the disparities.

Last, the article touches on the final report (large pdf) from the Texas Center for Educational Research on the state’s one-to-one laptop pilot, which indicated modest improvement on some test scores in the experimental group over the control group.

THE one area where the students from lower-income families in the immersion program closed the gap with higher-income students was the same one identified in the Romanian study: computer skills.

Catherine Maloney, director of the Texas center, said the schools did their best to mandate that the computers would be used strictly for educational purposes. Most schools configured the machines to block e-mail, chat, games and Web sites reached by searching on objectionable key words. The key-word blocks worked fine for English-language sites but not for Spanish ones. “Kids were adept at getting around the blocks,” she said.

-+-

Unfortunately, all these studies are measuring the wrong thing. This is equivalent to giving a 1920s farmer a new radio, then measuring the increase or decrease in his crop yield. Why would you expect his crop yield to increase after he’s been given this new communication technology?

A better tool for increasing crop yield would be a tractor, which could plow more than his horse. But, the radio — the highest available technology at the time — would be considered a failure because it did not directly result in higher crop yield.

The farmer would have received timely news and weather reports which perhaps would have indirectly affected his yield. Also, the farmer and his family would have been exposed to an increase in entertainment, experiencing the latest from New York and LA. This cultural acclimation would be accompanied by a wider appreciation of world and national events, which would have had no effect on farming but would have benefited the farmer and his family nonetheless, albeit indirectly. Perhaps the farmer would learn something useful from an agricultural program, but again, looking for a direct increase in crop yield completely misses the point.

Ultimately, computers at home or in one-to-one programs should never be expected to increase academic scores, just as a farmer’s radio shouldn’t be expected to directly increase his crop yield. Home computer functions simply do not correlate well to traditional test taking. But computers do have value in several other areas, in the creative programs and games they run, and the communications capabilities they offer to students. But discerning whether an experimental group earns higher test scores than the control group is simply measuring the wrong thing.

References:
Stross, R. (2010, July 10). Computers at home: Educational hope vs. teenage reality. The New York Times, p. BU3.

Vigdor, J. L, & Ladd, H. F. (2010, June). Scaling the digital divide: Home computer technology and student achievement. National Bureau of Economic Research. [Online.] Available: http://www.nber.org/papers/w16078

Beyond Second Life

Tony Bates refers us to an article in the Chronicle by Jeffrey Young: After Frustrations in Second Life, Colleges Look to New Virtual Worlds.

The article details the challenges universities have faced when trying to integrate SL into lessons. Consequently, they are exploring other venues for instruction that offer more controls and fewer distractions.

Sometimes this leads to additional problems. Few companies in this specialty are as established as SL’s parent, Linden Labs. Some have gone broke, taking virtual classroom space with them when the plug was pulled.

A couple of promising efforts either underway or coming this year include Open Cobalt from Duke University, funded by the NSF and the Mellon Foundation, and OpenSimulator which leases virtual space for instructional purposes.

Several initiatives are out there to offer classroom space to educators at no cost to them. Young notes Aaron E. Walsh over at Boston College hosts about 2,000 educator accounts on Education Grid, a world devoted to online instruction that Walsh set up through his project, the Immersive Education Initiative. The mix on Education Grid is about 80% university profs and 20% secondary teacher accounts. The IEI leases space from OpenSimulator.

To counter the academic exodus, SL now offers a version of its software universities can host on local servers, which effectively prevents outsider access and the ability for students to wander over to red light districts.

It’s interesting to see the idea mature from a fanciful notion, to gritty reality, to something tailored for specific educational needs. For instance, initially universities set up virtual spaces identical to real world lecture halls. This resulted in unwieldy virtual space that was hard to navigate. It’s also interesting to see the day coming when SL will be considered “old hat” by professors and students, who will be using newer, more robust environments geared specifically for virtual education from the ground up.

IBM Helps Universities to Innov8

Here’s a nice article by Julie Moran Alterio from Gannett, appearing in the Asbury Park Press (“From the Jersey Shore to You”) on IBM’s new business leadership skills game, Innov8. Taking a page from the military’s America’s Army game, IBM hopes to instill desired skill sets in those “fuzzy” areas that games are so good at teaching, such as leadership, teamwork, social skills, and real world problem solving.

Innov8 came about through IBM’s corporate case challenge, which involves B-school teams from two universities competing with one another to provide a solution to a business problem. IBM VP Sandy Carter noted that 40 of the 44 teams from Duke and U. North Carolina suggested using a videogame to help people develop needed skill sets for business acumen. Since January, Carter has shepherded development of Innov8, using some of the students from the case challenge to help design the game. Pilot studies were completed earlier this year and the game is now ready for prime time, to be offered free to 2000 universities worldwide.

Gameplay should be familiar to World of Warcraft players. It’s a 3-D virtual interactive environment (VIE) with human avatars. Players assume a female avatar tasked with solving various business-scenario dilemmas. NPCs provide helpful dialogue.

Jim Lawler, an information systems associate professor over at Pace University, is given prominent mention. He worried game dynamics would be difficult to master, thus detracting from lessons. However, he was won over after quickly mastering the game. His key quote: “Enrollment is lower in computer science and information systems nationally. This is what schools have to do, integrate these kind of games and tools.”

David Rejeski is also mentioned prominently in the article:

More corporations and the U.S. government are starting to see the potential of games to teach serious subjects, said David Rejeski, director of the Serious Games Initiative at the Woodrow Wilson International Center for Scholars in Washington, D.C.

“The thing about games that’s really nice is you can fail softly,” Rejeski said.

The Apply Group, a high-tech consulting firm, predicts that one in five of the Global Fortune 500 will adopt gaming for learning by 2012.

My take: IBM has long been at the forefront of top companies that “get it” with gaming and Web 2.0 technologies. For instance, the company has obtained considerable virtual real estate in Second Life and holds online meetings there with avatars showing up from personnel spread across the globe. It is heartening to see this effort to help train business students in appropriate skill sets. Offering the game to universities free of charge is a good way of helping B-schools graduate students with the knowledge and skills needed by IBM and other big corporations.

References:
Alterio, J. M. (2007, November 26). New video game teaches students business and computer skills. Asbury Park Press. [Online.] Retrieved November 28, 2007 from: http://www.app.com/apps/pbcs.dll/article?
AID=/20071126/BUSINESS/711260306/1003