The new PISA! Still Reviewing and Reading…

So much progress, so much exemplary assessment. So much data.

This will take some some number of weeks to process…. and I will try to link to the best bits. So far the Economist treatment is looking pretty thorough.

And of course the PISA web site itself has a visualization tool… i think…

The interactive “problem solving” exercises — using “MicroDYN” systems and “finite-state automata” in particular look really interesting.

The test is here;  more results and more start here.


Reuters shines some light…

A new piece from Renee Dudley and Reuters.   Some I know were disappointed by “yet another multiple choice test”, or with the continued dominance of “the number-two pencil”, while others were nevertheless satisfied with this somewhat new version of the test if only for a couple (handful?) of content-related changes they were happy to see.  Some less so.  Some were enraged.

Read “Crash Course: College Board faces rocky path after CEO pushes new vision for SAT“.


Another Kristof Krissis – TIMSS results used and abused

The headline pretty much tells the tale:

Nicholas Kristof Is NoTIMSS-2011math rankingst Smarter Than an Eighth-Grader – 

American kids are actually doing decently in math and interpretation, but he’s not.

By Eugene Stern

You can read the original piece here or the Slate version here, but the summative bit gets right to the point, after the why-should-we-be-surprised cherry-picking of two extreme examples to misrepresent the results of the 2011 Trends in International Mathematics and Science Study (TIMSS)  assessments; as shown here (see graphic or follow link to source).

Eugene Stern wraps up with these observations:

But Kristof isn’t willing to do either. He has a narrative of American underperformance in mind, and if the overall test results don’t fit his story, he’ll just go and find some results that do. Thus for the examples in his column, Kristof literally went and picked the two questions out of 88 on which the United States did the worst, and highlighted those in the column. (He gives a third example too, a question in which the U.S. was in the middle of the pack, but the pack did poorly, so the United States’ absolute score looks bad.) Presto! Instead of a story about kids learning stuff and doing decently on a test, we have yet another hysterical screed about Americans “struggling to compete with citizens of other countries.”

Kristof gives no suggestions for what we can actually do better, by the way. But he does offer this helpful advice:

Numeracy isn’t a sign of geekiness, but a basic requirement for intelligent discussions of public policy. Without it, politicians routinely get away with using statistics, as Mark Twain supposedly observed, the way a drunk uses a lamppost: for support rather than illumination.

So do op-ed columnists, apparently.

The propensity of some in the public eye to yearn for “stories” and narratives to support their heartfelt policy positions to the degree that they mis-understand statistics or misinterpret information is unfortunate;  but here we have the conflation of two trends — one pointed up by books like “The Manufactured Crisis: Myths, Fraud, And The Attack On America’s Public Schools” from 1996;  the other by googling “kristof disingenuous“.

In any event, if we want “trends”, 2011 data probably isn’t the place to start.  Sadly, TIMMS 2015 numbers are a probably over a year away.  But meanwhile, there is no shortage of new educational assessment data from the 50 states to dig into, so perhaps policy champions can go there and try, despite the tempting sound-bites and stories, to resist making stuff up.