AAPPL Results Analysis

Published by SraSpanglish on May 26, 2017May 26, 2017

I have been using AAPPL rubrics for a few years, but I had never administered the test. Ordinarily I’m a very anti-test type person, but in the face of waning school budgets and waxing xenophobia, I wanted something tangible to point to in order to demonstrate the merits of learning a language, something that would put a feather in our cap that only languages can provide.

That feather is the seal of biliteracy, or here in North Carolina, the “Global Languages Endorsement.” Now a four-course sequence is pretty much impossible here at the early college, but I felt pretty confident a handful of my kiddos could “Pass an external exam approved by the North Carolina Department of Public Instruction”–meaning I1 or better on all 4 tests.

In fact, nearly all of the students who took all 4 tests got at least one intermediate ranking: 2/3 got two or more, and eleven (including six native speakers) actually earned the seal!

Here’s what my first ever AAAPL reports look like:

Background

Three things you should know about my classes before we break down their AAPPL results:

The As and I5 are my native speakers. The I2s and I3s, however, are not.
I made everybody–EVERYBODY–do the interpersonal listening/speaking, coughing up some of my hard-earned teacher dollars in some cases so that I could get a baseline in the one category that was a little interpretive, a little productive.
Students had to pay for their own AAPPL tests. Most were interested enough in taking a stab at the seal of biliteracy (“Global Languages Endorsement”) that they ponied up. If they weren’t, I made my own pretend AAPPL test for reading, listening, and writing for them.

That being said, the unenthused do not account for all of the wild variation in speaking. All of the N1s are among their ranks and most of the N2s; I kind of suspect that enthusiasm was a larger factor than ability in many of those. There were some N4s and even I1s among them, though.

Seeking to understand the variation, I disaggregated the data a bit, cutting out Spanish III and then separating by class. See, I noticed that while 2/3 of first period was intermediate in speaking, only 1/5 of fourth period did. Now 1/2 of 4th took no other test, so that might make their results artificially high in the other categories. Also, I’m certain 2 got ripped off because of recording issues in 4th, although 2 got “UR” for their recordings in 1st as well.

The only explanation I can think of is how the classes were assembled:

First period was made up of 23 Spanish II students. Fourth period only had 15 because Spanish III–7 native speakers who skipped I plus one really dedicated senior–was mooshed in with them.
All of the Spanish II kiddos who indicated they wanted to be actors in the language festival (that got moved on us without warning and which I am not at ALL still bitter about a month later) were in first period–though they only made up half of it.
All of the kids in 4th period Spanish II had opted for singing–which is a group event where those who did not feel as confident could potentially hide out in the background.

Results

Reading

I confess I kind of expected these would be higher, but I can see where going from “just pick out what you understand” and getting scored only on that versus the AAPPL matching format could set some back.

Listening

These results are pretty in-line with what I’d expect, again, with the new, less loosey-goosey format to the interpreting. I had hoped for more intermediate, but this was always a problem area, and I have no doubt that nerves factored in here.

Speaking

What the…? I am fairly certain that recording issues factored in here. One of my very very brightest students logged an N3, and another at N2. They have not performed at that level since the first half of Spanish I. Two other bright ones got “UR” for their official score–I could hear a lot of breathing and cutting out. This kind of makes me want to make writing my default for the future.

Writing

Holy camole! How is it they were just barely intermediate in reading, but knocking the top off in writing? For one, looking at the answers they submitted, I have to say the scorers took the part about “in a way that your teacher and others who are used to the writing of language learners can understand” VERY seriously.

Conclusions

Looking at last year’s results, you can see that the bulk of the class was still at Novice High, with a few stragglers in Novice Mid–exactly where North Carolina expects them to be at the end of Spanish I.

As you can see this year, over half the class is intermediate in just about everything! Listening is a bit of a problem, possibly because of those personalized IPAs last year. Taking out the “unenthused” from the speaking data, the chart looks a lot closer to the others, too:

I’m still concerned about those who were not confident or eager enough to take all 4 sections, and the preponderance of those still stuck near Novice Low in speaking, even though I’m blown away by how many are solid Novice Mids! I would like to point out, though, that last year about 1/2 got I1 or better for reading, but now it’s 2/3–IF we don’t count the native speakers!!

Considering not only the shift from personalized IPAs to AAPPL format, but ALSO the full calendar year without Spanish class, I have to say I’m pretty pleased that growth was mostly consistent!

Now if I can just get some funding next year…

AAPPL Results Analysis

Background