EDPuzzle and Integrated Performance Assessment

My first EDPuzzle IPA mashup went pretty smoothly this week. I think it really helped students break down the listening into manageable chunks to make the overall task approachable. I’m also really glad that I chose to add in my little summary audio commentary at various intervals because students were still exposed to a speedy Spaniard, but they also had something that gave them a foothold on the task.

Now for each segment, I not only had my summary comment, I had 2 open-ended question slots: one for a question to see how well they got the main idea, and one for kids to show what they know by picking out whatever words they could hear and telling me what they mean.

First of all, this progress tracking feature is super cool:

Notice nobody was finished watching, but I could tell who hadn’t even started because they weren’t pink yet! So I’d saunter over, make sure their computer’s starting right, they followed the log-in instructions…and that they’re not trying to do last Monday’s homework instead. If you have multiple choice questions, it’ll even tell you how well they’re doing on this page: Help, Good, or Excellent! 

Then after the assessment is complete, the grading’s pretty simple.

Check out how it collapses the ones you’ve graded! And you have all of the answers to the same question together for quick comparison!

Grading
There’s a comment spot under each answer, and then you click the check mark if the answer is good, the X if it’s not. This is tricky, since I’m grading IPAs holistically using the AAPPL interpretive rubric. Do I put a check mark only if they reach the desired level? Do I put an X if they interpret anything at all inaccurately?

I decided since the rubric is really where the grade will be, then the X should only communicate if A) they didn’t tell me what the words they listed actually MEAN in English to prove they UNDERSTOOD or B) they were WAY off with their interpretation or missing essential components of it. Aside from that, I jotted a note or two in the comment to explain the problem if they were close or to clarify false cognates, i.e. lectura.

As for the time to grade, it took me a little half an hour to go through each question and list of things they understood by the second class, and then around another half hour to compare to the rubric. Notice how once I did grade them, it sorted them into Help, Good, or Excellent! for me. With my quick color-coded analysis, I guessed the “Good” would be my N1s (since I did give them a check mark if they weren’t WRONG) and those in the 92% range within  Excellent! would be my N2s.  To evaluate more carefully, I did go ahead and click on their breakdowns–which are SUPER cool:

Notice how you can see how many times they rewatched! This will come in handy when we start breaking intermediate ground! The “Next Student” button above their breakdown was also a godsend.

I looked at my notes where I had indicated problems as well as the frequency of phrases and sentences in their responses (HINT: definitely make note of how many phrases/sentences you see when you are commenting) and compared to the AAPPL rubric for the final analysis. Overall, the color coding was not terribly accurate for my purposes, as most of the 92’s were actually N1 (though one was N3), and several of the 100s were N2.

Reflection
Everyone (who didn’t have computer issues) was able to answer all 6 questions for a 2-minute video –plus maybe another minute of my audio commentary–within an hour. Some finished as quickly as 30 minutes in (HINT: this is where personal goal and portfolio assignments come in handy, so they can keep productive while they wait for their project partners for the interpersonal step.) In retrospect, I think 6 questions was a bit much for the first attempt for novices, and along with my audio comments, it really stretched out the video to a longer text than they are prepared to deal with efficiently, so I might aim for 1 1/2 minutes and 4 questions next time.

I have also asked the PHENOMENAL EDPuzzle staff how students could potentially display their answers for their e-portfolios. A screenshot of their results might do for now, but how cool would it be if the answers could show at the different intervals just like they did when they were answering the questions? Fast-forwarding and skipping would be essential for display purposes, though.

One thing I know for sure from this assessment, though, is we need to do more listening exercises with authentic texts. Even my Novice Highs and Intermediates were a bit blown away by the accent and the speed. I did not have nearly as many kids consistently Novice Mid as I would have liked at the end of the first 6 weeks, but I think I could also increase their success by selecting the IPA video before writing the unit TPRS story to make sure that essential vocabulary is also repeated several times in my voice–and their voice–too.