From Facts to ARTifacts: Balancing Content and Context in Technology-Assisted Assessment

“I’d like to use my lifeline…”

I’ve really not watched Who Wants to Be A Millionaire more than a few times, but my pop culture literacy is pretty high, and I am quite familiar with the concept of “lifelines”. Apparently the show has dropped “Phone a Friend“, “Three Wise Men” and “Ask the Expert” and now only allow jumping the question and asking the audience.

Listening to Julian Stodd’s podcast this morning on how learning is changing and the learning spaces needed in organizations (particularly in respect to social learning / co-created stories), I was struck by a statistic he mentioned – that

most smartphone users unlock their phones about 80 times a day.

To be frank I probably use mine even more (this might make a good self-study). It is my vade mecum – my “go with me“. My memory has deteriorated due to age and the after effects of chemotherapy (called “chemo-brain“). But that’s quite OK, since the things I look up or record as a reference are “facts”, not “thinking”. There are, as Ewan McIntosh often says, the “Google-able questions”…the stuff we need but that does not necessarily

arouse our minds.

They are the makeup base to the vibrant colours of lipstick and eyeshadow…the foundation garments to the grande ensemble…the canvas to the final masterpiece.

But I have to know where to look…

I have to know who can help me in my pursuit… (I discuss this more in my post about knowledge as connection).

I need to consider the validity and usefulness of my sources.

After the finding (or retrieving) and assessment of information I need to store it somewhere to access in the future. Along with that I need a system – whatever method that works for me – for organizing and managing this “knowledge”. In essence,

our connections are our lifelines

This was perhaps always true…it’s difficult to survive as a loner. From early times we benefitted from membership in a tribe and the division of labour that that social structure afforded. Nothing much has changed in the nature of connected knowledge (and survival)…as independent as we may perceive ourselves to be, we all need our “lifelines”.

What has changed exponentially is the degree of connectedness, as well as the acceleration of transmission. I don’t have to wait for the library to open to do my research…I don’t have to wait for people in my city to wake up if I have a question at 4 am, because I’m connected via social media to people in all time zones. I don’t have to search laboriously through a text because I can use tags to find content.

And that’s a good thing. It doesn’t diminish my intelligence – it enhances it.

It leaves room for the big thinking.

And isn’t that what we want from students (or any learner of any age?)

As educators, we are tasked with the duty of guiding students in their pursuit of knowledge and then assessing that acquisition of knowledge in order to “report” it to some higher authority. In fact, just today a friend on Twitter reminded me:

I agree not everything can be a collaborative endeavor, but I also think that certain structures can be set up by the student himself and as a cohort of learners (including the teacher in a “co-learner” role), that will facilitate individual expression of learning.

This morning I chanced upon a blog post by Graham Brown-Martin entitled The Future of Learning: What is the Purpose of School and the Role of Edtech?

When teaching the International Baccalaureate course “Theory of Knowledge” for several years, I often posed that very question (the point of school?) to my 16-18 year old students. They were particularly intrigued by the ways in which “knowledge” is changing due to technology (they read Clive Thompson’s Smarter Than You Think and David Weinberger’s Too Big To Know), as well as the rethinking around education that is happening as a result (for this I would recommend Seth Godin’s Stop Stealing Dreams: What is School For?)

While I’ve always been a constructivist and, as I became more immersed in “edtech” and my personal use of social media, a connectivist , my recent work has propelled me in the study of creativity – how it works, how to manage it, and how to use it.

That led me to rethink something…

I think we can’t dismiss the importance of learning (and therefore teaching) content. As I illustrated in my post about Remix, I still believe the stickiest learning comes from making– especially if it’s coupled with some juicy reflection. But whenever I speak about remix or creativity for that matter, I emphasize that it comes down to dot-connecting…

..but first you must have the dots!

These “dots” are the bits and pieces of knowledge…we re-contextualize them by mashing them up, or interpreting them in different ways…

Perhaps we can consider three types of “knowledge”:

known stuff

(we know off the top of our head, from studying or from experience)

searchable stuff

(things we have an inkling about but need to look up to be sure)

wonderable stuff

(the complex things – often with no definitive answer – that we may form theories about based on a variety of knowledge experiences, including experiential as well as academic)

We remix ideas from the “known” and “searchable” stuff to shape our thinking in the “wonderable” stuff. Ideally, the fruit of our thinking is insightful and at times even “original” (or at least “unconventional”).

We move from “regurgitate” to “create”.

This all leads me to argue for something I tried out successfully my last 10 years or so of classroom teaching –

open resource assessments

By “open resource” I mean open notes, open book, open Internet (including social media). This is different than groupwork or a team challenge. Each student is held accountable for his own demonstration of learning, although students may be accessing a co-created resource like a class wiki or peer blogs. In fact, I’ve even asked students to purposefully refer to their classmate’s blog posts and hyperlink them in their final exam essays. When offering students a chance for all open assessments there are a few things to consider:

It’s Harder Than They Think

Prepare students by clarifying that this is not an off-the-cuff type of task. They need to organize sources ahead of time. Personally, I think it’s imperative that students are not clueless about what types of things they might be asked to do or know. We are not there to “trick” them, we are there to help them demonstrate and articulate what they’ve learned with as many tools as possible. If a student is unprepared (as in doesn’t have all his notes or has no idea what key words he should use in a search), then he will waste a lot of time on the retrieval process rather than on the production.

Personal Voice Reigns Supreme

Perhaps the most important is that each assessment task – whether it be text, video, audio, or some sort of creative project – be filled with individual voice. If all students are, for example, drawing from the same list of YouTube videos as a resource, they should each express their thinking as uniquely as possible. Of course, this is something to be practiced throughout the course with formative assessments, so they have a good handle on it when it comes to any summative piece.

This is why I love creative assessments like different forms of digital storytelling – and why I think Remix is the highest form of learning. It’s also why I loved to use video blogging (aka “vlogs”) as a final exam or as a reflection on a unit. With vlogs, students get personal…I’m not sure if it’s “confessional” nature of the medium itself, or perhaps that they can do it in their own, comfortable space (like a bedroom at 11 pm). Vlogs show voice intonation and other paralinguistic cues (body language) that reveal emotion. I want to know not only what my students know, but how they feel about it – how poignant something was for them.

(please check out my G+ community on Vlogging For Assessment and Critical Reflection)

Breadcrumb Trails are a Must

As with any remix, you should seek to overtly acknowledge the lineage of their sources and influences. I say “influences” because even if not directly quoted, works and people who have influenced your thinking and creative process should be referenced. This is so whoever is assessing your work may see how your thinking developed and why, and it’s also good for replication (I love how the science community has always done this with their research, so that one can replicate an experiment exactly).

So..what should we “know”?

After noticing how Wired Magazine distilled things into four categories, I thought this might be a great way to organize curriculum. I think any type and level of learner would benefit from any or all four of these categories:

People to Know

Hottest Ideas

Essential Works

Must-Follow Feeds

This works for both historical and contemporary context as well. That being said, I believe a thematic rather than linear approach to any subject is the best (though with History I tended to organize chronologically within the theme, so that if we were studying “Power” Henry VIII came before Stalin, naturally).

I want to end by sharing one interesting final exam I gave to my World History students.

The Artifact Bag Assessment

I think this approach could be used in any subject and done completely offline (the open source being books and notes) or digitally (with access to the Internet and any class websites or online resources).

What is it?

Each student gets a different bag (I used large ziplocs) filled with printed “artifacts” and a set list of questions. I organized my questions by theme. It’s pretty meaty, but students should have practiced these types of analyses prior to the exam. Artifacts included photos of artwork, photography pieces, cartoons and illustrations, pieces of texts / quotes (primary and secondary), video links and sound clips (music and speech). There were pop culture references mixed in with purely historical ones, so that students could see how the events of History have influenced our world today.

Why is it fun?

The neat part is that each person gets a different combination of artifacts in the bags- I had enough artifacts that no two students got the exact same thing, although they might share a resource or two.

Why does it work?

The questions are scaffolded from “known” and “searchable” to “wonderable” (and remixable). There’s more Why? than What?, and more What if? than Why?

Moreover, there’s no possibility for “cheating” ( I really hate that term), because students are allowed all access to resources (except fellow students in the room). They can’t chat with others in the room (though you could make this a team exercise), but may draw from co-created resources.

The key is sense-making.

Learners use curated resources to make sense out of something they are confronted with, and to consider the relevance of the artifact to human knowledge in general, as well as to their own lives.

The assessment demands “basic” knowledge, but expects the student to push further in their thinking.

(just thought I’d throw this in..I did include an essay in addition to the artifact analysis..it’s not necessary but perhaps gives more context..in other words, you could “beef it up” with an essay challenge)

Change It Up

This particular assessment was writing-centric. It could easily be a vlog, or some other type of creative project that tasked the learner with making something as well as providing commentary. Some students wrote by hand and others on their laptop. If you wanted to make it all digital, you could provide each student with a customized digital folder of artifacts, or even have students create “artifact” collections (approved by you) for each other.

If you wanted to focus on one type of artifact, such as all images, sound clips, or quotations, you could.

If you try it, please let me know in the comments or on Twitter @amyburvall.

I hope there will come a time when we don’t demonize our devices – and more importantly, our connections – in the classroom. The more we teach students how to maximize the use of these tools rather than use them only for frivolous purposes the more they will be savvy and sophisticated individuals who can balance the time spent on the screen with the time spent “in the moment” (for want of a better phrase because sometimes I’m very much “in the moment” when I’m on social media). They can develop etiquette around them, and negotiate in various learning spaces with ease and in a gentile manner.

Because what is wrong about phoning a friend…even if the friend is AI?

#julianstodd #connectedlearning #smartphone #curation #digitalstorytelling #connectivism #sociallearning #vlogs #knowledge #assessment #artifactbag #ewanmcintosh #remix