Week 3 - DIY - Blended Assessments of Learning

Reflections 

Good exercise.. Demanded a detailed review of assignments. I had to rationalize their nature, sequence, and design ... especially as to how they built upon one another and addressed the learning objectives as stated in my course blueprint and module design sequence. To do this correctly, would require a great deal more time then I was in a position to give. The iterative nature of this activity that the Blendkit 2016 - Chapter 3 materials eludes, certainly becomes evident. I have also come to realize just how rich and powerful the interactive video strategy that I have only recently discovered and implemented is going to be. This assignment opened my eyes to this. It also forced a more detailed examination about how it can best be administered.

Note that I can also see tremendous value in using webbased screencasts to show students (especially second language learners)  how to get started on a project or assignment - whether that is in a face to face or online context. The multi modal nature of them plus the 24/7 access to them helps make instructions so much more clear to students and at a time and place of their convenience. As a result, I have begun to experiment with recording any "how to" (demonstrations) that I have done in class. I have also the luxury of seeing when and how frequently they are being used. Student response to them is almost always excellent too. 

I also wanted to note the pragmatic side of using objective based quizzes. They certainly have the capacity to lighten an instructor's marking load albeit typically for low level cognitive skills (though the examples presented in the Effective Assessment Examples article seem to suggest that this not always be the case - worth a follow up). But what I find such quizzes are especially useful for is how they better support trial and error than when done via an in the class activity. However, I've also come to learn the need to design these in such a way that they do NOT support random guessing. I have found that I can do this by only giving students access to a final score of a quiz .. so they are not clear as to which questions they got incorrect. The quiz also needs to be based on a battery of questions thus giving students a chance to redo the quiz but with little likelihood of getting the same questions. It should also be set up to support a minimal mastery rate (e.g. 80%). I have found that the strategy promotes frequent use of an online quiz and as a result, the quiz in effect becomes more than an instrument for collecting scores, but a learning activity in itself.


Background

Will target my current POTB COL 140 course at Zayed University for blended learning.

POTB attempts to take the current regular COL 140 - English Composition 1 Course that is given over a regular 15 week period and to compress it into 7 weeks. This puts significant pressure on students (and me the instructor) to cover the same material / syllabus but over a much shortened time frame. The use of blended learning strategies may permit students to engage with course concepts more frequently, in a manner that is more engaging, and at a time and place more convenient to them (outside of class). 


Interactive strategies (tools) that I am likely to use

• real time collaborative document / lesson plan (etherpad - example, Google doc) 
• interactive video + reading comprehension, sentence writing (Edpuzzle - example)
• synchronous paragraph writing (Padlet - example,  etherpad)
• synchronous sentence writing (Today's Meet - example)
• class review (Kahoot) 
• vocabulary development (combination of Google form - to collect student written definitions; Quizlet - taking student written definitions to author a Quizlet) 

My course blueprint

• Wk 1 - DIY - Blendkit 2016 - blended learning blueprint - COL 140 - POTB - Graph - James Buckingham - ver 1 (pdf) 

My module schedule 

•  Wk 2 - DIY - Blendkit 2016 - Module Schedule_COL 140 POTB - James Buckingham (pdf)

 
---------------------------------------------------

Plan for completing Week 3 - DIY

• read Week 3 - DIY assignments
• review reading 
• review my notes 
• review examples 
• complete Week 1 - DIY assignments 

read Week 3 - DIY assignments

1. Create Assignment Instructions [pdf file; size=102kb]For each graded assignment through which you will assess student learning,  use this process to create written assignment instructions to share with students.
2. Configure Online Quizzes [pdf file; size=78kb]Online quizzes are one of the most common assessment methods employed in blended courses. If you decide to employ online quizzes in your blended course, use this checklist to work through the logistics of setting up your online quizzes.
3. Revise Course Blueprint, Syllabus, and other course documents as appropriate.As you work through the logistics associated with your online quizzes and other graded assignments/assessments, you may find that you want to adjust your original plans as articulated in your earlier documents. Revising them will allow you to stay focused on designing one, consistent whole.

review reading 

• Week 3 - Blended Assessments
• Week 3 - reading reaction  

review examples (found in

• Using Bloom’s Taxonomy to Create Multiple-Choice Questions
• Effective Assessment Examples
• Question Improvement Suggestions
• Sample - portfolio 

FINAL results submitted 

• Wk 3 - DIY - Blendkit 2016 - Module_interaction - COL 140 POTB - James Buckingham (pdf)
• Wk 3 - DIY - Blendkit 2016 - Module interaction (detailed) - COL 140 POTB - James Buckingham 

Week 4 - reading reactions - Blended Content and Assignments

Questions to Ponder

• In what experiences (direct or vicarious) will you have students participate during your blended learning course? In what ways do you see these experiences as part of the assessment process? Which experiences will result in student work that you score?
• First of all ... what do we mean by direct or vicarious experiences? Direct seems to suggest that the student listen, digest and respond to a question with a direct (predictable or desirable response)... while vicarious suggests that the student do much the same but with a less predictable, more creative outcome (no less desired). The focus here in on experience. I would ideally seek to have students demonstrate an understanding of something through "direct" response and eventually encourage them to become more discriminatory in the use of a "concept" or "idea" with a more "vicarious" experience (one more authentic). The "direct" response would lend itself more to Formative assessment .. while the later would lend itself to being more Summative. 
• How will you present content to students in the blended learning course you are designing? Will students encounter content only in one modality (e.g., face-to-face only), or will you devise an approach in which content is introduced in one modality and elaborated upon in the other? What will this look like?
• I've cheated a bit this time. Based on my interpretation of the readings, I should be presenting the course in modules and standardizing the way that I present each module (e.g. headings and subheadings) so that the student comes to know what is expected in each. I should like to design these so that online activities complement face to face activities and vice versa so students also become familiar with these processes and can then focus just on the tasks at hand (and not be burdened with having to learn a new approach or sequence). Would expect to invite students to view online an introduction to a concept .. perhaps reflect on it, then have them come to class where these ideas are presented in more detailed, then invite students to practice these concepts both in class and online (graduated in difficulty) and then as an instructor observe student online efforts to pinpoint general problems that can be discussed in class but then also work with problems individual students may be having.  
• Will there be a consistent pattern to the presentation of content, introduction of learning activities, student submission of assignments, and instructor feedback (formal and informal) in your blended learning course? How can you ensure that students experience your course as one consistent whole rather than as two loosely connected learning environments?
• I think I've already answered this question in my response directly above. Yes by trying to standardize the presentation, types of engagement and methods of review, I breed familiarity and comfort in students thus focusing their attention less on technology and more on the concepts to be learned. 
• How can specific technologies help you present content, provide meaningful experiences, and pitch integration to students in your blended course? With your planned technology use, are you stretching yourself, biting off more than you can chew, or just maintaining the status quo?
• I can choose technologies that help provide students with the appropriate type of interaction that I'm seeking from them and at a time, manner and place that is most appropriate for them too. Case in point, how to invite students to write more frequently and get feedback more frequently so that they can begin to master writing? I do so by inviting them to use tech tools in class that invite them to write anonymously yet still get feedback at the early stages of their writing.... and then gradually move towards making their work more public as they are expected to move to making their writing "more public". 

My notes

Seems to be stating the obvious but online work needs to be relevant to in class work and/or experiences .. and it needs to be perceived as such by the learners. In other words, "integrate online with face to face, so there aren't two separate courses". To do otherwise, invites the question from learners "Why are we doing this?"   The pay back? Student should be able to be more engaged in the learning practice .. learning activities are not add ons but actually reinforce earlier presented concepts.

But how? Article seems to stress the importance of using a basic framework and repeating the use of it so students become more familiar with it and actually come to expect it. Recommendation? Present materials or topics in a module form and even structure each module in a consistent organizational manner .. so the focus is on the content of the module rather than on the need to relearn the organizational structure of a module each time  (e.g. how it all fits together)

I've seen all sorts of methods for this.. I've even used a few of my own. This one was (and still is) used for introducing a "learning tool" for promoting targeted integration into a teacher's practice.

• Introduction 
• What is it? (description) 
• Why bother? (rationale) 
• What's in it for me? (benefits, theory)
• How can I realize those? (methods) 
• Obstacles (managing problems) 
• What others have said? (testimonials)  
• Your thoughts? (invite sharing of experiences - successes / failures; reinforcing or re-defining best practices)
• References 

Another more simple but so elegant framework is this one (currently being used to help in the design of a badge earning ecosystem). 

• Learn (acquire knowledge / understanding of a topic)
• Do (practice applying / mastering the topic within a safe circle of learners / supporters) 
• Share (sharing the mastery with others outside a circle of learners; sharing the experience with peers)  

There is a mention of Dee Fink who seems to echo the above with

• plan for student encounter with content (learning) 
• participate in active experiences (do) 
• share personal reflections / experience (share)  

Then there was the topic of learning activities for active learning (chart below). I reflected on the work that I've doing in COL 140 and which of these activities may already be in place.

I noticed how I was already involving students in direct experience of writing and responding to a prompt in a Basic Composition course that I teach, through the use of Today's Meet (sentence, short response), Padlet (paragraph response), and EdPuzzle (listen, learn, reflect, respond on short answer). Also noted was how the mobile learning project in the same course offered much more freedom to experience and apply what the student had learned to realize a something much more engaging and potentially authentic.

I have noticed how consistent use of these tools makes it easier for students to respond .. especially as they become more comfortable not only with using the tool but in how I use it to benefit them.

Closing point 

I found much of this week's content was already somewhat familiar to me but that was also somewhat reassuring. Some of the key concepts - consistently using a design framework, keeping f2f and online activities relevant and supportive of one another - were already evident to me. What was perhaps helpful was the need to clearly identify the interaction between student and teacher if one hoped to  stimulate a desired outcome. For example, the need for the teacher to be clear about the type of  feedback to give.... if we are expecting a certain response to a question from a student. It seems so obvious and yet it needs to be stated.

Video

References

A Working, Self-Study Guide on: - GuidetoCourseDesignAug05.pdf. (n.d.). Retrieved from https://www.deefinkandassociates.com/GuidetoCourseDesignAug05.pdf

Fink, D. (n.d.). A self-directed guide to designing courses for significant learning. Retrieved from https://www.deefinkandassociates.com/GuidetoCourseDesignAug05.pdf

Hirtz, S., & Harper, D. G. (2008). Education for a Digital World: Advice, Guidelines and Effective Practice from Around Globe. Commonwealth of Learning, Vancouver and BC Campus, Canada. Retrieved from http://dspace.col.org/handle/11599/52

Kaminski, J., & Currie, S. (2008). Planning your online course. In Education for a Digital World: Advice, Guidelines, and Effective Practice from Around the Globe. Retrieved from http://www.colfinder.org/materials/Education_for_a_Digital_World/Education_for_a_Digital_World_part2.pdf

Liyoshi, T., & Vijay Kumar, M. S. (n.d.). Opening Up Education. Retrieved March 25, 2016, from https://mitpress.mit.edu/books/opening-education

NCAM. (n.d.). Invent + Build / Web + Multimedia / Accessible Digital Media Guidelines. Retrieved March 25, 2016, from http://ncam.wgbh.org/invent_build/web_multimedia/accessible-digital-media-guide

Online @UCF. (2016, March 1). Open Source Digital Content - List | Diigo. Retrieved March 25, 2016, from https://www.diigo.com/list/onlineucf/repository

Survey of Technology Tools for Student Engagement. (n.d.). Retrieved March 25, 2016, from https://docs.google.com/document/d/1ttqi7NuVhEAdhcki_G-y-k13ef-0sw9Y74Mg1m9uoHc/edit?usp=sharing&usp=embed_facebook

Week 3 - reading reactions (before / after reading)

This week was a bit of a disappointment but perhaps it should be viewed as reassuring too. Much of what was said and written was largely already known to me prior to taking the Course. A few new approaches and perhaps a refresher on ideas that I hadn't come into contact in quite some time. 

Table of contents

• Focus Question 
• Objectives 
• Questions to Ponder 
• Notes from reading 
  • Not a whole lot new learned here 
  • There were a few bright spots however
    • non graded multiple choice quizzes
    • presenting video examples or explanations 
    • presenting non credit, online practice exams 
    • key online assessment tool features 
      • T/F alternative
    • informal assessment design checklist 
    • one sentence summary 

Focus Question

How will I determine students are learning?

Direct creation of learning artifacts by students that demonstrate their understanding of concepts through their application of concepts to realize something. Ideally whatever is produced is authentic in the sense that it invites application to a situation / simulation similar to one where these same skills and concepts would need to be applied. 

 Authentic assessment—assessing student abilities to apply knowledge, skills, and attitudes to real world problems—is not only possible in an online environment; it is getting more popular. (Thompson, n.d.)

Objectives

After completing all of this week’s activities, you should be able to:

• Determine  assessments appropriate for the learning objectives you’ve established in your blended course
  • assessments are aligned with learning objectives; methods lead practicing skills and competencies that are to be tested in assessments (everything connects) 
• Revise existing course documents as appropriate based upon assessment decisions
• Configure online quizzes for your blended course
  • self directed, formative assessment quizzes 
• Create written student instructions for each graded assignment in your blended course 
  • checklist

Questions to Ponder (before reading)

• How much of the final course grade do you typically allot to testing? How many tests/exams do you usually require? How can you avoid creating a “high stakes” environment that may inadvertently set students up for failure/cheating?
  • How much of the final course grade do I typically allot to testing? For the most part, it has not been determined by me but by those responsible for the course syllabus. However, as a rule, I would want to opt to have assignments weighted equally throughout the course and also have assignments be sequenced in such a way that they build upon earlier assessments and build towards subsequent assessments. 
  • I've learned that putting more than 20% weight on any one assessment typically means making it "high stakes".  The draw back? Performance anxiety - a one off "disaster" can effect a student's final grade - thus explaining the "high stakes" nature of it. We want students to demonstrate their learning and unless managing "stress" is considered an important part of the assessment, a learning objective, then it should be minimized as much as possible. How to avoid it? Simple. Don't make one assignment so high stakes. Another option is to design an assessment so that it is ALSO a learning experience. Even invite students to make notes or drawn on notes that they think will help support them in the completion of an assignment - perhaps even make this a group / collaborative activity so they once again, engage with course contents, negotiate understanding and meaning prior to completing the assignment. Invite students to even use these notes to help them complete an assessment.

• What expectations do you have for online assessments? How do these expectations compare to those you have for face-to-face assessments? Are you harboring any biases?
  • I think that I should value online assessments as much as any other assignment that is given in a face to face context. However, I can imagine a healthy dose of skepticism from peers and supervisors (even students) who believe that when done sight unseen, students are inclined to "take advantage" of the situation to round up help and thus undermine the validity of the testing. Yet the few articles that I've read, surprisingly indicate that the incidence of cheating online is not even marginally significant when compared with that in a face to face context. 

• What trade-offs do you see between the affordances of auto-scored online quizzes and project-based assessments? How will you strike the right balance in your blended learning course?
  • Trade offs with auto scored online quizzes? Immediately what springs to mind is how objective testing is essentially inviting practice in lower order thinking skills (memorization, identification, classification) but far less likely to be taxing higher order thinking skills. However, any course of study does require a basic competency in the discourse of its subject and these sorts of online quizzes become useful for both ensuring students make them a focus of study and for monitoring student success in understanding them. Project based assessment would be far more welcomed for its ability to promote application, judgement and synthesis of concepts and thus promote higher order (critical thinking) skills. But the caveat with them is in how much more time they require to be reviewed and assessed. In a MOOC with participants that number in the 100s or even 1000s, this is just not feasible. What then needs to be entertained is the use of peer review with a clear and comprehensive rubric - perhaps supported by an instructor's own reflections and guidance (thus the peer assessment also becomes yet another "learning / application / judgement experience"). The balance? Between what is humanely possible for the instructor and what is most beneficial to the students - essentially choosing the "biggest bang for the buck".

• How will you implement formal and informal assessments of learning into your blended learning course? Will these all take place face-to-face, online, or in a combination?
  • My (yet to be identified) blended learning course would definitely see a combination of the two and ideally with one supporting the next. By this I mean that the concepts studied and learned for one assessment are continued / built upon / reapplied / reinforced in subsequent assessments so there is a cumulative reinforcement of them.