Week 5 - reading reactions - quality assurance in blended learning

Table of Contents 

• Questions to Ponder (before reading)
  After reading 
• Key take aways?  Blended learning quality - says who?
• What about online learning standards? Do they apply?
• So what to do? 
• journal one's work 
• use formative and summative feedback on your efforts 
• invite peer review 
• and traditional ways to measure effectiveness? 
• Engage in Scholarship of Teaching and Learning
• References  

Questions to Ponder (before reading)

• How will you know whether your blended learning course is sound prior to teaching it? How will you know whether your teaching of the course was effective once it has concluded?
• I would prefer to know more about the general nature / background of my students first - especially their interests, competencies. With these, I have a clearer idea as to how I might be able to fully motivate them. I am also fully aware how even with this, I'm forced to make some assumptions that will need to be checked once the Course has started. That translates into frequent surveys and/ or focus groups. Everything else that is part of planning a course is informed by this. This typically starts with a look at desired learning outcomes (typically driven by course stakeholders) and then the available learning resources (limitations on classroom access,  timings, frequency). With these clarified, work can begin on designing instructional methods. 
• With which of your trusted colleagues might you discuss effective teaching of blended learning courses? Is there someone you might ask to review your course materials prior to teaching your blended course? How will you make it easy for this colleague to provide helpful feedback?
• Who would I trust to give me an effective review of my blended learning courses? Ideally this would happen with others who also have an understanding of what is involved in designing blended learning courses. I can realize much of this by making transparent much of the design work listed above - perhaps through Google docs of some sort of online portfolio. 
• How are “quality” and “success” in blended learning operationally defined by those whose opinions matter to you? Has your institution adopted standards to guide formal/informal evaluation?
• Good question. How do we typically measure quality and success in any learning venture? Ultimately that can be seen in the scope and richness of the learning artifacts that learners produce as a result of the course methods .. and then a look at how well these align with the course's learning outcomes. The operative word here is "effectiveness" and then the problematic nature of measuring that effectiveness. It should actually be easier to do with blended learning now that more and more of the processes used can be made transparent to others. 
• Which articulations of quality from existing course standards and course review forms might prove helpful to you and your colleagues as you prepare to teach blended learning courses?
• Not to elude the question but Quality is typically being measured in terms of "effectiveness" and "efficacy" and the discovering of both involves examining all of the variables that went into the design of the Course plus the learning artefacts .. but most important of all  

"Did it change anything for the learner?" 

• How is that last point measured? Feedback from the learners. How to get quality feedback from learners? Surveys, an examination of the learning artifacts, focus groups. Also important will be alignment - checking to see that what we "teach" is driven by the desired "learning outcomes" .. and that this has been learned by the student as demonstrated by the "learning artifacts". 

After reading

Key take aways? Blended Learning Quality - says who?

Still early innings on blended learning. By that I mean, there seems to be so little research on blended learning in general and especially on how to measure quality and effectiveness. Very problematic to try and identify what constitutes the best combination of online and face to face interactions. The same could be said for any instructional design - traditional face to face included. The article suggests that this may be partly due to the almost complete absence of any standards. (No surprise to learn) Attempts have been made to suggest what contributes to "quality" blended learning. Noted here is 

• rigorous learning assessment Picciano and Dzuiban, 2007)
• responsiveness of learner characteristics (Skibba et al, 2014)
• student engagement (Vaughan et al, 2014) 
• high quality faculty development (Dzuiban, Hartman and Mehaffy, 2014)  

Important to note that the major issue with any set of standards is the risk of them becoming prescriptive. I think Thompson makes a real valid follow up point here when he notes that there may be a tendency for any standards that do exist, to be taken up and applied indiscriminantly by educators. Perhaps in their zeal to apply something to elevate the quality of their blended learning work, they are just grateful to find anything and thus may not take the time to critically evaluate such standards - especially in the current status (history) of BL.  Yet it is clear that we need to be critical of any such standards in the current "exploratory" phase of blended learning. Key questions? How were any such standards arrived at? By whom? In what context? 

What about online learning standards? Do they apply?

Because online learning has been subjected to much more scrutiny in comparison with blended learning, what research has been done here may hold some answers.  Big surprise here was to read how such standards have emerged not from traditional educators but from for-profit companies. Perhaps this is motivated by the power of standards to attract the attention of educators and thus help market / promote their product (e.g. Blackboard's Exemplary Course Program). These are usually presented in the form of a checklist (e.g. CSU Chico's Rubric for Online Instruction, and Illinois Quality Online Course Initiative)

So what to do? 

 

Journal ones own work. 

Observe your work and attempt to identify your own standards. Check and evaluate. Experience is the best judge of this..

 

Use formative and summative feedback on your efforts 

This involves soliciting students for feedback on any / all of your designed interventions as to student perceptions of their purpose and effectiveness - during the course or after a course ends. 

 

Invite a peer to review your work  

Invite one's Blended Learning peers (wherever to find these?) to engage with my own review of my work and to critically challenge it. Invite peers to review resources, activities, assessment strategies, course structure and communication strategies. Essentially what I am hearing here is the need to mobilize a community of practice that is focused on "blended learning" where such communities become the incubator to sharing ideas, and over time, help identify best practices. Perhaps they also ascend to the lofty heights of creating a set of "minimum guidelines" or even standards that most blended learning educators could support.



And traditional ways to measure teaching effectiveness? 

A teacher's effectiveness continues to be measured in how they make a difference to students and their quest to succeed. Factors that have been identified here include 

• organization of the course
• knowledge of course material
• ability to communicate clearly with students 
• frequency and quality of feedback to students 

 

Engage in the Scholarship of Teaching and Learning  

Good quote.

Those of us involved with blended design will need to adopt the attitude of learners, examining our practices and seeking continually to improve based upon the most current information available. Perhaps this is done in dialogue with trusted colleagues. - Thompson

References 

Blackboard. (n.d.). Exemplary Course Program | Blackboard. Retrieved March 28, 2016, from http://www.blackboard.com/consulting-training/training-technical-services/exemplary-course-program.aspx

Blended Learning in K-12 - Wikibooks, open books for an open world. (n.d.). [wiki]. Retrieved March 28, 2016, from https://en.wikibooks.org/wiki/Blended_Learning_in_K-12

CSU, Chico. (n.d.). Welcome to the Exemplary Online Instruction (EOI) website - Exemplary Online Instruction. Retrieved March 28, 2016, from http://www.csuchico.edu/eoi/

Eades, J. (2014, September 30). 3 Ways To Measure Training Effectiveness. Retrieved March 28, 2016, from http://elearningindustry.com/3-ways-measure-training-effectiveness

Hirtz, S., & Harper, D. G. (2008). Education for a Digital World: Advice, Guidelines and Effective Practice from Around Globe. Commonwealth of Learning, Vancouver and BC Campus, Canada. Retrieved from http://oasis.col.org/handle/11599/52

ION Initiatives. (n.d.). Quality Online Course Initiative. Retrieved March 28, 2016, from http://www.ion.uillinois.edu/initiatives/qoci/index.asp

Morrison, D. (2015, May 26). How “Good” is Your Online Course? Five Steps to Assess Course Quality. Retrieved from https://onlinelearninginsights.wordpress.com/2015/05/26/how-good-is-your-online-course-five-steps-to-assess-course-quality/

Picciano, A. G., Dziuban, C. D., & Graham, C. R. (2013). Implementation of Blended Learning for the Improvement of Student Learning. In Blended Learning: Research Perspectives. Routledge.

Singh, H., & Reed, C. (2001). Achieving Success with Blended Learning. Centra Software. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.114.821&rep=rep1&type=pdf

Thompson, K. (2005a). Constructing educational criticism of online courses: A model for implementation by practitioners. Retrieved from http://purl.fcla.edu/fcla/etd/CFE0000657

Thompson, K. (2005b). What is Online Course Quality? Retrieved from http://ofcoursesonline.com/wp-content/uploads/2008/06/thompson_onlinecoursequality.pdf

Thompson, K. (2011, June 27). BlendKit Course: BlendKit Reader: Chapter 5. Retrieved from https://blended.online.ucf.edu/blendkit-course-blendkit-reader-chapter-5/