People will adopt LOs if they are seen easy to use and useful (Lau & Woods, 2009).  Ideally there should be formative and summative evaluation at each stage.  There are several ways to evaluate the functionality.  Some of these methods are tied back to the Analysis and Design Phase. Many are similar (if not the same) as with online tutorials.

Several authors recommend use of Multimedia Educational Resources for Learning and Online Teaching (MERLOT) process of evaluation and review (Cochrane, 2005; Moisey, Ally, & Spencer, 2006).  MERLOT Evaluation Criteria for peer reviews has three categories of evaluation standards

• Quality of Content
• Potential Effectiveness as a Teaching-Learning Tool
• Ease of Use

In addition to MERLOT, there are other checklists for evaluation that can be considered and incorporated into the evaluation process. The Learning Object Review Instrument (LORI) and the Southern  Regional Education Board’s “Checklist for Evaluating SREB-SCORE Learning Objects” (SREB-SCORE) are two examples.  Evaluations can incorporate a rating system or scale in response to questions and/or criteria.   Evaluations should consider the following

Content

• Does the content present valid concepts, models, and skills? Is it error and bias free?
• Does the content present educationally important concepts, models, and skills?
• Is the content sequenced logically and effectively?
• Will the content supply the content and learning experiences to achieve learning outcome? Are there enough activities to support outcomes?

Ease of Use:

Elements that affect ease of use include:

• Are the labels, buttons, menus, navigation elements, text, and general layout of the interface consistently placed?
• Is the module flexible and easy to navigate? Can users return to start, exit, and go where they choose?  Is the interface is intuitive and/or are clear instructions  provided ?
• Does the design apply Universal Design for Learning principles
• Is text legible, concise, and clearly written?  Are graphs and charts labeled?   Do color, music,  design support learning goals and instruction ?
• Design and interface attractive and consistent throughout?

Effectiveness as teaching tool.

Merlot cautions that evaluating actual effectiveness for teaching and learning requires actual use by students and faculty.  This will be addressed later.  But a checklist can evaluate potential effectiveness.

• Are learning goals and objectives declared within the content? Are the learning goals appropriate for the learners? Do the learning object’s activities, content and assessment align with the goals and objectives?
• Does the LO provide timely feedback and tailor instruction and activities according to learner needs?  Is the feedback encouraging, letting learner know how to improve performance?
• Does the LO motivate? Is the content relevant to the learner? Is it realistic?
• Does the learning object provide engaging and interactive learning? Does it offer chances to increase understanding, knowledge, and skills? Will the materials improve teaching and learning of materials?
• Is the learning object reusable? Can it be used in different learning contexts and by learners from various backgrounds?
• Is the learning object comprised of one or more learning experiences, each addressing a single learning objective? Can it stand alone as a learning resource?

This evaluation process does not include the actual assessment of learning.

Sample Checklists

MERLOT: Evaluation Criteria for Peer Review

Learning Object Review Instrument: LORI

Checklist for Evaluating SREB-SCORE Learning Objects

EDUCAUSE 2001: Learning Object/Module Checklist

Sources

Cochrane, T. (2005). Interactive QuickTime: Developing and evaluating multimedia learning objects to enhance both face-to-face and distance E-learning environments. Interdisciplinary Journal of Knowledge and Learning Objects, 1, 33-54.

Lau, S., & Woods, P. C. (2009). Understanding learner acceptance of learning objects: The roles of learning object characteristics and individual differences. British Journal of Educational Technology, 40(6), 1059-1075.

Multimedia Educational Resources for Learning and Online Teaching. (n.d). Evaluation Criteria for Peer Reviews. Retrieved from http://taste.merlot.org/evaluationcriteria.html

Moisey, S. D., Ally, M., & Spencer, B. (2006). Factors affecting the development and use of learning objects. American Journal of Distance Education, 20(3), 143-161

Nesbit, J., Belfer, K., & Leacock, T. (n.d.). Learning object review instrument (LORI: User Manual.  Retrieved from http://www.elera.net/eLera/Home/Articles/LORI%201.5.pdf

Southern Regional Education Board. (September 2007).  Checklist for evaluating SREB-SCORE learning objects.  Retrieved from http://publications.sreb.org/2007/07T05_Checklist_Eval_SREB-SCORE.pdf