Several of the attendees in the room are also looking at the relationship between their student evaluations and components of their quality assurance process. This made for interesting discussions:
Ideas for refining process and research: Do an analysis to see if the questions on the course evaluation fit into the same concepts that we think we are measuring (for aligning our standards and the questions on the course evaluation).
Issues of measurement: What do student evaluations measure? What do the external reviewers measures? and the faculty member self-reviews? Issues with measurement of learning that happens outside the LMS – course activities in the publisher’s textbook site; live sessions; web 2.0 tools, etc.
Common use of chunking content; breaking up videos; what prior knowledge would students need before they do this course? where is the instructor presence? how do you design a course so that it has instructor presence?
Online experiential learning – is it case studies? no, actually it really is real-life experiential learning; connections to employers; the reflection piece is really important too
Side note/thinking: instructional designers work from theory; instructional designers curate content. The more I realize what instructional designers do, and what really good instructional designers do, like Melanie, I wonder if we are going to get to to a place where some of the best courses really are online, not f2f? who is supporting this type of learning f2f? not enough in higher ed, I think
Let’s assume creativity and consistency are at opposite ends of the spectrum – are they?
Consistency – template standard production line model
Creativity – it’s different each time it’s taught because it changes based on the student
Extreme example – wholly standardized course – it starts with best practice; updates are difficult; faculty feels disengaged; can be restricted by the LMS; how do we have faculty bring their experience to the course; how do we not demotivate them;
Extreme example #2 – wholly creative; non-consistent; innovative techniques for design; build the course as it goes; design tailored to faculty, authentic experience based on the student needs; confusion on the part of the learner; longer development time; lack of continuity across the program (accreditation)
Consistency is used to create the order that enables creativity.
How?
Consistency can enable creativity
Consistency: standard verbiage, course build, core principles (i.e. chunking, assessment of prior learning),
Creativity can come in how you deploy the content; navigation
Questions to consider further: What are additional ways to find the balance? What are some course design elements that should be standardized? What pressures have you encountered when trying to find the balance? Where can faculty add their own flair? What about assessment? can faculty adjust points and scores? but not outcomes.
Personal experiences – that’s where faculty can add their flair – sharing video/audio with their stories
4 ways to find the balance
Have a starting point. Design principles. Standard terminology, standard look and feel, blueprint, an umbrella theme and structure
Give ’em options. Give both instructional designers and faculty options. Options on lesson delivery, suite of tools, assessment strategies, different models. Let them choose. Give lots of choice. Let them add videos, redo videos as needed.
The 40/30/30 rule. 40% of the course is the core and designed with consistent rigor; publisher level quality, scalability, and revised for substantial changes in the core content. 30% of the course is flex. Tailored content, regionalism, personal tastes of the instructor, quickly changeable by faculty. 30% of the course is the teach part. Personality and context; constructive suggestions, timely feedback; instantly amenable by faculty. This part can change.
i.e. The core is the Lesson; the flex is readings and media; the teach: the announcements in the course.
We don’t want the core to get to the point where it inhibits creativity.
Be flexible. Let’s train on critical thinking, attention to detail, problem solving, learning principles and NOT on just the tools and the blueprint and the exact way to do it. Instead, teach the instructional designers relationship building – to work with the faculty from what they want to do. Then learn what the faculty needs instead of just saying we do it this way.
Things I want to work on more
Articulated core learning principles (i.e. feedback to students) that guide the course design
Things to think about with revision process; how does it come? do you need a form? do you need a ticketing system?
How to allow more flexibility in some of our high enrollment classes with multiple sections
Different ways to do labs: hands-on manipulatives in person, remote labs, virtual labs, lab kits, etc.
Who is saying which method is more effective, why are they saying it, and based on what criteria? This is a paper that Jim is about to publish. Lesson for me in my budding research: write the journal article first, then use the conference presentation to get the word out and to find potential future research partners; to inspire future needed research; or to share additional results that weren’t included in the paper.
He developed a classification system that can be used for both online and non-traditional, and that can apply to all scientific disciplines.
The engineering lab lit review that he used to inspire his work is this paper: Ma, J., and Nickerson, J. V., “Hands-on, Simulated and Remote Laboratories: A Comparative Literature Review”, ACM Computing Surveys, (38:3) Article 7, 2006, pp. 1-24. pdf
There isn’t a central repository for this information, and so this research on traditional and non-traditional labs isn’t moving along very quickly.
Interesting things came up in the studies: geographic location of the studies, gender, level of the student, is there an equivalent learning for students.
Most undergrad labs are about following directions, getting a specific outcome, scientific communication, using and testing a hypothesis, etc.
Most of outcomes measured in the lab studies were on Knowledge/Understand level. Hmm. Are we really doing in labs what we want to be doing?
The second highest is Perception – which is student’s interest in science. Labs make science interesting for the student.
The big question is, where do the students need the physical manipulatives? Student prior experience with the manipulatives is a consideration as well. Additional research is needed, which includes being really clear on which outcomes are being measured.
One of the participants shared how they offer a program with online and f2f delivery both; and when students get to the face to face clinical work in the hospital, they have the same success rates. Jim argued that we need to get more of these results published.
Reflection: I need to get this paper and share it with our faculty who are thinking of online labs. The issue of labs is really important for access to scientific learning experiences. In addition, how students learn various types of knowledge and skills is important as we try to deliver in a wider variety of ways.
What I really appreciated about this presentation is digging down into the learning outcomes at the foundation of labs, and then considering from that foundation the issues of non-traditional labs.
Juggling concept: the more things you add for your online learning program, the more likely is that a ball will drop. It’s ok. Pick it up and keep moving forward.
Interesting Notes
25 % of faculty think their institution are pushing online education too much (not the 85% we might think).
At the beginning they asked everyone to stand up, and then sit down if you don’t have this in place: one person who oversees online education; have an online advisory board; have an instructional designer; have policies and procedures.
Strategies and Audience Ideas
Have one person who oversees online education for your campus. Don’t add it to someone else’s load as an add on to their other responsibilities. Include things like proctoring, policies, tutoring online, student services, support services – available for ALL online programs.
Have an Online Advisory Board. Should include Faculty, students, administrators, business services, program directors, instructional designers, student support services.
Offer high quality faculty development opportunities. Ideas include: Online learning competency assessment (for people who say they’ve taught online); Online learning academy (6 weeks in the summer for 3 days a week – for those who are totally new); OLC workshops; during the school year accelerated online learning academy; elearning scholars institute – OLC workshops for faculty who want to dabble in online learning. And then after they take the workshop, they teach it to other faculty at a brown bag lunch. Weekly sessions. Use tips and tools to get them in the door, and then teach them something about pedagogy.
Create faculty buy-in. Consider what motivates and demotivates faculty. Student need is a very high motivator. A national credential for training. Something that recognizes faculty at graduation – like a stole to wear. Competencies are important.
Hire an instructional designer. Best way to increase the quality of online learning at your institution. Create an online education factbook for your institution. Impact of online courses on the graduation rate. Impact of instructional designer on retention rate. Partner your ID with your “grumpy guy” to sell resistance faculty on online learning. Two models: they are a consultant and training or are they actually building and designing the course? Interesting thought: the ID on the higher ed side is similar to a media specialist on the K12 side.
Create a policies and procedures manual or handbook for online education. i.e. what are the policies: do we pay more for online courses? can I start my online course a week early? when do I give the final exam in my online course? can I teach my course through my personal blog? student evaluations of instructors, i.e. enrollment in online shouldn’t exceed face to face, dropping online courses, workload, response time, policy for changing instructors in the middle of the term, credit hour. Example of the online handbook at UWLAX. You might need a document with policies and a document with best practices. University approved policies, faculty expectations, best practices etc.
Provide financial support for training and course / program development. Give certificates, buy stoles, give incentives.
Ensure high quality student support services for online learners. Library, student ID, assessment, communication, student communities, personal services, etc.
Create a strategic road map for your institution. For example, who is responsible for these different issues and getting them done. How will you get there. Think logistically about what it takes.
Collect and use data. Number of online courses, gen ed online courses, students taking online courses, graduation rate in six years if they took online course. Tell faculty about it – those who teach online, those who don’t. Share share.
Dare to be different.
Things To Work On
An online program policies and procedures manual. This is ours and this is Brian’s. Improvements could be made on ours!
Have a workshop on ways to reduce cheating (same level of cheating for online and f2f)
Build more faculty training options
Intellectual property clarity. For example one option might be: university has a non-exclusive in perpetuity license for the content; and if the university funds creation of media, the university owns that media.
Bonus: Brian is an excellent example of how to interject humor into your teaching.
