Peeking Under the Hood: A Reflective Case Study of a Unique MOOC Collaboration

Abstract

This case study describes a collaborative process by various teaching and distance support units to create and run a Massive Open Online Course (MOOC) in the subject of Leadership. Whereas typical collaboration arrangements in the literature involve partnerships of various teaching and support units, the collaboration of course developer, instructional designers, librarian, and course facilitator is a missing but needed voice in the MOOC and online learning ecosystem to support student learning and success. This collaborative example combines a case study of the development, implementation, and administration of a MOOC. Documenting and commenting on this collaborative arrangement benefits future course development projects by understanding and encouraging collaboration at any scale.

Introduction

MOOC Evolution

While the Massive Open Online Course (MOOC) hype of 2012 has died down (Pappano, 2012), MOOCs are still very present, with 50 MOOC-based degrees announced between 2017 and 2019 (Shaw, 2019). Furthermore, Class Central, which indexes MOOCs from Coursera, EdX, Udacity, and FutureLearn, reports “in its eighth year, the modern MOOC movement has reached 110 million learners, excluding China” (Shaw, 2019). Searching Google Scholar for MOOC literature since 2018 returns approximately 15,000 citations.

The enduring appeal of the MOOCs may be two-fold. For faculty, their open nature allows them to share their knowledge beyond the university, interact with learners globally, and try new pedagogies (Hollands & Tirthali, 2014; Lowenthal, Snelson, & Perkins, 2018). As universities look for ways to operate and innovate in a climate of decreasing state funding, MOOCs, an open educational resource, provide an avenue for innovation and recruitment, as well as sustainability as an institution of higher education.

This case study documents a collaborative MOOC development of a regional university still experimenting with MOOCs and contributes to the literature on the lived experiences of the collaborators in developing and facilitating the MOOC. As a regional comprehensive university with over 10,000 students, Austin Peay State University (APSU) offered its first MOOC in 2016 and its second MOOC in 2018, both on the Canvas platform. Both offerings were experimental and decentralized (Haywood & McLeod, 2014; Voss, 2013). The first offering involved the course developer and Distance Education (DE), while the second offering included additional personnel from both inside and outside DE in addition to the course developer.

Overview of MOOCS at APSU

APSU’s first MOOC, “Emotional Toughness Training,” was developed and launched on the Canvas platform in 2016. It had over 2,000 registrations and 800 active participants. With the success of the first MOOC, a second course topic was sought with options including Homeland Security and Leadership. While a plethora of Homeland Security courses was and still is available through MOOC sites, the selection of leadership offerings was sparse. Thus began the conversation between the Course Developer [Developer], who was the Chair of the Leadership and Organizational Administration department, and the Instructional Designer [Designer 1] that led to the development and release of APSU’s second MOOC. After brainstorming some preliminary ideas on a MOOC that would cover leadership in general terms, they settled on a title used by the APSU Leadership program: “Discover the Leader in You.” This title would serve as a unifying theme for the MOOC, putting the focus on learners assessing the leadership traits they already had, and reflecting on their leadership.

First APSU MOOC: Emotional Toughness Training. The initial goal of the first MOOC was marketing for APSU programs, linking the experiences within the course to experiences that learners might encounter while taking an APSU course. Module activities consisted of self-analysis questionnaires, discussion, and a short quiz, all based on videos, book excerpts, and articles written for the Tennessean over the years by the faculty author. Learners could move around to different modules out of order but were required to pass each module quiz to release the free certificate at the end. Two distinctive features of the MOOC included a world map on the course homepage, which functioned as the navigation scheme with a dotted line representing the learner’s path through the course, and a live Q& A session with the faculty author. The majority of learners lived in the United States (54%), were 35-54 in age (46%), female (65%), and held a Master’s (33%) degree. The MOOC was highly regarded, with over eighty percent of participants giving the course at least four stars.

Second APSU MOOC: Discover the Leader in You.The first MOOC was a success in terms of the number of participants and the feedback received. From this outcome, a second APSU MOOC was developed. Like the first MOOC, it needed to offer value to learners. The second MOOC would once again market APSU programs and courses. When DE started looking around at available leadership courses in late 2016 during the earliest stages of discussing the MOOC, there weren’t any free leadership courses on other MOOC sites like EdX and Coursera. EdX had started its MicroMasters program in September of 2016, a system of courses that required payment for certification in different areas, but during our initial talks, there were limited offerings, most of which cost money. The purpose of the APSU leadership MOOC wasn’t to be the first, though. It was to showcase APSU’s talented faculty and market APSU programs while providing a structured forum for student inquiry, reflection, and interaction with other learners that would encourage learners to develop an on-going study of leadership in daily life experiences. With this second MOOC, APSU was still exploring MOOCs as an educational avenue for the campus and the department.

Context

The goal of this case study is to provide an example of what it looks like to go from regular course development, often a solo affair by a faculty member, to a MOOC development, which by its open nature and scale requires collaboration to be successful and support student learning.

MOOC Literature Reviews and Stakeholder Perspectives

In “The Changes in Massive Open Online Courses (MOOCs) Studies Between 2012 and 2017 – A Review of Literature,” Alzahrani (2018) surveyed a wide range of studies on MOOCs between 2012 and 2017, winnowing down 979 studies into a smaller group of 37 to examine. These studies focused on student views (25 of 37), course developer views (5 of 37), and combined student and course developer views (7 of 37) of the MOOC experience. This paper both adheres to and exceeds the criteria of Alzahrani’s pool of studies. While the Developer’s view is represented in this paper, it also incorporates a depth of creator views that extends beyond just the Developer, a depth that includes a librarian, instructional designers, and class administrator stakeholders. One element missing from the APSU MOOC is a student view, which is mentioned in Alzahrani’s configuration.

In their study of 76 MOOCs, Margaryan, Bianco, and Littlejohn (2013) found a need for increased instructional quality. In terms of the APSU experience, the issue of instructional quality is tied to the content itself – traditional APSU courses draw on a richer vein of supporting materials in including textbooks and supporting articles. Some elements of the APSU MOOC did emulate some design frameworks seen in traditional APSU courses, such as the use of a combination of discussion boards, quizzes, and essay work.

Challenges of Scale and Openness

MOOCs by nature of their scale and openness bring new challenges and opportunities and often require greater collaboration across universities and partners than online course development. Much of the literature on MOOCs has rightfully examined learner success and how they progress or do not progress through MOOCS depending on their motivations and other factors. Designing for different types of learners, from the sampler to the certificate earner, can be very challenging in terms of engagement (Ho et al., 2014). While many may intend to pursue certification (Chung and Ho, 2016), MOOCs do not usually carry the same weight as a course in a degree program; thus, MOOC time on task may take a backseat (Eriksson, Adawi, & Stöhr, 2017). A few studies have begun to look at the course developer’s perspective and experience in developing and teaching MOOCs. What seems to be missing is the documented collaboration of stakeholders in the university to develop and facilitate a successful MOOC. Searching the educational literature (ERIC & Education Full Text) for Massive Open Online Course, faculty, instructional designer, librarian, and facilitator generally yielded no results, but some combinations of two terms, such as faculty and librarian, yielded results. Thus the following studies on faculty perspectives, innovative uses of MOOCs, and partnerships help show the benefits and challenges faced in different MOOC development instances.

Faculty Perspectives. In a 2018 mixed-methods study of over 180 MOOC course developers on the Coursera and edX platforms, researchers found that while almost half of them had no prior online teaching experience, over 80% agreed they would teach a MOOC again (Lowenthal, Snelson, & Perkins, 2018). The strongest motivations for teaching were interest in the format, sharing expertise, and belief in the value of open education. The researchers note that course developers saw value in the MOOCs more for professional learning than for replacing traditional degreed learning and noted issues in support and sustainability that would need to be addressed for MOOCs to have a viable future (Lowenthal, Snelson, & Perkins, 2018).

Zheng, Wisniewski, Rosson, and Carroll (2016) make a case for faculty support as a critical element of MOOC learner success. The researchers interviewed fourteen faculty members in different disciplines with experience in teaching at least two MOOCs as well as traditional online courses. When faculty members were interviewed by Zheng et al. (2016), they spoke of their motivations and challenges in teaching online at such a scale from the unprecedented massive and global reach and professional growth to the logistical complexities of collaborative work and feedback to individual learners. From their interviews, the researchers distilled six recommendations to improve the MOOC teaching experience, including providing faculty support from beginning to end, project management leadership, and feedback and communication technologies (Zheng et al., 2016).

Innovative use of MOOCs. In an early experiment of “how MOOC content might be incorporated into traditional degree programs and environments that serve mainstream American college students in order to improve student outcomes and/or reduce costs” (Griffiths, Mulhern, Spies, & Chingos, 2015, p. 1), the findings were mixed. While the hybrid classes using the MOOCs posed no harm to students in terms of outcomes, student dissatisfaction was more significant in the hybrid classes than the traditional standard classes as students felt they had less time with faculty (Griffiths et al., 2015). Faculty, while overall satisfied in achieving their goals, found that adapting the courses both in terms of aligning content to their lecture schedule and platform to be considerable, particularly in introductory courses (Griffiths et al., 2015). In a follow-up study with a single university and course in the University of Maryland system, they found moving the MOOC modules to the second half of the term led to higher student satisfaction with the learning experience. By the second half of the term, the faculty member had laid the groundwork, and students were more comfortable with the technology utilized in the course (Griffiths, 2015).

Unlikely Partnerships. While these two instances of partnership may seem obvious once stated, it’s easy to get stuck or busy and not look at possible partners on campus. In the first instance, the faculty member wanted to expand and scale her mentorship, while the second instance documents what happens when a librarian is part of a college-wide initiative to develop MOOCs.

Faculty-Librarian Partnership. When an Engineering faculty member with substantial experience in flipping the classroom, looked for a way to scale her mentorship of individual faculty members into a more sustainable community of practice, she partnered with a librarian. Together, they researched the educational literature regarding innovative pedagogy for a National Science Foundation (NSF) grant proposal (Harp Ziegenfuss, & Furse, 2016). The faculty member, who was also an Associate Dean, brought to the table considerable grant-writing experience, recognition as an outstanding teacher, and considerable experience in teaching flipped courses (Harp Ziegenfuss, & Furse, 2016). The librarian’s strengths included a background in higher education and leadership, experience in teaching a MOOC on designing online courses, and teaching an undergraduate flipped research methods course (Harp Ziegenfuss, & Furse, 2016). As a result, they were successful in their efforts to win the grant, design the initial course, and then redesign into a shorter six-week MOOC, which they co-taught several times before handing over to the teaching and learning center. Participants included higher education faculty, K-12 teachers, faculty developers, administrators, and corporate trainers (Harp Ziegenfuss, & Furse, 2016).

Librarian Perspective in a College-wide Initiative. Davidson College, a liberal arts college, started its own MOOC initiative in 2013. The librarian, pulled in to help with copyright and intellectual property, created an online guide to help faculty as they developed MOOC proposals (Swan, 2017). She later reviewed proposals for copyright and intellectual property considerations. From there, she expanded her scope of work to include information literacy instruction by recording a video and creating a course guide for the Representations of HIV/AIDS MOOC. The Davidson College initiative serves as a guide in campus-wide systematic development, in which four MOOCs were launched on the edX platform.

Situating the APSU Leadership MOOC

This paper seeks to look at the APSU MOOC project in ways that both continue and expand existing research:

• Qualitative in nature. Our work is primarily reflection-based and aligns with existing MOOC research.
• Stakeholder focused. Our work includes a course developer view of the MOOC; however, it expands the creator-centric view by including distance learning professionals, class administration, and library stakeholders.
• Stakeholder led. Our work focuses on a stakeholder-led (as opposed to an overarching institutional) initiative.

Development - A True Team Process

The Developer has developed and taught multiple online leadership courses at APSU. The Leadership MOOC both resembled and differed from the content found in the for-credit courses.

Similarities and Differences between the MOOC & Departmental Courses

Similarities included the following:

• Common topics - Topics in the MOOC had roots in topics covered in APSU Leadership classes. As an example, the Leadership MOOC looks at an individual’s strengths. That was a frequent topic of study in the APSU curriculum, particularly in LDSP 2100, Foundations of Leadership.
• Typical structure and assessment methods - The structure of the MOOC, particularly the assessment methods of discussion boards and short quizzes, resembled the structure of APSU courses developed and taught by the Developer. Also, the modular format of the MOOC was similar to that in courses employed at APSU.

Differences included the following:

• Resources - Typical Leadership courses at APSU used one to three texts. In contrast and owing to copyright concerns, the MOOC relied on articles readily accessible to the general public.
• Depth of inquiry- Subjects covered in the MOOC received less in-depth inquiry than they would have in an APSU Leadership course. Reasons for this include resource limitations (as described in the previous item), an abbreviated schedule, and the nature of the MOOC audience versus APSU students.
• Specialization of the topic - While APSU’s LDSP 2100, Foundations of Leadership serves in part as an introduction to the study of Leadership, other courses in the Leadership program focus on particular topics such as leadership theory, organizational dynamics, and power.
• Nature of the audience - APSU students take their courses for credit, and that creates a strong(er) commitment to the course. On the other hand, the MOOC would more likely appeal to the casual learner.

MOOC Outline

Through the work sessions between the Developer and Designer 1, an outline of the MOOC took shape for the “Discover the Leader in You: Six Questions to Ask Yourself” MOOC. The goal of the MOOC was to equip learners with tools to become better leaders. While developing themselves as both scholars and practitioners of leadership, the learners would address six questions in the modules in the MOOC:

• Module 1: Who are You? Define Your Leadership. Learners define the unique role of a leader in an organization (including a distinction between leadership and management).
• Module 2: What Makes Your Leadership Different? Learners consider an individual’s traits and strengths and their application to leadership.
• Module 3: When are You a Leader and When are You Not? Contrast the Roles of Leaders, Followers, and Colleagues. Learners contrast the varied roles that leaders fulfill.
• Module 4: Where is Your Power Supply? Evaluate the Tools of Leadership. Learners evaluate how effective leaders use diverse tools to accomplish goals.
• Module 5: Why is Your Leadership a Challenge? Embody the Change Agent. Learners explore how the leader serves as a catalyst for change.
• Module 6: How do You Develop Other Leaders? Build a Legacy of Leadership. Learners examine the importance of leadership development for organizational success.

The Developer collaborated with the IDs to produce written, visual, video, and audio content over several months. The next section explores the IDs’ roles and contributions to the MOOC.

The Role of the Instructional Designers

As mentioned before, the Developer collaborated with the DE team to produce the MOOC content. The team had three instructional designers at the time with different specialties who collaborated and contributed to the MOOC project in different ways. Designer 1 had pedagogical expertise and worked with the Developer to flesh out overall course objectives, module topics and objectives, assessments and activities (aligned with course and module objectives), and module instructional content to be curated by the Librarian. Designer 2 had content organization and accessibility expertise and also served as the informal project manager, including serving as the liaison between DE and Canvas (MOOC platform). In preparation, Designer 2 learned the Canvas LMS for the course developer and designer’s perspectives since APSU used Desire2Learn (D2L). The Canvas Instructional Designer [Designer C] set up the course shell, and Designer 2 created the course structure and components in the shell. Eventually, Designer 2 also uploaded course content and set up the assessments, activities, and course badge for successful completion. Simultaneously, Designer 2 also kept the project worksheet updated to chart progress and created a customized survey to get student feedback upon MOOC completion. Designer 3 had multimedia expertise and helped develop the MOOC components, such as instructional videos with closed-captions, images, and overall instructional technology pieces.

Multimedia Development and MOOC Instructional Videos

Designer 3 worked closely with Developer to create module videos that covered specific content and aligned with objectives. Scripts and PowerPoints for the videos served as the primary module instructional content. After the Developer created the content and put it in a blank PowerPoint template, Designer 2 created detailed PowerPoints. She also managed narration recording, and created and edited the module videos using Camtasia. At least one video per module was created and “chunked” under 10 minutes to ensure learners would stay engaged.

A typical multimedia meeting for the MOOC. To produce the module videos, Designer 3 scheduled weekly meetings with the Developer during which they would produce content in a step-by-step process over a 2-3 hour session, one module at a time. Before the meeting, the Developer created an unformatted (all white slides, no themes) PowerPoint template that outlined the concepts to cover within each module video. A typical meeting began by Designer 3 and Developer reviewing the unformatted PowerPoint and making revisions. Once the content was finalized, audio narrations were recorded using Camtasia based on the PowerPoint content. Once the volume and pacing were adjusted for the videos, Designer 3 split up each slide of content as a separate audio recording to avoid perfecting large chunks of audio and facilitate editing. Designer 3 would then produce a final video by stitching the narration to the content, adding music, transitions, and other elements to bring the videos to life. Videos were then exported to DE’s YouTube account to create accurate closed captions for accessibility and provide a link for the Canvas platform.

Course Developer, Instructional Designer, and Librarian Development

Simultaneously the Developer also collaborated with the APSU Librarian [Librarian] to search and curate various open educational resources (OERs) to be used instead of a textbook as supplemental material for the MOOC.

The open course format. While courses in a university’s Learning Management System (LMS) are covered by vendor contracts and licenses for databases and online resources, determining if open courses fall within the contract requires working with legal counsel.

Open resources on the web. Over the past several years, there has been an explosion of open education resources, which can be used in MOOCs without concern about vendor contracts. Repositories like MERLOT and Open Commons and open textbook publishers such as OpenStax and Open TextBook Library provide high-quality resources. Also, most federally-funded research grants require that the research be published openly without paywalls.

Nature of the MOOC audience and business magazines. Most magazines are available online; however, paywalls often limit one to a couple of articles. Selecting benevolent websites is beneficial for open courses. Another potential issue is finding the right level of quality for the course. For this course, most of the supplemental articles for optional reading by learners came from Forbes, which matched criteria in terms of quality, reputation, and liberal access to articles.

MOOC Assessments

As outlined in a previous section, each module had its objectives. Assessments for each module included:

• A Micro Quiz - to check self-progress and retention of main concepts in the module.
• Discussion 1: "How Does this Relate?" gave learners an image and asked them what it represents to them in the context of the content covered in the module.
• Discussion 2: "Represent This" asks learners to take something associated with the module and represent it symbolically using descriptive text, a visual, or video/audio.
• Discovery Question Assignment, a thought-provoking, and reflective assignment required learners to tie all the concepts together and bring in personal and professional experiences and insights. These assignments would occasionally include related tools like a SWOT analysis rubric.

A MOOC is born - Finishing touches

MOOC Module Design and Map. Each of the six modules consisted of an overview including objectives and tasks, review lectures and suggested readings, a micro-quiz, discussions, and a discovery question assignment. To complete each module, learners would:

• Start with the module overview, objectives, and tasks.
• Review content video and read suggested articles.
• Complete the micro quiz to check themselves (10 points).
• Engage in two module discussions - "How Does this Relate?" and "Represent This" (10 points each).
• Complete the Discovery Question Assignment (10 points).

Each module was organized, as shown in Figure 1, to maintain a consistent structure throughout the course.

Figure 1: MOOC Module Structure

MOOC Completion Badge. The instructional designers decided to create a course completion badge, as shown in Figure 2, with two key purposes: get the APSU brand name out to learners and encourage learners to complete the MOOC (which was free, but offered no formal credit or recognition). As Knight and Casilli (2012) note in their case study on open badges, badges “have long been used successfully to set goals and motivate, and represent and communicate achievements and success” (p. 279). While badges go back as far as the 1300s (“Badge," 2019), people today are more familiar with the merit badges of the scouting movement begun in the early 1900s (U. S. Scouting Service Project, 2017) which “highlight the achievement of the scout in terms of a clearly defined skill or knowledge set demonstrated at an established standard or level of practice, a model that suits the educational context especially well” (Gibson, Ostashewski, Flintoff, Grant, & Knight, 2015). In education, badges “can play a crucial role in the connected learning ecology by acting as a bridge between [various learning] contexts, making these alternative learning channels [such as MOOCS] and types of learning more viable, portable, and impactful” (Knight and Casilli, 2012, p. 280). Furthermore, Law (2015) reports that “data show that learners in an informal environment are willing to pay for certification and recognition of unsupported informal learning” (p 232).

Figure 2: MOOC Completion Badge

MOOC completion criteria included:

• reviewing all MOOC content, including instructional video and module articles;
• scoring 85% or more in all graded assignments including micro-quizzes, discussions, and discovery questions for all modules; and
• completing a MOOC welcome survey (created by Canvas) at the start of the course as well as a MOOC exit survey (customized by Designer 2) asking for learner experience and feedback while taking the MOOC. The survey results are described in detail later under Course Facilitation.

Development Challenges and Solutions

The MOOC on Canvas. As mentioned previously, Instructure’s Canvas LMS was used to host the Leadership MOOC. Since APSU uses D2L, there were some technical issues for both the instructional designers in designing and developing the MOOC and the Facilitator in facilitating the MOOC once launched.

Instructional Designer’s Knowledge of Canvas. Designer 2 was in charge of working with a blank Canvas course shell to create the MOOC design. To familiarize herself with this new LMS, Designer 2 enrolled and completed the “Canvas Network Training Course,” which covered topics related to using Canvas as a course developer, builder, and instructional designer earning the “Canvas Groupie” and “Canvas Rockstar” badges upon successful completion. Additionally, Designer 2 referred to Canvas Guides and Canvas Community pages as needed for reference or troubleshooting. Having previous experience with LMS in the past, such as Blackboard, D2L, Moodle, and WebCT, helped to quickly gain an understanding of the Canvas platform from a design and development perspective. Designer C also advised on the right resources and guides when needed in the design process.

Course Facilitator’s Knowledge of Canvas. Once the MOOC went live, the Facilitator was in charge of the course and interacted with learners and graded assessments. While the Facilitator was familiar with other LMS’s, she experienced a learning curve in Canvas after the MOOC launched. To strengthen her knowledge of the MOOC LMS from a facilitating perspective, the Facilitator reviewed asynchronous training videos. She also found that her prior experience with Blackboard, D2L, and WebCT LMS’ as both a student and course developer helped in learning to navigate the Canvas LMS.

Role of the Canvas Support Team. Technology, whether used as a novice or expert, becomes challenging when the technology fails to function, or there is glitch in the system that might affect learner progress. Designer 2 and the Facilitator worked with the Canvas support team when assistance was needed with technical issues that exceeded their level of expertise, such as an issue that caused a delay in learners receiving their badges after completing the required MOOC criteria. One learner needed the badge as verification that she completed the course as a professional development opportunity for an employer, while another pursued the badge as a symbol of accomplishment that he could share with his children. Designer C corrected the last module, which enabled learners to trigger the automatic release of the badge once all course requirements were complete. Until the problem with awarding the badge was resolved, the Facilitator utilized a manual process as a temporary solution to issue badges.

Course Facilitation

The course was designed so that all learners, whether an entry-level employee or a senior-level administrator, could improve or build their leadership skills. Through reflective activities, learners shared accomplishments and personal challenges that led them to enroll in the MOOC. In preparation for developing a leadership plan in Module 6, one learner wrote about her experience with emotional intelligence as a leader. She spoke of the need to create a plan that would help team leaders in her organization learn about “positive and negative reinforcement, defense mechanisms, and coaching.”

“In my case, I would like to learn about anger management skills. I am honest to say that I sometimes expect too much from my members. As a result, I can sometimes feel very disappointed when a certain member does not deliver their task well.” (Learner)

Discussion board activities encouraged creative and flexible thinking about leadership attributes. The first discussion activity in each module generated an average of 103 original posts and asked learners to examine a picture and then relate it to a leadership topic in that module’s readings. The second discussion activity, which generated 90 original posts on average, asked learners to choose and post a picture or video (or written response) that represented a topic in the module.

The structure of the discussion board paved the way for rich and engaging discussions. Learners expressed their personalities through humor or thought-provoking quotes and interesting photos. Learners also shared valuable resources that could be added to a leadership toolkit for future use. (Facilitator)

The Facilitator worked with the Developer and the Designer 2 on the development of a rubric for discussion assignments and discovery activities. The team wanted to ensure learners received notification of the evaluation criteria for each assignment. After the course, twenty learners received a digital badge, which verified their participation in the course. The release of the digital badge resulted in 2 Facebook Shares, 10 LinkedIn shares, and 11 Twitter shares.

Participant Profile and Demographics

Registration in the Leadership MOOC opened in May 2018 and closed November 2018. The total enrollment number reached 1,034, with approximately 2% of learners receiving a badge for successfully completing all course requirements. The fluidity and design of the course provided opportunities for learners to complete all module activities in any order. In addition, learners could drop in and select module activities based on interest. Of the 1,034 enrollments, 24% of learners (246) accessed various sections of the course, while 6% of learners (63) completed at least one activity. Participants agreed or strongly agreed that course materials (90%) and activities (89%) had a positive impact on their learning experience with only a 2-5% difference lead to Agree over Strongly Agree. One learner noted, “I enjoyed the interactive class activities. Reading other people's responses, descriptions, and representations gave me a different perspective on leadership and many times, it mirrored my own thoughts even more clearly.”

Figure 3: Welcome Survey Participants’ responses to how the course will meet personal or professional goals.

Demographic data gathered in the Welcome survey from 246 participants reveals that learners were female (57%), curious to learn more about the topic (49%), and intended to be active (44%) by spending two to four hours a week (36%) on the course. Gender demographics in the APSU MOOC are similar to a 2015 MOOC report by Canvas, in which women comprised 62% of learners.

Successes and Challenges

The greatest success was the opportunity for learners to earn a badge in the course. Twenty learners finished the course with a tangible badge to demonstrate their mastery of a subject area. The badge was essential for those who needed to document professional development training for promotions. When offering badges, it is essential to provide a certificate option for learners to demonstrate mastery as not all countries accept badges as verification of professional development.

The creative and flexible aspects of the discussion board were very successful, and learners contributed very engaging responses. Learners who did not have a natural inclination for creativity struggled with responding to a photo; however, they quickly caught on and contributed meaningful posts.

One challenge in the course for both learners and the Facilitator was overcoming language barriers. Although the course was delivered in English, some assignments were submitted in a language other than English. While there are translators available to assist with translating documents, there were some instances where the Facilitator could not copy the text into a translator. Demographic data from the Leadership Welcome survey showed that English is the primary language for 81% of respondents; however, 18% of respondents indicated that English is not their primary language. To minimize concerns related to language, including a downloadable translator in the LMS and offering video transcriptions in multiple languages, would be especially helpful.

Figure 4: Discover the Leader in You Survey participants’ responses to English as a primary language and location

Figure 5: Survey participants' responses to English as a primary language and location, as reported in the 2015 Canvas Network Progress Report.

Next Iterations

Challenges and Opportunities

In the course of developing and implementing the Leadership MOOC, team members faced both challenges and opportunities in four different aspects.

Scale, Collaboration, and Project Management. “The volume of activities associated with the MOOC development exceeding those of the typical university online course, particularly, the number of video and audio items needed exceeded what had been done previously”(Developer). For the veteran Developer involved, it was the first MOOC with which he had been involved. In comparison to his typical university course, the MOOC required him to work with various colleagues on a multitude of MOOC components. “For me personally, I enjoyed the experience. In particular, I appreciated working with my fellow APSU colleagues in developing and building the MOOC.” [Developer]

Closely related to scale is the need to start with the end in mind and work backward, especially in a project involving multiple individuals from different departments. The instructional designers took the lead on the project management using the deadline checklist provided by Canvas in the course administrative area. Working on a new course development, particularly a MOOC, brought new challenges such as scheduling time to work with the Developer to make sure all scripts could be recorded, which entailed needing to find blocks of time to record and edit the videos.

It is also essential to know if there is flexibility in the launch dates by both the individuals involved and the platform. In this case, because this was a collaborative project, and outside the normal working load for some individuals, the launch was delayed from the end of spring to mid-summer.

Copyright and OER. One of the main challenges for the IDs was navigating copyright and finding materials that could be used in the MOOC to provide a variety of content. Designer 1 reached out to the Library and Librarian for assistance in investigating copyright permissions for potential materials, including journal articles and locating open educational materials. Due to initial time constraints, team members did not end up pursuing permissions through the Copyright Clearance Center (CCC) OER partners. Once the launch was pushed back, the Developer and the Librarian worked together to seek out materials and linkages that were acceptable to use in a MOOC open to the public. While the availability of materials ultimately shapes what can and cannot be done, the Developer realizes that it affects the finished product. Lessons for the next time include starting as early as possible and beginning with open access journals and open educational resources (OERs) for which there are no paywalls or restrictions.

Technology. Team members encountered technical issues in working with a different LMS in developing and implementing the MOOC. In this case, this involved both learning how to use the new system inside out, and third-party integrations such as badging components. All of this takes time and is more difficult while facilitating a MOOC in situ.

As a result of technical difficulties, the Facilitator recommended that a test of a complete, integrated system be performed in the future to ensure the system is functioning correctly and meets specified requirements for third-party integrations such as the badging component, Badgr.

Community Engagement.
Anticipating who and how many learners are going to enroll in a MOOC is difficult at best, and requires an understanding of who enrolls in MOOCs from the limited, but growing data available. In most cases, it would be valuable to start with sound design principles as Designer 3 did, creating video content that would be exciting for potential learners enrolled in the MOOC as she worked with the Developer. While obtaining an accurate breakdown of MOOC learners ahead of the course start might be impossible, getting a quick pulse early on of demographics and enrollment metrics might go a long way in identifying potential barriers and establishing intervention methods.

The Next MOOC

Possible Topics. After the success of the “Discover the Leader in You” MOOC, the Developer believes future opportunities would come from looking at focused leadership-related topics, such as decision making and problem-solving in the context of new college learners or leadership and power or creativity.

Course Design and Development. From incremental improvements to community engagement to developing in a whole new fashion, the three instructional designers brainstormed what they thought would enhance future MOOCs. First, set up course badging more granularly to increase motivation and recognize achievements by breaking down accomplishments into module-level or activity-level badges. Second, automate more course activities for easier and quicker grading to help with both course facilitation and granular badging. Additional engagement ideas include talking to past learners about what they looked for in a course and enjoyed; incorporating videos with embedded knowledge checks and responsive multimedia that scaffolds depending on student answers; and lastly, integrating case studies with optional, ungraded discussions where learners share stories about leaders from their own countries or cultures. Looking to the future, one idea to put a unique spin on course development would be to develop the MOOC using solely mobile technologies such as a phone's video, video editing apps, and an LMS app.

Course Facilitation and Assessment. The Facilitator had several recommendations to identify potential sticking spots ahead of time, the key being to bring the Facilitator in early with a clear set of expectations and time to gain knowledge of the course content and system. The Facilitator can help check that the assignment instructions are explicitly communicated and that there is a rubric for each module to help learners gain a better understanding of grading and feedback. Once the course is underway, have someone in-house who can assist with instructional design concerns and consider translating course content into two or three languages by installing translator technology. Lastly, provide an alternative credential, such as a certificate, for learners who live in countries where badging is not accepted.

The Future of MOOCs

While MOOCs entered the world with a bang, they have settled into the role of a significant educational tool drawing increased scholarly interest. A Google Scholar search for MOOCs since 2019 generated over 9,000 results, and searching education subscription resources such as ERIC and Education Full-Text generated over 200 results published since the beginning of 2019. Understanding the MOOC learning experience and target audience can foster improvement, but educators also need support in designing to address their significant developmental and legal considerations and costs. Regardless of how an institution uses a MOOC, designing online education is no longer a solo educator activity – today, it is a "team sport" that involves multiple players fulfilling specific roles, based on specialized knowledge and abilities (Chu & Hogue, 2019). To enable educators to develop MOOCs, higher education institutions need to understand the role of MOOCs in the education trajectories of current and potential learners (McClure, 2019).

References

Alzahrani, A. A. (2018). The Changes in Massive open online courses (MOOCs) studies between 2012 and 2017--a review of literature. World Journal of Education, 8(4), 59-71.

Badge. (2019). OED Online. Retrieved from https://www-oed-com.ezproxy.lib.apsu.edu/view/Entry/14550?result=1&rskey=NfSUUz&

Blackmon, S. J. (2016). Through the MOOCing glass: Professor’s Perspectives on the future of MOOCs in higher education. In S. J. Blackmon & C. H. Major (Eds.), MOOCs and Higher Education: Implications for Institutional Research (pp. 87-101). San Francisco: Jossey-Bass.
Canvas Network. (2015). Progress Report 2015.

Chu, H., & Hogue, B. (2019). A strategic leader for student success: An argument for the Chief Academic Technology Officer. Retrieved from https://er.educause.edu/articles/2019/5/a-strategic-leader-for-student-success-an-argument-for-the-chief-academic-technology-officer

Chuang, I. & Ho, A. (2017). HarvardX and MITx: Four years of open online courses -- Fall 2012-Summer 2016. Retrieved from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2889436

Eriksson, T., Adawi, T. & Stöhr, C. (2017). Time is the bottleneck: A qualitative study exploring why learners drop out of MOOCs. Journal of Computing in Higher Education 29(1): 133-146. https://doi.org/10.1007/s12528-016-9127-8

Gibson, D., Ostashewski, N., Flintoff, K., Grant, S., & Knight, E. (2015). Digital badges in education. Education and Information Technologies, 20(2), 403-410. doi:10.1007/s10639-013-9291-7

Griffiths, R. (2015, February 5). Blended MOOCs: Is the second time the charm? [Blog post] Retrieved from https://sr.ithaka.org/blog/blended-moocs-is-the-second-time-the-charm/

Griffiths, R., Mulhern, C., Spies, R., & Chingos, M. (2015). Adopting MOOCS on campus: A collaborative effort to test MOOCS on campuses of the university system of Maryland. Online Learning, 19(2): 1-15. http://dx.doi.org/10.24059/olj.v19i2.523.

Harp Ziegenfuss, D., & Furse, C. (2016). Opening up collaboration and partnership possibilities: Re-valuing library resources, skill sets, and expertise. Digital Library Perspectives, 32(2), 103-116. Retrieved from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2381263

Haywood, J. & Macleod, H. (2014). To MOOC or not to MOOC? University decision making and agile governance for educational innovation. In Kim, P. (Ed.), Massive open online courses: The MOOC revolution (pp. 56-70). New York: Routledge.

Hollands, F. M., & Tirthali, D. (2014). MOOCs: expectations and reality. Full report. Center for Benefit-Cost Studies of Education, Teachers College, Columbia University, NY.

Knight, E., & Casilli, C. (2012). Mozilla open badges. In D. G. Oblinger (Ed.). Game Changers: Education and Information Technologies (pp. 279-284). EDUCAUSE.

Law, P. (2015). Digital badging at The Open University: Recognition for informal learning. Open Learning: The Journal of Open, Distance and e-Learning, 30(3), 221-234.

Lowenthal, P., Snelson, C., & Perkins, R. (2018). Teaching massive, open, online, courses (MOOCs): Tales from the front line. International Review of Research in Open and Distributed Learning, 19(3).

Margaryan, A., Bianco, M., & Littlejohn, A. (2015). Instructional quality of massive open online courses (MOOCs). Computers & Education, 80, 77-83.

McClure, M. W. (2019). MOOCs, learners, higher education and their paradoxes. In W. Archer & H. G. Schuetze (Eds.) Preparing Learners for Life and Work: Policies and Reforms Affecting Higher Education’s Principal Mission (pp. 157-178). Boston: Brill Sense.

Pappano, L. (2012). The year of the MOOC. The New York Times. Retrieved from https://www.nytimes.com/2012/11/04/education/edlife/massive-open-online-courses-are-multiplying-at-a-rapid-pace.html?pagewanted=all&_r=0

Shah, D. (2019, December 17). Online degrees slowdown: A review of MOOC stats and trends in 2019. Retrieved from https://www.classcentral.com/report/moocs-stats-and-trends-2019/

Swan, S. G. (2017). Davidson College. In Alman, S. W., & Jumba, J. (Eds.). MOOCs now: Everything you need to know to design, set up, and run a massive open online course (pp. 75-87). Santa Barbara, CA: Libraries Unlimited

U. S. Scouting Service Project. (2017, November 28). Merit Badge History. Retrieved from http://usscouts.org/mb/history...

Voss, B. D. (2013). Massive open online courses (MOOCs): A primer for university and college board members. Washington, DC: Association of Governing Boards of Universities and Colleges.

Zheng, S., Wisniewski, P., Rosson, M. B., & Carroll, J. M. (2016). Ask the course developers: Motivations and challenges of teaching massive open online courses. In D. Gergle (Ed.), Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (pp. 206-221). San Francisco: Association for Computing Machinery.

Appendix A

The following supplemental files have been placed in a Dropbox file, http://bit.ly/ojdla, and can be downloaded.

• Objectives_Course and Module Levels.docx
• Survey_Welcome.docx
• Survey_User Experience.docx
• Leadership Roles.xlsx