Abstract

This paper outlines the competency-linked project of a total redesign of a fully online master’s program that linked every deliverable and assessment in each course with a competency model. Extensive mapping was completed and then applied to every course in the master’s program over a two-year period. Graduate students' perceptions of connecting new course knowledge to existing knowledge on the course topic, prior to current skills, and the perceived ability to apply course knowledge to the workplace were examined, analyzed, and reported. Lastly, recommendations on finding competencies, mapping, and spreading the word are detailed.

Background

The purpose of the Master of Education in Training and Development in the College of Education at North Carolina State University is to prepare adult learners for careers such as instructional designers, trainers, and program managers or evaluators throughout various industries. Courses throughout the program prepare students to engage in curriculum development, manage people and programs, and apply learning to a wide variety of situations. The program redesign between 2017 and 2019 required course assignment modification to incorporate specific industry competencies throughout topics covered in each course as well as course restructuring to serve students for both regular 15-week semesters and summer 5-week semesters. The purpose of the program redesign was to improve student learning, develop students’ critical thinking skills, increase labor market outcomes, and establish community partnerships. Competency-based learning has been proven to be beneficial for students on all education levels (Gross, Tuchman, & Patrick, 2018).

Designing Courses for the Learner

Regardless of whether a course is designed for online, face-to-face, hybrid, blended or flipped learning, every course should be designed with the learner in mind. Learner-centered instruction places the student at the center of instruction so students are active learners rather than passive participants (Altay, 2014). The five overarching themes for each course’s modules allowed us to section out students’ course materials, readings, and assignments so everything tied together with the main theme. As we did this, we linked industry competencies and course learning objectives to everything within each course. Instead of students merely going through rote steps and memorization in program courses, the changes required them to engage more with the materials and apply what they learned throughout the course to various assignments.

These assignments were even stronger since not only were the course objectives mapped to the assignments and readings, but the assignments also had an extra layer of rigor since they incorporated various competencies related to the concepts covered in class. Some assignments covered multiple competencies while others only incorporated one. Regardless, students were able to see which competencies they had met and achieved at the end of each assignment, module, and course. To ensure students applied critical thinking skills and integrated skills they would likely apply in the workforce, most assignments were created to be mock projects of what students would likely encounter in the workplace in various training, learning, and educational settings.

Designing to Support Course Rigor, Ensure Alignment, and Aid in Student Outcomes

Once each course was associated with a high-level competency area, then the course objectives, assignments, and readings were analyzed. Doing so created an illustrative approach to see which competencies went with which course throughout the program, what competencies aligned with which assignments throughout a course, which objectives needed to be rewritten or revised, and where gaps existed throughout the various program courses. To ensure every course throughout the program met the previously established course objectives and aligned with the competencies being linked throughout, every course was redesigned in some shape or form to support rigor, ensure alignment and aid in increasing student outcomes. The end goal was to ensure students knew they not only mastered the course objectives but were also proficient in various industry competencies by the end of the course. The redesign ensured each course had a higher level of rigor with students applying knowledge gained instead of merely regurgitating it through multiple choice question quiz or exam questions and seeing how each course connected with the real world. Critical thinking is defined as a way of thinking that requires the thinker to analyze, assess, and reconstruct knowledge through problem-solving and effective communication (Foundation for Critical Thinking, 2019). Assignments were revised to provide students more application-based opportunities rather than rote memorization or passive learning of concepts. The program redesign aimed to purposefully require students to apply critical thinking skills and those learning outcomes are not only increased but also tied to real-world application.

Aligning Industry Competencies to Program Courses

Industry competencies related to learning technologies, change management, managing learning programs, and instructional design were aligned to program courses associated with those specific competencies. Every assignment in each course is mapped and aligned with at least one-course objective and typically one industry competency. Some assignments are mapped and aligned with several. To ensure we aligned and mapped course objectives and competencies with the correct course, we aligned and matched each course in the program to an industry competency (i.e., learning technologies, change management, instructional design, etc.).

Figure 1
Course names and competencies

Research Questions

In order to see the connections among students’ preparedness and skill levels after completing the various courses, the following research questions guided the study.
Research Question #1: How do graduate students perceive participation in an online competency-based course impacts their own work preparedness?


Research Question #2: How do graduate students perceive participation in an online competency-based course impacts their skill level and Connection to prior knowledge?
Research Question #3: Is there a statistically significant relationship between the connection of prior to knowledge for graduate students in an online competency-based course?

Data Collection

Researchers for this study created an end-of-course survey to collect students’ perceptions related to, prior skill level vs. current (end-of-course) skill level, level of connection of new knowledge to prior knowledge, and the relationship of the level of connection of new knowledge to prior knowledge with perceived ability to apply knowledge in the workplace. The data that was collected and analyzed indicated that students in a competency-based course that is linked to specific competencies gained significant knowledge on skills needed to pursue training positions. Incorporating competencies provides students the ability to transition from graduation to the job knowing they have achieved industry-specific competencies that are needed in the workplace.

Data Sources

Four online graduate level courses were selected from a master’s program that is fully online which had recently had a complete redesign to be competency-based. Data was collected from courses that were taught in the Fall 2019 semester. Student survey data was obtained from the following courses:

  1. EAC 556: Organization Change in HRD (ATD Competency Area: Change Management)
  2. EAC 580: Designing Instructional Systems in Training & Development (ATD Competency Area: Instructional Design)
  3. EAC 581: Advanced Instructional Design (ATD Competency Area: Instructional Design)
  4. EAC 582: Organization & Operation of Training & Development Programs (ATD Competency Area: Managing Learning Programs)

Participants

The participants were graduate students currently enrolled in courses that were designed for a fully online master’s degree program in Training and Development with a focus on Instructional Design during the Fall 2019 semester. Data was specifically collected from online students from four of the 12 required graduate-level courses (n=87).

Findings for Research Question 1: Prior to Current Skill Level of Course Topic.

How students perceived participation in a graduate online competency-based course impacted their skill level was examined through survey questions that asked students about their prior skill level of a course topic and also asked them their current (end-of-course) skill level. Data from the survey around work preparedness, Prepared to apply knowledge to future work (FCP) and Prepared for future job in field (FJP), were analyzed. Findings indicate there is a statistically significant difference in prior skill level and current skill level of online graduate students after completing a CBE designed course.

Table 1

Prior skill level vs. current (end-of-course) skill level of students who completed a competency aligned online course


Findings for Research Question 2: Connection of Current to Prior Knowledge

In order to answer research question two, participants were asked to score on a likert scale of 1-5 , ‘Course activities helped me make connections of new to prior knowledge’. Data from the survey around skill attainment and connection, Prior skill level vs. current (end-of-course) skill level (DIFF) and Course activities helped me make connections of new to prior knowledge (OVN) were analyzed. The scores from this survey question help examine the impact of their connection of currently obtained new knowledge to prior knowledge. Findings indicate that students perceived a strong connection of the new knowledge in the course with prior knowledge on the topic.

Table 2
Students who completed a competency aligned online course perceptions of connection of new knowledge to prior knowledge



Findings for Research Question 3: Connection Between Course Knowledge and Workforce Application

Data on students' perceptions of connecting new knowledge to old knowledge related to their perceptions of being able to apply their new knowledge in the workplace was analyzed. Findings indicate that as connections prior to new knowledge increased so did the ability to use new knowledge in the workplace (r=.73).

Future Recommendations

Recommendation #1: Competency selection. Not all fields have an existing list of competencies to draw from or align with for education purposes. It is helpful to create a list of competencies from experience, findings from the literature and have the list of competencies examined by other stakeholders such as workplace leaders, hiring agents, accreditation and credentialing agencies, and other experts who have a solid understanding of the skills needed for success in their field.

Recommendation #2: Mapping. Just as course designers map overarching course learning objectives to smaller module learning objectives, when mapping competency-linked courses you map the course deliverables, content, and readings to the competencies. This helps organize the content and deliverables to ensure all competencies are addressed in the curriculum and measured through assignments, quizzes, discussion forums, and projects.

Recommendation #3: Make it known. It is imperative that students are informed that the course is competency-based (or linked) and which specific competencies are aligned throughout the course. Providing a map of the alignment between course and module learning objectives with competencies is helpful so that students see the connection between their learning and the competencies. Students, therefore, possess a narrative to identify their new skills. They can then share those strengths by stating their proficiency in specific competencies in written form on resumes and/or in verbal form during job interviews.

References

Altay, B. (2014). User-centered design through learner-centered instruction. Teaching in Higher Education, 19(2), 138-155.

Association for Talent Development. (2013, January 3). ASTD debuts new competency model, redefines training excellence. https://www.td.org/insights/astd-debuts-new-competency-model-redefines-training-excellence

Foundation for Critical Thinking. (2019). Our concept and definition of critical thinking. https://www.criticalthinking.org/pages/our-conception-of-critical-thinking/411

Gross, B., Tuchman, S., & Patrick, S. (2018). A National Landscape Scan of Personalized Learning in K-12 Education in the United States. iNACOL.