Assessment of Program Learning in Accelerated Capstone Courses

Abstract

A culminating experience is incorporated into most graduate programs. Capstone courses are one methodology for structuring this experience. These courses are designed to integrate student learning throughout their studies, making them a natural point for program and student learning assessment. The development of capstone courses must consider the perspective of stakeholders with an interest in the program’s outcomes. Challenges in curriculum development can occur when seeking to embed assessment within accelerated adult programs. A case-study based capstone presents an opportunity for the effective demonstration of student application of program learning. An example is presented of a case-based capstone designed to meet the needs of stakeholders for effective program assessment.

Introduction

In graduate programming, the capstone course is the place where assessment of student learning culminates. Just as the capstone of a structure serves to lock the stones in place, the capstone course is intended to “cement” knowledge gained throughout the program. To ensure program learning objectives are met and for continuous improvement it is important for programs to assess student learning throughout the curriculum. This paper will discuss the purposes of the capstone, challenges of designing effective curriculum supporting program assessment in accelerated adult programs, and describe an effort to meet these challenges.

The Context

A culminating experience is incorporated into most graduate programs. They are structured in a variety of ways, including: formal courses, senior seminars, independent, portfolios, examinations, internships, and service-learning requirements (Sum, 2015, van Acker & Bailey, 2011). Designers of these culminating experiences face tensions around teaching content verses developing skills, integration and consolidation of knowledge verses professional development, encouraging specialized knowledge verses a broader context (Mowbray, 2015). Accelerated online learning adds another layer of challenge due to its natural time constraints and often widely distributed student population. The development process is further tested by the multiple stakeholders to be considered. Students, faculty, academic administrators, accrediting bodies, and others have an interest in the outcome of learning.

The Stakeholders

Each stakeholder in the process of this culminating experience have priorities that can conflict. Students, as the reason for the program’s existence, are the leading stakeholder. In general, they seek relevant content that has value to employers. As stewards of knowledge, faculty seek confirmation of their work by students, peers, and administrators. Faculty are interested in the success of their students, but are also aware of the value of high student evaluation scores. In addition to control of curriculum content, they hold a responsibility for passing on their own acquired learning. Academic administrators determine which programs will be pursued and maintained, allocation of resources, and the evaluation methods for programs and faculty. Finally, there are the other interested parties. Accrediting bodies, for example, desire to assure payers and the public regarding the quality of the programs they review. (Todd & Magleby, 2005). Each of these stakeholders depend on some form of program assessment.

The Capstone

In pursuit of an “authentic” culminating experience, programs often seek to integrate previous learning through a capstone course. Learning in the program, up to the point of the capstone, is established through content-driven courses which seek to develop understanding and competency in the given subject matter. While these curricula sometimes seek to establish an interconnectedness between content areas and scaffolded learning, they can only scratch the surface. In contrast, a capstone experience is not based on a specific body of knowledge, but rather, the integration of student learning from content-specific courses (Seidel et al., 2019). This integration focus lends itself naturally to the assessment of program-level student outcomes.

The Adult Learner

Adult learners in accelerated programs are independent and self-directed and are always integrating new content into everyday life experiences. They value problem-centered learning since it aligns with the challenges seen in their environments (Knowles, 1984). Many students in online accelerated programs have extensive work and family responsibilities. Time management is a significant issue. This, along with the accelerated learning pace, results in a perception of learning as task-oriented and compartmentalized, with the focus primarily on the course at hand (Schlichting & Fox, 2015). Design of the capstone course must consider these facts in concert with the perspective of other stakeholders.

The Example

One university sought to design a capstone course for their MBA-Healthcare Administration program in a manner which provided an authentic experience and met the needs of stakeholders for program assessment. While comprehensive examinations, standardized testing, student portfolios, and course-based assessments all play a role in assessment systems, the program felt case studies covering the core content areas provided the best mechanism for summative assessment. Graham (2014) notes the benefits of using case study methodology as: student analysis of issues, use of higher order skills, application of theoretical knowledge, self-reflection, opportunities for the improvement of teamwork and communication skills, critical thought, and practical application. Case studies provide student-centered experiences in real-life situations based on topics that demonstrate theoretical concepts in a healthcare setting (Powell et al., 2014).

A “casebook” text, designed to focus on the strategy and operations of several types of healthcare organizations, was selected (Sidel & Lewis, 2017). The course was designed for delivery in a six-week, asynchronous manner. Each week contained a case report assignment, a case reflection, and development of a portion of a comprehensive consultant report to a health system CEO based on a more expansive case study. The first week of the course set the context for the healthcare organizations under consideration and outlined the requirements for case reports students were to develop in weekly assignments. Students formulated case responses to problem considerations presented by the instructor. These problem considerations focused on the core content areas covered in the program (finance, governance, human resources, strategic development, etc.). The weekly case reflection required students to compose a journal entry contemplating ways program course content informed their development of the case report, and the applicability of these topics in their role as health administrators. Finally, students composed one portion of a comprehensive consultant report based upon deliverables from a consulting contract outlined in a memorandum from the healthcare system. The deliverables covered each of the key content areas. The completed consultant report was due during week six of the course.

The Outcome

The MBA-Healthcare Administration program has six learning outcomes. These outcomes cover the mission of the program, and student competencies in critical thinking, analysis, ethics, communication, and personal management. Grading rubric scoring provided data on the assessment of individual student learning in the program. End of course survey results and faculty course feedback provided the data used to evaluate the effectiveness of the capstone course.

Student end of course surveys were examined for the year before and after the implementation of the new capstone curriculum. Significant improvement in survey scores on course content was noted (p = .0004). This finding is valuable given the priority assigned to students as stakeholders. The second major stakeholder, the faculty, were also surveyed. Findings from faculty course feedback indicated significant improvement as well (p = .0004).

The Future

The amount of data collected to-date is not substantial enough to inform changes in the MBA-Healthcare Administration program overall. However, the favorable response by students and faculty demonstrate the capstone is moving the program in the right direction. Two areas for consideration in future iterations of the course are the inclusion of more formal instructor interaction and a team-based/cooperative learning component. The issue of formal instructor interaction outside grading and feedback begs the question of whether students, at this stage of knowledge and skill development, should be able to formulate problem responses independently. The reality is that many employers expect graduates to begin work with little supervision or training. Finding the right balance of interaction is a topic of current discussion.

The second area is just as important. The ability of graduates to not only work in teams, but to lead those teams cannot be understated. Both students and employers see the value of teams, yet in the accelerated online format, implementation can be challenging. The issues of time and distance are amplified. Our experience is students and faculty find team assignments problematic. The program is evaluating various technological improvements to better facilitate group assignments, but recognizes some issues are just part of the nature of teamwork in the current environment.

Conclusion

The development of an authentic capstone experience can contribute to quality assessment of program learning by students. They allow for the integration of content-knowledge and prepare students for the post-graduation environment. Capstone courses utilizing a case-based curriculum are an effective vehicle for assessing student learning in graduate programs. In accelerated online programs, the case design aligns with the preferences of a key stakeholder group, adult students. The priorities of other stakeholder groups can also be met through this methodology. While this work demonstrates the use of a case-based capstone for one university program, it is hoped that it will inform and spur the thinking of others.

References

Graham, J. (2014). Case-based learning in an electronic learning environment. Perspectives of Innovations, Economics, & Business, 14(4), 194–203.

Knowles, M. S. (1984). Andragogy in action: Applying modern principles of adult learning. Jossey-Bass.

Mowbray, J. (2015). The postgraduate capstone experience: Negotiating the pedagogical tensions. Journal of Learning Design, 8(2), 43–52. https://doi.org/10.5204/jld.v8...

Powell, D. C., Saint-Germain, M., & Sundstrom, L.-M. (2014). Using a capstone case study to assess student learning on NASPAA competencies. Journal of Public Affairs Education, 20(2), 151–162. https://doi.org/10.1080/152368...

Schlichting, K., & Fox, K. R. (2015). An authentic assessment at the graduate level: a reflective capstone experience. Teaching Education, 26(3), 310–324. https://doi.org/10.1080/104762...

Seidel, L. F., Lewis, J. B., & Arcand, C. (2019). The authentic capstone experience: Beginning the conversation. Journal of Health Administration Education, 36, 93–110.

Sidel, L. F., & Lewis, J. B. (2017). The middleboro casebook: Healthcare strategy and operations (2nd ed.). Health Administration Press.

Sum, P. E. (2015). Capstone courses and senior seminars as culminating experiences in undergraduate political science education. In J. Isheyama & W. J. Miller (Eds.), Handbook on teaching and learning in political science and international relations (E. Simon, Narr.). Edward Elgar Publishing. https://doi.org/10.4337/978178...

Todd, R. H., & Magleby, S. P. (2005). Elements of a successful capstone course considering the needs of stakeholders. European Journal of Engineering Education, 30(2), 203–214. https://doi.org/10.1080/030437...

van Acker, L., & Bailey, J. (2011). Embedding graduate skills in capstone courses. Asian Social Science, 7(4), 69–76. https://doi.org/10.5539/ass.v7...