ANALYZING PROGRAM EFFECTIVENESS BASED ON FIVE-YEAR AGGREGATE SURVEY DATA: A DESCRIPTIVE STUDY OF METHODS, RESULTS AND PATTERNS

File
Barredo R.1, Raynes E.1, Wenger K.1, Tan J.1
1Tennessee State University, Physical Therapy, Nashville, United States

Background: Program effectiveness refers to the extent by which a program is successful in meeting its goals and objectives. An analysis of program effectiveness involves the use of metrics used by the program to provide empirically-based data that help inform program effectiveness and promote programmatic change.

Purpose: The purposes of this study are as follows:
(1) to determine how the program is meeting its graduate student outcomes;
(2) to identify emergent patters from exit, alumni, and employer surveys; and
(3) to identify from the aggregate data programmatic strengths and weaknesses.

Methods: This study utilized a descriptive research design through a review and analysis of survey data. Survey results from exit (at the end of the students’ matriculation), alumni (six months after graduation), and employer (one year after graduation) surveys for the Classes of 2009 to 2013 were used. The items in the survey instrument were similar with all administered surveys and listed all of the program’s graduate outcomes. Respondents used a Likert-type scale (Excellent 4, Good 3, Fair 2, Poor 1) to assess whether or not they (for exit and alumni surveys) or their employers (for employer survey) were able to perform these outcomes. Data analysis included descriptive statistics and pattern analysis. To preserve the anonymity and protect confidentiality, names were coded and protected from disclosure.

Results: The program´s criterion for success was a rating greater than 3 for each outcome statement. Mean scores demonstrated that the program met this threshold. Additionally, pattern analysis indicated that mean scores increased between exit and alumni surveys. The same pattern existed between alumni and employer surveys. The researchers assumed that the increase in mean scores may be attributable to a change in the respondents’ life roles and the benefit of hindsight. Consistent among all three surveys were the following outcomes with the highest mean scores: practice in a safe, legal, and ethical manner; and provide therapy to a diverse population with respect to patient differences. Also consistent among all three surveys was the following outcome statement with the lowest mean score: use the principles of evidence-based practice.

Conclusion(s): The results indicated that the program was effective in meeting its graduate student outcomes. Mean scores for all survey items increased from the time the exit survey was administered to the time the alumni survey was given six months later. This increase may have resulted from a change in life roles and the benefit of hindsight— from being new graduates with no work experience to being physical therapists with full time employment. The graduate outcome with the lowest mean score—use the principles of evidence-based practice— may be explained in part by the perceived lack of time, demands of work, and access to resources. More research is needed to confirm these explanatory models.

Implications: In this study, the DPT program utilized graduate, alumni, and employer surveys to determine program effectiveness.

Funding acknowledgements: None

Topic: Education

Ethics approval: This study has been approved by the Institutional Review Board of Tennessee State University.


All authors, affiliations and abstracts have been published as submitted.

Back to the listing