Nursing Assessment and Evaluation Miranda Aitken State

Running head: NURSING ASSESSMENT AND EVALUATION
Nursing Assessment and Evaluation
Miranda Aitken
State University of New York Institute of Technology
1
NURSING ASSESSMENT AND EVALUATION
2
Differentiation between Assessment and Evaluation
Billings & Halstead (2012) outline four steps in the teaching-learning continuum: assessment,
planning, implementation, and evaluation. Assessment can be further divided into three components:
curricular attributes, faculty attributes, and student attributes (Billings & Halstead, 2012). Assessment
means obtaining information for the specific purpose of understanding and improving student learning
(Billings & Halstead, 2012). Authentic assessments are strategies for learning that require students to
demonstrate the same competencies, or combinations of knowledge, skills and attitudes that they need
to apply in a similar situation in the professional workplace (Raymond, Smith & Gray 2013).
Evaluation is a means of appraising data or placing a value on data gathered through one or more
measurements, and is the final step of the teaching-learning model (Billings & Halstead, 2012). It is also
a process that involves a series of actions including: identifying the purpose of the evaluation, identifying
a time frame, determining when to evaluate, selecting evaluators, choosing the evaluation
design/framework or model, selecting an evaluation instrument, collecting data, interpreting data,
reporting the findings, using the findings, and considering the costs of evaluation (Billings & Halstead,
2012). In other words, evaluation ends in a judgment, whereas assessment ends in a conclusion.
Two common forms of evaluation are formative and summative evaluation, each with different
advantages and disadvantages. Formative evaluation takes place during the program and looks to
identify progress towards purposes, objectives, or outcomes to improve the teaching and learning
(Billings & Halstead, 2012). Summative evaluation takes place at the end of the activity or instruction and
emphasizes to what extent objectives were met (Billings & Halstead, 2012).
NURSING ASSESSMENT AND EVALUATION
3
Stakeholders, Roles & Program Evaluations
Curriculum evaluation is an ongoing process to ensure the delivery of quality education. The role
of the nurse educator, with regard to evaluation, is to use the data collected to identify the strengths and
weaknesses of a lesson or curriculum to improve the student learning experience. It is considered a
competency of nurse educators by the National League for Nursing and is an expectation for national
accreditation (Giddens & Morton, 2010). The administrators provide the vision and leadership, all faculty
members participate in curriculum evaluation, data collection and analyses, and the formulation of
recommendations for program decision making (Keating, 2011).
Stakeholders should be involved in the process, and there should be congruency between
expectations of the stakeholders and the program’s mission, philosophy, goals, and outcomes (Billings &
Halstead, 2012, p. 508). Oermann & Gaberson (2014), note that the consequence of evaluation and
interpretation of results to students and other stakeholders must be considered. Stakeholders are
anyone with a vested interest in the outcome of the program including administrators, educators,
community members, regional accrediting bodies, legislature and students. Other stakeholders include
the local health care agencies that serve as clinical placement sites, and employers who offer jobs to
graduates. Students are considered to be valid and reliable informants about educational policies,
procedures, and resources, contributing effectively through focus groups, course evaluations, surveys,
and fieldwork evaluations (Carlson, 2013).
The institution should have a plan for program evaluation and include the various stakeholders.
Many institutions establish advisory committees of the various constituent stakeholders for additional
information to ensure that program outcomes or goals are appropriate, to provide input for curriculum
planning, and to develop evaluation questions and tools to determine if market needs are being met
(Billings & Halstead, 2012, p. 509).
NURSING ASSESSMENT AND EVALUATION
4
Program evaluation is the process of judging the worth or value of an educational program
(Oermann & Gaberson, 2014, p. 365), and can be defined as a systematic operation that involves data
collection, observations and analyses, and are necessary to respond to accreditation reviews by state,
education and professional accrediting bodies (Billings & Halstead, 2012). The purposes of evaluation
are to facilitate learning; to identify problems, ineffective teaching practices, and curriculum deficits; to
make decisions regarding the assignment of grades, determining whether to offer promotions or tenure;
to improve products such as replacing textbooks or adding content to a study module; to judge
effectiveness and whether goals and standards are being met; and to judge cost effectiveness (Billings &
Halstead, 2012, p. 424).
Conceptual Models of Evaluation
Choosing an evaluation model is a way to organize the process and represents the ways the
variables to be evaluated are arranged, observed, or manipulated to answer the evaluation question
(Billings & Halstead, 2012). Advantages to using an evaluation model include making priorities about the
variable and providing structure to all involved in the process. Conceptual models of evaluation include
decision-oriented models, such as the context, input, process, and product (CIPP) model, logic models,
benchmarking, evaluation processes models, formative and summative evaluation (Keating, 2011).
Decision-oriented models focus on internal standards of quality, value and efficacy in effort to
make decisions about the program Oermann & Gaberson, 2014). Originally intended to focus on
program improvement instead of proving something about the program, CIPP education evaluation
models have proven useful across a variety of educational and non-educational evaluation settings (Frye
& Hemmer, 2012). Context evaluation identifies the target population and assesses its needs; input
identifies and assesses system capabilities; and process detects defects in the design or
implementation; and product evaluation is a collection of descriptions and analyses of outcomes and
correlates them to the objectives- resulting in the interpretation of the results (Billings & Halstead, 2012,
NURSING ASSESSMENT AND EVALUATION
5
p. 427). This model is more agile than more linear models, such as the logic model, in terms of
evaluating programs with dynamic and complex elements. A CIPP Context evaluation study is typically
conducted when a new program is being planned, or when an established program is undergoing
planned change or must adapt to changed circumstances, such as a program with new leadership (Frye
& Hemmer, 2012).
Frye & Hemmer (2012) note that a logic model approach can be very useful during the planning
phases of a new educational project or during program revision, and can assure that the educational
program, once implemented, actually focuses on the intended outcomes. It is a model that may be
especially useful when more than one person is involved in planning, executing, and evaluating a
program (Frye & Hemmer, 2012), and to gain an understanding of the proposed program. The process of
developing a logic model should begin by bringing all key stakeholders together to share an
understanding of the proposed program, thus helping to build consensus (Hulton, 2007).
Benchmarking refers to the measurement and comparison of selected criteria with previously
recognized ideal criteria (Billings & Halstead, 2012, p. 428). Benchmarking allows programs to measure
their own success and standards or comparisons to similar institutions. This model may allow institutions
to compete for students, motivate faculty to strive for excellence, or collaborate with other institutions to
share strengths and improve programs (Keating, 2011, p. 299). The National Council Licensure
Examination (NCLEX) pass rates are typically used as a benchmark measuring the quality of institutions.
Formative evaluation strategies include course evaluations, student achievement measures,
teaching effectiveness surveys, satisfaction surveys (may be faculty, administration, student or staff),
NCLEX readiness, retention/attrition rates, and cost-effectiveness of the program (Keating, 2011, p.
300). One method of formative program evaluation is the use of standardized testing, such as the
Assessment Technologies Institute (ATI), in which students are tested each semester and ranked
NURSING ASSESSMENT AND EVALUATION
6
nationally (Billings & Halstead, 2012). This can be used to determine student progression through key
concepts in the curriculum (Billings & Halstead, 2012, p. 425).
The accreditation model uses a peer-reviewed, evidence-based evaluation approach.
Accreditation models typically use a combination of self-study and site visits by a team of peer reviewers.
When program assessment is based on accreditation, the goal is assessing whether the program meets
quality standards for the type of program (Oermann & Gaberson, 2014). The purpose of accreditation is
to ensure that at least the minimum standards of quality are met, provide recognition for funding, and
ensure consistency in quality across academic programs (Keating, 2011, p. 315). The Commission on
Collegiate Nursing Education (CCNE) accredits baccalaureate and master’s degree programs. Formally
known as the National League for Nursing Accrediting Commission (NLNAC), the Accreditation
Commission for Education in Nursing (ACEN) accredits all levels of nursing education: practical,
diploma, associate, baccalaureate, master’s, and clinical doctorate (Oermann & Gaberson, 2014, p.
366). The ACEN accreditation program is founded “on the belief that specialized accreditation
contributes to the centrality of nursing for the public good and provides for the maintenance and
enhancement of educational quality through continuous self-assessment, planning, and improvement”
(ACEN, 2014). Current accreditation criteria reflect the importance of evaluating program outcomes to
ensure quality and increase accountability (Oermann & Gaberson, 2014).
Nursing education programs must be approved by the state board of nursing in order to operate
and by the regional accrediting body to seek national accreditation (Billings & Halstead, 2012). While
obtaining national accreditation is voluntary, the public perception of the institution is linked to national
accreditation. Also, if schools wish to compete for external funding or prepare students for graduate
education, they will want to meet all levels of accreditation (Billings & Halstead, 2012, p. 504).
Accreditation indicates to employers that the student graduated from a quality program where specific
standards were met (Keating, 2011). The accreditation process also requires an evaluation plan that
NURSING ASSESSMENT AND EVALUATION
7
includes curriculum review, a timeline, a theoretical framework for evaluation, preparing the accreditation
report, and preparing for the site visit (Keating, 2011). An interview conducted with the director of the
department revealed these insights into the process:
1. What value did you have in the accreditation process prior to your accreditation experience?
JM- “I did not have a full appreciation for the amount of work that was necessary to prepare for the selfstudy. We spent hours preparing, writing and rewriting before sending out the self-study. I was quite
surprised how rigorous the site visitors schedule was and the need for coordinating many departments
besides the nursing department for interviews.”
2. How did your impression of the accreditation process change following the visit and subsequent
accreditation decisions?
JM- “The visitors we had were extremely knowledgeable and helpful but also critical of certain standards.
We had a visit in 2008 which ended with a warning status and return visit in 2010. The site visitors are
truly peer evaluators who I believe wanted to see our program succeed and they told us about many
other resources we could use to prep for the following visit. There were a couple of occasions where I
contacted the visitors for clarification and guidance; they were not punitive but supportive in their
suggestions.”
3. What advice would you give to new nurse educators preparing for their first accreditation experience
and why?
JM-“I would recommend that anyone preparing for their first accreditation read thoroughly through the
required “Standards and Criteria” for ACEN or CCNE. If the organization has a list of recently accredited
programs, I would contact as many as I could to find out how the process went and if they had any
suggestions based on recommendations made to them following their site visit. Some schools will even
NURSING ASSESSMENT AND EVALUATION
8
share their self-study when they know that the administrator is new and will consult with them on an as
needed basis. I do know that ACEN holds annual conferences on prepping for upcoming accreditation
site visits and I plan to attend one in the next couple of years to prepare for the 2018 site visit.”
4. How much time is involved in preparing for the accreditation process?
JM-“One should plan on at least 3 to 4 years. The faculty need to be involved in the process and there
are also other academic departments that need to be consulted in responding to the individual
standards. Using a quality improvement process of constant evaluation and reevaluation is imperative to
the success of the site visit. There needs to be at least 3 years of data to report and it takes 6 months to
a year to write the self-study and update the Systematic Program Evaluation Plan. It is also wise to have
a proof reader, such as an English Professor to assure that the study is written in “one voice”, and to
assure there are few if any grammatical errors. Additionally, if there is someone on the campus who can
help display the data in a graphic design it helps explain the data in a more professional and visual
manner.”
5. How are the faculty in the program involved in the process?
JM-“During our last accreditation in 2010, each faculty member had a Standard [ACEN] to begin
researching and collecting data. We worked as a department and held multiple meetings to organize
and prepare the outline for the self-study document. Additionally, faculty had to ‘live the study’ to be able
to respond to the questions asked by the visitors. Lastly, the faculty need to help prepare the students so
they understand what to expect when and if questioned by the site visitors.”
These statements certainly clarify the depth and commitment necessary from all internal
stakeholders involved in the accreditation process.
NURSING ASSESSMENT AND EVALUATION
9
Focus on the Future
The National League of Nursing (NLN) reports that in the future, changing socioeconomic factors,
developments in health care delivery and professional issues unique to nursing will lead to
transformations in nursing education, and subsequently, nursing evaluation (NLN, 2013). Ten trends the
NLN notes are going to play key roles in the new millennia of nursing education are (NLN, 2013):
1. Changing Demographics and Increasing Diversity
2. The Technological Explosion
3. Globalization of the World's Economy and Society
4. The Era of the Educated Consumer, Alternative Therapies and Genomics, and Palliative Care.
5. Shift to Population-Based Care and the Increasing Complexity of Patient Care
6. The Cost of Health Care and the Challenge of Managed Care
7. Impact of Health Policy and Regulation
8. The Growing Need for Interdisciplinary Education for Collaborative Practice
9. The Current Nursing Shortage/Opportunities for Lifelong Learning and Workforce Development
10. Significant Advances in Nursing Science and Research
The NLN reports that changing demographics and increasing diversity are affecting health care
priorities as well as the practice of nursing (2013). This is due to the aging population and estimates that
by 2020, more than 20 percent of the population will be 65 and older, with those over 85 constituting the
fastest growing age group (NLN, 2013). With this advanced age, comes the increasing acuity of patients
encountered in healthcare settings. Additionally, significant increases in the diversity of the population
affect the nature and the prevalence of illness and disease, requiring changes in practice that reflect and
respect diverse values and beliefs (NLN, 2013).
Student demographics are also changing. Ethnic and racial diversity of nursing schools has
increased dramatically, and students are entering schools of nursing at an older age and are bringing
NURSING ASSESSMENT AND EVALUATION
10
varying college and work experiences, as well as more sophisticated expectations for their education
(NLN, 2013). Shaw & Degazon (2008) discuss the need for recruiting and retaining culturally diverse
students into the nursing education to ensure that the diversity of the workforce aligns with that of the
general population (Keating, 2011, p. 362). Continued efforts need to focus on identifying obstacles to
nursing careers and to enhance access to nursing education for ethnic minority students.
Schools of nursing must increase their minority recruitment, retention, and graduation rates while
ensuring that all graduates are culturally competent (Keating, 2011, p. 160). Curricula and program
evaluation need to consider these factors as key elements of the program in order to meet the demands
of our changing healthcare environment.
Upon graduation from this program, I will hopefully continue my career as an educator. I am
currently responsible for the certification of an outpatient diabetes program. There are similarities as the
certification from the American Diabetes Association not only acknowledges the quality of the program,
but is directly associated with whether the program qualifies for reimbursement from Medicare. As part of
the recertification process, curriculum must be reviewed and revised, quality and education of the
educators must be maintained and reviewed, and although the certification is in effect for four years,
already we are collecting our accomplishments and planning for the recertification in 2018. This course
has definitely improved my understanding and appreciation for the evaluation process and will
undoubtedly be beneficial wherever my career as a nurse educator takes me.
NURSING ASSESSMENT AND EVALUATION
11
References
Accreditation Commission for Education in Nursing [ACEN] (2014). Retrieved from:
www.acenursing.org
American Association of Colleges of Nursing, (2014). About QSEN. Retrieved at:
http://www.aacn.nche.edu/qsen/about- qsen
Billings, D. M. & Halstead, J. A. (2012). Teaching in nursing (Fourth ed.). St. Louis, MO: Saunders
Elsevier.
Carlson, N. (2013). Beyond consumers and stakeholders: students "reflecting-on-action" as active
partners in program evaluation. Education Special Interest Section Quarterly, 23(2), 1-4.
Frye, A. W., & Hemmer, P. A. (2012). Program evaluation models and related theories: AMEE Guide No.
67. Medical Teacher, 34(5), e288-e299. doi:10.3109/0142159X.2012.668637
Giddens, J., & Morton, N. (2010). Report card: an evaluation of a concept-based curriculum. Nursing
Education Perspectives, 31(6), 372-377. doi:10.1043/1536-5026-31.6.372
Hulton, L. (2007). An evaluation of a school-based teenage pregnancy prevention program using a logic
model framework. Journal Of School Nursing (Allen Press Publishing Services Inc.), 23(2), 104110. doi:10.1177/10598405070230020801
Keating, S. B. (2011). Curriculum development and evaluation in nursing (2nd ed.). New York, NY:
Springer Publishing Company.
National League for Nursing (2013). The future of nursing education: Ten trends to watch. Retrieved
from: http://www.nln.org/nlnjournal/infotrends.htm#7.
NURSING ASSESSMENT AND EVALUATION
12
Oermann, M. H., & Gaberson, K. B. (2014). Evaluation and testing in nursing education (4th ed.). New
York, NY: Springer Publishing Company.
Raymond, J. E., E., Smith, R., & Gray, J. E. (2013). Learning through authentic assessment: An
evaluation of a new development in the undergraduate midwifery curriculum. Nurse Education In
Practice, 13(5), 471-476. doi:10.1016/j.nepr.2012.10.006
Quality and Safety Education for Nurses (QSEN), (2014). Competencies. Retrieved at: www.qsen.org

Download Report