Becoming an Evidence-based School /Organization
Approximate reading time: 4 minutes
¿What kind of data does your school/organization collect? What is done with this data? How is it analyzed? Does it inform decision-making? How does leadership learn from the data? In this article, I tell you how having a constructive evaluation culture can benefit your school, organization, or educational program. And when I say evaluation, I don't mean tests, but rather evaluation as a learning process.
What is program evaluation and why is it important in educational organizations?
Program evaluation is a systematic process of collecting, analyzing, and using information (or data) to assess the effectiveness, efficiency, and impact of programs, policies, or projects, with the ultimate goal of improving them and informing decision-making.
Generally, organizations (and schools especially) generate a lot of data. However, in my own experience as an educator not all educational institutions have the knowledge and resources to organize and analyze all this information. In fact, many overlook precious data that can be collected and used for learning and improvement.
Even when some schools have established data collection tools and processes, these tend to be used mainly to comply with protocols and regulations of the Ministries of Education and other regulatory entities in the system. Information is rarely collected and used to learn about the organization itself or to make evidence-based decisions.
To gain insight into how schools collect and use data, I surveyed administrators and teachers from several schools, both private and public, in the Dominican Republic. The results confirmed many of my initial hypotheses:
a) Many schools do not have a system or strategy for data collection and analysis in place.
b) Evaluation in schools focuses primarily on student academic performance.
However, instruction is not the sole purpose and operation of a school or learning organization, and student performance is certainly not the sole or most important outcome.
There are many other operations and indicators that influence the quality and overall health of educational programs.
According to my survey results, schools often fall on different parts of the evaluation spectrum. Identify where your school/organization falls:
How do education organizations benefit from cultivating a program evaluation culture?
The main reason for establishing an evaluation culture is that it helps make better decisions. Consider the following examples:
Example 1:
A school is losing students. Each year, there is an increase in the number of students leaving and a decrease in the number of students enrolling. The school’s administration and staff may wonder what is happening and what can be done about it. Is this loss of students related to the quality of instruction? Perhaps the school climate? Or the cost-benefit ratio? Could it be due to factors external to the school, such as demographic changes? Is the loss of students concentrated in a specific grade or level?
If the school gathers information from different sources to analyze this phenomenon, it could answer these questions and make a decision based on the true cause of the problem.
Example 2:
An organization that provides foreign language programs in a touristic community plans to modernize its English program. What components would be worth including? How different would this new program be from the previous one? What changes or efforts would be required from each stakeholder in the organization to implement this new program? Is it better for the organization to design its own program or invest in a standardized one? Who can be consulted to develop an appropriate curriculum? What are the needs of the students and the community? What do the organization's English instructors think about the plan? What does research say about effective English programs?
If this organization answers each of these questions by collecting and analyzing adequate information, it will likely make a decision that will meet the needs of the institution and benefit all stakeholders.
Where and how to begin?
Even when there is awareness about the importance of evaluation, some organizations wonder when, where, and how to begin. The question of timing is easily answered. As Kusek and Rist explain in their book Ten Steps to a Results-Based Assessment and Monitoring System:
Whenever there are concerns or questions, it’s a good time to gather evaluative information.
The "where" and "how" are a bit more complex. The starting point depends on what you want to evaluate and the evaluation questions. The questions usually indicate the path to follow. Is it a design question? Is it an impact question? A process question? Is it about a cost-benefit?
Once the purpose of the evaluation is determined, two crucial components come into play:
a) Involving the right people and b) asking the right questions.
But that's a topic for another article.
I hope this information has been helpful to you!
Would you like to implement an evaluation system in your organization but don't know where to start?
Our consulting services can help you. Email us at connect@eduactivo.ca