External evaluation report EXTERNAL EVALUATION REPORT of the European eLearning Forum for Education Project by Tjeerd Plomp, ELFE project external evaluator Oosterbeek, the Netherlands 16th January 2006 This project has been carried out with the support of the European Commission in the framework of the eLearning programme. The information expressed in this publication reflects the views only of the author. The Commission cannot be held responsible for any use which may be made of the information. 1 External evaluation report Content Content .................................................................................................................2 1. Introduction to project – purpose of report ........................................................3 1.1. The ELFE project ........................................................................................3 1.2 Project approach..........................................................................................4 1.3 Structure of the report ..................................................................................4 2. The external evaluation of the ELFE project .....................................................5 2.1 Tasks and activities of external evaluator ....................................................5 2.2 Specific role of the external evaluator in the ELFE project...........................5 2.3 Specific evaluation activities and tools .........................................................6 2.4 Some problems encountered and addressed ..............................................7 3. Findings of external evaluator ...........................................................................8 3.1 On the methodology followed.......................................................................8 3.2 On the process of conducting the project.....................................................9 3.3 On the results of the project .........................................................................9 4. Some final remarks and recommendations.....................................................10 2 External evaluation report 1. Introduction to project – purpose of report The ELFE project has been conducted by ETUCE (European Trade Union Committee for Education). The aim of this evaluation report is to assess whether the objectives of the ELFE project, expressed in the project proposal, have been achieved through the means proposed. To provide a context for reporting the evaluation findings, first the project aims and approach will be summarized (1.1), followed by the project approach (1.2) and a summary of the structure of this report (1.3). 1.1. The ELFE project As stated in Chapter 1 of the ELFE report, schools in the 21st century are facing many challenges to continue to make education relevant for the information society in which all citizens have to be life-long learners and many workers are expected to be able to work in teams, to acquire independently knowledge and skills and to be able to communicate in a variety of contexts and using a variety of means. As indicated in the project application, it was felt by ETUCE that there is much experience regarding the use of e-Learning 1 in tertiary education, but very little regarding primary and secondary education. In the latter ones, due to the age of the students, it is very important to discuss how the social learning environment can be dealt with in connection with eLearning methods. In this context, the overall aim of the project is to create a European eLearning Forum for Education (ELFE) that will allow ETUCE and the wider society to get a better understanding of the strengths and the weaknesses of using ICT in those two stages of education. The Forum is a concept that covers three seperate elements or components: 1) the ELFE website includes a forum section for debating the use of ICT in education, 2) the whole ELFE-website and ETUCE's additional information regarding the ELFE activities is used to provide information about the activities and raise interest in the topic both at school level and at union level, and finally 3) the two ELFE conferences, where at the first one schools meet with pedagogical experts, and at the second teacher unions meet to debate union policy and policy recommendations regarding the use of ICT in education. So ELFE as a forum includes processes (using information from schools) evoking teacher unions to put the use of ICT in teaching and learning on their agenda, which is necessary for drafting policy recommendations. Through peer reviewing of previous experiences of eLearning pilot projects carried out in five different European countries (Denmark, UK, Germany, Norway and Portugal), the ELFE project has worked on the following specific aims: 1. analyse and share good experiences and identify good practices in using ICT in schools; 2. study the possibilities of transferring these good practices to other schools and countries; 3. create a debate on how the European policy on eLearning and use of ICT in education should be developed. The project has been conducted by the project coordinator and the ELFE Steering Committee, supported by the ETUCE-secretariat, in charge of the project management. The project coordinator chaired the Steering Committee and organised and monitored the project, whilst the ETUCE-secretariat organised and coordinated all operational activities in the project. 1 ) In the context of this project, e-Learning is used as the use of ICT (modern information and communication technology) to enhance teaching and learning so as to make it more studentcentered and meeting the challenges of the 21st century. In this and the ELFE report eLearning and the use of ICT will be used synonymously. 3 External evaluation report The Steering Committee (SC) consisted of nine (9) persons: representatives from Teachers’ Trade Unions from five European countries, viz Denmark, England, Germany, Norway and Portugal, three experts in the area of ICT in education, and an external evaluator, who also participated actively as ‘critical’ friend’ in the project. Apart from the Trade Unions represented in the Steering Committee, other Teacher Unions (members of ETUCE) were involved in the project by discussing at two conferences the project findings as well as the conclusions and policy recommendations to the EU and the ETUCE and its membership. 1.2 Project approach In the project document approved by the EU, it was proposed that the first two project aims would be addressed by organising a number of school visits in the five countries participating in the ELFE project. For Aim #1: Analyse and share good experiences and identify good practices in using ICT, ELFE Steering Committee members have visited in each country three schools that were considered in the country as demonstrating in some way or another good practices in using ICT for teaching and learning. The ELFE teams consisted of two representatives of teacher unions (one from the host country) and one expert in ICT in education, To realise Aim #2: Study the possibilities of transferring these good practices to other schools and countries, a programme of school-to-school exchange visits of the 15 schools identified for Aim #1 was arranged in a way that each school would visit a school in another country, and would also host another school from another country for a number of days. It was believed that well-structured school-to-school reports would be a source to draw a number of conclusions related to Aim #2. The ELFE project elaborated the first two project aims in a number of analysis questions reflecting two perspectives that are important in this project, viz. the availability of an ICT infrastructure (as a necessary condition for pedagogical use of ICT) and the pedagogical and organisational aspects of intensive use of ICT in the school. Information needed to answer these questions was obtained from the school principal, teachers and students of the schools visited via interviews, as well as in two school questionnaires (for the principal and for the ICT coordinator). In each participating country, the three schools were chosen not based on a single criteria, viz. the best example of ICT for teaching and learning, but on a balanced set of general and specific criteria (see Chapter 3 of the report for an elaboration). To realise Aim #3: Create a debate on how the European policy on eLearning and use of ICT in education should be developed, a conference was organised in November 2005 with representatives from the EU and from all Teachers’ Unions represented in ETUCE. The main purpose of the conference was to debate a draft policy paper on the use of ICT in education prepared by the ELFE team. After the conference the draft policy paper was amended and afterwards adopted by the board of ETUCE. 1.3 Structure of the report Section 2 of the report presents the contexts in which the external evaluator had to operate and a summary of the evaluation activities. This will be followed in Section 3 by the evaluation findings whereby a distinction will be made between the methodology followed, the process of conducting the project and the results of the project. Finally in Section 4 some final remarks and recommendations will be presented. 4 External evaluation report 2. The external evaluation of the ELFE project First the assignment for the external evaluator will be presented (2.1), followed by a discussion of the specific filling-in of the role of the external evaluator for this project (2.2), the evaluation approach applied (2.3) and how a number of problems were encountered and addressed (2.4). 2.1 Tasks and activities of external evaluator The task of the external evaluator is according to the work plan to draft two evaluator’s reports: Deliverable I: an evaluator’s report at the end of work package 1 (first three phases of the project), i.e. after the school visits of the ELFE teams and the first conference in November 2004 in Brussels, Deliverable II: containing external evaluator’s report from the 3 following phases: school-to-school visits and final conference where the findings of the study be discussed. To fulfill these tasks, the external evaluator was expected to (see contract): Interview the persons involved in the project Analyse all the documents produced during the project by the different actors (pedagogical experts, teachers) Analyse and evaluate the questionnaires answered by the participants in the project. Participate in the seven Steering Committee meetings in Brussels Perform one visit to the selected schools with the Steering Committee members in Phase II and two visits in Phase IV Assist to the launching and follow up Conferences. 2.2 Specific role of the external evaluator in the ELFE project The tasks and the activities of the external evaluator, as specified in the work plan and summarized in 2.1, illustrates that the ELFE project would benefit much more from an external evaluator who would actively participate in the project - by being an active member of the Steering Committee; participating in the visits of the selected schools; analysing and evaluating the answers of the participants in the project; writing reports on the findings in the respective phases of the project, etc. - than from an evaluator playing his/her role in a passive way, from a more ‘distant’ position. From the very beginning of the project it was decided that the external evaluator would be an ‘evaluating participant’ and a ‘critical friend’. Both components of this metaphor are equally essential. The ‘friend’ role implied a participation in the project activities on ‘equal footing’ with all members of the Steering Committee (SC). The qualification ‘critical’ was operationalised by the external evaluator by challenging the project at critical stages and guiding and inducing the SC to make explicit how they were planning to conduct (each component of) the study. This combination of both being ‘critical’ and a ‘friend’ can be illustrated by the following example. During the SC meeting of January 26, 2004, the project evaluator queried the project how it would structure the visits of the selected schools so as to assure that (i) the project aims would be addressed per visit, and (ii) it would be possible to do a cross-school analysis resulting in a report based on systematic analysis of data collected in the school. This ‘critical’ query resulted in the conclusion that the ELFE project had to elaborate an operational research design, i.e. no set of analytical questions (derived from the project aims) and no instruments for collecting data – this was concluded less than three months before the first school visits would take place. In his role of ‘friend’ the evaluator provided the project with a number of drafts, viz. a set of analytical questions, a set of instruments for data collection (both interview check lists and questionnaires), as well as a structure for reporting the school 5 External evaluation report visits in a way that cross-case analysis would be possible. The SC has reviewed, amended and adopted these. In retrospect, the fact that the external evaluator participated in the ELFE project as an ‘evaluating participant’ who perceived his role as a ‘critical friend’ has prevented a situation in which an evaluator would have concluded at the end of the project period that the project may have had many weaknesses in its design and execution. To the contrary – the role of continuingly critically assessing the quality of the proposed steps and activities in the respective phases of the project has resulted in a similar continuous process of adaptations and improvements during the various phases of the project. Another benefit of this approach is that the intensive interactions with the project leader and the SC have prevented that the ELFE project would draw conclusions that would be an ‘overclaiming’ of what the data collected would allow for. This is being illustrated by what has been written at the end of section 2.1 of the ELFE report: “Given the small number of schools involved in this study, it is important to realise that the information of the schools do not allow for any generalisation to the education systems of the participating countries or any other country. But the project data formed a rich input for the conference discussions, which together with results and conclusions from related studies – have led to nuanced and relevant recommendations.” In addition to this it should be mentioned, that the ELFE SC in its meeting explicitly expressed an appreciation of the decision the project made in its planning stage, viz to have an external evaluator participating in the role of ‘critical friend’. The SC recognised that it had been forced to reflect more methodologically and that the project has been accomplished in a more systematic way resulting in better outcomes. 2.3 Specific evaluation activities and tools The specific role as external evaluator who participated in the study as a critical friend, resulted in it that evaluation activities were very much linked to a number of SC meetings during which the design and the operational parts of the study were discussed. The query for operationalization of components in the project (see above) resulted in the invitation by the project coordinator to draw up proposals for what information should be collected, for data collection procedures, and for a structure for the reports of the school visits. A memo on this (dated March 31, 2004) was discussed and with some amendments approved in the 3rd SC meeting (2 April, 2004). As a follow up the 3rd SC meeting, the evaluator further elaborated these components in close (electronic) interaction with all members of the SC, resulting in a number of documents: criteria for schools to be selected an overview of analysis questions as operationalization of project aim #1 interview schedules and questionnaires2 data collection manual matrix structure of the school visit reports (a matrix structure with two dimensions, viz. analysis questions and data collection instruments) A reason for proposing the specific matrix structure was to make it possible to compare the ELFE findings also with another larger study that has used a similar matrix structure. The ELFE SC supported the idea of using the instruments from the IEA (International Association for the Evaluation of Educational Achievement) and adopted the model. At a later stage of the project, also a proposal for the school-to-school visits (Aim #2) has been drawn up that has been approved by the SC. 2 ) the design for this part of the study was adapted from the IEA study SITES-Module 2. We acknowledge generosity of the International Association for the Evaluation of Educational Achievement (IEA) for allowing the project to use their instruments. 6 External evaluation report Of each school visit also a ‘narrative report’ was written that had the same structure as the matrix structure school report. Schools were asked to review the narrative reports and propose corrections where needed. This provided the ELFE project with an opportunity to validate the matrix structure school reports that served as the basis for the data analysis for Aim #1. The narrative reports had an ‘external’ function: they were placed on the website of the ELFE project, whilst schools could use them for dissemination purposes. Other activities that I conducted as external evaluator were: Presenting the project findings on Aims #1 and #2, as included in Chapters 1 to 5 of the ELFE final report. The approach chosen in these findings reflects that (from a research perspective) the ELFE project is a collection of case studies in various contexts. As a consequence, no generalizations to educational systems have been made, but trends could be reported, as well as observations and illustrative phenomena that appeared to be rich input for the discussions in the Steering Committee, the ELFE conferences and the policy recommendations drawn up by ETUCE. Presenting the ELFE findings at both conferences. 2.4 Some problems encountered and addressed The developments leading to the need for a more operational project design (as described in 2.3), put a certain strain on the project. It was not anticipated that much time had to be invested in developing the tools and instruments, in the period following the 3rd SC meeting of April 2, 2004, whilst the school visits were scheduled as of mid-June 2004. As a consequence the draft instruments for data collection as well as the proposed structure for the school visit reports could only be reviewed and commented on in a very narrow time interval, and the first two school visits also served as a sort of ‘try out’ for these instruments. A few conclusions emerged from these first visits: What worked well: the structuring of the data collection the structure of the matrix structure school visit reports the written questionnaires for the principal and the ICT coordinator. Some problems: the interview checklists were too long for the time available for interviews with the principal, teachers and students the variation among the schools (such as primary – secondary, general – vocational, differences between schools in pedagogical ICT use, cross national differences) forced the ELFE visitors to adapt the interviews to the specific context of the respective schools The recognition of these problems resulted in the decision that ELFE visitors should have the freedom to adapt the data collection to the specific context of the schools – within the limit of the topics listed in the instruments, should address the relevant topics listed in the interview checklists without being forced to address each detailed question included in the interview checklists as elaborations of the topics BUT should report the school visits according to the matrix structure school reports that had to be approved by each member of the ELFE team visiting that particularly school. The importance of the last point is that the ‘cross-school’ analyses and therefore the report of the project findings could be done on the basis of these matrix structure school reports. We can clearly affirm that – given the narrow time frame in which the project had to operate this approach provided a good solution: justice could be done to the variation between schools and countries, whilst at the other hand enough ‘standardization’ of the process was obtained to allow for well-considered ‘cross-school’ analyses. 7 External evaluation report 3. Findings of external evaluator This section on findings addresses the following topics: the methodology followed, the process of conducting the project and the results of the project. 3.1 On the methodology followed The project had at its start a clear work plan listing the activities to achieve the project aims: the SC meetings to discuss, elaborate and decide upon how to conduct the project; the school visits by SC members and the school-to-school visits to collect data needed and relevant to realize the project aim; the website as a platform for communication and debate; the conferences each serving its own goal: o the first one to discuss with representatives of the schools and a number of experts the preliminary findings o the second one to discuss with the membership of ETUCE the findings and draft policy recommendations. I believe that the work plan reflected a good approach to realize the project aims and to provide input for debate among the national teacher unions (the membership of ETUCE) resulting in valuable recommendations both to the EU and the ETUCE member organizations. Some detailed comments: The composition of the Steering Committee (teacher union representatives and experts) worked out well. The experts in the SC assured that a proper balance between the union perspective and the broader educational and scientific perspectives on integrating ICT for pedagogical purposes was obtained. Besides, the union representatives played a key role in arranging the visits to the schools. Also the selection of countries showed a good representation of Western Europe: two Nordic countries, two big mid-European countries and a Mediterranean country. Of course, many other countries could have been selected, but the project aims and the limited scope of the project justified this variation of countries and this balance in the selection of countries. The selection of schools within the countries resulted in a variety of schools reflected primary and secondary education, general and vocational education. The variety of national contexts added to the variety among the 15 schools that were selected for the ELFE project. Important was that the project took as starting point that ‘innovative pedagogical use of ICT’ is very much locally defined as countries differ very much in culture and policies, and therefore ‘innovative developments’ in such varied contexts are almost by definition locally determined. I conclude that the case study approach followed in the project was appropriate because the project did not strive for generalizations of system characteristics or indicators, but aimed at getting a better understanding of the strengths and the weaknesses of using ICT in primary and secondary education as an input for a discussion within ETUCE on policy recommendations to its membership and the EU. The way the case study approach was operationalized was good and the contribution of the IEA (International Association for the Evaluation of Educational Achievement) who allowed the project to using their instruments must be commended on. The research design worked out well as the matrix structured school reports from ‘ELFE teams’ appeared to be a good basis for cross-school analysis, and the few problems that occurred in the early stage of the school visits could be solved (see 2.3 and 2.4). 8 External evaluation report 3.2 On the process of conducting the project Just a few remarks will be made on the process aspects. The collaboration of the schools with the project was in general fine. Schools gave the ELFE visitors all information needed. The ELFE teams not only met with the supporters of ICT in teaching and learning within the school, but also with those who had doubts or were opposed. It was clear that the five participating countries are very different culturally and in their education policies, resulting in a variety of attitudes among schools and unions to the use of ICT in education. This has given the project some challenges, but it also provided an example of the effect of cross-European projects. Not only the SC meetings and the conferences, but certainly also the school-to-school visits provided platforms for discussion and sources of inspiration. Several schools reported that they have received new impulses and inspiration from the examples they saw in other countries. This did not only apply for e.g. the Portuguese schools where the integration of ICT in teaching and learning has not yet taken off very widely. Also schools with already advanced developments were inspired by seeing and experiencing how other schools had integrated ICT of teaching and learning. Another noteworthy point was the ELFE conferences were attended by trade union representatives from all EU and EFTA countries. So although the input for the discussion was based on the findings from 15 schools from five countries, participants representing trade unions from all countries endorsed the policy recommendations. 3.3 On the results of the project As written in the project proposal, the “overall aim of the ELFE project is to create a European eLearning Forum for Education (ELFE) that will allow ETUCE and the wider society to get a better understanding of the strengths and the weaknesses of using ICT in those two stages of education”. The Forum is a concept that covers three seperate elements or components: 1) the ELFE website includes a forum section for debating the use of ICT in education, 2) the whole ELFE-website and ETUCE's additional information regarding the ELFE activities is used to provide information about the activities and raise interest in the topic both at school level and at union level, and finally 3) the two ELFE conferences, where at the first one schools meet with pedagogical experts, and at the second teacher unions meet to debate union policy and policy recommendations regarding the use of ICT in education. So ELFE as a forum includes processes (using information from schools) evoking teacher unions to put the use of ICT in teaching and learning on their agenda, which is necessary for drafting policy recommendations. Through peer reviewing of previous experiences of eLearning pilot projects carried out in five different European countries (Denmark, UK, Germany, Norway and Portugal), the ELFE project has worked on the following specific aims: 1. analyse and share good experiences and identify good practices in using ICT in schools; 2. study the possibilities of transferring these good practices to other schools and countries; 3. create a debate on how the European policy on eLearning and use of ICT in education should be developed. Reviewing the project activities a few conclusions can be drawn with respect to these aims. On Aim #1: this aim has been realized as the ELFE schools can be seen as exemplary practices in their countries. The findings, reported in Chapters 1-5 of the final report, have been shared and discussed in the two conferences of the project. On Aim #2: school-to-school visits was the approach to address this aim. The visits took place during the last part of the project and it is too early to expect transfer of good practices in one ELFE school to other ELFE schools. Needless to say that transfer of good practices to nonELFE schools could not be realized. However, the project aim referred to ‘study the possibilities of transferring these good practices’ and with respect to this one may conclude 9 External evaluation report that in general (one exception) school delegations were enthusiastic about the opportunities offered and picked up many good ideas. One may conclude that a program of such school-toschool visits may contribute to transfer of good ideas and practices, but the context of a project like ELFE is too limited to make this happen to a substantial extent. On Aim #3: distinction has to be made between the ELFE website and the more general discussion that the project organized through its conferences. The ELFE website has been used in the project for sharing information, mainly meeting minutes and reports. It had a reasonable number of hits, but it has not been used much as a platform for debate. One may wonder why this is the case. One possibility is that is was too optimistic to expect that a project website would develop into a broad platform of debate. In case ETUCE wants such a debate platform among its members one may argue that another approach could have been more successful. It is positive that ETUCE has decided to support the ELFE website also after the end of the project to keep the debate area open after the adaption of the ICT policy paper and as a mean of distributing the ELFE findings to a broader audience. However, the debate on how the European policy on eLearning and use of ICT in education should be developed has taken place with enthusiasm at both conferences organized by the project. Teacher unions from all EU and EFTA countries participated in the second conference during which the findings of the study and draft policy recommendations were discussed and unanimously accepted. These have been adopted by the ETUCE Executive Board, on 7th December 2005. It is still open for comments from the member organisations. Finally, it will be put forward to the ETUCE General Assembly on December 2006. It should be stressed as a positive outcome that the ELFE project accomplished its major aim, viz it produced a set of recommendations for which the ELFE findings were used as the major source of inspiration. None of the ELFE findings were denied in the policy paper and it is in a way interesting that the Teachers Unions recognised findings like the need that the school leadership has to develop a vision for the pedagogical use of ICT and to determine clear priorities. One may conclude that the policy paper and its recommendations are a good and valued outcome of the ELFE project! 4. Some final remarks and recommendations In Section 3 already a number of remarks have been made on the functioning of the project and the SC as the main body in conducting the project. In this section only some final remarks and recommendations will be presented. The ELFE project as a meeting place of union representatives and researchers: The ELFE project has been an interesting meeting place between union representatives and researchers, among these the external evaluator. One platform of meeting was the SC, where the discussion focused on how to operationalize the work plan to realize the project aims. In general, the SC functioned quite well: it was a forum of discussion where clearly the union perspective and the researchers’ perspective confronted and complemented each other. For the researchers it was important to experience that the union representatives were not only looking at legal conditions and conditions of employment for teachers, but were equally open for the needs of education and schools to meet the demands of the information society. Besides the SC functioned as a good platform to discuss and elaborate how a good action plan had to be elaborated in an operational project design that in the end allowed for ‘crossschools’ analysis (see also below under ‘scientific approach or not?). The ELFE project functioned also as a good meeting place between union representatives and researchers at the two conferences. At the first conference, researchers from various countries (England, France, Netherlands, Ireland) provided input to the discussion, whilst at the second conference a small number of researchers served as sparring partners in the discussions about the draft policy recommendations. 10 External evaluation report Added value of the ELFE project for the various target groups of the project: Following the previous point, the ELFE project wanted to serve various target groups. As stated in the project proposal they are: - Teachers and Teachers’ unions, forming the main target group. They will exchange the experiences of eLearning pilot projects in a number of European countries from a pedagogical point of view. - Students in Europe, who will benefit indirectly from the project, as far as best examples and practices for their education will be identified. - Policy makers form another target group for whom the results of this project will be relevant (education sector). The project will contribute to their awareness of the changes that the use of ICT implies in education (both pros and cons). One can conclude that the project served most of these target audiences quite well: - Teacher Unions were actively involved in discussing the finding and drawing up policy recommendations; - Teachers were very much the focus in operationalizing the project aims and as a result the policy recommendations focus very much on the interest of teachers (of course next to the wider interest of education as a whole); - Policy makers were always in the picture as target group which clearly appears from the policy recommendations included in the project report, endorsed by the ETUCE Executive Board. - Students functioned as target group in different ways. In a direct way, students’ interests appeared clearly in the analysis questions, and students were interviewed during school visits, which is reflected in the reporting of the findings (see Chapter 5.3). In an indirect way the interest of students were clearly addressed as in discussing the project design, the findings and the policy recommendations the Teacher Union members in the SC had a clear focus on what education is needed in the information society, and were not only looking at legal and working conditions of teachers. Scientific approach or not? One of the points of discussion between the SC (especially the union representatives) and the external evaluator has been the point whether the ELFE project should be ‘scientific research’, or should stay a project that would provide good input for discussions on policy recommendations on pedagogical use of ICT. As external evaluator I have taken the position that also a project like ELFE (aiming at practical outcomes) should have a strong ‘chain of reasoning’ in the design of the route from aims towards results (like in scientific research) even where it is not aiming for contributing to the scientific body of knowledge. Informed decision making and informed discussions on policies need valid and reliable data and not just ‘impressions’ from school visits. Whether one want to state that the ELFE project followed a scientific approach or not, the approach chosen had to result in good data that would provide solid input for the discussions on policy issues. It was important that I could convince the SC that such a systematic approach had to be chosen and I want to compliment the SC letting it be convinced. More time needed for detailed project design: The action/work plan of the ELFE project had proper phases, but it appeared that in the stage of proposal development not much thought was given to the detailed project design. As a result, too little time was available for elaborating the action plan into an operational design with analysis questions, data collection instruments and a data analysis plan. With reference to what has been stated about this in the previous sections as well, it is strongly recommended that a follow-up project assures sufficient time for those components, including time for trying out and field testing instruments for data collection. On the role of the external evaluator: In section 2.2 I wrote that the ELFE work plan project illustrates that the project would benefit much more from an external evaluator who would actively participate in the project - by being an active member of the Steering Committee; participating in the visits of the selected schools; analysing and evaluating the answers of the participants in the project; writing 11 External evaluation report reports on the findings in the respective phases of the project, etc. - than from an evaluator playing his/her role in a passive way, from a more ‘distant’ position. From the very beginning of the project it was decided that the external evaluator would be an ‘evaluating participant’ and a ‘critical friend’. Looking back at the end of the ELFE project, I can conclude that from my perspective the ‘critical friend’ role of the external evaluator worked out quite well in the project. As evaluator, I could focus on the key question: with what design, means, methods and techniques will the project realize its aims. I stayed out of the discussions on the interpretation of the results, and could stick to my role of keeping the project focused. In this report, I have given an account of how I perceived and fulfilled my role and my conclusion is that this approach is a suitable one for ‘field-oriented’ projects like ELFE and that – if applied properly with mutual respect and recognition of each other’s role – it can be satisfactory for both the SC of the project and the external evaluator. 12 External evaluation report 13
© Copyright 2024 ExpyDoc
