Development-oriented assessment: from grades to words
Summary
Description of the Fontys ICT assessment policy in the face of formative review and assessment in the full-time study programmes and part-time study programmes (Education for Professionals), in addition to and in accordance with the applicable TER.
Fontys RACI ICT
To learn more about how this page fits within RACI, see the RACI domains Governance and Education.
This lemma describes the policy from Fontys ICT in the face of formative review and assessment in the face of assessing and review in the full-time study programmes and part-time study programmes (Education for Professionals), in accordance with the applicable TER. In the coming period, this policy will be applied in the development of the new educational units across the different curricula. Chapter 2 presents the vision on education and assessment discussed in connection with each other. Both are inextricably bound up with each other. Chapter 3 focuses on the integral semester assessment based on portfolio. Chapter 4 deals with the assessment quality and finally Chapter 5 deals with the organisational embedding.
In this chapter, we will first briefly outline which vision is to be adopted. Fontys ICT has on education, what principles we apply and how this translates into testing and assessment.
Fontys ICT wants to train students according to socially constructivist principles that are implemented in powerful learning environments. Simons defines powerful learning environments as:
'learning environments in which learners are challenged to actively learn together, with a clear view of the functionality of learning and what has been learned (application perspective), in which learning is situated in a concrete context and in which the authenticity of that context is as high as possible' (Simons, 2000, p. 44).
Fontys ICT-education is current, occupationally-oriented, innovative and development-oriented. Students learn to create within a knowledge domain and work field that constantly changes and innovates. In the education the activation, instead of reproducing, of knowledge is therefore highly valued and stimulated. Education is meaningful, is lifelike and is linked to the professional context.
Metacognitive skills as self-responsibility, self-reflection and feedback play an important role in the process from development to practitioner. Lecturers know the students, are in dialogue with them and provide specific, concrete and meaningful feedback. There is a lot of interaction and contact between lecturers and students, between teachers themselves and between students themselves. Personal attention and trust form the basis for this. As the student progresses in the study programme, the student's own responsibility for the learning process increases and the supervision becomes less.
Student centred education does justice to the (individual) characteristics of all Fontys ICT students and their learning processes. The learning environment is rich in feedback and encourages student activity. The Fontys ICT learning environment wants to bring out the best in students and lecturers and thus to stimulate excellence in a talent-oriented way . In the education of Fontys ICT, lecturers and students work together to create knowledge. Students are seen and known and are approached personally. They learn in interaction with each other and in interaction with the lecturers. This makes demands on the working posture of both students and lecturers. Growth and development of students and lecturers go hand in hand.
In the vision of Fontys ICT a ('qualified to start') professional is explorative, inquisitive, in search of challenges and inspired by new possibilities and applications of ICT. Partly as a result of this, he is constantly aware of the continuously developing ICT work field. Both the applications themselves, the way in which they are achieved, the platforms on which they will be performed but also the tools with which they are made change constantly and at a fast pace. In this dynamic world, the ability to acquire new knowledge and skills is constantly evolving and will continue to do so throughout the working life in ICT. Assuming a certain knowledge base, an explorative attitude and a certain level of work and thinking, this skill in this field of expertise is more important than being able to reproduce ready knowledge. After all, some of it is already outdated or superseded after just a few years. It is therefore that the (starting) professional is aware of the need to continue to develop and is able to steer this himself in a professional context, where attention is paid to both the personal and the professional component. This control includes self-awareness (what do I want), the capacity for critical self-reflection (what am I good at?, what is not yet) and a certain purposefulness (what does it take to achieve my goals).
The educational model including the assessment programme of Fontys ICT is aimed at turning incoming students into skilled (master) professionals. In the propaedeutic phase the incoming student is guided towards an active and enterprising attitude to study. During the propaedeutic phase the student learns increasingly to be able to take responsibility for its own learning process in preparation for the bachelor's phase. The learning units increase in size and complexity during the propaedeutic phase and certainly in the bachelor's phase and steering the study programme where assessing serves as an activating element, is diminishing in importance and is giving way to guidance on self-chosen ways and means to learning objectives to achieve and demonstrate. Steering of the learning process is gradually changing into guiding it in a way that is appropriate to the individual student's needs. In the master's phase, the steering of the learning process lies entirely with the student. The student determines the content of his education, whereby the complexity of the assignments and solutions is aimed at applicability in other sectors. In addition, the student can make use of flanking education that is offered through workshops and inspiration sessions, among other things.
Fontys ICT's lecturer is constantly aware of his development as a lecturer. By having a learning attitude towards one's own field of expertise, professional practice, student learning, or other aspects, he's an inspiring role model for students. He has an explorative attitude and is involved in the learning and development of their students. The lecturer works in a student centred way, ensures a learning environment with meaningful and qualitative feedback and inspires the student by establishing a link with professional practice and other elements of the curriculum.
Student centred education asks for student centred assessment. This is in line with the literal meaning of the term 'assessment': 'sit next to someone' (Dochy et. al., 2015). By sitting next to the learner throughout his learning process, the lecturer provides guidance and development-oriented feedback and 'feed forward' on the student's overall performance. Not only the student thus gains insight into his learning process, also the lecturer gets a picture of the student and his development during the education by collecting and interpreting information on the performance of the student. This brings the formative function of assessment, or 'assessment for learning' in a central position (Dochy et. al., 2015). In addition, Cohen-Schotanus (2012) also underlines the importance of frequent feedback moments, for the quality of learning and thus the promotion of study success.
During the feedback moments, both the student and the lecturer gain insight into the student's learning process with regard to the learning outcomes. Recording of oral feedback is mandatory and must be recorded at least in FeedPulse. In principle, it is up to the student to note the feedback themselves in FeedPulse. However, it is the lecturer's responsibility to ensure that the feedback is actually noted.
In addition to the development-oriented feedback in words, the development-oriented feedback scale is used. The purpose of this feedback scale is to provide the student with an answer to the question: 'Where are you at the moment in your development with regard to learning outcomes? The Project Leader Programme Implementation (PLOU) coordinates the formative assessment for an education semester (see section 5.2.3) and decides whether or not to use the development-oriented feedback scale throughout the semester, in addition to the substantive feedback. The development-oriented feedback scale is used according to the following assessment guidelines:
Development-oriented feedback scale
It is important to note here that this is an absolute scale. It is a snapshot of a student's current growth based on the established learning outcomes. It may therefore also be the case that a student's development is declining compared to the previous measurement moment, for example when he does not continue to show certain behaviour in practice that was determined earlier.
Within the semester, there are at least three formative assessment moments where feedback is given by assessors for each learning outcome using the developmental feedback scale. The last of these three formative assessment moments is no later than two weeks before the assessor meeting. This allows the student to take up and process the feedback received.
Assessment guidelines
In all cases, the assessors decide on the assessment by mutual agreement. The only set rule in assessment is:
This rule allows the student to focus on key areas of improvement. It should not be interpreted as the minimum requirement to achieve a Satisfactory assessment.
Guidance
This development-oriented feedback scale can be used as follows:
Fontys ICT believes that the number of study components that will be assessed in a summative way, must be limited. After all, we strive to ensure that the focus of the student and its learning process is aimed at more integral learning objectives over the long term. The aim of this is to prevent studying from becoming equivalent to obtaining test results whose intrinsic meaning, the goal and the inspiration are out of the picture. With development-oriented feedback, we want to focus on this: it is the feedback that shows the development and appreciation of performance. As the number of credits to be earned increases, the student gets in addition a growing interest in meeting all the requirements of the assessment. For a good return, it is therefore important that the study programme consists of a limited number of large components (Cohen-Schotanus, 1994).
Professional task
Central in each semester is the professional task. This is an authentic assignment, who preferably comes straight from the professional practice and is attached to an external client. There will be teamwork in order to acquire the knowledge and skills that are central to that semester in an integrated way. In addition, the professional task is the ideal environment in which the student develops into an ICT professional (professional development). After all, students are confronted with the professional context, which calls for action as an ICT-professional and which presents them with challenges and problem-solving skills. Both the group and the individual in the group, receives feedback from the tutor, course lecturers and, where possible, from the external client. Education for Professionals (OvP) is working on a case study that students carry out in their own professional practice. This line will be continued in the Master's programme and supplemented with cases from research groups and Partners in Innovation.
In order to be able to carry out such a professional task in a professional manner, the student must obtain the necessary professional baggage (intelligence, skills and attitude) (Kloppenburg, 2011 in van Berkel, Bax and Joosten-ten Brinke, 2014). This is why the professional task surrounded by flanking education.
Flanking education
Flanking education at Fontys ICT includes workshops and courses from different fields of expertise, knowledge-sharing sessions, feedback moments, or other educational activities in which students actively acquire the necessary knowledge base and vocational skills and apply them in more complex professional duties such as the professional task. The flanking education supports the learning process of the student.
Meaningful context
The professional task and the flanking education always form a meaningful, coherent context in which students work towards the learning objectives belonging to a semester. Every semester at Fontys ICT is therefore practice-related. Train students to an ICT-professional by means of education in an authentic and meaningful context, asks that the testing and assessment are also geared to this. Therefore, the starting point is to make an integral assessment.
Integral assessment
By integral assessment we mean the assessment of craftsmanship by looking at a combination of the application of knowledge, skill and attitude and the underlying processes. According to Klarus (2004) there is an inseparable focus on cognitive activity (substantive) and acting (process and professional development). A complex question and learning from a social constructivist viewpoint is an integrative matter. Information from multiple formative measuring moments, based on various assessment methods, on various layers of the pyramid of Miller, by several assessors contribute to a more reliable and generalisable, summative opinion (from Berkel, Bax and Joosten-ten Brinke, 2014).
The summative assessment takes place at semester level. In order to be able to assess the complex set of knowledge, skills and professional development in an integrated manner, a portfolio is chosen as the form of assessment. The portfolio should reflect the efforts made, the development and results of the student of that semester (Arter & Spandel, in Dochy, Heylen & Van de Mosselaer, 2002). This concerns both the subject matter and the professional development. In addition, the feedback from lecturers and possibly from external involved persons or fellow students is also included in the portfolio. Every semester the student closes the portfolio and in the following semester he starts with a new portfolio.
The summative portfolio assessment takes place during the assessor meeting. Here, there is an assessment given in the form of U (unsatisfactory), S (satisfactory), G (good) or O (outstanding). In the case of an S, G or O the student has successfully completed the entire unit of study and in the case of a U, he has not. This means that the student has shown at least the expected level of the relevant semester. The appreciation of the quality of the performance the student has delivered, becomes visible in the feedback. During the assessor meeting a delegate of the examination chamber is present, who takes notes of the argumentation of the student's assessment. The summative result is passed on to the examination chamber, which can only see whether the judgement has been arrived at in the correct manner.
In order to be able to make a reliable integral judgement, the assessment programme - which is to be specified per semester in a concrete form in the accompanying semester guide - should provide a wealth of information through a combination of validated products of learning, such as professional products, reflections and feedback received from lecturers, students and external stakeholders. Information is triangulated across evaluators and test moments, as a result of which decisions are not based on the judgement of one assessor or on the judgment in one situation or on an occupational task (from Berkel, Bax & Joosten-Ten Brinke, 2014). Fontys ICT is for an integral assessment also inspired by the assessment system for complex performance of Daan Andriessen (2014), a system that strikes a balance between analytical and holistic assessment. In order to be able to assess at a higher level, the level of professional qualifications, it comes down to assessing on the basis of competence and knowledge of the facts; the assessors are therefore required to have a high level of professionalism (Bulterman-Bos & the Muynck, 2014).
The student builds during the semester his portfolio consisting of several validated products of learning, which, per student - depending on the selected teaching method - may differ (learning path-independent assessing). Examples of validated products of learning are: made-to-measure professional products, assignments, tests (presentations, reports, performance assessments, deliveries of professional task and criterion-oriented interview) including the formative feedback and feed forward that the student regularly receives. This feedback is related to criteria, designed in rubrics or assessment dimensions, and helps the student to demonstrate the learning outcomes belonging to the semester. Because criteria of the study programme include various aspects or elements (intelligence, competences, professional development), it is necessary that feedback also covers these different aspects. In addition, feedback should cover both products and processes in order to optimally promote learning. For this purpose, various formative feedback moments are required, on the basis of which the student has had several opportunities to improve his portfolio to be updated, prior to the integral portfolio review. Per semester is determined which mandatory elements (products of learning) should include students in their portfolio and how much free space there is.
In order to be able to determine the content of the portfolio properly, the function of the portfolio must first be determined. In this way, the type of portfolio can be determined, for example: assessment portfolio, development portfolio, presentation portfolio. At Fontys ICT, two functions are combined, so that each portfolio contains a development and assessment component. A development portfolio represents the efforts and development of the student. An assessment portfolio stands for the assessment and formal validation of the requested learning outcomes, showing that the student has achieved them. With a presentation portfolio the student shows who he is, the portfolio serves as a showcase in which competences are demonstrated by the student (SLO, z.j.; Ouwehand & van der Zanden, 2009).
However, the extent to which these various components are emphasized may vary from one semester to the next. In table 1 is shown how a component can be expressed in the various types of portfolios:
Table 1
Development and assessment components for various types portfolio
Component | Portfolio Fontys ICT (Development and Review) | Other, namely. Showcase portfolio (Presentation) |
---|---|---|
Target | Making the efforts visible, development and results of the student. | Making products visible where the student is proud of or shows what he's good at. |
Content | Structured and organised collection of validated products of learning like professional products, reflections and received feedback from lecturers, students and external stakeholders. | Products from the student of which he is proud. |
Influence of the portfolio on the learning process | Portfolio building is a dynamic process and forms input for guidance and assessment. | The portfolio shows final results. This can play a role in applications. |
The role of the portfolio upon reflection | The student becomes aware of the learning process. | The student becomes aware of where his qualities lie and what he is proud of. |
Role student | Degree of selection depends on teaching method. | Selection of the content of the portfolio. |
Role lecturer | Guidance and assessment as a continuous process. | Support in the selection of, and reflection on, the content of the portfolio. |
Review | Summative portfolio assessment by several lecturers. | Reviews can be included in the presentation portfolio if desired. |
Remark. Adapted and taken over from of Dochy and others. 2002, Heylen & van de Mosselaer, 2002; Ouwehand & van der Zanden, 2009; FHICT, 2017; M. van Diggelen, persoonlijke communicatie, 28 June 2018; SLO, (n.d.).
At Fontys ICT the portfolio contains a mix of validated products of learning, regardless of what function it has. The use of a mix of learning products leads to meaningful feedback and feed forward. This is aimed at promoting the development of (learning process) of the student.
Regardless of the chosen teaching method the student is supported by the lecturer in his portfolio-accrual. The degree of independence of the student and the complexity of the learning outcomes which, by means of the portfolio must be demonstrated, increase during the study programme (from propaedeutic phase as far as graduation). Table 2 provides guidelines on the way in which each of the following lecturer can offer guidance to a student at the portfolio-development, which, at every teaching method can be applied.
Table 2
Guidelines for lecturers to assist students with portfolio development
With the choices in this assessment policy the aim is to develop assessment assessment programme tests of sufficient quality. A number of quality aspects set out below comments the way in which they are secured. This is followed by a short comment on the way the assessment quality is guarded.
The Dublin Descriptors are the starting point for the design of our education. Substantive objectives relate to the Dublin descriptors 'Knowledge and insight' and 'application of knowledge and insight' and are related to the exit qualifications from the Domeinbeschrijving Bachelor of ICT, an edition of the HBO-I, the umbrella organisation of the universities of applied sciences ict-study programmes in the Netherlands. The process and attitude objectives relate to 'application of knowledge and insight', 'judgement', 'communication' and 'learning skills'. Learning objectives have been derived from this. The learning objectives form the basis for rubrics or assessment dimensions and provide guidance for formative feedback. Education has been translated from the learning objectives into educational activities. This is specified in the semester guide. The PLOU oversees the coherence between and level of professional tasks, learning objectives, testing methods and evaluation criteria. By choosing authentic assignments and problems, which are closely related to tasks from professional practice, Fontys ICT increases the validity even further. This is secured by involving the work field in the education. This approach stimulates the development of thought processes that are used in professional practice for carrying out assignments or solving problems.
To increase reliability, the evaluation criteria are explicitly laid down in the semester guide belonging to a unit of study. Student development is central. From the point of view of reliability, this means that claims from a student are repeatedly applied against the assessment criteria. Partly for this reason, formative assessment and feedback is an important part of the process and precedes the summative portfolio assessment. In order to increase reliability, several examiners are involved. The dialogue between different examiners, who each bring in their professional judgment, plays a crucial role. By agreeing, through dialogue, on the competence of a student, the reliability is increased. The emphasis will be on intersubjectivity instead of objectivity. Intervision and calibration sessions, where the focus is on the interpretation of assessment criteria, further contribute to increased reliability.
By explicitly describing and publishing learning objectives and assessment criteria - embedded in an assessment model - and bringing them to the attention of students, the aim is to achieve transparency. Specifically to each student this transparency will be further pursued through the use of development-oriented feedback referring to the learning objectives and/or evaluation criteria. By including meaningful feedback in Canvas, the LMS of Fontys ICT, in conjunction with the assignments and the work submitted, the Canvas courses provide a good representation of the student's learning process, development and performance. Canvas courses therefore also function as an assessment file at the same time (as described by Jaspers, M. & From Zijl, E., 2014), that both for student and lecturer provides insight into the extent to which students are able to control learning objectives.
The assessment quality is guarded, on the one hand, because assessment is also included in the educational evaluations, as described in the quality cycle from Fontys ICT. The evaluation shall also cover the manner in which the testing and assessment is to be carried out. In addition to evaluations by students about the educational period, lecturers also carry out a assessment analysis. This assessment analysis is a procedure appropriate to a portfolio founded assessment and will be included in the assessment file together with any improvement actions, so that they can be included in the next performance. By linking the performance and control of tasks to different roles as much as possible, a self-reinforcing process is sought. The assessment file contains the following parts: description of the assessment method, the assessment model, a representative set of products and assessments, an overview of all assessments and an assessment analysis.
The lecturer and in particular the lecturers' team play an important role in the assessment process of formative feedback on the way to a summative assessment. Consultation and calibration sessions between lecturers are important for the quality of assessment (Andriessen, 2014) and thus also of education. Knowledge sharing is an important component of professionalism and therefore deserves constant attention (Dochy et. al., 2014, p. 67). Fontys ICT asks that every lecturer within three years after the permanent appointment further professionalises, including in the field of didactics and assessment. Where the consultation between lecturers is rather an informal form of professionalisation, Fontys ICT offers more formal forms of lecturer professionalisation. The vision of student centred education is extended to personnel policy (HR), and professionalisation; student centred education is implemented in a lecturer centred manner. Lecturers can build a portfolio based on their own development question or area of interest. The emphasis is on development and reflection in daily work. In addition, lecturers have the possibility of training or guidance in the field of didactics and assessment. Lecturer teams and individual lecturers are guided by a team of educational specialists.
Within the assessment policy of Fontys ICT different roles are distinguished on behalf of quality assurance of assessment. The roles are filled in by lecturers, students and/or participants from the work field. In the following paragraphs, the different roles are explained.
Across Fontys ICT there is one examination board for all study programme variants (full-time Dutch, full-time English, part-time) and main subjects within, from which rooms are controlled, each responsible for the implementation of the assessment policy within a variant, main subject or study phase. The examination board has final responsibility for formulation, implementation and adjusting of the assessment policy. In a list of sub-tasks, the following are involved:
In the process of summative assessment the examination board secures that the procedures for determining the final result on the basis of the portfolio assessment were carried out correctly.
Within Fontys ICT has been chosen for an active acting assessment committee, consisting of members of all Fontys ICT fields of study, who not only oversees the assessment quality, but also as an active discussion partner cooperates in improving and securing the assessment quality. The assessment committee shall report to the examination board and conducts regular audits to verify the implementation of the assessment policy in Fontys ICT and the assessment quality is monitored. In addition, the assessment committee gives advice, coaching and training where necessary, for the purpose of assessment policy to be perpetuated within the organisation.
The Project Leader Programme Implementation (PLOU) coordinates in addition to the educational implementation also the formative assessment for an education semester. From the point of view of educational development, for each semester an assessment programme is established with laid down in it the form, content and planning of formative assessment in accordance with the assessment policy and frameworks from the TER. The PLOU submits this assessment programme to the examination board for determination. Then the PLOU takes care of the implementation of the assessment programme. To this end, the PLOU designates test developers and examiners from the list of examiners appointed by the examination board. After execution of the unit of study, the PLOU evaluates the formative assessment, processes the results and, on that basis, proposes any improvement actions to be taken. These improvement actions, together with the evaluation results, are submitted for approval to the examination board. After approval, the improvement proposals are executed for the next implementation of the unit of study. The PLOU is responsible for applying for the test files in Canvas and securing them in practice.
If within a field of expertise a formative assessment is planned, a test developer will draw up a concrete test/assignment and an assessment model. The assessment programme of an educational activity provides the framework for each test to be drawn up. Where, for each performance, the same test can be taken, as in the case of product reviews, for example, then the test development in the development of an educational activity has been carried out by the 'vakeigenaar'. A test developer is always an examiner. In addition, various people can provide input to the test developer during test development. Initially these are lecturers, but especially later in the study programme students and experts from the work field weigh in. After approval from the test reviewer, the test developer submits a test, including the corresponding assessment model, to the project leader programme implementation for the purpose of determination.
The test reviewer provides a test developer with feedback on a test/assignment drawn up by the last one, including the assessment model. Only after approval of the test reviewer the project leader programme implementation can determine a test. A test reviewer is in all cases a appointed examiner by the examination board.
The assessment is always carried out by one or more examiner(s) appointed by the examination board. However, they can also base their judgement on the opinions of others, such as experts from the work field and students. Determining (formative) assessment methods is always done by examiners, appointed by the examination board. The evaluation of (formative) assessment is based on information from overall quality assurance. This information shall be sent to the project leaders educational implementation and the examination committees provided by the quality coordinator.