For the quality of applied research within Fontys ICT

Summary
In the context of the second Fontys ICT Ambition plan 2018-2023 OPEN UP, the R&D theme aimed to link education and applied research. This involved first identifying what characterises applied research and how the quality of those characteristics can be determined or described. This led to the creation and elaboration of a model with underlying tools. The model has four dimensions (relevance, soundness, transferability and valorisation) that characterise applied research and with which the quality of applied research can be expressed as the span between the four dimensions. The underlying set of instruments provides tools to put the dimensions into practice and to assess, maintain and develop that quality through measurement, organisation and products/outcomes. The set of tools is part of the learning line investigative problem solving which bridges research and education.

Fontys ICT is transforming into a knowledge institute that integrates education, research and co-creation with external partners. With Research Based Learning (RBL), students acquire the necessary (research) skills to become better ICT professionals.

The quality of research activities is assessed according to the standards of the Branch Protocol for Quality Assurance in Research (University of Applied Sciences, 2022) [1]. These standards focus on the organisation and relevance of research groups and their research.

Combining research and education brings additional challenges that need to be addressed:

• Semester-based education is not in sync with long-term research projects;
• Lack of proper documentation and transfers jeopardise continuity;
• Learning outcomes and learning pathways compete with research goals.

The model, as shown in Figure 1, distinguishes between four dimensions that together characterise applied research. Two dimensions focus on external and two on internal knowledge transfer.

Figure 1
The model showing the four dimensions characterising applied research and their tools and motto.

External knowledge transfer is about value and interaction with partners, networks, agenda and society. We distinguish two outward-facing dimensions:

• Valorisation: the added value our ICT graduates (to-be) contribute to the work field;
• Relevance: the positioning of our research themes in relation to national research themes and relevant issues with partners.

Internal knowledge transfer involves research you can rely on by providing an educational specialist framework, working and study environment for students, lecturers, researchers and project leaders or organisation. We distinguish two internally focused dimensions:

• Soundness: ensure reliable and valid research by embedding research methodologies in practice and education;
• Transferability: ensure project continuity and extension through good handovers and life-cycle management.

Although the dimensions can be treated independently, when placed in order they correspond to the natural process of research: starting with a relevant research question, applying sound methodologies leading to relevant (partial) professional products and ensuring continuity through knowledge transfer. Finally, products and knowledge show their added value for the work field.â

To assess, maintain and improve the quality of applied research, we define instruments that work on the dimension to:

• [a] measure quality and;
• [b] define the organisational structures through which quality is maintained and controlled and;
• [c] as a product/outcome that makes research quality visible or tangible.

The dimensions in the form of arrows can be used to create an overall picture, graphically depicted in Figure 2, that represents the quality of applied research.

Figure 2
Independent measurability dimensions and radar plot of quality applied research .
Note: the arrows indicate that each dimension is independently measurable. A radar plot (span of dimensions) shows the quality of applied research and which dimension(s) need additional attention.

Some notable tools include:

• Relevant positioning using the Enable-Create-Position (ECP) model;
• Robustness through DOT framework and design research paradigm;
• Transferability through life cycle management;
• Valorisation in Technology Readiness Levels (TRL) transitions and in Communities of Practice (CoP).

A design research process can be mapped (Figure 3) as a regulatory cycle akin to the Plan-Do-Check-Act (PDCA) quality cycle.

Figure 3
Sequence of the four characteristic dimensions applied research as corresponding to the PDCA quality cycle and the regulatory cycle of design-oriented research.

Relevance
A relevant project aligns with research positioning, which in turn is socially embedded. A relevant practice-oriented research project therefore starts from an appropriate research challenge and solid orientation (problem analysis).

Dignity
At the heart of any research project are the activities that address the challenge. The validity and reliability of these activities can be externally assessed through the use of well-defined and commonly understood design research patterns, research strategies and research methods.

Transferability
Following sound research methods leads to clear handovers, i.e. products that transfer knowledge to a follow-up or adjacent research project.

Valorability When relevant, robustly conducted and transferable research is opened to the outside world, it shows its added value. Interaction in communities with an active research portfolio gives further direction to more relevant research.

Table 1 provides an overview of tools by dimension characteristic of applied research.

Table 1
Overview of instruments (product/outcomes, organising and measuring) per dimension.

A gap analysis was performed on the four-dimensional model using the standards Branch Protocol for Quality Research (Basic-level Teaching Qualification (BKO) [1] and the Fontys SCOD model indicators [2]. Figure 4 shows this analysis schematically.

Figure 4
Mapping SCOD indicators and Basic-level Teaching Qualification (BKO) standards to the four dimensions model.

Although the Basic-level Teaching Qualification (BKO) 2023-2028 revised the standards from five to four, the analysis shows that the standards are largely covered. However, the coverage is not complete when it comes to [a] steering, [b] ethics governance and [c] self-evaluation.
Although the model does include tools that can be used for organising knowledge, it does not focus on steering. This makes the model particularly suitable for use at the operational level.

Ethical governance falls under steering. In addition, ethics is part of sound methodology. That aspect was elaborated within the OPEN UP ambition plan by the Impact on Society theme group. In cooperation, the two ambition themes have developed new DOT framework method maps in the field of ethics. Peer reviews and assessments are measuring instruments within the soundness dimension and are now applied in the mutual evaluation of (Fontys ICT) research groups.

1. Vereniging Hogescholen (2022). Branch protocol Quality Assurance Research (BKO): quality assurance system for applied research universities of applied sciences 2023-2028. Accessed from https://www.vereniginghogescholen.nl/kennisbank/praktijk-en-onderzoek/artikelen/brancheprotocol-onderzoek-2023-2028.
2. Crasborn, Koster, Van Duijl et al (2020). SCOD model: building blocks for quality indicators [Internal report].