REPORT ON THE EVALUATION ACTIVITY
The assessment of the pilot training course on Quality Assurance, AQUA.TS included the following tasks.
Task 1
Elaborate a SWOT analysis, mainly regarding the quality assurance activity in your organisation
Goal: to evaluate the following participants’ knowledge and competences
- elaboration and use of SWOT analysis, as an evaluation/self-evaluation instrument of quality management
- the content and the meaning of the four elements of the SWOT analysis (Strong points, Weak points, Opportunities, Threats).
Task 2
Elaborate a strategic plan for the implementation of one of the ten EQAF indicators (at your own choice) in your work institution/organisation
Goal: to evaluate the following participants’ knowledge and competences
- the needed steps for implementing an EQAF indicator
- the elaboration and use of a strategic plan, as an evaluation/self-evaluation instrument of quality strategy
- the elaboration of SMART objectives
- correct objectives’ quantification and its importance for the evaluation/self-evaluation process.
Task 3
Fill in the EXCEL model of AQUATS TOOLKIT for your work institution/organization. Please indicate your comments and/or concerns in the end of row cells.
Goal: to evaluate the following participants’ knowledge, competences and attitudes
- the critical and comprehensive thinking regarding the content and the requirements of the toolkit, as an evaluation/self-evaluation instrument of the quality assurance system
- the responsible and proactive attitude regarding the implementation of the EQAF indicators in the participants’ institutions/organisations.
Task 4
Please indicate 3 – 5 strong points and 3 – 5 weak points of the AQUATS TOOLKIT, in order to use it for evaluation and/or self-assessment of the quality assurance system in your work organisation
Goal: to evaluate the following participants’ knowledge, competences and attitudes
- the critical and comprehensive thinking regarding the content and the requirements of the toolkit, as an evaluation/self-evaluation instrument of the quality assurance system
- the responsible and proactive attitude regarding the implementation of the EQAF indicators in the participants’ institutions/organisations.
TOOLKIT. The most frequent strong points and weak points that the training participants have mentioned.
| Strong points | Weak points |
| -Enables the European principles integration and recommendations, and it is appropriate to the European and national objectives.
-Creates a structured framework for quality assurance in the VET. -Encourages the development of a result-based research and analysis at the system level. -Enables an in-depth monitoring and analysis of the process of quality assurance in VET organizations. -Tackles, in a detailed manner, multiple issues and situations of the quality assurance process. -Enhances a clear and specific definition of the concepts and categories of quality assurance in VET. -Highlights the obligations of every VET-involved person, especially the socio-economic responsibilities of the management and authority staff. -Provides support for the identification and analysis of the trainers’/ students’/employers’ expectations. -Contributes to the identification of possible partnerships in VET. -Provides support and information for further development of training programs. |
-Some items are not applicable to any type of VET organization; especially for smaller VET organisations, newly established ones or newly initiated/accredited programs this approach is not totally appropriate.
-The learners’ employability indicator is a little too high at the moment because this is a delicate situation on the present unstable labour market. Therefore, at this time, more emphasis should be put on the quality of teaching and on evaluating indicators. -Some items are difficult to be implemented in some countries because of the gaps inherent in the systems, and not due to the organizations’ shortcomings. -Some questions are unclear, confusing, or do not specify the requirements. -Some items have not enough answering options to cover all possibilities. -Some answering options are unclear or do not specify the option. -The answering options are irrelevant for some items. -It is a thick, branching out and difficult to use assessment tool.
|
Examples (no. of indicator – no. of the row in the EXCEL file)
| 1. | Some items are difficult to be implemented in some countries because of the gaps of the system and not due to the organizational shortcomings. | |
| 5 – 19 | If you use tools to check the causes for which your learners do not complete the VET courses, do you take corrective actions to reduce the dropout rate? | |
| 5 – 24 | If you assess the competences needs of the labour market in the design of VET programs, do you make such an assessment in a systematic way? | |
| 2. | Some questions are unclear, confusing or not specific enough. | |
| 4 – 17 | Do you use a tool to assess training needs in the labour market that is been used /evaluated at least twice a year? | Unclear. |
| 8 – 11 | Have all the measures been taken in your organisation to overcome the barriers? | All measures?
What barriers? |
| 3. | Some items have not enough answering options to cover all possibilities. | |
| 8 – 17 | Which is the completion rate of training programs realized by disadvantaged groups (vis-à-vis the total number of disadvantaged people in training)? | The reasonable rate is about 50% in many countries. |
| 2 – 18 | Do you plan your training courses on an annual basis to:
1 – Improve key competencies of trainers/teachers |
There are situations when planning is done every 2 years or randomly, when needed, for example (and not annually) – for which there are no answering options. |
| 3 – 10 | How often do you align the planning of your programmes with national, local and/or European VET policy? | Annual adjustments can be done to local and national policy and every two years, for example, to the European policy – there are no answering options for this situations (it is recommended that formulations containing “AND” to be avoided because of possible difficulties in answering). |
| 3 – 11 | How many ways your organisation has organised to respond to the need for adapted training provision?
1 – 1 tool/procedure |
The question is not clear enough and there is no answering option for more than one method/procedure. |
| 5 – 24 | If you assess the competences needs of the labour market in the preparation of VET programs, do you make such an assessment in a systematic way?
Yes, No, No, I don’t make this action/measures |
There is no answering option if this action is taken randomly or when needed. |
| 4. | Some answering options are unclear or not specific. | |
| 2 – 14 | If a trainer updating training and specialisation plan exists, what percentage of the total number of the trainers/teachers participates yearly in training activities?
1 – Less than 70% the software will consider good: answer n. 3 |
There is no option for 3. |
| 2 – 18 | Every year, do you plan your training courses in order to:
1 – Improve key competencies of trainers/teachers
|
To this sort of questions, ticking YES or NO is needed or the answering options only?
If a course is organised every two year or when needed (is requested from people, for example), there is no answer option for this situation.
|
| 5. | The answering options are irrelevant for some items. | |
| 7 – 14 | Have the target groups found a coherent job with the carried out path, where they can put into practice the skills acquired in training?
YES / NO |
This question with the YES/NO answering options is irrelevant (if just 1% succeeded for example, will it be considered YES?). A percentage answering scale with five steps – including 0% maybe – should be more appropriate. |
| 8 – 19 | If yes, have they found a consistent job with the skills acquired during the training?
YES / NO |
This question with the YES/NO answering options is irrelevant (if just 1% succeeded for example, it will be considered YES?). A percentage answering scale with five steps – including 0% maybe – should be more appropriate. |
Dr. Serban Adriana