Gender Equality in Academia and Research

Monitoring and evaluation as part of the process of change

A Gender Equality Plan is meant to address several issues at once, and to rely upon a complex set of measures. Hence, from its earliest stage, monitoring and evaluation instruments are to be foreseen. Such instruments allow among others to assess the progress that is made towards targets, based on indicators.

Monitoring and evaluation instruments are however often lacking, which undermines the transformative potential of planned actions. If objectives are not indexed on relevant progress, success or outreach indicators, it is difficult to assess whether the organisation is being actually transformed. This might also reduce the commitment of stakeholders towards those objectives.

For these reasons, monitoring and evaluation instruments are firstly to be seen as tools supporting effective actions and creating accountability. Secondly, by providing indicators against which actions can be assessed and resources allocated, they also enhance the knowledge about on-going changes.

Baseline assessment (or ‘ex-ante evaluation’)

Evaluation is often conceived only as ex-post. Yet, Gender Equality Plans are better designed if relying upon a comprehensive assessment of the status of gender equality.

This assessment can take different forms. An audit can be carried out with the support of external and impartial expertise. Internal knowledge about gender and the institution itself can also be mobilised. Different tools can be used for investigating gender inequalities, bias and imbalances at all levels, including pilot studies, surveys, focus groups, interviews or ethnographic observation.

Have a look at the section on ‘analysing and assessing the state-of-play in the institution’. Gender Equality Plans carried out with appropriate resources and expertise, such as those supported by the European Commission, often draw on comprehensive diagnoses.

As a result, actions are planned starting from the knowledge of actual situations and processes. They are also more tailored to the needs of the institution.

Additionally, ex-ante evaluation, by drawing attention to gender-related issues, helps securing support, raising awareness, and mobilising stakeholders. It has the potential to give voice to stakeholders who are not in a position of advantage, and to create support for changes. Open, collaborative processes, involving different categories of stakeholders, can help building consent around the diagnosis of the situation, and foreseen solutions.

Legal requirements often exist to gather basic data on gender equality in research and higher education institutions (e.g. on the number of women and men in professorship positions). These efforts are usually formal. However, they may already provide a baseline for the analysis. Therefore, ex-ante evaluation should integrate existing instruments so as to increase its legitimacy, and make the most of available data.

Monitoring implementation

Monitoring actions once they are being implemented is key to inform about how they address actual needs, and effectively support change. Gender Equality Plans should mobilise a large number of different actors within the organisation. It is deemed important for those who pilot actions to have a good command of what is going on.

Monitoring does not only enable to see where and how actions are being implemented. It can also indicate whether or not a transformative dynamic is going on. Well-thought monitoring mechanisms can help identifying and addressing potential sources of resistance to change. Last but not least, a virtuous cycle can make monitoring tools part of a continuous enhancement process.

As such, these instruments can be both part of the piloting of the actions, and external to it, in order to grant implementation both with a sight on overarching goals, and a more distant stance on the actions. In all cases, gender expertise will be required, potentially along with other expertise on change dynamics or other specific issues tackled by the Gender Equality Plan.

Actions can thus be regularly monitored by setting up appropriate indicators and follow-up instruments. Indicators should be implementation-oriented, and adapted to the purposes of the action. Actions aimed at increasing the participation of women in senior positions should not only be monitored by looking at figures.

Assessing the enrolment in supporting schemes (such as training, mentoring or reserved fellowships) and their impact on the actual gender balance is relevant. But indicators on the transparency of recruitment, promotion and evaluation procedures are also needed.

Similarly, when tackling gender-based violence and sexual harassment, assessing reported cases and the functioning of alert and resolution mechanisms should not be the only concern. Checking how actual cases are qualitatively dealt with can effectively support action, and measuring how a gender sensitive culture is developing in the organisation, for instance by means of a periodic survey, can also effectively support action.

Monitoring sessions with core and/or extended teams, in our experience, are crucial moments of self-reflexivity about the processes which have been set in motion, and how to strategically steer the project to achieve more.(Marina Cacace, L'Assemblea delle Donne per lo Sviluppo e la Lotta all'Esclusione Sociale (ASDO), evaluator of the EU-funded STAGES project)

Evaluation to be context-sensitive

Evaluation is taking an ever greater place in research organisations and higher education institutions. These institutions’ overall performance with regard to management, financial health, innovation and outputs are increasingly evaluated. Evaluation has often become key to access public funding, and international rankings play a major role in the strategies of research organisations.

Researchers themselves are not immune to evaluation and beyond peer-reviewing, their achievements are being scrutinised along different scales. But evaluation is not only meant to support scientific excellence. Actions aimed at transforming research institutions to prevent gender bias, including in the way researchers are being evaluated, also need to be assessed.

Such assessment should take into account context-specific features, such as, but not exclusively:

• The type of organisation
• The research areas covered by the planned actions
• Who is involved and targeted by the process of change
• The existence of prior gender or other equality policies
• The current status of the organisation: is it going through a broader process of change?

Quantitative indicators

Quantitative indicators are relevant whenever they are adapted to the objectives of the planned actions. It is widely acknowledged in research on evaluation that there is no evaluation, and hence, indicators, that can fit for all. Quantitative indicators most often include:

• the number of female candidates for positions in which they are under-represented
• the number of women and men in selection panels (for recruitment and promotion)
• horizontal sex segregation in respective categories of occupation
• the number of women and men targeted and reached by gender awareness-raising or training actions planned
• gender ratios in accessing research grants (and other resources, e.g. laboratories or personnel)
• the gender pay gap among different categories of staff, including researchers

This list is by no means exhaustive, and indicators can also be set to measure respective positions of men and women with in relation to work-life balance, leaves, evaluation scores (…). Such indicators help building accountability for the successes or failures of implemented measures. They also create perspectives.

Yet, alone, they are either little predictive for long-term transformations or changes that can be steady and collateral to the main objectives.

Transforming complex organisations, challenging processes, routines and power relations that contribute to shape the distribution of positions among researchers, takes time. This scale of time is not necessarily the one ascribed by the evaluation. Hence, attention must also be paid to short-term and mid-term milestones and potential achievements.

Changes in prospect for greater equality are only possible with the support and engagement of key stakeholders and of a large part of the community. Evaluation should thus support existing dynamics and help measuring and addressing successes and challenges that are likely to emerge along the way.

Qualitative indicators

Quantitative indicators are not enough; qualitative indicators are also needed. Those may look at dimensions such as:

• Mainstreaming of gender knowledge and awareness among the different categories of staff, including researchers. This can be measured for instance by the relevance given to knowledge creation on gender equality within the institution, its institutionalisation (in the form of dedicated programmes or departments), the dissemination of such knowledge across disciplines and research areas, to be evidenced by seminars, research projects with a gender component, etc. As regards gender awareness, the attention given to gender by different categories of stakeholders, through communication initiatives, codes of conduct, activities centred on gender-related aspects can also help evidencing changes.
• The uptake of the gender equality objectives set by the Gender Equality Plan by different categories of stakeholders, to be for instance reflected in the different framings of gender inequalities within the organisation and their evolution towards a greater gender awareness.
• The actual transformation towards greater gender-sensitivity of both formal and informal practices as the effect of implemented actions, notably in the areas of human resource management, decision-making, evaluation and governance.
• The diffusion of a gender equality culture in terms of work conditions, verbal and non-verbal interactions, so as to reflect changes regarding the management of work-life balance, awareness on sexual harassment and other aspects of gender-based violence, non-sexist communication.

Qualitative indicators can contribute to a better knowledge of the process of change itself. They may bring evidence that change happens and that gender equality and awareness are not out of sight.

Qualitative indicators have also a stronger learning potential. They support self-reflexivity and may provide indications for a continuous enhancement of the implemented measures and actions.

Resources for sound monitoring & evaluation

Have a look at the section on ‘analytical measures, monitoring and evaluation’, where resources for sound monitoring and evaluation are indicated. These resources draw upon the experience of EU-funded projects and complex gender mainstreaming strategies. They have also the potential to be replicated in a variety of contexts.

Yet, each organisation operates in a different institutional and disciplinary context, and is confronted with different challenges as concerns gender equality. Hence, it is wise to reflect upon the actual conditions for available indicators, to be used meaningfully in your own institution.

Similarly, in those contexts where expertise is available either from public or private structures, it is useful to mobilise the expertise of external evaluators for gender audits and/or ex-post evaluation of implemented measures.

When drawing upon external expertise on evaluation, it is yet recommended to bring together external evaluators with people in charge of implementing change within the institution, so as to co-design monitoring and evaluation instruments adapted to your goals and constraints.

Need inspiration?

Evaluation as key to sustainability and further enhancement

Ex-post evaluation of gender equality initiatives is often lacking. This considerably undermines the potential of gender mainstreaming. Evaluation does not only provide evidences of actual changes or failing attempts. It also enlightens the positive dynamics brought by gender mainstreaming strategies, and the opportunities they bring. Beyond the objectives ascribed, implemented measures are likely to produce positive side-effects:

• Strengthened sense of community, more transparent recruitment, appraisal and evaluation procedures;
• Stronger pluri-disciplinarity in research;
• Improved working conditions.

Assessing the impact of these short-, mid- and long-term transformations is an opportunity to enhance the support to gender equality policies. It also paves the way for future, even more resolute actions, and offers a valuable knowledge for their design.

A thorough, context-sensitive and mixed evaluation approach should not be an overburden, but helps your strategy to make a difference.

Click below to continue to the next step and read how to ensure the sustainability of your actions. You can also go back to the previous step.