Skip to main content
Search by keyword

National Digital Skills Strategies & Action Plans - How can we monitor their effectiveness?

Community discussion - Monitoring the effectiveness of national digital skills strategies and action plans

The Issue

Many countries, in different ways, are currently working on the definition of digital skills strategies. One of the main challenges is the need to identify reliable ways to verify the effective implementation of these strategies, by researching tools able to monitor and measure their progress.

Many countries have the issue of how to better measure the impact of the National Digital Skills Strategies and Action Plans, for effectively verifying the impact that the planned actions may have on the target population, together with the opportunity of recalibrating them in order to achieve the defined goals.

Thus, it would be very useful to compare countries' practices in monitoring the actions results and the effective implementation of the strategy.

The Italian Experience

In April 2020, the Ministry for Technological Innovation and Digital Transition launched the Italian Coalition for Digital Skills and Jobs, which is one of the 23 European coalitions. The Coalition builds on “Repubblica Digitale”, a multi-stakeholder initiative that promotes digital skills at all levels of the Italian economy and society. The initiative aims to identify and engage as many stakeholders as possible (e.g., businesses, public entities, NGOs etc.), by creating a cultural change based on those competences improvement necessary to fully realise the benefits of digital transformation. Up to now, more than 200 organizations have joined the Italian Coalition and they have launched more than 250 projects.

The Italian Coalition is led by the Technical Steering Committee of Repubblica Digitale, coordinated by the Department for Digital Transformation - Presidency of the Council of Ministers and formed by representatives of Ministries, public entities, representatives from universities, research institutes and citizens' association of the National Coalition for Digital Skills.

During the last year the “National Strategy for Digital Skills” was approved. In the document, 4 lines of intervention have been identified coherently with the four pillars of the European Coalition for Digital Skills and Jobs:

  • Higher Education and Training
  • Active workforce
  • ICT specialist skills
  • Citizens
  • Priorities and lines of action have been associated with each line of intervention. The Strategy has then been complemented by an Operational Plan with a roadmap and specific actions.

Dashboard definition

The current Plan addresses the 41 lines of action identified in the Strategy through 110 actions and provides a dashboard of over 60 indicators to monitor the impact on the 4 lines of intervention. Each action also includes appropriate milestones, result and outcome indicators and target values.

The dashboard is based on the indexes included in the Digital Economy and Society Index (DESI) of the European Commission and the Digital Maturity Indexes (DMI) elaborated by the Digital Agenda Observatory. The impact indicators selected for the dashboard allow a benchmarking activity with European countries that have the socio-demographic characteristics most similar to Italy, such as France, Spain, Germany, Poland.

The Plan includes targets for 2025, based mainly on both DESI and Eurostat indicators, such as the following:

  • 70% of the population with at least basic digital skills and elimination of the gender gap in digital skills;
  • duplication of the population with advanced digital skills (78% of young people with higher education, 40% of workers in the private sector and 50% of civil servants);
  • increase by 50% in the share of SMEs using ICT specialists;
  • increase by five times in the share of the population using public digital services (64%).

Each target value has been identified upon the considerations of some criteria:

  • reducing the gap with other European countries and reaching one of the first three positions compared to the countries most similar to Italy in socio-economic and demographic characteristics;
  • the objectives defined by the European Commission (e.g., European Agenda for Digital Skills and the Action Plan for European Digital Education 2021-2027);
  • the need to identify challenging target values that can be concretely reached with the resources available (with a “validation analysis”).
  • Monitoring process
  • The monitoring model will allow to evaluate the progress of the operational plan on a six-monthly basis, evaluating the effectiveness of both the conducted actions and the overall plan while recalibrating them where necessary, and considering the ever-changing context according to the Plan-Do-Check-Act (PDCA) continuous improvement model.

In particular the monitoring model of the Plan is two-levels:

  • on the specific action, in terms of progress and outcomes achieved, effectiveness assessment (currently via a pilot project)
  • on the overall plan, in terms of coverage level of the actions on the expected impacts (outcomes indicators vs impact indicators) and adequacy of the improvements (based on the data annually provided by ISTAT and EUROSTAT)  compared to target values values

The first monitoring report of the strategic plan will be released in November 2021.

Community Discussion

In this context, and following on from our Community Workshop on 18th October, we would like to start a discussion with the wider Digital Skills & Jobs Community in order to compare models and methods, and to effectively monitor the implementation of a digital skills strategy and, in particular:

  • digital strategy skills and action plan design
  • setting of indicators and identification of target value
  • data source and collection
  • monitoring methodologies and tools
  • improvement cycle

In order to better understand which are the most successful measures each country implemented in monitoring the metrics of their National Strategies and Action Plans, we propose to start the discussion by sharing these elements:

  • Explanation of the monitoring strategy used
  • Results achieved
  • How it is designed to be effective
  • How it can be improved
  • Top 5 key points for a successful digital skills strategy monitoring and improvement cycle

Please share your own experiences and thoughts in the comments below!