Automating recoding under classification updates using textual data

This template is a fast starting pack for AIML4OS projects that use Python. For R projects, see this Github repository.

Introduction

National statistics’ offices (NSOs) around the world, have to align their statistics to international standards, such as the industry classification NACE. Over time, international standards need to be updated to reflect changes in society. In 2025, NSOs in Europe started transitioning from NACE revision 2.0 to 2.1. This update creates a number of challenges including recoding large registers and holding two standards in production over an extended period of time.

This tutorial provides examples from France and Norway to illustrate examples of automating some aspects of these process.

Motivation

The recoding process can be highly resource demanding. If only manual methods are used, many NSOs would not be able to complete the task in a timely manner. Automating through rule-based systems is also time consuming to set up and rigid in a way that can be innefficient and inaccurate. Therefore, machine learning and deep-learning approaches are becoming more commonly used.

…