The Quality of French census

July 2010

The Quality of French census

Page 2

The quality of the French population census July 2010

An « old » but good paper…

› Written en 2008 by Olivier Lefebvre and Michel Cezard› Three parts :

–A complete presentation of the new french population census method

–Compliance on • the quality criteria of the UN (individual enumeration, universality,….)

• the criteria shortlisted by the EuroStat code of best practices (relevance, accuracy,…)

–The management of the census quality

Page 3


II - In line with quality criteria

› The essential characteristics of the census (defined by the UN) :

‐ Individual listing

‐Universality

‐Simultaneity

‐Capacity to supply data on small fields

‐Regularity

Page 4


Individual listing

›Data for each individual and each household interviewed in the census

›Organised in a « detail » file allowing for all kinds of comparisons

› Sampling rate sufficient for all kinds of cross-tabulations

Page 7


Capacity to supply data on small fields

› Stems from both universality and individual listing›Garanteed by the sampling rate (100% in the small

communes and 40% in the large ones)› The census will supply data on all the communes and

on the « IRIS », districts of about 2000 inhabitants

Page 8


Regularity

›Criteria met by construction because one supplies annual data

Page 9


In line with quality criteria (2)

›Criteria adopted by Eurostat– Relevance– Accuracy and reliability– Timeliness and punctuality– Accessibility and clarity– Comparability– Coherence

Page 11


Accuracy and reliability

› An imprecision linked to the sample survey, but of low amplitude and lower than the one resulting from the «ageing » of data

›Data collection, control and computation methods which considerably reduce the omission bias

Page 12


Timeliness and punctuality

›Recent data –permanently 2 to 3 years for detailed data against 18

months to 10 years in the former system

–results of annual surveys available at the beginning of the following year and integrated to the provisional estimates (at the regional and national level)

› Punctuality : –Legal requirement (legal populations of the communes)

–It guarantees the permanence of the system

Page 13


Accessibility and clarity

› Accessibility guaranteed by resorting to the internet as medium for dissemination

› Systematic feed-back to the communes, with the possibility to re-disseminate this data

›Clarity : important pedagogical and documentation work in the website.

Page 16


III - The management of the census quality

›General principles– control quality step by step–A continuous evaluation approach

› Exemples :–Sampling frames–Data collection–Data processing–Dissemination

Page 17


CENSUS STEPS

Located

buildings register

(R.I.L)

Data Collection Sampling

Data Collection preparation

Households Communities

Communities register Receipt, Checking

Restitution to the mayors Dissemination

Preparation of data entry

Validation

of the data

Estimation of datas :

population and detailed datas

Imputation of missing datas

Coding

Automatic then manual

recovery

Data entry

(external service provider)

Addition of the weights and updating of geography

Page 18


General principe 2

› A continuous evaluation approach› Enabled through the annual feature of the operation› After each campaign, decisions of evolutions› Implemented in N+1, but mostly N+2 or N+3

Campaign

Evaluation

Improvements

Page 19


Sep.Juin Jui. Août Oct. Nov. Déc. Mar JuinAvril MaiJan Fév. Sep.Jui. Août Nov. Déc. Jan

Year N-1 Year N

The schedule of a campaign

Sampling

RIL Expertise

Setting-up of organisation

Preparation

Training

Communication

Collection

Reception

Controls

Data processing

Oct.

Dissemination

Page 20


› Any improvement as effects at several points of the process : formulars, formations, protocols, softwares, traitements,…

› Sometimes an improvement has his first impact within 2 or 3 years.

Page 21


The national commission for the evaluation of the census

›Under the rule of the CNIS (National Council for Statistical Information)

› Presided over by a senator› It associates communes, the data users (researchers,

administrations, municipalities), the Insee› It enables an exchange on data collection, control

protocols, useful in improving the processes› It must give an opinion on the texts governing the

organisation of the census

Page 22


The management of quality, process by process : some exemples

› Sampling frames›Data collection›Data processing›Dissemination

Page 23


1 - Sampling frames

› The « quality » survey of the RIL (adress register in large communes)

› An annual survey to assess – The surpluses and the deficits– The surplus-deficit balance– The communes for which the quality is deemed mediocre

› The sample in 2008: 340 000 addresses and 900 000 dwellings› The results

– 1% of dwellings in deficit, 1% in surplus– one commune out of eight beyond 2.5% of deficit– A level of quality higher than that of usual geographical registers – A regular improvement process

Page 24


Sampling frames (2)

› The action plans› To improve the quality of the RIL with targetted operations

– Decided and implemented by the Regional Directorates in relation with the communes

– As a function of the strong points and the weak points detected locally

– Training of communes, fieldwork…› Allowing for an operational management of quality because targetted on

specific problems and based on the closeness between the regional teams and the communes

› The future efforts: reduce the surpluses and arrive at level surplus-deficit balance

Page 25


2 - Data collection

› The control procedures during the data collection

› The follow-up and control upon the initiative of the communes

› The controls at the Insee after the data collection (reception-control, in office control, controls in the field)

Page 26


The control procedure during data collection

› A precise data collection protocol

›Checked by the municipal officer

› And by the supervisor

Page 27


Follow-up and control system upon the initiative of the communes

›Upstream from the data collection–Validation of the sample

–Control-validation of the field to be observed carried-out by the enumerator

›Follow-up of the data collection–Follow-up of the progress of work of the enumerator

–Follow-up of key indicators : FLNE(Non response housing forms), vacant dwellings, quality of the log book of the tour-visits

Page 28


The controls at the Insee after data collection

Successive « sorting » › The step of reception-registering of the questionnaires

– Establishment of indicators– Communes with insufficient scores subject to in-depth control

(approx. 15% of the communes representing 30% of the population)

›Desk controls– Verification with the help of the housing Tax files– On the exhaustiveness of the dwellings, the FLNE, the

structure of main/not main residence

› In field controls

Page 29


The in-field controls

› They are done by Insee agents› They relate to the communes whose quality is

presumed insufficient›Most often, they confirm the data collection› The errors detected are corrected

N.B. these controls are not representative, therefore their results cannot be extrapolated

Page 32


Results of controls of FLNE

› In 85% of the cases, the household enumerator finds the information

› Most often, the FLNE does concern a main residence and the number of persons is correctly estimated

› In 15 % of the cases, FLNEs have been wrongly provided (for non-main residences)

› !!! These figures are not representatives. Controls are made on most difficult cases, where the proba of error is higher

Page 33


3.1 - Data processing : Data entry

›outsourced to indepent service provider

›Quality criteria specifically defined in the specifications– securing of transports and data– maximum rate of error by category of variables

›… and evaluated by an independent control process (sample that has been input twice and arbitration)

Page 34


3.2 – Data procesing : coding of industry and profession

– sample Quality control by twice coding :•Assessing the quality of the automatic coding and manual correction

•Estimating a « threshold » of percentage of bulletins « non codable »

–The contributions of this approach :•adding to the expert system files •the improvement of training and protocols (precisions added in codification rules)

•A finer management of quality, without going in for « excess quality »

Page 35


3.3 - Data processing - Data editing and imputation

–Anayse the share of non responses or incoherent responses

–Imputation of missing data or data presumed non-coherent (hot-deck procedure)

–They are fine-tuned year after year, in view of the quality of the variables produced

Page 36


4 - Dissemination

›A step prepared with the users–Define with them the products and services : two groups

of the CNIS

–Assess their satisfaction : groups of the CNIS and satisfaction surveys

›A systematic validation of the data before dissemination

–At the national level and in each regional directorate since 2009

– Today only at national level.

Page 37


Any questions ?

Documents

The Quality of French census