Navigation – Plan du site

AccueilNumérosHors-sérieRandomised Experiments and the Ev...

Randomised Experiments and the Evaluation of Innovative Placement Schemes for the Unemployed

Évaluation par expérimentation aléatoire de dispositifs de reclassement innovants pour les demandeurs d’emploi
Marc Ferracci et Florine Martin
Traduction de Andrew Clark
p. 65-83

Résumés

L’amélioration des dispositifs d’accompagnement et de placement des chômeurs est un enjeu essentiel des politiques actives du marché du travail. Alors que de nombreux travaux académiques s’intéressent aux effets d’interventions centrées sur les demandeurs d’emploi, cet article se propose d’évaluer des méthodes de travail innovantes visant à accroître l’efficacité des conseillers chargés du placement. Au moyen d’une expérimentation aléatoire sont ainsi testés deux dispositifs distincts. Le premier a pour objectif d’inciter les conseillers à mieux organiser leur temps de travail en leur permettant de concentrer ponctuellement leurs efforts sur un nombre réduit de demandeurs d’emploi. Le second consiste à augmenter les ressources dédiées à la recherche d’offres d’emploi et au positionnement des candidats sur ces offres. Les résultats obtenus montrent que les deux dispositifs peuvent améliorer le taux de sortie du chômage, mais que cet effet positif ne vaut pas pour toutes les populations de demandeurs d’emploi, ce qui soulève la question des effets externes de ce type d’actions.

Haut de page

Texte intégral

Article published in French in Travail et Emploi, n° 135, juillet-septembre 2013.

  • 1At the end of June 2012, the statistics from Job Centres provided by the Direction de l’animation (...)

1The budgets devoted to labour-market policies are not inexhaustible. In 2012, the amount of money allocated by the Budget Law to the programme “Work and Employment” was 11% lower than that in 2011, at a total of 10.2 billion euros. In addition, the subsidies granted to the Pôle emploi (the Public Placement Operator) remained at their 2010 and 2011 levels even though there were substantially more jobseekers to be helped.1 These budget restrictions accentuate, if it was necessary to do so, the need for the systematic evaluation of the policies designed to help the unemployed. The current constraints force us to come up with methods that increase the efficiency of placement schemes without increasing the amount spent.

2In this context, the current article presents the results of a field experiment aimed at increasing the efficiency of the help given to the unemployed, emphasizing the role of the methods used by caseworkers rather than the details of the programme itself (the frequency of interviews with the unemployed, for example). Two innovative schemes are tested using original data collected by a Private Placement Operator, Sodie, working for the Pôle emploi. Compared to the existing work on evaluation in France and elsewhere, the particularity of these schemes consists in the targeting of placement caseworkers themselves, rather than jobseekers. The first scheme aims to help caseworkers better organize their work time, by concentrating from time to time their effort on a small number of jobseekers. The second increases the resources dedicated to the search for job offers and the positioning of candidates regarding these offers. The results show that both mechanisms may increase the exit rate from unemployment, but that this does not hold for all types of jobseekers. This raises the question of the existence of spillover effects from such schemes.

Existing Literature

  • 2See Ferracci (2007) for a review of the literature on the efficiency of reforms to the public empl (...)

3There are by now a number of quantitative evaluations of the effect of enhanced job-search programmes for the unemployed, as compared to the standard counselling they receive in this respect. In France, since the beginning of the 2000s, and even more so since the constitutional reform of 28 March 2003 which authorised random experiments by local authorities, this type of evaluation has grown rapidly. In their meta-analysis of labour-market policies, David Card, Jochen Kluve and Andrea Weber (2010) conclude that job-search counselling policies are generally more effective than subsidised employment programmes in the public sector, and produce faster results than do training programmes. A number of pieces of work on French data have revealed a positive effect of job-search counselling on job-finding by the unemployed. Bruno Crépon, Muriel Dejemeppe and Marc Gurgand (2005) use historical data from the French Public Employment Agency (Agence nationale pour l’emploi – ANPE) to analyse the effects of a variety of programmes on both the return to work and the duration of the subsequent job. There are many evaluations of this kind in other countries which have produced broadly consistent results. Recently, Pathric Hägglund (2009) noted, using an experimental approach in Swedish data, the positive effect of a counselling programme in reducing the length of the unemployment spell. Equally, Richard Blundell and co-authors (2004) evaluate a counselling programme targeted at youth unemployment in the UK, the New Deal for the Young Unemployed programme, and find positive results. The French results in this context are thus in line with those found in other countries:2 counselling programmes increase the job-finding rate of the unemployed, regardless of the type of unemployed or of the agency providing the programme.

  • 3This is particularly the case as both theoretical and empirical work on the outsourcing of public (...)

4These kinds of programmes are costly, however, and the costs have to be compared to the ensuing benefits.3 Recent work has thus attempted to establish the cost-benefit ratio of counselling programmes for the unemployed. One example is Crépon and co-authors (2011) with respect to a programme run by a private placement operator aimed at the recipients of the French minimum welfare payment that is provided to those who are not entitled to unemployment benefits (Revenu minimum d’insertion RMI). While their results do show a positive impact of this programme in terms of recipients’ return to work and benefit receipt, they conclude that the programme is not always profitable in the sense that the value of the benefits does not outweigh the costs. Hainmueller et al. (2011) come to a contrasting conclusion in a different context. In German data, the reduction in caseworkers’ caseload is found to reduce the duration of unemployment, due to a rise in both the penalties applied to the unemployed (as caseworkers now have more time to analyse each individual case) and the number of jobs proposed to the unemployed under this scheme. Contrary to Crépon et al. (2011), the authors conclude that the long-run cost of hiring more caseworkers is outweighed by the positive effects of a smaller caseload.

5While these programmes generally improve the placement of the unemployed, it does not then follow that greater resources devoted to such programmes will automatically produce a favourable cost-benefit figure. As such, we should consider, either at the fringe or the heart of the Public Employment Agency, the introduction of alternative placement schemes that are potentially more efficient. This search for efficiency has led to an increasing use of outsourced services that compete for the placement of the unemployed, with private providers sometimes being argued to be more reactive and innovative in their counselling activities. While the outsourcing of placement services is now widespread in OECD (Organization for Economic Co-operation and Development) countries, France has only relatively recently considered taken this route, starting in the early 2000s.

6A number of evaluations have specifically considered the outcomes of private providers compared to those of the Public Employment Agency. Crépon et al. (2011), already cited above, use double-blind randomisation to show that an intensified counselling programme for the young unemployed with qualifications, carried out by an external service provider (an association, which has a particular legal status, or a private placement operator), produced quicker transitions to lasting jobs. Luc Behaghel and co-authors (2012) appeal to a randomised trial and obtain similar results: intensified counselling programmes for the unemployed, provided by a private operator or a specific unit of the ANPE (Cap vers l’entreprise – CVE), improve the employment prospects of recipients, whether they receive unemployment benefits or not. In addition, Behaghel et al. (2012) emphasize that the effect of the programme run by the CVE is larger and comes about more quickly than that of the private provider.

7The experiment carried out by Behaghel et al. (2012) was accompanied by qualitative interviews with those involved in administering the programme (the private operator and the CVE) and telephone interviews with a representative sample of jobseekers who received either intensified counselling or the normal counselling programme (Gratadour, Le Barbanchon, 2009; Capelier, Mizrahi, 2008). The results from this qualitative research revealed the broad similarities between the intensified programmes proposed by the private operator and the CVE, even though there were some differences. The CVE proposed more employment offers to jobseekers than did the private operator, while the latter concentrated more on help with job search. However, the isolation of the qualitative research from the quantitative evaluation makes it difficult to see how these different practices affected jobseekers’ return to work. The data in Behaghel et al. (2012) does not then allow us to determine which particular methods used by either the private provider or the CVE had a positive impact on job finding. Equally, the qualitative evaluation was not carried out in the traditional framework of a randomised control trial, which, under certain conditions, allows causal relationships to be established. These limitations helped inspire the evaluation work that we describe in the current article.

8Overall then, the results in the existing literature do not allow us to evaluate the impact of innovative placement schemes by the caseworkers who are responsible for finding jobs for the unemployed. In particular, there is no work looking at the effect of caseworkers’ better understanding of the labour market in order to improve the job offers that are proposed to the unemployed. However, field work has suggested that practices in terms of placement schemes for the unemployed vary widely, and in particular among private operators (Divay, 2010). The aim of the current article is to help fill this gap by testing, via a randomised experiment, a number of schemes designed to improve caseworker efficiency. We also address the question of the external validity of our results, and the possibility of generalising these types of scheme. The following section outlines the context in which our analysis took place, and then the details of the experimental scheme and the resulting data.

The Context of the Experimental Evaluation

The Pôle Emploi and Sodie

  • 4For more information, see http://www.sodie.com/.
  • 5A consultancy group for Works Councils, Hygiene, Safety and Work Conditions Committees and Trade U (...)
  • 6A consultancy group for firms and local regions in the fields of local economic development, train (...)

9Sodie4 is part of the Alpha Group, which is involved in the entire employment process via Sécafi5 and Sémaphores.6 Sodie was initially active, in the 1980s, in the retraining of workers from the Steel industry and has since worked on regional revitalisation programmes and the redeployment of workers in a range of industries and occupations. It is today a human resources service and consultancy company that is specialised in labour-market transitions.

  • 7The individuals the most detached from the labour market were defined in the tender document as jo (...)
  • 8As over the duration of the programme, the Job Centre was not able, in some areas, to supply Sodie (...)

10In April 2009, Sodie submitted a bid to the tender announced by the Public Employment Agency for measures aiming to place two particular groups of jobseekers: those in the “Trajectoire Emploi” measure (which we will subsequently call TRAJECTOIRE) directed at those who are the most detached from the labour market,7 and in the “Licenciés économiques” measure (subsequently called LIEC) targeted at the unemployed who had been laid-off and who had signed a Personalised Retraining Convention (Convention de reclassement personnalisée – CRP) or a Professional Transition Contract (Contrat de transition professionnelle – CTP).8

11In addition to the target groups, the TRAJECTOIRE and LIEC measures differed in two other ways: the duration of the programme proposed to beneficiaries (six months for TRAJECTOIRE, but a maximum of twelve months for LIEC) and the outcomes accepted by the Public Employment Agency as authorising the ending of the treatment programme. In the TRAJECTOIRE measures, these outcomes were only permanent employment or a temporary contract of at least six months’ duration, while in the LIEC measure they also included long-duration training or the creation (or reopening) of a firm.

  • 9This type of variable payment schedule is by now the norm for outsourced contracts (Ferracci, 2007 (...)

12The types of successful outcomes recognised by the Public Employment Agency are important, to the extent that they determine whether Sodie receives the variable component of its payment. The payment schedule for the private provider in this programme is as follows: 50% of the amount is paid when the individual treatment programme starts, 25% if the individual is placed (but only 10% if this placement is in a training programme) and another 25% if the placement job lasts at least six months (ditto for the creation/reopening of a firm).9 Payment of the variable component is conditional on proof of placement and duration being supplied to the Pôle emploi.

13The tender was split up into a number of different geographical areas, and Sodie was the successful bidder in 14 of these (representing 22% of the total): Est francilien (Seine-Saint-Denis and Val-de-Marne), Centre and Pas-de-Calais for the TRAJECTOIRE measures; Nord-Pas-de-Calais (two areas), the Île-de-France region other than Paris (three areas), Rhône-Alpes (two areas), Poitou-Charentes, Burgundy, Brittany and Lorraine for the LIEC measures.

14The placement programme ran for two years starting in September 2009, with Sodie being involved with over 45,000 jobseekers in the LIEC programme and almost 20,000 in the TRAJECTOIRE programme. The scale of the jobseeker-placement programme resulted in Sodie opening 110 new sites in the regions concerned and recruiting 100 new employees.

15The size of the programme, the company’s willingness to innovate, and the contributions and limits of the private operator vs. CVE evaluation noted above led Sodie’s Management to engage in an autonomous programme of analysis and evaluation. The aim was to develop methods that would allow the caseworkers working with jobseekers (who Sodie calls Human Resources Caseworkers – RH) to evaluate the results of their work and consequently to improve their work methods. To this end an “Analysis and Evaluation Group” of staff from Groupe Alpha (of which Sodie is part) was established: this consisted of members of Sodie’s Management and Operational teams (the project Director, the project supervisor and the RH caseworker), members of the group’s analysis and forecasting centre, the manager of the computer system dealing with the data on jobseekers and job offers, as well as academics. This evaluation by Sodie led to a number of practical conclusions being drawn, which will be discussed in detail below.

Profiling, Experimentation, Fieldwork: the Three-Point Evaluation of Sodie’s Pôle Emploi Programme

  • 10These three components were set out in a letter from the Groupe Alpha’s analysis and forecasting c (...)
  • 11The use that caseworkers make of this profiling information is interesting in that it allows us to (...)

16The evaluation of Sodie’s Pôle emploi programme consisted of three distinct parts.10 First, the individual and career characteristics of the jobseekers concerned were recorded and analysed, in order to identify the factors associated with a return to work and establish a profiling system. This latter consisted of the estimation of the ex ante probability of finding a job, given their observable characteristics, and was designed to provide caseworkers with a decision tool. As a complement to their own judgement of the individual in question, profiling allows caseworkers to select appropriate measures for each jobseeker in their portfolio. It is important to underline that this profiling information was made available to caseworkers only in offices that were randomly selected in one of the two schemes evaluated (see the paragraph below entitled “Description of the schemes evaluated”).11

Box
Fieldwork Methodology

  • 12Employment areas were classified as dynamic or depressed according to the diagnostics of Sodie’s a (...)

The fieldwork was carried out over a ten-month period between June 2010 and March 2011. It consisted of 35 semi-structured interviews with individuals working for Sodie:
– RH caseworkers;
– RENT caseworkers;
– RENT co-ordinators;
– Project supervisors;
– The management team.
The choice of those interviewed in this qualitative survey should allow recurring issues in the workplace to be identified. All of the regions covered by the Pôle emploi programme were visited. One site per region was chosen according to a number of criteria in order to ensure that the survey covered the outcomes in the following dimensions:
– the measures offered at the site (TRAJECTOIRE, LIEC [CTP or CRP], or both);
– dynamic or depressed employment areas
12;
– the size of the site (in terms of the number of jobseekers);
– whether the site was set up in the local Pôle emploi or not.
These interviews were carried out following a set of guidelines and lasted for an hour and a half on average. All interviews were fully transcribed.
The qualitative work consisted of two phases:
– an exploratory part in order to validate the interview guidelines (three sites visited and around ten interviews in June 2010);
– the subsequent general interviews between September 2010 and March 2011 producing 25 more interviews.

  • 13See the paragraph below entitled “Operational difficulties and their solutions”.

17Alongside the profiling, two pieces of analysis were carried out: a qualitative field survey (see the Box) and a randomised control trial of a number of placement schemes. These two were undertaken in parallel, with the results of the field survey allowing the stumbling blocks in the practical implementation of the experiments to be identified.13 The fieldwork consisted of a set of observations and semi-structured interviews in a sample of Sodie sites that dealt with jobseekers in the context of the TRAJECTOIRE and LIEC measures. Interviews were carried out with the project managers and supervisors, the RH caseworkers and the caseworkers who were in charge of identifying appropriate job offers in the labour market (known as the conseillers relations entreprises - RENT). The aim was twofold. First, to better understand the relationship between Sodie’s work organization (the different types of job and how tasks were split between them, in particular between the RH and RENT caseworkers, the hierarchical links, and the degree of autonomy) and its economic model (billing, the establishment of the sites, etc.), and how these two had changed with the Pôle emploi programme. Second, to establish how these two affected the placement of jobseekers.

18The second analysis was a randomised control trial of a number of schemes designed to improve jobseeker placement: this is described in the following sections.

Innovative Placement Scheme Experiments

Description of the Schemes Evaluated

  • 14Ibid.

19The approach taken by Sodie’s analysis and evaluation group was to first carry out a pre-experimental test of three schemes in six sites in the first half of 2010. These three schemes resulted from discussions in the analysis and evaluation group between academics and those responsible for jobseeker placement in Sodie. This pre-experimental phase allowed the practical difficulties associated with each scheme to be identified and corrected, as far as possible.14 The schemes were a monthly placement engagement (the PARI scheme), greater resources for the service that identified job offers and matched them to jobseekers (the RENT scheme) and the monthly publication of individual RH caseworker performance (the RANK scheme).

20Under this last RANK scheme, RH caseworkers were ranked at each site according to their placement performance, with the results being sent monthly to all RH caseworkers on the site. No penalty or bonus was associated with this ranking. The idea was to motivate caseworkers by allowing them to compare their outcomes to those of their peers, and to encourage co-operation and the exchange of good practice between those with better and worse performance in the same workgroup. This scheme, that was particularly backed by Sodie’s management who wanted to provide performance incentives to caseworkers, was not in the end retained in the experimental phase as it was introduced over the entire company immediately after the pre-experimental test as a result of its almost-zero cost. Only the PARI and RENT schemes were experimentally tested.

  • 15Even though caseworkers were made aware of the issue, this scheme can lead to the cherry-picking o (...)

21The caseworkers in the PARI scheme made a monthly placement engagement with respect to the jobseekers in their portfolio (five placements or 10% of their jobseekers), which jobseekers were identified by name ex ante. In practice, at the beginning of each month, caseworkers selected the jobseekers for whom they made this engagement using the computerised monitoring application. This scheme came out of the observation made by operations supervisors at the sites during the exploratory phase of the qualitative survey that caseworkers sometimes had problems with the organization of their time, consecrating too much time to jobseekers who were in fact quite capable of looking for a job on their own. The PARI scheme was designed to help caseworkers take the time each month to review their entire portfolio and identify the jobseekers who a priori would be likely to benefit the most from help. Caseworkers could then, with the help of the project supervisors, set up their schedules and the initiatives to be undertaken for these identified jobseekers, while not forgetting the other jobseekers in their portfolio. The person responsible for the coordination of the experiment was in charge of collecting and compiling these placement engagements over all of the sites, and chasing up, if needs be, caseworkers who did not make their monthly engagements on time. The PARI scheme was thus presented as methodological aid for caseworkers, aimed at improving their efficiency by helping them organize their time.15

  • 16The open labour market is that where all job offers are published, as opposed to the hidden labour (...)
  • 17At Sodie the department identifying appropriate job offers in the labour market (RENT) is organize (...)

22Under the second scheme tested (RENT), greater resources were affected to the service that identified job offers and matched them to jobseekers. Four RENT assistants were recruited for this purpose over the duration of the experiment, and were set the task of identifying job offers uniquely for the sites that were randomly chosen for this scheme. These assistants were told to look on the open labour market,16 and identify, for a restricted portfolio of jobseekers, relevant job offers from the main available sources. In addition, they ensured that CVs were sent off and followed up each of the applications sent to firms. As well as canvassing for job offers, the RENT scheme thus includes a component regarding the positioning of jobseekers relative to the job offers. This increase in resources and improved matching of job offers and jobseekers should improve placement performance.17

Data and Outcome Variables

  • 18The data from the Job Centre programme thus offer a number of perspectives for future research bey (...)

23The data used to constitute the test and control groups, and evaluate the schemes, come from the internal database used to keep track of jobseekers, called “RHapSodie”. The use of an internal database from a placement company is one of the original aspects of this experiment. RHapSodie allows caseworkers to enter and visualise all of the jobseeker’s individual characteristics: qualifications, career to date, career goals, etc. The identified job offers collected by the service also appear in the same programme. These are available to the human resources caseworkers who can directly match job offers and jobseekers. Caseworkers can also monitor each job application via RHapSodie: this shows the data on the help given to and placement of the jobseeker, the type of services proposed, meetings or training sessions attended, the exit date from the scheme and the type of placement (job, training, creation or reopening of a firm).18 Efforts were made to ensure that caseworkers understood the importance of entering all appropriate data into the programme.

24Our outcome variable is placement in a job lasting at least six months (see above) within a certain time after entering one of the two schemes (TRAJECTOIRE or LIEC). All jobs lasting at least six months count as positive outcomes here, whether permanent, fixed-term or temporary. For the LIEC scheme, long-duration training, of over six months and 150 hours, or the creation or reopening of a firm also count. It is important to note that in the RHapSodie database, exits to employment or training were only billed when the jobseeker brought proof of placement (an employment contract or attestation of admission into a training programme, for example). This way of checking exits does introduce some uncertainty, as certain jobseekers do not send these attestations back to Sodie. As such, the exit rates here should be considered with some caution, especially when they are compared to work in which placement is orally declared by the jobseeker, as in the OPP/CVE evaluation (Behaghel et al., 2012).

Random Allocation and the Comparison of Jobseeker Characteristics

25The random draw took place over the 101 Sodie sites that were opened in the context of the Pôle emploi programme, spread over eight different regions and including participants in the two schemes, TRAJECTOIRE and LIEC. The three sites that participated in the pre-experimental phase were excluded from this draw. The draw took place over sites, and not jobseekers, for both ethical and practical reasons. It seemed difficult to have on the same site jobseekers who would benefit from some additional help and those who would not. In addition, the very nature of the schemes to be tested rendered differentiation across jobseekers on the same site tricky. The experiment was carried out over a ten-month period, between September 2010 and June 2011. Sites were randomly assigned to one of three groups: one of the two schemes to be tested (PARI and RENT) and the control group. The individual assistants in the RENT scheme were also randomly allocated to the different sites that were chosen for the RENT scheme. Random allocation by site, rather than by individual, means that there are two types of jobseeker at each site: those who are already signed up when the experiment started (the stock) and those who entered unemployment after this date (the flow). To the extent that these two groups are not comparable with respect to their potential duration of unemployment, we estimate different effects for the stock and flow groups. In addition, to ensure a degree of homogeneity by site size in the three different groups above, sites were stratified by size before the draw took place. Sites were split up into 34 lots of three sites of similar size, and the allocation into groups (PARI, RENT and control) took place within each lot.

26The objective of this random draw was to constitute two populations of jobseekers in the TRAJECTOIRE and LIEC groups who were similar in terms of their individual characteristics, and so who only differed in terms of the placement schemes that they were offered. Tables 1 and 2 present some descriptive statistics suggesting that this goal was reached, for the stock and flow populations respectively. For each measure in the table, the first three columns show the means for a number of individual characteristics in the RENT, PARI and control groups. With the individuals in these three columns having been allocated randomly, we want to be sure that their mean characteristics are similar. There are certainly occasional differences, but these are small and only rarely significant (the stars in the following columns of the table indicate statistical significance). The three experimental groups thus have similar observable characteristics. We can extend this notion of similarity: if the random allocation rendered the groups similar in terms of their observable characteristics (such as jobseeker age and education), we have good reasons to believe that they are also similar regarding their unobservable characteristics that may play a role in their placement (like their motivation and social network).

27Beyond the validation of the experimental approach of the comparison of means, Tables 1 and 2 also inform us about the characteristics of the individuals in the TRAJECTOIRE and LIEC measures. Mechanically, individuals in the TRAJECTOIRE measure have characteristics which make their placement more difficult. These are summarised in an individual propensity score showing the probability of being in a job one year after having entered unemployment. This score comes from the profiling model developed by Sodie in the context of the evaluation of the Pôle emploi programme. As expected, the average score is lower (around 0.15) for individuals in the TRAJECTOIRE measure than for those in the LIEC measure (around 0.35).

Table 1: Mean Individual Characteristics in the Experimental Groups – Stock Population

Table 1: Mean Individual Characteristics in the Experimental Groups – Stock Population

Note: *, **, *** indicate significance at the 10%, 5% and 1% levels.
° The reconversion variable is a dummy for the ROME (Répertoire opérationnel des métiers et des emplois) code of the desired job being different from that of the previous job. °° The individual score is calculated by regressing employment status one year after entering unemployment on a set of individual characteristics. °°° Site size is the number of individuals on each site, across all different measures.
We thus ensure that the three groups consist of similarly-sized sites.

Source: Sodie

Table 2: Mean Individual Characteristics in the Experimental Groups – Flow Population

Table 2: Mean Individual Characteristics in the Experimental Groups – Flow Population

Note: *, **, *** indicate significance at the 10%, 5% and 1% levels.
° The reconversion variable is a dummy for the ROME (Répertoire opérationnel des métiers et des emplois) code of the desired job being different from that of the previous job. °° The individual score is calculated by regressing employment status one year after entering unemployment on a set of individual characteristics. °°° Site size is the number of individuals on each site, across all different measures.

Source: Sodie.

Operational Difficulties and their Solutions

28The experiment ran into a number of operational difficulties, and a number of changes had to be made compared to the initial scientific protocol and methodology. First, regarding the randomisation, a number of sites were excluded from the experiment after the initial draw. Three as they decided to introduce, during the experimental period, their own version of the RENT scheme even though they had been allocated to either the control or the PARI group.

  • 19For the laid-off who are at the beginning of their CRP/CTP, this refusal to apply for the jobs pro (...)

29A number of operational difficulties also came to light with respect to the implementation of the two experimental schemes. With respect to the increase in the human resources of the RENT service, the four new assistants were only recruited in October 2010, one month after the start of the experiment, and the duration of the learning phase regarding the search for job offers and the positioning of candidates differed between assistants. In addition, given the turnover in the team, in some periods (notably May and June), there were only three assistants instead of four. During these periods the sites allocated to the missing assistant were reallocated to her colleagues. Another problem encountered in the RENT scheme required the priority treatment of some jobseekers. At the start of the experiment, the RENT assistants were asked to identify job offers for all the jobseekers at their sites who had a CV available. Given the large number of jobseekers, and the frequent refusal of some jobseekers to apply for the jobs offered,19 it was decided that the four RENT assistants should concentrate their efforts on the jobseekers who were the most detached from the labour market (in the TRAJECTOIRE group) and individuals who were at the end of their CTP/CRP, who were more likely to accept to apply for a job. The refusal to apply for jobs fell after this reorientation, making the team’s work more effective.

30There were also changes in the PARI scheme during the experiment, with profiling information on jobseekers being made available to caseworkers in February 2011. This decision tool was designed to help caseworkers with their choice of candidate, but also revealed how the jobseeker candidates were selected. Profiling information was supplied in two phases in order to see how behaviour changed: in February and March 2011, this information was given to caseworkers in half of the PARI sites, and only starting in April 2011 did all caseworkers in PARI sites benefit from this information.

Results

Treatment Effect and Spillovers in the Experimental Sites

31The outcome measure for each scheme is calculated over all individuals in the treated sites, in order to take any spillovers, positive or negative, into account. Being a jobseeker at a RENT site does not necessarily mean that the individual will personally be supported by the new team. However, the information about available jobs given to identified jobseekers may well be passed on to others whose CV was not sent off in an application to this job by the RENT assistants. This is a positive spillover, but it is also possible to imagine negative spillovers. These could pertain in the PARI scheme if caseworkers select certain jobseekers from their portfolio for their (temporarily) undivided attention, to the detriment of the jobseekers who were not selected. These types of spillovers justify the use of outcomes for all jobseekers at the site to evaluate the schemes, and not just those who were chosen by caseworkers in the PARI scheme or by the assistants in the RENT scheme.

32The evaluation was carried out for the two groups, TRAJECTOIRE and LIEC, and for the stock and flow unemployed. The stock unemployed are those who were in one of the two groups prior to the start of the experiment in September 2010, while the flow unemployed joined one of the two groups during the experiment, between September 2010 and June 2011. The separate analysis of the outcomes in terms of lasting jobs for the stock and flow unemployed allows the evaluation of treatment efficacity for jobseekers who benefit from the start of their unemployment period (for example, the flow unemployed who were laid-off) and jobseekers who had already benefited from help before the start of the experiment (for example, jobseekers in the stock TRAJECTOIRE group).

Results by Sub-Populations

  • 20It should be remembered that these exit rates are conditional on the caseworkers obtaining proof o (...)

33Table 3 shows the impact of the RENT and PARI schemes on the probability of leaving unemployment for a job or training for the LIEC group. The figures in parentheses indicate the average exit rate for the population under consideration.20 The estimated coefficients thus reflect the percentage-point effect on this exit rate of the schemes. Both schemes have a positive effect for the flow unemployed, regardless of the time period considered, with the effect size being significant and somewhat larger for the PARI scheme. This latter increases the probability of leaving unemployment for a job or training by six percentage points 12 months after joining the scheme (the analogous figure for the rent scheme is five percentage points). The results are more mixed for the stock unemployed, with the RENT scheme only having a positive impact at six months, and the estimated coefficients for the PARI scheme always being insignificant.

Table 3: The Impact of the Schemes on the Probability of Leaving Unemployment for Work, the Creation or Reopening of a Firm, or Training. Linear Probability Models

Table 3: The Impact of the Schemes on the Probability of Leaving Unemployment for Work, the Creation or Reopening of a Firm, or Training. Linear Probability Models

° Given the outcome variable, it should be noted that the stock population is likely heterogeneous, as it consists of individuals who were helped for between 0 and 3 months for the 3-month exit rate, and between 0 and 12 months for that at 12 months. This makes the comparison with the flow results more difficult.
Notes: *, **, *** refer to significance at the 10%, 5% and 1% levels. The standard errors are clustered at the site level.
The table should be read as follows. The RENT scheme increased the probability of a lasting exit from unemployment 12 months after joining the scheme for the flow unemployed in the LIEC group by five percentage points compared to the control group.

Source: Sodie.

  • 21Given that treatment in the TRAJECTOIRE group is limited to six months, we do not present a twelve (...)
  • 22Ferracci and co-authors (2014) use data from the national Assedic database to show that a rise in (...)

34Table 4 presents the impact of the two schemes on the exit rate for those in the TRAJECTOIRE group. Here, the RENT scheme is always associated with positive outcomes for the flow unemployed. The notable result here is that the effect of PARI in this group is now insignificant at three months and significantly negative at six months (two percentage points lower).21 Neither scheme produces significant effects for the stock TRAJECTOIRE group, regardless of the time horizon under consideration. The negative impact of PARI here may reflect a negative spillover, similar to those recently found for other active labour-market policies :22 the PARI scheme may involve less effort being expended on non-selected jobseekers. It is possible that this reduced attention mattered less in the LIEC population, which is made up of individuals who are on average less detached from the labour market and so more autonomous in terms of job search. On the contrary, not being selected as part of the monthly placement engagement could have more serious effects for individuals in the TRAJECTOIRE population, as they likely require more attention than do the LIEC jobseekers.

Table 4: The Impact of the Schemes on the Probability of Leaving Unemployment for Work, Linear Probability Models

Table 4: The Impact of the Schemes on the Probability of Leaving Unemployment for Work, Linear Probability Models

° Given the outcome variable, it should be noted that the stock population is likely heterogeneous, as it consists of individuals who were helped for between 0 and 3 months for the 3-month exit rate, and between 0 and 12 months for that at 12 months. This makes the comparison with the flow results more difficult.
Notes: *, **, *** refer to significance at the 10%, 5% and 1% levels. The standard errors are clustered at the site level.
The table should be read as follows. The RENT scheme increased the probability of a lasting exit from unemployment 12 months after joining the scheme for the flow unemployed in the TRAJECTOIRE group by five percentage points compared to the control group.

Source: Sodie.

  • 23This information was not given to caseworkers on RENT sites and in the control group until April.

35While the jobseekers in the TRAJECTOIRE group then seem to have suffered more from this “neglect”, we do not actually find that caseworkers select those who are the least detached from the labour market in terms of their monthly placement engagements. If anything, the opposite holds, as can be seen from the use that the caseworkers in the PARI sites make of the individual score. From April 2011 onwards (and starting in February 2011 for half of the group), caseworkers in the PARI scheme were informed of the score of all jobseekers in their portfolio.23 This information changed the caseworkers’ selection with respect to their placement engagements : starting in February 2011, the average score of the selected jobseekers actually fell on average. This was also the case at each individual site. Once they were informed of the individual score, caseworkers thus deliberately selected jobseekers with less favourable observable characteristics.

Practical Lessons and the Potential Generalisation of the Results

36The results of this experiment revealed that the RENT scheme leads to significantly better outcomes in the sites in which it was applied. This is not particularly surprising as the combination of the better identification of job offers and greater effort in matching these to jobseekers should lead mechanically to higher exit rates. However, the generalisation of schemes like RENT, either in companies like Sodie or at a wider level in the Public Employment Agency, does pose the question of the cost-benefit ratio. The recruitment of new assistants and the operational costs of the scheme during the experiment, as well as the costs of replacing the assistants who quit the scheme, were sufficiently costly to dissuade Sodie’s management from generalising this scheme to all sites at the end of the experimental period.

  • 24This partnership agreement was signed on the 10 December 2012 with the Qapa platform.

37Despite this conclusion, it should be underlined that this evaluation, in both its quantitative and qualitative dimensions, did lead Sodie to change its practices in terms of collecting job offers. In the context of the qualitative survey, the detailed analysis of the activity of the caseworkers in charge of the identification and matching of job offers revealed that they considered that looking for job offers took up too much of their time, to the detriment of the analysis of jobseekers’ CVs and the matching of jobseekers to job offers. Inasmuch as there are a number of online recruitment services which currently provide the automated collection of a large number of job offers, Sodie’s management decided to enter into a partnership with one such provider in order to have access to its job offers.24 The aim here was to concentrate the work of the RENT caseworkers on the activity in which they were of the greatest value: the matching of job offers and CVs.

38On the contrary, the financial potential of the PARI scheme, when correctly targeted, is clearer. The associated costs are only small, apart from those associated with the follow-up of caseworkers’ monthly placement engagements by a coordinator who also chases up caseworkers who do not submit their engagements on time. As such, the success of the PARI scheme, even though it was limited to the LIEC group who had been laid-off, indicates that investments in innovative work methods can increase placement efficiency with an effect size of the same order as that of (more costly) intense counselling programmes. This is one of this article’s most important results. The limited resources of the Public Employment Agency in the context of budget cuts can be offset by systematic reflection regarding the work methods of caseworkers, followed by the general introduction of the practices with the best results.

39We should however remain cautious about any generalisation of schemes similar to PARI. The qualitative survey run by Sodie underlined the tension that an obligation in terms of results and increasing caseloads put on caseworkers’ activities. This tension is shared by other private service providers, but also to a certain extent by caseworkers in the public sector. In this context, it may be difficult to manage, across decentralised sites, programmes with a considerable methodological component. In the PARI scheme, the requirement to make engagements and their follow-up were sometimes badly received by caseworkers. As such, rather than any accelerated general introduction, the analysis of the practical difficulties encountered during the experiment argues for the organized permanent exchange of best practices. From this point of view, the experiment revealed different effects of the two schemes on the stock and flow unemployed, who likely have unemployment of different durations. The targeting of schemes at those who are most likely to profit from them, here those who have been laid-off for the PARI scheme, would increase the efficiency of the counselling programmes.

40This article presents the framework, method and results of a randomised experiment carried out by a Private Placement Operator aimed at testing the effect of two innovative placement schemes on the exit rate from unemployment. The first (RENT) intensified job collection and matching activity, in order to increase the number of job offers proposed to jobseekers. This increased the placement rate, although the considerable associated costs would appear to be an obstacle to its more general introduction. The second (PARI) consisted of a methodological aid given to caseworkers, by encouraging them to temporarily concentrate their efforts on a restricted group of jobseekers. This scheme, which is far less costly than the RENT scheme, also produced positive effects, but only on those who had been laid-off.

41The research here contributes to the empirical literature on the efficiency of placement programmes for jobseekers from a relatively novel standpoint: that of the caseworkers who are in charge of placement, rather than the jobseekers themselves. The results reveal that providing support to caseworkers may be just as effective as increasing the resources allocated to the placement programme.

42Future work in this area could extend these results in two different directions: first, by carrying out a formal cost-benefit analysis of the schemes taking into account the revenue generated for the private placement operator, part of which is conditional on results; second, by paying particular attention to the spillovers created by these schemes (positive spillovers from information circulation, or negative spillovers from the neglect of some jobseekers) using the data collected during the experiment that has not yet been exploited. In particular, the PARI scheme would appear to deserve more detailed investigation in order to understand the changes in caseworkers behaviour, notably when these latter are provided with jobseekers’ profiling scores.

Haut de page

Bibliographie

Behaghel,, L., Crépon, B., and Gurgand, M. (2012). Private and Public Provision of Counseling to Job-Seekers: Evidence from a Large Controlled Experiment. IZA Discussion Paper, no 6518, Bonn: Institute for the Study of Labor.

Blundell, R., Costa Dias, M., Meghir, C., and Van Reenen, J. (2004). “Evaluating the Employment Impact of a Mandatory Job Search Assistance Program.” Journal of the European Economic Association, 2(4), 569-606.

Capelier, T., and Mizrahi, R. (2008). «  L’évaluation qualitative de la mise en œuvre des expérimentations.  » L’accompagnement renforcé des demandeurs d’emploi, no 2, Paris : ANPE-Dares-Unédic.

Card, D., Kluve, J., and Weber, A. (2010). “Active Labour Market Policy Evaluations: a Meta-Analysis.” The Economic Journal, 120(548), F452-F477.

Crépon, B., Dejemeppe, M., and Gurgand, M. (2005). Counseling the Unemployed: Does it Lower Unemployment Duration and Recurrence? IZA Discussion Paper, no 1796, Bonn: Institute for the Study of Labor.

Crépon, B., Gurgand, M., Kamionka, T., and Lequien, L. (2011). Is Counseling Welfare Recipients Cost-Effective? Lessons from a Random Experiment. Document de travail, Malakoff : Crest.

Crépon, B., Duflo, E., Gurgand, M., Rathelot, R., and Zamora, P. (2013). “Do Labor Market Policies Have Displacement Effects? Evidence from a Clustered Randomized Experiment.” The Quarterly Journal of Economics, 128(2), 531-580.

Dares (2012). «  Demandeurs d’emploi inscrits et offres collectées par Pôle emploi en juin 2012.  » Dares indicateurs, 2012-50.

Divay, S. (2010). «  Nouveaux opérateurs privés du secteur public de l’emploi  : les pratiques des conseillers sont-elles novatrices  ?  » Travail et Emploi, 119, 37-49.

Ferracci, M. (2007). «  Améliorer le service public de l’emploi  : ce que disent les faits.  » Revue française d’économie, 21(3), 75-135.

Ferracci, M., Jolivet, G., and van den Berg, G. J. (2014). “Evidence of Treatment Spillovers Within Markets.” The Review of Economics and Statistics, 96(5), 812-823.

Gratadour, C., and Le Barbanchon, T., «  Les expérimentations d’accompagnement renforcé de l’Unédic et de l’ANPE  : contenu des accompagnements et opinion des bénéficiaires.  » Premières synthèses, 41.2.

Hägglund, P. (2009). Experimental Evidence from Intensified Placement Efforts Among Unemployed in Sweden. IFAU Working Paper, no 16, Uppsala: Institute for Labour Market and Evaluation Policy.

Hainmueller, J., Hofmann, B., Krug, G., and Wolf, K. (2011). Do Lower Caseloads Improve the Effectiveness of Active Labor Market Policies? New Evidence from German Employment Offices. MIT Political Science Department Research Paper, no 2011-22, Cambridge (Mass.): Massachusetts Institute of Technology.

Haut de page

Notes

1At the end of June 2012, the statistics from Job Centres provided by the Direction de l’animation de la recherche, des études et des statistiques (Department for the Coordination of Research, Studies and Statistics – DARES) revealed a rise of 6.9% in the number of registered jobseekers on a year-to-year basis (Dares, 2012).

2See Ferracci (2007) for a review of the literature on the efficiency of reforms to the public employment agency.

3This is particularly the case as both theoretical and empirical work on the outsourcing of public services has underlined that private operators act in order to maximise their profit, given the financial incentives under which they operate. In an extension of their evaluation, Behaghel et al. (2012) consider the determinants of private-operator efficiency, and conclude that financial incentives play a key role.

4For more information, see http://www.sodie.com/.

5A consultancy group for Works Councils, Hygiene, Safety and Work Conditions Committees and Trade Unions.

6A consultancy group for firms and local regions in the fields of local economic development, training and employment, auditing, efficiency and project management (in terms of housing, taxation etc.).

7The individuals the most detached from the labour market were defined in the tender document as jobseekers:
– who do not have any part-time activity;
– who require ongoing support in their job search;
– who face difficulty in finding a job and want a permanent position;
– who live in depressed employment areas;
– who are entering the labour market for the first time.

8As over the duration of the programme, the Job Centre was not able, in some areas, to supply Sodie with sufficient jobseekers who had been laid off to attain the minima required in the contract, the LIEC measures were extended to jobseekers laid off on economic grounds, but who had not necessarily signed a CRP or CTP, and were hence only entitled to the general regime of counselling and unemployment benefits.

9This type of variable payment schedule is by now the norm for outsourced contracts (Ferracci, 2007). The schedule provides incentives for private operators to perform efficiently, but may also lead to selection issues: caseworkers may concentrate their efforts on the jobseekers who are actually the least detached from the labour market.

10These three components were set out in a letter from the Groupe Alpha’s analysis and forecasting centre: “An OPP on the couch. The evaluation of Sodie’s jobseeker placement programme”, La lettre du CEP, No 9, February 2012. This is available online (in French): http://www.groupe-alpha.com/data/document/lettre-cep-9-opp-sur-divan15-02-2012.pdf (accessed 27 May 2016).

11The use that caseworkers make of this profiling information is interesting in that it allows us to identify any cherry-picking behaviour, in which more attention is paid to jobseekers who are less detached from the labour market and so increase the firm’s income. The section below on the experimental results will come back to this point.

12Employment areas were classified as dynamic or depressed according to the diagnostics of Sodie’s analysis group, largely based on Insee (Institut national de la statistique et des études économiques) and ANPE data, such as the Workforce Requirements survey (Besoins en main-d’œuvre).

13See the paragraph below entitled “Operational difficulties and their solutions”.

14Ibid.

15Even though caseworkers were made aware of the issue, this scheme can lead to the cherry-picking of certain jobseekers, with those considered to be difficult cases left to one side in favour of those selected for the monthly placement engagement. This possibility is however limited by the fact that, as in the RANK scheme, the PARI scheme is not associated with any penalty or bonus. The analysis of the empirical results reveals that, in fact, jobseekers who were the most detached from the labour market were not put at a disadvantage by the PARI scheme (see the paragraph “Results by sub-populations” below).

16The open labour market is that where all job offers are published, as opposed to the hidden labour market.

17At Sodie the department identifying appropriate job offers in the labour market (RENT) is organized regionally around four job types:
– RENT caseworkers: one per site in the context of the Job Centre programme. This caseworker identifies job offers on the hidden labour market by going to meet local firms which have jobs open;
– RENT assistants: again one per site. The assistant may be asked to enter the job offers identified by the RENT caseworker into the computerised application, as well as considering the fit between jobs and jobseekers when sending off the CVs transferred by the human resources caseworkers and following up on job applications;
– Telecanvassers: these are the individuals who ring firms to collect job offers. The amount of telecanvassing differs between sites;
– The RENT co-ordinator manages the service identifying appropriate job offers in each region.
As such, while the RENT department traditionally looked for job offers on the hidden market, the new RENT assistants recruited in the context of the Job Centre programme identified jobs only in the open market by
consulting the main available sources.

18The data from the Job Centre programme thus offer a number of perspectives for future research beyond the analysis in the current paper. The conclusion returns to this point.

19For the laid-off who are at the beginning of their CRP/CTP, this refusal to apply for the jobs proposed by the RENT assistants may reflect that unemployment benefits are fairly high, at 80% of last gross wages, so that a number of these laid-off unemployed may have taken a wait-and-see attitude.

20It should be remembered that these exit rates are conditional on the caseworkers obtaining proof of placement and duration, and are thus not directly comparable to the rates obtained from jobseekers themselves during telephone interviews in other evaluations.

21Given that treatment in the TRAJECTOIRE group is limited to six months, we do not present a twelve-month exit rate here.

22Ferracci and co-authors (2014) use data from the national Assedic database to show that a rise in the number of trained unemployed in a local labour market reduces the exit rate for the unemployed without training, probably reflecting displacement. A similar result is found for counselling programmes for the unemployed in the experiment in Crépon et al., 2013.

23This information was not given to caseworkers on RENT sites and in the control group until April.

24This partnership agreement was signed on the 10 December 2012 with the Qapa platform.

Haut de page

Table des illustrations

Titre Table 1: Mean Individual Characteristics in the Experimental Groups – Stock Population
Légende Note: *, **, *** indicate significance at the 10%, 5% and 1% levels.° The reconversion variable is a dummy for the ROME (Répertoire opérationnel des métiers et des emplois) code of the desired job being different from that of the previous job. °° The individual score is calculated by regressing employment status one year after entering unemployment on a set of individual characteristics. °°° Site size is the number of individuals on each site, across all different measures. We thus ensure that the three groups consist of similarly-sized sites.
Crédits Source: Sodie
URL http://journals.openedition.org/travailemploi/docannexe/image/6833/img-1.png
Fichier image/png, 28k
Titre Table 2: Mean Individual Characteristics in the Experimental Groups – Flow Population
Légende Note: *, **, *** indicate significance at the 10%, 5% and 1% levels.° The reconversion variable is a dummy for the ROME (Répertoire opérationnel des métiers et des emplois) code of the desired job being different from that of the previous job. °° The individual score is calculated by regressing employment status one year after entering unemployment on a set of individual characteristics. °°° Site size is the number of individuals on each site, across all different measures.
Crédits Source: Sodie.
URL http://journals.openedition.org/travailemploi/docannexe/image/6833/img-2.png
Fichier image/png, 63k
Titre Table 3: The Impact of the Schemes on the Probability of Leaving Unemployment for Work, the Creation or Reopening of a Firm, or Training. Linear Probability Models
Légende ° Given the outcome variable, it should be noted that the stock population is likely heterogeneous, as it consists of individuals who were helped for between 0 and 3 months for the 3-month exit rate, and between 0 and 12 months for that at 12 months. This makes the comparison with the flow results more difficult.Notes: *, **, *** refer to significance at the 10%, 5% and 1% levels. The standard errors are clustered at the site level.The table should be read as follows. The RENT scheme increased the probability of a lasting exit from unemployment 12 months after joining the scheme for the flow unemployed in the LIEC group by five percentage points compared to the control group.
Crédits Source: Sodie.
URL http://journals.openedition.org/travailemploi/docannexe/image/6833/img-3.png
Fichier image/png, 24k
Titre Table 4: The Impact of the Schemes on the Probability of Leaving Unemployment for Work, Linear Probability Models
Légende ° Given the outcome variable, it should be noted that the stock population is likely heterogeneous, as it consists of individuals who were helped for between 0 and 3 months for the 3-month exit rate, and between 0 and 12 months for that at 12 months. This makes the comparison with the flow results more difficult.Notes: *, **, *** refer to significance at the 10%, 5% and 1% levels. The standard errors are clustered at the site level.The table should be read as follows. The RENT scheme increased the probability of a lasting exit from unemployment 12 months after joining the scheme for the flow unemployed in the TRAJECTOIRE group by five percentage points compared to the control group.
Crédits Source: Sodie.
URL http://journals.openedition.org/travailemploi/docannexe/image/6833/img-4.png
Fichier image/png, 17k
Haut de page

Pour citer cet article

Référence papier

Marc Ferracci et Florine Martin, « Randomised Experiments and the Evaluation of Innovative Placement Schemes for the Unemployed »Travail et Emploi, Hors-série | 2015, 65-83.

Référence électronique

Marc Ferracci et Florine Martin, « Randomised Experiments and the Evaluation of Innovative Placement Schemes for the Unemployed »Travail et Emploi [En ligne], Hors-série | 2015, mis en ligne le 30 décembre 2015, consulté le 29 mars 2024. URL : http://journals.openedition.org/travailemploi/6833 ; DOI : https://doi.org/10.4000/travailemploi.6833

Haut de page

Auteurs

Marc Ferracci

Université de Nantes, Laboratoire d’économie et de management de Nantes-Atlantique (LEMNA), Centre de recherches en économie et statistiques (Crest-ENSAE), Laboratoire interdisciplinaire d’évaluation des politiques publiques (LIEPP-Sciences Po), Chaire de sécurisation des parcours professionnels; marc.ferracci@ensae.fr

Articles du même auteur

Florine Martin

Équipe de recherche sur l’utilisation des données individuelles en lien avec la théorie économique (Érudite), Université Paris-Est Marne-la-Vallée; Sodie; florine.martin@sodie.com

Articles du même auteur

Haut de page

Droits d’auteur

CC-BY-SA-4.0

Le texte seul est utilisable sous licence CC BY-SA 4.0. Les autres éléments (illustrations, fichiers annexes importés) sont « Tous droits réservés », sauf mention contraire.

Haut de page
Rechercher dans OpenEdition Search

Vous allez être redirigé vers OpenEdition Search