Performance management through outcomes is very much in vogue in further education (and elsewhere) at present. The benefits of concentrating on outcomes are compared favourably, first with input management which concentrates typically on measuring and controlling resources invested in delivering services, but this gives little indication as to what has been achieved through this investment.
Next in an apparent hierarchy is measurement of outputs, but here measuring the number of widgets made, or courses delivered in further education still fails to identify the ultimate return from resources invested. When courses delivered are combined with the results achieved by students, we move closer to an outcome. Indeed, the combination of student retention rates to complete their courses and the passing of qualifications to form ‘success rates’ has, for the last 15 years, at least, become the pre-eminent measure of performance in further education.
The limitation of success rates as that pre-eminent measure has, however, been challenged in the recent (ish) FE White Paper, the implementation through detailed legislation and regulation is now underway. A key argument against success rates is that, although initially effective in focussing colleges and others in focussing on student success, it has fallen foul to an almost immutable rule that as a measure of performance becomes pre-eminent it eventually begins to subvert and unbalance systems and institutional behaviours.
Hence, it is argued that providers were incentivised to put students on courses that were insufficiently challenging, to achieve success more easily; when financial considerations are added to this tendency there was also a risk that students were directed toward courses that were cheaper to deliver in terms of infrastructure and staff costs.
The alternative that underwrites much of the thinking in the White Paper is to focus on outcomes; here, the ‘outcome’ that the White Paper concentrates upon is progress to further study and/or employment. In this context, the completion of a course and the achievement of a qualification are quite properly identified as simply steppingstones to a beneficial outcome. The argument is then that focussing on a beneficial outcome will then generate more balanced and desirable behaviours and performance in providers.
This is where the idea of Russian dolls comes in – there is an argument that every outcome is simply an output toward a further outcome. A job is indeed an outcome, but it is also an output toward the outcome toward a good job, however that might be defined, whether by salary, social value, or other means. But even a good job is only an output in terms of a life well-led and a healthy society. Each outcome is then only another output in a hierarchy of succeeding outcomes.
Moreover, with each succession complexities in measurement ensue. Getting a job outcome, much less a well-paid and rewarding job, will be harder for college in an area of high unemployment, than in a more prosperous area. If performance management rewards colleges in terms of job outcomes, the college in a deprived area doing really difficult work risks becoming poorer with compared to a college serving a more prosperous area where job outcomes are easier to achieve – the very antithesis of ‘levelling-up.’
Advocates of such an approach may argue that a weighted formula could address such situations, but this seems likely to produce still more complexity in FE which the White Paper argues elsewhere has prevented the Sector from realising its full potential.
This argument against the simplistic use of outcomes illustrates a further complication – that the further performance management develops away from the input of resource management and the output of student retention and completion, the more that the ‘outcome’ is shared between providers, funders, agencies and government.
Outcomes are, in this logic, inherently better measures of overall system performance, than individual actors within the system. Addressing underperformance at the system level should therefore be the objective and at this level, I think there are at least two preconditions: first, greater trust between providers, agencies, and government, and second better information on the destinations of students, not only in the job (or course) that they progress into after their course but across their lives. Tracking destinations in this way is obviously an enormous job but in the era of big data and AI, the tools are becoming available.
==========
Author: Martin Doel
Following a 28-year career in the Royal Air Force, which culminated in him serving as Director of Training and Education for the Armed Services, Martin Doel joined the Association of Colleges as their Chief Executive in 2008. In this role he worked closely with the government to develop the skills and education policy, and was a member of the Apprenticeship Reform Board. In 2016 he was awarded a CBE for his contribution to further education. He now advises Purlos on building its footprint in the UK Further Education market.