Credit Scoring and Credit Control XVI Pre-conference Workshop … · 2019-08-14 · • Up-Selling – What is this and what are the potential benefits and disadvantages? • Over-rides

Second level

Third level

Fourth level

Fifth level

© ThreeCs LimitedCredit Scoring and Credit Control XVI

Pre-conference Workshop

Credit Scoring and Credit Control XVI

Pre-conference Workshop 28/08/19

ImplementationPractical Considerations to Sit Alongside the Theory

David B. EdelmanThreeCs Limited

[email protected]

Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Practical Considerations to Sit

Alongside the Theory

• Characteristic Selection– What are the practical considerations we need when selecting which

characteristics to include in the modelling process?

– Model Selection• Which of the same practical considerations apply when selecting which model to use?

• Setting the Scorecard Cut-off– What are some of the options we use in selecting the cut-off?

• Up-Selling– What is this and what are the potential benefits and disadvantages?

• Over-rides– What are these, what are the key things to consider when managing

them, and how to set our over-ride strategy?



2

Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Characteristic Selection• When we select a characteristic for the modelling

process and, therefore, for possible inclusion in the final

model, what are the features that we are looking for in

this characteristic?

• For example, would hair colour be a good characteristic?



3

Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Desirable - Legality

Some characteristics may be very powerful statistically but are not permitted by law or by agreement.

This varies from country to country

Typically,– Gender

– Full-time / part-time employment – correlated with gender

– Race, colour, religion, country of origin, etc.

There are fewer restrictions - once we leave the consumer area, e.g. data protection and discrimination are less of an issue

- once we move from application to behaviour



4

Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Desirable - Stability

Characteristics, attributes and therefore scores are not

subject to cultural or legal changes

Examples:

• Living with Partner – Married / single

• Single after marital breakdown – single / married / divorced?

• Residential Status – tenant / living with partner

• Retired / Unemployed

• Legal Status – sole trader / partnership / lim. co.



5

Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

• We want characteristics which, either on their own or in combination with other characteristics, help us to separate along the spectrum of scores,

• the very low risks

• the moderately low risks

• the medium risks

• the high risks

• ... but how do we measure this?

Desirable – Ability to Differentiate

Between Goods and Bads



6

Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Information Value / Power• One measure that we use is the Information Value and Power

• Information value =

• ( (% Good - % Bad) * Ln (% Good / % Bad) )

Power = Information Value * 1000

The higher the power, the more the characteristic, when considered on its own, is able to separate out the cases into different levels of risk.

Rule of Thumb

Power < 50, discard characteristic

50 < Power < 200, discard characteristic unless there is a strong reason to include it in the modelling process

Power > 200, include characteristic in the modelling process, but select one from each group of obviously highly-correlated characteristics


Pre-conference Workshop 7

Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Desirable - Intuitive Sense

• The characteristics in a scorecard should make sense.

• The score weights attached to the attributes should make directional and relative sense.

• There is no formal test – just an application of some common sense.



8

Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Desirable - Simple to Obtain

We need information that is simple to obtain and reliable.

Characteristic:

• Total £ Credits last 6 months / Total £ Debits last 6 months

• Divided by

• Total £ Credits last 2 months / Total £ Debits last 2 months

Might be very powerful – but how to source the information?



9

Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Characteristic Selection

• Relevance – ability to predict risk – power of characteristic

• Legal

• Stable

• Intuitive

• Simple to Obtain

additionally

• Verifiable

• Clear and Unambiguous

And if the characteristics have these properties, then the model

is a long way to having them as well



10

Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited



• Characteristic Selection

• Setting the Scorecard Cut-off

• Up-Selling

• Over-rides



11

Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Score

% P

opul

atio

n

Cut-

off

Reject

Rate

Bad

Rate

Scorecard

12Credit Scoring and Credit Control XVI


The scorecard simply ranks cases in terms of risk.

The business decides where to set the pass mark.

Different businesses have different business models and

operate at different points on the risk spectrum.

Setting the Cut-off – Business Strategy

Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Setting Cut-off

• What factors might we use to set the cut-off?

• How do you decide who to accept and who to reject ?

• In behavioural scoring or collections scoring, how do we

decide which treatment to give to which account?

• What thresholds might we apply to these factors?



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

What Factors do we Use to Set Application

Scorecard Cut-off’s?

• Acceptance Rate• Bad Rate• Loss• Marginal Bad Rate• Breakeven Position• Maximum volume subject to some constraint

• Profit and Loss of Tranches of Lending• Profit and Loss of Product• Projected Provisions• Balance Sheet Growth



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Strategy Trade-off Table / Run Book

This is a report created to show, for a wide range of cut-

off’s, what will be the impact on:

• Acceptance Rate

• Bad Rate

• Marginal Bad Rate

• Decline Rate for Good accounts



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Strategy Setting

0%

5%

10%

15%

20%

25%

140 160 180 200 220 240 260

ScoreNew Scorecard - Options

Current Scorecard

Use data extracted from the Run Book to consider strategic options



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Cut-off and Business Strategies

• Each position on the curve of the new scorecard represents a

different risk appetite with different consequences for the

operation and the financials of the business.

• As an example, we are going to consider four different, but feasible and rational, options for setting the scorecard cut-off, to help us decide where to set the cut-off.

• Each option has quite different effects on the dynamics of the P&L Account and ultimately on the business’s Balance Sheet.

• We refer to an extract from the Run Book for this scorecard development. This extract reveals the current Rates and Numbers of Accepted Accounts and Bad Accounts.



Second level

Third level

Fourth level

Fifth level



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Setting the Cut-off - ScenariosA Same acceptance rate

B Same Bad Rate

C a 1/5th reduction in acceptance rate

D Same Value of Losses

For each scenario, we can evaluate the impact on

Acceptance Rate

Number of Good cases

Number of Bad cases

Bad Rate



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Options - Summary

Cut-off Acceptance

Rate

Bad

Rate

# Good’s # Bad’s

A

B

C

D *

Current Scorecard

Acceptance Rate 35.2% Bad Rate 3.91%

= change in # = number of

* We will make a reasonable assumption



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

• This scorecard is calibrated so that a score of 200 gives the same acceptance rate as the previous scorecard.

• With the same cut-off and the same Acceptance Rate of 35.2%, we use a better scorecard to remove some Bad Accounts that we took on last time and replace them with some Good Accounts.

• The better scorecard reduces the Bad Rate at the cut-off (or in the 200-204 band) from 9% to 4.54%. This is the Marginal Bad Rate.

• The better scorecard reduces the Bad Rate from 3.91% of all accepted applications to 2.57% of all accepted applications.

• This generates 591 fewer Bad accounts – reduced losses – and also 591 more Good accounts – higher income. These groups of 591 accounts are the “swap sets”.

21

Option A – Same Acceptance Rate



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Option B – Same Bad Rate

• Option B uses the fact that the organisation is comfortable with a Bad Rate of 3.91%.

• We can use the better scorecard and accept many more cases. We can do this with a scorecard cut-off of 180. This raises the Marginal Bad Rate at the cut-off from 4.53% to 7.84%, up by more than 70%.

• This generates more Bad Accounts but many more Good accounts.

• Acceptance Rate increases from 35.2% to 52.0%. This generates 2540 Bad accounts, an increase of 820. This also generates an additional 20180 Good accounts.

• This strategy will introduce a much larger scale to the business, with the consequences of that.



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Option C – Reduce Acceptance Rate by 1/5th

• Option C uses an improved credit assessment tool to raise significantly the quality of the book.

• We need an Acceptance Rate of 4/5 * 35.2% = 28.16%.

• We can do this with a scorecard cut-off of 210, giving an Acceptance Rate of 28%.

• This generates fewer Bad Accounts but also fewer Good accounts.

• The Bad Rate falls to 2.11%, and the Acceptance Rate falls from 35.2% to 28.0%

• This generates 737 Bad accounts, a reduction of 983. It also generates a reduction of Good accounts totalling 8017.

• Option C represents a reduction in business but a much higher quality of new business.



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Option D – Same Value of Losses

• Option D assumes that the organisation is comfortable with the existing number of Bad accounts, and the existing losses.

• We shall assume that the average loss on a Bad account is unchanged.*

• Therefore, we need a cut-off that offers c.1720 Bad accounts.

• This is at 190.

• Acceptance Rate increases from 35.2% to 43.2%

• This generates 1719 Bad accounts, the same number of Bad cases but a higher number of Good cases. The Bad Rate falls to 3.18%.

• This generates an additional 10001 Good accounts.

• * If we think that the average loss on a Bad account has increased by 3% or

5%, this can be factored in.



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

0%

5%

10%

15%

20%

25%

140 160 180 200 220 240 260

Score

A

A Same Acceptance Rate - Reduced Bad Debt

B Same Bad Debt Rate – Much Higher Acceptance Rate

C Reduced Acceptance Rate – Much Reduced Bad Debt

D Same Bad Debt Value – Higher Acceptance Rate

C

B

D



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Options - SummaryAcceptance Bad Good’s Bad’s

Rate Rate

Current 35.2% 3.91%

A 200 35.2% 2.57% 591 (591)

B 180 52.0% 3.91% 20180 820

C 210 28.0% 2.11% (8017) (983)

D 190 43.2% 3.18% 10001 (1)



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Loss / Expected Loss

• Some organisations use Expected Loss in setting cut-off.

• Expected Loss =

– Probability of Default (PD) * %

– Loss Given Default (LGD) * %

– Exposure at Default (EAD) $ / £ / €

• Many organisations focus on Bad Rates – which will be aligned with PD only



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Loss / Expected LossOne implication from using EL is that … we will accept

some higher PD cases with low LGD.

In a mortgage book, for example, this will mean that we will

take on some higher risk (PD) cases where the LTV is

low and so we are less likely to lose ultimately.

This is really “lending against security” which is against

standard bank lending principles.

It also generates additional work for the collections

department – although it may be “good” work for them as

they can collect / recover much of the sums due.



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Setting the Cut-off – Next Steps• Assess profitability of each option

• Understand constraints – operational, capital, regulatory

• Data and modelling challenges– Profit on a good account and Loss on a bad account

– Profile of future applicant population

– Income from X-selling product v. customer

– time horizon

– accurate knowledge of costs

– quality of earnings – penalty fees - a good source of income?

– amount classified and percentage recovered

– fixed costs apportionment

• Too good for credit

• Getting Executive approval



29

Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited





• Up-Selling

• Over-rides



30

Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Up-Sell – Definition

• Applicant applies for a personal loan of £12000.

• Assessed as satisfactory – pass – accept.

• Affordability assessment suggests that we would lend up to £17000.

• Do we offer (i.e. try to up-sell to) £17000?



• Do we offer only the £12000 the applicant requested?



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Up-Selling

• This is a way for the credit function to support the

business in its drive to higher profits.

• Should we do this?

• What are the advantages and disadvantages?



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

There may be a difference between

applicant asking for £12K

applicant asking for £12K but our assessing that they

could afford to borrow £17K and they then borrow £17K

Reputational issue?

Performance difference? - need to record both amounts

33

Up-Sell Issues



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Up-Sell Issues

• Pricing

• Higher borrowing may be at lower interest rates.

• TescoBank - pricing from July 2019

• £4800 @ 10.5% 36 months £154.94 per month

• £5200 @ 3.5% 36 months £152.24 per monthBorrow £400 more and it actually costs £97.20 less – a gain of £497!

• Business loss

• An extra loan – to cover sunk marketing and sales costs

• Lower value loan – higher interest rate

• Customer benefit



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Up-Sell Issues

• Sales

• How is increased limit to be sold / communicated?

• What is the process?

• What is the justification?

• Is it affected by what is being purchased?

• Sales incentives on value of sales

• Sales incentive on value of income e.g. value * Interest Rate



Second level

Third level

Fourth level

Fifth level



The Moral Dimension

• What would constitute responsible borrowing?

• What would constitute responsible lending?

• A dilemma for Ethics man!

36

Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited





• Up-selling

• Over-rides



37

Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Over-rides

• Definition

• Operational Management

• Strategy



38

Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Over-Rides

• What are they?

• Who sanctions them?

• Recording, managing and tracking over-rides



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Over-rides – what are they?

• Cases where the outcome/decision is contrary to the

scorecard recommendation.

– Recommended “Approve” cases but which are declined –

High Side Over-rides – HSO’s

– Recommended “Decline” cases but which are approved –

Low Side Over-rides – LSO’s

• Both need to be recorded for analysis and tracking



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Low Side Over-rides

• Usually, LSO’s are more problematic than HSO’s

• If approved by underwriters - may be OK

• If approved by branch staff / dealers / brokers –

usually not OK as a process



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Low Side Over-rides

• How do they occur?

• What reasons do people use to try to justify them?



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

• Good Customer What is a good customer? Whose definition? Consistency from one operator or member of branch staff to another?

• Profitable Customer Do we know? Do we know accurately? Or are we just making some assumptions, perhaps based on number of products held?

• I don’t understand why it failed While education about the policy and the process should be implemented, it is not the role of the sales operator to understand every decision. Perhaps, we should have a mechanism for them to refer isolated cases.

• We used to accept this type of proposal We have changed our policy because it needed to be changed. Or we may have changed bureau. To over-ride just takes us back to where we were.

• I think we should experiment with this one We can consider experimentation - but it is not the role of the branch staff or dealer to decide on which cases should be included in an experiment.



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

• Only failed by a few points If you approve a case failing by 3 points, then next week you will be asked to approve a case failing by 5, and then by 6. This erodes our cut-off decision. Again, perhaps we should have a mechanism for them to refer isolated cases.

• Value of connection How do we measure this? It is true that declining the application may jeopardize the relationship. However, it can work in the other direction. Granting a child a loan may annoy the parent.

• Cross-subsidisation How do we identify which product(s) are actually subsidising which?

• I need it to reach my sales target Even if we have sympathy with this, why should we allow them to over-ride the case at 5 p.m. on a Friday? There is almost certainly a case much more deserving, closer to the cut-off, that was declined earlier in the week / month.



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Potentially Acceptable Reasons

• Controlled and planned experimentation with cut-off

and/or policy

• Underwriter selection of anomalous good cases

among declines using Additional Data Not available

to the scorecard



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Low Side Over-rides

• Need recording and tracking

• Should be managed as normal

• May tell us something about our process - especially

if they perform well

• Can we amend credit criteria and take them on in the

normal course of events?



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Beware of Cherry-Picking

0%

2%

4%

6%

8%

10%

12%

14%

16%

18%

20%

<200 200-219 220-239 240-259 260-279 280-299 300+

Score

Bad

Dev

Sample 1

Sample 2

Sample 3



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Portfolio 1: Age < 30

0%

10%

20%

30%

40%

50%

60%

0 - 529 530 -

539

540 -

549

550 -

559

560 -

569

570 -

579

580 -

589

590 -

599

600 -

609

610 -

619

620 -

629

630 +

0

1000

2000

3000

4000

5000

6000

Volume Bad Rate Predicted Bad Rate



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Cherry-Picking

LSO’s may be subject to cherry-picking or very careful selection

If so, we would not want to take on all cases, for example, in the

scoreband below the current cut-off.

This may be for reasons of:

• Risk

• Processing

• Business objectives

Much better - try to identify the rules of the cherry-pickers and

implement these into the system.



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

High-Side Over-Rides

How do these occur?

• Operational checks implemented after the scorecard:

– credit policy

– verification checks, e.g. Money Laundering, Matching at Bureau

• Underwriter Caution or inability to deal with new types of risks or

inability to price accordingly

• The applicant has an existing account with the bank in arrears

• The organisation has had a bad credit experience with the

applicant



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

HSO’s

Usually represent less of an issue – as they are “only” a

lost opportunity rather than an actual exposure.

We don’t know how they would have performed.

Cases which pass the scorecard cut-off and are

rejected will, on average, be of lower quality than

those which pass and are accepted … but

would they have been profitable?



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Over-Ride Analysis

For the HSO’s we do not know how they would have

performed. We have another reject inference problem.

Analysis:

• Try to understand why they were rejected.

• Accept a sample of them and see how they perform.



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Over-Ride’s – Some Strategy Options

Through analysis, develop our strategy for if and when

we allow over-rides.

Over-

rides OK

LSO’s

only

HSO’s only

within15

points of

cut-off

HSO’s

only for

certain

reasons

No Over-

rides

Volume

Bad Rate

# Bad’s

Profit



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited

Over-rides - Summary

• Overrides should be processed through a credit / underwriting / review process.

• Manual processing costs increase when overrides are in place.

• We need to extract value from this additional cost.

• Over-rides should not generally be subject to “special handling”

• Over-rides should be tracked and analysed.

• Over-rides generate uncertainty when analysing alternative strategies.

• We should try to control the over-ride percentages.



Second level

Third level

Fourth level

Fifth level

© ThreeCs Limited





• Up-Selling

• Over-rides

Enjoy the Conference

David B. Edelman

ThreeCs Limited

[email protected]



Documents

Credit Scoring and Credit Control XVI Pre-conference Workshop … · 2019-08-14 · • Up-Selling – What is this and what are the potential benefits and disadvantages? • Over-rides