Upload
andra-roberts
View
223
Download
0
Embed Size (px)
Citation preview
Valuation Office Agency
Experience of Automated Valuation Modelling (AVM)
in England
Tim Eden BSc MRICS IRRV
Deputy Director of Council Tax
Valuation Office Agency
Valuation Office Agency
Presentation overview
A “fly through” of the VOA AVM experience including: Background The challenge The extent of data capture Changes in data management Process improvement and change Model development Training Achievements
Valuation Office Agency
To be a world class organisation providing valuation and property
services for the public sector
Valuation Office Agency
VOA - Our purpose
To provide a fair and robust basis for taxes, which help to pay for public services; and to help drive better use of property in the public sector by: compiling and maintaining accurate and comprehensive valuation lists for
local taxation providing accurate valuations for national taxes delivering expert advice on valuations and strategic property management developing and maintaining a comprehensive
and up to date property database advising policy makers on valuation and
property issues
Valuation Office Agency
VOA - Our structure
Country VOA VOA Deals with: -
No. of Offices
Local Domestic Property Taxation
Local Non-domestic
Property Taxation
National Property Taxation
Other Government
Property Services
England 67 Yes Yes Yes Yes
Wales 7 Yes Yes Yes Yes
Scotland 5 No No Yes Yes
Valuation Office Agency
“Recent” Taxation History1956 General National Valuation – to Point Rental Value
1963 General National Revaluation
1973 General National Revaluation
1982 General National Revaluation (Cancelled)
Non-Domestic Domestic
1990 National Revaluation with Antecedent Valuation Date
(AVD) 1 April 1988
Individual-based Taxation called Community Charge (“Poll Tax”)
1993 National Property Valuation for Council Tax to Capital Value
Bands (AVD 1 April 1991)
1995 Revaluation AVD 1 April 1993
2000 Revaluation AVD 1 April 1998
2005 Revaluation AVD 1 April 2003
2007 Revaluation AVD 1 April 2005 (Postponed)
Valuation Office Agency
Council Tax (CT) - Context
CT + Non-Domestic Rates together raise c. £40bn pa
CT raises £18.5bn pa 22m domestic properties in England 1.3m domestic properties in Wales Clients
Department for Communities & Local Gov’t (DCLG) in England
Welsh Assembly Gov’t (WAG) in Wales
Valuation Office Agency
CT – 1993 List Bands (England)
A - up to £40,000 B - £40,001 to £52,000 C - £52,001 to £68,000 D - £68,001 to £88,000 E - £88,001 to £120,000 F - £120,001 to £160,000 G - £160,001 to £320,000 H – over £320,001
Valuation Office Agency
CT - Revaluation
Local Government Act 2003 included requirement to undertake revaluations in Wales (2005) and England (2007)
In Wales AVD 1 April 2005; list came into force on 1 April 2005
In England AVD 1 April 2005; list was to come into force on 1 April 2007
Draft lists were to be published 1 September 2006
Valuation Office Agency
The Challenge
New banding scheme not known Necessary to produce individual
valuations & then overlay new bands Scale of task Complexity of housing types Vagaries of the market Inconsistent, paper-based records
Valuation Office Agency
The Challenge – Market Vagaries Market analysis across England
Free flow of money Historically low interest rates Fluctuating volumes in sales
Sales Relative to Same Month in Previous Year - Aug '05 Data
0%
20%
40%
60%
80%
100%
120%
140%
160%
Jul-04 Aug-04 Sep-04 Oct-04 Nov-04 Dec-04 Jan-05 Feb-05 Mar-05 Apr-05 May-05 Jun-05
Month
Pe
rce
nta
ge
Birmingham
East Anglia
East Midlands
Leeds
Liverpool
London-City
London-North
London-South
London-Westminster
Manchester
Newcastle
North West
Reading
Sheffield
South East
South West
St Albans
Wessex
West Midlands
Western
England Total
Sales Relative to Same Month in Previous Year - Aug '05 Data
0%
20%
40%
60%
80%
100%
120%
140%
160%
Jul-04 Aug-04 Sep-04 Oct-04 Nov-04 Dec-04 Jan-05 Feb-05 Mar-05 Apr-05 May-05 Jun-05
Month
Pe
rce
nta
ge
Birmingham
East Anglia
East Midlands
Leeds
Liverpool
London-City
London-North
London-South
London-Westminster
Manchester
Newcastle
North West
Reading
Sheffield
South East
South West
St Albans
Wessex
West Midlands
Western
England Total
Valuation Office Agency
The Challenge – Housing Types Profile of Housing Stock
80% Houses/Bungalows 20% Flats/Maisonettes Over 1/3rd of all flats in Greater London Around 1/6th of properties at least 100 years old BUT high proportion of new properties in sales evidence!
National Age Profile
0.00%
2.00%
4.00%
6.00%
8.00%
10.00%
12.00%
14.00%
16.00%
18.00%
<1900 -1918 -1929 -1939 -1954 -1964 -1972 -1982 -1992 -1999 >1999
Age (Era)
Co
un
t (
wh
ere c
od
ed
)
National Age Profile
0.00%
2.00%
4.00%
6.00%
8.00%
10.00%
12.00%
14.00%
16.00%
18.00%
<1900 -1918 -1929 -1939 -1954 -1964 -1972 -1982 -1992 -1999 >1999
Age (Era)
Co
un
t (
wh
ere c
od
ed
)
Valuation Office Agency
The Solution!
Develop an AVM Skill-up staff Learn from others
Cole Layer Trumble (CLT) – computer modelling expertise, AVM software
KPMG – programme & project management support
IAAO – statistical training Digitise data
Valuation Office Agency
Began in 2002; worldwide research Mass data capture/enhancement Model development Many innovations and lessons
These aspects have been very important to the success of the project
• Improving Business Decisions
• AVM design
• MRA models
• data collection • 22 million values• data cleansing
• data enhancement• business case• political issues
• value reviews
• staff planning and allocation
VOA - AVM Journey
Valuation Office Agency
Attribute Data Investigation
Investigation commences in 2002 considering: Use of existing “attribute” codes already available on VOA IT
database (“Group”, “Type”, “Area”) Valuer and caseworker engagement to determine useful
codes Data availability & maintainability
CT List Maintenance work External standards e.g. IAAO (6 year cycle) Impact of differing local practices
Ability to undertake sales investigation activity
Pre-contractor appointment; so limited AVM expertise
Valuation Office Agency
Main Attributes to be digitised
Architectural Style and construction quality – “Group” Property Type e.g. House Semi-Detached – “Type” Age (coded as era e.g. G – 1965-72) Area m² - external area (houses); internal area (flats) No. of rooms No. of bedrooms No of bathrooms Number of floors (houses); floor level (flats) Parking Conservatory (type and size) Outbuilding details Value Significant Codes
Valuation Office Agency
What about Sales Data?
Access to Land Registry and Stamp Duty Land Tax information
Address irregularities – creates matching problems and manual effort
Need to establish whether transaction at “Market Value”, special circumstances etc.
Data understanding significantly assists AVM geared to support this process “Value” understanding working with AVM is key
to the process
Valuation Office Agency
Process Development
Consider existing process The impact of AVM – scale and profile Skills
Existing skills and abilities of targeted staff The need to re-train certain staff
Resource availability New process to reflect Revaluation needs
and business as usual IT constraints and capabilities Understand relationship between sales and
model performance
Valuation Office Agency
Before
Sales received
Suspect sales coded
Sales searched to support each valuation case “on paper”
At a Revaluation “Beacon Sales” reviewed and recorded “on paper”
Property Data
(Group, type, floor area, age, bedrooms,
rooms, garage)
Property Data
(Group, type, floor area, age, bedrooms,
rooms, garage)
Models
(MRA, algorithms
and variables)
Models
(MRA, algorithms
and variables)
Locality Definition
(area from which
comparables derived)
Locality Definition
(area from which
comparables derived)
AVM
AVM analysis output
reviewed by
qualified staff
AVM analysis output
reviewed by
qualified staff
Decision to change, retain or include current
data and coding
Decision to change, retain or include current
data and coding
Sales Data
1)New Sales continually updating
analysis set
2) Value significance considered
Sales Data
1)New Sales continually updating
analysis set
2) Value significance considered
After
The Analysis Process
Valuation Office Agency
Initial Issues with Modelling Process
No first hand knowledge of sale Data time lag Data gaps due to
“Permissible development” – the public need not inform the Billing Authority (BA)
BAs internally not acting in a joined-up way – billing, planning & building control
BAs not acting in joined-up way with VOA Are we maintaining lists or data or both?
“Condition” not an attribute
Valuation Office Agency
Integrating the AVM
Investigation work commenced late 2002 Procurement process 2003 Collaboration of in-house team and
successful contractors/partners - Cole Layer Trumble (CLT)
Working on BA areas where data capture is advanced (note had only started in 2003!)
Challenge of integrating AVM technology with existing VOA IT systems
Valuation Office Agency
Getting Started
Started with a simple additive model £ = B0 + B1 * size + B2 * Detached * Size + B3 * Terraced *
Size + … + Bn * Date of Sale * Size
1993 Band initially used to support analysis, but quickly stripped out of valuation models
Postcode sectors used as proxy for location – VOA developed “localities” Postal areas are too crude Designed to support mail delivery, not to reflect influences
on the property market!
Valuation Office Agency
Performance Improvement (1)
Better understanding of subject/sale data helped to: Determine usability rules for raw data Determine usability rules for a sale
This understanding fed Comparable selection Model specification Consideration of rarely occurring variables Consideration of locality relationships
Valuation Office Agency
High Correlations
Group 55 separate codes
Age11 separate codes
Type
32 separate codes
Parking
Floors/
Floor Level
BedroomsRooms
AreaBathrooms
Correlation among variables
High correlation compromises
modelling stability
Valuation Office Agency
Localities – illustrative photograph
Local Authority housing estate
Local Authority housing estate
Privately built housing estate
Privately built housing estate
Created bespoke localities (neighbourhoods)
Valuation Office Agency
Mapping & Localities
In excess of 10,000 localities Regular boundary review required Thematic mapping as part of process X-Y co-ordinate becomes necessary
data X-Y suitable for comparable selection Issue of who maintains the data
Valuation Office AgencyMapping Example
Valuation Office Agency
Performance Improvement (2)
Move to multiplicative (log linear) model structureLog (£) = B0 + B1 * log (size) + B2 * Detached + B3 * Terraced
+ …
+ Bn * Date of Sale + Bp * Log (Locality Adjustment Factor)where Bi determined from sales set using MRA
This enabled improvements in: - Locality Adjustment Factor (time adjusted median
price per square metre) Locality Grouping – support for comparable selection Central Modelling across the whole country
Valuation Office Agency
Benefits of Central Modelling
Central Modelling enabled: Central recognition of national modelling
patterns and data issues Modelling “constraints” could be imposed
by the centre ensuring model coefficients are consistently applied
Effective direction of effort Calculation of market trend information to
support VOA modelling and wider government market appreciation
Valuation Office Agency
Performance Improvement (3)
Integrating X-Y & Mapping allowed mass calculation of “plot size”
Issues with map plots are: Plot bleed Non-alignment with the property
transferred Maintenance of data
Valuation Office Agency
Model Development - Lessons
Create a multi-skilled and focused R&D team Select several representative areas to test Ensure proper debate on proposals Predict overall likely modelling gains Sense check & gather feedback from local
staff Business model to relate model performance
and cost/benefit Produce individual cost/benefit for each
proposal Promote external verification e.g. IAAO
Valuation Office Agency
Supporting Decision Making (1)
VOA recognised management of a national valuation delivery required consistency in valuation decisions
Property level confidence score required COD/COV inadequate existing banding too remote
Score needed to support decisions at several levels: Strategic:
Business case development Process definition Data collection & enhancement
Tactical: Resource planning
Operational: Resource allocation Value review or band review
Valuation Office Agency
Supporting Decision Making (2)
Confidence estimate for each subject property needed to reflect the aspects which would reduce accuracy:
Data quality Data availability Market variability Model accuracy
Adequate coverage for accuracy in the MRA Adequate comparables
Valuation Office Agency
Confidence Model Confidence model based upon indicators from MRA and comparables Related actual errors to the dispersion amongst the comparables Comparable sales approach provides a number of other indicators for
confidences: comparability distance weighted estimate (average adjusted) vs. MRA estimate the overall COV for the model on the sales set which tells us something
about the underlying uniformity in the market and sales base MRA modelling includes a control model which considers current CT band So Confidence Model is:
Likely Error = A+ B dispersion of comparables+ C x average distance between comparables+ D x absolute value of (ln (mkt est. / control model est.) ) + E x model standard error+ F x absolute value of (ln (weighted est. / MRA est.) )
Valuation Office Agency
Confidence Model Maintenance
Re-calibrated with every calibration iteration Periodically calibrated to valuer judgement of
estimate output Can be used individually or aggregated for
decisions at varying levels Developed between VOA, John Thompson
(Cole Layer Trumble), Dr Jim Abbott (EDS) Presented at IAAO CAMA/GIS conference
February 2006
Valuation Office Agency
Summary of Models (1)
Multiplicative MRA and comparable selection Operational delivery Undertaken locally Specified centrally
Broad Based Model Market Trends Linearisations e.g. Group and Type variables to
allow wider comparability Develop constraints for local models Quality assurance on operational delivery
Valuation Office Agency
Summary of Models (2)
Control Model Operates in background of local model Highlights data irregularities
Confidence Model Uses data from all models Calibrated to valuer opinion Provides information for use at all levels
Valuation Office Agency
Managing Business Change
Addressing the time lag Review whole data process Use electronic transfer to bring in sales
data Consider how to engage with taxpayer
New skills Control modelling and train professionals Develop support and maintenance staff
Rigorous application of project management methodology
Valuation Office Agency
IT Development
VOA has successfully integrated AVM technology with existing IT; this required: Improving data flow and management Enhanced mapping tools “Customising” 3rd party “off-the-shelf” product Introducing workflow to manage delivery
Lead-in time has created tensions IT supplier’s understanding of business and
new technology does not happen overnight!
Valuation Office Agency
Conclusions/Lessons Learned (1)
Other data sources create issues: Often aggregated Currency – frequency of update Niche provider with limited depth Definition – what creates the data Cost – Is it worth it to you? Your records may be adequate and better
than elsewhere anyway!
Valuation Office Agency
Conclusions/Lessons Learned (2)
How do you measure data quality Sale data quality
Mostly measured by model Dependent upon proper sales review process
Subject data quality Adoption of external standards not always
possible e.g. IAAO 6 year cycle Data sampling using existing maintenance
activities can support the view
Valuation Office Agency
Conclusions/Lessons Learned (3)
Time lag Understand the delivery requirement Address steps in the data process Work closer with data “partners” Central modelling can identify areas of
concern Proximity to data
You cannot easily replace local knowledge of market and meaning of sales.
Valuation Office Agency
Conclusions/Lessons Learned (4)
Modelling Define “attributes” with knowledge of AVM
techniques Balance modellers desire for more data of
imperfect market and cost to complete/maintain
Stabilise data prior to commencement of modelling
Valuing to “Band” does not loosen the rigour of modelling and data management
Valuation Office Agency
Conclusions/Lessons Learned (5)
Use AVM to direct effort Raw data analysis COD & COV are not the only measures “Frequently used comparables” Thematic mapping
Use formal modelling hierarchy Don’t rush to deliver. Inefficiencies result
Stabilise Model approach to ensure consistency Data stability and consistency
Valuation Office Agency
Conclusions/Lessons Learned (6)
Create clear lines of communication Local Management Local technical/modelling
Training Make timely Consider delivery mechanism (e-learning,
workshop) Project Management Structure A quality model can be let down by a poorly
defined and delivered process!
Valuation Office Agency
Some VOA achievements
Digitisation of over 22 Million records Data completeness approaching 100% Over 120 surveyors trained in AVM techniques Market analysis nationwide, including mapping
localities across the whole country Model performance well within recognised
standards e.g. median COD of 353 BAs is 9.97 Proven IT platform for mass appraisal at
national level.
Valuation Office Agency
Questions?
Tim Eden BSc MRICS IRRVDeputy Director of Council [email protected]
Valuation Office Agency
Achieving World Class