Lee Kennedy, Mark Ecker, & Hans Isakson (Winter 2022) AVM Testing and Evaluation using AVM Performance Metrics, Real Estate Finance, 38:3, 210-249
Abstract
Currently-used industry procedures independently test the credibility of AVM valuations, via their AVM Performance Metrics, calibrated using actual sales data. In particular, the vendor-reported Forecast Standard Deviation (“FSD”), self-generated by an AVM provider, together with the valuation for a target property, is empirically corroborated using 455,563 housing sales, each valued by as many as 15 AVMs in an AVM-by-FSD analysis. The AVM-by-FSD analysis reveals that AVM providers are empirically underreporting their vendor-reported FSDs for 267 (72.8%) of the 367 AVM/FSD combinations. Furthermore, a composite index of any collection of AVM Performance Metrics, called the AVM Error Score (“AVM-ES”), is introduced to compare, rank and grade competing AVMs, at any desired geography and sampling unit. The AVM-ES is illustrated using two sets of housing sales data. The first expands the AVM-by-FSD analysis by using eleven AVM Performance Metrics (not just the FSD) to rank each of the 367 AVM/FSD combinations. The second analysis compares the performance of twelve AVMs for a common set of 1,193 sales in Clark County, Nevada using ten (mostly different) AVM Performance Metrics. Lastly, given a set of AVM performance thresholds, the AVM-ES can assign a collective ordinal grade (e.g. ‘Strong’, ‘Reasonable’, ‘Acceptable but Weak’ or ‘Poorly’ performing) to classify overall AVM performance.
Keywords: Automated Valuation Models, AVM-by-FSD Analysis, AVM Performance Metrics, Failure Magnitude, Failure MAPE, Failure Rate, FSD, Percentage Sales Error, Principal
Component Analysis