# AVM Testing and Evaluation using AVM Performance Metrics

> Kennedy, Ecker and Isakson measure the accuracy of AVMs and their FSDs, demonstrating that most of the time FSDs underreport forecast error.

- Source: https://avm.digisavvy.dev/blog/avmtestingandevaluation/
- Author: AVMetrics
- Published: August 21, 2024
- Category: Whitepapers

---

Lee Kennedy, Mark Ecker, & Hans Isakson (Winter 2022) AVM Testing and Evaluation using AVM Performance Metrics, Real Estate Finance, 38:3, 210-249

**Abstract**

Currently-used industry procedures independently test the credibility of AVM valuations, via their AVM Performance Metrics, calibrated using actual sales data. In particular, the vendor-reported Forecast Standard Deviation (“FSD”), self-generated by an AVM provider, together with the valuation for a target property, is empirically corroborated using 455,563 housing sales, each valued by as many as 15 AVMs in an AVM-by-FSD analysis. The AVM-by-FSD analysis reveals that AVM providers are empirically underreporting their vendor-reported FSDs for 267 (72.8%) of the 367 AVM/FSD combinations. Furthermore, a composite index of any collection of AVM Performance Metrics, called the AVM Error Score (“AVM-ES”), is introduced to compare, rank and grade competing AVMs, at any desired geography and sampling unit. The AVM-ES is illustrated using two sets of housing sales data. The first expands the AVM-by-FSD analysis by using eleven AVM Performance Metrics (not just the FSD) to rank each of the 367 AVM/FSD combinations. The second analysis compares the performance of twelve AVMs for a common set of 1,193 sales in Clark County, Nevada using ten (mostly different) AVM Performance Metrics. Lastly, given a set of AVM performance thresholds, the AVM-ES can assign a collective ordinal grade (e.g. ‘Strong’, ‘Reasonable’, ‘Acceptable but Weak’ or ‘Poorly’ performing) to classify overall AVM performance.

***Keywords*****:** Automated Valuation Models, AVM-by-FSD Analysis, AVM Performance Metrics, Failure Magnitude, Failure MAPE, Failure Rate, FSD, Percentage Sales Error, Principal  
Component Analysis

###### [https://www.researchgate.net/publication/351634353_AVM_Testing_and_Evaluating_Using_AVM_Performance_Metrics_-_May_14_2021-1](https://www.researchgate.net/publication/351634353_AVM_Testing_and_Evaluating_Using_AVM_Performance_Metrics_-_May_14_2021-1)