Several studies have examined the issue of racial bias in Automated Valuation Models (AVMs) by analyzing various measurements of AVM valuation errors in relation to the sales price. One such study conducted by the Urban Institute utilized three measurements to determine the difference between the sales prices and AVM values:
- The average difference to capture overvaluations or under valuations.
- The absolute difference captures the magnitude of inaccuracy.
- The percentage magnitude of inaccuracy, which is the magnitude of inaccuracy divided by the sales price.
The study found that AVM inaccuracy in majority-Black neighborhoods is 3.1 percentage points greater than in white neighborhoods, even after controlling for home values, neighborhood characteristics, and turnover rate. Another study conducted by HouseCanary examined the relationship between the racial composition of a census tract and the likelihood of a negative valuation gap. They found no evidence of racial bias in their automated comp and AVM tools.
Veros conducted a study in five major U.S. cities to assess how its AVM performed in ZIP codes with different racial compositions. The study analyzed several measurements, including the proportion of properties the AVM undervalues and overvalues by more than 10% or 15% and the size of the AVM’s Median Absolute Error (MAE). The study found no evidence of a positive correlation between under valuations of the AVM and the proportions of minority populations in a neighborhood. Similarly, overvaluations of the AVM were not positively correlated with the proportion of the white population. The AVM was found to be no less accurate in minority neighborhoods, based on multivariate regression analysis.
- The studies have used one or more of the following methods to test their models for racial bias:
Proportions of vastly over and under valuations of errors. - Aggregated mean or median errors to show the differences in the center of the distribution of errors with respect to racial composition.
- Absolute median errors or the variance of the errors.
These methods enable the computation of model errors and test the correlations of these errors with the proportion of minority households.
