The AVMetrics Blog

Appraisal Foundation IAC Open Letter to Congress

IAC Opinion Letter to Congress with IAC Listing

On September 13, 2017, AVMetrics participated with its many partners in The Appraisal Foundation in sending an open letter to the House Committee on Banking, Housing and Urban Affairs urging the committee to refrain from weakening the oversight of the Appraisal Subcommittee (ASC).   The letter – linked to above – emphasizes the importance of the ASC’s National Registry and the benefits of consistency brought about by ASC’s oversight.

The letter proposes several enhancements to the ASC system, including improvements in background checks for appraisers and a standard definition of a “Federally Related Transaction.”   Other proposed enhancements detailed in the letter are designed to improve consistency and transparency.

Lee Kennedy Speaks at Appraisal Institute Symposium

On Monday, June 26, 2017 the Appraisal Institute’s Northern California Chapter hosted an educational seminar in Oakland, CA, with Lee Kennedy as an invited expert. The panel was moderated by Paul E. Chandler, MAI, and Lee’s co-panelists were Michael Simmons of AXIS Appraisal Management and Todd Krell of CrossCheck Compliance. The topic of this educational session was Third Party Vendors, Tools and Compliance: The Role of AMCs and AVMs in the Appraisal Process.  The panel was part of the Commercial and Residential Symposium entitled The Role of Valuation Experts in the Current Regulatory Environment and provided both state (7hrs) and Appraisal Institute Continuing Education.

Synopsis:

Appraiser and Appraisal Management is at the heart of quality loan production. How does a lender know it is using competent appraisers providing quality reports? Policies for the empanelment of appraisers, procurement of vendors and review and quality control of assignments must be documented, managed and audited. An expanding selection of data and technology options are available to lenders to manage all aspects of collateral due diligence. Tools include fully integrated loan origination systems, appraisal management platforms and robust data and review tools. This session will review how to find, screen and manage different third party providers, including AMCS, AVM sellers and QA compliance firms.

  • AVM Cascades and Their Applicable Uses
  • Commercially Available AVMs
  • Big Data Solutions
  • AMC Onboarding Check List
  • Third Party Oversight
  • AMC Auditing
  • Quality Assurance Testing
  • Performance Monitoring

Cascade vs Model Preference Table® – What’s the Difference?

In the AVM world, there is a bit of confusion about what exactly is a “cascade.” It’s time to clear that up.  Over the years, the terms “cascade” and “Model Preference Table®” have been used interchangeably, but at AVMetrics, we draw an important distinction that the industry would do well to adopt as a standard.

In the beginning, as AVM users contemplated which of several available models to use, they hit on the idea of starting with the preferred model, and if it failed to return a result, trying a second model, and then a third, etc.  This rather obvious sequential logic required a ranking, which was available from testing, and was designed to avoid “value shopping.”[1]  More sophisticated users ranked AVMs across many different niches, starting with geographical regions, typically counties.  Using a table, models were ranked across all regions, providing the necessary tool to allow a progression from primary AVM to secondary AVM and so on.

We use the term “Model Preference Table” for this straightforward ranking of AVMs, which can actually be fairly sophisticated if they are ranked within niches that include geography, property type and price range.

More sophisticated users realized that just because a model returned a value does not mean that they should use it.  Models typically deliver some form of confidence in the estimate, either in the form of a confidence score, reliability grade, a “forecasted standard deviation” (FSD) or similar measure derived through testing processes.  Based on these self-measuring outputs from the model, an AVM result can be accepted or rejected (based on testing results) in favor of the next AVM in the Model Preference Table.  This application reflects the merger of MPT rankings with decision logic, which in our terminology makes it a “cascade.”

CriteriaAVMMPT®Cascade“Custom” Cascade
Value EstimateXXXX
AVM RankingXXX
Logic + RankingXX
Risk Tolerance + Logic + RankingX

 

The final nuance is between a simple cascade and a “custom” cascade.  The former simply sets across-the-board risk/confidence limits and rejects value estimates when they fail to meet the standard.  For example, the builder of a simple cascade could choose to reject any value estimate with an FSD > 25%.  A “custom cascade” integrates the risk tolerances of the organization into the decision logic.  That might include lower FSD limits in certain regions or above certain property values, or it might reflect changing appetites for risk based on the application, e.g., HELOC lending decisions vs portfolio marketing applications.

We think that these terms represent significant differences that shouldn’t be ignored or conflated when discussing the application of AVMs.

 

Lee Kennedy, principal and founder of AVMetrics in 2005, has specialized in collateral valuation, AVM testing and related regulation for over three decades.  Over the years, AVMetrics has guided companies through regulatory challenges, helped them meet their AVM validation requirements, and commented on pending regulations. Lee is an author, speaker and expert witness on the testing and use of AVMs. Lee’s conviction is that independent, rigorous validation is the healthiest way to ensure that models serve their business purposes.

[1] OCC 2005-22 (and the 2010 Interagency Appraisal and Evaluation Guidelines) warn against “value shopping” by advising, “If several different valuation tools or AVMs are used for the same property, the institution should adhere to a policy for selecting the most reliable method, rather than the highest value.”

Allison MacDonald Recognized by CSUCI as 2017 Distinguished Alumna

We were proud to read the following in CSUCI’s newsletter:

Many CSUCI alumni have found tremendous success over the years. But recognition comes for those who take that success and use it to help others. This year one very compassionate and talented alumna demonstrated an unwavering commitment to the cause. Please help me congratulate your 2017 Distinguished Alumni Awardee, Allison MacDonald ’07. Allison was an active student while here at CI and took advantage of all we had to offer. After graduation, her drive and hard work led her to career success. Almost immediately and unsolicited she came to us asking to give back. Her dedication to CI and its students in time, talent and treasure were unmatched. Her true dolphin spirit embodies the CI Way. Congratulations Allison!

Sincerely,

Tania R. Garcia

Director of Development, Alumni Relations

Read more about Allison here.

AVMetrics is proud to have Allison as a key member of the team.  Many congratulations, Allison, for all you do for giving back to the CSUCI community and for all you do here at AVMetrics.

How AVMetrics Tests AVMs

Testing an AVM’s accuracy can actually be quite tricky.  It is easy to get an AVM estimate of value, and you can certainly accept that a fair sale on the open market is the benchmark against which to compare the AVM estimate, but that is really just the starting point.

There are four keys to fair and effective AVM testing, and applying all four can be challenging for many organizations.

  1. Your raw data must be cleaned up, to ensure that there aren’t any “unusable” or “discrepant” characters in the data; differences such as “No.” “#” and “Num,” must be normalized.
  2. Once your test data is “scrubbed clean” it must be assembled in a universal format and it must be large enough to provide reliable test results, even down to the segment level for each property type within each price level within each county, etc. and this might require hundreds of thousands of records. 
  3. Timing must be managed so that each model receives the same sample data at the same time with the same response deadline.
  4. Last, and most difficult, the benchmark sales data must not be available to the models being tested.  In other words, if the model has access to the very recent sales price, it will be able to provide a near-perfect estimate by simply estimating that the value hasn’t changed (or changed very little) in the days or weeks since the sale. 

AVMetrics tests every commercially available AVM continuously and aggregates this testing into a report quarterly; AVMetrics’ testing process meets these criteria and many more, providing a truly objective measure of AVM performance. 

The process starts with the identification of an appropriate sample of properties for which benchmark values have very recently been established.  These are the actual sales prices for arm’s-length transactions between willing buyers and sellers—the best and most reliable indicator of market value.  To properly conduct a “blind” test, these benchmark values must be unavailable or “unknown” to the vendors testing their model(s).  AVMetrics provides in excess of a half million test records annually to AVM vendors (without information as to their benchmark values).  The AVM vendors receive the records simultaneously, run these properties through their model(s) and return the predicted value of each property within 48 hours, along with a number of other model-specific outputs.  These outputs are received by AVMetrics, where the results are evaluated against the benchmark values.  A number of controls are used to ensure fairness, including the following:

  • ensuring that each AVM vendor receives the exact same property list (so no model has any advantage)
  • ensuring that each AVM is given the exact same parameters (since many allow input parameters that can affect the final valuation)
  • ensuring through multiple checks that no model had access the recent sale data, which would provide an unfair advantage

In addition to quantitative testing, AVMetrics circulates a comprehensive vendor questionnaire twice annually.  Vendors that wish to participate in the testing process complete, for each model being tested, roughly 100 parameter, data, methodology, staffing and internal testing questions.  These enable AVMetrics, and more importantly our clients, to understand model differences within both testing and production contexts, and it enables us and our clients to satisfy certain regulatory requirements describing the evaluation and selection of models (see OCC 2010-42).

AVMetrics next performs a variety of statistical analyses on the results, breaking down each individual market, each price range, and each property type, and develops results which characterize each model’s success in terms of precision, usability and accuracy.  AVMetrics analyzes trends at the global, market and individual model levels, identifying where there are strengths and weaknesses, and improvements or declines in performance.

The last step in the process is for AVMetrics to provide an anonymized comprehensive comparative analysis for each model vendor, showing where their models stack up against all of the models in the test; this invaluable information facilitates the continuous improvement of each vendor’s model offerings.

AVMNews coming to AVMetrics

For the last 15 years, we have all been indebted to Perry Minas for running AVMNews.  He’s curated the AVM industry’s best articles and shared them via his email newsletter to hundreds of industry insiders.  This month Perry is retiring, and November 2016 will be his last issue of AVMNews – at least his last as a solo performer.  Picking up in 2017, AVMetrics will take over the heavy lifting of compiling and distributing AVMNews, but Perry will stay involved, overseeing his creation.

AVMetrics is proud to continue the tradition of AVMNews, and we plan to take this opportunity to ask readers for their suggestions.  Some early suggestions are more articles on

  • Industry trends
  • Technical articles
  • Interviews with AVM builders
  • Appraisers use of AVMs
  • Modeling trends
  • How customers are using AVMs.

We welcome your comments and suggestions.  Anyone who would like to receive the compilation of articles six times a year can subscribe to AVMNews for free via the following link: AVMNews Subscription.

This newsletter is intended to be a compilation of interesting and noteworthy articles, news items and press releases that are relevant to the AVM industry.  The opinions and positions presented herein are those of the authors and are not necessarily shared by AVMetrics, its employees, principles or agents. Unless specifically stated otherwise within an article or advertisement, AVMetrics does not endorse any opinions, positions, products or services published in AVMNews.

Raising the De Minimis Threshold – Fear Not!

Background

There is a lot of controversy about appraisals and Appraisers these days, and the FFIEC proposed rule change – increasing the de minimis threshold to $500,000 – allowing for an appraisal exemption and the use of an evaluation in lieu of an appraisal – has sparked anxiety in the world of collateral risk.  Our colleagues at the Collateral Risk Network (CRN) expressed their opposition to the proposal. Not surprisingly for a group of its size, there are diverse opinions at the individual membership level of the group.  Our opinion is that the change – far from being the catastrophe imagined – will in fact have some important benefits.

A Place for De Minimis

While the CRN and certain appraiser blogs expressed skepticism – to put it mildly – we believe that there is a place for an appropriate de minimis level, even the $500,000 level now being considered.  On low risk transactions, evaluations (as opposed to full appraisals) can be appropriate and even beneficial for risk management of the overall lending system.

Here’s why.  Lending volumes tend to scale up and down faster than the supply of appraisers.  As a result, boom cycles in the lending business can place extreme pressure on appraisers.  This scenario makes quality control extremely challenging.  The option to leverage efficient evaluations on low risk transactions can improve the risk management of the entire system by devoting limited appraisal resources to their highest and best use.  In other words, when you place strain on a system, something has to give, and raising the de minimis threshold enables lenders to focus scarce resources on the riskier transactions.

Evaluations and the Credit Crisis

The CRN expressed concern about allowing the mistakes of the recent Credit Crisis to be repeated, and we could not be in more agreement.  However, their letter insinuated that evaluations (specifically BPOs and AVMs) were to blame for inflated valuations.  Of the vast number and type of quality problems experienced during the credit crisis, evaluations were not a major contributing factor.  In fact, we are not aware of any reported cases of AVMs being blamed for the quality problems experienced during the credit crisis.

Appraisals as a Source of Market Analysis

Strangely, the CRN comments suggested that reviewing individual appraisals is an important source of market trend analysis for investors during overheated markets.  We find this highly improbable.  The typical single-family appraisal may contain microanalysis of neighborhoods or small markets that lenders may find informative, but most Investors already access market and economic trend data via other sources, including their own or 3rd party economic analyses and risk management tools.

Existing Quality Control Infrastructure for Appraisals

The CRN letter makes the case that appraisals benefit from an extensive regulatory framework and quality control infrastructure surrounding their use, making them inherently safer for the industry to rely upon.  We note that much of the same quality control infrastructure and practices were in place before the last crisis.  Much of that appraisal quality control depends on the same people and practices – e.g., “desk appraisals” performed by other appraisers – making them subject to similar risk factors.  In other words, appraisals are not a guarantee against risk.  They are simply one tool in the toolbox – an effective and comparatively expensive tool – but they should not be an exclusive tool when evaluations can be effectively employed in lower-risk scenarios. .

Application of Evaluations

We believe in using the right tool for the job, and we believe that there is a place for evaluations in prudent lending practices. Relying on additional risk measurements, rather than just focusing on a one size fits all de minimis level can provide a formula for better risk management.  For example: A $350,000 transaction at a 40% LTV for a pay stub borrower has less need for an appraisal; an evaluation might be able to suffice.  Better to allocate that valuable appraisal resource to a $225,000 transaction at 90% LTV.  Raising the de minimis, while providing additional guidance for other measures, provides lenders and investors more flexibility to make smarter risk management decisions, and it releases valuable appraisal resources to be used where they can have the most benefit.

Now that the FFIEC has recently closed its commentary period regarding the proposed de minimis lending threshold of $500,000, we expect to receive final communication from the FFIEC during 2016.
We anticipate that lenders will adapt to the new regulations incrementally, with quality controls designed for the new thresholds, not discarded with the bathwater.

Lee Kennedy & Mike Coyne,

AVMetrics, LLC.

Being Part of the CSUCI Community Means Being Active

Our local community is a team, together we effect the most positive of changes! Many thanks to all those who attended the CI Alumni & Friends Happy Hour at the Hangar event (Thursday, 6/16). We raised over $12,000 for student scholarships and it was a blast doing it. AVMetrics is proud to be major sponsor and an active part of the CSUCI community.