Why Is RESNET Limiting Its Options for Improvement?

In case you haven't heard, last year RESNET created a task force to look at ways to improve the consistency of results among certified home energy raters (aka HERS raters). In mid-August, executive director Steve Baden sent an email about the latest developments in the process. He also attached a document titled Quality Improvement Working Group RESNET QA Options (pdf) that outlines the areas of consensus and gives three options for changing the way quality assurance (QA) is done. Unfortunately, there's a glaring omission from the list.

The need for improvement exists

We've been a HERS rating provider, now called HERS QA provider, since 2009. We serve independent raters from Arizona to Maine, and we make sure our raters know what they're doing and do things right. Because we don't let them get away with not meeting the HERS Standards or ENERGY STAR new homes program guidelines, we hear their complaints about raters with other providers who aren't such sticklers. Here's one I got this summer:

It's getting more and more frustrating on our end and is to the point where [one local HVAC company] is not wanting to bid on projects with a builder that has [our company] as the raters. Meanwhile, [that HVAC company] is pushing paperwork / installation issues through other raters that are apparently not having any issues getting Energy Star.

Yes, there are problems. Not all are egregious violations. Part of what's motivating RESNET is that if you hire several raters to come into a home to do a rating, you'll get a spread of results that's broader than it should be. Apparently, some of the big production builders that build homes in different markets have experienced this when they build the same home in a similar climate and get different results when different raters rate the home.

When a provider does QA on their raters, the results for the HERS Index are supposed to be within 3%, but different providers and QA Designees (QADs) don't always interpret the standards the same way. Each one has their ways of doing things and might emphasize or de-emphasize certain aspects of the rating process. So you can have variance in the results even without malfeasance.

RESNET's oversight of providers has improved but...

The HERS rating industry is set up to have QA of raters done by their providers. Providers are accredited by RESNET, which oversees and does QA on the providers. Energy Vanguard Energy Ratings has to submit an annual report to result documenting the results of the QA we do. In our first three years as a provider, however, RESNET wasn't doing a lot of oversight. One of those years, we never even got a report back from RESNET about how we did with our annual report. Also, despite RESNET having guidelines calling for enhanced oversight or field visits to providers, we haven't had any of that. (We're getting our first online review this fall.)

That's definitely changed, though. Laurel Elam took over as QA Manager for RESNET, and she's on the ball. In the past two years, she's not only looking carefully at the reports submitted by all providers, she's acting on what she finds. Here's what a recent email from RESNET announced about the process:

The reviews found numerous instances where providers fell short of complying with RESNET quality assurance provisions. As a result 43 Rating Quality Assurance Providers were placed on administrative probation. An additional 56 Providers had findings that needed correction to be compliant with RESNET Standards.

That's great! I'm happy to see that RESNET is starting to hold providers to the requirements in the HERS Standards.

But, it's still not enough. Laurel is doing a wonderful job and RESNET is lucky to have her, but she's not a technical person. She's never been a HERS rater and isn't a QAD. I don't think the QA Manager needs to have those qualifications, but RESNET has to have technical people involved in the oversight process. They do have at least one technical person, Abe Kruger, who assists on a contract basis, but he's not fulltime.

What RESNET has been doing over the past two years since Laurel took over is the important step of making sure the t's are crossed and the i's dotted. What RESNET has not been doing is looking at the words and seeing if there should even be a t or an i in them or maybe if they should have used a different word altogether.

In other words, RESNET needs to dive in deep enough to determine, not only if providers are doing enough QA and keeping records properly, but also if their QA process is good enough to assure the quality of ratings. A big step here would be to repeat some of the rating data file QA done by providers on raters.

Providers are required to do this for 10% of all ratings done by a rater, so RESNET, it seems to me, should be doing this for some percentage of every provider's file QA. RESNET wouldn't have to travel for this, and I think it would turn up a lot of problems and force all providers to make sure they do it right.

The 3 options

Back to the Quality Improvement Working Group now. This process has been going on since last year, and they recently released a document outlining the issues and their options for improvement (pdf). Here they are:

  1. RESNET staff do all QA on raters and ratings.
  2. RESNET creates a pool of contractors to do all QA on raters and ratings.
  3. RESNET contracts with a pool of third-party regional QA contractors to do all QA on raters and ratings.

I can't see RESNET going with number 1. They probably won't want to do the work involved with number 2 either. I think, if they continue down this path, they'll probably go with number 3, and we'll have ICF, CSG, and ClearResult doing all the QA.

All three of these options take QA away from the current providers. The task force and working group have said they want all QADs to act as "agents of RESNET." Providers will still exist but will not do QA anymore.

It's an important document for the future of the HERS rating industry. Please read it, think about it, discuss it, and participate in the public comment process. Unfortunately, the deadline for public comment was at the end of August. RESNET gave an unusually short 10 days for comments, whereas they normally allow 30 days.

The glaring omission

Let me state again that this process is necessary. RESNET definitely needs to do something improve the consistency and quality of the HERS ratings being done. The list of consensus items in their recent document has some good points. For example:

"QA contractors should not be performing QA on Raters and ratings where a conflict of interest exists."

"The RESNET Home Energy Rating and QA processes must be more clearly defined to achieve greater consistency."

"Qualifications of the companies and/or individuals who provide QA services, including accountability, must be more stringent and more clearly defined to create greater consistency and quality in how QA is delivered."

Unfortunately, they seem to think that because the majority of providers also do ratings, no providers should be doing QA. I beg to differ. There are a few of us who are providers but do little or no ratings. Energy Vanguard does a handful of ratings per year, nearly all for Habitat for Humanity, many of them pro bono. We made the decision early on not to compete against our raters. Building Efficiency Resources (The BER) is another company with this model, and so is Southface.

Why is RESNET ignoring the benefits of this model? Rather than including only options that take QA away from providers, why not have a fourth option that allows providers to do QA but not on any ratings they do themselves? This is the fox-in-the-henhouse issue people have been talking about for years because of all the big providers that do their own QA.

This model can work, but it requires RESNET to take the next step I described above and look not only at providers' administrative compliance but also technical compliance. As a member of the recently disbanded RESNET QA committee (replaced by the new QA committee tasked with creating an ANSI standard), I've been saying this over and over. The current structure can work but RESNET needs to do more thorough QA on providers.

What happens when QA is taken away from providers?

If RESNET continues down the current path and chooses one of the three options above (probably the third), why would anyone want to be a provider? The big rating companies will probably continue, but there's not much in it for small providers like ours. We'll become document chasers and discipline enforcers, and I have no interest in that.

If RESNET chooses one of the three options above, I think QA could actually get worse for independent raters. Many providers do more than the minimum QA. In our providership, we look at nearly every REM/Rate file that gets sent to us, not just the 10% required by RESNET. Will the new QA companies do that?

And providers won't know as much about QA if they're not doing it. When raters call them up to ask questions, providers won't know as many of the answers. They may even steer raters in the wrong direction because the QA companies may do things differently.

The collapse of QA

Another problem with the three options above is that it could lead to the collapse of the QA process from three levels to two. Right now it's RESNET => providers => raters. It seems like RESNET thinks that handing all the QA off to "independent" QA companies (or RESNET staff) will solve all the problems so the collapse won't matter. I think they're wrong. Let's keep three levels and beef up RESNET's QA, which will reverberate all the way down.

I don't think the folks on the task force and the working group have fully considered all the options or the consequences.

Two changes that could make all the difference

The following two changes are really all RESNET needs to do to improve the QA of HERS ratings:

  • Providers should not be allowed to do QA on their own ratings.
  • RESNET should do more technical QA on providers.

The latter is what I was talking about above when I said RESNET should redo some percentage of each provider's rating data file QA. Combined with the great job Laurel Elam has been doing with administrative QA, this could would allow RESNET to discover problems that might otherwise never be found.

Let's get it right

If you're a home energy rater, QAD, or provider, please get involved in this process. Read the documents. Check out the resources I've listed below. And most of all, let your voice be heard.

External Resources

RESNET quality improvement process

The BER's webinar on an alternative path

The BER's white paper on quality improvement

Texas HERO's response to RESNET's quality improvement process

Views: 171

Tags: HERS, RESNET, assurance, energy, home, quality, rater, rating


You need to be a member of Home Energy Pros Forum to add comments!

Join Home Energy Pros Forum

Latest Activity

Chris Stratton commented on Jim Gunshinan's blog post Our First Year with Solar Panels, by Chris Stratton
"@ Jonathan 1. Cost of PV array was about $15k before 30% federal tax credit. 2. The financial…"
1 hour ago
Chris Stratton commented on Jim Gunshinan's blog post Our First Year with Solar Panels, by Chris Stratton
"@ Brennan Hey thanks! Yeah, it's time for an update on the energy consumption now that…"
1 hour ago
Chris Stratton commented on Jim Gunshinan's blog post Our First Year with Solar Panels, by Chris Stratton
"@Grey re: estimated vs. actual production Good question! The estimated first-year annual production…"
1 hour ago
Diane Chojnowski's video was featured

People Who Save Energy 1: Builders

Millions of people work to save energy in the United States. They work in every state, across industries and technologies. To show the diverse arc of this growing workforce, we visited North Carolina. This video tells the stories of some of these…
3 hours ago
Jonathan Nadle commented on Jim Gunshinan's blog post Our First Year with Solar Panels, by Chris Stratton
"Hi Jim, Thanks for your well thought-out and written piece.  I agree with your comments on…"
20 hours ago
Leo Klisch commented on Jim Gunshinan's blog post Our First Year with Solar Panels, by Chris Stratton
"WOW! 913kwh/12= 76kwh/month. How is that possible? I live in central Minnesota and started with…"
20 hours ago
Leslie Jackson replied to Leslie Jackson's discussion DIY Filter Solution for an Evaporative Cooler, anyone?
"Thanks Rich, I hope they thought of that, but you are on to something. I think I learn more doing…"
23 hours ago
grey staples commented on Jim Gunshinan's blog post Our First Year with Solar Panels, by Chris Stratton
"Curious to know how the kWh production turned out relative to what the installer indicated you…"
Brennan Less commented on Jim Gunshinan's blog post Our First Year with Solar Panels, by Chris Stratton
"Hey Chris, y'all are kicking butt!! I'm sure your new HVAC won't use any energy,…"
Rich Manning replied to Leslie Jackson's discussion DIY Filter Solution for an Evaporative Cooler, anyone?
"Have them call a HVAC contractor and install a filter system"
Rich Manning replied to Jerry Needham's discussion Condensation on ceiling and walls of master bedroom
"Looks like a leaking ridge vent"
Jonathan Nadle liked Jim Gunshinan's blog post Our First Year with Solar Panels, by Chris Stratton
Matt Peterson posted a blog post

Gas Line Installation In Portland OR Code Basics

When you're looking to undertake any kind of a remodel or repair in your home it's important to…See More
Brennan Less liked Jim Gunshinan's blog post Our First Year with Solar Panels, by Chris Stratton
Pearl Home Certification posted a blog post

Press Release: HomeSmart International Joins the Pearl Broker Advantage Program

Vienna, Virginia and Scottsdale, Arizona – July 19, 2018 – Pearl Certification and HomeSmart…See More
Franco Oyuela commented on Allison A. Bailes III's blog post The Ductopus or the Centipede - Which HVAC Duct System Is Better?
"Many older homes were constructed without air ducts. Ductless systems can provide much needed air…"
Franco Oyuela commented on Chris Laumer-Giddens's blog post The Ductless Mini-Split Heat Pump's Big Brother - CITY MULTI
"The compact design and long piping system gives homeowners free reign for designing a ductless…"
Terry Hill's event was featured

Passive House Institute US 2018 Conference at Boston

September 19, 2018 at 6pm to September 23, 2018 at 7pm
Annual conferenceSee More
Franco Oyuela commented on Allison A. Bailes III's blog post How the Heck Does a Heat Pump Get Heat from Cold?!
"Like conventional central heat pumps, ductless mini-split heat pumps manipulate refrigerant to move…"
Leslie Jackson replied to Leslie Jackson's discussion DIY Filter Solution for an Evaporative Cooler, anyone?
"Thanks, Larry! You are probably not confused! Apply skin, not a Band-Aid ... or something like…"


  • Add Photos
  • View All

© 2018   Created by Home Performance Coalition (HPC)   Powered by

Badges  |  Report an Issue  |  Terms of Service