Why Is RESNET Limiting Its Options for Improvement?

In case you haven't heard, last year RESNET created a task force to look at ways to improve the consistency of results among certified home energy raters (aka HERS raters). In mid-August, executive director Steve Baden sent an email about the latest developments in the process. He also attached a document titled Quality Improvement Working Group RESNET QA Options (pdf) that outlines the areas of consensus and gives three options for changing the way quality assurance (QA) is done. Unfortunately, there's a glaring omission from the list.

The need for improvement exists

We've been a HERS rating provider, now called HERS QA provider, since 2009. We serve independent raters from Arizona to Maine, and we make sure our raters know what they're doing and do things right. Because we don't let them get away with not meeting the HERS Standards or ENERGY STAR new homes program guidelines, we hear their complaints about raters with other providers who aren't such sticklers. Here's one I got this summer:

It's getting more and more frustrating on our end and is to the point where [one local HVAC company] is not wanting to bid on projects with a builder that has [our company] as the raters. Meanwhile, [that HVAC company] is pushing paperwork / installation issues through other raters that are apparently not having any issues getting Energy Star.

Yes, there are problems. Not all are egregious violations. Part of what's motivating RESNET is that if you hire several raters to come into a home to do a rating, you'll get a spread of results that's broader than it should be. Apparently, some of the big production builders that build homes in different markets have experienced this when they build the same home in a similar climate and get different results when different raters rate the home.

When a provider does QA on their raters, the results for the HERS Index are supposed to be within 3%, but different providers and QA Designees (QADs) don't always interpret the standards the same way. Each one has their ways of doing things and might emphasize or de-emphasize certain aspects of the rating process. So you can have variance in the results even without malfeasance.

RESNET's oversight of providers has improved but...

The HERS rating industry is set up to have QA of raters done by their providers. Providers are accredited by RESNET, which oversees and does QA on the providers. Energy Vanguard Energy Ratings has to submit an annual report to result documenting the results of the QA we do. In our first three years as a provider, however, RESNET wasn't doing a lot of oversight. One of those years, we never even got a report back from RESNET about how we did with our annual report. Also, despite RESNET having guidelines calling for enhanced oversight or field visits to providers, we haven't had any of that. (We're getting our first online review this fall.)

That's definitely changed, though. Laurel Elam took over as QA Manager for RESNET, and she's on the ball. In the past two years, she's not only looking carefully at the reports submitted by all providers, she's acting on what she finds. Here's what a recent email from RESNET announced about the process:

The reviews found numerous instances where providers fell short of complying with RESNET quality assurance provisions. As a result 43 Rating Quality Assurance Providers were placed on administrative probation. An additional 56 Providers had findings that needed correction to be compliant with RESNET Standards.

That's great! I'm happy to see that RESNET is starting to hold providers to the requirements in the HERS Standards.

But, it's still not enough. Laurel is doing a wonderful job and RESNET is lucky to have her, but she's not a technical person. She's never been a HERS rater and isn't a QAD. I don't think the QA Manager needs to have those qualifications, but RESNET has to have technical people involved in the oversight process. They do have at least one technical person, Abe Kruger, who assists on a contract basis, but he's not fulltime.

What RESNET has been doing over the past two years since Laurel took over is the important step of making sure the t's are crossed and the i's dotted. What RESNET has not been doing is looking at the words and seeing if there should even be a t or an i in them or maybe if they should have used a different word altogether.

In other words, RESNET needs to dive in deep enough to determine, not only if providers are doing enough QA and keeping records properly, but also if their QA process is good enough to assure the quality of ratings. A big step here would be to repeat some of the rating data file QA done by providers on raters.

Providers are required to do this for 10% of all ratings done by a rater, so RESNET, it seems to me, should be doing this for some percentage of every provider's file QA. RESNET wouldn't have to travel for this, and I think it would turn up a lot of problems and force all providers to make sure they do it right.

The 3 options

Back to the Quality Improvement Working Group now. This process has been going on since last year, and they recently released a document outlining the issues and their options for improvement (pdf). Here they are:

  1. RESNET staff do all QA on raters and ratings.
  2. RESNET creates a pool of contractors to do all QA on raters and ratings.
  3. RESNET contracts with a pool of third-party regional QA contractors to do all QA on raters and ratings.

I can't see RESNET going with number 1. They probably won't want to do the work involved with number 2 either. I think, if they continue down this path, they'll probably go with number 3, and we'll have ICF, CSG, and ClearResult doing all the QA.

All three of these options take QA away from the current providers. The task force and working group have said they want all QADs to act as "agents of RESNET." Providers will still exist but will not do QA anymore.

It's an important document for the future of the HERS rating industry. Please read it, think about it, discuss it, and participate in the public comment process. Unfortunately, the deadline for public comment was at the end of August. RESNET gave an unusually short 10 days for comments, whereas they normally allow 30 days.

The glaring omission

Let me state again that this process is necessary. RESNET definitely needs to do something improve the consistency and quality of the HERS ratings being done. The list of consensus items in their recent document has some good points. For example:

"QA contractors should not be performing QA on Raters and ratings where a conflict of interest exists."

"The RESNET Home Energy Rating and QA processes must be more clearly defined to achieve greater consistency."

"Qualifications of the companies and/or individuals who provide QA services, including accountability, must be more stringent and more clearly defined to create greater consistency and quality in how QA is delivered."

Unfortunately, they seem to think that because the majority of providers also do ratings, no providers should be doing QA. I beg to differ. There are a few of us who are providers but do little or no ratings. Energy Vanguard does a handful of ratings per year, nearly all for Habitat for Humanity, many of them pro bono. We made the decision early on not to compete against our raters. Building Efficiency Resources (The BER) is another company with this model, and so is Southface.

Why is RESNET ignoring the benefits of this model? Rather than including only options that take QA away from providers, why not have a fourth option that allows providers to do QA but not on any ratings they do themselves? This is the fox-in-the-henhouse issue people have been talking about for years because of all the big providers that do their own QA.

This model can work, but it requires RESNET to take the next step I described above and look not only at providers' administrative compliance but also technical compliance. As a member of the recently disbanded RESNET QA committee (replaced by the new QA committee tasked with creating an ANSI standard), I've been saying this over and over. The current structure can work but RESNET needs to do more thorough QA on providers.

What happens when QA is taken away from providers?

If RESNET continues down the current path and chooses one of the three options above (probably the third), why would anyone want to be a provider? The big rating companies will probably continue, but there's not much in it for small providers like ours. We'll become document chasers and discipline enforcers, and I have no interest in that.

If RESNET chooses one of the three options above, I think QA could actually get worse for independent raters. Many providers do more than the minimum QA. In our providership, we look at nearly every REM/Rate file that gets sent to us, not just the 10% required by RESNET. Will the new QA companies do that?

And providers won't know as much about QA if they're not doing it. When raters call them up to ask questions, providers won't know as many of the answers. They may even steer raters in the wrong direction because the QA companies may do things differently.

The collapse of QA

Another problem with the three options above is that it could lead to the collapse of the QA process from three levels to two. Right now it's RESNET => providers => raters. It seems like RESNET thinks that handing all the QA off to "independent" QA companies (or RESNET staff) will solve all the problems so the collapse won't matter. I think they're wrong. Let's keep three levels and beef up RESNET's QA, which will reverberate all the way down.

I don't think the folks on the task force and the working group have fully considered all the options or the consequences.

Two changes that could make all the difference

The following two changes are really all RESNET needs to do to improve the QA of HERS ratings:

  • Providers should not be allowed to do QA on their own ratings.
  • RESNET should do more technical QA on providers.

The latter is what I was talking about above when I said RESNET should redo some percentage of each provider's rating data file QA. Combined with the great job Laurel Elam has been doing with administrative QA, this could would allow RESNET to discover problems that might otherwise never be found.

Let's get it right

If you're a home energy rater, QAD, or provider, please get involved in this process. Read the documents. Check out the resources I've listed below. And most of all, let your voice be heard.

External Resources

RESNET quality improvement process

The BER's webinar on an alternative path

The BER's white paper on quality improvement

Texas HERO's response to RESNET's quality improvement process

Views: 161

Tags: HERS, RESNET, assurance, energy, home, quality, rater, rating

Comment

You need to be a member of Home Energy Pros Forum to add comments!

Join Home Energy Pros Forum

Latest Activity

Brett Little posted events
yesterday
Ben Thompson replied to Stuart Perkin's discussion TrueFlow in the group Energy Auditing Equipment for Sale, Trade or to Purchase
"Hi Stephen. Do you still have a Trueflow that you would sell? Ben Thompson,…"
yesterday
Ben Thompson replied to Landon's discussion WANTED: Blower door complete set Retrotec or Minneapolis in the group Energy Auditing Equipment for Sale, Trade or to Purchase
"Hi Landon. I have a Minneapolis blower door set-up that I would like to sell. It's in…"
yesterday
Ben Thompson added a discussion to the group Energy Auditing Equipment for Sale, Trade or to Purchase
Thumbnail

Wanted: TrueFlow plate

Seeking an Energy Conservatory TrueFlow plate (grid) in good condition. Ben Thompson.…See More
Friday
Scott Partlo replied to Jerry Needham's discussion Closed cell on roof deck to prevent mold.
"You say that 2" of closed cell foam is applied OVER the roof deck.  Where exactly was the…"
Friday
ayazfudugo1 replied to Tom Smith's discussion Are Energy Efficient Windows worth the investment?
"For both new and existing homes, Energy efficient windows are an important consideration Heat…"
Friday
Profile IconTodd Williams and Haley Keegan joined Diane Chojnowski's group
Thumbnail

Hall of Shame

In this group, members share an array of images from the field, showing the kinds of issues…See More
Friday
Todd Williams liked Diane Chojnowski's group Hall of Shame
Friday
Home Energy Magazine posted a blog post
Thursday
Leona Zoey posted a blog post

Landlords Are Your Friends

While it may be easy to forget given their apparent position of power over you, landlords usually…See More
Thursday
Haley Keegan joined Building Performance Institute's group
Thumbnail

Building Performance Institute (BPI)

BPI is the nation's premier standards development, quality assurance and credentialing organization…See More
Thursday
Donald Hester replied to Jerry Needham's discussion Closed cell on roof deck to prevent mold.
"For the roof shingle issue that is an unfortunate issue with many home inspectors that have been…"
Thursday
Profile IconJason Green, Dennis Smith, Alex Ortega and 2 more joined Home Energy Pros Forum
Thursday
tedkidd replied to Quinn Korzeniecki's discussion BPI CEO Larry Zarker's Review of "The Home Comfort Book" in Healthy Indoors Magazine in the group Building Performance Institute (BPI)
"...free chapters of book available here - http://bit.ly/HPchapterlist"
Thursday
tedkidd liked Quinn Korzeniecki's discussion BPI CEO Larry Zarker's Review of "The Home Comfort Book" in Healthy Indoors Magazine
Thursday
William Collins's video was featured

Foam Kits vs. Spray Sealants Gun

How does the Foam Kit stack up to the Spray Sealants Gun when it comes to simplicity? When you integrate the Spray Sealants Gun to a 16lb Canister, you creat...
Thursday
Scott Becnel added a discussion to the group Energy Auditing Equipment for Sale, Trade or to Purchase
Thumbnail

Brand new DucTester at a discount

Brand new DucTester at a discountI have a brand new 300 DucTester system from Retrotec for sale at…See More
Wednesday
Scott Becnel joined allen p tanner's group
Thumbnail

Energy Auditing Equipment for Sale, Trade or to Purchase

Discuss the pros and cons of the equipment you are interested in prior to purchase. Post equipment…See More
Wednesday
Lucyna de Barbaro liked Home Energy Magazine's blog post AeroBarrier: The Silver Bullet to Zero Energy Affordability, by Paul Springer
Wednesday

Photos

  • Add Photos
  • View All

© 2018   Created by Home Performance Coalition (HPC)   Powered by

Badges  |  Report an Issue  |  Terms of Service