News‎ > ‎

RESNET Chapter 6: Sampling... it just isn't getting the job done

posted Oct 31, 2014, 3:09 PM by Stephen Mogowski
Imagine taking a poll at the latest mega-subdivision in your hometown:
You interview everyone who just bought a home.  Maybe its their first home or their energy efficient dream home!

Now imagine the look on their faces when you tell them that only 1 out of 7 of the homes constructed in their subdivision were actually inspected!  For the mathematicians out there, that comes in at just under 15%.  I suspect your new subdivision is less than pleased.

For those that don't know, sampling is an acceptable strategy for RESNET Sampling Providers to implement in order to certify a large amount of homes instead of testing each one individually.  There are many benefits to the builder (less cost) as well as the rater (charge per certification not per inspection) but there are huge drawbacks to this strategy as well.

Before we get started, there are rules to sampling... for instance you need a "first seven" or "first three" lots to pass consecutively before the sampling process can begin.  And if a home fails, you test two more... if one of those fails, you test the remaining "batch" of 7 homes.  I get all that, I once helped certify thousands of homes per year for ENERGY STAR.  And yes, it was all due to the sampling process.

The theory behind sampling is sound.  If you see a problem, you fail it and reinspect on the next houses that normally wouldn't have been inspected had the item passed initially.  It is built off the premise that if the same contractors are doing the work at all the houses, their craftsmanship should be the same.  This is problem #1.  

All framing/insulation/air sealing/etc crews are not created equal.  The construction industry in general is made up of a lot of temporary employees in large part to due to the volatility of our market with respect to the economy.  So assuming that one skeleton crew is going to outperform another is no guarantee.

The 2nd problem with sampling is that not all homes have the same design features.  So if the inspected home was a one story without a band joist but houses 3 and 4 of the batch are a two story home with a band joist-- there could be issues that go unnoticed.  This could be repeated several times before the problem is caught in a test home.  But once it's caught, there is no going back in time to fix the problems of the previously certified two story homes.  Those homes are flawed but receive the same certification and marketing label that the inspected homes receive.

Moving forward.  "Builder Sign Offs" are a method of allowing problems with a lot to be fixed and repaired without triggering sampling protocol.  The following passage from ENERGY STAR v3 Checklist Rev07 shows that you can have up to eight (!) problems and have them corrected and move forward... but what about the other six houses in the batch?  Do we honestly believe that all builders are going to go through and verify that each of these issues is being resolved at their other active lots?  And the problem with this oversight is the Builder Sign Off could be repeated again and again!

"The only exceptions to this rule are in the Thermal Enclosure System Rater Checklist, where the builder may assume responsibility for verifying a maximum of eight items. This option shall only be used at the discretion of the Rater. When exercised, the builder’s responsibility will be formally acknowledged by the builder signing off on the checklist for the item(s) that they verified."

The last issue I have with sampling relates to simple economics.  At the heart of every company there is a financial side.  And it is also important to keep your clients happy.  Builders do not like to "fail" inspections for a multitude of reasons but I've found that the main one is that construction delays are costly and reflect negatively on the manager's performance.  Failed inspections also trigger sampling protocol which requires more site visits for the energy rater.  More site visits equal less profit for the energy rater. is in the best interest of all parties (financially speaking), to pass or "Builder Sign Off" most lots.  This is compounded by the fact that subcontractors are often paid piecework so their workmanship can be substandard if they are trying to maximize the number of jobs they complete.

At the end of the day, RESNET's Sampling Protocol wasn't written to be exploited.  It called upon the good nature of raters and builders alike to build MORE homes efficiently.  And the Protocol has worked-- there are many, many builders who build ENERGY STAR that wouldn't be able to if there wasn't a Sampling Protocol.

But I challenge whether it is time, as our industry gains momentum, to revoke RESNET Ch. 6 and start giving quality inspections to all new energy efficient homes.

Thanks for reading!