Case Study: Do Smart Thermostats save money?

“Smart thermostats save you money”.

I feel like I’ve heard those words so frequently that I just kind of assume it’s true.

But is it actually true? And if so, what kind of savings are we talking?

I wanted an answer to these questions so I spent a couple months testing different smart thermostat features in my home. I used the awesome data collection power of my ecobee thermostat to record the data. Here’s what I found:

Overview

The vast majority of potential savings can be achieved by simply creating a schedule. To do that, an expensive smart thermostat is not necessary. All you really need is a programmable thermostat.

However, cost is not the only consideration. Ultimately, any savings should not come at the expense of comfort or convenience. I want my home to save and be comfortable, but I don’t want to worry about constantly adjusting my thermostat.

Smart thermostats can achieve slightly higher savings than programmable thermostats while simultaneously being easier to use and way more effective at keeping the right areas at the right temperature.



How Much Is The Average Savings With A Smart Thermostat?

Several years ago, a study by Nest concluded that customers saved an average of 10 to 12 percent on heating costs and 15 percent on air conditioning costs.

Ecobee simply claims that their smart thermostat can save you up to 23% on your energy bills. Their claim is based on a household that leaves their current thermostat set to a constant temperature of 72F.

The amount an individual household saves can vary quite a bit depending on your existing situation and thermostat habits.


How Does A Smart Thermostat Save You Money?

A smart thermostat uses a combination of scheduling, geofencing, and motion sensing to know when to turn the heat up or down.

A smart thermostat isn’t some mystical device that changes the laws of physics in order to use less energy. The only way to spend less is to heat or cool less.

The smart part is deciding when to do it, so you don’t notice.

thermostat_front_away

Are you a set it and forget it thermostat operator or do you micro-manage your thermostat?

The households that will save the most money are the ones that either

  1. have a programmable thermostat that they don’t utilize or
  2. have a manual thermostat that tends to stay set to a single temperature 24/7.

The Experiment

What will I test? How will I measure it?

I will test 3 different smart thermostat features. I will start with a control phase and then add features with each subsequent phase. Each phase will last for 3 weeks so that random daily fluctuations have a chance to get averaged out.

Phase 1: Control – constant 70F

Phase 2: The effect of scheduled set backs

Phase 3: The effect of using a remote sensor

Phase 4: The effect of a smart thermostat preheat feature

How to determine which settings save more?

There’s a simple way to compare two setups and it requires only one measurement: average indoor temperature. This is the method I will use for my calculations.

Here’s why it works:

1. During cold weather, your home is constantly losing heat to the outside environment. In order to maintain a constant temperature inside, your heating system simply needs to replace the amount of heat lost.

2. The rate a home loses heat depends mainly on the temperature difference between the inside and the outside.

For example, if I keep my home at 70F and the outside temperature is 30F, that is a difference of 40F (70-30 = 40).

I can expect to require more heat if the outside temperature changes to 20F (70-20 = 50).

Similarly, if I prefer my home to be set to 65F, I can expect to require less heat (65-30 = 35).

A smaller difference means less heat loss which ultimately means less heating is required.

3. Therefore, all we really need to do is record the average indoor temperature. If our thermostat’s settings result in a lower average indoor temperature, we can be fairly sure that:

a. our home lost less heat, which means

b. our heating system didn’t have to replace that heat, which means

c. we saved money.

My furnace and thermostat setup

My home is two full floors plus an unfinished basement and an attic living space. We spend most of our time during the day on the main floor and at night we sleep in our bedroom on the second floor.

I have one forced air furnace in my home. It’s a natural gas, single-stage furnace.

All heating is controlled by my ecobee3 smart thermostat. It has a temperature sensor on the thermostat which is located on the main floor. I also added a remote temperature sensor in my bedroom on the second floor.

The data recording of ecobee is AMAZING for this experiment. It logs each piece of data I need every 5 minutes. From the ecobee web portal, I can download the data for any given period of time. Then, I can analyze the data with spreadsheet software.


Phase 1 – Constant temperature

The first step of my experiment was to determine a baseline which all my other tests would be compared to. So, I set my thermostat to a constant 70F day and night for three weeks straight.

To get an idea how my home’s temperature fluctuates throughout the day, I graphed the average temperature by time of day for each sensor.

The blue line is the temperature sensor on the thermostat. The thermostat is using that sensor for all heat control. That’s why it’s basically a straight line. You can see that the average temperature is between 0.5F to 1.0F below the set point of 70F. The actual average was 69.3F.

The red line is my bedroom sensor and you can see it slowly rises throughout the night. I suspect the main reason for this is that we shut the door to our bedroom while we sleep. After we get up in the morning, we open the door and the temperature seems to fall back in line with the rest of the home.

This tells me that my bedroom is getting more heat than is needed. While that may seem like a bad thing, it’s actually a perfect opportunity to use the smart thermostat’s remote sensors to save a bit extra. I will show how much in Phase 3.


Phase 2 – Set a schedule

For phase 2, I wanted to find the savings you might achieve by simply setting a schedule. You don’t really need a smart thermostat for this. Any programmable thermostat will have the ability to set a schedule. However, it’s much easier to set and change a schedule with a smart thermostat.

I set the thermostat to 70 during the day from 6:00 am to 10:00 pm and set it to 65 from the hours of 10:00 pm to 6:00 am. Here’s the resulting temperatures:

This is pretty much what I expected after seeing the results of the first test. The thermostat sensor averages about 0.7 degrees below the set point. Again, the bedroom with the door closed at night gets too warm. But during the day with the door open, it stays cooler than the rest of the house.

There is one anomaly to make note of. The thermostat temperature spikes at 1pm, while the bedroom sensor temperature dips at the same time. The only reason for this I can think of is that during this time of year and during a short period of the afternoon, the sun must shine directly on the thermostat and cause it to read a higher temperature. In turn, the furnace runs less and the rest of the house (bedroom sensor included) gets cooler.

Now for the real question…Does the schedule change actually save money?

As I explained above, the simplest way to compare two settings is to get the average temperature of each. Whichever average temperature is lower, should require less heat.

Constant 70ScheduleDifference
69.3 F67.8 F1.5 F

It should have been easy to see simply by looking at the graph, but the calculation confirms that the schedule temperature is significantly lower.

Next, I can use that difference in temperature to calculate the potential percent savings.

Outdoor temp% Savings
10 F2.5%
15 F2.8%
20 F3.0%
25 F3.4%
30 F3.8%

Savings with a daytime setback

I work at home so I keep the temperature set to my “home” settings during daytime hours. However, lots of people’s homes are empty during the day. For those people, it’s another opportunity to save on energy costs by setting the temperature back during the day (see the dashed green line on the figure below).

NOTE: The temperature is higher during the daytime hours because I assume during the 2 weekend days, you set the thermostat to the home settings.
Night setback only With daytime setbackDifference
67.8 F66.6 F1.2 F

Additional savings with a daytime setback:

Outdoor Temp% Savings
10 F2.0%
15 F2.2%
20 F2.4%
25 F2.7%
30 F3.1%

Extra savings without the morning heat?

I noticed that my furnace has to run for about 2 hours straight on an average winter day to heat the house from the overnight temperature to my morning temperature.

If you are someone that leaves for work shortly after, you may wonder if it’s worth it to heat the house for that hour long period between getting up and leaving for work.

How much could you save by just tolerating the morning cold while you get ready (see the red dotted line on the figure below)?

Well, according to my calculations, not much:

Night and daytime setback No morning heatDifference
66.6 F66.3 F0.3 F

This minimal temperature difference results in a minimal savings:

Outdoor Temp% Savings
10 F0.5%
15 F0.6%
20 F0.6%
25 F0.7%
30 F0.8%

If you can tolerate the morning cold while you get ready for work, you will likely save yourself less than 1% on your heating bill.


Phase 3 – Using remote sensors

My bedroom is the only occupied room at night and it gets too much heat. I can solve this problem with a remote sensor.

For this phase, I continued using the schedule set in Phase 2 but this time, the thermostat uses the remote sensor in my bedroom for heating decisions during the “sleep” period (10pm – 6am).

This drops the temperature in my bedroom to the desired level. As an added benefit, it also drops the temperature in the rest of my unoccupied house. This should help me save on energy costs.

Here’s what actually happened:

Notice the bedroom temperature stays very close to the set temperature at night. That’s great. It means the sensor is working. Also, notice that the thermostat temperature falls significantly below the set point in the early hours. That’s also great. It means energy is being saved.

Now take a look at the Phase 3 thermostat temperature compared to the Phase 2 thermostat temperature:

If the Phase 3 average is lower than the Phase 2 average, that would mean the addition of the sensor saved energy:

Thermostat sensor controlWith bedroom sensor controlDifference
67.8 F67.6 F0.2 F

With only a 0.2F difference, it doesn’t seem like using the sensor saved much. However, I was disappointed to see there is another “problem” that shows up around 10pm when the schedule switches to sleep mode.

When the schedule is set to change, the ecobee slowly averages in the temperatures of the sensors that will be used in the next schedule setting. Since the bedroom sensor is typically colder during the evening hours, this brings the average temperature down. Meanwhile, the set point is still at 70F. Therefore, the heat kicks in and we see that “bump” show up right around 10pm.

Without the extra bump, it looks like the savings would have been much more significant.

When sensors don’t save – What if our bedroom was too cold at night? This time adding the sensor would cause the system to run more than before. Again, the bedroom would be comfortable for us, but the rest of the house would be heated to several degrees warmer than usual.

In this case, using remote sensors by themselves aren’t really the right tool for the problem. Instead, I would consider using a vent booster and possibly a smart vent system.


Phase 4 – Will smart recovery kill your savings?

For this phase, I kept all the same settings as Phase 3. In addition, I turned on the smart recovery option for heating.

The smart recovery feature starts heating your home ahead of time so that when your schedule changes, the home is already at the new temperature setting.

Since it turns on the heat earlier, I don’t expect smart recovery to save energy. In fact, I expect it to use more. The question is, how much more?

For my schedule, smart recovery is activated in the early morning hours so that the furnace can recover from the cold “sleep” settings (65F) to the “home” settings (70F) by 6am.

Again, we see the bump right before the switch to sleep settings caused by the remote sensor in the bedroom. However, since both Phase 3 and Phase 4 have the same problem, it sort of cancels out as long as I only compare those two.

Here’s how it compares to Phase 3:

You can see that the heat turns on somewhere between 4am and 5am with the preheat enabled. Whereas, it’s closer to 5:30am without the preheat. Here are the average temperatures of each:

No preheatWith PreheatDifference
67.6 F67.6 F0.0 F

Well, that I did not expect. Preheat should have a higher average temperature.

At first glance, this doesn’t make much sense. But, what I didn’t show you is the average outdoor temperature for each period:

No PreheatWith Preheat
Outdoor Temp23.8 F16.9 F

The outdoor temp during the preheat test was much lower, which caused the indoor temp to drop much faster during the sleep period. This brought the overall average temp down. Unfortunately, I can’t make any solid conclusions about preheat.


Final Thoughts

After all this, did I actually learn anything about smart thermostats?

Yes!

Here are the main things I learned:

  • In my opinion, the most valuable feature of my smart thermostat is the ability to use remote sensors. Although it didn’t save the amount I expected, it added necessary comfort while I sleep.
  • Smart home/away settings don’t save as much as I thought. As it turns out, I don’t leave the house very often so there aren’t many opportunities for smart home/away to save me money. However, if you are regularly gone for long periods of time and prefer to use smart home/away rather than a schedule, it would certainly add up to significant savings.
  • If you have a nighttime setback and a daytime setback, I thought that eliminating the morning reheat in between could save a significant amount. It turned out to be only a minimal amount of savings.
  • Finally, 70F is waaay too hot for sleeping in winter.

After tallying up the potential savings, I decided the amount of money I can save on my HVAC bill isn’t big enough to justify sacrificing any comfort. It’s more important for me and my wife to be comfortable in our own home than it is to save $20/month.

However, there’s no sense in being outright wasteful. So if I can save a bit and be more comfortable at the same time, it’s a win.

I can do that with my smart thermostat and I suspect that you could do it too. It doesn’t require any complex automation or custom code. All you need is one of these full-featured smart thermostats.


P.S. If you have any other money saving tips for smart thermostats or ideas you want me to test, let me know in the comments below.

15 thoughts on “Case Study: Do Smart Thermostats save money?”

  1. That’s fantastic if 60% of participants benefitted from the thermostat. I just barely had my home built. I’m still trying to decide what type of heating I want. I may just go with a furnace to make it easier. The thermostat would be a nice addition.

  2. There is no denying the fact that there are many advantages of using a WiFi thermostat and the energy saving are one of the most important. These thermostats are a lot more convenient to use than a traditional or a standard programmable thermostat and are loaded with many different features to ensure a high level of comfort, allow you to have a better control on the heating and cooling system and also enable you to manage your energy costs efficiently.

  3. Wow! I never knew that a smart thermostat could be this efficient. It is a one-time investment and hopefully, can help in saving me a lot of energy and money. It will help increase the efficiency and will make things easier in the long run. Thanks for sharing!

  4. Hi Eric,
    I live in the Middle East.
    We do not need to heat up the house we only have AC.
    Every room has it’s own AC system and the monthly electricity bill is very bad… it goes from USD 800 in Winter to USD 1.500 in Summer (the house is on two floors for a total surface below 300m2). If I were to implement smart thermostat in the entire house I’d need 7. Do you think it would really make sense in the short term (1 to 3 years)?

    1. Great question! And wow, that is a huge electricity bill. I suspect you are using mini-splits? If so, you can buy AC controllers (like these). An average cost per unit is about $100. So for a total of $700, you can convert to smart control. How much you could save depends on your habits so I can’t say for sure whether you could save money with them.

      1. I would like to make sure I understand. In the best case scenario you save 4 to 10% and it makes 20$/month so your heating bill is 200-500$ per month. So you pay 6 000-12 000$/year in energy? My god, you must live in a huge mansion!

        1. Not really. My home is about 2000 ft2 and winter heating bills are typically about $250 but that depends on weather and the price of natural gas. You sound like you’ve never paid a utility bill before.

  5. what no one mentions is that it takes more energy to recover from a temp setback than it does to simply leave the thermostat at a temp you’re comfortable with. If you lower the heat setting for 8 hours when you go to work then bring it back up before you get home more energy is used to do that than to simply maintain a single setting the entire time. Your equipment has to work harder longer shortening it’s service life as well. The only benefit is to thermostat manufacturers and installers in the form of $$$. Unconvinced? Leave the thermostat alone for a month and compare your energy usage. Not as sexy as wifi connected devices and the ability to change the settings from the other side of the world but if you want to use less energy and save money on high-tech gadgets, a $12 thermostat will do just fine.

  6. Thorough and well written test. The only thing I would have done different is to run the baseline test with a regular non-programmable thermostat, because how do I know the Ecobee isnโ€™t adjusting since the start of the test.

    Also, it seems that the smart thermostat doesnโ€™t seem to really save you that much, however over time Iโ€™m sure it adds up.

  7. This is exactly the type of real-world, empirical study I was looking for. Excellent study design, data and conclusions.

    Smart thermostats are way over-rated, and the savings touted by the manufacturers of these devices are not to be believed.

    The human body is a living, constantly fluctuating, analog entity. No mathematical, electro-mechanical system of HVAC control can take into account all the environmental and personal factors that enter into the dynamic human question, “Do I feel hot or cold?” And that’s what determines whether a thermostat gets adjusted or not and how often it needs to be adjusted.

    Unless you spend a lot of time away from your home ( an empty house ), and can therefor make use of an “away” schedule, a smart thermostat will save you very little.

  8. re: “Since it turns on the heat earlier, I donโ€™t expect smart recovery to save energy. In fact, I expect it to use more. The question is, how much more?” — It can actually save energy, depending on the behavior it is replacing. As you mentioned, outdoor temperature would be a factor in the pre-heat results. So then, for example, if you had been scheduling your heat to turn up at 5 am to make sure it’s the desired temp by 7 am, and you switch to a smart pre-heat that still has you warm at 7 am but varies day to day in turning on the heat between 5 and 6:30 depending on the how cold it is outside, then all the days it decides your heat doesn’t really have to be on as early as 5 am to get you warm by 7 are days it saves energy. It might only cost energy if it decides it should turn on even before 5. (Numbers for illustrative purposes only, of course.)

Leave a Comment

Your email address will not be published. Required fields are marked *