Calculating House consumption - reliably!

I am overwhelmed with data - from GivEnergy reports, Octopus import and export (in 10 second intervals if I want it with an Octopus Home Mini), etc, etc....

BUT - I don't seem to have a way to accurately know daily home usage. The GivEnergy reports do have a consumption total but I presume this is derived from the "Home - half hourly" data and every day that has one or more gaps in the graph.

In trying to save energy knowing accurately how much we are actually consuming would seem important.

I am just starting with Home Assistant and GivTCP so maybe that will give a solution. I presume the data is there in the inverter but some of it is not being sent to the cloud for some reason (inverter to router is strong and our internet is good)

Meanwhile, any thoughts welcome!

[unknown] We are in the same boat. We know exactly what our consumption was from DCC downloaded data (weekly, daily, hourly and 30-minute summaries available) for the last 13 months and indeed for the 12 months prior to solar installation. We still have access to the DCC data and can see import (but sadly not export although the meter registers it) at the same level of granularity. But now, we are reliant on the Givenergy cloud to tell us our actual consumption.

There are three issues here for us;

1. Data is lost from time to time due to comms failures and so the cloud data is incomplete
2. There are patent errors in the calculations - the numbers just don't add up correctly on the portal, even over a long period of time. Whether this is complicated by the fact that we have two hybrid inverters, I don't know. I have reported this but in fairness, they're dealing with much bigger issues currently. Even the house consumption on the web portal summary only shows net for one inverter.
3. The EM115 which Givenergy uses to control the inverter power, while certified for metering and performing really well in inverter control, registers different import/export from my smart meter (the de facto correct standard as far as I'm concerned). We have two EM115s as each hybrid needs its own and the values registered by these two devices are different, even though they are situated in the same DIN enclosure and their CTs are adjacent to each other on the same cable.

What I conclude from all of this is that the Givenergy portal is an excellent indicator of solar generation, battery energy input/output and house import/export. The portal itself is very informative and rams a tremendous amount of information into each page but as far as I can make out, the house consumption data is incomplete, unreliable and has errors, so I'm not looking to use it directly for calculations of savings.

What we are looking to do is compare our actual import kWh with historic data and use current prices to calculate it that way. We haven't changed lifestyle since the solar installation although we do now have an EV, so there's some tweaking needed to factor in PV to EV (and off-peak to EV) replacing petrol that we would have bought otherwise.

At the high level, we did do some ballpark calculations around the above model and worked out that our system should pay for itself in just under 7 years. Considering that we didn't do it as an investment, but rather a "help the planet" and reduce rocketing energy bills, it looks like a good investment as well.

Tim We are in the same boat

Glad it is not just me that finds this frustrating!

Tim , out of interest, how far out are the readouts on the EM115's themselves compared with your smart meter?

And, does it look like the GE inverter uses the cumulative energy values from the EM115's, or is the inverter carrying out it's own power/time integration before passing energy data to the cloud?

If I'm understanding the problem here, I've completely solved this recently. I don't have a givenergy system yet but I wanted to know my real usage and solar export in real time and in one place, and with the history.

So this is solvable, it's 100% accurate, and it will survive brief disconnections. And it doesn't rely on "the cloud" which can be turned off at any time. It's perfect.

But... it'll cost you.

Requirements:
Around £80
Home Assistant installed on either a raspberry pi computer or spare PC, virtual machine, whatever. Sounds like OP already has this, but if you're not OP and want to do this, be aware that you should probably be "technically inclined" to use this. Home Assistant is brilliant, but it's a bit of a pig to setup sometimes, and this is process will be no exception, so keep that in mind.

Step 1:
Install this app, take the time to get it set up and give it time to sort itself and start showing your data: https://www.glowmarkt.com/#/faqs/apps/how-do-i-find-bright I'm not sure if a human has to request things at their end, so keep that in mind given our proximity to Christmas.
Once numbers start appearing (if the costs are wrong it's due to your smart meter having the wrong prices, see the FAQ in the app) you're good to go for step 2. I recommend you get this working before step 2 as that's when you're spending money and they will require the username you setup in this app when you order to set up the IHD.

Step 2:
Buy this: https://shop.glowmarkt.com/products/display-and-cad-combined-for-smart-meter-customers - ensure you fill out the form correctly with your username (setup in step 1 when you set up the app).
Once this arrives it should "just work" but you will have to set it up to work on your home wifi.
You now have two IHDs telling you the same thing.

Step 3:
This is where it gets a bit complicated.
Watch this video: https://www.youtube.com/watch?v=7tnKTNZd9Dc
This guy will teach you how to set up your new IHD to talk to Home Assistant. Follow this guide. Again, this is a bit of a pig to set up. I had to uninstall and reinstall the MQTT broker addon before it worked for some reason and I had to manually edit a configuration file and insert the serial number to get things working.
I found that I got a few days of history imported automatically. Unfortunately you don't get much but the IHD will reimport any missing data for when it's turned off or can't get data for a while for whatever reason. You're probably okay for a day of two of outage but after that I've found my smart meter deletes the data so keep that in mind.

Step 4:
Set up the energy dashboard in Home Assistant. It will ask you for various sensors and they should all be present if step 3 was followed correctly and everything is working.
These are my settings: https://i.imgur.com/WBBNcsU.png
Only the Electric and Gas sections will be applicable here. The Solar is from a separate integration for my Solis setup.

You'll then get proper import and export values. If you add in your solar (like I have) then you'll also be able to see the self consumption of solar generated. You can also add your givenergy battery here to complete the picture.

This is 100% accurate as it reads from your meter and it's what you get billed so it should be as precise as it gets. No clamps, no guesswork, just real numbers from your meter which is what you're paying for. Anything battery/solar related will have to come from elsewhere, of course, but this solves the "this is what I'm paying/getting paid" problem, which is nice.

This is what my energy dashboard looked like in Home Assistant for yesterday: https://i.imgur.com/adbw0va.png

Home Assistant does an excellent job of displaying this data and allowing me to "geek out" should I want to. An involved process to set up, but very much worth it in my opinion.

ribs If I'm understanding the problem here, I've completely solved this recently. I don't have a givenergy system yet but I wanted to know my real usage and solar export in real time and in one place, and with the history.

Thanks for all that - will examine it in detail.
One quick question: What does does Bright/Glow do that is not already possible via the HA Octopus integration and API?

[unknown] The bright App is developed my Hildebrand who are a "DCC Other User". By downloading their app and signing up to their service (you also have to enter your MPAN), they provide you with access to your DCC summarised data (weekly ... 30 minutes etc) by MQTT subscription and by API, if that's your bag. If you also buy one of their dongles, you can get near-real-time data from your smart meter, just as your IHD does. The dongle posts your real-time data up to their cloud and you can pull it back using their API with a refresh of every 6 seconds or so. It is much more dynamic and reliable than the Givenergy API and about the same refresh as the shortest period with Givetcp.

We have had this service for over 12 months and it is good at reporting what the smart meter reports, but it doesn't tell you what your house consumption is, just the energy import (and export if you have it set up).

[unknown] how far out are the readouts on the EM115's themselves compared with your smart meter?

I've looked at meter readings registered in the Givenergy portal and a midnight snapshot which I get from the GlowAPI at the start and end of the off-peak period. The two EM115s are within 0.15% of each other with an import registered by the Givenergy system as 477kWh between 23/11 and 22/12, whereas the smart meter (data from GlowAPI snapshot) delta for the same period is 457kWh. In the summer these numbers would be less than 30kWh, but this time of year with poor solar and two EVs to keep charged, the volume is much larger and provides some better data to compare.

The portal looks to take the meter import value from one of the EM115s rather than averaging.

To be clear, you don't have to push raw data to "the cloud" if you don't want. My IHD works entirely on my local network.

Regarding getting the consumption data (and not just "import"), I do get that. Home Assistant figures it out from the import from the metre and the solar generation from the cloud.

Today I had a pretty rubbish day for solar generation, so this is what I'm seeing (split over multiple screenshots on my phone): https://imgur.com/a/Tuc4l53

I'm assuming the math is good with this. Looks good at first glance to me. It does also have a storage battery option and I know there is givenergy integration so that should also slot in without issue.

[unknown] To be clear, you don't have to push raw data to "the cloud" if you don't want. My IHD works entirely on my local network.

Your IHD does not push any data to the cloud, your smart meter does that itself. The data is published to the DCC cloud from where your energy supplier accesses the meter readings so they can work out your bill. The Bright app reads the historic data sent by the smart meter to the DCC cloud via the Hildebrand "other user" API route. This is what you are agreeing to when you create an account and enter your MPAN etc. If you also buy the Glow/Bright/Hildbrand dongle, it connects to your wifi and to your smart meter over zigbee (same as your IHD) and this data is published to the Glow cloud (ie outside the DCC). This data stream is the one that updates every 6 seconds or so.

If you don't want to push raw data to the cloud, don't have a smart meter.

[unknown] Another thought, but manumatic. Use your MCS meter and smart meter. MCS meter import is charge to battery; MCS meter export is PV + battery discharge to house; Grid import is either house consumption or battery charge and grid export is either battery or PV to grid.

Grid import = House consumption + MCS Import
MCS export = House consumption + Grid export

House consumption = Grid import - MCS import + MCS export - Grid export

The accuracy of the MCS meter is no better than the EM115 or smart meter, but is designed as part of the microgeneration scheme to register system import/export. I suppose using the EM115 meter data in the same way would also work with a similar level of accuracy.

Tim I'm pretty sure it's pulling from the smart meter.

When I place the IHD into a position where I have known weak signal to the meter, but strong wifi, the meter doesn't work at all. Neither the original IHD from my provider or the new one will work in this position. The Hidlebrand IHD needs both decent meter and wifi signal to work.

The meter may also be pushing things to the cloud for my energy provider to use, but that's not my concern and that's for them to maintain and pay for (which I guess I'm ultimately paying for, but whatever...)

In terms of what I want, which is local monitoring to my home assistant instance, everything appears to be local from what I can see, which I what I want.

Hi,

I am finding the EM115 is reporting some energy used as reactive and my SMETS2 meter is showing it as active, the power factor between the 2 meters is incorrect with the inverter off, currently talking to Octopus about it.

I am finding about 1.5-1.8kwh a day in difference due to this between meters depending on if the heat pump has run, how many cycles the washing machine has done and if the fridge has had to cycle a lot.

With the new beta update EM115 shows 0w but 0.288a, my IHD displays this as 65w and therefore charging me. Owen Brothers say this is reactive power and not active hence it reading 0w and the inverter not ramping up/down to match (as its not real power).