Overnight top-up prediction algorithm

I was just trying to think of the algorithm that the control software would need to adopt in order to figure out how much cheap tariff to pick up overnight, just for fun (Givenergy mode 1, Octopus Go). Maybe you can improve it. I've made assumptions based on the current Octopus Go tariff where cheaper electricity is available overnight 00:30-04:30.

kWh expected tomorrow by Solcast prediction. The area under its graph needs to be found which is easy enough from the raw data. It is a prediction based on the orientation, angle, location and capacity of the panels so should be reasonably accurate but it seems sensible that maybe a rolling correction factor for say, the last 30 days, between actual and predicted generation is used. So, Solcast predicted kWh tomorrow = s. Prediction correction factor, c.

I think we then need to know the likely usage tomorrow which is based on a rolling average. We want to exclude the overnight cheap charge tariff which is when we are looking to top up the battery and very large swings in use from EV charging happen. How to work this out? I think we have only a rolling estimate again over say, the last 30 days. Call it u.

I think that this means that the anticipated discharge from the battery tomorrow, given the expected PV generation and battery discharge is u-sc. (Where u > sc).

If this is right, I wonder if it could prove to be more beneficial than simply moving the manual slider controlling the overnight charge? Seems less crude to me but it could be that the correction factor and averages being used make it too vague to be useful?

Probably holes all over this, would be keen to see where they are or how the above could be improved.

I am essentially battery only. I would like a 'thing' where it looks at the outstanding battery % charge ( I have 22kwh batteries ) and then calculates the battery charge value so that the battery is 100% charged at 05h30.

The idea is that the inverter would have the lowest heat buildup possible - as the battery charge vale will be the lowest possible.

I would like to know how to automatically change this charge value daily, after running the calculation automatically at 11:30pm.

Epatrickt

Before buying my battery I ran a simulation with a full year of 5-minute generation/consumption data with that question in mind. Conclusions:

• The daily prediction approach won't stop your battery from discharging in the morning if the sun is forecast for the afternoon... aim to keep some float throughout the day from the overnight charge.
• There's no universal golden rule (I have an ASHP, EV, 5.2kWp of PV, 10.4kWh battery).
• The Solcast data is an indication but seems far from perfect.
• My rules of thumb: 100% charge every night the ASHP is running, 70% charge for the rest of the year, put the EV on the granny charger on sunnier days. "Lost" electricity of _£20 per year versus the ideal, so maybe that's as good as it will get?

Epatrickt This is in development at present, Forecasting day ahead load power, PV power, battery charge and discharge power, Then if you have a smart tariff bolted onto this, it will charge overnight just enough to cover your peak consumption. We hope to roll this out in the next month or two.
I hope this helps.

Danlambert (GivEnergy) you guys are amazing. Thank you - and GivEnergy, for being so active to make your (our) system even better. 👍

I wrote some python that does this. It looks at the hourly cloud cover forecast for the following day, combines this with predicted use to calculate how much to charge the battery during cheap rate. Its been running reliably on my home NAS for over a year with good results. It uses openweathermap api for the cloud cover forecast but could converted to use other forecast such as provided by forecast. Its opensource and available here along with richer documentation :- https://github.com/lockedj/solarbatteryoptimize

Whilst being able to calculate your overnight charge level would be useful, I look at things on a balance of risk basis.

I am on Octopus Go, and charge to 100% overnight. If I "overcharge" for the solar generation the following day it has cost me 5p per unit. However this is offset by the solar generation being exported at 4.5p per unit so my net cost is 0.5p per unit (although the current Go rate of 7.5p would give a net cost of 3p per unit). If however, I "undercharge" the cost of import during peak go rate is 24p per unit. On this basis for every unit I "undercharge", it would take 48 units of marginal solar generation saving to offset it (or 9 units on current Go tariff).

Martyn the algorithm and parameters I use are on the pessimistic side so it tends to overcharge the battery rather than undercharge. It really pays off in the summer months when there is enough sun to fulfil the needs of the house charge the battery and the immersion but it also benefits at other times of year.

locked That's impressive. I suppose that you can work out how many grid kWh you have avoided overnight importing by adding up the difference between the max and actual over a year. I suppose that assumes you erring on the side of caution does a good job to avoid having to pick up kWh during the day. Was it worth it? Maybe GivEnergy should pick up your code!

I'll dive in as I work in data science. I'd simply train a model using a regression algorithm summed across our shared data. Happy to jump on a call with the givEnergy devs to try this out even if this is on AWS where I am an Azure engineer

Last night would have been the perfect night to not charge our batteries, as we're experiencing fantastic production (just as forecast).Has anyone figured out the right API call to temporarily suspend the smart tariff overnight charge (without deleting the smart tariff altogether)?

adamdoran Just use the battery.API with /chargebattery function to change the mode, set start/end & target SOC

https://kb.givenergy.cloud/article.php?id=54
tbh I dont use the smart tariff at all. I dont temperary override it, I just submit my own charge request via my own automation every night, which means I can use any algorithm I like to work out target SOC. Worked perfectly for a very long time now.
I changed form Agile to Go Faster 5. I changed my algoritm without issue to determine charge slots. Solcast API easy to use directly as is the ability to work out how much power you may need based on hostorical records. Pretty easy maths.

I still err on the side of caution though. 1 missed optimisation looses all the benefits . If I over charge by 10p and get hot water as a result thats not a problem. If I undercharge and have to use peak rate power I am not so happy

Seye Out of interest, what approach have you taken to the algorithm?

@locked 's code considers some of the basics (cloud cover and a daily usage template) and my current plan of summer/winter settings is more conservative. With different domestic load profiles, PV array sizes, battery sizes, import/export tariffs, etc, developing a one-size-fits-all algorithm makes for an interesting project.

Seye thanks, that's very useful. I completely agree with regard to how misjudging this problem would get expensive. We originally had a single 8.2kWh battery, and so filling it up in the day wasn't difficult, but then went to two of them in January. What I'm finding now is we're frequently getting through the day with well over 50%, but the nightly charge means there's very little room to harvest the sun come morning. It would be good to manage the capacity in more of a Hysteresis-like fashion, so we're not likely to drain the battery intra-day, but do ensure there's room to harvest the sun properly.

I have occasionally found that on a sunny morning it makes more sense to prioritise charging one of the cars, because they can take around 3.4kW from the Hybrid inverter as AC, as opposed to just 2.6kW for the GivEnergy batteries. (it's annoying seeing <1.4kW following out to the grid, below which the Zappi won't activate.)

I actually just noticed in the "Solar Power Forecast" tab that "Load Power" is now being collected alongside the predicted/actual solar generation levels. I suspect this is part of the foundations for the functionality @Danlambert (GivEnergy) mentioned above. Much as it would be fun to do this for myself, It's marvellous to think that when GivEnergy releases this feature it's going to immediately cause a significant reduction in the amount of energy pulled from the Grid by a significant number of households.

adamdoran ok, factors in my algorithm:
1) The solar forecast
I chose pesimitic value so half way between 10% and main prediction. I found that mapped quite well to reality

2) Power usage
I work out the base load for my use over the day using historical data
3) Spare Power from PV ( Basic version )
subtract base load from PV forecast. Charge battery up to meet the gap
4) Spare Power ( advanced version )
look at the PV curve. I dont get much PV until mid morning. My cheep rate finsihes at 04:30 so I will ALWAYS use battery before 10:30 as a minimum. That means I always need at least 45% SOC to cover that period. Use the half hourly solar forcast to predict when PV will be above demand. Use that to ensure enough charge in the battery to meet that time. Also then work out how much more spare PV is avaialble for the rest of the day to charge the battery so that there is enough to carry over the evening and early night
5) Spare Power ( customised advanced version)
Add in non baseload e.g. a washing machine etc. Determine if that can be satisfied via PV or not after all the other factors. If not, just put it on at night when its cheep.

I got all that working pretty well, only a couple of miscalculations. Then I worked out having energy diverted to hot water just makes it so much easier. I also have smart plugs that can charge a 12v battery, my EBike, A heater etc. I don't export much energy at all

ANd dont forget, if you dont use all the energy one day its still in the battery. So when you charge overnight the and set a target <100% due to your algorithm, it will just draw less from the grid to top it up as you are just topping it up. You don't have to fully charge / discharge the battery every day.
Work out the profile over days / weeks to determine the true value of detailed optimisations. I did, which is why I switched most off

Side note:
Since energy prices are rising, my balence will change shortly. It will be more expensive per kwh for Gas than cheep rate electricity. Even more reason to heat water by PV if there is spare. I may even give it a little boost on off peak power too ( as if PV has heated it up then it wont actually use any power over night due to thermostat, but ensure plenty for morning routines just in case yesterday was cloudy)

I know this is an old thread - sorry for resurrecting it! I just thought I'd share with you an article I put together where I use a very simple automation in Home Assistant to set the GivEnergy SOC based on the following day's solar forecast. It's nowhere near as accurate as the Python script, but it's far simpler to implement and to be honest, has been giving me great results for the last couple of weeks. If you're looking for something a bit better than manually setting the SOC value every night while you wait for GivEnergy to release their dynamic charging feature, I reckon this is a good bet.

https://www.speaktothegeek.co.uk/2022/06/givenergy-giv-ac-3-0-battery-home-assistant-and-solar-forecast-automation/

good stuff! I like Homeassistant and gictcp and use it in addition to my python Giv automation. A couple of comments, if someone is only interested in automating the battery charge based on weather I'd argue my python script is simpler to setup and use (its been simplified over time). If interested in a richer smart home experience integrating different devices... into a single portal then home assistant is great but does take a more tech naunce to host / setup. Or even better use both to get the best of both worlds :-) Automating how much to charge with cheap rate has worked really well, its never going to be perfect given the nature of weather forecasts but it had made a big difference to our bills.

locked

SpeakToTheGeek locked ...and then there were 3 😁. Mine uses a similar approach to @locked, but with solcast for the weather and yesterday's consumption as the reference load. Www.github.com/salewis38/palm

locked probably a stupid question, but how do I set this up and what do I need, this isn't something I've ever tried to do so is all new to me?

Thanks

dm0806 the code and documentation is available here https://github.com/lockedj/solarbatteryoptimize. Take a look and if it is not clear please let me know

Danlambert (GivEnergy) This is in development at present, Forecasting day ahead load power, PV power, battery charge and discharge power, Then if you have a smart tariff bolted onto this, it will charge overnight just enough to cover your peak consumption. We hope to roll this out in the next month or two.
I hope this helps.

@Danlambert (GivEnergy) has this been released yet or near?

Just looking at how to set up my battery now I have switched to Octopus Go.

I recently bought a Pi 400, stuck Home Assistant on it and am running this HA automation on it. It seems to work pretty well.

https://youtu.be/IBXoD6KUtxk

Zakalwe nice link, thanks for sharing. The basic approach described doesn't account for when in the day the sun will be shining versus consumption so will be a bit more hit and miss, especially on days with changeable weather.

stevelewis

That's a fair point, though to be honest we are getting into ever diminishing returns by that stage.

The other thing that I am conscious of is the real cost of under-charging. I am on Intelligent Octopus, so every kWh that I use from the battery during the day means that I save £0.32 (peak rate is £0.40 compared to £0.075 off-peak). The flipside to that equation is that if I run out of battery during the day then every kWh consumed is costing me five times the cost.....
This will be more important to manage during the winter when the PV output will be minimal.

Zakalwe We are in a similar position. Come the autumn, we'll move from the EdF GoElectric Flat Rate (29p/kWh) to the GoElectric35 which will give us 5 hours every night at 4.5p/kWh. Standard rate on this option is nearly 40p/kWh, At that price differential, we'll be finishing the off-peak period with probably 80% SoC every night. If the sun shines, we can charge a bit, if it shines a lot, we'll get 4.1p/kWh from Octopus SEG. Putting too much in the battery would be costing 0.4p/kWh. I'm not sure we'll even bother to check the forecast.

My automation (link in earlier post in thread) is conservative, it determines tomorrows forecast generation and takes into account expected usage but sets a min. The combination of expected use + min mean there should always be enough from cheap rate +battery + sun to not pull from the grid at peak rate (other than occasional high demand demand that solar+battery cannot handle)

locked same here (I'm probably more conservative still with actual usage and an average of P10 and P50 values which factors in uncertainty in the weather forecast), the one in the video was a lot more basic. As @Tim mentions, if the potential saving is trivial due to the import/export rates, the risk/reward case is very different to the use case with deemed exports.

M_J it's in the same queue as sorting out multiple hybrids/plants, i think the queue's called 'promised but no plan' 😃

ivell hopefully lower priority than fixing the SoC issue for those who have been suffering from this.

locked I know I asked you this a while ago, but given it runs in python, do I need a separate a second raspberry pi to the one I already have running HA? I'm fairly new to this but given I may need to start charging my battery overnight a bit more through winter, am keen to learn to get this working

dm0806 No. The most common implementation seems to be to install Ubuntu on your Pi, then Docker and Portainer. Then install Home Assistant in a container, then install Givtcp in another container.

Easiest is HAOS direct on the Pi. GivTCP AddOn

TheDragon (GivEnergy) if it's just the top-up prediction/register change, the easiest is the Python I pulled together. Runs natively on Rasp Pi, PC, phone, etc Www.github.com/salewis38/palm

Hi All, I can see a few great custom solutions are out there! Sorry for the bump, but I have ordered a Givenergy 9.5 system and am trying to figure out in advance how I could use this. One specific issue I want to solve for is preventing drain from the battery to an immersion heater I use overnight. I believe this can be done by setting the battery to charge to a particular level (or is there another setting to prevent discharging?). I am trying to figure out if it’s possible to combine that with anyone’s custom solar prediction automation?!

Put the immersion to run during the charge time slot for your batteries. Whilst the charge time is running the batteries shouldn't discharge even if full. This will, of course, change as soon as youtr charge period has ended.

Bear in mind if you don't want your batteries to charge to 100% on a particular night, you can set a charge to percentage.

However it is also a discharge to the house/immersion/etc percentage!

E.g. if you say charge to 50% and your battery is at 75%, the battery will continue to kick out power until it gets to 50% or until the timed charge window ends.

Think of that percentage as "I'd like the battery to be this full at the end of the timed charge"

TX200 thanks, this is what I am struggling with - it might need some slightly more complicated logic adding in to a prediction algorithm to stop it discharging to immersion if the battery is sufficiently full already. Is there a mode that tells the battery to do nothing - ie neither discharge nor charge? If so I could create some kind of IF statement that charges if below the required amount, or sets the battery to stay fixed if above it, and then switches to standard mode after the immersion window is done. With current Economy 7 rates and Octopus outgoing paying 15p, even once you consider round-trip efficiency there’s little reason not to charge to 100%. Even if overcharging you lose maybe 2.5p per kWh overcharged. By the time I actually get my battery installed it could look totally different of course!

Boffinboy set the charge and discharge rates to zero?

Don’t forget to change them back though 😆

Thanks. Not got the kit yet, and not a programmer, will have to take a look when I get it and see if I can modify some of the code shared here. Presumably I can just add something like that within the code shared here, or have it estimate what to charge to, then if it’s already above that set the discharge rate to 0 for a period of time.

Boffinboy In your case, simply charge to 100% all the time as you suggest... it's not worth over-thinking it!

I'm on an old FIT scheme with 50% deemed export so don't get paid by the kWh for export and therefore have a very different use case.

stevelewis you’re probably right! May change when rates change, by which point I will have the battery to actually tinker with

As you have no rush why not consider a myenergy eddi and have the immersion controlled automatically?

positor1 fair question. Essentislly cost and time! I’ve gone for a Wi-Fi immersion timer that cost £45 and will take almost no time for electrician to install vs Myenergi kit or alternatives which are much more expensive and have availability issues I think. My goal is not to divert solar, but rather to reduce gas use by a) not using it to heat water, and more importantly b) allow me to turn down flow temperature on system boiler to 50 degrees or below and ensure efficiency via condensing mode in heating season whilst still having tank at 60 (no separate hot water priority in my setup currently). In summer I can then decide whether to use a little bit of gas for hot water, or continue to use immersion overnight for a couple of hours. If I get on well with it, and decide to solar divert, then maybe in a year or so i can switch to a different solution (maybe Giv will release theirs!). I figure over this winter I will save the cost of the Wi-Fi timer install in efficiency gains on the heating bill.

@stevelewis finally got my battery installed and whilst I’m on E7 and not Flux I was planning to set up your palm to run on my Synology NAS. Unfortunately I’m struggling! I was hoping to run it as a scheduled task, but it’s not working. When I try to run via SSH I get an error about a missing module - presumably the version of Python on the NAS lacks some extra needed bits and pieces.

I think I should probably be running it with Python in a Docker container, but that’s beyond my capabilities at the moment…. I have a few things running in Docker, but those were straightforward to setup as they were already standalone containers.

I also wanted to ask what the second charge slot feature is, and whether that can be disabled? Is that intended for use with Flux, and if so - how does it work?

Any advice much welcomed!

Danlambert (GivEnergy) This is in development at present, Forecasting day ahead load power, PV power, battery charge and discharge power, Then if you have a smart tariff bolted onto this, it will charge overnight just enough to cover your peak consumption. We hope to roll this out in the next month or two.

Guessing this didn’t happen?