Inverter timeouts

I was having considerable problems with Inverter Timeouts with both the GE app and Wonder Watt. I wasn't convinced that the Wifi was the root cause generally because the Wifi connection did not drop at the times that the timeouts were occurring (timeouts are visible in the GE log in the portal and individual Wifi connections are visible in the log in my router and these never coincided). However I did suspect that local traffic to the inverter was overwhelming the dongle.

I use 7 instances of the Android Inverter app, each on a 5 second poll, along with one instance of the BBC Basic app plus a couple of instances of the GE app. These give me views at a glance all over the house (mostly on old and very old phones and tablets that have become disused over time - the oldest is over 10 years old running Android 6). You can imagine the internal traffic this creates to the inverter. WW commands often went through all 4 retries before failing the command - most annoying when I missed a charge or discharge slot on Agile or ended in the wrong mode.

With The WW guys help I reduced traffic to a single instance of the Android app and increased the retry period ( for 4 retries) to 2, 8 and 24 seconds. Reducing the number of polls to one device did not eliminate the timeouts but those that did occur usually needed just a single retry. I re-enabled all the Android Inverter apps but changed most of them to 60 second poll interval. This did bring back the timeouts but now the longer retry intervals came into their own such that by the 4th retry the inverter seems to have recovered. With a very busy night of schedules, most commands got through with 0 or 1 retries. A couple took 2 retries. Just a single command used all 3 retries. In this command the first three attempts got an Inverter Offline response but the 4th one worked. Separation of the retries was 2, 8 and 25 seconds.

So if you're also plagued by this problem, consider if you might have a lot of traffic to the inverter causing it to become overwhelmed. And if you use an app or home automation, implement a retry sequence that allows for at least 30 seconds recovery time.

I had inverter timeouts long before I used the Monitor for GivEnergy inverter app. Today I have a single instance of that app, which is not currently set to run in the background, and a single instance of the official app. I still get timeouts.

Windy Miller One suggestion is to proxy the inverter so that the androids (and/or bbc app) connnect to, say, a rpi which is mirroring the real inverter registers. All the bits and pieces are available as python code using modbus protocol. I have a dummy server which both android and bbc are happy to connect to.

I'm surprised it held on so well with 8 connections polling every 5 seconds to be fair!

hoggy I agree but rejecting an individual command with busy rather than collapsing in a heap would have been a preferable error condition to what was effectively a DDOS attack!

I wrote the original post to help people understand that timeouts are sometimes caused by things other than Wifi issues and what I found helped in such a situation as I hadn't read such a remedy myself on the forum.

Windy Miller A good highlighting of a potential issue, as you say a potential DDOS attack

I’ve not seen any evidence that the GivEnergy portal retries commands when it sends them to the inverter, so things like a ‘reset to defaults’ or maybe even changing the inverter clock between UTC and BST might continue to have timeout issues. May be worth further dropping the poll frequency of your android devices, or at least the less frequently used ones.

I know that if I set the givtcp autorun period to less than 15 seconds then this can cause similar comms issues between the inverter and the portal, especially on older slower inverters

geoffreycoan I have 6 of 7 android apps now polling at 60 second intervals which is the limit of the settings dialog. The other is at 10 seconds as this is the one most viewed. I haven't re-enabled the bbc one as I didn't find it as useful as the android ones. I just had an old iphone and thought it a good use. I'm not seeing failed commands anymore whereas previously I was getting about 1 in 10 commands from WW failing (after exhausting 3 retries over 14 seconds).

Windy Miller I agree but rejecting an individual command with busy rather than collapsing in a heap

I haven't looked but I'm assuming traffic to/from the portal to the inverter is UDP over an encrypted connection hence the use of TCP port 7654. UDP is fire and forget, its a connectionless protocol so no way to reject anything.

The modbus on this kit is slower than a slow thing on a dead day in modern comms terms so it takes a relatively long time to do anything, hence there's a lot more possibility of comms collisions.

Add in local comms (like givtcp/HA) and you can get that to a stage where a fairly high proportion of comms will fail.

Proxy it or poll less are the only viable options with this sort of stuff.

There is a 3rd option that kind of ties in with @DD 's proxy in that you can take advantage of the "spammy" nature of these inverters and use "listen" modes instead. So only 1 device is actually polling the inverter and the rest of the clients simply passively pick up the broadcast messages on the network.
The Async library has this function (and works well) but I'm unaware if any 3rd party apps take advantage of it (GivTCP doesn't use this either)

hoggy It'd work on a flat network but I suspect would rapidly become a monumental PITA to implement/manage while retaining some sanity on anything even slightly more complex.

Proxy seems like the logical solution, which I assume was GE's intent with the app requiring the portal to communicate with the inverter. Didn't really work though as evidenced by this thread 😃

15-20 years ago with the growth of the Internet of Things, comms used to be as bad as this. Nowadays you would implement something that used a higher level protocol that kept a connection open so that it could respond with a busy if necessary. It just shows how old some of the technology is in the GE kit.

I'm not a software engineer (as you can probably tell) but our products (I'm now retired) used to implement a messaging stack that meant you could bidirectionally communicate between nodes through whatever network you had (and it could work on serial ports like Modbus). It was open-source but I can't remember its name. We implemented it precisely because the nodes on the network would get out of step with each other and make wrong decisions. All the nodes had a consistent view of the world even though only connected by a slow network. When these nodes controlled motors it was important they all did the same thing at the same time. Worked much better as a result.

Is this what you mean by a proxy?

Windy Miller Is this what you mean by a proxy?

No, I was thinking more of a device that pretends to be an inverter as far as the network is concerned, but is actually just cacheing the results from the connection to the real inverter.

Windy Miller I'm not a software engineer (as you can probably tell) but our products (I'm now retired) used to implement a messaging stack that meant you could bidirectionally communicate between nodes through whatever network you had (and it could work on serial ports like Modbus). It was open-source but I can't remember its name.

MQTT maybe?

geoffreycoan Doesn't ring a bell. But then, being a hardware engineer, I used to fall asleep during software presentations 😅 . This was more than 15 years ago. I think we ran it on networked MPC561 processors which were used in lots of Engine Control units at the time. Network would have been CANbus.

Windy Miller timeouts are sometimes caused by things other than Wifi issues

Absolutely spot on. Early in the development of my app, I was running two copies (in two virtual machines on the same computer) and noticed one was running well whilst the other had a huge error rate. Realising that the app could conflict with itself, I developed a technique I call "Time Travel" which adjusts the timing when a conflict is detected.

Unfortunately the dongle often can't cope with doing two things at once.

hoggy use "listen" modes instead

This is an area I am currently working on. Even a passive listener has to send keepAlive packets or the dongle will drop the connection. Early days, but I have something working already on my kit, watch this space!