The 5 minute 'snapshot'

As I understand it, using the as yet unreleased GivTCP component you can get a significantly higher frequency of update from the Inverter than the portal currently allows for, which is great, but it leaves me with a question...

When the Inverter pushes data back to the GivEnergy cloud platform every 5 minutes, why does it just send a snapshot from the moment before, why doesn't it send any logged data since the last interval?

One reason I could think of is that that could be a lot of data so why not take a snapshot every 30 seconds / minute and push that accumulated data, so the portal would still update every 5 minutes, but you'd be able to see a bit more of what happened within those 5 minutes?

I think you have already answered your question, that would be a lot of data. In my opinion the 5 min snapshot for a lot of the values is a waste of time, especially for current consumption, PV etc. Only the cumulative data is useful.

chris_p With regards the 'a lot of data' statement, I was more meaning if it sent everything logged by the Inverter over a 5 minute period, assuming the Inverter works on a one second frequency.

I've just checked the API call the Meter Data screen makes and for a whole day's worth of data it amounts to _250kb, so I can't imagine the bandwidth overhead of sending several snapshots from a five minute interval being particularly high, even when scaled out over their entire customer base.

Damien Well someone at GivEnergy has done the calculation and this can only be a cost issue with regards to whatever data center they use.

Not sure where they are install number wise (my user ID puts me in the 1,500's back in 2020 so must be pushing >5K now. If anyone with a new install would like to share their rough ID no. (not the exact one) then that would be a clue) - even at 250kB that's still 1.2GB a day of data at 5min intervals per day @5000 users.
It'll only get worse now they are ever more popular.
No ongoing subscription & a product expected life of >10years remember.

hoggy Your ID? No idea what number I would be. Going by my EON username I got I would be customer 152*** lol

THALL Think I ended up with my number from the API at one point but forget where.
Easiest way to find it is: old portal > Account Details > Actions > Edit > then see what the digits are after "/editUser/" in the URL at the top I guess.

Damien detailed data between pushes is not stored on the inverter, thats why you get no data when wifi goes down, you dont get a recovery packet when conection is restored. Firmware updates and probably new local hardware in the dongle is required to deliver that capability.

The cummulative registers are just that so they are ok, the instant registers are lost after the next write to them

as per @hoggy , just from an individual persepective....
Each user isn't a lot of data. Add it up for all users, for all days, every day after commissioning. It does add up. 250kb at 5min = 15000kb ( 14.6Mb ) per day = 5346mb (5gb) a year. Then multiple for the number of years you have your system. Is guarenteed for 10 years so use that. 50Gb. Typical cloud storage costs for 50Gb are about $15 a year. Ot a vast amount but a perpetual cost with zero revenue to pay for it. Multiple that up by the number of customers and its understandable why it may be an issue for sustainability

Maybe a subscription service that provides higher refresh rates at a cost to cover storage may become an option if people really want it and cant use local mechanisms?

My inverter card says 3.9kw - the other card 3.3kw charge and 0.8kw discharge. Which to me sounds about right.

Seye I'm not sure I'd want it constantly pooling up data between intervals on some presumably limited write cycle flash storage inside the inverter (unless you could switch out the storage via an SD card in the dongle)
Other than the cumulative totals, settings and serial numbers I can't think of any other data that is persistently stored.

Tesla found this out the hard way when their 1st gen MCUs ate it after too much writing, corrupting their limited storage.

Frankly the sooner its easy for someone to just install a pre made app on a local machine and have real time readouts in a browser the better. I'm computer literate far more than most and I look at the thread about local control and I'm just lost. I don't think anyone yet has just put together a simple to follow walkthrough for people to use in normal English.

hoggy

For your clue sluthing...

CE2051 June 2021
CE2133 December 2021

dbt85 there's a document coming shortly that runs through setting up docker, pulling GivTCP down, linking inverter IP to it and setting up variables.
@anabanet thanks for that although as they start with letters I think that must be inverter serials? not user IDs, I've not worked out the numbering pattern with those (mine starts SD...) + I guess it depends on if the stock has been sat for a while.
Think the only place you can suss out your userID is the old portal thing I mentioned above.

hoggy 3875 I joined GE 31st July 2021

THALL 58XX here, and that was late November, 2021.

adamdoran interesting, so must be well over 6k now then.
As a very rough guide if we assume 80% of this 6k have just the 1x 8.2 battery and nothing else (which would be a reasonable approximation for to cover the older DOD ones and those that have more or less) that puts us at nearly 40MWh of distributed storage.
That's pretty good going in a short space of time!

hoggy

I look forward to it.

hoggy according to our installer, GivEnergy is one of several vendors (Tesla being the only other one I remember) whose equipment can be instructed to discharge remotely by the National Grid, in the event it's struggling to maintain the necessary frequency, so that theoretical 40MWh becomes a very useful capability for the nation.

Seye they migrated to AWS during the beta portal phase IIRC.

Think that clue is hiding in the API stuff but I do recall having a conversation about it with one of the dev team at some point.

It is cost driven, a combination of data and data calls per day soon gets expensive for them and ultimately its not a huge company over charging for hardware either.

I know there is an intention to offer a subscription service for higher frequency data calling for those that want it to fund those server side costs and I believe the intent was a variety of packages to suit differing users demands.

That said all of that takes time to implement and develop, with plenty of items that were touted still unresolved and new ideas still coming in targeting the resources of the dev team is a tough balancing act.

Although an option to output more frequent data to a local network setup/storage would be great and gets around all of this.

yandards yep, it appears they're using AWS Elastic Beanstalk. I noticed the DNS name beta.givenergy.cloud is an alias to AWS recently:

% dig beta.givenergy.cloud +short
betaportal2-env.eba-vvmbmw2s.eu-west-2.elasticbeanstalk.com.
...

I've been recording the five second metrics from my inverter for nearly two weeks now, and the InfluxDB database is 23MB, which means I'm never likely to throw any of it away, but scale that up to six thousand users, and things would get really expensive. The "high resolution" data is mostly good when you're curious about exactly what's happening when the sun starts shining / a cloud floats overhead / the kettle is switched on, but beyond that I think summarised data will do nicely. (with the small caveat that it would be better if the five minute values were cumulative, rather than simply snapshots.)

hoggy #3500 - Yes I sent the serial numbers....

THALL 3500 15th July 2021