Core update and GivTCP/MQTT problem

Core 2023.5.4 on my RPi2 caused a lot of hassle with GivTCP. To cut a long story short all GivTCP functions were lost as messaging to the MQTT Broker was refused after the core update. The error logged said “Connection refused: not authorised”. All local users should be able to access the broker without credentials, adding username and password to MQTT config didn’t help. I installed MQTT Explorer and it could connect quite happily to the broker, so I think this is a core problem.
I’ve reverted to a 2023.5.3 backup and everything GivTCP is restored.
I suggest running a full backup prior to changing to 2023.5.4 core.

x5toledo Thanks for the heads up.

I've been running 2023.5.4 since its release last week, no issues whatsoever with MQTT or GivTCP. Running on a Pi4 though so perhaps that's the difference. I've never tried HA on a Pi2 though, I didn't think it would work at all so am surprised. Pleased to hear it, but still surprised!

x5toledo mine seems to have updated fine on my HP thin client, as SpeakToTheGeek mentions it might be the Pi2 for some reason

I'm also running 2023.5.4 Core, and no issues for me, although I do use separate dockers for HA, MQTT and GivTCP.

I've never been a massive fan of the PI2 and used it really for expedience and to see if Home Assistant was any use to me. Anyway, it decided to have a server Error 500 sometime this morning, so it's the end of the road. I've ordered a thin client with proper mSATA, so will rebuild now I know what needs to get done with Home Assistant.
Having energy management automations such as checking if the car is plugged in and then switching on the Zappi at certain levels of battery is very useful at this time of year. I've not been able to control the battery, GivTCP has always been something that doesn't act and writes to error log - again, almost certainly due to the antiquated hardware! So I'd like to get some GivTCP / Intelligent Octopus sense during the autumn. Getting a flag for car charging can be done through the carmaker's APIs, I need to stop the battery from discharging when the car is juicing.
I also have found Home Assistant to be invaluable for home brew management :-)

x5toledo I run Ubuntu 22.04 and Docker on one of these:

https://www.seeedstudio.com/ODYSSEY-Blue-J4125-v2-128GB-p-5532.html

It runs all my dockers including Portainer, AdGuard, GivTCP, HomeAssistant, HomeBridge, MQTT and Watchtower.

Never missed a beat and powered by USB-C PD.

I used to use a Pi 4, but this thing just beats it in every direction.

ProximusAl That Odyssey looks a proper piece of kit! I have to say that Home Assistant sitting on Ubuntu would be my preference due to experience with this flavour of Linux.
Do you run Supervised? If not, do you miss Supervisor? TIA

x5toledo No I run core in a docker. It does everything I need with HACS installed in HASS.

I run HA on a small intel NUC- loaded Proxmox (Dead easy, just download a file from the site, install onto a USB, boot onto it from the nuc,). Once installed its all done through the web interface.
Then follow these instructions to install HA onto proxmox as a VM https://community.home-assistant.io/t/installing-home-assistant-os-using-proxmox-7/201835
Fully supported and you can run other things on the NUC in dockers - I tend to use portainer on each docker as its easy to see the configuration. I run GivTcp in a container rather than an addon, but really only because that's originally how it was done.

Thank you @anglefire and @ProximusAl - I’m back up and running. I made a VirtualBox installation to make all the mistakes in then clean installed Ubuntu on the thin client. Docker installation followed, along with Portainer - it’s been a while since using docker for me, I can fully understand why you recommended Portainer! Home Assistant install is easy after this.
GivTCP was harder for me as so long since using Docker, but I successfully packaged one of the longest docker run commands ever to make this happen (I wasn’t inclined to learn Compose); Eclipse MQTT seemed straightforward; deConz put up some resistance but that’s because things have changed since first using it with the Pi.
With HA data mapped to a Docker volume at /var/lib/docker/volumes/homeassistant_data/_data configuration.yaml is editable with nano so that was restored once I checked the validity of all sensors. I could have done the same with automations but this was a time to purge a few and test each one.
Finally, I dug out some of the Lovelace dashboard hidden files from a tar archive and used these to recreate the dashboards. Apart from some sensor renaming, this all went quickly.
Everything is working well. Today will be all about backups 🙂

I’ve a portainer stack file that you could have used as a start for givtcp - oh well you are working now so all good!

I just use this:

docker run -d \
--name GivTCP \
--net=host \
--restart=always \
-e NUMINVERTORS=1 \
-e INVERTOR_IP_1=192.168.76.150 \
-e NUMBATTERIES_1=2 \
-e MQTT_OUTPUT=True \
-e MQTT_ADDRESS=127.0.0.1 \
-e MQTT_USERNAME=mqttuser \
-e MQTT_PASSWORD=mqttpassword \
britkat/giv_tcp-ma:latest

Obviously changing the inverter IP address and NUMBATTERIES

ProximusAl Blimey, that's short!

This is mine - with the security stuff removed.

version: "3.9"
services:
GivTCP:
image: britkat/giv_tcp-ma:latest
ports:
- "1883:1883" # MQTT
- "6345:6345" # REST (add increments for each invertor)
- "3000:3000" # This should match the WEB_DASH_PORT ENV below
- "6379:6379" # Redis for job queueing
- "9181:9181" # RQ Dashboard to view job scheduling
environment:

Critical to update

- NUMINVERTORS=1 # Set this to the number of Invertors in your setup, then replicate the next two lines for each invertor (changing the last number of the ENV)
- INVERTOR_IP_1=192.168.1.137 # Set this to the IP address of your Invertor on you rlocal network
- NUMBATTERIES_1=1 # Number of battery modules installed and connected to the above invertor
- MQTT_OUTPUT=True # "True" if you want to publish your data to MQTT, "False" otherwise
- MQTT_ADDRESS=192.168.1.170 # IP address of an existing MQTT broker, or leave as "127.0.0.1" to use the internal broker
- MQTT_USERNAME=mqtt-user # Username of your existing broker, if needed. Not required for internal broker
- MQTT_PASSWORD=PWD # Password of your existing broker, if needed. Not required for internal broker
- HOSTIP=192.168.1.154 # External IP address of the docker host (needed for web dashboard)
- CACHELOCATION=/config/GivTCP # Location of cache data, this folder can be mapped to a persistence storage outside the container
- TZ=Europe/London # Set to your Timezone

Optional

- MQTT_TOPIC= # Root topic to publish data to for Invertor 1. If left blank it will default to GivEnergy/<invertor_serial_number>
- MQTT_TOPIC_2= # Root topic to publish data to for Invertor 2. If left blank it will default to GivEnergy/<invertor_serial_number>
- MQTT_TOPIC_3= # Root topic to publish data to for Invertor 3. If left blank it will default to GivEnergy/<invertor_serial_number>
- MQTT_PORT=1883 # Port of your existing broker, leave as "1883" for internal broker
- LOG_LEVEL=Info # Level of logs to be reported: "Error", "Info" or "Debug"
- PRINT_RAW=True # If True this will publish all inverotr data unprocessed as well as standard data
- SELF_RUN=True # If True the container will self-run and connect and publish data. If "False" the you will need to trigger externally via REST
- SELF_RUN_LOOP_TIMER=10 # Wait time between every read command to the invertor
- INFLUX_OUTPUT=False # "True" if you want to publish your data to InfluxDB, "False" otherwise
- INFLUX_URL= # URL of an external Influx instance
- INFLUX_TOKEN= # Access Token for your Influx instance
- INFLUX_BUCKET= # Data Bucket of your Influx instance you want data sent to
- INFLUX_ORG= # Influx instance Organisation
- HA_AUTO_D=True # If True (and if MQTT_OUTPUT is True) this will publish Home Assistant Auto Discovery messages to the broker
- HADEVICEPREFIX=GivTCP # Defines the preficfor all entities in Home Assistant for invertor 1
- HADEVICEPREFIX_2=GivTCP2 # Defines the preficfor all entities in Home Assistant for invertor 2
- HADEVICEPREFIX_3=GivTCP3 # Defines the preficfor all entities in Home Assistant for invertor 3
- PYTHONPATH=/app
- DYNAMICTARIFF=False # When set to true DAYRATESTART and NIGHTRATESTART times are ignored and tariff rates are changed by a dedicated entity/function
- DAYRATE=0.240 # Price in £/$ for your daytime electricity per kWh
- NIGHTRATE=0.240 # Price in £/$ for your night time electricity per kWh
- EXPORTRATE=0.150 # Price in £/$ for your export electricity per kWh
- DAYRATESTART=04:30 # Time in HH:MM when your day time\expensive tariff kicks in
- NIGHTRATESTART=00:30 # Time in HH:MM when your night time\cheap tariff kicks in
- WEB_DASH=False # Enable the web dashboard
- WEB_DASH_PORT=3000 # Port to serve the web dashboard on
- SMARTTARGET=False # Enables the PALM capability, requires the three ENV below to be set
- GEAPI= # API Key for the GivEnergy Cloud
- SOLCASTAPI= # API key for Solcast
- SOLCASTSITEID= # SiteID for Solcast site (first array)
- SOLCASTSITEID2= # SiteID for Solcast site (second array)
- DATASMOOTHER=low # Set the data smoothing to most agressive setting (High, medium, low, none)
- PALM_WINTER=01,02,03,10,11,12 # Comma delimited list of the winter months (01=January, etc)
- PALM_SHOULDER=04,05,09 # Comma delimited list of months in which consumption can vary so SOC calculation should be more cautious
- PALM_MIN_SOC_TARGET=25 # Lower limit for state of charge (summertime)
- PALM_MAX_SOC_TARGET=45 # Lower limit for SoC limit in shoulder months
- PALM_BATT_RESERVE=4 # Battery reserve for power cuts (minmum of 4%)
- PALM_BATT_UTILISATION=0.85 # Usable proportion of battery (100% less reserve and any charge limit) on a scale of 0-1
- PALM_CHARGE_RATE=2.5 # Inverter charge/discharge rate in kW
- PALM_BATT_CAPACITY=10.4 # Nominal battery capacity in kWh

restart: always
privileged: true

anglefire I run mine on the host, not port forwarded bridge stuff. So much easier

ProximusAl This is essentially just what is written on the website!
GivTcp is on a different mini pc to where HA is hosted as it happens - both run proxmox and are in a cluster so easy to manage or even move.