Recovering Home Assistant without a backup

This morning the house circuits that are on my inverter EPS pinged off and then back on again which lost the internet connection and forced a reboot of my home assistant server as these are on the EPS circuit board.

Home Assistant came back OK, but then it pinged off and on and again, and after about 3 or 4 more of these I went and switched the changeover switch so the EPS fusebox was fed directly from the grid and went to look at what HA was telling me. Unfortunately the multiple power cuts had killed my Home Assistant VM and it wouldn't then boot.

Fortunately I use the excellent Google Drive Backup add-on for HA that takes a backup every night and copies it to a dedicated Google drive account I have, ... but when I went to get the latest download I found that the drive was out of space and the last backup taken was on 31st March.

I was able to restore that backup so I lost 3 days of data, but subsequently I worked out how to access and extract the backup taken last night that was in my corrupt HA virtual disk, so here's a brief guide to how to do this if its useful to anyone else.

Needless to say you need to be taking backups, either manually, via the Google drive backup add-on or the Samba backup add-on. I was taking backups they just were not getting saved outside of the HA virtual disk.

I use the free Oracle VM virtualisation software running HA as a VM under Windows 10. If you use a pi or other approach the steps will be different.

1. Download the latest VirtualBox HASS software and create a brand new HA virtual server
   https://www.home-assistant.io/installation/windows#start-up-your-virtual-machine

2. Start the virtual machine and let HA set itself up. http://homeassistant:8123

3. Get to the point where you are ready to start the onboarding https://www.home-assistant.io/getting-started/onboarding/

4. If you have a backup on a NAS drive, Google drive or wherever, download the backup onto the Windows PC, select 'restore from backup' and HA will restore the backup and you'll only have lost the data after the last backup point.

5. In my case I did this OK and HA was now running but had lost all data from 31/3, 1/4, 2/4 and this morning 3/4. I realised that there was more recent backups locked inside the old (corrupt) HA VDI file. Process to get these out is as follows.

6. Download and install the free DiskInternals Linux Disk Reader https://www.diskinternals.com/linux-reader/ (you don't need the pro version)

7. Run Linux Disk Reader. First step is to open the (corrupt) old virtual disk. Drives/Mount Image/Containers, select Virtalbox virtual disks (worth noting that Linux reader does support other virtual disk types).
   Click Next, navigate to where your corrupted HASS VDI is, select it and click Open

8. You'll now see a number of additional volumes for the Linux disks within the VDI. The one you want is the one that contains hassos-data - /mnt/data in my case

9. Double click the hassos-data volume, then navigate through the supervisor and single click the backup folder. We're going to extract all the HA backups as we can't tell which is the latest

10. Click Commands/Save, choose where to save the files to, and then let it extract all the backups to the Windows PC.

11. Having got the most recent HA backups on the PC you can restore that into HA by following step 4 above (or if HA is already running, go to System / Backups, three dots in the top corner, Upload and upload the backup into HA and you can restore it from there).

Hope this helps someone else. I only lost half a day of data as a result of being able to restore from the backup from within the VDI disk

geoffreycoan
You really are spectacularly unlucky with your HA crashes. Maybe it is time to invest in a UPS to back-up your EPS?

geoffreycoan With all the help you have offered us, you didn't deserve this.
You must be relieved to have recovered what you have.
You have written a useful step-by-step procedure for recovery or installation on a fresh installation, which I have copied for reference. Thanks.

I run predbat on a raspberry pi 4 and am thinking about getting a pi 5.
I will need to start afresh but now know how to get my predbat installation onto a new pi.

I have just upgraded HA to 2014.3.3 and predbat still runs.
I have added a copy of the backup onto my Google drive using the Google Drive Backup add-on for HA.
Thanks for reminding me about backups ....

Rob

The moral of the story - which we should ALL do, but don't - is test your backups regularly.

PianSom LOVE my UPS, keeps 2 PC's with monitors, modem and home phone going for 20-25 minutes which is enough time to shut down if its an actual power outage.

Vestas Or at least check the available space where you back them up to regularly

Jellybaby Part of the annoying thing was that I had run out of space on the google drive, had received an alert about it, cleared some space and thought all was well. But seems that it had run out of space again and I didn’t get a subsequent alert as the backup stale binary sensor hadn’t been reset after the first error.

I need to keep less of the really old backups on Google drive, or maybe just create another google account for long term storage.

Will look at APC UPS’s on ebay as well as they seem quite reasonably priced used

geoffreycoan thanks, i never tried the backup but i am glad it would work if and when needed, i am using google drive backup option.

in your post you mentioned, you lost data. Does this mean you only lost history of entities state values and all devices/entities/automations were restored?

DeepS

geoffreycoan oof, cursed google slacking on the notifications.

For UPS I went back and forth between Cyberpower and APC and I went with the
CyberPower CP1500EPFCLCD-UK PFC Series, 1500VA/900W, 6 Outlets (4 IEC + 2 UK), 2 USB Charging Ports, AVR, Active PFC compatible, Tower

Had it since December 2022 and last week something tripped the main breaker in the house but none of the little ones (usually happens a few times a year and we don't know why) and it ticked over fine,

I got the front display showing grid voltage, I know its not the main grid voltage just whats coming out that plug because GE portal reports +3v and I go with that since its coming off the main feed.

No USB HDD attached for backups as well?

deepanshus when I restored from the backup, everything came back, all the config, automations, entities, dashboards, appdaemon apps, predbat config save files, even a custom shell script I wrote to save power up events into predbat, all came back fine.
All I lost was the event state history from midnight to about 1pm yesterday when I finished the restore. I can live with that.

@Jellybaby I probably only need a small UPS, just enough to run the internet router and my HA PC. Don’t need lots of outlets on it. Good idea about using a USB HDD for extra backup storage, I have a couple in my desk drawer I never thought of using.
Annoyingly I have an old NAS drive on the network with all my music collection on it, but I’ve forgotten the windows share password and then I couldn’t get it working on my mac a few years ago so gave up. Am loathe to reformat it though !

I updated HA yesterday to 2024.3.3
Predbat is running fine and I am hopeful that the upgrade has sorted out some of the strange predbat status report warnings I was receiving.
However, my Predbat settings seemed to have been switched back to default settings. I noticed this when I spotted that I was no longer in Expert mode. I have spent some time today putting tweaks back (if I could remember them). Some of the replies in this thread have been useful though and I have scoured through the excellent documentation once again.
The only settings that survived were within my apps.yaml file.
Perhaps I could have restored Predbat if I had just backed that up prior to updating, rather than a full backup. But I should now have a clean start.

Is there a way to copy the Predbat settings?

Rob

Rbor
Actually, I just updated also. I had no issues. But then I am slightly eccentric in that I don't run HAOS, I run HA in Docker, and AppDaemon (with Predbat) in another Docker container.

HAOS is basically just Docker, with HA managing the Docker containers (and calling them Add-ons). To lose data from an Add-on is pretty serious stuff. Updating the top level should not lose data in a subordinate or sister container. And it is odd that apps.yaml survived.

In any case, to answer your question - yes, it is possible (and easy) to backup Predbat settings. As ever, see the excellent docs

There’s a HA dropdown to save/restore your predbat settings https://springfall2008.github.io/batpred/customisation/#saving-and-restoring-predbat-settings

I have seen occasional reports of predbat settings getting lost during upgrades of HA, but never seen it myself. It may be due to the way appdaemon runs in one container and calls out to HA to hold the actual predbat config as HA entities. Don’t know what causes it.

2024.4 came out today. I’ll wait a few days before upgrading

PianSom
geoffreycoan
Thanks both.
Embarrassingly, I can't believe that I missed this, especially as I have used it previously!
I have saved my current settings and have deleted the old settings.
I never looked at the file before and it contains everything that I need.

I will wait for Geoffrey's rundown on his HA4.4 update.
I will copy my settings before carrying out the update with full backup selected.
I will then save a copy to the Google drive using Google Drive Backup add-on.

I finally now have my appdaemon.yaml file correct. I had the incorrect url set.
I now finally get 9 versions of predbat.log.

Learning predbat is a long journey but I am getting there.

Looks like I may have a busy weekend if the predicted Agile plunge prices materialise.
I just need to maximise my usage of electricity and hope for some negative rates.

Rob

geoffreycoan
Can you expand on your step 4?

4. If you have a backup on a NAS drive, Google drive or wherever, download the backup onto the Windows PC, select 'restore from backup' and HA will restore the backup and you'll only have lost the data after the last backup point.

When you restored from your google drive backup, did you point HA directly to the google drive file which is a compressed .tar file?
I have decided to update my raspberry pi setup and have just purchased a Raspberry pi5 and a fast SSD drive.
Having spent most of today setting up my new setup, I am at the point of trying to transfer my HA set up, happily running away on my pi4.

Thanks

Rob

Rbor You basically have to just get the .tar file (don’t uncompress it) into Home Assistant in the /backups directory.

Several ways of doing this.

• Using the Google drive add-on within home assistant you restore the backup from the Add-on UI
• If you have Samba share running on Home Assistant you can copy the file into the correct folder
• Or within HA, go to Settings/ System/ Backups, click the 3 dots in the top right corner and ‘Upload backup’ to import the backup file into HA

The easiest for you Rob may be to:

• create a brand new HA instance on your new Pi 5
• install and setup Google backup add-on
• on your old Pi, take the backup and let it copy to the cloud
• shutdown your old Pi
• On the new Pi, using the Google backup add-on, download the backup just taken onto the new Pi
• Then restore that backup on your new Pi

geoffreycoan thanks.
I will let you know how I get on.

I have managed to set up a Samba share on my pi. What would we do without informative websites and videos?
I just need to copy my initial installation from an SSD card onto the SSD drive, initiate an HA installation, and link to a fresh Google drive backup I can move that onto the pi.
I am going to run this headless from my Mac, iPad, etc. on ssh. …. and all on a footprint not that much bigger than a credit card with 1 lead attached for power.
I then intend to repurpose my old pi for my old record collection which I have slowly been digitising.

I have had my own problems as the comms unit on my meter has stopped working. Octopus are sending out an engineer to fix (or replace it) but I am not receiving my 'real data' currently. Luckily the meter stores everything for 13 months but it may take a month to get the engineer visit and for my data to be updated.

Rob

You can't just move HA from Pi4 to Pi5, its entirely different architecture. It won't work and you'll spend many hours in the cli trying to sort it out. It definitely won't work if you have a NVMe drive.

Best bet is to try what geoffreycoan suggested re new install.

Vestas I received the pi5 and NVMe drive on Tuesday and I have set it up, with a clean 64 installation on a 32Gb SD card. I then tweaked required settings and have set up a Samba share. I have then been able to copy my Google drive backup to the SD card.
I have copied the 32Gb contents over to the NVMe.
My plan is to then follow Geoffrey's plan, restoring from the backup file during HA onboarding.
I have tried running directly from the NVMe and it is so much faster than an SD card, almost instantaneous.

Alternatively, I may continue to run HA with predbat on my old pi4, which I have hooked to an unused 500Gb drive via USB3. I have other plans for purposing either the pi4 or pi5.

Hope my plan looks sensible. 🤞

Rob

Rbor You might be surprised how fast a (suitable) SD card is on the Pi5 - they basically doubled the bandwidth so there's not a lot in it between SD and NVMe for running HA in terms of read/write times (especially with some of the 8GB memory being used as cache).