How to update Nextcloud

Intro

Nextcloud

Nextcloud is an open source server suite. It has a variety of uses from hosting collaborative office software, video chat, to files, or your calendar and contacts. It's self-hostable and open source, with a focus on being community-driven.

I've been using it for 4 years now and needed to do a bit of maintenace. Here's how that went!

Backstory

I was using NextcloudPi, which is focused on being easy to install without much hassle. However, I had to update the OS since Debian Buster was reaching end of life. Instead of using the broken ncp-dist-upgrade command (always make backups!!), I decided it would be a good time to learn how to manually set up a Nextcloud server on my own.

It wasn't as hard as I thought, but there were a few hiccups along the way.

The Process

Backing up the server data

The first step was to buy another harddrive to specifically backup my nextcloud installation's data since I only had one 8TB drive running (I keep backups of my files by other means, don't worry lol). Once I did that I felt more confident about proceeding with the upgrade process (although it doesn't actually touch the data it turns out.)

I used rsync to copy everything over. You can do it from the pi, but it was 4x faster to just plug in the drive to my desktop PC so I did that instead since otherwise I would have had to use one of the USB2.0 slots on the Raspberry Pi.

Read the manual!

Once I had the data backup sorted I took my time reading Nextcloud's administrator docs all the way through. It was definitely better maintained than the NextcloudPi docs, which was another reason I decided a manual installation might be a better bet than using a one-size-fits-all script maintained on volunteer labor. Nothing against that, there's just limits when relying on it, especially for my setup that veered from the default.

The most helpful sections of Nextcloud's docs were the sections about installation and server configuration, and migrating a server, how to backup/restore an installation. The OCC commands section was also important for understanding how to interact with Nextcloud itself.

There's some details on hardening the server, tuning performance and memory caching that I would also recommend reading about, but aren't strictly required to get things running. However, it will improve your server a lot.

Lastly, if you're updating from an old server like I did, be sure to replace the section in the URL for the docs where it says "stable" or "latest" with the actual version number of your instance. In my case I was still on 23. The instructions are mostly the same across versions, but there are slight differences that make it worthwhile to be safe.

Upgrading the server

Now I was ready to upgrade the server! It basically went like this:

  1. Put the server in maintenance mode
  2. Back up the current installation and data so that I could go back to it if needed.
    • This includes the mysql database (mariaDB), nextcloud folder in /var/www/nextcloud and the data drive.
    • Be sure to transfer this to the new installation so you can import them, but don't modify the original drive at all.
  3. Install a fresh copy of the latest RaspberryPi OS (formerly Raspbian) on a new drive.
    • Enable SSH by creating a file on bootfs named ssh and a separate userconf.txt to create a new user.
    • Note - userconf.txt contains a single line:
      • username:(encrypted password by running openssl passwd -6)
    • Once you're in you can run sudo raspi-config to set things like GPU memory to only 16MB and localization.
    • Be sure to run sudo apt update && sudo apt upgrade once you're in!
  4. Install all the PHP dependency packages, mysql sever, setup the Apache server and Let's Encrypt certbot on the new install.
  5. Restore the database from the previous Nextcloud installation.
    • This will take a while if it's big! Especially on a microSD card.
    • My ssh connection would get stuck and eventually timeout, but you can use htop on a separate shell connection to see if mariadb is running, once it's done you're good. Didn't have that issue on SSD.
    • If you want to be extra sure that the database was fully imported you can run this query and see if it's in the ballpark of your backup file's size.
      • SELECT table_schema "nextcloud", ROUND(SUM(data_length + index_length) / 1024 / 1024, 1) "DB Size in MB" FROM information_schema.tables GROUP BY table_schema;
  6. Restore the previous installation's Nextcloud instance using rsync -Aavx nextcloud /var/www
    • If you make any changes, update that in nextcloud/config/config.php
  7. If data-fingerprint is set in nextcloud/config/config.php then run:
    • sudo -u www-data php occ maintenance:data-fingerprint
    • This will let clients know that the data has been restored, don't redownload it.
    • Note: You need to either use the full path (/var/www/nextcloud/occ) or cd into /var/www/nextcloud to run occ without the full file path.
    • To cd into /var/www/nextcloud you can access the root prompt with sudo su - since the path is protected.
  8. If you have memory caching like I did then you'll want to install redis and configure /etc/redis/redis.conf
    • Set unixsocketperm 770
    • Uncomment and copy the socket file location (I set mine to /run/redis/redis-server.sock)
    • Update host for redis in nextcloud/config/config.php to the socket file.
    • Run usermod -a -G redis www-data so your server (Apache for me) has access to it. (This may vary by system, see docs).
  9. Set up fail2ban and do any other hardening
  10. Restart the apache, redis-server and fail2ban services
    • Either sudo systemctl restart [service-name]
    • or sudo service [service-name] restart
  11. Check your Nextcloud and web server logs to see if there's any errors.
    • This can be set in nextcloud/config/config.php and I set my logs to /var/log/nextcloud.log
    • Apache has an error log in /var/log/apache2/error.log
    • May be worth checking /var/log/syslog for good measure.
  12. Troubleshoot any errors in the system logs.
    • These may seem complicated, but it can be as simple as a bad configuration, not having set the right permissions or a missing dependency.
    • You can disable optional things (like memory caching) and work backwards until errors stop being thrown, then figure out what's causing the issue.
  13. At this point I disabled maintenance mode and my server was working!
    • However, we still need to update nextcloud itself.
  14. Re-enable maintenance mode before updating Nextcloud.
  15. I followed the process to manually update nextcloud, but you could try the automatic updater.
    • MAKE SURE YOU MEET THE MINIMUM REQUIREMENTS FOR EACH VERSION BEFORE UPDATING!
    • Remember to check that you're reading the right version's manual (see number/name in doc's URL)
    • I stopped at Nextcloud version 25 since RaspberryPi OS doesn't have the PHP 8 packages in the default repository yet. You could install a custom PPA to get it, but I'll just wait until it's upstream when they've updated to the latest Debian stable (likely October 2023), since I'll have to update all that again anyway.
    • To bring over old 3rd party apps from previous nextcloud versions I used rsync -Aavx --ignore-existing nextcloud-old/apps/ nextcloud-new/apps/

In theory that's the whole process! Now, I have successfully managed to get everything updated.

However, I did have some setbacks...

Setbacks and Troubleshooting

NextcloudPi

The first setback I ran into that forced me on this process in the first place was that the NextcloudPi updater didn't work. Before running it I used dd to backup my installation, and when it broke everything after the update I was able to re-image my drive back to the initial state. That allowed me to come back to this process when I was finally ready months later.

Migrating the server

When starting I wasn't sure if I needed to install a fresh instance of nextcloud first, but that's unnecessary. Basically once all the dependencies are installed in setup (i.e: php, apache, etc) the old instance's nextcloud folder can be copied into /var/www and it should work. Just restart the web server after it's finished copying.

NextcloudPi used redis for caching and used a password. Newer versions of redis just use ACL (user permissions) to determine if a user should have access to the cache. Long story short, I just removed the password from Nextcloud's config.php since it was the old way of doing things and added the www-data to the redis group so the apache server could use it.

From there I resolved any issues that the admin panel showed (when you log in with the admin user on your Nextcloud server through the web and go to General View under Admin Configuration). It's pretty good about providing a guide link when available.

Drive imaging

One issue that was really weird was when I tried to move from the microSD card I was building the new installation on to the SSD that I use for better speed, dd wasn't working properly. As far as I know the swap file in /var somehow breaks things and I still don't know what went wrong. fsck and ddrescue were unhelpful.

In the ended I decided to just go through the migration process of installing a new copy of Raspberry Pi OS on the SSD this time and copying over the appropriate files from the microSD card which ended up being faster. Your mileage may vary, but if you figure it out let me know!

What I learned

The biggest things I learned from this process are:

  • logs are super helpful
  • by manually installing Nextcloud I learned how it works better.

Logs

The logs allowed me to see immediately where an issue was appearing. Some things were easy to diagnose, like maintenace mode being enabled made it so that clients trying to connect to the calendar weren't able. Or the fact that I didn't have an accessible temp folder for tempdirectory in the Nextcloud config. Others were a little less clear, but at least helped point me in the right direction like when I was getting redis errors (basically permissions weren't set right).

Nextcloud

By walking through the process to manually install Nextcloud I now have a better idea of the full stack that runs Nextcloud as software. It relies on the classic "LAMP" stack (Linux, Apache, MySQL, PHP), as well as a few other tools like memory caching.

In short that means Nextcloud runs on:

  • a Linux OS
  • Apache for responding to web requests.
    • Serves the PHP code of Nextcloud.
  • MariaDB is the database for everything from files to users.

These components can be swapped out for others (hence "the stack"), but they all come together to create the server that responds to requests. This was a helpful experience to get a better grasp on that.

It's also easier to understand where I should make configuration changes now that I don't have NextcloudPi intervening in ways that are unclear. It would reset certain configurations because it overrides them with whatever was set in ncp-config. Now I can just configure each part of the stack in the way they were intended by changing files in /etc.

Contact Me

Currently opens your default mail client.