Backgrounds

Recently, I have a lot of Linux servers created. Those servers are really a lot that I'm tired upgrading those every day.

file

And I just keep doing the same thing:

  • Login
  • Run sudo apt update
  • Run sudo apt upgrade -y
  • Run reboot

Why upgrade those automatically?

  • Save time.
  • Get security updates as soon as possible.
  • Packages are always very clean and new, so less painful to upgrade in one time.
  • Historically, running apt upgrade has been very safe. Few accident reported due to apt upgrade.
  • Running apt upgrade usually won't delete existing libraries.
  • Microsoft Windows is updating automatically.

From the above factors, it is true that automatic upgrade is very useful.

However, here's the catch: I find this is not recommended for most Linux users. We will continue the discussion next.

How to configure Ubuntu server to upgrade weekly

To upgrade automactially, first write an upgrade.sh file:

echo "Updating sources..."
sudo apt update

echo "Upgrading packages..."
sudo DEBIAN_FRONTEND=noninteractive apt upgrade -y

echo "Removing deprecated packages..."
sudo apt --purge autoremove -y

echo "Business app upgrade..."
# Some business app upgrade scripts.
# For example, install the latest business app.
sudo reboot

And use crontab -e to edit the profile:

0 2 * * 0 /home/anduin/update.sh # Every 2 AM at Sunday.

But... Maybe you shouldn't do that!

There are several reasons that you SHOULDN'T do that!

Investigation

The most critical question is: as an operation and maintenance, we often need to investigate problems. When investigating a problem, we need to be able to fully reproduce the situation and dependency tree at the time. These are very important for us to reproduce the problem.

If we auto-update, we might mask the real problem.

AirGap stable

For many customers, they need extremely stable services that do not tolerate any changes. For the server, changes come with risk. Imagine we are building the flight control system of an airplane: it must never be changed after the test is passed, because any change could cost lives.

May break when upgrading

The new version may have bugs. If we need to be stable enough, functional updates should be delayed and only security patches and bug fixes should be updated. Automated apt upgrade may upgrade some software to larger functional updates. These functional updates may cause breaking changes and cause business interruption.

Rebooing might be fatal

First of all, the update procedure usually brings a reboot. Many programs have difficulty restarting: the cache needs to be rebuilt after restarting. The cache may not necessarily be fully synchronized. For example: game system. Restarting a game server may interrupt ongoing matches.

When shall we upgrade the server automatically?

  • The system doesn't matter. Tolerate availability degradation.
  • The system is stateless. Rebooting won't affect it working.
  • The system has been configured with a perfect backup\snapshot.
  • The system is always connected to the Internet.
  • The system needs to get the latest functional update.

I recommend using automatic updates only if all of the above conditions are met.