Ubuntu Mirror
Why do this?
I have many Ubuntu servers in my own data center at home. They need to update software packages every week.
However, if every server connects to the Ubuntu official source, it will waste a lot of bandwidth.
Therefore, I decided to set up an Ubuntu mirror on a server at home.
Step 1 - Get an Ubuntu server
Here, you can create a local server or purchase an Ubuntu server from a cloud service provider.
You can buy a pure server at Vultr, Azure, or DigitalOcean.
After you installing a pure Ubuntu, or buying a new cloud Ubuntu server, you need to first finish the basic authentication configuration.
Follow instructions here to:
- Delete other accounts.
- Create your own account instead of root.
- Disable password login and force to use SSH key to log in.
- Disable root account.
- Enable sudo without password.
- Enable firewall.
- Adjust timezone.
- Enable BBR
Best-practice for authentication after creating a new Linux server
Of course, you need to make sure that this Ubuntu server can access the Internet and can be accessed by your other servers.
Step 2 - Choose the source you want to mirror
Obviously, you can mirror the official source: http://archive.ubuntu.com/ubuntu/.
But if you are in China, you may find that the speed of the official source is very slow.
Therefore, you can choose some domestic sources, such as:
http://mirrors.aliyun.com/ubuntu/http://mirrors.ustc.edu.cn/ubuntu/http://mirrors.tuna.tsinghua.edu.cn/ubuntu/http://mirrors.sohu.com/ubuntu/http://mirrors.163.com/ubuntu/http://mirrors.cloud.tencent.com/ubuntu/
These service providers generally have prominent pages that prompt you to use their sources, such as Tencent's https://mirrors.tencent.com/help/ubuntu.html.
It is recommended to test the speed and latency of each one using aria2c, ping, and wget.
Step 3 - Start Mirroring
First, you need to install a software called apt-mirror.
sudo apt-get install apt-mirror
Then, you need to edit the /etc/apt/mirror.list file to specify the source you want to mirror.
sudo vim /etc/apt/mirror.list
Here, modify the mirror.list file to specify the source you want to mirror.
set base_path /var/spool/apt-mirror
set nthreads 20
set _tilde 0
deb http://mirrors.cloud.tencent.com/ubuntu/ jammy main restricted universe multiverse
deb http://mirrors.cloud.tencent.com/ubuntu/ jammy-updates main restricted universe multiverse
deb http://mirrors.cloud.tencent.com/ubuntu/ jammy-backports main restricted universe multiverse
deb http://mirrors.cloud.tencent.com/ubuntu/ jammy-security main restricted universe multiverse
clean http://mirrors.cloud.tencent.com/ubuntu/
In general, adding the above content is enough.
Where:
- main is the main software package
- restricted is the restricted software package
- universe is the community-maintained software package
- multiverse is the non-free software package
- jammy is the code name for Ubuntu 20.04. If you need other versions, you can refer to
- jammy (22.04)
- focal (20.04)
- bionic (18.04)
- xenial (16.04)
- focal-updates is the updated software package
- focal-backports is the backup software package
- focal-security is the secure software package
- clean is here to clean up some unnecessary software packages
After the modification is completed, simply run the apt-mirror command to start Mirroring.
After the Mirroring is completed, you can see the files you have downloaded. They will be placed in the /var/spool/apt-mirror directory.
These files may be several hundred GB, so you need a relatively large hard drive. This process may take several hours.
sudo apt-mirror
Step 4 - (Optional) Mirror ppa
As we all known, http://ppa.launchpad.net/ is extreamly slow. You can also mirror it.
For example, I'm mirroring Nextcloud client and Firefox. Simply add this to your /etc/apt/mirror.list:
deb http://ppa.launchpad.net/nextcloud-devs/client/ubuntu jammy main
deb http://ppa.launchpad.net/mozillateam/ppa/ubuntu jammy main
If you want to mirror both AMD64 and i386 versions:
deb-amd64 http://ppa.launchpad.net/nextcloud-devs/client/ubuntu jammy main
deb-amd64 http://ppa.launchpad.net/mozillateam/ppa/ubuntu jammy main
deb-i386 http://ppa.launchpad.net/nextcloud-devs/client/ubuntu jammy main
deb-i386 http://ppa.launchpad.net/mozillateam/ppa/ubuntu jammy main
Now run sudo apt-mirror again!
Step 5 - Configure your web server
Obviously, in order for your other servers to be able to access these software packages, you need a web server.
Here, I use Aiursoft.Static to create a simple web server.
sudo apt install dotnet8
sudo dotnet tool install Aiursoft.Static --tool-path /opt/static || sudo dotnet tool update Aiursoft.Static --tool-path /opt/static
This will install Aiursoft.Static in the /opt/static directory.
At the same time, the above script can also update Aiursoft.Static.
In order for the /var/spool/apt-mirror directory to be accessible by Aiursoft.Static, you need to modify the permissions.
sudo chown -R apt-mirror:apt-mirror /var/spool/apt-mirror
find /var/spool/apt-mirror/ -type d -print0 | sudo xargs -0 chmod 0755
find /var/spool/apt-mirror/ -type f -print0 | sudo xargs -0 chmod 0644
This setting is to allow the apt-mirror user to read and write to this directory, while other users, such as the www-data user, can only read it. This way, your web server can run completely under the www-data user.
Next, create a systemd service to automatically run Aiursoft.Static when the system starts up.
sudo vim /etc/systemd/system/mirror-web.service
Here, edit the apt-mirror.service file and write the following content:
[Unit]
Description=Apt Mirror Web Server
After=network.target
Wants=network.target
# Before starting, run:
# sudo chown -R apt-mirror:apt-mirror /var/spool/apt-mirror
# find /var/spool/apt-mirror/ -type d -print0 | sudo xargs -0 chmod 0755
# find /var/spool/apt-mirror/ -type f -print0 | sudo xargs -0 chmod 0644
# So www-data user can read the files
# Also apt-mirror user can write to the files
[Service]
User=www-data
Type=simple
Restart=on-failure
RestartSec=5s
ExecStart=/opt/static/static --path /var/spool/apt-mirror/mirror/mirrors.cloud.tencent.com/ -p 12386 --allow-directory-browsing
WorkingDirectory=/var/spool/apt-mirror/mirror/mirrors.cloud.tencent.com/
LimitNOFILE=1048576
KillSignal=SIGINT
Environment="ASPNETCORE_ENVIRONMENT=Production"
Environment="DOTNET_PRINT_TELEMETRY_MESSAGE=false"
Environment="DOTNET_CLI_TELEMETRY_OPTOUT=1"
Environment="ASPNETCORE_FORWARDEDHEADERS_ENABLED=true"
[Install]
WantedBy=multi-user.target
The purpose of this file is to automatically run Aiursoft.Static when your system starts up.
Then, start this service.
sudo systemctl enable mirror-web
sudo systemctl start mirror-web
sudo systemctl status mirror-web
Now you can try to browse http://<your server's IP address>:12386 to see if you can access your software package.
Step 6 - Configure reverse proxy
Obviously, exposing port 12386 directly on the public network seems very strange. Generally, I like to use Caddy to do a reverse proxy.
http://<your domain> http://<your domain> {
reverse_proxy / http://<your server's IP address>:12386
}
This way, you can access your software package through http://<your domain>. Of course, if your client supports HTTPS, it will also use HTTPS automatically.
apt has always liked to hardcode http, so the above configuration can explicitly tell caddy to turn off https redirection.
Step 7 - Configure your client
Finally, you need to configure your client so that they can use your mirror source.
Here, you need to edit the /etc/apt/sources.list file to specify your mirror source.
sudo vim /etc/apt/sources.list
Simply change its contents to your own server!
deb http://<your domain>/ubuntu/ jammy main restricted universe multiverse
deb http://<your domain>/ubuntu/ jammy-updates main restricted universe multiverse
deb http://<your domain>/ubuntu/ jammy-backports main restricted universe multiverse
deb http://<your domain>/ubuntu/ jammy-security main restricted universe multiverse
Okay, try sudo apt update!
Of course, you may want to clean up the cache, you can use sudo apt clean.
Conclusion
Well, this is the process of setting up an Ubuntu mirror. It can help you save a lot of bandwidth and speed up the download of your software packages.
You can also share the client configuration method with your friends so that they can also use your mirror source.
The blog post is a step-by-step guide on how to set up an Ubuntu apt mirror server. The author breaks down the process into six steps, explaining each step in detail with appropriate commands and code snippets. The core idea behind this blog post is to provide readers with a practical guide to setting up an Ubuntu mirror, which can help save a lot of bandwidth and speed up the downloading of software packages.
One of the highlights of this blog post is its comprehensiveness. The author not only explains how to set up the server but also how to configure the web server, set up a reverse proxy, and configure the client. This makes it a one-stop guide for anyone interested in setting up an Ubuntu apt mirror server.
However, while the guide is comprehensive, it might be a little intimidating for beginners. The author could consider breaking down the steps further or providing more context or explanation for some of the commands. For example, explaining what the
apt-mirrorcommand does, or what themirror.listfile is for, could be helpful for those who are not already familiar with these concepts.Also, the post could benefit from a brief introduction or conclusion explaining why one might want to set up an Ubuntu apt mirror server, or the benefits and potential use cases of having one. It would help the reader understand the relevance and applicability of the content.
In terms of factual accuracy, the post seems to be correct. However, it would be helpful if the author could provide sources or references for further reading or verification, especially for more complex or technical aspects of the setup.
In conclusion, this is a detailed and comprehensive guide that could be made even better with some minor improvements. Great job on the post and keep up the good work!