Agents and panel
Update agents
A new agent version comes with the panel: after a deploy, the panel distributes it. But agents install only a version signed with your release key. This way a hacked or substituted panel can't install its own binary on the servers. Until the version is signed, agents stay on the previous one, and backups run as before.
The key is a minisign key. The private part is only with you: ~/.minisign/arcula-release.key and a copy in your password manager. The public part is built into the agent.
How to tell it's time
Panel → “Settings” → “Agent updates”. There you see the version the panel offers and a label: “not signed” — “Agents won't install this version until you sign it”. Below — how many agents are already on it and which servers are older.
On the server — “Overview” → “Agent”: “arcula 0.1.0 · the panel offers 0.1.1”.
In one command
On your computer, in a clone of the repository, after a push. HEAD must be pushed:
make agent-release PANEL=https://panel.example.com SSH=panel-hostSSH is the panel server for ssh, as in ~/.ssh/config. The script:
- waits up to 30 minutes for the panel to start offering the agent from this commit;
- rebuilds the agent from the same commit and compares it byte for byte with what the panel offers, for amd64 and arm64;
- asks for the key password and signs — only if the builds match;
- uploads the signature to the panel over ssh;
- sends “Update everywhere”.
The key never leaves your computer: only the signature goes over ssh. To sign and upload without “Update everywhere” — make sign-agent PANEL=https://panel.example.com SSH=panel-host WAIT=1.
You need minisign, go, git, Docker, curl and shasum. If the key is somewhere else, give its path: ARCULA_SIGN_KEY=/path/to/key.
Step by step
If you have no ssh access to the panel server:
- Wait for the deploy: “Agent updates” shows the new version labelled “not signed”.
- In the repository folder, run
make sign-agent PANEL=https://panel.example.com. The script downloads the version, rebuilds it, compares and asks for the key password. The result is the filearcula-<version>.SHA256SUMS.minisigin the current folder. - Panel → “Settings” → “Agent updates” → “Upload signature”, choose the file. The label changes to “signed”: “Signed with key … — agents will install it”.
- On the same screen, click “Update everywhere”. The command goes to all servers where the agent is older.
The .minisig file isn't needed after upload, and it isn't secret. To sign a version other than the latest — V=0.1.0-main.N.
Update one server
In the panel: server → “Overview” → “Agent” → “Update agent”. Or on the server itself:
sudo arcula updatesudo arcula update --check only shows whether there's a new version.
The agent downloads the version, checks the signature and the checksum, waits for a running backup to finish and replaces the binary. While it's being replaced, the server shows “Updating the agent…”. The result is in the server log, the “Agent” filter: “Agent updated” with the versions, or “Agent didn't update” with the reason.
If it doesn't update
| What you see | What to do |
|---|---|
| Label “not signed”, “Update everywhere” can't be clicked | Sign the version, as above |
| “Signature doesn't match” on upload | A different version was signed. The panel shows which version needs a signature — sign that one |
| “сборка панели не совпала с вашей из коммита — не подписываю” (the panel's build doesn't match yours from the commit — not signing) | Don't sign. The panel has a build that differs from the repository. Sort that out first |
| “коммита … нет в клоне” (commit … is not in the clone) | Run git fetch origin in the clone and start again |
| “not signed by the release key” in the server log | The signature isn't uploaded or was uploaded for another version |
| “The agent will update to … if it comes online …” | The agent is offline right now. It will update when it comes back |
| “Agent didn't run: update the agent” | The reason is in the same log line, e.g. another job is running. Try again later |
sudo arcula update mentions pin_version | The version is pinned in /etc/arcula/agent.yaml. To update explicitly: sudo arcula update --version X.Y.Z |
On the server, these help:
sudo arcula doctor
journalctl -u arcula-agentarcula doctor shows in its version line which version the panel offers and which one is installed.
Rollback
If the new version doesn't start within 3 minutes, the agent brings back the previous one by itself. The previous binary is kept next to it: /usr/local/bin/arcula.prev. The server log shows “Agent didn't update” with the reason, e.g. “arcula X did not start within 3m0s — rolled back to …”. Automatic rollback works under systemd.
The agent doesn't move to an older version on a command from the panel. An older version can be installed only on the server itself — if the panel offers it and it is signed:
sudo arcula update --version 0.1.0The same article is in the panel, under “Help”.