El Preservador: SSHing Responsibly on iOS

El Preservador: SSHing Responsibly on iOS
Keeping terminal sessions alive with El Preservador

With La Terminal, we strive to offer an SSH client that just works on iOS.

With this in mind, I set out to make design decisions on behalf of users that would provide an experience that works out of the box while providing the best terminal emulator experience they deserve.

A platform feature that has long plagued SSH clients on iOS and iPadOS is that these operating systems are designed to reduce battery consumption, and thus are more than happy to suspend your application and terminate your network connections when your app goes to the background. This has led app developers to develop creative solutions to this problem over the years.

There are three approaches SSH clients have taken to solve the background disconnect issue. Let me list them and explain how they work:

  • Keeping the application running in the background: there are several features that one can use on iOS that will keep the application running. Apple will let you use those capabilities if you have a good reason to do so. Many terminal emulators support this, and the most common solution is to use location tracking to keep your application alive.
Why does an SSH client need to track my location? 

The downside of this approach is that your app is always running, you consume more battery than you should, and you need permission from the user to grant you location access. Location tracking raises privacy concerns for the users (are they selling my location?) and defeats the careful iOS tuning that goes into saving the user’s battery life.

  • Using the mosh protocol: Mosh is a terminal emulator that runs on your server and is connected locally to your server. When you run applications under mosh, it interprets the commands and keeps the contents on the server. If your connection ever drops, you can reconnect to mosh, and mosh will send the state of the terminal back to your client. The reconnection happens transparently when your connection drops or when you roam.

    Mosh requires the user to install a server-side component on the target system. After an initial connection over ssh, mosh exposes a port to the internet for subsequent connections. The major drawback is that the terminal emulation in mosh is a subset of the terminal capabilities of La Terminal, so you end up with a degraded terminal experience. While it works great for most applications, it prevents innovative capabilities from surfacing until Mosh is updated. Mosh is also limited to certain terminal emulators due to the mosh library license.

  • Using a session saving system: Another approach is to use a session saving system like tmux or GNU screen. These systems are terminal emulators that run on your server, and it is possible to re-establish the connection and restore your session. When paired with a modern SSH client, the experience is very similar to what mosh does, except that they do not expose an additional port to the internet - it just uses a fresh new SSH session and restores the state.

    This approach shares some of the downsides of mosh, like having its terminal emulator and only exposing a portion of the terminal capabilities to the clients. It also requires tmux to be installed on the server. In my testing, I found that the versions of tmux deployed in the wild can be pretty old, further limiting the available capabilities. La Terminal supports using tmux as a session preservation system and will gracefully fall back to the more limited capabilities in older versions of tmux.

Double-width and double-height fonts are supported by La Terminal, but intermediate emulators erase this capability
An example of capabilities being erased: Double-height text in El Preservador versus tmux

One of the goals of La Terminal (and SwiftTerm) is to provide a state-of-the-art terminal emulation. And not only to support a high-fidelity emulation for xterm and DEC VT100 to VT520 but also to incorporate new capabilities under development by the terminal community.

I was not happy with either burning the battery and the blue reminder of shame omnipresent in your phone, nor the limited terminal emulation from tmux. These solutions pulled me back into a world of limited features, fallbacks, and unpleasant experiences.

So I designed a different solution, which I call El Preservador, and is the default session preservation system in La Terminal.

El Preservador is a small program that runs on your target host, it requires no manual installation, and its sole role is to act as a server-side proxy buffer to ssh.

Unlike tmux or mosh, it does not attempt to interpret the meaning of the buffer stream and instead leaves the emulation to La Terminal. When iOS suspends your network connections, El Preservador continues running on the server. When your application returns to the foreground, La Terminal creates a new connection to your machine and reconnects to El Preservador, restoring your session where you left it. One downside is that El Preservador is paired to the running instance of La Terminal, so if you kill the application on iOS, it will not attempt to restore an existing session like tmux or mosh.

This shows how La Terminal can display inline images in the terminal, but when you use tmux the image is absent.
Another example of capabilities getting erased by an intermediate terminal emulator: Native Sixel image support works when using El Preservador versus tmux.

La Terminal solves the background disconnect problem without consuming any battery while in the background and without reducing the fidelity of your terminal on the remote end.

By default, new host configurations in La Terminal use the Native Keep Alive option, powered by El Preservador. You do not need to install or upgrade any software on the remote host to use El Preservador - this is handled automatically by La Terminal.

How La Salchicha is made

This graphic shows how a regular connection, with no session preservation works:

Boring picture of a phone connecting to a server, with an arrow indicating an uninspired SSH connection.
Regular SSH connections

Since the SSH protocol is stateful and iOS will happily kill the network connection when La Terminal is suspended, I came up with a different approach.

The idea is to treat the SSH connection between your iPhone and the remote server as temporary and discardable. So all the traffic is still encrypted and secured via SSH, but rather than launching your shell, it launches El Preservador - a small buffering program on your server that then connects locally to your SSH server.

Then I create an SSH tunnel over the temporary connection that connects your session to El Preservador, and it is over this tunnel that I launch a regular SSH session. La Terminal and El Preservador shake hands, and you have yourself an established connection. You will notice if you run a command like "who" that you show up as being connected from the local host, not a remote server (either localhost or 127.0.0.1)

Diagram showing how La Terminal uses El Preservador
La Terminal creates dynamic tunnels on demand to reconnect to the long-running Preservador on your remote host

As you go on with your day and respond to the latest joke posted to the squad chat group, try to keep up your Duolingo streak, or even blink for a second, iOS will try to satiate the battery gods and suspend your application and kill your network connection.

When you return to La Terminal, she immediately notices that the connection is gone. Fear not, as she will launch another transient SSH connection to the server, authenticate herself with El Preservador, and they will exchange information like "What was the last thing you heard me say" and then resume the operation from that point on.

Agent Installation

To install the agent on the remote system, La Terminal first determines the architecture and operating system it is running on. If it has a supported agent for that platform, it opens an SFTP channel to copy the agent to the remote system and set the permissions accordingly. You can find the agent in $HOME/laterminal/ which is both versioned and labeled for the architecture.

Subscribe to La Terminal Blog

Sign up now to be the first to know.
Guenter Gibbon
Subscribe