Idea
Managing configuration files for essential CLI tools such as vim and readline across multiple environments is an interesting challenge. Whether you’re running sudo vim to edit system files or SSH-ing into a new server, configs get lost. It’s a common headache to keep them in sync and avoid conflicts, especially on shared accounts like root. I’ve created a solution that is:
- Transparent and fast. Just use the commands normally - configs will be migrated where needed. No slow file transfers or double authentication will occur. Absolutely minimal requirements are imposed on the destination.
- Works recursively. If destinations are chained, dotfiles propagate accordingly. For example, you can: ssh host1 -> sudo su -> ssh host2 -> su user -> …
- Session-only. Other persons will not be affected by your configs on the remote, even if the remote user is shared.
- Multi-user. Multiple users can utilize this solution simultaneously, even if the remote user is shared.
- Doesn’t pollute history. Intercepts calls to the affected commands and follows their interface, keeping history compliant with the original version.
- Compatible with scripts. Works only on the main interactive session of destination.
- Source-extensible. As a part of my workspace repository, it allows to add custom things to profile without touching source-controlled base files.
Implementation
The setup on the destination environment is handled by a single auto-generated file (the “remote profile”). The easiest way to compose it is to use a template engine.
I use gomplate. It works really well and has everything I need. Importantly, its go template syntax is already familiar to me and has been used to generate the blog you’re currently reading. However, I dislike it for being bloated in terms of features and binary size.
1. Remote profile
Remote profile is a self-sufficient bash script that wraps up remote .profile and extends it with our custom stuff. It is static and stores everything inside. To spin up remote configuration, one should export two variables, source it, and source newly generated rcfile. These variables are:
$WS_IS_COMPAT- controls whether we’re in compatibility mode, script does nothing if not$WSCOMPAT_DIR- defines the temporary directory where to store populated configuration and scripts
Inside remote profile, we:
- Source original
.profile,/etc/profile. - Prepend PATH with
$WSCOMPAT_DIRto make transferred executables easily accessible. - Configure editors. Usually this means making files inside
$WSCOMPAT_DIRdirectory and instructing programs where to look for them using environment variables. - Save and chmod executables that will migrate remote profile to the next destination.
- Set arbitrary environment variables according to our preferences.
- Render bashrc wrapper, which
- Uses aliases or functions to intercept commands that switch environment and calls our wrappers (to make the solution recursive)
- Does arbitrary interactive-only session configuration
- Do other customizations
Below you can find a template file of the remote profile. It loads and processes other files as templates recursively (dotfiles are also templated). It must be re-rendered when its children change.
config/workspace-compat/profile.tmpl
#!/bin/bash
if [ -z "$WSCOMPAT_DIR" ] || [ "$WS_IS_COMPAT" != 1 ]; then
exit 0
fi
[ -f /etc/profile ] && . /etc/profile
[ -f ~/.profile ] && . ~/.profile
export EDITOR=vim
export SHELL=/bin/bash
export PATH="$WSCOMPAT_DIR:$PATH"
cat << 'WSCOMPAT_EOF_04tcIQE7' > "$WSCOMPAT_DIR/bashrc"
{{ tmpl.Inline (file.Read "config/workspace-compat/bashrc") . }}
WSCOMPAT_EOF_04tcIQE7
export VIMINIT="source $WSCOMPAT_DIR/vimrc"
cat << 'WSCOMPAT_EOF_04tcIQE7' > "$WSCOMPAT_DIR/vimrc"
{{ tmpl.Inline (file.Read "config/vim/vimrc.tmpl") . }}
WSCOMPAT_EOF_04tcIQE7
export INPUTRC="$WSCOMPAT_DIR/inputrc"
cat << 'WSCOMPAT_EOF_04tcIQE7' > "$WSCOMPAT_DIR/inputrc"
{{ tmpl.Inline (file.Read "config/readline/inputrc") . }}
WSCOMPAT_EOF_04tcIQE7
export LESSKEYIN="$WSCOMPAT_DIR/lesskey"
cat << 'WSCOMPAT_EOF_04tcIQE7' > "$WSCOMPAT_DIR/lesskey"
{{ tmpl.Inline (file.Read "config/less/lesskey") . }}
WSCOMPAT_EOF_04tcIQE7
cat << 'WSCOMPAT_EOF_04tcIQE7' > "$WSCOMPAT_DIR/wssh"
{{ tmpl.Inline (file.Read "bin/wssh") . }}
WSCOMPAT_EOF_04tcIQE7
chmod +x "$WSCOMPAT_DIR/wssh"
cat << 'WSCOMPAT_EOF_04tcIQE7' > "$WSCOMPAT_DIR/wsudo"
{{ tmpl.Inline (file.Read "bin/wsudo") . }}
WSCOMPAT_EOF_04tcIQE7
chmod +x "$WSCOMPAT_DIR/wsudo"
2. Dotfiles
Dotfiles may need to differ on the master and destination environments. Firstly, they are different in terms of capabilities - the first one is connected to your window manager and has access to a well-defined list of packages and plugins, whereas second one is designed to work anywhere. Secondly, the remote nature of the latter may require some commands to work differently (for example, to make use of your host clipboard, image viewer, etc).
In order to keep the code reused, they are processed as templates. For the master (native) target, dotfiles should be rendered to their primary location in $XDG_CONFIG_DIRS or $HOME. Rendering in compatibility mode is managed by the parent template in the previous section.
To detect the target, simply check the environment variable IS_COMPAT.
config/vim/vimrc.tmpl
...
{{ if ne .Env.IS_COMPAT "1" }}
...
{{ else }}
...
{{ end }}
If configuration file does not use template engine directives, it will be rendered literally.
Right now, in compatibility mode, 4 programs are configured separately:
- vim: VIMINIT=“source /path/to/file” method
- less: LESSKEYIN="/path/to/file" method
- readline: INPUTRC="/path/to/file" method
- bash: rcfile (supplied by the caller script)
By the way, I find it useful to render config files as templates besides this task, as it allows to keep generic settings in one place, do inline calculation and adapt to the specs/capabilities of computer I install them on.
3. Command wrappers
They are special handlers of commands that involve switch of environment, such as ssh and sudo. Interface of these commands must be a subset of the interface of original commands in order to keep them transparent in history.
Pre-command approach is highly appreciated as it does not lead to double context switching. For now, their names must be unique so they don’t pollute PATH and break programs. I’ve chosen w prefix which means workspace.
Basically, every wrapper should do everything that the base command does and achieve the following:
- On remote, set
WS_IS_COMPAT=1 - On remote, set
$WSCOMPAT_DIR. Some implementations use the folder in/tmp. To avoid conflicts between different users of this solution, variable$WSCOMPAT_MASTERis passed along the chain of switches to be used as part of$WSCOMPAT_DIR. It stores the name of the user who initiated the session. - Merge local “remote profile” with custom scripts from local
$WSCOMPAT_DIR/profile*.shand transfer it to remote. Some fancy stuff is done with heredoc separators because wrapper script must transfer itself. - On remote, source “remote profile”.
- On the remote, invoke interactive command if it was supplied. If not, invoke shell with generated
rcfileand appropriate parameters
Wrapper may fall back to the base command if it detects non-interactive use.
bin/wssh (ssh)
#!/bin/bash
set -e
if [ $# -lt 1 ]; then
echo "Usage: $0 <hostname> [ssh-options]"
exit 1
fi
if [[ "$*" == *" -- "* ]]; then
exec /bin/ssh "$@"
fi
[ -z "$WSCOMPAT_MASTER" ] && WSCOMPAT_MASTER="$(whoami)"
read -r -d '' init_cmd << WSCOMPAT_EOF_9oqn7IVF || true
export WS_IS_COMPAT=1
export WSCOMPAT_MASTER="$WSCOMPAT_MASTER"
export WSCOMPAT_DIR="/tmp/wscompat_${WSCOMPAT_MASTER}_\$(whoami)"
rm -rf "\$WSCOMPAT_DIR" && mkdir -p "\$WSCOMPAT_DIR" && chmod 700 "\$WSCOMPAT_DIR"
cat << 'WSCOMPAT_EOF_84EbM3la' > "\$WSCOMPAT_DIR/profile"
$(
sed 's/WSCOMPAT_EOF_/WSCOMPAT_EOF__/g' "$WSCOMPAT_DIR/profile"
for file in "$WSCOMPAT_DIR/profile"*.sh; do
if [ -f "$file" ]; then
sed 's/WSCOMPAT_EOF_/WSCOMPAT_EOF__/g' "$file"
fi
done
)
WSCOMPAT_EOF_84EbM3la
. \$WSCOMPAT_DIR/profile
# pam, systemd - wrap in login shell
exec /bin/bash --login --norc --noprofile -c "exec /bin/bash --rcfile \"\$WSCOMPAT_DIR/bashrc\""
WSCOMPAT_EOF_9oqn7IVF
exec /bin/ssh -t "$@" "$init_cmd"
When I need to execute remote command, I always use -- syntax because it doesn’t require double quotes, so no nesting and extra escaping of quotes occurs. I decided not to implement full argument parsing to handle quoted remote commands.
As for sudo, there is a dichotomy between sudo -u and sudo su. Dealing with sudo -u seems a lot easier (no double switching), and this option is more simple. Since we get everything as arguments, it is possible to eliminate su completely at the parsing stage.
bin/wsudo (sudo)
#!/bin/bash
parse() {
sudo_user="root"
sudo_command=""
sudo_login_shell=""
local args=("$@")
local i=0
local arg_count=${#args[@]}
expand_flags() {
local expanded=()
for arg in "$@"; do
if [[ $arg =~ ^-[a-zA-Z]{2,}$ ]]; then
local chars="${arg:1}"
for ((j=0; j<${#chars}; j++)); do
expanded+=("-${chars:$j:1}")
done
else
expanded+=("$arg")
fi
done
args=("${expanded[@]}")
arg_count=${#args[@]}
}
expand_flags "$@"
while [[ $i -lt $arg_count ]]; do
case "${args[$i]}" in
-u|--user)
((i++))
if [[ $i -lt $arg_count ]]; then
sudo_user="${args[$i]}"
fi
((i++));;
-i|--login)
sudo_login_shell=true
((i++));;
-s|--shell)
((i++));;
--)
((i++))
sudo_command="${args[*]:$i}"
break;;
-*)
((i++));;
su)
((i++))
local su_user=""
local su_args=()
while [[ $i -lt $arg_count ]]; do
su_args+=("${args[$i]}")
((i++))
done
local su_expanded=()
for arg in "${su_args[@]}"; do
if [[ $arg =~ ^-[a-zA-Z]{2,}$ ]]; then
local chars="${arg:1}"
for ((j=0; j<${#chars}; j++)); do
su_expanded+=("-${chars:$j:1}")
done
else
su_expanded+=("$arg")
fi
done
local k=0
local su_count=${#su_expanded[@]}
while [[ $k -lt $su_count ]]; do
case "${su_expanded[$k]}" in
-|-l|--login)
sudo_login_shell=true
((k++));;
-c|--command)
((k++))
if [[ $k -lt $su_count ]]; then
sudo_command="${su_expanded[$k]}"
((k++))
fi;;
--session-command=*)
sudo_command="${su_expanded[$k]#*=}"
((k++));;
-m|-p|--preserve-environment)
((k++));;
-s|--shell)
((k++))
if [[ $k -lt $su_count && "${su_expanded[$k]}" != -* ]]; then
((k++))
fi;;
-*)
((k++));;
*)
if [[ -z "$su_user" ]]; then
su_user="${su_expanded[$k]}"
((k++))
else
((k++))
fi;;
esac
done
if [[ -n "$su_user" ]]; then
sudo_user="$su_user"
fi
break;;
*)
sudo_command="${args[*]:$i}"
break;;
esac
done
}
parse "$@"
[ -z "$WSCOMPAT_MASTER" ] && WSCOMPAT_MASTER="$(whoami)"
read -r -d '' init_cmd << WSCOMPAT_EOF_ahkiem3K || true
export WS_IS_COMPAT=1
export WSCOMPAT_MASTER="$WSCOMPAT_MASTER"
export WSCOMPAT_DIR="/tmp/wscompat_${WSCOMPAT_MASTER}_\$(whoami)"
rm -rf "\$WSCOMPAT_DIR" && mkdir -p "\$WSCOMPAT_DIR" && chmod 700 "\$WSCOMPAT_DIR"
cat << 'WSCOMPAT_EOF_daePe0Ph' > "\$WSCOMPAT_DIR/profile"
$(
sed 's/WSCOMPAT_EOF_/WSCOMPAT_EOF__/g' "$WSCOMPAT_DIR/profile"
for file in "$WSCOMPAT_DIR/profile"*.sh; do
if [ -f "$file" ]; then
sed 's/WSCOMPAT_EOF_/WSCOMPAT_EOF__/g' "$file"
fi
done
)
WSCOMPAT_EOF_daePe0Ph
. \$WSCOMPAT_DIR/profile
WSCOMPAT_EOF_ahkiem3K
if [ -z "$sudo_command" ]; then
sudo_command="exec /bin/bash --rcfile \"\$WSCOMPAT_DIR/bashrc\""
fi
if [ "$sudo_login_shell" ]; then
exec /bin/sudo -u "$sudo_user" \
/bin/bash --login --noprofile --norc -c "$init_cmd; cd; $sudo_command"
else
exec /bin/sudo -u "$sudo_user" \
/bin/bash --noprofile --norc -c "$init_cmd; $sudo_command"
fi
Finally, inject them instead of the basic commands in the interactive shell via aliases in the rcfile below, which is transferred through remote profile and invoked as --rcfile. It sources the default rcfile aka .bashrc and adds customizations.
config/workspace-compat/bashrc
[ -f ~/.bashrc ] && . ~/.bashrc
alias ssh='wssh'
alias sudo='wsudo'
su() {
if [ "$1" = "-" ]; then
shift && wsudo - -u "$@"
else
wsudo -u "$@"
fi
}
su from interactive root in the form I use was superseded by sudo.
That’s all the wrappers for today. Next candidates are probably chroot and docker exec, which I don’t use.
4. Master environment
Obviously master (native) environment should not source these scripts, so it needs a bit of configuration - it should know where to look for the remote profile.
$WSCOMPAT_DIR must point to a folder where you rendered profile template. You can add extra initialization scripts that follow glob profile*.sh to that folder - they will be merged into the target’s profile. Local $WSCOMPAT_DIR will not be populated with configs as we’re in native mode.
I recommend setting $WSCOMPAT_DIR in the local .profile or its callees. My $WSCOMPAT_DIR points to .config/workspace-compat and is exported from .config/workspace-compat/env.sh.
For the better shell experience, wssh and wsudo scripts should appear in $PATH (which is best set in the local .profile due to its incremental nature). I install them to .local/bin. To make this truly unified and seamless, I also make sudo and ssh aliases, just like in compatibility mode.
alias ssh='wssh'
alias sudo='wsudo'
Overriding su here is useless since native user is not root.
5. Generate
IS_COMPAT=1 gomplate \
--file config/workspace-compat/profile.tmpl \
--out $WSCOMPAT_DIR/profile
IS_COMPAT variable is accessible inside template files.