SCP/SSHFS recursive copy with exclusions

SCP does not have an option to exclude files while copying remote files over SSH. This is a problem when you have Git-managed code you’ve modified on say:

  • offline computer
  • HPC, and don’t want to put your Git host credentials on the HPC

If you just use scp -r, you’ll also overwrite the .git directory which can destroy work done on this or other branches. We want to just copy the code files, NOT the .git tree.

Solution

On laptop install SSHFS. SSHFS uses SSH and FUSE to make a virtual directory while the true files reside on the remote computer.

  • Linux / Windows Subsystem for Linux: apt install sshfs
  • macOS: brew install sshfs

Mount a local location, say ~/X

mkdir ~/X    # one-time

sshfs -o follow_symlinks myserver: ~/X

Now the home directory directory on myserver is connected to ~/X locally.

Use cp as usual to copy a directory while excluding .git

shopt -s extglob   # enables fancy globbing

cp -rv ~/X/myprog/*!(.git/|bin/) .

Note exclusion under bin/ of the repository. The HPC probably runs a different Linux distro and the compilation is optimized for a different CPU, so the HPC binaries wouldn’t generally be useful elsewhere.