Scientific Computing

Git SSH over port 443

Certain networks may block traffic on port 22, which causes failure for Git operations like:

ssh: connect to host github.com port 22: Connection timed out fatal: Could not read from remote repository.

The solution for this is to at least temporarily configure SSH to use port 443, which is typically open for HTTPS traffic. This can be persistently done by editing the user SSH config file usually located at “~/.ssh/config”, here for GitHub:

Host github.com
  User git
  IdentityFile ~/.ssh/github
  Hostname ssh.github.com
  Port 443

where “~/.ssh/github” comes from GitHub SSH key setup.

Likewise for GitLab:

Host gitlab.com
  User git
  IdentityFile ~/.ssh/gitlab
  Hostname altssh.gitlab.com
  Port 443

An alternative is to specify the port directly in the Git remote URL like:

git push ssh://user@host:PORT/path/to/repo.git main

Or specify environment variable GIT_SSH_COMMAND.

Recursively convert line endings with dos2unix + mac2unix

To recursively convert line endings from DOS \r\n and old macOS \r to Unix \n format in all files recursively, use “dos2unix” and “mac2unix” commands. For example, it’s not uncommonn that Matlab .m files shared among groups end up with DOS line endings, which can cause issues across different operating systems. When viewed with “git diff” or similar, files might have ^M characters at the end of lines, indicating DOS or old macOS line endings. Run the following command using GNU Find to convert in place all .m files:

find . -type f -name "*.m" -exec sh -c 'dos2unix -e "$1" && mac2unix -e "$1"' _ {} \;

Set Git to use the correct line endings to avoid line ending conflicts in Git repositories.

Windows Git Credential Manager

Git over SSH can use SSH-agent to remember SSH keys for Linux and Windows. For those not using Git over SSH but tiring of typing the Git password, on Windows the Git Credential Manager can fix that situation. The install is small and quick, and to use Git Credential Manager, be sure via

git remote -v

that the connection is https:// and not ssh:// where appropriate. Upon trying to git push or git pull, the Git Credential Manager will appear. It works with 2-factor Authentication as well, storing in Windows Credential Manager. The keys can be deleted from Windows Credential Manager if needed, and / or the Personal Access Token automatically generated can be deleted from GitHub.

Note: If using automatic Git push SSH that would need to be disabled to allow git push to use https so that Git Credential Manager is invoked.

Detect executable file arch

For the simplest execution and best performance, an executable file binary architecture should generally correspond to the CPU architecture of the operating system it runs on. The late 1990s and early 2000s saw a proliferation of CPU architectures including SPARC, PowerPC, DEC Alpha, MIPS, Itanium, and x86. From the late 2000s through about 2020, the dominant CPU architecture for personal computers was x86_64. The 2020s saw the rise of ARM64 across operating systems including Windows on ARM, Apple Silicon, and Linux.

Given the development time and maintenance burdern of supporting multiple CPU architectures, there are in general situations where a user needs to run an executable file on a different CPU architecture than the executable file was built for. It’s a nice practice for program developers to print their native CPU architecture in their program’s about dialog to help users be aware of the CPU architecture of their executable files such that they can seek a native executable if available. For example, a 32-bit x86 executable might run on a 64-bit x86_64 operating system (via API thunking and environment emulation like WoW64), but a 64-bit x86_64 executable generally cannot natively run on a 32-bit x86 operating system.

With some performance overhead and compatibility limitations, executable JIT translation like macOS Rosetta 2, Windows Prism emulation, or Linux FEX emulation can allow executables of one architecture to run on a different CPU architecture. The operating system level translation or emulation development is typically a large investment in software development and optimization, and is not universally available for all architecture combinations. Apple Rosetta 2 translates x86-64 instructions to ARM64 on Apple Silicon, allowing x86-64 executables to run on ARM64 macOS. On Windows, Prism emulation generally allows x86-64 executables to run on ARM64 Windows. On Linux, FEX emulation allows x86-64 executables to run on ARM64 Linux with real-time API call forwarding.

The LIEF library can be used to detect the CPU architecture of Windows executable files from Python.

Meld Git difftool / mergetool

Git users often use Meld to graphically resolve 2-way differences and 3-way merges.

Configure Git to use Meld:

git config --global diff.tool meld
git config --global merge.tool meld

On Windows, Meld can be installed by:

winget install -e --id Meld.Meld

Troubleshooting

If Meld was installed manually and Git can’t find Meld, configure Git to use Meld by setting the path to Meld.exe like:

git config --global mergetool.meld.path "$Env:ProgramFiles\\meld\\Meld.exe"

On Windows if also using MSYS2, don’t add Meld.exe to environment variable Path as it has libstdc++.dll that conflicts with MSYS2 G++. The symptom is the G++-built executable will segfault silently.


Alternatives:

Python for Windows on ARM

Anaconda Python is working toward Windows on ARM support. For now, Anaconda / Miniconda Python work for Windows on ARM via the built-in Prism emulation. To use native ARM64 Python, which could be useful for benchmarking or maximum (best) computing performance, use plain CPython install for ARM64 such as:

winget install Python.Python.3.14

Upon installing and starting, one sees the ARM64 designation in the Python dialogs.

GDL GNU Data Language build

GDL (GNU Data Language) is a free/libre open-source program that runs a good percentage of IDL code. GDL is actively developed and easily installed by:

  • Linux: apt install gnudatalanguage
  • macOS: use weekly gdl-macOS-arm64-standard.dmg. We do this instead of Homebrew because the homebrew/science tap for gnudatalanguage is currently unmaintained.
  • Windows: get the latest release

Building GDL source uses the GDL build script “scripts/build_gdl.sh” to get the prerequisites. If Anaconda Python is present, conda deactivate first to avoid library problems when building GDL.

git clone https://github.com/gnudatalanguage/gdl

cd gdl/

cmake -B build --install-prefix=$HOME/gdl

cmake --build build --parallel

(optional) Check the install. You will see several plots appearing and disappearing automatically during this test, which takes a few minutes.

cmake --test-dir build -V

Install (do not use sudo):

cmake --install build

Do not build on an ExFAT / FAT32 drive, as the build will fail since symbolic links are not allowed on ExFAT / FAT32. If cmake reports libeigen being too old, install LibEigen3 or:

cmake -Bbuild -DEIGEN3=OFF

To use the Linux distro’s older version of GDL, just use /usr/local/bin/gdl or similar, or rename ~/.local/bin/gdl to ~/.local/bin/gdl0.98 or similar.

Troubleshooting build:

  • Runtime search path conflicts: temporarily comment out those paths in ~/.profile (typically from Anaconda Python, libreadline, libhistory, libz, libjpeg.so)
  • Problems with LZMA, try disabling HDF5: cmake -DHDF5=OFF

Clear Pacman database lock

If upon attempting Pacman operations a failure occurs like:

failed to synchronize all databases (unable to lock database)

This may occur if the system was interrupted during a Pacman operation, leaving a lock file that prevents further package management operations. The lock file is located by:

$(pacman-conf DBPath)/db.lck

which is typically “/var/lib/pacman/db.lck”. Check no other Pacman process is running:

ps -ef | grep pacman

Then the Pacman lock file can be removed:

rm $(pacman-conf DBPath)/db.lck

GitHub outage workaround with SSH instead of HTTPS

Anecdotally we have observed that during GitHub outages, Git over SSH operations may have a better chance of succeeding than Git over HTTPS operations. This includes cloning repositories.

Rather than reconfiguring .gitconfig to use SSH, simply clone using the SSH URL instead of the HTTPS URL.

For example, instead of:

git clone https://github.com/user/repo.git

Assuming Git over SSH is setup on the computer:

git clone git@github.com:user/repo.git

CMake VS 2026 on GitHub Actions

The CMake variable CMAKE_GENERATOR can be used with GitHub Actions to specify the Visual Studio 18 2026 generator. Currently, the runner image GitHub Actions runner image windows-2025-vs2026 is used until the “windows-latest” runner image incorporates VS 2026.

jobs:

  msvc:
    runs-on: windows-2025-vs2026

    steps:
    - uses: actions/checkout

    - name: Print CMake version
      run: cmake --version

    - name: Configure CMake
      run: cmake -B build -G "Visual Studio 18 2026"

    # and so on