macOS is 10x slower than Linux Link to heading

macOS runs 10x slower than Linux when it comes to POSIX utilities that are pre-installed on these systems.

Not all Unices are made equal

If you are a software developer, you are probably familiar with a suite of POSIX tools that are pre-installed on various Unices, such as Linux, macOS, and BSD. These tools include cd, ls, diff, cmp, sort, awk, cut, grep, etc. These tools are standardized by POSIX, making all Unix OSes feel very similar. However, this does not mean these tools are the same in all the Unices.

POSIX defines an interface and not implementation. Hence, there are multiple implementations of these tools, and each OS decides which implementation to ship with. For example, virtually all Linux distributions are equipped with GNU implementations of these tools.

# on Linux
diff --version
diff (GNU diffutils) 3.10
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Paul Eggert, Mike Haertel, David Hayes,
Richard Stallman, and Len Tower.

On the other hand, macOS and BSDs are equipped with BSD implementations and derivatives.

# on macOS
diff --version
Apple diff (based on FreeBSD diff)

As you can imagine, Linux is an order of magnitude more widely adopted BSDs, encouraging more people to contribute to GNU utilities than BSDs. In addition, Apple’s macOS is a closed source and hence only a handful of Apple engineers get to work on these utilities for macOS. The result is that most of these utilities are quite outdated and hence not as optimized as the GNU counterparts.

Let me show you a proof. Below graph summarizes runtime of several utilities run on the same hardware. The Linux bars represent runtime of those utilities run using GNU implementation while macOS bars represent runtime of those pre-installed on macOS, derivatives of the BSD implementations.

In all of these tests, these utilities run much faster on Linux. In particular, diff from Linux runs 10x faster than diff from macOS. These are just handful of utilities I tested, but I won’t be surprised to see more POSIX utilities running faster on Linux than macOS.

Solution Link to heading

If you are a macOS user, like myself, it is not the end of the world. You could easily install GNU implementations of these tools and use them as the default. In fact, this is the first thing I do when I am setting up a new Mac. The easiest way is to install via homebrew.

# cmp and diff
brew install diffutils

# cut and sort
brew install coreutils

If you are interested in an alternative installation, you could try nix package.

# cmp and diff
nix-env -iA nixpkgs.diffutils

# cut and sort
nix-env -iA nixpkgs.coreutils