15

Installing R packages that need compiling in macOS is broken since Big Sur. Here is such an example.

> install.packages('nlme')

  There is a binary version available but the source version is later:
      binary  source needs_compilation
nlme 3.1-150 3.1-151              TRUE

Do you want to install from sources the package which needs compilation? (Yes/no/cancel) Yes
installing the source package ‘nlme’

trying URL 'https://cran.rstudio.com/src/contrib/nlme_3.1-151.tar.gz'
Content type 'application/x-gzip' length 805592 bytes (786 KB)
==================================================
downloaded 786 KB

* installing *source* package ‘nlme’ ...
** package ‘nlme’ successfully unpacked and MD5 sums checked
** using staged installation
** libs
"gfortran-4.8" -fno-optimize-sibling-calls  -fPIC  -Wall -g -O2  -c chol.f -o chol.o
gfortran-4.8: warning: couldn’t understand kern.osversion ‘20.1.0
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/usr/local/include   -fPIC  -Wall -g -O2  -c corStruct.c -o corStruct.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/usr/local/include   -fPIC  -Wall -g -O2  -c gnls.c -o gnls.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/usr/local/include   -fPIC  -Wall -g -O2  -c init.c -o init.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/usr/local/include   -fPIC  -Wall -g -O2  -c matrix.c -o matrix.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/usr/local/include   -fPIC  -Wall -g -O2  -c nlOptimizer.c -o nlOptimizer.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/usr/local/include   -fPIC  -Wall -g -O2  -c nlme.c -o nlme.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/usr/local/include   -fPIC  -Wall -g -O2  -c nlmefit.c -o nlmefit.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/usr/local/include   -fPIC  -Wall -g -O2  -c pdMat.c -o pdMat.o
clang -mmacosx-version-min=10.13 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/usr/local/include   -fPIC  -Wall -g -O2  -c pythag.c -o pythag.o
"gfortran-4.8" -fno-optimize-sibling-calls  -fPIC  -Wall -g -O2  -c rs.f -o rs.o
gfortran-4.8: warning: couldn’t understand kern.osversion ‘20.1.0
clang -mmacosx-version-min=10.13 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o nlme.so chol.o corStruct.o gnls.o init.o matrix.o nlOptimizer.o nlme.o nlmefit.o pdMat.o pythag.o rs.o  -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
ld: framework not found CoreFoundation
clang-7: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [nlme.so] Error 1
ERROR: compilation failed for package ‘nlme’
* removing ‘/Library/Frameworks/R.framework/Versions/4.0/Resources/library/nlme’
* restoring previous ‘/Library/Frameworks/R.framework/Versions/4.0/Resources/library/nlme’
Warning in install.packages :
  installation of package ‘nlme’ had non-zero exit status

The downloaded source packages are in
    ‘/private/var/folders/_3/b8whcf8d1bb8w_lr2rrscb5m0000gp/T/RtmpWCjwfs/downloaded_packages’

I've tried reinstalling xcode and Command Line Tools and completed Brew update + upgrade. Any suggestions appreciated.

geotheory
  • 21,259
  • 25
  • 104
  • 184
  • Just spotted this (ignored) question which looks related https://stackoverflow.com/questions/65047685/r-package-installation-fail-ld-framework-not-found-corefoundation-clang-7-err – geotheory Dec 11 '20 at 13:17
  • 2
    Lots of other people stumbling upon the same issue. https://pbs.twimg.com/media/EbLLgorUcAEpWYf?format=png&name=900x900 – IRTFM Dec 13 '20 at 21:20
  • Thanks @IRTFM. Though very fristrating! – geotheory Dec 13 '20 at 23:07
  • Might be better to communicate with the R-MAC-sig mailing list. – IRTFM Dec 14 '20 at 07:24
  • 1
    I'm stuck with the same problem. I've tried `r-macos-rtools` but the installation does not work for Big Sur. Here is some info on how to fix the installation, though I was not able to: https://github.com/rmacoslib/r-macos-rtools/issues/42 – epsilone Dec 15 '20 at 19:58

1 Answers1

25

This has been challenging, but here are the steps I used compile R packages from source on MacOS Big Sur:

  1. Reinstall xcode command line tools

(don't believe Software Update if it says 'up to date' - he lies - brew doctor said my version was actually old)

sudo rm -rf /Library/Developer/CommandLineTools
sudo xcode-select --install
  1. Install gcc & llvm via Homebrew (instructions for installing Homebrew) or, if you already have gcc/llvm installed, skip to the next step
# WARNING: This can take several hours
brew install gcc
brew install llvm
  1. If you already have gcc & llvm installed via Homebrew:
brew cleanup
brew update
brew upgrade
brew reinstall gcc
brew reinstall llvm
  1. Link some headers into /usr/local/include
sudo ln -s /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/* /usr/local/include/

# I believe you can safely ignore warnings like this:
#ln: /usr/local/include//tcl.h: File exists
#ln: /usr/local/include//tclDecls.h: File exists
#ln: /usr/local/include//tclPlatDecls.h: File exists
#ln: /usr/local/include//tclTomMath.h: File exists
#ln: /usr/local/include//tclTomMathDecls.h: File exists
#ln: /usr/local/include//tk.h: File exists
#ln: /usr/local/include//tkDecls.h: File exists
#ln: /usr/local/include//tkPlatDecls.h: File exists
  1. Edit your ~/.R/Makevars file (if you don't have a file called Makevars in your ~/.R/ directory, create it) and include only these lines:
FLIBS=-L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin19/10.2.0 -L/usr/local/gfortran/lib -lgfortran -lquadmath -lm
CXX1X=/usr/local/gfortran/bin/g++
CXX98=/usr/local/gfortran/bin/g++
CXX11=/usr/local/gfortran/bin/g++
CXX14=/usr/local/gfortran/bin/g++
CXX17=/usr/local/gfortran/bin/g++

LLVM_LOC = /usr/local/opt/llvm
CC=/usr/local/gfortran/bin/gcc -fopenmp
CXX=/usr/local/gfortran/bin/g++ -fopenmp
CFLAGS=-g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe
CXXFLAGS=-g -O3 -Wall -pedantic -std=c++11 -mtune=native -pipe
LDFLAGS=-L/usr/local/opt/gettext/lib -L$(LLVM_LOC)/lib -Wl,-rpath,$(LLVM_LOC)/lib
CPPFLAGS=-I/usr/local/opt/gettext/include -I$(LLVM_LOC)/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include
  1. Compile a package from source in R/Rstudio
# Compile the nlme package from source
install.packages("nlme", type = "source")

# To check whether openmp is enabled, compile data.table:
install.packages("data.table", type = "source")

Update (2022/04/05)

I had an error when updating "RcppAlgos" (couldn't find gmp.h or libgmp). I checked gmp was installed (brew install gmp), then added /usr/local/include to CPPFLAGS and /usr/local/lib to LDFLAGS in the ~/.R/Makevars file to solve the problem, i.e.

cat ~/.R/Makevars
LOC=/usr/local/gfortran
CC=$(LOC)/bin/gcc -fopenmp
CXX=$(LOC)/bin/g++ -fopenmp
CXX11 = $(LOC)/bin/g++ -fopenmp

CFLAGS=-g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe
CXXFLAGS=-g -O3 -Wall -pedantic -std=c++11 -mtune=native -pipe
LDFLAGS=-L$(LOC)/lib -Wl,-rpath,$(LOC)/lib,-L/usr/local/lib
CPPFLAGS=-I$(LOC)/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/usr/local/include

FLIBS=-L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin19/10.2.0 -L/usr/local/gfortran/lib -lgfortran -lquadmath -lm
CXX1X=/usr/local/gfortran/bin/g++
CXX98=/usr/local/gfortran/bin/g++
CXX11=/usr/local/gfortran/bin/g++
CXX14=/usr/local/gfortran/bin/g++
CXX17=/usr/local/gfortran/bin/g++
jared_mamrot
  • 14,156
  • 3
  • 17
  • 38
  • 1
    Thanks Jared, I get `$ git -C "/usr/local/Homebrew/Library/Taps/homebrew/homebrew-core" fetch --unshallow fatal: --unshallow on a complete repository does not make sense` Any idea why? – geotheory Dec 17 '20 at 13:38
  • My homebrew is currently a mess due to QGIS, but hopefully once I've resolved that in next day or so your solution will work. – geotheory Dec 18 '20 at 00:38
  • 1
    Brew fixed and your workflow fixed this install. Many thanks :) – geotheory Dec 20 '20 at 00:44
  • After doing all this I still get the error about `ld: framework not found CoreFoundation` – Florag Dec 24 '20 at 09:32
  • Trying to figure out why it didn't work.... one thing is that my version of gfortran seems to be using the libraries of a different gcc version to that specified in the Makevars file above. Mine has `/usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0`. Changing the makevars to use that doesn't help. – Florag Dec 24 '20 at 09:55
  • Thanks for responding! Have now gone through it all again both in whole, and rerunning `brew reinstall gcc` again after seeing your message.. I still have the x86_64-apple-darwin15/6.1.0 folder there. Any suggestions would be very, very welcome! – Florag Dec 24 '20 at 12:31
  • Just noticed that when I run `reinstall gcc` it seems to pour it to `/usr/local/Cellar/gcc/10.2.0`. Any idea why it would use this address instead? – Florag Dec 24 '20 at 13:53
  • 4
    After all steps and restarting I got `ld: framework not found CoreFoundation`. Installing fresh gfortran for R/Mac worked for me: https://github.com/fxcoudert/gfortran-for-macOS/releases/tag/10.2-bigsur-intel – speleo Dec 29 '20 at 21:45
  • 1
    @speleo, that's it! Installing gfortran from that link solved it for me. Was a bit hesitant given that the repo is just from a researcher, but it's got my R working again, so many, many thanks! – Florag Jan 03 '21 at 12:01
  • 1
    There's actually an alternative suggestion to installing that gfortran link. That is, to use the version of gfortran that brew provides with gcc, but I think this requires different makevars lines to those provided above. See this link: https://github.com/Homebrew/homebrew-cask/issues/96135 – Florag Jan 03 '21 at 13:41
  • Glad you got it working @Florag! Sorry I was offline over Christmas/New Year's and couldn't help you solve the issue. My understanding was that `brew reinstall gcc` linked brew's 'gcc' fortran to /usr/local/gfortran but apparently that's not the case on other people's systems - thanks @speleo for the additional troubleshooting! – jared_mamrot Jan 03 '21 at 22:37
  • @speleo and @jared_mamrot, you both saved me, thank you! I get the message `ld: warning: dylib (/usr/local/opt/llvm/lib/libunwind.dylib) was built for newer macOS version (11.0) than being linked (10.16)` after using the fresh gfortran linked. Do you also? – user551504 Feb 02 '21 at 03:04
  • 1
    @user551504 that warning is related to the statically linked llvm libraries. You can avoid the warning by removing llvm (`brew uninstall llvm`) and the references to llvm from the .R/Makevars file, and many packages will still compile but some will fail. Unfortunately, there are still some packages that I couldn't get to compile after following these steps - e.g. rstan - but the majority of packages compile successfully. – jared_mamrot Feb 04 '21 at 23:19
  • @jared_mamrot what if it says `Error: unexpected symbol in "sudo rm"` in the first step? – nilsinelabore Apr 20 '21 at 01:04