julia quasi-broken after gcc update

[tcovert]

Hi @staticfloat.

I just did a brew update, which lead to a new version of gcc (I am now using version 7.10 apparently). Now I get linear algebra warnings when I start julia, like this:

tcovert-imac:~ tcovert$ julia
WARNING: Error during initialization of module LinAlg:
ErrorException("could not load library "libopenblas"
dlopen(libopenblas.dylib, 1): Library not loaded: /usr/local/opt/gcc/lib/gcc/6/libgfortran.3.dylib
  Referenced from: /usr/local/opt/openblas-julia/lib/libopenblas.dylib
  Reason: image not found")
WARNING: Error during initialization of module CHOLMOD:
ErrorException("could not load library "libcholmod"
dlopen(libcholmod.dylib, 1): Library not loaded: /usr/local/opt/gcc/lib/gcc/6/libgfortran.3.dylib
  Referenced from: /usr/local/opt/openblas-julia/lib/libopenblasp-r0.2.19.dylib
  Reason: image not found")
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.5.1 (2017-03-05 13:25 UTC)
 _/ |\__'_|_|_|\__'_|  |  homebrew-julia release
|__/                   |  x86_64-apple-darwin16.4.0

julia> 

My guess is that the only thing that needs to change is for julia to look for libgfortran in /usr/local/opt/gcc/lib/gcc/7/ instead of /usr/local/opt/gcc/lib/gcc/6/ but I don't know how to make that happen.

I tried uninstalling julia, arpack-julia, suite-sparse-julia and openblas-julia, and re-installing, but the error persists. FWIW I had a similar problem with R this morning and I solved it by re-building it from source, which I guess is not possible with julia home-brew?

Any advice? Thanks.

[joshbode]

Same issue here.

This fixes it for me until the julia and openblas-julia bottles are eventually rebuilt:

$ brew reinstall --build-from-source openblas-julia
$ ln -s /usr/local/Cellar/openblas-julia/0.2.19/lib/libopenblas{_penryn,}p-r0.2.19.dylib

The second command is a bit of a hack to avoid rebuilding julia from source, which is looking for /usr/local/opt/openblas-julia/lib/libopenblasp-r0.2.19.dylib and the name (e.g. penryn) may be slightly different depending upon your hardware. Alternatively, instead of the symlink, do:

$ brew reinstall --build-from-source julia

[tcovert]

@joshbode Thanks for the suggestion - that worked. I ended up just building openblas-julia from source anyway. I didn't realize that if you do brew install --build-from-source julia, home-brew only compiles julia and not the dependencies!

[dpo]

@tcovert If you want to build all dependencies from source, you can do

HOMEBREW_BUILD_FROM_SOURCE=1 brew install julia

[aseemmahajan]

Hi @staticfloat, I'm still getting an error where libopenblas looks for libgfortran in the gcc6 rather than the gcc7 directory, even after reinstalling gcc, julia, open-blas, suite-sparse, and arpack from their source (I've pasted the errors and warnings below). Would you have any suggestions? Thanks!

WARNING: Error during initialization of module LinAlg: ErrorException("could not load library "libopenblas" dlopen(libopenblas.dylib, 1): Library not loaded: /usr/local/opt/gcc/lib/gcc/6/libgfortran.3.dylib Referenced from: /usr/local/opt/openblas-julia/lib/libopenblas.dylib Reason: image not found") WARNING: Error during initialization of module CHOLMOD: ErrorException("could not load library "libcholmod" dlopen(libcholmod.dylib, 1): Library not loaded: /usr/local/opt/gcc/lib/gcc/6/libgfortran.3.dylib Referenced from: /usr/local/opt/openblas-julia/lib/libopenblasp-r0.2.19.dylib Reason: image not found") WARNING: Error during initialization of module LinAlg: ErrorException("could not load library "libopenblas" dlopen(libopenblas.dylib, 1): Library not loaded: /usr/local/opt/gcc/lib/gcc/6/libgfortran.3.dylib Referenced from: /usr/local/opt/openblas-julia/lib/libopenblas.dylib Reason: image not found") WARNING: Error during initialization of module CHOLMOD: ErrorException("could not load library "libcholmod" dlopen(libcholmod.dylib, 1): Library not loaded: /usr/local/opt/gcc/lib/gcc/6/libgfortran.3.dylib Referenced from: /usr/local/opt/openblas-julia/lib/libopenblasp-r0.2.19.dylib Reason: image not found") * core WARNING: Method definition f(Tuple{Vararg{Int64, #N<:Any}}, AbstractArray{#T<:Any, #N<:Any}) in module Main at /usr/local/Cellar/julia/0.5.1/share/julia/test/core.jl:723 overwritten at /usr/local/Cellar/julia/0.5.1/share/julia/test/core.jl:729. ERROR: LoadError: LoadError: could not load library "libopenblas" dlopen(libopenblas.dylib, 1): Library not loaded: /usr/local/opt/gcc/lib/gcc/6/libgfortran.3.dylib Referenced from: /usr/local/opt/openblas-julia/lib/libopenblas.dylib Reason: image not found in jlplt_openblas_get_config_25010 at /usr/local/Cellar/julia/0.5.1/lib/julia/sys.dylib:? in openblas_get_config() at ./linalg/blas.jl:88 in versioninfo(::IOStream, ::Bool) at ./interactiveutil.jl:228 in versioninfo() at ./interactiveutil.jl:191 in include_from_node1(::String) at ./loading.jl:488 in include_from_node1(::String) at /usr/local/Cellar/julia/0.5.1/lib/julia/sys.dylib:? in macro expansion at ./util.jl:230 [inlined] in runtests(::String) at /usr/local/Cellar/julia/0.5.1/share/julia/test/testdefs.jl:7 in (::Base.##620#621{##16#24,Tuple{String},Array{Any,1}})() at ./multi.jl:1030 in run_work_thunk(::Base.##620#621{##16#24,Tuple{String},Array{Any,1}}, ::Bool) at ./multi.jl:1001 in #remotecall_fetch#625(::Array{Any,1}, ::Function, ::Function, ::Base.LocalProcess, ::String, ::Vararg{String,N}) at ./multi.jl:1055 in remotecall_fetch(::Function, ::Base.LocalProcess, ::String, ::Vararg{String,N}) at ./multi.jl:1055 in #remotecall_fetch#629(::Array{Any,1}, ::Function, ::Function, ::Int64, ::String, ::Vararg{String,N}) at ./multi.jl:1080 in remotecall_fetch(::Function, ::Int64, ::String, ::Vararg{Any,N}) at ./multi.jl:1080 in macro expansion at /usr/local/Cellar/julia/0.5.1/share/julia/test/runtests.jl:44 [inlined] in (::##15#23{Array{Any,1},UInt64})() at ./task.jl:360 while loading /usr/local/Cellar/julia/0.5.1/share/julia/test/core.jl, in expression starting on line 2399 in #remotecall_fetch#625(::Array{Any,1}, ::Function, ::Function, ::Base.LocalProcess, ::String, ::Vararg{String,N}) at ./multi.jl:1056 in remotecall_fetch(::Function, ::Base.LocalProcess, ::String, ::Vararg{String,N}) at ./multi.jl:1055 in #remotecall_fetch#629(::Array{Any,1}, ::Function, ::Function, ::Int64, ::String, ::Vararg{String,N}) at ./multi.jl:1080 in remotecall_fetch(::Function, ::Int64, ::String, ::Vararg{Any,N}) at ./multi.jl:1080 in macro expansion at /usr/local/Cellar/julia/0.5.1/share/julia/test/runtests.jl:44 [inlined] in (::##15#23{Array{Any,1},UInt64})() at ./task.jl:360 in sync_end() at ./task.jl:311 in macro expansion at ./task.jl:327 [inlined] in (::##11#19)() at /usr/local/Cellar/julia/0.5.1/share/julia/test/runtests.jl:37 in cd(::##11#19, ::String) at ./file.jl:59 in include_from_node1(::String) at ./loading.jl:488 in include_from_node1(::String) at /usr/local/Cellar/julia/0.5.1/lib/julia/sys.dylib:? in process_options(::Base.JLOptions) at ./client.jl:265 in _start() at ./client.jl:321 in _start() at /usr/local/Cellar/julia/0.5.1/lib/julia/sys.dylib:? while loading /usr/local/Cellar/julia/0.5.1/share/julia/test/runtests.jl, in expression starting on line 26 ERROR: could not load library "libopenblas" dlopen(libopenblas.dylib, 1): Library not loaded: /usr/local/opt/gcc/lib/gcc/6/libgfortran.3.dylib Referenced from: /usr/local/opt/openblas-julia/lib/libopenblas.dylib Reason: image not found in jlplt_openblas_get_config_25010 at /usr/local/Cellar/julia/0.5.1/lib/julia/sys.dylib:? in openblas_get_config() at ./linalg/blas.jl:88 in versioninfo(::Base.AbstractIOBuffer{Array{UInt8,1}}, ::Bool) at ./interactiveutil.jl:228 in runtests(::String, ::Int64) at ./interactiveutil.jl:570 in runtests(::String) at ./interactiveutil.jl:560 in eval(::Module, ::Any) at ./boot.jl:234 in eval(::Module, ::Any) at /usr/local/Cellar/julia/0.5.1/lib/julia/sys.dylib:? in process_options(::Base.JLOptions) at ./client.jl:242 in _start() at ./client.jl:321 in _start() at /usr/local/Cellar/julia/0.5.1/lib/julia/sys.dylib:? Error: staticfloat/julia/julia: failed Failed executing: /usr/local/Cellar/julia/0.5.1/bin/julia -e Base.runtests("core") /usr/local/Homebrew/Library/Homebrew/formula.rb:1859:in block in system'
/usr/local/Homebrew/Library/Homebrew/formula.rb:1797:in open' /usr/local/Homebrew/Library/Homebrew/formula.rb:1797:in system'
/usr/local/Homebrew/Library/Taps/staticfloat/homebrew-julia/julia.rb:182:in test' /usr/local/Homebrew/Library/Homebrew/formula.rb:1694:in block (2 levels) in run_test'
/usr/local/Homebrew/Library/Homebrew/formula.rb:887:in with_logging' /usr/local/Homebrew/Library/Homebrew/formula.rb:1693:in block in run_test'
/usr/local/Homebrew/Library/Homebrew/extend/fileutils.rb:14:in block in mktemp' /usr/local/Homebrew/Library/Homebrew/extend/fileutils.rb:74:in block in run'
/usr/local/Homebrew/Library/Homebrew/extend/fileutils.rb:74:in chdir' /usr/local/Homebrew/Library/Homebrew/extend/fileutils.rb:74:in run'
/usr/local/Homebrew/Library/Homebrew/extend/fileutils.rb:13:in mktemp' /usr/local/Homebrew/Library/Homebrew/formula.rb:1687:in run_test'
/usr/local/Homebrew/Library/Homebrew/test.rb:28:in block in <main>' /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib/ruby/2.0.0/timeout.rb:66:in timeout'`

[dpo]

@aseemmahajan Make sure you don't have old builds of dependencies lying around. E.g., what's in /usr/local/Cellar/openblas-julia? If that doesn't help, could you paste here the output of brew gist-logs openblas-julia and brew gist-logs julia? (Those commands output URLs.)

[prairie-guy]

I had a working (brew installed) version of julia until I upgraded to OSx 10.12
I am now unable to successfully install julia with: brew install julia
I am using OSx 10.12.5 (16F73)
I have successfully executed the following: brew update, brew upgrade, brew doctor

Here is the output from gcc --version:

$ gcc --version Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk/usr/include/c++/4.2.1 Apple LLVM version 8.1.0 (clang-802.0.42) Target: x86_64-apple-darwin16.6.0 Thread model: posix InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

Per this thread, I have tried all of the following:

• brew reinstall --build-from-source openblas-julia

• brew reinstall --build-from-source julia

• HOMEBREW_BUILD_FROM_SOURCE=1 brew install julia

In all cases, I removed the dependencies before trying the installation of julia.

If I do a vanilla install and then launch julia, here is the output:

`$ julia
WARNING: Error during initialization of module LinAlg:
ErrorException("could not load library "libopenblas"
dlopen(libopenblas.dylib, 1): Library not loaded: /usr/local/opt/gcc/lib/gcc/6/libgfortran.3.dylib
Referenced from: /usr/local/opt/openblas-julia/lib/libopenblas.dylib
Reason: image not found")
WARNING: Error during initialization of module CHOLMOD:
ErrorException("could not load library "libcholmod"
dlopen(libcholmod.dylib, 1): Library not loaded: /usr/local/opt/gcc/lib/gcc/6/libgfortran.3.dylib
Referenced from: /usr/local/opt/openblas-julia/lib/libopenblasp-r0.2.19.dylib
Reason: image not found")

Version 0.5.1 (2017-03-05 13:25 UTC)
homebrew-julia release
x86_64-apple-darwin16.4.0 `

I'm stuck. Any advice would be greatly appreciated. I have been a long time julia user and have used it flawlessly in the past on OSx and Ubuntu without issue.

Thanks in advance.

[prairie-guy]

On further investigation, I note from the error:
Library not loaded: /usr/local/opt/gcc/lib/gcc/6/libgfortran.3.dylib
that gcc/6/ does not exist.

Instead, I have:
/usr/local/opt/gcc/lib/gcc/7/libgfortran.4.dylib

Not sure how to resolve this.

[prairie-guy]

Per a previous request, I have executed the following (still getting an error):

brew reinstall --build-from-source openblas-julia
brew reinstall --build-from-source julia

brew gist-logs openblas-julia
https://gist.github.com/f091b18a65bd3557c42e45d5072dbd39

[dpo]

Try brew uninstall --ignore-dependencies --force $(brew deps --recursive julia) followed by brew install julia.

[prairie-guy]

Thanks for helping!

I tried your suggestion:
brew uninstall --ignore-dependencies --force $(brew deps --recursive julia)
brew install julia.

Appears to be the same problem:
$ julia WARNING: Error during initialization of module LinAlg: ErrorException("could not load library "libopenblas" dlopen(libopenblas.dylib, 1): Library not loaded: /usr/local/opt/gcc/lib/gcc/6/libgfortran.3.dylib Referenced from: /usr/local/opt/openblas-julia/lib/libopenblas.dylib Reason: image not found")

WARNING: Error during initialization of module CHOLMOD: ErrorException("could not load library "libcholmod"

dlopen(libcholmod.dylib, 1): Library not loaded: /usr/local/opt/gcc/lib/gcc/6/libgfortran.3.dylib Referenced from: /usr/local/opt/openblas-julia/lib/libopenblasp-r0.2.19.dylib Reason: image not found")

Version 0.5.1 (2017-03-05 13:25 UTC) homebrew-julia release x86_64-apple-darwin16.4.0

gist-logs:
brew gist-logs openblas-julia
https://gist.github.com/8bfd683023afd72e7c84c502f294d071

brew gist-logs julia
https://gist.github.com/109f43b0d2324a1b77c6db902188356d

[dpo]

Sorry, the last command should have been HOMEBREW_BUILD_FROM_SOURCE=1 brew install julia to build from source. I'm not sure there are updated bottles for openblas. Have another try if possible.

[prairie-guy]

duo,

Still no luck. Per your suggestion, I tried:
brew uninstall --ignore-dependencies --force $(brew deps --recursive julia)
HOMEBREW_BUILD_FROM_SOURCE=1 brew install julia

Here are the dependencies:
gmp
mpfr
gcc
libffi
staticfloat/julia/llvm37-julia
pcre2
fftw
openssl
libssh2
libgit2
mbedtls
staticfloat/julia/openblas-julia
staticfloat/julia/arpack-julia
staticfloat/julia/suite-sparse-julia

Here are the last 15 lines leading up the to compile error of julia:
checking for sendfile in -lsendfile... no
checking for socket in -lsocket... no
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... no
checking sys/ahafs_evProds.h usability... no
checking sys/ahafs_evProds.h presence... no
checking for sys/ahafs_evProds.h... no
checking for pkg-config... yes
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating libuv.pc
config.status: creating Makefile
config.status: executing depfiles commands
config.status: executing libtool commands
make: *** [julia-deps] Error 2

gist-logs:
brew gist-logs julia
https://gist.github.com/8a8302dffab1963002cc358d523f0389

Not sure if it is an issue, but I note quite a few warnings as follows:
ld: warning: object file (/usr/local/opt/suite-sparse-julia/lib/libumfpack.a(umf_i_analyze.o)) was built for newer OSX version (10.12) than being linked (10.8)

Moreover, every warning is related to: suite-sparse-julia

Thanks again for you help. (I know I can just install julia as a dmg download, but I really like using brew to be my one-stop whenever possible.)

[prairie-guy]

Finally, I am up and running! It's a hack, which I don't understand, except that it works. Perhaps someone can look at this and figure out what I need to do to come up with a better fix.

This works:
brew uninstall --ignore-dependencies --force $(brew deps --recursive julia)
brew install gcc
cd /usr/local/opt/gcc/lib/gcc/
ln -s 7 6
cd 6
ln -s libgfortran.4.dylib libgfortran.3.dylib
brew install julia

julia then launches as: Version 0.5.1 (2017-03-05 13:25 UTC)

All tests are passed with:
/usr/local/opt/julia/bin/julia -e "Base.runtests()"

Clearly it appears to be a problem with julia or one or more of it's dependencies looking for gcc6 vs gcc7. I would really like to figure out if this is a problem on my side or if this is an issue caused by the brew packages themselves.

In any case, thanks for you help. Any feedback most welcome.

[dpo]

Here, you tricked the Julia bottle into believing that you have gcc 6 installed. I don't have much experience with the Julia build system, so it would be interesting if someone with experience could comment on the build error you reported above. @staticfloat ? @tkelman ?