Can't connect to a TLS-enabled Vault on JRuby

[dmaze]

We're using JRuby-9.1.16.0 against a TLS-enabled Vault. The very most minimal Vault-client script:

require 'vault'

Vault::Client.new.help('/')

Fails when you run it:

$ rvm use jruby-9.1.16.0@vault --create
$ gem install vault
$ VAULT_ADDR=https://localhost:8200 ruby tmp.rb
OpenSSL::SSL::SSLError: no cipher match
        ciphers= at org/jruby/ext/openssl/SSLContext.java:508
      set_params at org/jruby/ext/openssl/SSLContext.java:562
         connect at /Users/dmaze/.rvm/rubies/jruby-9.1.16.0/lib/ruby/stdlib/net/http.rb:903
        do_start at /Users/dmaze/.rvm/rubies/jruby-9.1.16.0/lib/ruby/stdlib/net/http.rb:868
           start at /Users/dmaze/.rvm/rubies/jruby-9.1.16.0/lib/ruby/stdlib/net/http.rb:863
           start at /Users/dmaze/.rvm/gems/jruby-9.1.16.0@vault/gems/vault-0.11.0/lib/vault/persistent.rb:698
  connection_for at /Users/dmaze/.rvm/gems/jruby-9.1.16.0@vault/gems/vault-0.11.0/lib/vault/persistent.rb:625
         request at /Users/dmaze/.rvm/gems/jruby-9.1.16.0@vault/gems/vault-0.11.0/lib/vault/persistent.rb:933
         request at /Users/dmaze/.rvm/gems/jruby-9.1.16.0@vault/gems/vault-0.11.0/lib/vault/client.rb:273
             get at /Users/dmaze/.rvm/gems/jruby-9.1.16.0@vault/gems/vault-0.11.0/lib/vault/client.rb:182
            help at /Users/dmaze/.rvm/gems/jruby-9.1.16.0@vault/gems/vault-0.11.0/lib/vault/api/help.rb:29
          <main> at tmp.rb:3

(Very low-level debugging with Wireshark indicates that this never gets past the initial TCP connection: something must be listening on the indicated port but it doesn't necessarily need to be Vault or anything knowledgable about TLS; you get the same behavior if you launch a parallel nc -l 8200 and point the script at that.)

It looks like Vault's default SSL client cipher suite is "TLSv1.2:!aNULL:!eNULL" but JRuby's OpenSSL implementation doesn't understand this:

$ irb
jruby-9.1.16.0 :001 > require 'openssl'
 => true
jruby-9.1.16.0 :002 > ctx = OpenSSL::SSL::SSLContext.new
 => #<OpenSSL::SSL::SSLContext:0x1a942c18>
jruby-9.1.16.0 :003 > ctx.set_params ciphers: "TLSv1.2:!aNULL:!eNULL"
OpenSSL::SSL::SSLError: no cipher match
	from org/jruby/ext/openssl/SSLContext.java:508:in `ciphers='
	from org/jruby/ext/openssl/SSLContext.java:562:in `set_params'

If you go in and look at the list of ciphers available by default:

$ irb
jruby-9.1.16.0 :001 > require 'openssl'
 => true
jruby-9.1.16.0 :002 > ctx = OpenSSL::SSL::SSLContext.new
 => #<OpenSSL::SSL::SSLContext:0x55a147cc>
jruby-9.1.16.0 :003 > ctx.ciphers
 => (elided)
jruby-9.1.16.0 :004 > ctx.ciphers[0]
 => ["DES-CBC-SHA", "TLSv1/SSLv3", 56, 56]

You will notice that they all advertise TLSv1/SSLv3, but none of them advertise TLSv1.2. JRuby has a long hard-coded list of available ciphers and in particular there is an SSL_TLSv1 bit, but not a bit for "TLS v1.2". There are some worrisome very old JRuby issues, like jruby/jruby#1738, that suggest that a fix in the underlying platform may not be coming soon.

It looks like I can work around this by setting an environment variable VAULT_SSL_CIPHERS. If I have to set it to something else, is there a good value to set it to? I can construct a working value based on https://wiki.mozilla.org/Security/Server_Side_TLS#Mandatory_discards

VAULT_ADDR=https://localhost:8200 \
VAULT_SSL_CIPHERS=ALL:!aNULL:!eNULL:!EXPORT:!RC4:!DES:!SSLv2:!MD5 \
ruby ./tmp.rb

but I can't tell whether I'm telling the system to use something that's known to be unsafe.

It'd also be helpful if the Vault client library could detect it was running on JRuby and use a different default, if that's appropriate.

[annmuor]

Just got the same problem in puppet vault module.

[jayabalan1992]

I have been experiencing the same problem with vault. As it's going to be on production, I am not sure how far this workaround is going to help me. Any solution would be much appreciated.

This error, I am getting while querying the vault for the secret by running puppet agent -t from a node.

Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Function Call, no cipher match

[annmuor]

@jayabalan1992 ssl_ciphers option will help.

[jayabalan1992]

@annmuor You mean setting the env variable as suggested by @dmaze ?

[annmuor]

@jayabalan1992 I'm using config to init puppet vault module and added this to config.

[evanphx]

That sounds like a compatibility bug in the JRuby OpenSSL package. Thoughts @headius?

[jayabalan1992]

I resolved it by setting the SSL ciphers to TLSv1:!aNULL:!eNULL

[mschuchard]

Setting ssl_ciphers to TLSv1:!aNULL:!eNULL no longer works with Vault 0.10.3, vault-ruby 0.12.0, and jruby-openssl 0.9.21 (essentially what ships with Puppet Enterprise 2018.1.4 and recent Vault/vault-ruby). The Error while evaluating a Function Call, no cipher match has returned.

[headius]

More recent versions of JRuby do support a broader range of ciphers, but I'm not sure how to test if this issue is resolved.

If there's still missing ciphers or additional work to do, could someone here confirm and open an issue on https://github.com/jruby/jruby-openssl?

[jduepmeier]

I think the problem is that jruby-openssl does not known TLSv1.2 in the cipher string.

TLSv1.2 seems not to be defined.

I have this small test code adapted from https://jkutner.github.io/2016/05/10/jruby-ciphers.html:

#!/usr/bin/env ruby
# frozen_string_literal: true

require 'openssl'
require 'net/http'

uri = URI.parse('https://localhost:8200')
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.ciphers = [
  'TLSv1.2',
  'AES128+GCM+SHA256',
  'AES256+GCM+SHA384',
  'AES128+SHA256',
  'ECDHE+RSA+AES256+GCM+SHA384'
].join(':')
http.ssl_version = :TLSv1_2
# http.ciphers = OpenSSL::SSL::SSLContext.new.ciphers.map do |c|
#   c[0].gsub("-", "+")
# end
puts(http.ciphers)

http.get(uri.request_uri)

If you run this with jruby test.rb (i used rvm use jruby) it will work. But not with only TLSv1.2.

My jruby version is:

jruby 9.2.6.0 (2.5.3) 2019-02-11 15ba00b OpenJDK 64-Bit Server VM 15+36 on 15+36 +jit [darwin-x86_64]

[chi110r]

Hi,
i face the same error with puppetserver 7 fresh installed. does anyone a selution?

[laugmanuel]

@chi110r did you fix the problem? I'm seeing the same problem with Puppetserver 7.9.0 (the previous 7.8.0 worked fine). The reason seems to be the bump of Jruby itself: https://tickets.puppetlabs.com/browse/SERVER-3133

[mleneveut]

Still the same error on Puppet Server 8.7.0 using jruby 9.4.8.0 (3.1.4) 2024-07-02

Gem vault-0.18.2

Adding Vault.ssl_ciphers = "ALL:!aNULL:!eNULL:!EXPORT:!RC4:!DES:!SSLv2:!MD5" solves the problem when running

/opt/puppetlabs/server/apps/puppetserver/bin/puppetserver ruby test.rb

But I still get the no cipher match error when calling the Puppet Function from an agent

Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Function Call, no cipher match

require 'vault'

module Puppet::Parser::Functions
    newfunction(:getVaultPassword, :type => :rvalue) do |args|
    
    vault_token = args[2]
    secret_path = "secrets/data/" + args[3]
    vault_address =  args[4]

    Vault.address = "https://" + vault_address
    Vault.token = vault_token
    Vault.ssl_ciphers = "ALL:!aNULL:!eNULL:!EXPORT:!RC4:!DES:!SSLv2:!MD5"

    Vault.with_retries(Vault::HTTPConnectionError, Vault::HTTPError) do |attempt, e|
            begin
                secret = Vault.logical.read(secret_path)
                unless secret.nil? || secret.data.nil?
                    retrieved_secret = secret.data[:data][:password]
                end
            end
     end

     return retrieved_secret
end