Summary

Nightly builds have started to fail in our CI/CD pipelines and the error is reproducible locally.

Errors are:

• cucumber features/host.feature:14 # Scenario: API key of a a host can be rotated.
• SimpleCov failed with exit 1Coverage report generated for Cucumber Features, RSpec to /src/conjur-api/coverage/coverage.xml. 1415 / 1538 LOC (92.0%) covered.

Steps to Reproduce

1. Check out repo locally
2. Run ./test.sh

Expected Results

Passing tests

Actual Results (including error logs, if applicable)

Tests fail

Reproducible

• Always
• Sometimes
• Non-Reproducible

Version/Tag number

Latest

Environment setup

Jenkins and local macOS env

Additional Information

The problem may have been caused by server behavior change in regards to rotation of API keys that can be found here.

Currently a lot of methods are global or use a global state, which makes it difficult if not impossible to maintain multiple Conjur API connections (configured differently) in the same process. Just a look through the list if Conjur::API class methods reveals many offenders, the biggest being .url_for. It's likely removing that and fixing what breaks will be a good first step.

The ruby client for the CyberArk Conjur API has tests today that are run only against the OSS.

As part of the configurable TLS change and because the client libraries are run and used by our customers we should additional tests that run against the EE to reduce the manual testing around these,

We recently moved our Jenkins configuration for master over to a pipeline by calling jenkins.sh from the Jenkinsfile. We would now like to split up the jenkins.sh script into more discrete steps to make the pipeline easier to work with.

When providing the URL to a custom authenticator, the URL should include the path prefix. The path prefix should not be assumed to be /authn.

Validate and fix any errors with the following workflows:

• Loading Policy with users and hosts with API keys disabled shows policy load success/failure, but not API keys
• Loading Policy updating users and hosts with previously enabled API keys shows policy load success/failure
• Loading Policy updating users and hosts with previously disabled API keys shows policy load success/failure and generated API keys
• Attempt to rotate disabled API results in a helpful error message
• Attempt to authenticate as a user/host with a disabled API key results in a authentication failure

Related to: cyberark/conjur/issues/1359

The master branch of the Ruby API is currently failing on an error before the cucumber tests even get run. It's happening because a new version of Cucumber was just released and we do not have the gem version pinned. Pin it back to the latest working version.

Following the release of Conjur v1.9.0 (DAP 11.7+) which included a medium security bug fix, the methods for API key rotation need to be updated to ensure they follow the new guidelines.

This should be fixed in the Ruby client in #181, and once those changes are in we will need a new release.

AC:

• There is a new release of the Ruby client including the fixes from #181

Is your feature request related to a problem? Please describe.

We need to configure custom connection parameters for accessing the Conjur API, like proxies, TLS certificates and other TLS-related options. However, we can’t apply these settings globally because we have other uses of RestClient with different parameters.

Describe the solution you would like

Given conjur-api has a configuration object, having an extra key for passing a Hash of RestClient options sounds like a natural solution.

Conjur.configuration.rest_client_options = {
  ssl_ca_file: "ca_certificate.pem",
  proxy: "http://proxy.example.com/"
}

I noticed conjur-api patches RestClient for applying custom parameters, meaning it potentially conflicts with other gems that use RestClient too. If conjur-api supported local RestClient options, that monkey patching would not be needed.

I also noticed Conjur::Configuration has options ssl_certificate and cert_file, but they do not seem to apply when using Conjur::API.new_from_key.

Describe alternatives you have considered

Isolating the calls to the Conjur in a separate Ruby environment so that the global variables do not interfere, but that sounds quite overkill.

• Make it always visible
• Add "Copy to clipboard" text that appears on hover.
• Change the image to an svg provided by Ty

Is the optimizations branch ready to pull? Is there anything backwards-incompatible in it?

What's the best way to finish it off?

Audit API appears to force a pattern like this:

audit_events = nil
api = Conjur::API.new_from_key admin_user, admin_password
api.events do |events|
  audit_events = events
end

With cyberark/conjur#1219, Conjur now supports managing authenticators through the API.

The Ruby API for Conjur should also include methods for interacting with these new endpoints from Ruby.

Currently, TokenExpiration is hard-coded to 4 minutes.

With Slosilo 2.1, tokens can have a customizable expiration. TokenExpiration should account for this possibility.

Currently, attempts to pass strings as arguments to Sequel filters will be met with a deprecation warning. We can solve this by integrating the auto_literal_strings extension:

http://sequel.jeremyevans.net/rdoc-plugins/files/lib/sequel/extensions/auto_literal_strings_rb.html

Since we don't server external requests with possum, the SQL injection concern mentioned in the above documentation should not be a concern.

Failing Jenkins Build

https://jenkins.conjur.net/job/cyberark--conjur-api-ruby/job/master/208/

Failing Tests

cucumber features/new_api.feature:5 # Scenario: From API key.
cucumber features/new_api.feature:13 # Scenario: From access token.
cucumber features/new_api.feature:24 # Scenario: From access token file.

Example Test Output

  Scenario: From access token file. # features/new_api.feature:24
    Given I run the code:           # features/step_definitions/api_steps.rb:1
      """
      token = Conjur::API.new_from_key("host/#{@host_id}", @host_api_key).token
      @temp_file = Tempfile.new("token.json")
      @temp_file.write(token.to_json)
      @temp_file.flush
      """
    Then I run the code:            # features/step_definitions/api_steps.rb:1
      """
      api = Conjur::API.new_from_token_file @temp_file.path
      expect(api.resource("cucumber:host:#{@host_id}")).to exist
      """
      expected #<Conjur::Host:0x000055650cbc2778 @id=#<Conjur::Id:0x000055650cbc2c50 @id="cucumber:host:app-515f935a...pocnhaOUc4QUZ5aDRqZ0xURzMwaGp3dmlycjQifQ==\""}, :username=>nil}, @v5_router=Conjur::API::Router::V5> to exist (RSpec::Expectations::ExpectationNotMetError)
      ./features/step_definitions/api_steps.rb:2:in `eval'
      (eval):2:in `block in <top (required)>'
      ./features/step_definitions/api_steps.rb:2:in `eval'
      ./features/step_definitions/api_steps.rb:2:in `/^I(?: can)? run the code:$/'
      features/new_api.feature:32:in `Then I run the code:'

Problem Description

When we use fully_escape to make Resource IDs URL safe, we don't currently escape the period (.) character. This can lead to resource URLs resulting in a 404 response from the Conjur API because it parses the period as an extension (.json) instead of part of the URL path.

Example for user with ID my.user:

"GET /api/audit/roles/cucumber%3Auser%3Amy.user?limit=10 HTTP/1.1" 404

Expected Outcome:

Periods should be escaped with the sequence %2E. With this substitution the request above is correctly routed and handled:

"GET /api/audit/roles/cucumber%3Auser%3Amy%2Euser?limit=10 HTTP/1.1" 200

Is your feature request related to a problem? Please describe.

At current, the documentation recommends also installing the conjur-cli gem in order to configure this gem.

Describe the solution you would like

We'd like to move all of the configuration into this gem, and the CLI gem can depend on conjur-api instead of the other way around.

Describe alternatives you have considered

n/a

Additional context

Once an update is made here, the CLI will also need to be updated to depend on conjur-api and to remove duplicate code.

In order to better understand how robust our test coverage is for the Conjur API Gem, we need the following:

• Code coverage is available in Jenkins ¹
• Code coverage is visible in CodeClimate repo page

¹ Ruby example: https://jenkins.conjur.net/job/cyberark--conjur/job/master/935/Coverage_20Report/

They really return URLs. I propose to have .url method instead and deprecate .host.

Currently we run our tests on Ruby 2.3, which is EOL.

This should be updated to use an active version of Ruby.

Also considering implementing, or filing a follow up issue to test against multiple Ruby versions, similar to what the conjur-cli does.

Currently the library doesn't re-request tokens on expiration. While it's not currently a problem (no long-running operations ATM), it's conceivable that it'll become one. Furthermore it makes interactive usage very inconvenient.

If the repo has a changelog that doesn't meet the standard, do try to change earlier entries to match the standard.
If the repo doesn't have a changelog use this as a starter:

# Changelog
All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).

## [Unreleased]

Acceptance criteria

• CHANGELOG.md exists and follows the KeepAChangeLog format. For reference consult https://github.com/cyberark/conjur/blob/master/CHANGELOG.md.
• Add parse-changelog validation step to Jenkinsfile. For reference consult https://github.com/cyberark/conjur/blob/master/Jenkinsfile#L27.

We should rename all possum references to use "conjur" terminology.

Working with the conjur API in a ruby repl (e.g. pry) is frustrating
because it seems to use ruby's default Object#inspect. This results in
objects being printed like the following (with large strings in place
of the ellipses):

# <Conjur::Role:0x007fbbf19323d0
@block=nil, 
@options= 
{:headers=> 
{:authorization=> 
"..."}, 
:username=>"..."}, 
@url= 
"...">

It'd be really helpful if #inspect returned a more concise and
descriptive string like #<Conjur::Role 'stripe:layer:layer1'>.

If I try to use the Ruby API with a CONJUR_SSL_CERTIFICATE environment variable set to a blank value, I get the error:
ERR OpenSSL::X509::CertificateError: header too long

Ideally the Ruby API will treat a blank value for this env var the same as if this env var were unset.

Add a setting Configuration.major_version with corresponding environment variable CONJUR_MAJOR_VERSION. This setting will select v4 or v5 compatible routing.

Methods which are not available in the selected version will raise NoMethodError.

In configuration.rb:

add_option :major_version

Create a utility method which will delegate a method call to a v4 or v5 provider based on the configuration setting.

Use this utility method to implement the differences in behavior between v4 and v5.

Skeleton implementation on https://github.com/cyberark/conjur-api-ruby/tree/major_version

Summary

The latest published gem is 5.3.2 from May 6. The version of the gem is consistent with the latest version bump from January 10, but it wasn't published on January 10 - and there are no commits in the project from May 6.

Expected Results

Gems are only published when the repo is tagged

Actual Results (including error logs, if applicable)

A gem was published without a tag

Additional Information

To address this, we should update the pipeline to publish on tag-triggered builds and ensure the CONTRIBUTING is clear that to release a new gem, you should add a tag to the project.