Ideas for improving Conjur-Gitlab integration experience

[cmellazchy]

The current method of using a separate container image with this binary, as shown in the provided examples, is not practical for real-world CI/CD workflows.

Challenges:

• Tooling Expectations: Developers prefer using container images that already include the necessary tools for their jobs, rather than relying on a separate image just to inject secrets.
  Sample scenarios:
  • Testing an application within its own container image in a CI job, where access to secrets is required.
  • The need to use container images bootstrapped with cli tools like kubectl or azure cli for deployment.
• GitLab's Limitations: Unlike GitHub Actions, GitLab lacks job composability features, making it difficult to integrate this project into workflows.
  • GitLab is gradually improving in this area with the introduction of CI Steps in the runner, which could help solve some of these composability issues. This is going to introduce re-usable steps similar to Github Actions.

To work around these challenges, I built a GitLab CI/CD component that:

• Includes a hidden job that developers can reference within their job scripts.
• It simply executes shell scripts stored inline within the hidden job at the point of reference.
• Handles:
  • Downloading the binary
  • Parsing variables
  • Fetching secrets
  • Injecting secrets into environment variables

This is a sample GitLab CI workflow. In this example, .conjur_fetch_secrets is a hidden job defined in the conjur-linux CI/CD component. It contains shell script logic to download the required binary, parse conjur secrets, fetch them and export it to the user-defined environment variables.

Developers can use any container image that suits their use case while still being able to retrieve secrets. They achieve this by simply including the shell script from the hidden job within their before_script section.

include:
  - component: $CI_SERVER_FQDN/internal/components/conjur-ci-utils/conjur-linux@~latest

app_test_job:
  image: ubuntu:22.04
  id_tokens:
    CONJUR_AUTHN_JWT_TOKEN:
      aud: $CONJUR_APPLIANCE_URL
  variables:
    CONJUR_SECRETS: >
      path/to/variable/username|APP_USERNAME;
      path/to/variable/password|APP_PASSWORD
  before_script:
    - !reference [.conjur_fetch_secrets, script]
  script:
    # Consume the secret from the environment variable
    - curl -u "${APP_USERNAME}:${APP_PASSWORD}" https://myapp.example.intel.com/records
    - echo "Application successfully tested!"

Additional Considerations

• The Go binary needs to be statically compiled for compatibility across different container images, including both glibc and musl-based distributions.
• The CI job environment must have curl or wget available to download the binary at runtime.

This solution might not be ideal, but I wanted to share my take and contribute to the discussion to improve the situation in the future. Perhaps once Gitlab CI Steps are the de-facto standard, we could do better.

[cmellazchy]

I haven't completely thought this through yet but here's an idea: With Conjur CLI now supporting JWT authentication in Conjur v13.4, perhaps we could just replace this binary with the Conjur CLI to fetch secrets. I understand that it might require adding additional env vars, checks, parsing logic etc.