Skip to main content

Sonatype for Azure DevOps

The Sonatype for Azure DevOps extension integrates with the Azure DevOps pipeline to run policy evaluations in the build workspace. It adds a new step within the build, during which applications are scanned by Sonatype IQ Server to identify any open-source security, license, or quality policy violations. It can be configured to fail the build or generate a warning. This allows the build maintainers to understand the reasons for build failures and plan a remediation strategy.

Main Features

  1. Perform a Sonatype IQ Server policy evaluation on files in the build workspace.

  2. Display scan results within Azure DevOps pipeline report.

  3. Provide a link to a comprehensive Sonatype Lifecycle policy evaluation report indicating violation details and remediation recommendations.

Installation

  1. Sign in to your Azure DevOps account at https://dev.azure.com.

  2. Click on "Organization Settings" button at the bottom left corner:

    126655777.png

  3. From the left-hand side menu choose "Extensions":

    126655776.png

  4. Click "Browse Marketplace" button in the top right corner of the page:

    126655775.png

  5. You will be redirected to the Azure DevOps Extensions marketplace.

  6. Enter "Sonatype IQ" in the search box to find the Sonatype for Azure DevOps.

  7. Click on the Sonatype for Azure DevOps tile, to access the extension's home page.

  8. Click on the "Get it free" button, to start the installation process.

  9. Follow the on-screen steps to complete the installation

  10. Sonatype for Azure DevOps icon will now be visible on the Organization Settings page as below:

    126655774.png

Updating to a newer version of Sonatype for Azure DevOps

By default, if a new version of the extension has been released in the Azure DevOps Marketplace, it is updated in your pipeline automatically.

For some cases, when there is a change in access permissions for the extension, manual user approval will be required to install the update. Follow the steps below:

  1. Go to "Organization settings" → "Extensions" (the same menu as the "Installation" section) → "Installed" extension tab

  2. An "Action required" message appears.

  3. Click on Sonatype for Azure DevOps.

  4. Click on the "Review" button.

  5. A menu with scope changes appears.

  6. Click on the "Authorize" button.

Sonatype for Azure DevOps is now updated to the new version.

Configuration

1. Sonatype IQ Service Configuration

  1. Open your project in Azure DevOps.

  2. Go to the "Project Settings" in the bottom left corner.

  3. Choose "Service Connections" from the menu:

    126655773.png

  4. Click "New Service Connection" button and choose "Sonatype IQ" from the list of connections:

    126655772.png

  5. Enter the URL and credentials for your Sonatype IQ server in the pop-up window and click the "OK" button:

    126655759.png

2. YAML Pipeline Configuration

  1. Select "Pipelines" → "Builds" from the left-hand side menu:

    126655770.png

  2. Create a "New build pipeline" if one does not exist:

    126655769.png

  3. The following message will appear:

    126655780.png

  4. Choose a repository where you host your project from the list above and follow the steps to create the pipeline.

  5. To edit an existing build pipeline, click on "Edit":

    126655768.png

  6. An azure-pipelines.yml file will be opened.

  7. Search for "Sonatype IQ" in the search box on the right-hand side of the page:

    126655767.png

  8. Click on found Sonatype IQ task extension to view a configuration screen:

    171672037.png

  9. Configure the Sonatype IQ task on the configuration page, as below:

    1. Sonatype IQ service connection (configured previously)

    2. Organization ID (optional): ID of the organization under which the application will be created if the automatic application creation is enabled and the application does not exist.

    3. Application ID: ID of the application to evaluate against as configured in Sonatype IQ

    4. Stage: Stage in IQ for the evaluation

    5. Scan Targets : Targets to perform policy evaluation are listed as comma-separated glob patterns (more details below). E.g. **/*.jar, **/*.json

    6. Use ${Pipeline.Workspace} as the base folder for scanning: If checked, the above scan targets are evaluated against the ${Pipeline.Workspace} folder and its descendants (more details below).

    7. Ignore IQ Server's system errors: Controls the pipeline outcome when the scan or evaluation fails to produce results for some (possibly intermittent) connection problem. Usually such a failure would result in a FAILURE of the pipeline, but ignoring system errors allows it to show as a WARNING on the pipeline. DNS and network connection failures can be ignored, but misconfiguration of Stages or Application IDs cannot be ignored.

    8. Ignore IQ Server's Scanning errors: Controls the pipeline outcome when there are scanning errors such as malformed files. Scanning errors result in a FAILURE of the pipeline if they are not ignored. Ignoring scanning errors will lead to pipeline finishing in a WARNING state instead of a FAILURE in case of scanning errors.

    9. Enable Debug Logging: Enables debug logging for IQ policy evaluation. This should be used only to troubleshoot any problem you may have when scanning a repository.

    10. Java System Properties (Advanced): Command line arguments to alter the behavior of the JVM. e.g. -Djava.net.useSystemProxies=false

  10. Click on the "Add" button, to update the YAML file with the above inputs.

  11. Save your build by clicking the "Save" button in the right top corner of the page:

    126655765.png

  12. Configuration of Sonatype for Azure DevOps is now complete and ready to run policy evaluations on your builds.

Scan Targets

The Scan Targets field allows for a fine-grained selection of the files on which the policy evaluation is performed. Scan targets are listed as comma-separated glob patterns. The supported glob patterns are described in the file matching patterns reference section of the Azure DevOps documentation.

The provided glob patterns are evaluated as follows:

  • If Use ${Pipeline.Workspace} as the base folder for scanning is checked, they are evaluated against the files located in the pipeline workspace directory (and its descendants)

  • Otherwise, they are evaluated against the files located in two directories:

    • ${System.DefaultWorkingDirectory} - the system default working directory - the local path on the agent where your source code files are downloaded. For example: c:\agent_work\1\s

    • ${Build.ArtifactStagingDirectory} - the build artifact staging directory - the local path on the agent where any artifacts are copied to before being pushed to their destination. For example: c:\agent_work\1\a

The union of all the files that match the provided glob patterns is used as target for the policy evaluation.

For example, the following scan target will consider for evaluation all the JAR and POM files found anywhere under the two directories mentioned above: **/*.jar, **/pom.xml

The Azure DevOps glob patterns also support exclude patterns (all above examples are include patterns). The exclude patterns start with a '!' character and must be specified after all other include patterns. For example, the following scan target will consider for evaluation all files except those under the '.git' directory: **, !**/.git/**

3. Optional: Using the classic editor to create a pipeline without YAML

  1. Create a new pipeline from the left menu, Pipelines → Build

  2. Click on the "Use the classic editor to create a pipeline without YAML" link at the bottom.

137203913.png

3. After creating a pipeline in the classic editor, add a Sonatype IQ pipeline task by clicking on the "+" icon on the "Agent job" tile:

126655779.png

4. Configure the Sonatype IQ task.

5. Enable the "Allow scripts to access the OAuth token" checkbox under the "Agent job" tile to enable widgets correctly, with settings as below:

126655778.png

Evaluating Policies

You can now start running policy evaluations as a part of your build.

Note that "NexusIqPipelineTask" appears as one of the tasks in the list, while running a build:

126655748.png

Accessing/Viewing Results

Open NexusIqPipelineTask to view a console output with the results of the evaluation:

126655751.png

The console output contains a summary of the policy evaluation and a link to the detailed report.

Click on the link on the console output to view the detailed report in Sonatype IQ Server.

Select the Sonatype IQ Build Report tab on the build for a detailed report with all the components and their correspondent violations:

126655747.png

Select the Sonatype IQ Summary Report tab on the build to see a summary report for the policy evaluation for the scanned components:

126655749.png

Add dashboard widgets for Sonatype IQ

For ease of use, the following widgets for Sonatype IQ can be added to Azure DevOps dashboard.

  1. Sonatype IQ Policy Evaluation widget: shows the policy evaluation results for the latest build.

  2. Trends for Sonatype IQ Policy Evaluation: shows a historical trend of Sonatype IQ Policy evaluations of the last 5 builds.

How to add Sonatype IQ widgets to Azure DevOps Dashboard:

  1. Go to the "Overview" → "Dashboards" and click the "Edit" button.

  2. On the right-hand side, under "Add Widget" search for the "Sonatype IQ"

    126655761.png

  3. Select the appropriate widget and click the "Add" button at the bottom right corner of the page. Click the "Done editing" button.

  4. Sonatype IQ widget now displays on the dashboard.

A sample screenshot for Sonatype IQ Policy Evaluation widget showing the results summary for the latest build:

126655752.png

A sample screenshot for Trends for Sonatype IQ Policy Evaluation widget showing the historical summary for the last 5 builds:

126655756.png

Running Sonatype IQ in Azure Self-hosted Agents

If you use an HTTP proxy within your infrastructure and use Azure self-host build agents, then the Azure DevOps agent options for a proxy can be specified and then will be automatically used while connecting to IQ. Refer: https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/proxy?view=azure-devops&tabs=windows

In the Azure provided sample command:

./config.sh --proxyurl http://127.0.0.1:8888 --proxyusername "myuser" --proxypassword "mypass"

This would appear in the scan output as it is passed through to the IQ scan client:

...
-p
127.0.0.1:8888
-U
myuser:***
...

Tips and Tricks

The Azure DevOps extension uses the Sonatype IQ CLI to perform the scan. Part of the IQ CLI scan process involves using git to determine the repository URL and commit hash. If native git is available on the agent then that is used, otherwise, jgit (java-git) is used. If jgit is used it tries to create some config files in the $HOME directory of the current user. If it does not have sufficient permissions to do that you may see logs on 'ERROR' level. These are not critical errors to the scan and can be ignored. To address the error, ensure native git is available or for jgit set the XDG_CONFIG_HOME environment variable to a directory that is writable by the build agent user.