GoCD is a popular Java CI/CD solution with a large range of users from NGOs to Fortune 500 companies with billions of dollars in revenue. Naturally, this makes it a critical piece of infrastructure and an extremely attractive target for attackers. In our previous article, Agent 007: Pre-Auth Takeover of Build Pipelines in GoCD, we demonstrated how unauthenticated attackers could impersonate build agents and access features that were previously protected by authentication mechanisms (CVE-2021-43287), leading to the disclosure of credentials and sensitive tokens for third-party services.
In this follow-up article, we describe three additional vulnerabilities discovered and responsibly disclosed by the SonarSource R&D team in GoCD 21.2.0 and below. First, a vulnerability that can be used by attackers impersonating build agents to force administrators to perform security-sensitive actions without their knowledge (CVE-2021-43288). Then, two additional vulnerabilities that could be chained with the first one to fully compromise the targeted instance by executing arbitrary commands (CVE-2021-43286, CVE-2021-43289) on the server hosting GoCD. These findings are already addressed by the latest release of GoCD: this article aims to share our root cause analysis and insights on how they could be exploited.
A threat actor taking advantage of these vulnerabilities could gain control of components within a release pipeline and leak intellectual property or include backdoors in the company's software. As an example, think about the SolarWinds hack, where attackers gained access to the software delivery pipeline and added a backdoor to critical software, leading to one of the most impactful supply-chain attacks thus far.
These three additional vulnerabilities in GoCD can be exploited by attackers who bypassed the mandatory authentication and obtained Agent privileges as presented in Agent 007: Pre-Auth Takeover of Build Pipelines in GoCD using CVE-2021-43287.
The first one is a Stored Cross-Site Scripting vulnerability that allows attackers to impersonate administrators after the visit of a poisoned job status page. To replicate what real-world attackers could do, we identified another two post-authentication vulnerabilities that can lead to the execution of arbitrary commands on the server when chained with the cross-site scripting vulnerability. Here is a representation of how they could be connected by attackers to compromise the server:
Attackers exploiting these findings could leak API keys to external services such as Docker Hub and GitHub, steal private source code, get access to production environments, and overwrite files that are being produced as part of the build processes, leading to supply-chain attacks.
All our findings including the ones presented in our first article were addressed in GoCD v21.3.0, available since October 26th.
Our exploit video demonstrates how the Stored Cross-Site Scripting can be triggered and used to take the control of an unpatched GoCD instance:
The three findings we describe in this article are all related to agent tasks and the way they communicate their results back to the GoCD server. From an architectural perspective, agents can be considered a special kind of user with a different HTTP API and means of authentication. They are identified with a UUID transmitted in the
X-Agent-GUID header and an HMAC of this value in Authorization.
They get new jobs by calling
/go/remoting/api/agent/get_work at regular intervals with a
GetWorkRequest packet. When a pipeline should run and an agent is chosen for the workload, the server provides the agent with all the necessary information. This includes the commands to run, and the secrets and environment variables to use.
While performing the pre-defined actions for their tasks, they send their status (e.g. building, passed, etc.), the console output, and eventual files and folders resulting from the build (also named “artifacts”) back to the server. These two last elements are sent over the
This first finding is related to the job status page, which displays everything about jobs, including tests, a tree display of artifacts (files, folders), and a console-like presentation of logs.
Let’s take a look at the source code behind this feature. GoCD implements its own server-side presentation layer: controller code has to create and fill
HtmlElement objects, which will later be sent back to the client after being processed by a
The rendering of the Artifacts tab is implemented in
, it iterates over
DirectoryEntry objects and call their
toHtml() method and passes it to the presentation renderer at
For both directories and files in the artifacts list, the final HTML code is generated based on the entry name, without further sanitization:
Attackers impersonating agents can exploit this weakness by sending artifacts with malicious names to inject arbitrary HTML elements into the page, such as
As shown in the capture below, the persistently stored payload will then be executed as soon as the job status page is opened by the victim. This page is likely to be visited by administrators if attackers deliberately fail CI jobs to get their attention.
The maintainers addressed this vulnerability in f5c1d2a, in which they introduced the use of
org.apache.commons.text.StringEscapeUtils to escape names of files and folders during their rendering as HTML elements.
With the help of the Stored Cross-Site Scripting vulnerability we described in the first section, attackers could force authenticated users to perform arbitrary actions without their knowledge, like disabling authentication or exploiting vulnerabilities that would not be reachable by the attacker otherwise.
To demonstrate this risk, the SonarSource Vulnerability Research team identified two additional vulnerabilities that can be chained with the Stored Cross-Site Scripting in order to gain arbitrary code execution on the GoCD instance.
The first finding is related to the way artifacts are written on the local filesystem: a parameter used by the application to craft the final destination path of the artifact is not validated. This behavior allows attackers to write files with arbitrary content to an arbitrary location.
A second vulnerability was discovered in the way GoCD processes the URLs of remote code repositories. Because of insufficient validation of these values, the behavior of external commands invoked by GoCD can be altered.
Console output and artifacts are sent over the
/go/remoting/files/ endpoint. The handler is found in
ArtifactsController.java, and is implemented as follows:
This code snippet is condensed for clarity, but three distinct steps can be identified:
, the value of
filePathis validated to prevent path traversal attacks;
, various objects are created to keep track of the current job, artifact name, etc and to format this data for the final stage;
, the artifact file is written to the local filesystem.
While the request parameter
filePath is validated to prevent path traversal vulnerabilities at
, that is not the case for the other request parameters, such as
Going deeper into the objects creation step (
), both a
JobIdentifier and a
StageIdentifier are instantiated. The role of these classes is to hold information about the CI job the incoming artifact is attached to, including values of the parameters
stageCounter, and so on. This information is later used to craft the path the artifact will be written to.
When the call to
putArtifact() is finally reached at
JobIdentifier object is used to craft the destination path of the artifact:
saveOrAppendFile() writes the file on the local filesystem:
The final destination path,
destPath, is based on
stageCounter, which is not validated: attackers can write files outside of the intended artifact directory.
When dynamically stepping through the code, several exploitation constraints arose:
- The name of the resulting file is fully controlled, but the file is written in a sub-folder whose name is not controlled and is based on the current job’s name;
filePathcan be empty, in which case the resulting file will be named with the current job’s name;
- When submitting a ZIP file, it will be safely extracted under a folder named based on the current job’s name.
Because of these restrictions, we didn't find a way to gain arbitrary code execution without another intermediary step, even with a powerful exploitation capability like this one. (Did you? Let us know!). Since the final part of the destination path is based on the job name, attackers could use the Cross-Site Scripting vulnerability to force administrators to first create a job whose name is the destination they want to write to.
To exploit this vulnerability, the next step is to identify files and folders that are writable by the user under which the GoCD server is running and that may have a security impact if created or modified. Attackers usually try to target configuration files or directories where plugins can be installed, but GoCD does not automatically reload them upon new changes.
We used the debugging tool strace to identify files that are accessed when browsing the GoCD interface, and noticed that the GoCD java processes tried to load Ruby (ERB) templates:
While intriguing at first, this behaviour can be explained by the presence of a Ruby On Rails application exposed under
/go/rails/. When reaching non-cached pages of this subsystem, the Ruby On Rails rendering engine searches for templates at several locations: here,
Creating one of them (e.g.
/go-working-dir/work/jetty-0_0_0_0-8153-cruise_war-_go-any-/webapp/WEB-INF/rails/app/views/shared/error.en.html.erb) and browsing an invalid page below
/go/rails/ loads this template, renders its contents and grants arbitrary code execution.
This issue was addressed by improving the validation of URLs and branch names in two commits on
- c22e042: the new method
stageCounteris a positive integer by using
Integer.parseInt()in POST and PUT handlers.
- 4c4bb47: the same method is applied in the GET handler.
By exploiting the Stored Cross-Site Scripting, attackers could also force administrators to create a new pipeline or configuration repository. This new repository would be cloned automatically by the server using external tools: for instance, referencing a Git repository will invoke the system-wide git command.
This logic is implemented in
domain/src/main/java/com/thoughtworks/go/domain/materials/. The method
checkConnection() of classes is called when the Test Connection button is clicked or when a repository is created:
For Git, it is implemented as follows in
If you remember our previous post about the PHP Supply Chain Attack on Composer, you have probably already identified the vulnerability: the variable
repoUrl is user-controlled, its format is not validated, and it is concatenated into the command line.
withArg() takes care of quoting the
repoUrl value, which mitigates the risk of a command injection but does not prevent attackers from adding unintended arguments with the prefix
The combination of three factors lead to a best-case scenario for exploitation:
- An argument can be added, without character set restriction;
git ls-remoterequires a positional argument, and the server will always add
refs/heads/masterin the call;
--upload-pack, an option to specify the path of the executable
--upload-pack=... in the URL field will result in the execution of the following command:
The refs/heads/master is the first positional argument: it forces git to treat it as a repository location. The value of the injection option
--upload-pack has the specificity to be invoked as an external command even in the case of local repositories. As an example, using
--upload-pack=”$(id>/tmp/id)” in the URL field confirms that attackers can gain arbitrary command execution:
We gave the focus on
git, but note that other handlers (SVN) were also vulnerable.
This issue was addressed with the commit 6fa9fb7, in which developers added stronger validation on user-controlled values, and started using the end-of-options delimiter -- standardized by POSIX.
|2021-10-18 - 2021-10-21||We report these findings to GoCD on HackerOne.|
|2021-10-18||GoCD confirms both issues.|
|2021-10-23||GoCD pushes patches on their GitHub repository.|
|2021-10-22||GoCD gives a heads-up about an important Security Fix coming up on their public Google Forum|
|2021-10-24||GoCD sends us the experimental installer for release v21.3.0.|
|2021-10-25||We verify the new version is secured against these vulnerabilities.|
|2021-10-26||GoCD releases version v21.3.0.|
|2021-11-04||CVE-2021-43286, CVE-2021-43288, CVE-2021-43289, and CVE-2021-43290 are assigned to these findings.|
In our previous blog post, we described a critical vulnerability that allowed unauthenticated attackers to get remote access to any GoCD installation. In this blog post, we described three additional vulnerabilities that could have been used by attackers to compromise a GoCD instance and to take over the underlying server.
We highly recommend that all users running GoC upgrade to the latest version (>= 21.3.0), since it includes patches for all the vulnerabilities we presented so far.
We would like to thank the GoCD Security Team which has been exceptionally responsive in the disclosure process. They reacted very quickly and worked with us to patch the vulnerabilities efficiently.