Icinga is a modern, open-source IT monitoring system with a web interface. Thanks to its specialized scripting language, it is highly configurable and can run checks on virtually any IT equipment. It also offers useful built-in plugins to query the state of services running on monitored hosts, such as running services, network traffic, or available disk space.
We recently discovered two code vulnerabilities in Icinga Web that allow attackers to compromise the server on which it is running by running arbitrary PHP code. As part of our research, we unveiled an unpatched bug in the PHP engine itself that enables the exploitation of one of the findings. This article presents the technical details of both vulnerabilities and how the maintainers fixed them.
It’s not common to discuss both PHP and C code in the same blog post; we will do our best to keep it fun. Let’s dive into it!
The most common way to deploy Icinga is to use the administration interface Icinga Web that communicates with the Icinga monitoring server.
We discovered a Path Traversal vulnerability (CVE-2022-24716) that can be abused to disclose any file on the server. It can be exploited without authentication and without prior knowledge of a user account. We also discovered CVE-2022-24715, which leads to the execution of arbitrary PHP code from the administration interface.
They can be easily chained to compromise the server from an unauthenticated position if the attacker can reach the database by first disclosing configuration files and modifying the administrator's password.
We strongly recommend updating your icingaweb2 instances to either 2.8.6, 2.9.6, or 2.10, even if they are not directly exposed to the Internet.
Although we won't be releasing a proof-of-concept, exploiting these findings is straightforward. We also recommend assuming that any secret present in the Icinga Web configuration (e.g. database credentials) could have been compromised; they should be rotated as a precautionary measure.
We assume that Icinga Web 2 was deployed using the upstream packages in version
2.9.5-1.hirsute and following the official documentation. As you will later see in the section CVE-2022-24715 - Remote Code Execution, this setup makes the exploitation slightly more complex for attackers and more interesting for us security researchers!
The Apache HTTP server is configured to dispatch all the incoming requests to
index.php using its module
mod_rewrite; this setup is very common for modern PHP applications to provide only one entry point:
This first script loads
webrouter.php and then tries to dispatch the request to the right software component based on the requested path:
- Important static resources (
css/icinga.min.css, etc.) are processed first, with support for the
ETagheader, minification and server-side cache;
- Dynamically-generated images (
png/chart.php) based on request parameters;
- Requests to paths starting with
lib/are handled by
- Everything else is handed to controllers.
Dynamic routers are always interesting components to review: they have to construct paths based on user-controlled data and are thus very prone to path traversal vulnerabilities; that’s what happens here!
The important code of
StaticController is shown below: it first iterates over existing libraries to find one matching the request URL and then concatenates the associated asset path to the a value provided by the client:
The code of
StaticController has two security issues:
- Libraries can declare an empty asset path, in which case the path to the file is constructed using only the user input; for instance,
- The user input can contain directory traversal sequences (
../), resulting in a final path outside the intended directory; for instance,
As a result, attackers can disclose any file of the local filesystem. We could confirm this vulnerability against the official demonstration instance, for instance by obtaining the contents of the file
Attackers can also target
incingaweb2 configuration files. Among other things, they contain database credentials used by the web interface.
If attackers can reach the database service, they can use these credentials to change the password of an existing account and gain authenticated access to the instance. We pursued this scenario and later found a way to execute arbitrary code on the instance thanks to this access (see below).
On non-default deployment, Icinga can also be told to use SSH private keys present on the local filesystem. They could be read using this technique and later pivot to other systems with the identity of the monitoring agent.
Authenticated users can edit resources to later reference them from other configuration files. One of the resource types is SSH keys, which require to be written to the local filesystem to be used.
We identified that no validation is performed on the parameter user of the
SshResourceForm at . It allows attackers to use directory traversal sequences (e.g.
../) to write the SSH key outside of the intended directory at :
Our first assumption was to consider this bug useless since SSH keys are validated with
openssl_pkey_get_private(); it doesn't sound easy to craft a PHP script that would also be a valid PEM certificate.
This function call being the only obstacle, it is worth investigating a bit deeper and taking the time to study its implementation. As mentioned in the documentation, this function is part of PHP's Cryptography Extensions; its code is located in php-src/ext/openssl.
While looking at this implementation in the PHP engine source code, one can notice a quirk specific to the OpenSSL module in PHP. Such libraries usually offer one way to load data, either based on the file's name that it will open and read or the data itself (in which case it's up to the user to handle any I/O operation).
Here, both methods are automatically supported: if the parameter
$private_key is prefixed with
file://, it reads the file for the user. Otherwise, this parameter is considered to be the value of the certificate.
This leads to some rather uncommon control flow in its implementation:
In the code snippet above,
zval *val is the internal representation of the private key submitted via the form.
val is binary-safe, which means that the PHP engine can work with the complete string even if it contains
NULL bytes by keeping track of its length in bytes alongside the data. However, the
libssl API (
BIO_*) only works with
NULL-terminated char arrays, which are inherently not binary-safe: processing will stop at the first
Attackers can use this quirk to circumvent the validation performed by
openssl_pkey_get_private() while keeping the ability to put arbitrary data in the resource file: PHP stops at the first
NULL byte while searching for the certificate on the disk, but the full data will be written to the destination file!
Attackers could then craft a payload in 4 parts:
- The mandatory prefix to enter the vulnerable code path,
- Path to a valid PEM certificate on the server, e.g.,
/usr/lib/python3/dist-packages/twisted/test/server.pemin our test virtual machine;
- The contents of the file to write, here a small PHP script executing an external command.
When installed using the official Linux packages, the PHP scripts of Icinga Web 2 are deployed under
/usr/share/icingaweb2. They are owned by the
root user and hence can't be modified with the identity of
www-data under which the HTTP server is running.
While this would prevent straightforward exploitation based on planting a PHP file under this directory and accessing them, we found another technique that attackers could use to obtain the execution of arbitrary code.
Icinga has a notion of modules, self-contained third-party code that extends the interface's capabilities (e.g., to add Grafana support). These modules are stored under
/usr/share/icingaweb2/modules by default, but administrators can also change this path directly from the interface.
global_module_path expects colon-separated paths from where modules are located. Changing this value to a path where the previously demonstrated vulnerability can write, say
/dev/, and enabling the new module named
shm allows executing arbitrary PHP code.
- The path is constructed;
realpath()is called: directory traversal sequences, symbolic links are resolved and ensure that the destination file exists;
- It made sure that the path resulting from the
realpath()call is still under the expected directory.
Further format validation is also performed on the value of the SSH resources before writing them to the disk to prevent the use of
We also reached out to the PHP maintainers to address the
NULL byte injection in the functions of the OpenSSL core extension. Because there isn’t any other function designed to validate the format of certificates, other software is likely using the same vulnerable functions.
We provided patches and test cases to ease their adoption by the maintainers; the bug ticket is still open as of the time of writing this article. Nevertheless, we chose to publicly document this bug as the security risk is deemed low, and an additional fix has been present Icinga Web 2 for several weeks.
|2022-02-15||We report the first path traversal vulnerability to Icinga.|
|2022-02-21||We report the second path traversal vulnerability to Icinga.|
|2022-02-23||Icinga acknowledges the vulnerabilities, GitHub advisories are created.|
|2022-03-10||The PHP bug is reported on the upstream bug tracker in #81713.|
|2022-03-14||Icinga releases icingaweb2 2.8.6, 2.9.6 and 2.10.|
In this publication, we covered the technical details behind two very similar vulnerabilities in Icinga Web 2, an IT monitoring solution. Both vulnerabilities can be combined within an attack in order to fully compromise the Icinga server. During the research of these vulnerabilities, we also discovered a bug in the PHP interpreter itself. We had a nice reminder that unintended quirks may be found in the implementation of a language’s built-in functions which can allow the exploitation of bugs that would be safe otherwise.
We strongly recommend not exposing such systems to Internet as-is: they should only be reachable by trusted source IP addresses (e.g., a VPN endpoint) or put behind a centralized authentication system.
We would like to thank the maintainers of Icinga and PHP for their prompt replies and help in addressing our findings.