Securing Your IT Infrastructure: The Critical Role of Server Patching

In the ever-evolving landscape of information technology, the security of your servers is paramount. One key aspect of maintaining a secure IT environment is regular server patching. In this blog post, we will explore the importance of patching servers and delve into a specific type of vulnerability, ‘buffer overflow,’ and the current warnings for GLIBC_TUNABLES from The Cybersecurity and Infrastructure Security Agency (CISA). 

Server patching involves applying the latest software updates, security patches, and bug fixes to the operating system, applications, and other software components on your servers. This ongoing process is crucial for several reasons, including: 

Security Enhancements

Patches often address known vulnerabilities in software. Failing to apply patches promptly can leave your servers exposed to exploitation by cybercriminals seeking to take advantage of these vulnerabilities. 

Performance Optimization

Updates may include performance improvements and optimizations that ensure your servers run more efficiently. This can lead to better overall system performance and responsiveness. 

Compatibility with New Technologies

As technology advances, software developers release updates to ensure compatibility with the latest technologies. Regular patching ensures your servers can seamlessly integrate with new hardware and software. 

Compliance Requirements

Many industries and regulatory bodies mandate that organizations adhere to specific security standards. Regular server patching is often a requirement for meeting these compliance standards.

Change Control

In a live environment where servers are actively supporting critical business operations, any disruption or downtime can have significant consequences. So, while the concept of patching is simple, patching production IT servers is a challenging and intricate process that necessitates careful consideration and adherence to ‘change control’ protocols. The complexity arises when balancing the imperative of applying security patches promptly with the potential risks associated with introducing changes to a stable system. Change control, a systematic process of managing changes to an IT environment, is crucial in this context, as it provides a structured approach to managing alterations to the production environment. Change control involves meticulous planning, testing, and validation of patches to ensure they do not introduce unforeseen issues or conflicts with existing configurations. Yet, they simultaneously manage to fix discovered vulnerabilities the patch might be intended to mitigate. 

Buffer Overflow Vulnerability

Vulnerabilities come in all shapes and sizes. One common exploitable vulnerability is ‘buffer overflow.’ A buffer overflow exploit is a sophisticated cyberattack that takes advantage of a programming flaw in software to compromise a system’s security. The vulnerability lies in the improper handling of data input by a program, typically due to inadequate bounds checking. In a buffer overflow attack, an attacker sends more data than a program’s buffer (a temporary storage area) can accommodate, leading to an overflow. The excess data spills into adjacent memory regions, corrupting or overwriting critical data structures, including the program’s execution stack. By carefully crafting the overflowed data payload, the attacker can manipulate the program’s behavior and inject malicious code into the system’s memory. When the compromised program continues its execution, it unwittingly runs the injected code, granting the attacker unauthorized access, control, or the ability to execute arbitrary commands on the targeted system. Buffer overflow exploits can be particularly dangerous as they often allow attackers to bypass security mechanisms, escalate privileges, and compromise the integrity and confidentiality of the system. 

One such buffer overflow is the subject of a recent CISA alert. In this alert CISA calls for immediate patching of Linux systems to mitigate a vulnerability in the GLIBC_TUNABLES Environment Variable. This variable in the GNUC Library allows system administrators to dynamically adjust various parameters. GNU-C Library, also known as GNU Lib C, or GLIBC, is the GNU ()Project’s implementation of the C library, a crucial component of the GNU operating system and many Linux distributions, providing essential functions for programs written in the C programming language. 

In what might only be labeled as a moment of comedic relief, the vulnerability has been dubbed “Looney Tunables.” First discovered in October, this vulnerability has been actively exploited by threat actor group Kinsing, a.k.a. Money Libra, to deploy malware in containerized environments for cryptocurrency mining.  

The Importance of Patching

The importance of patching servers in an IT environment cannot be overstated. Regular server patching is a fundamental aspect of maintaining a secure infrastructure, protecting against known vulnerabilities, and ensuring compliance with industry standards. Understanding specific vulnerabilities, like buffer overflow issues in Linux, and utilizing features such as GLIBC_TUNABLES, empowers administrators to proactively enhance the security posture of their systems. By staying vigilant and proactive in server maintenance, organizations can mitigate potential risks and create a more robust and resilient IT environment. If you’d like assistance in developing a mature patching program as part of a comprehensive cybersecurity program, please contact Ntirety.

HTTP and the Rapid Reset Vulnerability

The History of HTTP (HTTP/0.9)

The bedrock for data communication on the World Wide Web, a term that’s still valid yet rarely used today, is HTTP (Hypertext Transfer Protocol). HTTP is an Application Layer (OSI layer 7) protocol in the internet protocol suite. First released in 1991, the original version, HTTP, is now referred to as HTTP/0.9.  

Although crude by today’s standards, HTTP was effective enough to launch what would become a revolution in human communication. HTTP/0.9 was written as a plain document under 700 words and had one simple job, or in World Wide Web vernacular, one method: GET. Simply stated, the protocol could only request an HTML document:

GET /htmlpage.html 

General responses were extremely simple as well:

<html> 

 An old and simple html page. 

</html> 

It’s interesting to note that in these GET parameters, there is no indication of a server IP address or protocol port. In 1991, this information was simply not needed once connected to the server; there were no headers to convey session metadata and there were no status codes or error codes.  

HTTP Today: The HTTP/2 Flaw 

Fast forward to today, four iterations later, where the standard is now HTTP/2. HTTP/2 was released in 2015 and offers major advancements, making it more efficient and responsive to the demands of today’s web. While it’s a notable improvement over previous versions, as with many things, along with the good comes some bad. HTTP/2 has a flaw which provides malicious actors a simple method to create a Distributed Denial-of-Service (DDoS) attack against any web server, in an attempt to render network resources unavailable and disrupt operations.  

DDoS attacks against web servers are certainly not new, however a recent DDoS attack method, known as “HTTP/2 Rapid Reset,” leverages a flaw in the implementation of the protocol. For official information on this flaw see CISA’s alert on CVE-2023-44487. 

The flaw is nestled in the added ability for HTTP/2 to multiplex, which is a major advancement over HTTP/1.1. With multiplexing, HTTP/2 can initial multiple requests in parallel over a single TCP connection. The result means webpages containing several elements are now delivered over one TCP connection. It’s important to note that this flaw, by itself, does not lead to server or data compromise. It could, however, be used to divert attention while threat actors attack other areas in the target networks.  

In Figure 1 below, the HTTP/1.1 attack shows the serial nature of the previous protocol version and the limitation it causes in being able to rapidly establish Requests and Responses. The HTTP/2 attack shows how the multiplexing feature of HTTP/2 allows for a parallel attack, consuming more resources on the web server as it works to produce Responses more quickly. Finally, the HTTP/2 Rapid Reset attack demonstrates how the pervious parallel attack can be amplified by continuously sending the web server a Request followed by a Reset, which allows for an infinite number of Requests to be in flight. Here the webserver will have to do significant amounts of work creating and canceling requests, such as allocating new stream data structures, parsing the query and doing header decompression, and mapping the URL to a resource, eventually consuming available server or network resources, whichever occurs first.   

Rapid reset vulnerability.

Figure 1 – Image Source: thehackernews.com 

With some simple Python code utilizing the hyper library, a threat actor can use multiple threads and repeatedly sends a series of HTTP/2 requests with headers and RST_STREAM frames to generate a high volume of traffic. How simple might this be? We’ll first create a ‘send_requests’ function that might look something like this:  

def send_requests(): 

    conn = hyper.http20.h2.H2Connection() 

    conn.initiate_connection() 

    conn.connect(target_host, target_port) 

 

    for _ in range(num_requests): 

        headers = [ 

            (‘:method’, ‘GET’), 

            (‘:scheme’, ‘https’), 

            (‘:authority’, target_host), 

            (‘:path’, ‘/path/to/ resource’), 

        ] 

 

        conn.send_headers(1, headers) 

        conn.send_rst_stream(1, hyper.http20.errors.ErrorCodes.CANCEL) 

From here, we only need to initiate the request in multiple threads to amplify the effect: 

threads = [] 

for _ in range(10):   

    t = threading.Thread(target=send_h2_requests) 

    threads.append(t) 

So, with under 20 lines of code, it’s possible to create something that can be used as a base to create a DDoS attack against a web server. *Please note that the code in the example above is not complete and cannot, and certainly should not, be used or referenced to perform any type of DDoS attack. 

Mitigating Risks

While this flaw is known, vendors have been working to mitigate the risk. Organizations should also, as a rule, take proactive steps to mitigate risk and reduce the effects of DDoS attacks in general. This means organizations should have controls in place, including: 

  • Consistent application of patches to web servers/proxies 
  • Restricted internet access to web applications where possible 
  • Use of a Web Application Firewall (WAF) with rate limiting rules and geographic restrictions 

Organizations can also consider migrating to a hardened Platform as a Service (PaaS) provider like Microsoft Azure App Service, a managed service with built-in infrastructure maintenance, security patching, and scaling.  

Resource availability is critical in IT Security. Applications and data that are not readily available to authorized users can have a cascading effect on organizations, resulting in loss of productivity, loss of revenue, and loss of reputation. A DDoS attack like the threat described above can impact this availability. Organizations should – must – have a comprehensive security posture which allows them to protect their assets while actively monitoring and responding to alerts and incidents. If you want to create a comprehensive security program or bolster an existing program, the experts at Ntirety can help. Get started today by visiting us at ntirety.com.