Buffer Overflow Attacks: Methods and Best Practices

March 16, 2016  |  Garrett Gross

One of the best ways to improve IT security is for security specialists to understand, at a fundamental level, how different kinds of exploits work. They tend to fall into clusters, based on certain core ideas.

Among the most common forms, for instance, is buffer overflow attacks. The root idea is fairly simple: by inserting more data into a memory buffer than is supposed to be there, an attacker can cause a program to act in unexpected ways that support the attacker’s agenda.

A variety of results can emerge from buffer overflow attacks, ranging from an exception-driven crash to something much more sinister like executing rogue code inserted into computer memory by the attacker. Once this new code is running, it then becomes possible to achieve outcomes like corrupting enterprise data, monitoring and stealing private information, slowing performance of key systems, and many others.

Buffer overflow attacks are far from new to IT security. They’ve been around at least since the 1988 Morris worm, which rapidly spread across the Internet by taking advantage of problematic coding in the UNIX finger daemon. And they all rely on the same, basic premise of problematic coding pertaining to the boundaries of data structures. If the code assumes data can only possibly be of a certain length, an attacker need only exceed that length to cause problems.

You might wonder why programming languages don’t take buffer overflow attacks into account in their basic design, by enforcing rigorous boundary management at compile time. Many do, but they tend to be relatively recent languages such as Java. Older and more established languages, in particular C and its object-oriented cousin C++, as a rule do not — and they also have tremendous momentum in the development world. These languages have for decades been routinely leveraged to create some of the most fundamental elements of modern IT infrastructures, such as the operating systems Linux, UNIX, Windows, and Mac OS X, as well as a wide variety of applications and middleware.

So the unfortunate reality is that buffer overflow attacks, as a class of security threat, are not likely to go away any time soon. It’s just a question of how well security professionals can proactively address the problem or deal with it when it arises in a new form.

Let’s take a look at some of the different types of buffer overflow attacks.

Many kinds of buffer overflow attacks are possible

  • Stack attacks. The buffer here is the stack, a fairly small chunk of memory that programs use to manage call returns (among other things). By overwriting key areas of the stack with too much data, the attacker manages to trick the program to return to (that is, execute) his own code, located elsewhere in RAM, as opposed to the correct code. Stack overflows are the most common, well-known of all buffer overflow attacks.
  • Heap attacks. The heap is a much larger chunk of memory used to store more complex data such as images, or text, that relates to the program. The premise here is similar to the previous, but is trickier for the attacker to implement because the heap isn’t directly used to determine where in memory executable code is located.
  • Arithmetic attacks. These buffer overflow attacks emerge from the way C handles signed vs. unsigned numbers. Specifically, it’s possible to convert a negative (signed with -) number that requires little memory space to a much larger unsigned number that requires much more memory. A crash subsequently occurs and can be leveraged to yield an attack.
  • Format attacks. Text strings, rather like signed numbers, are sometimes converted automatically from a smaller format to a larger (such as by operating systems that require Unicode values). This means attackers can design a buffer overflow attack that exceeds the buffer length if the programmer hasn’t been careful to take into account the larger format.

Coming to grips with the problem of buffer overflow attacks

With that in mind, we can now consider some of the best ways this attack vector can be addressed by security professionals.

Clearly, the best possible opportunity is in the case of code that is actually written in-house, by a development team. Either they, or the quality assurance group, can discover buffer overflow attack possibilities by using automated testing tools or making an aggressive attempt to input unexpected data of inappropriate length during testing.

Also, of course, development teams aren’t limited to C! In fact, safe and automatic memory management, in which buffer boundaries can’t be overrun in these ways, is one of the primary strengths of languages such as Java, Rust (endorsed by Mozilla), and virtually all scripting languages such as Python, Perl, and others.

In the case of code developed externally (such as operating systems and “commercial off-the-shelf” applications), it’s also possible to do endpoint vulnerability scans looking for known buffer overflow attacks (usually associated with older, unpatched versions) that can be compromised with known malware. Similarly, intrusion detection systems can also be updated with rules that look for the behavior of systems that have already been compromised via known exploits. Security administrators can even block outright any attempt to access IP addresses or ranges associated with “bad actors” (eg, criminal organizations located in high-risk locations) from which such attacks often emerge.

The bottom line is that buffer overflow attacks are, and will remain, a primary concern for IT security. Criminal organizations and hackers are increasingly sophisticated in the ways they find and exploit vulnerabilities, and C and C++ continue to be extremely popular as development languages for perceived performance reasons.

So while organizations can try to apply pressure to in-house and external developers (to use better memory management practices and/or languages other than C and C++), the reality is that we can expect ongoing exploits in this area for the foreseeable future

Share this with others


Get price Free trial