Tuesday, January 6, 2015


Multiple LDAP APIs are Asking for Trouble

LDAP filter injection is a classic injection flaw that occurs when user-supplied values find their way into LDAP search filters ("queries") without proper encoding or input validation. The issue has been publicly described since at least 2002 [1] and I still find these flaws on a fairly regular basis. For those unfamiliar with it, these vulnerabilities most often show up in application login forms and can allow an attacker to extract the usernames that exist in the directory and often allow the extraction of attribute values from user objects. In older LDAP servers (or poorly configured ones), it is sometimes possible to extract user password hashes, since these are just an attribute on user objects. LDAP filter injection typically isn't as severe as SQL injection, but it can be serious, depending on how the application uses the filter and what sensitive attributes exist on objects.

Recently, I was describing to a client how to correct a filter injection in their code based on the LDAP API they were using and I was struck by how poorly constructed the API was. In this case, it was the "cfldap" tag provided by Adobe ColdFusion [2][3]. Throughout the documentation, I see no mention anywhere about the dangers of dynamically constructing search filters. In addition, there's apparently no function provided to escape special characters in user-supplied strings for use in filters or DN syntax. Of course any developer could read the fairly straight-forward RFC on the topic [4] and write an encoding function themselves to convert things like "(" to "\28". But as I mentioned in my previous post, one can't really expect developers to be experts in external technologies like LDAP, directory servers, and filter syntax. To add insult to injury, the documentation even provides code examples which are vulnerable to filter injection:

...
<form action="cfldap.cfm" method="POST"> 
    <input type="text" name="name"><br><br> 
    <input type="submit" value="Search"> 
</form> 
<!--- make the LDAP query ---> 
<!-- Note that some search text is required. 
    A search filter of cn=** would cause an error --> 
<cfif (isdefined("form.name") AND (form.name IS NOT ""))> 
    <cfldap 
        server="ldap.airius.com" 
        action="query" 
        name="results" 
        start="ou=People, o=Airius.com" 
        scope="onelevel" 
        filter="(&(cn=*#form.Name#*)(l=Santa Clara))" 
        attributes="cn,sn,ou,mail,telephonenumber" 
        sort="ou,sn" 
        maxrows=100 
        timeout=20000 
    > 
...

There you go, taking a value straight from the form submission and inserting it into a query. How can you expect the typical ColdFusion developer to protect their code from this kind of vulnerability when the API and documentation is effectively setting them up for failure? As one can imagine, ColdFusion isn't alone in this kind of API negligence:
  • PHP's LDAP API does provide an escaping function [5], but it isn't mentioned at all on the ldap_search page, and this page even provides a code example that is almost as vulnerable as ColdFusion's [6].
  • Under Java, the Apache Directory LDAP API does not appear to offer any escaping function and also doesn't mention anything about the risks of dynamically constructed filter expressions [7].
  • For .NET, I have yet to find a method that allows one to escape values prior to including them in search filters (outside of some provided in older C++ APIs). The DirectorySearcher class doesn't seem to contain any mention of the security risks of dynamic filter expressions [8]. Other MSDN pages [9][10] do discuss the escape syntax, but I haven't yet found any security warnings associated with it.
Ok, so if these API vendors aren't doing enough to dissuade developers from making search filter mistakes, what would things look like if they were doing it "right"? Well, we can start by taking inspiration from modern SQL query APIs. The data flow and syntax concerns are very similar, but a lot more attention has been paid to constructing SQL queries safely. The best solutions we have come up with include:
  • Query template APIs, A.K.A. "parameterized prepared statements". Here, programmers are expected to provide a query template along with a series of (potentially untrusted) dynamic values, each mapped to a particular element within the template. Encoding is automatically performed based on data type (or is unnecessary if the database server supports parsing the query templates).
  • Object-Relational Mapping (ORM) or more abstract APIs. Here, an abstract object representation is provided to developers which effectively eliminates the need to construct queries in most situations.
Indeed, these approaches have worked wonders for the safety of relational database access. Over the last several years I have noticed the frequency of SQL injection flaws drop dramatically for newer applications, since most are now leveraging APIs that use one of these approaches. So why don't we do the same for LDAP filter expressions? At very a minimum, each API should have a string escaping function available, along with documentation of the risks of injection. However, I'm not convinced that providing only an escaping mechanism is sufficient, as this approach hasn't been enough to protect SQL queries in the past.

For those keeping score, here's a summary of the limited set of LDAP APIs I've looked at so far and how well they fair in providing safe interfaces and documentation:


API Unsafe
Examples
Security
Risks
Documented
Encoding
Function
Filter
Templates
ORM-like
API
ColdFusion 10 YESNONONONO
PHP 5 YESNOYESNONO
Apache NONONONONO
.NET NONONONONO
python-ldap NOYESYESYESNO
Perl Net::LDAPNOYESNONONO

There's a lot of documentation to cover here, so it's certainly possible I missed something in one or more of these. If I misrepresented anything, post a comment below and I'll happily fix it.

References

  1. LDAP Injection: Are your web applications vulnerable? -- Sacha Faust, SPI Dynamics Inc.
    http://www.networkdls.com/articles/ldapinjection.pdf
  2. ColdFusion Documentation Wiki: cfldap
    https://wikidocs.adobe.com/wiki/display/coldfusionen/cfldap
  3. Adobe ColdFusion 10 - Querying an LDAP directory
    http://help.adobe.com/en_US/ColdFusion/10.0/Developing/WSc3ff6d0ea77859461172e0811cbec0eb56-7fe5.html
  4. RFC 4515: LDAP String Representation of Search Filters
    https://tools.ietf.org/search/rfc4515
  5. PHP: ldap_escape
    http://php.net/manual/en/function.ldap-escape.php
  6. PHP: ldap_search
    http://php.net/manual/en/function.ldap-search.php
  7. Apache Directory API: Searching (...)
    http://directory.apache.org/api/user-guide/2.3-searching.html
  8. MSDN: DirectorySearcher.Filter Property
    http://msdn.microsoft.com/en-us/library/system.directoryservices.directorysearcher.filter(v=vs.110).aspx
  9. MSDN: Search Filter Syntax
    http://msdn.microsoft.com/en-us/library/aa746475%28v=vs.85%29.aspx
  10. MSDN: Creating a Query Filter
    http://msdn.microsoft.com/en-us/library/ms675768%28v=vs.85%29.aspx

Monday, December 22, 2014


Why the Security Community Should Focus More on API Design

Every year, billions of lines of software code are written and deployed into production. While software security experts frantically review code and conduct penetration tests on some of these applications, a vanishingly small percentage ever undergo serious (or regular) security reviews. Unless something major changes in the current regulatory environment, this trend will continue indefinitely, as there are far more developers than security analysts in the market.

It is pretty obvious that simply playing whack-a-mole with individual vulnerabilities is not an efficient way for security experts and developers to spend their time. Perhaps, if developers understood more about security and the kinds of technical flaws that commonly rear their ugly heads, then many issues can be avoided up front. This has been a major goal of organizations like OWASP, which has been trying for over a decade to educate developers on the kinds of software vulnerabilities that are most common in a given situation and how they can best be addressed.

In my experience, developer education does work when it is well executed. In cases where I have been heavily involved in a software development team, working with them to both test their applications and also hold training sessions on the kinds of things they should watch out for, I've found that the quality of code improves a great deal. It's rarely the case that every developer on a given team takes a strong interest in security, or that the content really "clicks" with every developer. But if just one or two core developers on a team of 5-10 takes it seriously, then the problems sort of clean themselves up during the development cycle. Safer frameworks are developed, APIs are changed, and the basic development "norms" improve to where many categories of flaws become rare.

Developer education: Good, but not as effective as we would like

After a decade of working in software security, I'm convinced that the situation isn't getting better, despite many developer education efforts. Each year, we have more and more CVEs to go with more and more software products. Sometimes, entire classes of vulnerabilities become uncommon, but are only replaced by new kinds of issues to go along with the latest technology fad.

So if developer education is effective, why isn't it improving our situation? Are we simply not providing enough education? It may be true that employers are hesitant to provide extensive security education in a job market where this may train their employees for their next job. But for those that don't receive formal training, do developers learn nothing from the rare security review their applications are subjected to? No. I am optimistic, in that I think most developers do care about their code and how they deliver their end product. The majority of serious software flaws are trivial to fix. I would guess that perhaps 80% consist of a 1 to 5 line code change. And once a certain type of issue is understood, developers typically change their libraries to make it easy to avoid the the problem in the future. So what's going on here?  I suspect it is a combination of factors, but let us look at a one area that I don't think has been discussed at length before...

Consider the developer job market for a moment. As a career technologist it is easy for me, personally, to forget that many people don't stick with a highly technical developer position for their whole careers. But of course many developers stay in the field for only a short time, perhaps 5-15 years, after which they move on to management, to other technical roles, or retire. (I don't have any hard numbers on how long developers tend to stay developers. I'd love it if someone pointed me at hard statistics on this.) At the same time, each year a new batch of college graduates enter the workforce and will require training in secure programming. Wait: don't they learn security in school? No. In my experience, software security is simply not a priority of the higher education system (at least in the USA). The only computer science students that seem to come out of school with software security knowledge are those who have an intense interest in it to begin with.  These kinds of students will take all of the security electives and pick up the rest of what they need to know on their own. Unfortunately, these kinds of security-focused students rarely stick to software development for long, instead opting to move into a security role.

So here we are, trying to educate masses of developers on a wide array of technical issues to watch out for, but at the same time we lose previously educated developers each year. So any education we do in the industry can only reach a certain level of saturation. Each year a huge amount of code is still written by developers with no security training. Changing the way universities teach computer science could certainly help in this area, but this isn't a change we'll see overnight.  In addition, even if security training were provided in University, the specific technical pitfalls one needs to avoid always change over time as the underlying technology changes, making past education increasingly irrelevant.

So what else can we do, with limited resources, to reduce the quantity of vulnerabilities contained in new software each year?  I believe a more effective area to focus our efforts is on providing safer software development environments. When the development world made the transition from writing most software in C/C++ to using higher level languages, a major improvement came about in overall software stability and security. Programming languages that handle memory management for the developer eliminate a dozen or more classes of vulnerability (including buffer overflows, integer overflows, string format injections, use after free, etc). Of course, convincing developers to switch languages is not easy, to put it lightly. Fortunately, we don't need to: the development environments used today are much more dynamic than they used to be; being frequently updated by the core vendors and through new third-party libraries. So how can we, as security professionals, guide the ongoing refinements of development platforms to provide developers with safer environments?

In my mind, what really counts, what really makes a difference in the secure development equation, are the APIs used by developers on a day-to-day basis. Whenever a developer opens a file, runs a query, or fetches a URL, they are reliant on the platform's APIs as a significant component of their development environment. If these APIs are poorly constructed, many developers who are not familiar with the particular technology at hand are likely to make mistakes. On the other hand, if these APIs are well thought through, encourage secure data flows, and are well documented then the frequency of vulnerabilities is greatly diminished, particularly when novice developers are at the keyboard.


What makes for a good API?

This blog post is becoming dangerously close to a rambling manifesto... So I'll just briefly outline the key property I believe is needed for APIs that tend to produce secure software. (In future posts I'll give concrete examples to support my assertions.)

Programming interfaces tend to encourage software security when:
The most obvious way to do something happens to be the secure way

When learning a new API, programmers tend to read the minimal amount of documentation necessary before using it. There are many reasons for this, but primarily they are rooted in the fact that "All programmers are optimists."  ~Frederick Brooks, Jr.

After all, from the programmer's standpoint, if an API doesn't behave way he expects after a brief glance at the documentation, then it probably will break his software right away before he even checks it in, or at a minimum, during the Q/A cycle. Most of the time, this works fine and allows the programmer to achieve his goals quickly. The problem with this, of course, is that programmers and Q/A staff are eternally focused on how their software is intended to be used, and not how an attacker may choose to manipulate it. This is why it is so essential to present programmers with APIs that don't contain corner-case "gotchas" that could leave open attack scenarios that won't be caught by a Q/A team's typical test suite.

Often, developers are using third-party APIs to save them time: they don't want to learn the details of how a certain file format, database protocol, or other technology really works under the hood. They just want to achieve their goals quickly by building on what others have already accomplished. That's why being "obvious" is so important. This demands that API designers should:

  • Not assume that programmers are not going to read all of the API documentation
  • Not assume that programmers are particularly knowledgeable in the API's underlying technology
  • Ask programmers to expend a bit more effort whenever potentially dangerous features are exposed

This is not to say that APIs should not provide advanced features, provably prevent programmers from shooting themselves in the foot, or make any other unreasonable guarantees. This is simply about the default mode of operation, making conservative assumptions about API users competence, and trying to put yourself in your users' shoes now and then. These simple things can make a huge difference in the number of applications vulnerable to attack upon first release, which can free up the security community to do more fun a interesting things.