May 25, 1979: American Airlines flight 191 started down the runway at Chicago O'Hare Airport. Just before takeoff, the left engine tore itself completely off of the wing. This severed four critical hydraulic lines as well as disabling several safety systems. 20 seconds after takeoff, the lack of hydraulic pressure caused the left wing control surfaces to stop responding and the plane began to bank steeply to the left. 31 seconds after takeoff, the plane was a fireball on the ground, killing 273 people.  This remains the most deadly air accident in US history and is very well documented. While the airline industry has certainly learned a lot from this tragedy, I believe there are lessons that we, as software developers, can take from it as well.

What Happened?

Before we can draw any wisdom from this tragedy, we must understand the dramatic mechanical failure that caused the engine to free itself from the wing. The McDonnell Douglas DC-10 wing engines are attached to a large arm call the "pylon", which is then attached to the wing, as you can see here:


For various maintenance reasons, mechanics need to detach the engine and pylon from the wing. The procedure for doing this, as provided by McDonnell Douglas, calls for the removal of the engine first, followed by the removal of the pylon. However, this process is very time consuming, especially if you don't have a specific reason to detach the engine from the pylon. That's why several carriers, including American Airlines, independently developed procedures for detaching the pylon from the wing while the engine was still attached. AA's procedure involved using a fork lift to hold the engine and assembly while the pylon/wing bolts were removed and re-installed. McDonnell Douglas did not approve this procedure, and may have cautioned against it, but they could not dictate to any airline what procedures were used.

As it turns out, it is very difficult to manipulate a heavy engine and pylon assembly using a fork lift with the precision required to avoid damaging the aircraft. In the case of AA flight 191 aircraft, the rear pylon attachment point had been pressed up against the wing too hard, which created a fracture in the pylon's rear bracket. Over the next couple of months, this fracture widened with each take off and landing. When it finally failed, the engine's thrust pulled the entire assembly forward, rotating up and over the front edge of the wing. The engine/pylon took a chunk of the wing with it and cut the wing's hydraulic lines in the process. Inspection of other DC-10 planes after the crash revealed that similar damage had resulted from similar short-cut procedures used by both American and Continental Airlines.

... they provided a safer procedure in the manual. But for McDonnell Douglas, this was little comfort when all DC-10's in the US were grounded for 37 days.

Clearly, the majority of responsibility for the flight 191 accident lies with the airline maintenance staff, since they didn't follow the recommended procedure. The aircraft engineers at McDonnell Douglas may very well have anticipated the potential problems with trying to detach the pylon from the wing with the engine still attached, which is why they provided a safer procedure in the manual. But for McDonnell Douglas, this was little comfort when all DC-10's in the US were grounded for 37 days. This caused huge problems for the company in a competitive aircraft market. It was little comfort to the victims and those affected by the crash. Everyone loses in these situations, even those who are "right" about a seemingly arcane technical issue.

Lessons about People and Process

If software security is about People, Process and Technology, as espoused by Schneier, then these kinds of issues seem to fall squarely in the People and Process categories. Especially when technical pitfalls are documented, it is easy for engineers that are knowledgeable in a particular area to develop ivory tower syndrome and take the stance: "I told you not to do it that way, but if you want to shoot yourself in the foot, by all means..." But if our goal is to provide end-to-end safety or security, then this mentality isn't acceptable. As it turns out, there are things engineers can do, besides just documenting risks, to avoid People and Process problems with Technology. This is certainly not always the case: some problems simply cannot be addressed with Technology alone. But many can be mitigated if those problems can be anticipated to begin with.

Typically in software, the downsides of failure are not nearly as serious. However, the kind of displaced fallout that McDonnell Douglas experienced also shows up in software security. One example would be with open source blog software packages, such as WordPress. In a number of discussions I've had with clients and security folk, the topic of WordPress security has come up. Everything I hear indicates that WordPress has a pretty poor reputation in this area. In one way, this seems little odd to me, since I have briefly looked at the core WordPress code base a few times and they do a lot of things right. Sure, WordPress has its share of security issues, don't get me wrong, but the core software isn't that terrible. However, if you do a CVE search for WordPress, the number of vulnerabilities associated with WordPress plugins is quite depressing. To me, it is apparent that bad plugin security has hurt WordPress' reputation around security in general, despite the majority of vulnerabilities lying somewhat out of the core developers' control.

Two primary ways that engineers can help guide their technical customers (whether they be other programmers or maintenance crews) down a safe path: discourage dangerous usage and make safe usage much easier than the alternatives.

Discouraging Dangerous Usage

Let us return to the issue of mechanics trying to remove the engine and pylon assembly all in one piece. If the McDonnell Douglas engineers anticipated that this would be unsafe, then they could have made small changes to the engine/pylon assembly such that when the engine is attached, some of the mounting bolts between the pylon and wing were covered up. In this way, it becomes technically infeasible (short of getting out a hack saw) to carry on with the procedure that the airlines devised.

In the case of WordPress, if the core developers realized that many plugin authors keep making mistakes using, say, an unsafe PHP function (there are soooo many to choose from...), then perhaps they could find a way to deploy a default PHP configuration that disables the unsafe functions by default (using the disable_functions option or equivalent). Sure, developers could override this, but it would give many developers pause as to why they have to take that extra step (and then perhaps more of them would actually RTFM).

Making Safe Usage Easier

Of course, disabling features or otherwise making life difficult for your customers is not the best way to make yourself popular. A better way to encourage safety by developers (or mechanics) would be to devise faster/better solutions to their problems that are also safe. In the case of the airline mechanics, once McDonnell Douglas realized that three airlines were using a short-cut procedure, then they could have evaluated the risks of this and devised another procedure that was both fast and safe. For instance, if they had tested United's method of using a hoist (rather than a fork lift), they may have realized that a hoist is perfectly fine and encouraged the other two airlines to use that method instead. Or perhaps they could have provided a special protective guide, harness, or special jacks that would allow for fine control over the engine/pylon assembly when manipulating it.

In the case of WordPress, instead of just disabling dangerous interfaces in PHP, they could also provide alternative interfaces that are much less likely to be misused. For example: database access APIs that don't require developers to write SQL statements by hand, or file access primitives that make directory traversal impossible within a certain sub-tree. Of course it depends on the kinds of mistakes that developers keep making, but by adding APIs that are both safe by default and that save developers time, more and more of the developer population will gravitate toward safe usage.

Conclusion

Once again, it is easy to pass the buck on these kinds of problems and assume, as an API designer, that your users' poor choices are out of your control. It is also easy to assume that your users are just as technically savvy as yourself and won't make mistakes that seem obvious to you. But these are both bad assumptions and should be constantly questioned when it comes to ensuring the security of the overall system.

LDAP filter injection is a classic injection flaw that occurs when user-supplied values find their way into LDAP search filters ("queries") without proper encoding or input validation. The issue has been publicly described since at least 2002 [1] and I still find these flaws on a fairly regular basis. For those unfamiliar with it, these vulnerabilities most often show up in application login forms and can allow an attacker to extract the usernames that exist in the directory and often allow the extraction of attribute values from user objects. In older LDAP servers (or poorly configured ones), it is sometimes possible to extract user password hashes, since these are just an attribute on user objects. LDAP filter injection typically isn't as severe as SQL injection, but it can be serious, depending on how the application uses the filter and what sensitive attributes exist on objects.

Recently, I was describing to a client how to correct a filter injection in their code based on the LDAP API they were using and I was struck by how poorly constructed the API was. In this case, it was the "cfldap" tag provided by Adobe ColdFusion [2][3]. Throughout the documentation, I see no mention anywhere about the dangers of dynamically constructing search filters. In addition, there's apparently no function provided to escape special characters in user-supplied strings for use in filters or DN syntax. Of course any developer could read the fairly straight-forward RFC on the topic [4] and write an encoding function themselves to convert things like "(" to "\28". But as I mentioned in my previous post, one can't really expect developers to be experts in external technologies like LDAP, directory servers, and filter syntax. To add insult to injury, the documentation even provides code examples which are vulnerable to filter injection:

...
<form action="cfldap.cfm" method="POST"> 
    <input type="text" name="name"><br><br> 
    <input type="submit" value="Search"> 
</form> 
<!--- make the LDAP query ---> 
<!-- Note that some search text is required. 
    A search filter of cn=** would cause an error --> 
<cfif (isdefined("form.name") AND (form.name IS NOT ""))> 
    <cfldap 
        server="ldap.airius.com" 
        action="query" 
        name="results" 
        start="ou=People, o=Airius.com" 
        scope="onelevel" 
        filter="(&(cn=*#form.Name#*)(l=Santa Clara))" 
        attributes="cn,sn,ou,mail,telephonenumber" 
        sort="ou,sn" 
        maxrows=100 
        timeout=20000 
    > 
...

There you go, taking a value straight from the form submission and inserting it into a query. How can you expect the typical ColdFusion developer to protect their code from this kind of vulnerability when the API and documentation is effectively setting them up for failure? As one can imagine, ColdFusion isn't alone in this kind of API negligence:

• PHP's LDAP API does provide an escaping function [5], but it isn't mentioned at all on the ldap_search page, and this page even provides a code example that is almost as vulnerable as ColdFusion's [6].
• Under Java, the Apache Directory LDAP API does not appear to offer any escaping function and also doesn't mention anything about the risks of dynamically constructed filter expressions [7].
• For .NET, I have yet to find a method that allows one to escape values prior to including them in search filters (outside of some provided in older C++ APIs). The DirectorySearcher class doesn't seem to contain any mention of the security risks of dynamic filter expressions [8]. Other MSDN pages [9][10] do discuss the escape syntax, but I haven't yet found any security warnings associated with it.

Ok, so if these API vendors aren't doing enough to dissuade developers from making search filter mistakes, what would things look like if they were doing it "right"? Well, we can start by taking inspiration from modern SQL query APIs. The data flow and syntax concerns are very similar, but a lot more attention has been paid to constructing SQL queries safely. The best solutions we have come up with include:

• Query template APIs, A.K.A. "parameterized prepared statements". Here, programmers are expected to provide a query template along with a series of (potentially untrusted) dynamic values, each mapped to a particular element within the template. Encoding is automatically performed based on data type (or is unnecessary if the database server supports parsing the query templates).
• Object-Relational Mapping (ORM) or more abstract APIs. Here, an abstract object representation is provided to developers which effectively eliminates the need to construct queries in most situations.

Indeed, these approaches have worked wonders for the safety of relational database access. Over the last several years I have noticed the frequency of SQL injection flaws drop dramatically for newer applications, since most are now leveraging APIs that use one of these approaches. So why don't we do the same for LDAP filter expressions? At very a minimum, each API should have a string escaping function available, along with documentation of the risks of injection. However, I'm not convinced that providing only an escaping mechanism is sufficient, as this approach hasn't been enough to protect SQL queries in the past.

For those keeping score, here's a summary of the limited set of LDAP APIs I've looked at so far and how well they fair in providing safe interfaces and documentation:

API	Unsafe Examples	Security Risks Documented	Encoding Function	Filter Templates	ORM-like API
ColdFusion 10	YES	NO	NO	NO	NO
PHP 5	YES	NO	YES	NO	NO
Apache	NO	NO	NO	NO	NO
.NET	NO	NO	NO	NO	NO
python-ldap	NO	YES	YES	YES	NO
Perl Net::LDAP	NO	YES	NO	NO	NO

There's a lot of documentation to cover here, so it's certainly possible I missed something in one or more of these. If I misrepresented anything, post a comment below and I'll happily fix it.

References

1. LDAP Injection: Are your web applications vulnerable? -- Sacha Faust, SPI Dynamics Inc.
   http://www.networkdls.com/articles/ldapinjection.pdf
2. ColdFusion Documentation Wiki: cfldap
   https://wikidocs.adobe.com/wiki/display/coldfusionen/cfldap
3. Adobe ColdFusion 10 - Querying an LDAP directory
   http://help.adobe.com/en_US/ColdFusion/10.0/Developing/WSc3ff6d0ea77859461172e0811cbec0eb56-7fe5.html
4. RFC 4515: LDAP String Representation of Search Filters
   https://tools.ietf.org/search/rfc4515
5. PHP: ldap_escape
   http://php.net/manual/en/function.ldap-escape.php
6. PHP: ldap_search
   http://php.net/manual/en/function.ldap-search.php
7. Apache Directory API: Searching (...)
   http://directory.apache.org/api/user-guide/2.3-searching.html
8. MSDN: DirectorySearcher.Filter Property
   http://msdn.microsoft.com/en-us/library/system.directoryservices.directorysearcher.filter(v=vs.110).aspx
9. MSDN: Search Filter Syntax
   http://msdn.microsoft.com/en-us/library/aa746475%28v=vs.85%29.aspx
10. MSDN: Creating a Query Filter
    http://msdn.microsoft.com/en-us/library/ms675768%28v=vs.85%29.aspx