31

My SaaS company recently lost the bid for an enterprise software licensing deal.

One of the reasons the prospect gave for not choosing us as a vendor was:

the use of a WAF

I'm not an information security specialist, so I'm confused as to why the use of a WAF (Web Application Firewall) could be seen as a potential security vulnerability.

Is it best to avoid using a WAF? Could they be concerned because the use of a WAF suggests inadequate protection? Is their concern legitimate?

The full quote[^1] is as follows.

Technical Integration – $PROSPECT is a multi-national subsidiary of an industry leader in $INDUSTRY, which brings with it a lot of cyber security risks. This means our technical security standards are high, so $TECH_DUE_DILIGENCE_PROVIDER raised concerns about the implications of $REASON_1, the use of a WAF, and $REASON_2. Although in isolation these issues may seem minor, the cumulative risk and potential exposure was difficult to overlook – which ultimately led to $VENDOR being classed as a ‘high risk vendor' by IT.

[1^]: irrelevant or sensitive information redacted; emphasis mine.

End Anti-Semitic Hate
  • 3,144
  • 2
  • 27
  • 55
Anon
  • 311
  • 2
  • 5
  • 1
    But from that note, I'd ask for clarification. Doesn't make sense to me. "use of a WAF" is not an explanation. – schroeder Nov 27 '23 at 10:27
  • 3
    I'm afraid that the key is Although in isolation these issues may seem minor, the cumulative* risk and potential exposure was difficult to overlook*. By itself having a WAF as an additional defence line is not bad. But pretexting a WAF for lowering other defence lines is. – Serge Ballesta Nov 27 '23 at 10:50
  • @SergeBallesta I understand this can't be known from the other reasons given as they're redacted, but I don't believe the other two reasons are relevant. For example, reason #2 is actually the programming language our application is implemented in. Their complaint is that the language is too old, despite it first appearing in 1990. Their own technical stack [I believe] predominantly uses languages which first appeared in 1991 (Python), 1995 (JavaScript), and 1972 (SQL). Reason #1 is unrelated to network security. – Anon Nov 27 '23 at 11:06
  • 6
    @Anon, you only give partial informations. Most languages do have versions, some of them are no longer maintained. Using a no longer maintained thing (be it a language, framework or library) is seen as poor security practice. And using an old version even if maintained comes with an additional risk of using other old libraries or modules, some of them could in turn no longer be maintained. But for example PHP based applications are often seen as less secure than Java based ones, just because many third party PHP modules authors did not pay enough attention to security... – Serge Ballesta Nov 27 '23 at 11:23
  • @SergeBallesta You're absolutely right. In our case, the language we primarily use is absolutely still actively developed and maintained, which is why I don't think their feedback on that topic is legitimate. – Anon Nov 27 '23 at 11:25
  • 5
    ... Finally they are the client. That means that if they are consistent in their security referential, there is little you can do, past politely asking for explainations, telling them that you would like to later be able to provide them a response that they could accept. – Serge Ballesta Nov 27 '23 at 11:26
  • 3
    Web application firewalls tend to corrupt or block legitimate traffic. – Jasen Nov 28 '23 at 02:46
  • 1
    In most RFP/RFQs I have worked on, the presence of a WAF is more likely to be seen as a positive point or even a requirement than a negative point... But without more context it's difficult to judge. – jcaron Nov 28 '23 at 13:49
  • 1
    Question: is the WAF in question a solution from another SaaS, or some software you run on your own premises? – ᴍᴇʜᴏᴠ Nov 29 '23 at 13:10
  • @ᴍᴇʜᴏᴠ In our case, it's a [very] small application that we maintain, audit, and build from source. It runs adjacent to our web application. – Anon Nov 29 '23 at 14:02
  • Just reading interesting questions from the SE sidebar; for this one I had to look up what a WAF is, and the first paragraph on Wikipedia is already saying they're not recommended. (That's not the authority on security, but it's a sign that this company's reaction is not all that unexpected) – Raphael Schmitz Nov 29 '23 at 14:11
  • Related: Is a web application firewall necessary if the application is secure? – Bergi Nov 30 '23 at 13:14
  • 1
    Maybe they meant "use of an in-house WAF", I could see being a minus – Oliver Dec 07 '23 at 21:52

8 Answers8

35

A WAF is fundamentally a MITM attack against your own web services. It intercepts, and, in the event that it has a vulnerability, backdoor, poor handling of data transmission or storage, etc., has the capability to record and exfiltrate highly sensitive data including session tokens, credentials, private information, etc. It is an additional layer that, in order for your system as a whole to be secure, must also be secure against all the same sorts of threats. It is additional attack surface.

These are not merely theoretical considerations. Again and again, there have been compromises involving sloppy WAFs. And this makes sense: given that there is a correlation between viewing WAF as a "security solution" and having very naive ideas about security architecture and attack surface, a WAF is at least as likely to have vulnerabilities as a web application whose implementors only have mediocre security experience, if not more likely.

R.. GitHub STOP HELPING ICE
  • 8,440
  • 1
  • 25
  • 36
  • 13
    The same could be said for the IP-level firewall in front of the website, the load balancer, the router, the switch, the TLS off-loading module, etc. Are you saying that a WAF is a special risk? – schroeder Nov 27 '23 at 20:05
  • 10
    "a WAF is at least as likely to have vulnerabilities as a web application" -- citation needed – schroeder Nov 27 '23 at 20:06
  • 33
    "The same could be said for the IP-level firewall in front of the website, the load balancer, the router, the switch, the TLS off-loading module"--These don't decrypt traffic, they just pass it a long. In order for a WAF to work, it needs decrypted access to the application traffic. In other words, something with a WAF wouldn't be end-to-end encrypted. – nijave Nov 27 '23 at 20:16
  • 3
    @nijave With the exception of the TLS offloading module, whose single purpose is to decrypt traffic. – Robby Cornelissen Nov 28 '23 at 06:11
  • 9
    While I agree that the use of a WAF highly correlates with a poor secure SDLC, I'm hesitant to call a WAF a MITM attack. Assuming it's deployed on-premise (so it's not on the edge, like a CDN WAF, or a total detour to third-party servers). I'm hesitating because, by that definition, even your MVC framework would be a MITM. Or any library you use to process your data. Of course, you must assess the WAF's company reputation first but many WAFs are just augmented reverse proxies deployed autonomously. – Margaret Bloom Nov 28 '23 at 09:56
  • 5
    A WAF is a red flag because they all have a very high mismatch between their capability and the application's actual vulnerabilities. A WAF doesn't know how your application works and will only protect you from trivial (by 2023 standards) attacks. On top of that, this often means the WAF checks can be worked around more or less easily unless you are willing to hinder the experience of legitimate users. So, yes, WAF are for incompetents but I don't think they count as MITM. – Margaret Bloom Nov 28 '23 at 09:57
  • 1
    @MargaretBloom I'm a bit confused by the apparent value judgement in "WAF are for incompetents". Aside from that, I'd like to understand why the use of a WAF is inherently less secure than the same web application not sitting behind one. My prior understanding generally matched your characterisation of a WAF as essentially an augmented reverse proxy. – Anon Nov 28 '23 at 23:05
  • Would you still argue this point with the same strength if the WAF were implemented from scratch, essentially as another web application sitting in front of the primary web application? To Margaret Bloom's point about a WAF not knowing how your web application works — does that assessment change if the WAF is tailor-made to the web application? – Anon Nov 28 '23 at 23:22
  • 3
    @Anon "Why is the use of a WAF is inherently less secure than the same web application not sitting behind one?" - see the answer itself: it's a much larger attack surface. The WAF itself might have vulnerabilities - not just bugs of the kind that let it pass values that should have been blocked, but of the kind that let you take over the system on which the WAF is running. And taking over the WAF system is enough, as all sensitive data passes through it. – Bergi Nov 29 '23 at 00:34
  • 8
    @Anon "does that assessment change if the WAF is tailor-made to the web application?" - the assessment about mismatch and protection, yes. The assessment on the attack surface, no. However, I'd still consider it a sign of incompetence to build and run two web applications where one would have sufficed… it looks like security was only an afterthought, why not just build the tailored security into the primary application? – Bergi Nov 29 '23 at 00:38
  • @Bergi Thank you for clarifying. I don't disagree with you, but as a thought experiment, would you extend your reasoning to conclude that a [typical] microservices architecture is a disaster for security? – Anon Nov 29 '23 at 13:51
  • @Anon Not sure what you consider "typical", but afaict the best practice is a zero trust model where each microservice manages its own security. It's better than the monolith in terms of compartmentalisation (smaller impact if something is broken), but on the other hand there is indeed a larger attack surface (in particular if multiple services are publicly reachable) and a more heterogenous technology stack is harder to manage/overview/secure. At least it'll be easier to deploy updates… – Bergi Nov 29 '23 at 14:03
  • @MargaretBloom what kind of WAF are you working with? What kind of WAF are you all working with? The claims and assumptions and generalisations flying around in the comments are thick and heavy. No one is supporting any of their claims. – schroeder Nov 30 '23 at 08:01
  • @nijave "something with a WAF wouldn't be end-to-end encrypted" -- you have a fundamental misunderstanding of what E2E means and how it applies to a web app. A WAF would be one "end"... – schroeder Nov 30 '23 at 08:24
  • 1
    @Anon if you know the vulnerabilities in your application and can develop a WAF that filters exploits that use them, why would you not rather fix the vulnerabilities in your application? – Josef Nov 30 '23 at 09:46
  • 2
    @schroeder If the WAF has an underlying application that it proxies requests to, then I'd be hard pressed to call that an "end", by virtue of having a next step in the chain. I suppose it could be argued for host-based WAFs, but I'd find it hard to argue that WAFs in general aren't simply a step in the communication stack (and not the final one). – JHR Nov 30 '23 at 12:55
  • @JHR and how do you define the "end" in E2E when it's a web app? Web is inherently a long chain. – schroeder Nov 30 '23 at 13:58
  • 1
    @schroeder I have to admit that I do not work with any kind of WAF, so I cannot speak from operational experience. They have too many flaws imo, see https://www.macchaffee.com/blog/2023/wafs/ https://www.traceable.ai/blog-post/why-web-application-firewalls-arent-protecting-your-cloud-native-applications https://sphericaldefence.com/why-wafs-dont-work/ https://sourcedefense.com/glossary/limitations-of-waf/. I'm not saying a modern WAF cannot be an effective layer in a layered defence, but in the few projects I was involved the consideration was to rather spend resources on security elsewhere. – Bergi Nov 30 '23 at 14:00
  • @Bergi yep, they can be tricky, no doubt, and there are lots of different kinds, but the over-generalization from all the above is concerning. There are lots of apps where a WAF is a good and useful defense layer. And since ModSecurity is classed as a WAF, the reactions above are over-the-top. – schroeder Nov 30 '23 at 14:08
32

One possible explanation would be that they think you rely on a WAF for your security, rather than using it as an additional security layer. I've met people who have very insecure legacy sites (full of XSS, SQLi, etc), but they think that it's OK because "they have a WAF", and they think that's sufficient to protect them.

So if they asked a question like "How to you ensure that your website is secure?" and your answer was just "We have a WAF" then that would raise concerns. But if you gave a full explanation about what you do (secure development, patching, etc), and added to the end that you have a WAF as an additional layer, that shouldn't be an issue.

If you're using a cloud-based WAF, there could also potentially be concerned there about that, and who the third party is that you're exposing all their data to? They may not be happy with everything being sent to Cloudflare/Akamai/whoever.

But it does seem a rather odd response - and certainly something I'd go back to them and ask for clarification on.

Gh0stFish
  • 10,932
  • 2
  • 35
  • 36
  • 22
    Even if you're not relying on it as security, a WAF is a huge additional risk. It's a MITM that has access to all traffic, all access tokens, credentials, etc. and that's implemented by folks with very dubious ideas about security architecture and attack surface. – R.. GitHub STOP HELPING ICE Nov 27 '23 at 19:14
9

One thing I can think of, and I don't know if this applies to you, is if a WAF has been used instead of fixing application vulnerabilities. Sometimes companies do this, especially when it's difficult to make changes to the application. But it is a weak approach compared to fixing the vulnerabilities, as typically WAF controls can be bypassed by a determined attacker.

paj28
  • 33,442
  • 8
  • 96
  • 133
4

You mentioned your product is implemented in an older programming language dating to 1990. This suggests Python, Haskell or something more obscure (and if so likely worse).

The customer's security report has identified use of a Web Application Firewall in front of the application. One likely assumption many would make is that your application has a WAF because it is insecure without.

What does a WAF do? According to Wikipedia they monitor content and attempt to detect & block:

  • SQL injection,
  • cross-site scripting (XSS)
  • file inclusion
  • improper system configuration.

Wikipedia recommends against them saying "they also introduce a performance degradation and are easily bypassed by attackers."

You also answered, again in comments, that the WAF in question "In our case, it's a [very] small application that we maintain, audit, and build from source. It runs adjacent to our web application."

Now, if I was doing security diligence in a sector vulnerable to attacks:

  1. Old/ insecure languages are going to put me right off. Haskell is a good language, I don't want a Python appserver, anything else is likely a total load of cobblers.

  2. Having the WAF, and the generally understood purposes of a WAF, can easily feed into a perception that your underlying app & security may be very weak; and that you may be highly reliant on the WAF for security.

  3. Many of the kind of things that Wikipedia suggests a WAF protects against -- such as SQL injection -- I would consider absolutely unacceptable to be in the core application. I would not accept these anti-patterns present anywhere whether an attempt had been made to mitigate them at other levels or not.

  4. Perhaps the customer was willing to overlook security weakness of the core, if the WAF was solid. But what WAF are you using? Is independently known and reputable, something the customer can really trust in? Apparently not -- it's a small application you maintain and build yourself.

I'm presenting solely the critical view here so this will be negative, but ask yourself:

Would you, as a customer, trust a company with outdated technology, poor security and maybe using bad practices (SQL injections?) to maintain & build a security layer that needs to be 1000% secure?

Or would you instead prefer to choose a vendor with current technology (likely a memory-safe compiled language such as Java/Kotlin, C#, Rust, Go), good security and robust practices?

I am sure there is much we don't know, but perhaps this feedback could inform your company in regard of selecting & building a more secure technology architecture.

Thomas W
  • 203
  • 1
  • 4
  • 2
    I appreciate you're trying to empathise with the client, and not arguing whether or not any given language is "old". For me though, it raises the question: how old is any given programming language? And does it matter? Is Haskell as old as its first implementation? Or is it as old as its latest major compiler version? Or latest stable release? Essentially all RDBMS queries are done in SQL which first appeared in 1972, but I don't see anyone arguing that the use of SQL queries is a red flag. PHP arrived years after Haskell, but it certainly isn't safer, etc., etc. – Anon Dec 01 '23 at 12:04
  • @Anon In regards of security, concerns are probably memory safety and reliability (that code reliably does what was intended). Garbage collected or memory-safe languages help with the first, and became much more prevalent from 2000 onward. Compiled, type-checked, declarative and to some extent functional languages help with the second. – Thomas W Dec 04 '23 at 22:49
  • The general state of language technology in 1990 was imperative, non memory-safe compiled languages or weakly-typed interpreters (Perl, Python). Haskell as an academic language was well ahead as far as language design goes. From my perspective broad commercial acceptance of memory-safe languages came with Java 1.3 in 2000, which combined GC, threading, and network functionality in a portable language. I see a robust language foundation (memory-safety & compilation) as ideal for security. For me that is Haskell, Java, C#, Rust, Go. – Thomas W Dec 04 '23 at 22:54
3

The reviewer deemed the WAF a negative, when combined with the other 2 reasons which are not provided here. Therefore we can't really answer the question, because seriously, in general (ie without other reasons) a WAF is better than no WAF. Some situations where a WAF would be deemed negative, compared to no WAF in the same situation :

  1. if end-to-end encryption is required and the WAF is not sufficiently configurable (eg it cannot do anything if e2ee is needed, in which case a WAF is just another layer of maintenance, cost, and network lag)
  2. if the WAF is in-house
  3. if the WAF does not meet (or cannot be shown to meet) one or more essential standards
  4. if there is requirement for privacy (the WAF has to introspect)

There are probably more reasons. Really wish we could know what those 2 other reasons are.

Update

Here's an interesting article that I just came across, referenced in Hashicorp's newsletter, on that exact topic: https://www.macchaffee.com/blog/2023/wafs

Oliver
  • 131
  • 4
  • What standards for WAFs are there? – Bergi Dec 07 '23 at 22:19
  • 1
    SANS, NIST, NSA, ISO/IEC, OWASP, they may have recommendations / guides that the reviewer has in high regard, even if they are not standards in the official sense that's the idea – Oliver Dec 08 '23 at 22:48
  • 1
    I have to agree with this as well. WAF, by itself, is no reason to reject a solution. A real WAF only enhances the security of access to an application. There are many more companies that would reject a vendor if there was NO WAF in place. WAF plus $REASON_1 and $REASON_2 must be the real reason. – JD Allen Dec 08 '23 at 23:45
  • @Oliver "they may have" - do you have any concrete links? – Bergi Dec 09 '23 at 01:47
  • @JDAllen "A real WAF" sounds like a true scotsman – Bergi Dec 09 '23 at 01:48
1

What could be a consideration is that, even with a secure connection between Client->WAF, and then WAF->Your server, the WAF by definition has access to the raw request/response. In a case where data security (for example healthcare, finance) requires full encryption in transit, this is a big loophole, and essentially potentially provides a third party with access to the data.

Edit: My above discussed cloud WAF, and as such the concern that a third party has potential access to the unencrypted data.

YaronGo
  • 11
  • 2
0

As @Jasen mentioned in one of its comments, Web application firewalls tend to corrupt or block legitimate traffic.

In one of our project we had WAF & one of specific functionality our application was doing to upload different kind of document to our portal. We had seen major issues where WAF was blocking traffic pretty randomly because of some of rules (specifically XSS rule) was interpreting document we were uploading had malicious content.

There was no issue with documents we upload but after deep diving we saw some of meta-data of document are cause of concerns but again that too vary with machine from where these documents got created.

We end up modifying & disabling some of WAF rules because as we already had solid file scanner in place. So long story short, we didn't relay on WAF for security but there were additional security measures in place.

Considering this scenario, my assumption is that you are primarily relying on WAF for your security which might have raised red flag during review process.

Ashish Patil
  • 111
  • 5
  • "my assumption is that you are primarily relying on WAF for your security"

    I think this is where I'm struggling. I can't work out how anyone could just make that blanket assumption. You (and others) have outlined where the use of some software in front of your web application might make sense as part of a defence in depth strategy. The client in question jumped straight to the [wrong] conclusion that you did here.

     – Anon Dec 01 '23 at 12:16
0

I would never call a WAF a security vulnerability by itself. That is, unless there is a known issue with your specific implementation. There are three primary issues with using a WAF en lieu of finding/fixing vulnerabilities:

  1. The WAF, by definition, is unaware of the context (with regard to the backend application) of a request. Because of this it can only block certain types of attacks (typically injection style attacks). Even then, clever attackers can often evade the protection through various reformating/reencoding techniques that bypass the rules in the WAF. A WAF is never going to be able to detect and stop logic-level vulnerabilities, or at least not without very careful, custom tuning.

  2. Even if a WAF was 100% effective at blocking the attacks against your application, it still is acting only as a wall between the attacker and your vulnerable application. This means that if the attacker can get past the wall, the WAF will no longer protect you. Common scenarios where this could occur include:

    • The attacker lands a phishing foothold and can tunnel to the app from a "trusted" internal corporate IP address.

    • A poorly configured WAF may be circumvented by directly accessing the IP of the webserver or load balancer behind the WAF.

    • A server-side forgery attack might allow the malicious request to appear to originate from the same subnet or server as the vulnerable code, thus bypassing the WAF.

  3. It is difficult for a WAF to always stitch requests together and view them the same way the application does. This is due to the fact that each webserver, loadbalancer, proxy, etc. may chop up and interpret the request slightly differently. This means that new bypasses are always being discovered from transport-encoding, TLS parsing, cache-poisoning, and other attack methods.

The bottom line is that a WAF should always be viewed as a defense-in-depth technique or a quick mitigation while more thorough fixes are implemented. It is always an imperfect solution, unlike patching known vulnerabilities in the underlying software.

Bergi
  • 285
  • 2
  • 10
shellster
  • 628
  • 4
  • 6