Applications and WAFs
What is an app?
How does an app actually work?
The environment in which apps exist
So what is a WAF?
How are WAFs changing?
Where WAFs fall short
Shielding – a better solution
What is a ‘Shield’ ?
WAF and Shielding differences
The RedShield approach
What is a web application or ‘app’?
Most people in the modern world have heard the word ‘app’ (short for application) before, and common use is within the smartphone market, as that’s the most common ‘packaging’ of applications with which the general public interact in the modern era.
In previous decades of the computing industry, the most common implementation of an application was desktop software installed locally on a Microsoft Windows, or Apple MacOS platform through which a user was expected to interact through a keyboard and mouse to perform the activity the application was designed to facilitate. The most common applications that should be familiar from within this era are Microsoft Office (and its components Microsoft Word, Microsoft Excel, Microsoft PowerPoint and Microsoft Outlook), Lotus Notes, or Siebel CRM.
In the current age, applications are typically deployed to a much wider range of consumer endpoints such as Google Android smartphones and tablets, Apple iPhones and iPads, or are accessed remotely by use of an internet browser such as Google Chrome, Mozilla Firefox, Microsoft Edge or Apple Safari. An application (or “app”) is essentially a software program (or collation of 3rd party software modules in more complex application ecologies) that is deployed for use by end users to perform an activity through some form of Graphical User Interface (GUI). The intended activity of an application differs from one to another, and are defined by what is known as a use case during the application design phase.
An application differs from a service (which is also a software program, or collation of 3rd party software modules) in that a service does not have a graphical user interface intended for an end user to perform an activity. A service is often referred to as a “system” that sits “below” the application, and is typically a dependency of one or more applications to facilitate “internal” communication between software modules and components to facilitate the activity intended for the end user as part of the use case. They also provide communication functionality to “external” services offered by 3rd party providers that make additional functionality available that are not able to be facilitated by the application’s “internal” components. Common examples of this functionality are booking a job request with a courier or logistics company to collect or deliver a package, performing credit checks against finance requests for goods and services, sending marketing campaigns or advisories to a customer base, or online customer help chat functions. The most common form of service in the modern era in relation to applications is known as an Application Programming Interface (API) which are employed by many apps to pass input/out data between them.
For us to talk about Web Application Firewalls and Shielding as the purpose of this article, the layman’s definition does require some expansion.
We live in a world driven by commercial activity that is increasingly internet-based. This requires a way for people to interact with information in a highly available, user-friendly, and secure way. Web applications are the vehicles in which this information can be processed, and transactions can be made over the Internet. A web application is a visual, interactive representation of an ecosystem of software/hardware components designed for a particular end-use. Examples include social networking sites like Facebook and Instagram, ecommerce marketplaces like Amazon and eBay, and online banking applications most of us use almost to the exclusion of traditional physical bank locations. The part that a user can see and interact with is usually a very small component of the giant, complex machine driving the functionality behind that site or app.
How does an app actually work?
Apps are designed and built to respond to specific requests for information by the user through the use of a web browser, or mobile application. That request is handled first by a service (typically a web server of some description such as Apache, NGINX, or Microsoft’s Internet Information Services) then forwarded to a web-application server, where the requested task is actioned.
A response is then compiled according to the parameters of the request, and the logic of the application, and sent back along the same “channel” to be displayed to the end-user by what is accurately termed ‘the application layer’ of the TCP/IP stack.
As businesses and indeed our lives have become more dependent on applications, ways have been developed for them to share information between each other. Chief among these connectors of the modern app ecosystem – the APIs – are ostensibly an application in and of themselves. They present information to an end-user that’s made the request, the difference being that the end-user, in this case, is another machine, and it’s a stream of pertinent data rather than pixels on a screen.
The environment in which apps exist
The internet is a hostile environment, with skilled opportunists always looking for ways to access information they shouldn’t be able, and/or manipulate systems for their own ends. With this being the case, defenses need to be employed for the protection of all legitimate users. This is why a proactive approach is mandatory, as possible vectors for exploits need to be identified and fixed, instead of a reactionary stance where if you have been breached, it’s already too late.
Cybercrime is prolific. With increasing modernity and levels of professionalism, it is costing commercial industries trillions of dollars in productivity, information loss, reputational damage, and financial theft already, with estimates rising above the level of USD$6 trillion annually by next year, this already puts it several times the value of the illicit drug trade. With the widespread availability of automated tools, and vulnerability information, it has never been easier for even a novice to compromise a targeted system. A quote by R. S. Mueller III, FBI Director reads, “There are only two types of companies: Those that have been hacked and those that will be hacked.” Most knowledgeable security professionals would agree that this quote is well out of date, and to reflect the true reality and current landscape, should actually say, “There are only two types of companies: Those that have been hacked and those that don’t know they have been hacked.” as bastardised by Dmitri Alperovitch. The adoption of a raft of third-party vendors over traditional in-house solutions also provides further obfuscation of the vulnerability problem.
The obfuscation matrix
Due to the speed at which the technology landscape changes, the time-delay between vulnerability discovery, acknowledgement, analysis, impact assessments, dissemination of information, patching and remediation strategies and their approval, and then testing; organisations are, unsurprisingly, always behind the proverbial 8-ball. This also extends quite often to vendors themselves, where a wilful ‘ignorance is bliss’ is all too often a second line of defence.
As applications increase in the complexity of their design and number of sub-components used for the sake of additional functionality and speed, so too the attack surface expands exponentially. Defenses need to be layered, constantly reconfigured against known threats, be multifaceted, and possess a certain logic/intelligence in the way they handle incoming data. With any oversight or error, a breach becomes possible. While historically there has been some modicum of protection afforded by the limited number of capable threat-actors, this tide has shifted. Fuelled significantly by the rise of the hacking subculture in popular entertainment, and the commoditisation and ease of obtaining the tools and training supplied by a burgeoning dark market – including the buying and selling of vulnerability information – the sheer volume of attacks over the past decade has become a deluge. And while the vast majority of these attacks are trivial, they still continue to add to the demands and overall work and cognitive load of IT and security professionals. In the more significant cases, attackers can do anything from simply defacing publicly visible assets, compromise or exfiltrate sensitive data, gain credentials for users (or even ultimately administrators allowing for complete takeover of the server), cryptomining, and potentially weaponise the power of the machine(s) towards more nefarious activities like denial of service attacks, malware or email spam campaigns. And once an attacker is able to gain purchase in such a manner, they have the ability to all but guarantee future access through a multitude of techniques.
In this context of continually expanding threat and the growing awareness and focus on cyber risks, all too often a panacea is promised, and snake-oil delivered.
Ignorance isn’t bliss
So what is a WAF and when/why would you use one?
Web Application Firewalls (WAFs) are a valid line of defense against malicious attacks or the manipulation of HTTP traffic, but far from a total solution. They are appliances or modules that evaluate HTTP requests and responses with a binary ‘good/bad’ system governed by pre-set rules or programmed logic – a whitelist/blacklist. A WAF can be thought of as the proverbial ‘nightclub bouncer’ for identifying incoming and outgoing web traffic that falls outside of established rulesets, and therefore flagged as potentially malicious, and act accordingly; block, redirect, alert, or score (response by accumulation), according to the degree of risk dictated in the rulesets with which it’s been issued.
WAFs are typically deployed with a base, default ruleset like the OWASP ModSec CRS, designed to cover a large percentage of potential issues when correctly configured. It should be noted that this approach forms merely a first step as the binary filtering tool against commonly known web application exploits such as Cross-Site Scripting (XSS), SQL injection, and Cross-Site Request Forgery (CSRF), and crafted HTTP/HTTPS, XML-RPC and SOAP data packets.
While it ostensibly seems rational to protect your application; ‘if a user is inputting malicious content then thwart’, there are fundamental problems with this approach. The issue is that there will always be a way to circumvent these protections, and concurrently, legitimate transactions can always be blocked. This has the dual penalty of providing a false sense of security and/or also impeding functionality, both of which are impermissible cases for any operation. It’s important to understand that the ramifications of unintentionally stopping transactions can’t be overstated. If an end-user – be it human or API – isn’t able to use your platform to achieve the outcomes sort, then the fundamental reason for its existence is removed, and there are directly attributable impacts to your business.
This doesn’t mean we should ignore input validation. To the contrary, it is inarguably the first line of defence and should be held in primacy in the SDLC to ensure it’s baked-in not just tacked on. So while valuable, WAFs just aren’t the magic bullet they are all too often purported to be.
How are WAFs changing?
WAF’s at their core rely on signatures. The fundamental issue with this approach is that if you go broad to ensure you block as much malicious traffic as possible, you are inherently more likely to block legitimate user requests, or a false positive. The counter to this is the more granular, the simpler for an attacker to side-step the signature – the false negative issue. In the most common scenario, we just see default profiles applied, or even the WAF turned off completely to stop false positives. Should a capable security professional be involved, this turns WAF maintenance into a constant see-saw act of tuning to find the least offensive mix of false positives and false negatives; an untenable position as its core, but secondly, the process also carries a huge time burden in the context of a DevOps environment.
With the adoption of libinjection (a c-library to identify SQLi attacks) and other session-scoring methods, coupled with the false positives quandary, we have seen WAFs start to become something different. As it’s incredibly challenging to do so effectively and efficiently, WAFs are no longer instituted as a mechanism for blocking discrete attacks. The admirable notion of going to the root of the problem has been used as justification for what is purported to be a viable alternative; WAFs have been repositioned as tools for the identification and removal of perpetrators. Logically then, we’re seeing a pooling or bundling of mechanisms that achieve complementary and alike outcomes like DDoS detection.
The rise of Next Generation Web Application Firewalls (NG-WAFs) appearing on the market should come as no surprise. These obviously build on their precursors as the name would suggest, and now include defences against Distributed Denial of Services (DDoS) attacks as well as other ‘bot’ (software applications that run automated tasks, aka, scripts) attacks. And while they are certainly a substantial evolution, they aren’t the panacea that they’re often presented to be.
Where WAFs fall short…
A WAF detects exploit attempts as its raison dêtre – buffer overflows, SQL injections (SQLi), directory traversal, Local File Inclusion (LFI), and Cross-Site Scripting (XSS) are all through the vitiation of input validation routine. The issue arises when we see organisations so expectant on their WAF as a primary form of protection. We’ve seen WAFs in the Gartner magic quadrant fail to detect over 97% of SQLi attacks as defined by the OWASP SQLi cheat sheet among other shortcomings. To clarify, this is a clearly catastrophic level of failure by the ostensibly ‘technical leaders’ when it comes to the provision of expected protections. More egregious, the parameters for such protections that have been clearly laid out by the leading industry body are, ‘…simple, actionable guidance for preventing SQL injection flaws in your applications.” and have seemingly been altogether wilfully ignored.
To understand the use case and environment in which we’re living, we need to also look at the basis of the threats we’re dealing with too. A security compromise or breach can take place through a number of mechanisms. The most talked about, malicious traffic exploits, account for roughly 15% of breaches, while system glitches – the state in which an application might exist, unforeseen by its developers – account for some 25% of breaches according to IBM/Ponemon’s 2019 report. Looking at this context, we see WAFs have several limitations in real-world implementations.
WAFs aren’t able to resolve exploitable elements within an application, they only see as far as the rule-set with which they’ve been programmed. The task of resolving underpinning code or logic issues remains the responsibility of developers, and without these fixes, audits are failed, project deadlines not met, and fundamental legacy systems are forced from production, like vital ERP systems needing to be decommissioned in inopportune time frames.
By their very nature, WAFs are hamstrung by their essential dualist nature that limits their potential response to infinite permutations of an attack. This manifests as rulesets that are bypassable, as well as the blocking of legitimate requests.
As the ever-evolving threat landscape changes, new information must be injected for WAFs to stay on top of these threats. The sheer volume and growing sophistication of threat-actors means many vendors, let alone end-user organisations don’t have the requisite resources nor capability to constantly update this rule-set against emerging threats and test every application in use to ensure compatibility is maintained. And as we stack the signature bricks into an infinitely growing wall, the complexity and intraoperation of each becomes more and more unmanageable and legitimate traffic more likely blocked. And while Artificial Intelligence (AI) / Machine Learning (ML) are arguably the most likely candidates for addressing this in future, they are despite claims to the contrary, still in their infancy.
If we look at these three points holistically, it goes some way in explaining the changing approach we’ve seen in how WAFs are trying to now operate. While the sophistication and efficacy of their ‘updates’ has unquestionably improved, WAFs are still restricted to scoring users; essentially against a ‘database’, an inherently limited index of known malicious activities, geolocations, or previously identified bot behaviours.
An issue arises when we see methods employed that aren’t targeting vulnerabilities and aren’t presenting attack signatures. For example, the use of stolen credentials retrieved by the threat actor or procured from the dark web, or the scraping of unprotected content both go undetected by a WAF, or session hijacking where tools or a determined party is able to guess or induce session ID’s, allowing for that session to be taken over. Further, the use of bots can apply scale to this equation, making the limitations of a WAF become ever more apparent. And with the growth of botnets, the uptake of IPv6, and the boom in IoT deployments, the traditional limitations of the IP-centric nature of WAFs also becomes more apparent as threat actors render the IP rules of a WAF moot. The underlying problem is that we’ve an untenable balancing act between false positives or false negatives with no margin for error as either is impermissible.
The outcome, a WAF is no longer a tool for actively blocking attacks, merely a source of intel for hunting threat actors. So while WAFs can effectively guard against targeted vulnerabilities and identify and respond to malicious users, specific IP ranges, or even nation states, they are ill equipped to identify and respond to mechanisms that aren’t trying to exploit software vulnerabilities, but rather are targeting flaws in business logic. What’s more, to even maintain their relevance and stated effectiveness, they still require the substantial skill sets and experience to make full use of them and avoid the all-to-easy issue of misconfigurations. These errors can not only render the WAF an ineffective security device, but also impede legitimate traffic from your users.
So, is there a solution to this? Is there a way to not just identify malicious users, but to extinguish the very problems against which they can launch attacks?
The shortfall of WAFs and a better solution
What is a ‘Shield’
To understand this advantage, it’s best to look at the alternatives. Normally, when an application security issue is identified, the options are:
- Turn the app off while your dev team fixes the issue – a lengthy and expensive undertaking that also removes access to the application
- Leave the app live while your dev team fixes the issue – again, lengthy and expensive, and depending on the risk, typically unacceptable
- Upgrading the vulnerable component – possibly introducing new technical or commercial issues
- Block all transactions using a WAF, rendering the application safe, but non-functional
- Do nothing – you obviously leave yourself totally exposed to the identified vulnerability, as well as potentially negating cyber insurance claims. This may also be the situation forced upon you, waiting for months or years until a critical mass of users demands resolution from a vendor upon which you rely for your entire business.
While points one, two, three and five are fairly self-explanatory, how a WAF renders you safer carries implications that aren’t necessarily immediately obvious. Further, when examining what RedShield offers – in terms of both method and outcome – needs more clarity, most easily done in comparison and contrast with the stock solution of WAF deployment.
Differences between WAF and Shielding
If the WAF sees traffic that it knows is bad, it gets blocked.
A Shield is an in-path solution where all HTTP traffic for your application travels through RedShield where additional protections are added before passing through to end-users. Shielding doesn’t limit what a user can do, but rather changes the application transparently so that anything malicious has no effect, and without disrupting the user experience. So instead of just filtering like a WAF, shields actually fix the issue by changing the application behaviour, and without changing the underlying code itself. The flawed application is rendered safe while retaining its functionality – attacks are stopped, but transactions aren’t.
So while a WAF promises to protect through ever increasingly complicated policies, Shields address the root of the problem. For example, if your app accepts weak passwords, WAF policies might be deployed to check geolocation, frequency of attempts, the method of access, time of day, or countless other data points. The problem is, as compatibility issues inevitably come to light and the challenge of constant updates and maintenance becomes apparent, most businesses make their controls less and less effective to maintain app functionality. A Shield just implements better behaviour for the application.
Request traffic is shielded and sophisticated attacks are neutralized, eliminating vulnerabilities in your application without touching your code, and without disrupting any of your customers or users.
Response traffic is also shielded to harden against attack. Shielding improves cookie hygiene, session security, prevents data disclosure, and session hijacking, providing effective security against any known vulnerability.
The RedShield approach
…Then a new signature set is invariably released, and the process starts all over again. Eventually, it becomes increasingly necessary to remove the WAF or reduce its effectiveness to a point where it becomes essentially useless. And while some WAF providers do offer a managed service, this is typically just building more complex rules, leading to further compatibility issues. A WAF project also requires an all too scarce resource – the expertise to ensure correct and optimal configuration.
The RedShield approach is different – we stop hacks not transactions. We eliminate all known risks, then we add additional protections, and then we prohibit future threats.
We’ve mapped our process and tooling to the framework developed by the National Institute for Occupational Safety and Health in the United States – the NIOSH risk reduction hierarchy.
Instead of removing the contamination, a WAF just provides better safety boots. While this is certainly a good step and improves the immediate safety of workers, it should be done only as part of the larger hygiene sequence. RedShield cleans up the spill, puts in new flooring to prevent future spills, updates workplace protocols, and then provides the newer boots. You’re now not only better protected than previously, but you’re better protected in a safer environment.
RedShield is a service that reduces operational risk.
Engagement model – the RedShield stack
We’re able to deploy a Shield for your application often in a matter of minutes rather than the weeks, months, or even multi-year timelines typical with software remediation. And that’s if your dev team can solve it at all given they may not even have access to the problematic code as it sits with a third party or is otherwise obfuscated or inaccessible.
Our managed service then deals with the dynamic nature of newly identified issues, focused on both security effectiveness and maintaining applicational compatibility, thereby minimising business disruptions.
How RedShield engages
RedShield certainly helps in urgent situations and responds with a custom Shield to get you protected rapidly. Ideally, like most protections, Shielding is done strategically rather than as a tactical reaction. This is where RedShield is not a project, but a program where our service is aligned to your business and we manage the outcomes for your application security.
Shielding in the real world
Continued learning—case studies
The Preconceived Cloud Security Problem Businesses are overwhelming realizing the benefits of adopting cloud infrastructure strategies. Read more
What is a Session? A session is the period of time, post login, that the application is in communication with a user without requiring ... Read more
Shield A shield is a block of code that modifies application behaviour to fix a known exploitable vulnerability. Like a micro service, ... Read more
Application Shielding is a new tool for many organizations and the subject of several commonly held beliefs which may prevent a ... Read more
THE INTERNET: A BUSINESS ENABLER & RISK The internet is a huge business enabler; it provides: Lots of apps and services that Read more
Executive Summary The General Data Protection Regulation (GDPR) considers data protection as a fundamental human right of an ... Read more
The Challenge With cyber-crime now bigger than the drug trade, criminal organizations are treating data breaches in a structured ... Read more
THE CHALLENGE With cyber-crime now bigger than the drug trade, criminal organizations are treating data breach in a structured fashion. Read more
What is a Clientside Authentication? Authentication is the mechanism through which a user proves their identity. Once this has been ... Read more
Mega-Breach discovered July 2017 September 2017 Equifax disclosed a mega-breach of its systems, carried out though exploitation of a ... Read more
THE SITUATION After 3 year program of work, a tier-one APAC bank was ready to assess the effectiveness of their WAF deployment that had Read more
THE SITUATION As a payment transaction company processing millions of commercial transactions each day, ensuring the security of ... Read more
THE SITUATION A health care company was embarking on a company wide digital transformation program to achieve service delivery ... Read more
THE SITUATION A government agency had an old web application that was delivering core service capability. Although functionally ... Read more
THE SITUATION A university with a large public web application perimeter had a number of old web applications running on legacy windows Read more
THE SITUATION During a routine application penetration test an SQL injection vulnerability was discovered for a public banking ... Read more
THE SITUATION Late one afternoon, it became apparent that customers of a loyalty card service were being targeted by password reuse ... Read more
THE SITUATION A power retailer had a problem, they were facing the common trade-off between security and customer experience: Due to ... Read more
THE SITUATION A large APAC based insurer had engaged a range of external software development companies and contractors to implement a ... Read more
THE SITUATION To both enhance service and optimise cost of delivery a large US based healthcare provider had embarked on a full digital Read more
THE SITUATION A global telco had a number of applications that were subject to PCI regulations in a specific jurisdiction. One ... Read more
THE SITUATION In response to the increased cyber threat to its online presence plus regulatory compliance, a retail chain with ... Read more
THE SITUATION A massive UK organisation performed a penetration test on a critical internet facing application. The 15 year old ... Read more
THE SITUATION A business publishing commercially sensitive data on a periodic basis is subject to inevitable traffic spikes. To ... Read more
THE SITUATION When public sector entities merge, IT system harmony and cost reductions are the order of the day; especially when the ... Read more
THE SITUATION A major logistics company had followed a tender process to commission a complete logistics management application to move Read more
Over a 6-month period, multiple security testing companies revealed that the online service provider had made little to no progress on ... Read more
THE SITUATION For large payment transaction providers, policy and regulatory requirements must be met while simultaneously optimizing ... Read more
THE SITUATION For this mid-sized government department, approximately 100 browser-based applications run process essential data for the Read more
THE SITUATION As a power retailer maintaining millions of customer accounts, security of financial systems is central to the privacy of Read more
THE SITUATION Online offers boost sales – no doubt. This company publishes online offers on specific dates and site visitors validate ... Read more
THE SITUATION As a cloud-based, client management software service provider serving the healthcare sector, storing and protecting ... Read more
THE SITUATION A company wanted to move their ecommerce store workload to Amazon Web Services (AWS) but retain their customer care ... Read more
THE SITUATION A large government agency using Microsoft SharePoint had been attempting to put important security controls in place for ... Read more
All the healthcare company had to provide was the hostname, SSL cert and any vulnerability data. Then once RedShield had completed the ... Read more
RedShield - Presentation & Service Demonstration Read more
THE SITUATION As a payment transaction company processing millions of commercial transactions each day, maintaining the appropriate ... Read more
THE SITUATION After a public threat from a well-known hacktivist group, a prominent political party’s websites were subjected to ... Read more
THE SITUATION In the financial portfolio management business, the customer web portal is both critical to business continuity and ... Read more
THE SITUATION A large insurance company was introducing a new application platform where their digital insurance products were to be ... Read more
In the 2 years since deployment, the profile has remained in blocking mode and not a single false positive has been reported. To date, Read more
Test drive RedShield
During a test drive, you can see the value of RedShield on one of your websites.
We'll help you make your vulnerabilities vanish, removing the ability to exploit completely.
If you’d prefer one of our consultants to reach out to you as a first step, please leave your contact details.
Schedule a discovery call
Test drive RedShield
If you wish to take RedShield for a test drive automatically, you can get started in just a few minutes.