Essay: Automated Security Testing for Web Applications

Nowadays, the dependency on Web Applications is enormous. From bank transactions to online shopping to online data maintenance(cloud storage), everything runs on web based applications. On the other hand, the hackers are becoming more sophisticated and organized to break into these web applications. This Research Project attempts to find out the test automation tools for security of these kinds of web applications. Our focus is to look for the Black box testing tools available and its performance analysis (done by previous researchers). We also intend to find White box testing methods for secured web application development, if available, and how industry or researchers are trying to automate the testing process keeping security as priority. We also intend to find out the time to learn to use some of the tools and the ease of learning those tools.
Index Terms’Web Applications, security testing, test automation, Information security
I. INTRODUCTION
With the advent of Web 2.0, lots of user data is being generated every day. With large volumes of data, comes the responsibility of protecting/securing it from people with malicious intent. Lots of confidential work takes place over internet now a day. A user accesses Web application services to get the work done. Web applications programs are stored on a remote server and delivered over the internet and accessible via a web browser. These applications receive input and deliver output, usually in the form of HTML or XML. Web applications are dynamic and interactive often involve complex databases and servers at the back-end. Tremendous volumes of vital data are stored on these web applications. When the number of transactions increases on web then the deep security testing of web applications becomes very important. There is a need for better understanding of malicious cyber activities, so that better security practices (to prevent the malicious activities) can be involved into the Web application development. It is with rigorous testing that security can be assured. And testing takes time, thus automation is a way to expedite the testing process, so that with minimal human intervention, software quality (in terms of security) can be assured.
The batch and penetrate model was used during 1990’s which involves reporting and fixing the bug. The major drawback of this model was that there is a gap between the discovery of the bug and its patch being released
As shown in the figure 1, the applications remain vulnerable until the patch is developed and successfully deployed. This gives a period of time where the application remains vulnerable as shown in the time axis of the graph. The OWASP (Open Web Application Security Project) is one such non-profit organization which focuses on improving the security of software.
II. RESEARCH OBJECTIVES
1. To find out various security vulnerabilities and testing methods available.
2. To find out the Black box testing tools especially designed to perform the automated security tests and to analyze the cost to effectiveness comparison.
3. To find, if there are test first strategies available (White Box testing) for secured web application development. If it is, then is there any research done on automating the security testing for the web applications?
4. How Model-Based Testing helps in achieving automation in security?
5. Advantages and Disadvantages of Black Box and white box security testing tools.
6. To test learning of security testing on the vulnerable websites specially designed for programmers to learn about security aspects.
III. LITERATURE REVIEW
The initial research revealed many products being produced every year but are of little use to the real world threat scenarios. In [3] Bezemer et. al. discusses the security testing of AJAX web widgets and concludes how intricate it can become if it is tried to be automated. Every application that uses AJAX pages is unique in itself and thus every organization has to spend unique testing strategies for each website. The paper revealed no serious tools being revealed in the market. to test such applications.
In [5] the IBM researchers Artzi et. al., in 2011 introduced few basic algorithms to test website powered with java scripts. There research concluded with an algorithm for automated testing of JavaScript applications to discover programming errors. It is to be noted that the security testing of the basic constructs of web applications are still under research. The above research culminated into a limited capability product which can only be run on browsers characterized by Enjvs library.
So, researches like [1] [2] [6] [7] reveals the fact that automation in security testing is still under research. All these above research are done within past four years and so far, there is no product that claims implementing any of these testing methods.
The most common security vulnerabilities and methods of testing that industry accepts are described in detail on OWASP web portal.
It is also to be emphasized that the security issues that needs to be tested are still under discovery phase. So far, the most common security vulnerabilities found are tried to be automated via automated security scanning tools. The limitations of these tools, according to [10] is that they can test only the errors in the usage of functions and cannot test the errors out of faulty logic. For instance, use of strncpy() is preferred over strcpy() in C++. Our aim in this research is to find out the different security tools available currently in the market. The two basic security test automation tools falls under the category of black box testing and penetration testing(both are automated via customization).
IV. AUTOMATED SECURITY TESTING INTRODUCTION
Research objectives 2, 3 and 4 are accomplished by reading various papers and finding mostly used successful commercial web security testing software. The marketing research for Gartner indicated the most successful commercial testing tools available.
A. Importance of Automation in security testing:
The importance of security testing is well known. Security testing is a process to determine that an information system protects data and maintains functionality as intended.
The six basic security concepts that need to be covered by security testing are: confidentiality, integrity, authentication, availability, authorization and non-repudiation [12].
Major security breaches resulting in service downtime, loss of data, and brand damage cost businesses an average of $7.2 million dollars per breach according to Ponemon Institute 2012 report.[13] It is because of such reasons, security testing has become an overall part of the application development process.
Software security testing is a special kind of testing with the aim in validating and verifying that a software system meets its security requirements. Two principal approaches are used – functional security testing and security vulnerability testing [10].
Although security testing techniques are available for many years, little approaches are there that allow for specification of test cases at a higher level of abstraction, for enabling guidance on test identification and specification as well as for automated test generation [10].
Products and services are first tested to ensure proper ‘normal’ operation. This is sometimes complemented by vulnerability-specific tests mandated by ISO compliance, secure programming initiatives and risk management best practices. Vulnerabilities are most often exploited by manipulation of user input in URLs and web page form fields, all the way down to the bits and bytes in protocol packets.
The artifacts for security testing are code, generated binaries, class files, objects etc. and systems running the web application
B. Information security and automated testing:
The security tests include both White Box security testing, such as source code analysis and Black Box security testing, such as penetration testing. Gray box testing is similar to Black Box testing [18]. In a gray box testing a partial knowledge is of the application under test is assumed, for example knowledge of session management of the system. Security Testing can be integrated into Developers’ and Testers’ workflow[18]. Security testing during the development phase gives the first opportunity to ensure that individual components to be security tested before they are integrated with other components and build into an application. For this the developers can rely on the white box source code analysis tools. The process can be automated by tools developed my companies like Microsoft (preFAST and preFIX).
C. Security testing in the Developers workflow: Security Testing in the Coding Phase begins from the Unit Tests. The main objective of the security tests in this phase is to validate that code is being developed in compliance with secure coding standards requirements. Developers use source code analysis tools integrated into their IDE’s. Security test cases can be run using Unit test frameworks such as JUnit, NUnit and CUnit (which are adapted to verify security test requirements). For example testing of the input and output validation and boundary checks for variables could be validated by a unit test. Threat scenarios can be used with use cases and misuse cases[28] in the unit testing phase. A security misuse case is a variation on a use case and is used to describe a scenario from the point of view of the attacker.
D. Security Testing in the integration and Validation Phase: The integration system test environment is where testers simulate real attack scenarios. Security testing at this level can validate whether vulnerabilities are real and can be exploited by attackers. These type of tests are also referred as Ethical Hacking.
NOTE: The term SAST is used as White Box Testing and DAST is often used as Black Box testing throughout this paper
V. TOOLS AVAILABLE FOR AUTOMATED SECURITY TESTING:
Various tools are available and they are categorized by their nature as below:
There are four types of automated security testing techniques being used in the organizations:
‘ Black Box Testing
‘ White Box Testing
‘ Penetration Testing
‘ Model Based Testing
Black box security testing tools are sometimes called web application scanners or Dynamic Application Security Testing(DAST).DAST tries to identify architectural weaknesses and vulnerabilities in running web applications. Many organizations have started to use automated web application scanners. But their limitations should be understood, and testing frameworks should be planned appropriately [18].
White Box security testing sometimes referred sometimes as Static Application Security Testing (SAST). It is also known as source code analysis.
Static Analysis is done when source code is available and SAST is applied during the development phase. There are static analysis tools available like Microsoft’s static analysis tool called Scalable, Path Sensitive Source Code Analysis (PREfix) and Intra-procedural Source Code Analysis (PREfast) [19]. Microsoft has even developed Security Development Lifecycle (SDL) which is a security assurance process consisting of security practices grouped by seven phases as shown in the picture: [20]. The use of the SDL has been mandatory at Microsoft since 2004.
Fig. 2: Security Development Lifecycle (?? 2010 Microsoft Corporation.)
Static analyzers can be used both at the check-in time and at the final build. NASA also requires that every code change to mission critical applications go through static analysis. In essence Static tools analysis comes under white box testing.
Security requirements derivation through Use and Misuse Cases.: Like the use case diagram of UML, the designing of the security test cases needs to be thought over. Similar to use cases, security testers needs ‘misuse’ test cases in order to perform security testing. The limitation is missing some test cases that might turn out to be vulnerable to the application.
A. The ways of automating Black box testing:
A note on Selenium: Selenium is an open source functional test automation tool. Selenium IDE is a technology for QA testers and developers that allows recording of functional test sessions in the web application for future replay. Instead of having to manually test the web application functions every time a change is made, you can simply run the Selenium IDE test case again [24]. This tool is used as a supplement with other security scanners described below. For example, ZAP works with Selenium and so does OWASP Xelenium tool. A prerequisite with Selenium is that it uses Mozilla Firefox. Mostly Selenium acts as supplement for the applications i.e. it helps in writing test cases that will be executed automatically and a Security testing tool then intercepts the website in order to find any type of vulnerability in website.
Through the use of commercial software packages
‘ HP (SPI Dynamics) WebInspect & DevInspect
‘ IBM Rational (Watchfire) AppScan
‘ Cenzic Hailstorm
‘ NT Objectives NTO Spider
‘ Acunetix Web Vulnerability Scanner
‘ Configuring the open source Black Box testing tool to suit the particular business requirements.
‘ The open-source Black Box scanner projects are W3AF[29] and Powerfuzzer[30]
NOTE: The security intelligence blog Gartner[27] published 2013 Magic Quadrant for Application Security Testing (AST) report which gives comparison of all the leading commercial SAST and DAST soft wares
B. The ways of automating White box testing
Source code review is the process of manually checking a web application’s source code for security issues. Many serious security vulnerabilities cannot be detected with any other form of analysis or testing.
The best way to perform White Box testing is to run some security test cases against the code and try to expose vulnerability. Manual code scanning is sometimes done. Also, Selenium IDE can be used, if the test cases are ready. The most famous UNIX static code checker is lint; The Android SDK also provides lint that can help in identifying and correcting problems with the structural quality of the code, without executing the app or write any test cases. The Android lint tool is a static code analysis tool that checks the Android project source files for potential bugs and optimization improvements for correctness, security, performance, usability, accessibility, and internationalization [26].
Static Analysis tools are used to automatically detect defects, in terms of secured coding standards, in the source code. The use of Static analysis tool is encouraged because it keeps the security bugs in check early from the development phases
Source code review is the process of manually checking a web application’s source code for security issues. Many serious security vulnerabilities cannot be detected with any other form of analysis or testing.
C. Penetration Testing
Gary McGraw in [21] says, ‘If you fail a penetration test you know you have a very bad problem indeed. If you pass a penetration test you do not know that you don’t have a very bad problem’. Penetration testing is commonly known as Black box testing or Ethical Hacking [18]. Penetration testing is the most frequently and commonly applied of all software security best practices, in part because it’s an attractive late life-cycle activity and also the most commonly misapplied mechanism as well. Once an application is finished, it’s owners subject it to penetration testing as part of the final acceptance regimen.
Penetration testing has proven to be effective in network security. Testing is performed on networks and operating systems; the majority of the work is involved in finding and then exploiting known vulnerabilities in specific technologies.
Since the process is fairly repetitive, Penetration testing tools have been developed that automate the process. Many people today use web application penetration testing as their primary security testing technique [22].
Software penetration testing test’s success depends on many factors:
1. Tester skill
2. Tester Knowledge
3. Tester Experience
According to [22], Security penetration testing can be effective, as long as we base the testing activities on the security findings discovered and tracked from the beginning of the software life cycle, during requirements analysis, architectural risk analysis, and so on. A penetration test must be structured according to perceived risk and offer some kind of metric relating risk measurements to the software’s security posture at the time of the test.
Tools should definitely be part of penetration testing. There are basically two types of tools used for penetration testing and are as follows:
1. Static analysis penetration testing tools can examine software code, either in source or binary form, in an attempt to identify common implementation-level bugs such as buffer overflows.
2. Dynamic analysis penetration tools can observe a system as it executes as well as submit malformed, malicious, and random data to a system’s entry points in an attempt to uncover faults- a process commonly referred a to as fuzzing. The tool then reports the faults to the tester for further analysis.
According to Verdon[22] et. al., these two tools (Static and Dynamic analysis) have two major advantages:
1. When used effectively, they can perform most of the repeated work needed for basic software security analysis. Hence, they help in reducing the burden of reviewer and cost of analysis of the application.
2. Tools output lends itself readily to metrics, which software development teams can use to track progress over time.
Testers should use static and dynamic analysis tools uniformly at the component level. No customization of static analysis tools is necessary but for dynamic analysis tool will likely have to be modified according to the target component because they operate at API(Application Programming Interface) level.
Penetration testing should start prior to the system integration level in software development life cycle(SDLC). So, by applying penetration testing at the unit and system level , driving test creation from risk analysis, and incorporating the results back into an organization can avoid many common pitfalls so that, it can help in improvement of design, implementation and deployment phases.
D. Penetration Testing: The General Idea and Basic Steps:
The basic idea of model-based testing (MBT) is that instead of creating test cases manually, we use selected algorithms that are generating test cases automatically from a (set of) model(s) of the system under test or of its environment. While test automation replaces manual test execution by automated test scripts, model-based testing replaces manual test designs by automated test generation [12]. Regarding security testing, we are principally interested in locating critical system functionality with respect to the overall software architecture and in identifying security-critical interfaces, which might be an entry point for an adversary [7]. And also in [14] , For security testing, models of the system under test are mutated in a way that the mutants represent weaknesses or known vulnerabilities. These weakness or vulnerability models can then be used for test generation by various MBT approaches. The generated tests are used to check whether the system under test (SUT) is weak or vulnerable with respect to the weaknesses and vulnerabilities in the model. There are many approaches related to Model Based Security Testing and it’s Automation that are described below.
In [18], Wang et. al. presents a threat driven approach to MBST. In this approach, UML(Unified Modeling Language) sequence diagrams are used to specify threat in a model, i.e., event sequences that should not occur during the system execution. The threat model is then used as a basis for code instrumentation. Finally, the instrumented code is recompiled and executed using randomly generated test cases. If an execution trace matches a trace described by the threat model, security violations are reported and actions should be taken to mitigate the threat in the system.
Steps proposed by Schieferdecker et. al. [7] in generation of Security Tests using a model-based approach:
1. Identify security test objectives and methods: The test objectives define the overall goals of testing.
2. Design a functional test model: Security testing focuses either on testing the correctness of security functions or on testing the robustness against a dedicated misuse of the system. Thus, functional test models used for security testing describe not only the typical environment or usage of a system, but also adversary environments or atypical usages like attacks and hacking attempts.
3. Determine test generation criteria: Usually, there is an infinite number of possible tests that can be generated from a model, so that test designers choose test generation or selection criteria to limit the number of (generated) tests to a finite number by e.g. selecting highest-priority tests.
4. Generate the tests: The test generation is in MBT(Model Based Testing), typically a fully automated process to derive the test cases from a given test model as determined by the chosen test generation criteria. This is also true for MBST.
5. Assess the test results: During test result evaluation and test assessment, the quality of the SUT can be rated with respect to the test results as well as the quality of tests can be rated with respect to their fault and vulnerability revealing capabilities. However, it is still a research challenge to assess test results in MBST.
In [16], Security Functional Testing helps to verify whether the behavior of a product or system conforms to the security features claimed by the manufacturer.
According to Blackburn et. al., NIST (National Institute of Standards and Technology) and its sponsors initiated a multi-phase investigation to assess the use of a model-based approach to automate security functional testing. This automation will help security evaluation laboratories meet the demand for product testing. The automation approach is based on expressing a product’s security functional requirements in a model and using the supporting toolkit to automatically generate tests needed to verify security properties. The TAF (Test Automation Framework) integrates various modeling tools, like the SCRtool for modeling system and software requirements with the test automation tool T-VEC(Test Vector Generation System is commercially available from T-VEC Technologies, Inc.).
The TAF approach, customized with specific guidelines for modeling security properties and developing test drivers for databases, satisfies NIST’s requirements for an automated model-based approach to automated Security Functional Testing. This approach reduces the time and effort associated with security testing, while increasing the level of test coverage. These results demonstrate the feasibility of using model-based test automation to improve the economics of security functional testing [11].
VI. RESEARCH CONTRIBUTION:
A part of our research revealed the fact that Test first strategies doesn’t not exist (so far) for Security Testing. If the security bug can be predicted in advance, vulnerability issues can be tackled well in advance[21]. Unfortunately the security testing techniques available are based upon an application that was broken previously. Testing of an application for security vulnerabilities begins as soon as the integration of various components of a software is done.
Testing the ease of the tool and learn to write test cases for security vulnerabilities was our objective. The first step was to find the application with known vulnerabilities so that testing can be preformed. A little research led us to projects like OWASPS WebGoat, Google’s Gruere, INFOSEC Institute’s BodgeIt store project. Each application has its own limitation in terms of staring the project on one’s machine, configuring it for to run the test application. For instance, Google Gruere application could not be run as intended on the Safari Web Browser. The Bodgeit Project can only run on 64bit Machines with Eclipse installed on it. Since security testing involves yet another tool to intercept flow of request and response from the server, on a local machine installing the right type of server consumed a significant amount of time.
The small experiment to try to learn security vulnerability testing of Web application took a lot of effort in terms of making the environment suitable for learning. It was decided to test the BodgeIt application with ZAP penetration testing tool using Selenium IDE to for automating some test cases. The Bodgeit Application requires Tomcat server 6.
We decided to write a test case to test SQL Injection vulnerability, to test the validity of the tool ZAP. The BodgeIt Project defines the places where the SQL Vulnerability exist. On the user Login page if the user name is given as ‘1’ = ‘0 and password as ‘anypassword’, the system logs in. We made adjustments in the ZAP application program so that the Bodgeit Project can be proxied through ZAP so that the tool can intercept the flow of data. To automate the SQL injection testing another plugin SQL Inject Me by Security Compass as a firefox plugin needed to be installed so that multiple test cases can be submitted. As the multiple test cases are executed, ZAP intercept the running website on local machine and recognizes the vulnerability. The interesting observation was that there was some test cases that exploited the Bodgeit application, but ZAP did not recognize the vulnerability. We started looking for such incorrect results by ZAP and found that the tool itself has a bug which was reported as issue number 557 on Google projects[40].
All these blockers prove that the automation of security testing is still in its nascent stages with a lot to be improved in near future. Writing security test cases or mis-use cases itself requires significant expertise in security. Functional test cases are based upon the requirements well known in advance, security test cases are based upon the experience to foresee the future misuse of the application.
We had previous experience of using Black box testing tool called Accunetix Web Vulnerability Scanner. We used Moodle Learning Management System (LMS), an open source software e-learning platform.
There are many security issues like as authentication, availability, confidentiality and integrity attacks that were found in Moodle. We chose an old version with known vulnerabilities to test Accunetix Web Vulnerability Scanner s ecurity flaws of the Moodle.
Introduction of Acunetix Web Vulnerability Scanner:
Acunetix focuses on DAST tools. It offers a point solution and associated tools specifically designed for Web Application testing. Acunetix is used by information security specialists and penetration testing professionals looking for a reasonably priced, commercially supported Web application security testing tool with supporting tools and compliance reporting capabilities.
The conclusion of the tool usage was that it detected some of the known vulnerabilities, but not the one which involves a long attack vector. Those errors were not even hinted by the software. Those vulnerabilities can not be detected by automated security scanners because scanners checks the application against the general set of known issues. Though the use of such applications are recommended because it expedites the process of finding the common vulnerabilities early so that security experts can focus on deeply embedded vulnerability. The tool gives a hint to look for the possible areas to test for security vulnerabilities. But the reliance on such tools for security vulnerability is debatable.
VII. SUMMARY OF THE TOOLS DESCRIBED IN THIS PAPER:
Tool Name and Introduction How it Works
OWASP Xelenium Project: Used to identify Cross Site Scripting. Works along with Selenium functional testing tool Scans each text field of the provided web pages by making http requests. These requests are made using Selenium HtmlUnit Driver.
IBM Security AppScan Standard (previously known as IBM Rational AppScan Standard Edition): The tool tests Web applications for security vulnerabilities during the development process, when it is least expensive to fix such problems. It provides full coverage of the OWASP Top 10 for 2013 This trial version is a fully functional, unlimited version of the IBM Security AppScan Standard product. It works along with Selenium IDE. The trial version has an only restriction that scanning is limited to one site.
ZAP (Zed Attack Proxy): An open source, cross platform tool provided by OWASP. Ideal for developers and functional testers who are new to penetration testing. ZAP helps in finding vulnerabilities in web applications by performing penetration testing. It works with Selenium. Selenium runs test cases automatically and ZAP intercepts them in order to find out vulnerability if any in that web page of website.
Intra-procedural Source Code Analysis (PREfast): Microsofts tool PREfast prevents developers from checking in code with certain classes of bugs. PREfast uses a mix of syntactic and semantic analysis to look for these bugs. PREfast is currently being used as a check-in requirement for most Microsoft product groups.[25]
It is a static code analysis tool. PREfast uses your existing build structure and works by intercepting your native compiler. It is recommended that the build be divided into small (10 MB or less) sections, and run PREfast on each section. PREfast displays a log of the code defects encountered. Each line entry in the log shows a description of the type of defect, a PREfast warning number, the source location, and function where the defect occurred.[33]
Scalable, Path Sensitive Source Code Analysis (PREfix): Microsoft’s
PREfix is a tool used at compile-time to detect defects in C and C++ source code through symbolic evaluation techniques. PREfix simulates the execution of source code components along a selected set of program paths and queries the execution state in order to identify programming errors, all without requiring test cases or instrumentation. Detects errors in C/C++ code through null pointer, memory allocation, uninitialized value, resource state, library usage. It does Path Sensitive analysis by performing bottom up traversal of the graph that has been called[34].
HP WebInspect : HP WebInspect is an automated and configurable web application security and penetration testing tool that mimics real-world hacking techniques and attacks. [36] HP WebInspect Real-Time enables you to observe web
applications at code level while they are being attacked and then it uses this information to inform and guide the dynamic analysis while a test is still underway. It has the ability to see inside a running application, observing
and recording information about requests made to the application, the code the application executes and the values of variables inside the running program[35].
Cenzic HAilstorm (Web,mobile and Cloud Application Scanner): Cenzic provides the leading application security intelligence platform to continuously assess Cloud, Mobile and Web applications. Cenzic provides a development environment for applications security architects to build policies that define how applications are to be tested.
Cenzic’s Hailstorm detects vulnerabilities in both Web and AJAX-enabled sites, using differencing techniques rather than signatures. Hailstorm’s AJAX-specific features are aimed at catching authorization and authentication holes[37].
NT Objectives NTO Spider: NTOSpider On-Demand provides a responsive and scalable security solution with the ability to scan and report on thousands of application vulnerabilities within a short duration of time based on your immediate needs. NTOSpider can be customized to meet the business requirements. This can include a dedicated scanner appliance in your environment that can access and test your critical internal applications[38].
VIII. ADVANTAGES AND DISADVANTAGES OF VARIOUS TYPES OF SECURITY TESTING:
A. Black Box Testing:
Advantages:
1. Can quickly test the attack surface.
2. Gives clue about the commonly known vulnerabilities which crept in the software and gives hint.
Disadvantages:
1. Results tell you what vulnerabilities exist, not how or why they exist.
2. May be additional endpoints with vulnerabilities.
3. Provides less input for remediation.
Challenges in Automation for Black Box testing for security vulnerabilities?
The web application vulnerability scanners are the automated tools that check the application for security vulnerabilities. Without the need of the source code, these tools perform Black-Box testing, finding out the most common web vulnerabilities. Security threats to web applications like banking, FBI etc. are extremely crucial and the data stealing cannot be tolerated. There are security organizations that recognize the most common security threats among the web applications, but if proper care is not taken, developing a vulnerable website is quite a common practice.
If web application security testing is not automated using a proven automated web application security scanner that can test for thousands of potential security flaws, some if not all of the serious web application vulnerabilities can be overlooked.
For example, imagine the ERP(Enterprise Resource Planning) system has 200 entry points that need to be checked against 100 different web application vulnerability variants. That means that the penetration tester needs to launch at least 20,000 security tests. If every test had to take just 5 minutes to complete, it would take a web security specialist around 208 business days to complete a proper web application security audit of an ERP system.[18]
Most of the attacks involve some kinds of carefully crafted database queries, which if not checked properly, may populate the database with garbage value, or might even affect the structure of the tables.
We intend to critically find promise and effectiveness of automated tools and the limitations of those tools.
Broadly security testing of Web applications determines some of these following issues:
‘ Whether a confidential data stays confidential (i.e. it is not exposed to individuals/ entities for which it is not meant).
‘ Whether Users can perform only those tasks that they are authorized to do.
B. White Box Testing:
Advantages:
‘ Completeness and effectiveness
‘ Employing Static analysis tools drastically reduces the number of security vulnerabilities that penetrate into application only to be discovered in the Black Box testing phase.
‘ Accuracy
‘ Fast (for competent reviewers)
Disadvantages:
‘ Requires highly skilled security developers
‘ Can miss issues in compiled libraries
‘ Cannot detect run-time errors easily
‘ The source code actually deployed might differ from the one being analyzed
According to the research firm Gartner[27] ‘next-generation modern web and mobile applications require a combination of SAST and DAST techniques. Interactive application security testing[IAST] approaches have emerged that combine static and dynamic techniques to improve testing.” Figure 2 shows the comparative analysis of the various security testing tools plotted against Ability to execute versus completeness of vision. According to Gartner, IBM’s tool is the best among all.
Fig. 3: Gartner [July 2013]
C. Penetration Testing:
Advantages:
‘ Can be fast (and therefore cheap)
‘ Tests the code that is actually being exposed
Disadvantages:
‘ Front impact testing only!
‘ Penetration Testing is a late attempt to tackle security at the end of the development cycle. So penetration testing sometimes uncovers problems too late, at a point when both time and budget severely constrain the options for remedy and is prohibitively expensive.[21]
‘ Only skilled and experienced testers can successfully perform penetration testing.
‘ Penetration Testing can only identify a small representative sample of all possible security risks in a system. If a software development organisation focuses solely on a small (and limited) list of issues, it ends up mitigating only a subset of the security risks present (and possibly not even those that present the greater risk).[22]
Lack of developer training, misapplication of standard coding practices(like buffer overflow), poor choice of programming languages and their libraries are an important cause of failure of penetration testing.[23]
D. Model Based Testing:
E. Security Testing:
‘ Only find technical flaws in applications.
‘ Logical flaws cannot be tested through Security Testing
‘ Do not capture security state of application.
‘ Threat modeling helps here.
‘ Can require sophisticated users to drive them correctly.
‘ Can provide a false sense of security
Organizations that fail to integrate security throughout the development process often find that their software suffers from systematic faults both at the design level and the implementation(in other words, the system has both security flaws and security bugs).[21]
IX. CONCLUSION
There’s no perfect way to test for web security vulnerabilities. However, going about testing manually and relying on teams expertise alone is unaffordable because it might cost business a lot of money besides that some web application vulnerabilities might even go undetected. Integrating security automation into all stages of web applications software development life cycle is the need of the time. Thus multiple automated web application security scanner seems to be the possible solution. During from the unit testing phase writing security use cases and misuse test cases, and automating the testing using tools like Selenium IDE seems to be a good option.
False Positives are difficult to weed through and often require significant security experience to figure out which warning to be needs fixing. Much Vulnerability occurs only in specific environments and are discovered only when the application is run in that environment. Static tools cannot detect the vulnerabilities that can happen during the run time. To catch those vulnerabilities, Dynamic analysis tools needs to be used. Also static analysis tools cannot foresee the insecure deployment setting and thus miss the significant attack surface.
One of the limitations of traditional dynamic analysis is that the scanner is completely unaware of the inner workings of the application under test. Thus the possible solution is IAST. Tools like IBM (GlassBox) and HP (SecurityScope) have an enhancement for their DAST product that is sort of IAST-like.
Also it is discovered that There are issues where automation will not help and manual testing has to be done.
Model-based security testing (MBST) is a relatively new field and especially dedicated to the systematic and efficient specification and documentation of security test objectives, security test cases and test suites, as well as to their automated or semi-automated generation.
Almost all the tools that we have encountered so far runs only on windows. Unix based testing tools are still not prevalent. The learning curve remained low because of lots of system dependent tools and the lack of a common protocol among the different vendors.
ACKNOWLEDGMENT
Thanks are due to Dr. Joseph Chao, who introduced us to the Software Testing concepts and Automation tools. We would also like to acknowledge The Secured Software Engineering (CS 6310) course taught by Dr. Ray Kresman in Fall 2013 semester. We are thankful to The Bowling Green State University to give us the opportunity to collaborate and share our ideas among professors and peers.
REFERENCES
[1] X. Jia and H. Liu. Rigorous and automatic testing of web applications. Presented at Proceedings of the 6th IASTED International Conference on Software Engineering and Applications (SEA 2002). 2002, .
In this paper, Jia et. al. proposed an approach of automatic testing of web applications using formal specifications. A tool is proposed which accepts formal specifications in XML syntax as input, automatically generates test cases, executes the test cases and validates the test results.
[2] J. Bau, E. Bursztein, D. Gupta and J. Mitchell. State of the art: Automated black-box web application vulnerability testing. Presented at Security and Privacy (SP), 2010 IEEE Symposium on. 2010, .
In this paper, authors have tested 8 leading Black Box vulnerability scanning tools. They used a custom web application which has known vulnerabilities. They also tested previous versions of widely used web applications namely Drupal, WordPress and phpBB2. The aim of their studies is to determine the effectiveness of these tools in detecting vulnerabilities
[3] C. Bezemer, A. Mesbah and A. van Deursen. Automated security testing of web widget interactions. Presented at Proceedings of the the 7th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering. 2009, .
The paper presents automated security testing of AJAX web widgets. AJAX pages are composed of independent user interface components called widgets which are embedded in html pages.For example iGoogle. The authors proposed a dynamic analysis approach for automatically detecting security vulnerabilities to detect inter-widget interaction violations.
[4] S. Turpe. Security testing: Turning practice into theory. Presented at Software Testing Verification and Validation Workshop, 2008. ICSTW’08. IEEE International Conference on. 2008, .
This position paper proposes a research agenda for the field of security testing. This paper is an attempt to identify the questions that research failed to answer so far. Three categories of research problems were proposed: theory of vulnerabilities, theory of security testing, and tools and techniques.
[5] S. Artzi, J. Dolby, S. H. Jensen, A. Moller and F. Tip. A framework for automated testing of javascript web applications. Presented at Software Engineering (ICSE), 2011 33rd International Conference on. 2011, .
The authors proposed a framework for feedback-directed automated test generation for JavaScript. The generated tests can be used for detecting HTML validity problems and other programming errors.
[6] M. Buchler, J. Oudinet and A. Pretschner. Semi-automatic security testing of web applications from a secure model. Presented at Software Security and Reliability (SERE), 2012 IEEE Sixth International Conference on. 2012, .
In this paper, authors proposed a methodology for semi automatically testing web applications,starting from a secure model. A prototype has been implemented and evaluated on Webgoat, an insecure web application maintained by OWASP. It successfully reproduced Role-Based Access Control (RBAC) and Cross-Site Scripting (XSS) attacks.
[7] I. Schieferdecker, J. Grossmann and M. Schneider. Model-based security testing. ArXiv Preprint arXiv:1202.6118 2012.
Model-based security testing (MBST) is a relatively new field and especially dedicated to the systematic and efficient specification and documentation of security test objectives, security test cases and test suites, as well as to their automated or semi-automated generation. This paper provides a survey on MBST techniques security functional testing, model-based fuzzing, risk- and threat-oriented testing, and the usage of security test patterns.
[8] S. Artzi, A. Kiezun, J. Dolby, F. Tip, D. Dig, A. Paradkar, and M. Ernst, ‘Finding bugs in web applications using dynamic test generation and explicit-state model checking,’ TSE , vol. 36, no. 4, pp. 474’494, 2010.
In this paper Artzi et. al presented a dynamic test generation technique for the domain of dynamic Web applications. The technique generates tests automatically, runs the tests capturing logical constraints on inputs, and minimizes the conditions on the inputs to failing tests so that the resulting bug reports are small and useful in finding and fixing the faults.
[9] A. Doup??e, M. Cova, and G. Vigna, ‘Why johnny can’t pentest: an analysis of black-box web vulnerability scanners,’ in DIMVA , 2010, pp. 111’131.
This paper presented an analysis of 11 web vulnerability scanners and concluded that there is no strong correlation between cost of the scanner and functionality provided as some of the free or very cost-effective scanners performed as well as scanners that cost thousands of dollars.
[10] F. Y. Gu Tian-yang, Shi Yin-sheng & Yuan (2010): Research on Software Security Testing. World Academy of Science Engineering and Technology 69 2010.
[11] Mark Blackburn, Robert Busser & Aaron Nauman (2002): Model-based approach to security test automation .In: International Software Quality Week
The authors of this paper summarized the results of applying a model-based approach to automate security functional testing. The approach involves developing models of security function specifications (SFS) as the basis for automatic test vector and test driver generation. J. K. Author, ‘Title of chapter in the book,’ in Title of His Published Book, xth ed. City of Publisher, Country if not
[12] David Basin, J??urgen Doser & Torsten Lodderstedt (2006): Model driven security: From UML models to access control infrastructures. ACM Trans. Softw. Eng. Methodol. 15, pp. 39’91.
In this paper, Basin et. al. proposed a technique that combines different UML modelling languages with security modelling language in order to perform access control. So, it helps in modelling UML designs in order to perform security at the design phase.
[13] Linzhang Wang, Eric Wong & Dianxiang Xu (2007): A Threat Model Driven Approach for Security Testing.In: Proceedings of the Third International Workshop on Software Engineering for Secure Systems, SESS ’07, IEEE Computer Society, Washington, DC, USA, pp. 10′.
In this paper, the authors perform security testing related to threats in UML designing of sequence diagrams, so, that event sequences that are prone to threat should not occur during the system execution.
[14] Martin Weiglhofer, Bernhard K. Aichernig & Franz Wotawa (2009): Fault-Based Conformance Testing in Practice. Int. J. Software and Informatics 3(2-3), pp. 375’411.
In this paper, authors’ focuses on fault based conformance testing means, the software that has defect by design. Weiglhofer et. al. give example of Voice over-IP systems that are built in collaboration with different vendors. So, there are high chances of defect in either hardware or software that leads to security vulnerability for sure and hence this paper provides case study and tools for securing this point.
[15] Chandramouli R., Methodology for Automated Security Testing’, NIST Request for Proposal, Nov 1999.
This paper focus on which method is good for the automation of security testing, so that after writing test cases in a particular environment, software should not be vulnerable to security.
[16] NIST, Risk management guide for information technology systems – http://csrc.nist.gov/publications/nistpubs/800-30/sp800-30.pdf
[17] Gary McGraw, Beyond the Badness-ometer – http://www.ddj.com/security/189500001
[18] OWASP Testing Guide, v3,
https://www.owasp.org/images/5/56/OWASP_Testing_Guide_v3.pdf
[19] White Box Security Testing using code Scanning’ [Online] available at http://www.aspesdlc.com/offers/pickups_0386572/white_box_scanning_dobbs.pdf [Mar. 21, 2014]
The above link is a white paper provided to ASPE by Security Innovation. Only few concepts and informations is extracted because of the fear of it being a infomercial.
[20] ‘Secure Development Lifecycle’, http://www.microsoft.com/security/sdl/process/design.aspx [Online],
[Mar. 22, 2014]
[21] G. McGraw. Software security. Security & Privacy, IEEE 2(2), pp. 80-83. 2004.
[22] D. Verdon and G. McGraw. Risk analysis in software design. Security & Privacy, IEEE 2(4), pp. 79-84. 2004.
[23] B. Chess and G. McGraw. Static analysis for security. IEEE Security & Privacy 2(6), pp. 76-79. 2004.
[24] ‘Automated security testing with IBM Security AppScan Enterprise 8.7 and Selenium IDE’, Internet: http://www.ibm.com/developerworks/security/library/se-automated/index.html ,[Online], [Mar. 24, 2014]
This link gives step by step description of how IBM tool uses selenium for automating the security testing for web applications.
[25] ‘Program Analysis Projects’, Internet:http://www.microsoft.com/windows/cse/pa_projects.mspx ,[Online], [Mar. 24, 2014]
This website gives brief overview about all the tools that are used in error detection and correction, web vulnerability scanning, as provided by microsoft
[26] ‘lint’, Internet:http://developer.android.com/tools/help/lint.html ,[Online], [Mar. 25, 2014]
This website describes bout the static code analysis tool, ‘lint’, that is used in providing the mobile security from android applications.
[27] Gartner Magic Quadrant for Application Security Testing’, Internet: http://securityintelligence.com/gartner-magic-quadrant-for-application-security-testing-2013/ ,[Online], [Mar. 25, 2014]
[28] Alexander, I. Misuse Cases: Use Cases with Hostile Intent.IEEE Software 20, 1 (January/February 2003): 58’66.
[29] ‘Web Application Attack and Audit Framework(w3af)’.Internet: http://w3af.sourceforge.net/ ,[Online], [Mar. 26, 2014]
This website gives description of w3af, an open source web vulnerability black box scanner.
[30] ‘Power Fuzzer’, Internet: http://www.powerfuzzer.com/ ,[Online], [Mar. 26, 2014]
This website gives description of power fuzzer, an open source web vulnerability black box scanner.
[31] ‘Top 10 2013-Top 10’, Internet: https://www.owasp.org/index.php/Top_10_2013-Top_10 ,[Online], [Mar. 26, 2014]
This website describes the top ten web vulnerabilities in the year 2013 as provided by OWASP.
[32] ‘Sanitization (classified information)’, Internet: http://en.wikipedia.org/wiki/Sanitization_%28classified_information%29 ,[Online], [Mar. 27, 2014]
This website describes the basic definition and techniques for sanitizing the data while sending through link between sender and receiver.
[33] ‘PREfast Overview (Windows CE 5.0)’, Internet: http://msdn.microsoft.com/en-us/library/aa448765.aspx ,[Online], [Mar. 27, 2014]
This document provides brief overview of microsoft’s PREfast vulnerability scanner.
[34] ‘Program Correctness Tools’, Internet: http://drona.csa.iisc.ernet.in/~deepakd/pav/Lecture2.pdf ,[Online], [Mar. 27, 2014]
This document provides advantages, disadvantages and description of microsoft’s PREfix and PREfast tools for threat and risk scanning in web applications.
[35] ‘HP WEBINSPECT REAL-TIME’, Internet: http://h71028.www7.hp.com/enterprise/downloads/software/ESP-DTS003-031412-04.pdf ,[Online], [Mar. 27, 2014]
This websites gives detailed working procedure of HP WebInspect black box vulnerability scanner.
[36] ‘WebInspect’, Internet: http://www8.hp.com/us/en/software-solutions/software.html?compURI=1341991 ,[Online], [Mar. 28, 2014]
This websites gives brief overview and features of HP WebInspect black box vulnerability scanner.
[37] ‘Cenzic’s Hailstorm: Augment Your Web Security Toolbox’ , Internet: http://www.informationweek.com/cenzics-hailstorm-augment-your-web-security-toolbox/d/d-id/1044683? ,[Online], [Mar. 28, 2014]
This news blog talks about the hailstorm vulnerability scanner working i.e. how this scanner suspects the vulnerabilities in a website.
[38] ‘NTO Spider On-Demand’, Internet: http://www.ntobjectives.com/services/ondemand-saas-security-scanning/ ,[Online], [Mar. 29, 2014]
This websites gives brief overview and key concepts of NTO spider black box vulnerability scanner.
[39] ‘Acunetix Web Vulnerability Scanner’, Internet: http://www.acunetix.com/vulnerability-scanner/wvsmanual.pdf ,[Online], [Mar. 29, 2014]
This document talks about how acunetix web vulnerability scanner performs scanning of websites. This scanner tells about the famous vulnerabilities in web pages.
[40] https://code.google.com/p/zaproxy/issues/detail?id=557
GLOSSARY
Misuse cases: A security misuse case is a variation on a use case and is used to describe a scenario from the point of view of the attacker.
Data Sanitization: According to Wikipedia [32], When the sensitive information is removed from the document, so that, it can be transmitted over web is called as Data Sanitization..