ABSTRACT
This paper presents a systematic review on Model Based Testing (MBT) approaches for Aspect-Oriented Software Systems(AOSSs).A selection criterion was used to identify initially about 94 papers and then focus on only 21 papers .Detailed analysis of these papers shows that MBT approaches have been widely used for testing AOSSs. This paper proposed various evaluation metrics, defines the methodology and presents the comparative results.
Keywords
Aspect-Oriented Programming; Aspect-Oriented Testing; Model Based Testing; Coverage Criteria; Test- Cases.
1. INTRODUCTION
1.1 Aspect-Oriented Programming (AOP)
Paradigm
Aspect-Oriented Programming (AOP) is a latest programming paradigm in the field software engineering [1, 2]. Existing programming paradigms are not capable to modularize all concern which are required for developing complex software. It provides an effective mechanism that explicitly capture crosscutting concern into modular units called aspects.
1.3 Model-based Testing (MBT)
Model based testing (MBT) relies on explicit behavior of a system. Moreover a model depicts a particular aspect or view of a system. In software testing, there are two types of models such as structural models and behavioral models whereas structural model depicts the structural properties of the system and behavioral model represents the operational properties of a system. Nowadays, model based testing (MBT) getting popularity in the field of software testing because MBT can be used before and after, developing of the software system.
Motivation of this paper is to analyze the various model based testing (MBT) approaches of aspect-oriented software system (AOSS) on the basis of systematic proposed criteria as shown in figure 1. This will help to give the answers to various research questions which are faced by the various researchers who are involved in testing an aspect oriented software system (AOSS) using MBT approach.
This paper is divided into 4 different sections. Section 1 introduces the concept of Aspect-Oriented Programming Software testing and model based testing, section 2 presents the systematic review methodology and proposed evaluation criteria for selected relevant papers .Section 3 presents the various model-based testing approaches AOP, its relevant papers. Section 4 covers the conclusion and future scope of model based testing for AOSSs.
1.2 Software Testing
Software testing is an umbrella activity to all phases of software development life cycle (SDLC) which aim is to detect the maximum errors in software and provide quality software to customers. But effective software testing process relies on only quality (good) test cases, not on the quantity of test cases. Various testing techniques are being proposed by researchers to test aspect-oriented software systems. These techniques are broadly divided into three categories: code-based testing, model-based testing and fault-based testing. In this paper, our main focus on model based testing in AOSSs.
2. SYSTEMATIC REVIEW: SURVEY PLANNING AND METHODLOGY
A systematic review is a process to identify, evaluate and interpret the pertinent research about a particular research question[24].Planned steps in this work are such as :goal ,research question, source selection, search string, paper selection criteria ,categorize all relevant papers, apply grading to each relevant paper and complete the review process as shown in figure 1.
Goal: To categorize the MBT approaches from available literature.
Research question: What are the published MBT approaches for testing aspect-oriented software and what are their characteristics?
Source selection: ACM digital library, IEEE explore, Open Access Journals, Science Direct, Springer, World scientific, various journals and conference papers etc.
Search string: (‘model based testing techniques for aspect oriented software system or aspect oriented program’) or (‘testing AOP using UML based modeling’)or(‘testing AOP using activity diagram’)or(‘testing AOP using collaboration diagram ‘)or(‘Testing AOP using sequence diagram’)or(‘testing AOP using use case diagram’)or(‘testing AOP using state chart diagram’)or (‘state based testing for AOP’)or(‘modeling approaches for testing AOP’)
Paper selection criteria: 94 research papers were collected from which 73 papers were excluded as they were not found suitable for this systematic review. From the remaining 21 papers, currently 16 papers have been analyzed which were based on model based testing approaches for aspect-oriented software system (AOSSs). The list of the 89 identified papers are available in [4].
Categorization of selected papers: All selected papers are categorized into four different model based testing approaches as shown in figure 2.
Fig.2 Distribution Over the years of research papers according to characterization
Define Search String
Start Searching
Include only relevant papers
Categorize all relevant papers into UML based techniques
Assign Grade to each paper on the basis of metric’s parameters
Complete review process
Fig 1: A Systematic Review Process
Apply grading to each selected paper based on proposed evaluation metrics as discussed in subsection 2.1 and comparative study of each MBT techniques in AOSS as shown in Table 10.
2.1. Metrics for Evaluation
In order to perform a systematic review, several metrics have been proposed to evaluate the existing model based testing techniques for AOP. After widely studied the available literature, we have identified and proposed eight metrics on which quality of testing depends. Assign grade to each parameter which depends on the level of the work done by the author(s) in that particular parameter. The metrics for evaluation are as follows:
2.1.1 Testing techniques and algorithm
It describes how the testing techniques and algorithms are designed and defined.
Table 1. Testing Techniques and Algorithm
Parameters Assigned Grade
Technique and algorithm C
informally defined
Basic concept defined B
Technique and algorithm A
formally defined
2.1.2. Testing technique Implementation
It describes project’s size which was used for implementation of the proposed testing technique.
Table 2. Testing Technique implementation
Parameters Assigned Grade
Implemented with a small C
program
Implemented with a medium B
size projects
Implemented with live A
projects
2.1.3 Test case generation
It describes how researchers generate the test cases; manually or using an automated tool in their projects.
Table 3. Test case generation
Parameters Assigned Grade
Manually test case B
generation
Automated test case A
generation
2.1.4 Level of Testing
It describes the level of abstraction which is used to check the software behavior in model based testing for AOP.
Table 4. Levels of Testing
Parameters Assigned Grade
Mutation Testing E
Regression Testing D
Unit Testing C
Integration Testing B
System Testing A
2.1.5 Supporting tools
It describes how the values were evaluated; whether an automated tool is used or manually through some programs.
Ta
ble 5. Supporting Tools
Parameters Assigned Grade
Uses no tools C
Manually program was B
developed
Uses automated tools A
2.1.6 Input Data-Set required
It describes the size of input data set used for evaluating the results.
Table 6. Input Data-Set
Parameters Assigned Grade
Used Small size Input C
Data-Set
Used medium size input B
data set
Used large size input data A
set
2.1.7 Test coverage criteria
It describes various coverage criterion which have been used for generating test cases from software model.
Table 7. Test coverage criteria
Parameters Assigned Grade
No coverage criteria L
Conditional Coverage K
Branch Coverage criteria J
Control Flow Coverage I
Data Flow Coverage H
Use case Coverage G
State Coverage F
Integration Coverage E
Transition Coverage D
Sequence Coverage C
Advice Execution B
Coverage
Multi-aspect Integration A
2.1.8 Level of automation
It defines whether a particular approach is fully automated or partially automated.
Table 8. Level of Automation
Parameters Assigned Grade
No automation C
Partially B
Automated/Needed some
manual Effort
Fully Automated A
3. MODEL-BASED TESTING APPROACHES
Model-based testing approaches are characterize into four different categories based on the behavior models such as state-based testing, activity diagram based testing ,collaboration diagram based testing ,sequence/class/aspect / use case diagram based testing as shown in figure 2. Test cases are derive from aspect-oriented models which consider the behavior of programs while aspects are interacting with classes. Details of various model based testing approaches are as given in below subsections.
3.1 STATE-BASED TESTING
State-based modeling approach comes under the behavioral constructs of UML modeling .This approach is used for testing AOP which tests the state behavior errors during the interaction between the classes and aspects. This can be performed manually or by using some available automated tools.
Chien.H.L.et al. [5] proposed a state-based approach for testing AOP with a test model which abstract the state behavior of AOP while interacting the base module with aspects. Moreover, this proposed test model helps to generate the test cases which uncovers the potential state behavior errors those exist in AOP program. This approach can only apply to test a single unit of AOP because here consider only one base class at a time. Moreover, this approach only addresses the state based behavior between the base class and aspects introduced by method call join points but does not work while accessing variable join-point and precedence of advice .The limitation of proposed state-based test model is that it requires manual effort and there is no specific coverage criteria is used.
D. Xu et al. [4] proposed a state based incremental testing approach to test whether or not AOP and base class conforms to their expected behavior. This approach composes the state model of base class and aspects by an explicit weaving mechanism and then generate abstract test cases of AOP and corresponding base program, from this state model. Moreover, this paper also focuses on reusing test cases by following four rules which were defined in this paper and this approach significantly reduces the testing cost and also limits the criteria for prioritizing test cases to be reused.The limitation of proposed approach is that 1) proposed approach is tested by two small programs, 2) manual approach is being used and 3) only single branch-coverage criteria is being used.
D. Xu et al. [6] proposed an aspectual State Model (ASM).This model is directly built upon an object-oriented programs to design test cases and extended from FREE (flattened regular expression) state model in UML. This model presents aspect-oriented software in better way and provides a more effective test aspect implementation. Moreover, the proposed model supports a large scale project and conditional coverage criteria is being used.
D. Xu et al. [7] proposed a Model-based Aspect Checking and Testing (MACT) framework which is used for testing whether aspect-oriented program conform to their state model. Additionally, this framework helps to generate automated test cases by using two testing strategies such as: Structural’oriented testing and Property–oriented testing. These strategies are used to test aspect-oriented programs based on AOPs’s state model. The proposed strategy is applied on a large scale project and results shows that both strategies are powerful and coverage criterion are used such as state coverage,transition coverage etc.
M. Badri et al. [11] proposed an approach which integrates aspect-classes in incremental way .This approach works in two phases: 1) generates testing sequences 2) verifies the execution of the selected sequences. Additionally, this paper also proposed an automatic tool which supports the testing sequence generation as well as the verification process. This paper reveals three faults such as specification based faults, pre-condition based faults and exception based faults and also the proposed approach allows computing the various testing coverage criterion such as simple integration criterion and multi-aspects integration criterion according to the tested sequences.
M. Badri et al. [12] proposed an approach for unit testing of AOPs which is based on UML state charts. This approach is based on dynamic behavior and focuses on integrating one or more aspects to a class. The test sequences are generated through the UML state charts of an aspect-oriented program.
3.2 COLLABORATION DIAGRAM BASED TESTING
P. Massicotte et al. [8] proposed a technique of integrating one or more aspects which is based on collaboration diagram .The proposed strategy is divided into two phases:1)generating basic test sequences from collaboration diagram and verifies whether they are working correctly or not and in the second phase 2)integrating aspects incrementally to the collaboration diagram and verifies the integrated generated test sequences on the basis of certain testing criteria such as sequence coverage, transition coverage, simple integration coverage criteria and multi-aspects integration criteria.
P. Massicotte et al. [9] proposed a new aspects-classes integration testing strategy which is based on collaboration diagram in which focus on the integration of one or more aspects .This approach consists of two phases in which, first phase, generated test sequences based on interaction between collaboration and weaved aspects and in the second phase, verifies the process of the execution of the generated test sequence. The testing coverage criterion are such as transition coverage, simple integration coverage, multi-aspects integration and sequence coverage criteria.
P. Massicotte et al [10] proposed a new integration testing technique which is based on UML collaboration diagram. The proposed technique work in two phases: first, it generates the basic test cases whereas XML is used to describe the collaboration diagram and aspects. Thereafter, aspects are integrated iteratively to the original sequence and tested during the verification process. In the second phase, verifies the execution with proposed algorithm. Various testing coverage criterion are applied such as transition coverage, sequence coverage and multi-aspects integration coverage criteria.
3.3 SEQUENCE DIAGRAM/CLASS /USE CASE BA
SED TESTING
W. Xu et al. [13] proposed a model based approach for generating test sequences while interacting aspects with classes using two steps such as: 1) generate goal-directed flow graph 2) transform flow graph into a flow tree. This approach consists of three UML diagrams such as class diagrams, aspect diagrams and sequence diagrams and two test coverage criterion such as polymorphic and branch coverage.
D. Xu et al. [14] proposed an aspect-oriented model which is developed with the help of sequence diagrams and class/aspects diagrams. This model is used for generating the test cases and also verifies these test cases whether they are pass or fail and also reveals various faults such as strong/weak point cuts , incorrect precedence of aspects etc. Three coverage criterion such as polymorphic coverage, loop coverage and condition coverage are applied on this model.
Chitra B. et al[15] proposed a fault model which identifies the faults from sequence diagram by following steps such as: developing use case and sequence diagram of AOP, convert sequence diagram into meta data format using XML , a parser program capture the interaction between aspects and classes, apply various faults as specified in this paper .The proposed model supports manual approach and a large project (Online shopping system).
D. Xu et al. [16] proposed an approach for generating system test requirements by using aspectual use case diagram in which aspect-oriented Petri Nets are used to capture the AOP essential features such as point cuts, advice etc. This approach works in three steps: 1. Transform the basic flow of program into sequential structured Petri Net and each action is represented as a transition, 2. Expand the corresponding transition if any variation occur in the alternate flow, 3. Transform each sub-flow as a separate Petri Net and finally connects the net with the corresponding transition to form a loop structure. After transform aspect-oriented use cases into Petri net, generates the test cases from this formal Petri net model which covers various test coverage criterion such as use-case coverage, transition coverage and state coverage.
3.4 ACTIVITY DIAGRAM BASED TESTING
Charnpreet et al. [17] proposed an approach to test AOP using activity diagram which consists of three main phases: 1. Building activity model of primary concern and generates test sequences for it. 2. Building aspect models and integrate them with primary model and also generate test sequences .3.
Verifies the execution of the selected sequence. This approach is also capable of reveal four aspect specific faults such as incorrect advice type, weak or strong point cut strength and incorrect aspect precedence.
Somayeh et al. [18] proposed an approach of integration of crosscutting concern to primary concern and generate the test sequence based on integration between aspect and primary model using UML activity diagram. Moreover, the proposed approach is capable of reveals some aspect-specific faults.
4. CONCLUSION AND FUTURE WORK
In this paper, a systematic review is presented about model based testing (MBT) for aspect-oriented software systems (AOSSs) from the available literature and then classified them on the basis of UML diagrams based testing. In order to analyze various papers ,several metrics have been proposed and evaluation has been performed on model based testing approaches by using these metrics .This paper will improve the understanding of researchers about model based testing(MBT) approaches for aspect-oriented software system(AOSS) who are interested, to do the work, in this field.Our future work is to investigate manual testing techniques for AOSSs and try to automate them.
REFERENCES
[1] G. A. Colyer and A. Clement, ‘Aspect-Oriented programming with AspectJ’, IBM Systems Journal, Volume 44, 2005, pp. 301-308.
[2] G. Kiczales, J. Lamping, C. V. Lopes, J. J. Hugunin, E. A. Hilsdale and C. Boyapati,Aspect-Oriented Programming’ ,US Patent No. 6467086, 2002.
[3] Abdul Azim, Abdul Ghani and Reza Meimandi Parizi,’Aspect-Oriented Program Testing: An annonated Bibliography’, in the Journal of Software, volume.8, June 2013.
[4] Dianxiang Xu and Weifeng Xu. ,’A state-based incremental testing of aspect-oriented programs’, in the proceeding of the 5th International Conference on Aspect- Oriented Software Development, March 20-24,pg. 180-189,Bonn,Germany,2006.
[5] Chien.H.L and Chauan.W.C,’A State ‘ Based Testing Approach for Aspect-Oriented Programming”,in the Journal of Information Science and Engineering ,volume 24,pg. 11-31,2008.
[6] Dianxiang.Xu,Weifeng.Xu and K.Nygard ,’ A state-based approach to testing aspect-oriented programs ‘ in the proceeding of the 17th International Conference on Software Engineering and Knowledge Engineering ,July 14-16,2005.
[7] D. Xu, O. El-Ariss, W .Xu and L. Wang., ‘Testing aspect-oriented programs with finite machines’, in the Journal of Software Testing, Verification and Reliability,sept.2010 in Wiley Online Library.
[8] Philippe Massicotte, Linda Badri, and Mourad Badri.
‘Generating aspects classes integration testing sequences: a collaboration diagram based strategy’, in the Proceedings of
the 3rd ACIS International Conference on Software
ngineering Research, Management and Applications
(SERA’05), pp. 30-37, August 2005.
[9] Philippe Massicotte, Linda Badri, and Mourad Badri. ‘Aspects-classes integration testing strategy: an incremental approach’, in the 2nd International Workshop on Rapid
Integration of Software Engineering Techniques (RISE 2005), LNCS 3943, 2005, pp. 158-173.
[10]Philippe Massicotte, Linda Badri, and Mourad Badri,’Towards a tool supporting integration testing of aspect-oriented programs’,in the Journal of Object Technology,vol.6,No.1,pp.67-89,Feb 2007.
[11]Mourad Badri, Linda Badri and Maxime Bourque-Fortin,’Automated state-based unit testing for aspect-oriented programs: A supporting framework, ‘in the Journal of Object Technology, Vol.8, No.3, May-June 2009, PP.121-146.
P. Massicotte ,L. Badri and M.Badri ,’ Towards a tool supporting integration testing of aspect-oriented programs’, in the Journal of Object Technology, Vol. 6, No. 1, pp. 67-89, Feb. 2007. Mourad Badri, Linda Badri and Maxime Bourque-Fortin.,’Automated state-based unit testing for aspect-oriented programs: A supporting framework’, in the. Journal of Object
[12] Mourad Badri, Linda Badri, and Maxime Bourque-Fortin.,’ Generating unit test sequences for aspect-oriented programs: towards a formal approach using UML state diagrams’, In Proceedings of the 3rd International Conference on Information and Communication Technology, Cairo, 5-6 December 2005, pp. 237-253.
[13] Weifeng Xu and Dianxiang Xu., ‘A model-based approach to test generation for aspect-oriented programs’, AOSD2005 Workshop on Testing Aspect-Oriented Programs, Chicago, USA, March 2005.
[14] D. Xu, W. Xu, and W. E. Wong.,’ Testing aspect-oriented programs with UML design models’, in International Journal of Software Engineering and Knowledge Engineering, Vol. 18, No. 3, pp. 413-437, 2008.
[15] Chitra Babu and Harshini Ramnath Krishnan,’ Fault model and test-case generation for the composition of aspects’, SIGSOFT Software Engineering Notes, Vol. 34, No. 1, pp1-6, 2009.
[16] Dianxiang Xu and Xudong He.’ Generation of test requirements from aspectual use case’, in Proceedings of the 3rd Workshop on Testing Aspect-Oriented Programs, pp. 17-22, March 12-13, 2007, Vancouver, Br
itish Columbia, Canada.
[17] C.Kaur and S.Garg,’Testing Aspect-Oriented Software Using UML Activity Diagram’,in Internationational Journal of Engineering Research and Technology,ISSN:2278-0181,vol 1 Issue 3,May-2012.
[18] Madadpour, S.H.M.Hosseinabadi, V.Abdelzad,’Testing Aspect-Oriented Programs with UML Activity Diagrams’, in International Journal of Computer Applications, vol 33-No.8, Nov.2011.
[19] M.Shirol and R.Kumar ,’ UML Behavioural Based Test case Generation: A Survey’, in ACM SIGSOFT Software Engineering Notes, July 2013, Volume 38,Number 4.
[20]Manuel wimmer,Andrea Schauerhuber and Gerti Kappel,’A Survey on UML-Based Aspect-Oriented Design
Modeling’,in the ACM Computing
Survey, volume.43, No.4, October 2011.
[21]Zhanqi Cui,Linzhang Wang and Xuandong Li,’Modeling and Integrating Aspects with Uml Activity Diagrams’, in the SAC’09,march 2009.
[22] D.Mouheb,D.Alhadidi,M.Nouh,M.Debbabi,L.Wang and M.Pourzandi,’Aspect-weaving in UML Activity Diagram:A Semantic and Algorithmic Framework’,in Springer-Verlag Berlin Heidelberg 2012.
[23] Deepika Shukla,’Analyzing the comprehensibility of Aspect-Oriented Modeling and Design of Software System’,in International Journal of Computer Applications,Vol.95,June 2014.
[24] B.Kitchchenham,’Procedures for performing systematic review,’Joint Technical ReportSoftwareEngineering Group,Department of Computer Science Keele University,UK and National ICT Australia Ltd.,2004.
..