Test circumstance prioritization and regression test selection

1. introduction

The need for software screening process and its influences on software quality cannot be taken too softly. Software screening is a simple element of software quality assurance and presents an assessment of specification, design and coding. The higher visibility of software systems and the price associated with software inability are motivating factors for planning, through testing.

A variety of rules that act as testing goals are:

* Examining is a process of executing a program with the aim of finding problems.

* A good test case will have a good potential for finding an undiscovered error.

* An effective test case uncovers a fresh error.

Software maintenance is an activity, which includes enhancements, error corrections, marketing and deletion of obsolete functions. These adjustments in the program may cause the software to work incorrectly and could affect the other parts of the program, as developers maintain a software system, they regularly regression test it, hoping to find errors caused by their changes.

To do that, developers often create a short test suite, and then reuse it for regression tests. Regression testing can be an expensive maintenance process fond of validating customized software. Regression Test Selection techniques try to reduce the cost of regression trials by selecting lab tests from a program's existing test collection.

The simplest regression screening method, retest all, it is one of the conventional methods for regression testing in which all the checks in the existing test suite are re-run. This method, however, is very expensive and may require an undesirable amount of time to perform all tests in test collection. An alternative method, regression test selection, reruns only a subset of the initial test suite. In this technique rather than rerunning the complete test suite, we decide on a part of test collection to rerun if the cost of choosing the part of test collection is less than the expense of running the tests that regression test selection we can exclude. Naturally, this approach is unsatisfactory as well - test selection techniques can have significant costs, and can depart tests which could disclose faults, possibly lowering fault detection effectiveness. [1]

To decrease the time and cost during on tests process, another methodology, Test Cases Prioritization in a testing process can be advantageous for engineers and customers.

In Test Circumstance Prioritization techniques, test cases are executed in such a way, that maximum objective function like rate of mistake detection can be achieved.

In section 2 of the paper, we have described different types of Regression Test Selection techniques and we talked about various types of these types explain by various authors then getting into the facts of selective and prioritizing test situations for regression trials.

In this section, we also identify several techniques for prioritizing test conditions and we examine their ability to improve rate of problem detection, regarding to various authors.

In the next section, we in particular identify the Regression Test Selection techniques and Test Case Prioritization problems. Succeeding sections present our examination and conclusions

2. Regression testing

During a software development life routine, regression testing may begin in development stage of system after the detection and correction of errors in an application. Many modifications may occur during the maintenance phase where the software system is corrected, updated and fine-tuned.

There are three types of alterations, each arising from different types of maintenance. Relating to [2], corrective maintenance, commonly called "fixes", entails correcting software failures, performance failures, and execution failures to keep the system working properly. Adapting the system in response to changing data requirements or control environments constitutes adaptive maintenance. Finally, perfective maintenance addresses any improvements to increase the system handling efficiency or maintainability.

Based on of adjustment of specification authors identify two kind of regression testing, Intensifying regression testing entails a modified specification. In corrective regression tests, the specification does not change.

Table 1: Variations between Corrective and Progressive Regression Testing

According to [2], table 1 lists the major variations between corrective and intensifying regression assessment.

Regression testing is defined [3] as "the process of retesting the changed parts of the software and making certain no new problems have been introduced into previously examined code".

There are various regression testing techniques as given by various researchers are: (I) Retest all, (II) Regression Test Selection and (III) Test Case Prioritization. Retest-All Approach reuses all tests existing in test collection. It's very expensive as compared to other techniques. Within this statement our main give attention to Regression Test Selection and Test Circumstance Prioritization.

Let P be a procedure or program, let P' be considered a improved version of P, and let T be considered a test collection for P. A typical regression test proceeds as follows:

1. Select T' C T, a set of tests to execute on P'.

2. Test P' with T', building P"s correctness with respect to T'.

3. If required, create T", a set of new useful or structural testing for P'.

4. Test P' with T", building P"s correctness with respect to T".

5. Create T"', a fresh test collection and test background for P', from T, T', and T".

Although each one of these steps entails important problems, in this article we limit our focus on step 1 1 which involves the Regression Test Selection problem.

2. 1. REGRESSION TEST SELECTION

Regression Test Selection technique is less expensive as compare to retest all strategy. Regression Test Selection techniques reduce the expense of regression tests by choosing the subset of an existing test suite to use in retesting a revised program.

A variety of regression test selection techniques have been describing in the research literature. Authors [1] describe several families of techniques; we consider five most popular approaches often used in practice.

1) Minimization Techniques:

These techniques try to select minimal units of assessments from T that produce coverage of modified or affected helpings of P. One particular technique requires that every program statement put into or revised for P' be performed (if possible) by at least one test in T.

2) Safe Techniques:

These techniques select, under certain conditions, every test in T that can expose a number of faults in P'. One such technique selects every test in T that, when carried out on P, exercised at least one affirmation that is deleted from P, or at least one declaration that is new in or revised for P'.

3) Dataflow-Coverage-Based Techniques:

These techniques choose exams that exercise data interactions that contain been damaged by modifications. One such strategy selects every test in T that, when performed on P, exercised at least one explanation use pair that has been removed from P', or at least one definition-use match that is altered for P'.

4) Ad Hoc / Random Techniques:

When time constraints prohibit the use of your retest-all methodology, but no test selection tool can be found, developers often choose tests based on "hunches", or loose associations of lab tests with functionality. One simple technique arbitrarily selects a predetermined amount of exams from T.

5) Retest-All Technique:

This technique reuses all existing checks. To check P', the strategy "selects" all testing in T.

According to [3], Test Selection techniques are broadly labeled into three categories.

1) Coverage techniques:

These consider the test coverage conditions. These find coverable program parts that have been modified and select test conditions that focus on these parts.

2) Minimization techniques:

These act like coverage techniques except that they select minimum set of test circumstances.

3) Safe techniques:

These do not concentrate on conditions of coverage, in contrast they select those test cases that produce different productivity with a revised program as compared to its original version.

Regression test selection identifies the negative impact of improvements applied to software artifacts throughout their life pattern. In traditional techniques, code is modified directly, so code-based selective regression tests is used to identify negative impact of changes. In model-centric approaches, changes are first done to models, alternatively than to code. Subsequently, the negative impact to software quality should be recognized by means of selective model-based regression assessment. Thus far, most computerized model based evaluation approaches focus primarily on automating test technology, execution, and analysis, while support for model-based regression test selection is bound [4].

Code-based regression test selection techniques expect specification immutability, while model-based techniques choose abstract test circumstances predicated on model's improvements. Thus, in model founded Regression Test Selection techniques, the existing test collection can be categorized into pursuing three main types:

1) Reusable test instances:

Reusable test instances are test circumstances from the initial test suite that are not obsolete or re-testable. Hence, these test circumstances need not be re-executed.

2) Re-testable test instances:

Test circumstances are re-testable if they are non-obsolete (model-based) test case plus they traverse altered model elements.

3) Obsolete test circumstances:

Test conditions are obsolete if their input had been changed.

Regression Test Selection techniques may create new test conditions that test this program for areas which are not covered by the existing test situations.

Model based mostly Regression test collection selection that utilizes Unified Modeling Dialect (UML) structured Use Case Activity Diagrams (UCAD). The experience diagrams are commonly hired as a graphical representation of the behavioral activities of a software system. It signifies the functional tendencies of a given use circumstance. With behavior slicing we can built our activity diagram. This diagram gives us qualitative regression lab tests. Using behavior slicing each use case divided into a set of 'product of tendencies' where each unit of behavior signifies a consumer action. [5]

An activity diagram has generally six nodes:

1. Preliminary node

2. Individual Action node

3. System Processing node

4. System End result node

5. Condition node

6. Last node

2. 3. TEST Circumstance PRIORITIZATION

The main reason for test circumstance prioritization is to ranking test circumstances execution order to discover fault as soon as possible. There are two benefits helped bring by prioritization technique. First, it provides ways to find more pests under resource constraint condition and thus improves the exposed earlier; engineers have significantly more time to repair these insects [6].

Zengkai Ma and Jianjun Zhao [6] propose a fresh prioritization index called testing-importance of module (TIM), which combines two prioritization factors: problem proneness and importance of module. The main features of this prioritization strategy are twofold. First, the TIM value can be assessed by analyzing program structure (e. g. , call graph) by itself and it also can be assessed by incorporating program composition information and other available data (e. g. , source code changes). Therefore, this approach can be employed to not only regression assessment but also non-regression evaluation. Second, through analyzing program composition, we can create a mapping between fault severity and problem location. Those test cases covering important part of system will be assigned high concern and executed first.

As an outcome, the severe faults are unveiled earlier and the machine becomes reliable at fast rate. The main contributions of authors [6] are:

* They propose a new approach to evaluate the testing importance for modules in system by combining analysis of fault proneness and module importance.

* They create a test circumstance prioritization approach, which provides test instances priority end result by handling multiple information (e. g. , program structure information, source code changes) and can be employed to both new developed software tests and regression trials.

* They apply Apros, a tool for test circumstance prioritization based on the proposed strategy, and perform an experimental research on their procedure. The result suggests that Apros is a encouraging solution to increase the rate of severe faults detection.

Authors consider a sample system, which includes six modules: M1-M6 and there exist some call interactions between each module. A test suite includes six test conditions T1-T6 that includes the M1-M6 respectively. Some modules are reliant on each other. They have found fault proneness and fault severity by using TIM from this system. They conclude the prioritization result (T3, T6, T4, T2, T5, and T1) on the bases of analyzing structure of system. For calculating this final result they had developed some formulas and formula. [6]

They also do some experiment with two Java programs along JUnit test situations: xml-security and jtopas. They select three sequential variations of the two java programs and apply recently developed software trials and the regression screening. They perform some experiment for finding fault proneness and severe fault. They also introduce the importance of any module using weight truth.

Authors [7] explore value-driven method of prioritizing software system test with the objective of enhancing user-perceived software quality. Software testing is a challenging and expensive process. Research has shown that at least 50% of the total software cost is made up of examining activities. They conclude that, their way of prioritization of test situations is work effectively with regression and non-regression testing by analyzing this program structure.

They make a reach on prior TCP which have two goals: (1) to improve customer self-confidence on software quality in an inexpensive way and (2) to increase the rate of recognition of severe faults during system-level assessment of new code and regression trials of existing code.

They present a value-driven approach to system-level test case prioritization called the Prioritization of Requirements for Test (Interface). PORT predicated on pursuing four factors.

1) Requirements volatility

Is based on the number of times a necessity has been changed through the development cycle.

2) Customer priority

Is a way of measuring the importance of a requirement to the customer?

3) Execution complexity

Is a subjective measure of how difficult the development team perceives the implementation of necessity to be.

4) Fault proneness

Of requirements (FP) allows the development team to recognize the requirements that have acquired customer-reported failures.

They state in research paper, Prioritization of Need Test (Dock) has great impact on finding severe fault at system level. They are focus on Customer priority in TCP for enhance the fault recognition.

Today software companies will work on neutral manner. They placed neutral value to all requirements use instances, test instances and defects. To enhance the customer satisfactions in software executive world they may be showing a value-driven approach for system level screening. In these days Regression Test Circumstance Prioritization techniques use structural coverage conditions to choose the test situations. They are simply leading their ideas from composition level to code level TCP for both new and Regression exams.

This Paper has two main objectives: 1). Find severe faults previously 2). Improve customer self-confidence on particular system.

Researchers express several techniques [8] for prioritizing test cases plus they empirically evaluate their ability to improve rate of fault recognition"a measure of how quickly faults are discovered within the tests process. An improved rate of fault diagnosis during regression assessment can provide early on feedback on a system under regression ensure that you let developers start debugging and correcting faults sooner than might otherwise can be done.

Their results suggest that test case prioritization can significantly enhance the rate of fault detection of test suites.

Furthermore, their results showcase tradeoffs between various prioritization techniques.

Test circumstance prioritization can addresses a wide variety of objectives. Used, and depending upon the decision of goal, the test circumstance prioritization problem may be intractable: aims, an efficient means to fix the challenge would offer an efficient answer to the knapsack problem [8]. Authors consider nine different test circumstance prioritization techniques.

T1: No prioritization

One prioritization "technique" that authors consider is simply the use of no technique; allowing us consider "untreated" test suites.

T2: Random prioritization

Random prioritization in which authors randomly order the lab tests in a test collection.

T3: Optimal prioritization

An optimal ordering of test instances in a test collection for increasing that suite's rate of fault diagnosis. Used, of course, this is not a practical approach, as it requires understanding of which test circumstances will expose which faults.

T4: Total branch coverage prioritization

We can determine, for any test case, the amount of decisions (branches) for the reason that program which were exercised by that test case. We can prioritize these test situations in line with the final number of branches they cover by just sorting them in order of total branch coverage achieved.

T5: Additional branch coverage prioritization

Total branch coverage prioritization schedules test cases in the region of total coverage achieved. However, having performed a test circumstance and covered certain branches, more may be gained in succeeding test circumstances by covering branches that have not yet been covered. Additional branch coverage prioritization iteratively chooses a test circumstance that yields the best branch coverage.

T6: Total fault-exposing-potential prioritization

Statement- and branch-coverage-based prioritization consider only whether a declaration or branch has been exercised with a test circumstance. This account may mask a fact about test situations and faults: the power of your fault to be revealed with a test case will depend not only on if the test case reaches (executes) a faulty assertion, but also, on the possibility a fault for the reason that statement may cause a failure for your test circumstance. Although any sensible determination of this possibility must be an approximation, we wished to determine if the use of such an approximation could produce a prioritization strategy superior in conditions of rate of fault recognition than techniques predicated on simple code coverage.

T7:Additional fault-exposing-potential (FEP) prioritization

Analogous to the extensions made to total branch (or declaration) coverage prioritization to additional branch (or affirmation) coverage prioritization, we lengthen total FEP prioritization to generate additional fault-exposing-potential (FEP) prioritization. Allowing us account for the actual fact that additional executions of a statement may be less valuable than original executions. In additional FEP prioritization, after choosing the test circumstance t, we lower the award principles for all the test situations that exercise claims exercised by t.

T8: Total declaration coverage prioritization

Total assertion coverage prioritization is the same as total branch coverage prioritization, except that test coverage is measured in conditions of program statements somewhat than decisions.

T9: Additional declaration coverage prioritization

Additional statement coverage prioritization is equivalent to additional branch coverage prioritization, except that test coverage is measured in conditions of program claims somewhat than decisions. With this technique too, we need a way for prioritizing the remaining test circumstances after complete coverage has been achieved, and in this work, we do that using total assertion coverage prioritization.

2. 3. 1. Search Algorithms for Test Circumstance Prioritization

There are extensive search techniques for test circumstance prioritization, which are being developed and unfolded by various analysts in the field.

1) Greedy algorithm:

Works on another best search school of thought. It [9] decreases the estimated cost to attain a particular goal. Its benefits is that it's cheap in both execution time and execution. The cost of this prioritization is O(mn) for program containing m statements and test collection containing n test cases.

2) Additional Greedy algorithm:

This algorithm [9] uses the opinions from previous selections. It selects the maximum weight component from the part that is not already used by previously decided on elements. Once the complete coverage is achieved, the rest of the test situations are prioritized by reapplying the excess Greedy algorithm. The cost of this prioritization is O(mn2) for program containing m claims and test collection containing n test instances.

3) Hill Climbing:

It is one of the popular local search algorithms with two variants; steepest ascent and next best ascent. It's very easy and cheap to execute. However, this has cons of dividing O(n2) neighbors and is improbable to scale. Steps of algorithm are discussed in [9].

4) Hereditary Algorithms (GAs):

Is a search approach [9] predicated on the Darwin's theory of survival of the fit test? The populace is a couple of randomly generated individuals. Every individual is representing by parameters/parameters called genes or chromosomes. The basic steps of Genetic Algorithm are (1) Encoding (2) Selection (3) Cross (4) Mutation.

3. CONCLUSION

In this newspaper we discussed about Regression test selection and Test Case Prioritization Selection. Regression assessment is a style of testing that focuses on retesting after changes are created. In traditional regression screening, we reuse the same lab tests (the regression assessments). In risk-oriented regression testing, we check the same component features as before, but we use different tests. Any test can be reused, and so any test may become a regression test. Regression trials in a natural way combines with all other test techniques. Therefore we use Test Case Prioritization technique inside Regression Testing. Test prioritization makes strengthen our regression testing for finding more severe fault in early stages.

In this paper we mentioned about different factor of prioritization. Customer priority has a great impact on Dock. Our view about both test circumstance selection is, First version of test suite which developed by developer must have concrete test instances. Also at the same stage we should perform some prioritization. With early prioritization of test conditions we can reduce our cost, time, work and take full advantage of customer satisfaction

4. References

[1] Todd L. Graves, Mary Jean Harrold, Jung-Min Kim, Adam Porters, Gregg Rothermel, "An Empirical Research of Regression Test Selection Techniques",

Proceedings of the 1998 (20th) International Conference on Software Engineering, 19-25 April 1998 Webpage(s):188 - 197.

[2] Leung, H. K. N. , White, L. , "Insights into Regression Testing", Proceedings. , Discussion on Software Maintenance,

16-19 Oct. 1989 Page(s):60 - 69.

[3] K. K. Aggarwal & Yogesh Singh, "Software Executive Programs Documentation, Working Procedures, " MODERN International Publishers, Revised Second Edition - 2005.

[4] Naslavsky L. , Ziv H. , Richardson D. J. , "A Model-Based Regression Test Selection Technique", ICSM 2009. IEEE International Convention on Software Maintenance, 20-26 Sept. 2009 Page(s):515 - 518.

[5] Gorthi R. P. , Pasala A. , Chanduka K. K. P. , Leong, B. , "Specification-Based Method of Select Regression Test Suite to Validate

[6] Changed Software", APSEC '08. 15th Asia-Pacific Software Executive Convention, 3-5 Dec. 2008, Page(s):153 - 160

[7] Zengkai Ma, Jianjun Zhao, "Test Circumstance Prioritization predicated on Research of Program Structure", APSEC '08. 15th Asia-Pacific Software Executive Meeting, 3-5 Dec. 2008, Page(s):471 - 478

[8] Srikanth H. , Williams L. , Osborne J. , "System Test Circumstance Prioritization of New and Regression Test Conditions, " 2005 International Symposium on Empirical Software Anatomist, 17-18 Nov. 2005, Page(s):10 pp.

[9] Rothermel G. , Untch R. H. , Chengyun Chu, Harrold M. J. , "Test Case Prioritization: An Empirical Study", (ICSM '99) Proceedings. IEEE International Seminar on Software Maintenance, 30 Aug. -3 Sept. 1999, Page(s):179 - 188

[10] Zheng Li, Tag Harman, and Robert M. Hierons, "Search algorithms for regression test case prioritization, " IEEE Trans. On Software Anatomist, vol 33, no. 4, April 2007.