Global Sources
EE Times-Asia
Stay in touch with EE Times Asia
EE Times-Asia > Embedded

How to overcome the triage challenge

Posted: 17 Sep 2012 ?? ?Print Version ?Bookmark and Share

Keywords:regression testing? bug fixing? triaging process?

When regression tests fail, an engineer must settle which failing tests should be further analyzed and by whoma process called triage. The challenge is the dearth of information about the bug and its creator at this early stage. This session outlines a novel method that addresses the issue by driving the testing to gather enough information to do fully automated triaging. This paper explains how this innovative approach solves the triage challenge, thus accelerating the bug fixing process by 30%.

Regression testing is performed regularly through-out a project in order to ensure that the quality never dips, regresses. Regression testing is normally automated in terms of kicking off the tests and putting together the test results. But the analysis of the test results is not automated. The first thing that needs to be done is to look at the test failures, decide which are real errors and which are just intermittent computer or network issues and then assign the bug to the appropriate engineer. This process is called triaging. As it is impossible to know the exact nature of a bug, before it has been thoroughly analyzed, triaging has to be done while having very little information available at this early stage. The methods used are ad hoc, based on experience, association or static assignment, where an engineer is assigned for each test to first look at the failure. The triaging process takes time as the test results needs to be translated into bug reports and if the results are sent to the wrong person(s) then it will take longer before the issues are fixed.

What is needed is a tool that provides automatic triage which immediately sends the bug reports to the person who committed the faulty revision in order to accelerate the bug fixing process. The earlier the correct engineer gets the bug report the sooner the bug will be fixed. In order for this to work, automatic triage needs to be very robust so that it always sends the bug reports the right way and must be fast to run and use limited amount of resources so that it is efficient to use in large systems. This paper looks at the technology behind a new tool called PinDown, which has a novel approach to automatic triage that addresses these issues.

Figure 1: Automatic triage tool in system.

Automatic triage
Mapping test results to revision information: The Version Control System (VCS) contains the code of the device under test, the test bench, the tests and all the changes made to all of its files. The VCS contains information about each revision of each file or module such as: the committer of the revision, the time of the commit and the code lines that were updated. This is the information needed for triaging: who (committer), what (revision) and when (time of commit). What is needed is a tool that can map test failures to a revision in the VCS in order to extract all the information needed for triaging, including both creating a bug report and assigning it to the right person.

In order to take advantage of the information available in the VCS an automatic triage tool consequently needs to interface against both the VCS and the regression test system in order to map test results to revision information (figure 1). The automatic triage tool also needs to be able to communicate the result of its diagnosis automatically to the bug tracking database, email and files on the web and on the normal file system. The automatic triage tool also needs to save the test results and its diagnosis in a test result database to allow for speedy analysis and to be able to amass historical test results information that can be used in its analysis.

1???2???3???4?Next Page?Last Page

Article Comments - How to overcome the triage challenge
*? You can enter [0] more charecters.
*Verify code:


Visit Asia Webinars to learn about the latest in technology and get practical design tips.

Back to Top