Tuesday, July 28, 2009

Annoying bug tracking systems

Bugs are extremely annoying, but unfortunately bug tracking systems are also very annoying. They are several existing systems used with open source software. Unfortunately they more or less all sports the same problems.

One which is rather known is the connection of the bugs between upstream (the original project) and downstream (the distros), and in between downstream. Each project and each distro has its own bug tracking system. As a result of this, first, bugs might end up being duplicated: in general the users report their problem in the tracking system of their distro, excepted for some advanced users who directly follow upstream and report there as well. Second, the bugs get reported to places which are not read by the authors of the project, which reduces a lot the chances to have them fixed. So far, only lauchpad has tried to answer this problem, by automatically following upstream, but this does not resolve the problems to link the bug reports in-between downstream. Of course, it doesn't seem reasonable to have only one global bug tracking system for all the projects in the world. What would be useful is a mechanism which allows to transmit a bug from downstream to upstream, just by decision of the bug triaging team. It is then up to the triage team to differentiate bugs related to packaging problems to the bugs which have to be treated upstream.

However, an even bigger problem is that the bug tracking systems are actually bug report tracking system. The main concept is not a "bug", but the "report from someone about a bug". The main description about a bug is the first description ever written about it... which is one less likely to be precise: written before anyone ask for precisions, by someone who is a simple user. Generally, valuable information about the bug (eg: why and when it happens, how it can be fixed) is only available disseminated in the comments, or, even worse, in the comments of bug reports marked as duplicates. To correct this, the main concept should be shifted from "bug report" to "bug". Each bug could have assigned to it several bug reports, and bug reproducers (people would are affected by this bug and most likely able to confirm that the bug is fixed). The main description of a bug would be much closer to a wiki page, which can be updated little by little, as more things get known. Moreover, associated to the bug, there should be test cases, which allow to reproduce the bug. Each test case can be either manual, or automatic, in which case it could be possible to run the test case regularly and automatically against the latest version of the project to detect if the bug is still present. A bug would also be associated to fixes and workarounds. The fixes are patches, which can be commented, and which eventually become a commit in the code repository of the project.

A third problem is that when someone finds a bug in a project and reports it. The reporter should first check for previous reports of the bug. However, if the bug has been corrected in the mean time (typically, in the development version), it will not be easily findable because marked as closed. That is why the bug tracker should also be more aware of the versions of the projects, usually ordered as a tree. So that for each version of the project, it is possible to know which (known) bug is present, likely present, or fixed. When someone search for a bug, he or she will also indicate the version on which the bug was seen, and the search will show bugs even now closed. Not only this avoid to create a duplicate bug report, but also allow the reporter to use a workaround if it is available, without having to update to a new version.

Lots of perspectives to improve bug tracking systems. Maybe it is possible to modify bugzilla or launchpad in this directions. Maybe first, a proof of concept should be implemented. What other important features are missing in current bug trackers?

1 comment:

Bell Brown said...

defect tracker helps businesses report and resolve problems that occur in products or services. In the software industry, the term defect tracking is often used synonymously with the term bug tracking.
Software defect tracking system