Questions & Answers
Get your Rally questions answered.
Question
How to track test case time
Question posted 06/16/09 by Tony1983, last edited 06/17/09 , tagged Configuration Customization, Integrations, Planning, Requirements Management, Resource Management, Scheduling, Status Tracking, Test Management, Usability, User Stories and Tasks
12308 Views, 11 Comments
Title:
How to track test case time
Summary:
So I am trying to figure out how to best track testing time in a story. The test cases do not seem to have a time field. How do you track how much testing time will contribute to a story and then track time over multiple test case runs to see if you are improving test time or not.
Thoughts
Comments
Hi Tony - there is a duration field on the Test Case Result object where you could enter the time for a Test Case Result or the time of the run. You could also add a custom field to the Test Case object for a 'time estimate' if needed.
Hope those ideas help.
Reply to this Comment
so how do people go about tracking testing time associated with a user story? I would think its a common problem seems odd you need to add a custom field for it.
Reply to this Comment
Hi Tony - fair question - I'd be open to the community to see what they say.
The only other thing I could think of would be to have a task associated at the story level - not great, but that would at least be at the story level.
By the way, just to know - is the need for the time tracking related to billing needs or after the fact estimating evaluation?
Reply to this Comment
Tony:
We are looking at adding a Time Tracking module to Rally. I would be interested in discussing whether that module could help track tester time. The input I receive so far from other testers is a potential need to populate the Test Case Result duration field from the Timesheet, but I am guessing testers may also need to track non-test case time as well?.
Reply to this Comment
actual run time is needed but the total time it takes to develop an app is the dev time + test time (and doc, PM...) So only providing the ability to ad times to dev tasks and not test cases seems weird. I am used to estimating test time for new features and currently in rally I have no way to do this without a hack
Reply to this Comment
Tony:
The question is whether a Test Case is the right element to enter time against, vs a test task or a test case result.
To estimate testing time for new features, you could add a "Test it" task for each feature and enter an estimate on that task for how long it will take to test the feature. Using Time Tracking, testers can then enter how long it actually took them to complete that task. Would that be a workable model to track testers time?
Reply to this Comment
well isn't a testing task really a test case or collection of test cases? So a single story may have several developer tasks that are the steps dev needs to complete the story. the QA equivalent is a test case. so for story X I need to execute these 15 test cases to complete the qa portion of the story why not have the time estimates in the test case. adding a test task to a story then also adding test cases to a story seems redundant.
Reply to this Comment
Tony:
In the scenario above (with the Rally Quality Manager module enabled), you would create the 15 test cases, each associated with the one story to test. You would then create a test set to include the 15 test cases. You can then set an estimate on the test set. Test sets allow you to schedule a group of test cases multiple times (for regression testing).
Would tracking time at the test set level make sense to you?
Reply to this Comment
During my tests of rally I came across this issue too and it is fairly complex. A test case or collection thereof is not a test task but a description of what to test. Testcases can and often have to be testet multiple times. So a test case is not the right place to track testing time. At the moment the best approach to estimate the time needed for testing in my opinion is to schedule some tasks in the story:
TA1 Write Unit-Tests
TA2 Test Cases Run 1
TA3 Fix Defects from Test Cases Run 1
TA4 Test Cases Run 2
This can be repeated more often if required of course....
What would be needed is some way to track testing/bugfixing time in contrast to development time. Using the tasks from the example works well for estimation but later there is no easy way to say which time was dedicated to development, which was dedicated to running tests and which time was used for fixing the defects because all time tracking has been done in the tasks that do not have a connection to the the test cases or the defects.
Reply to this Comment
This is in line with the recent interviews I have done as part of the Time Tracking beta. Testers will want to track their time against a task, and Time Tracking will be supporting this process. If you use a task custom field to indicate Dev work vs. QA work, the Time Tracking reports will generate hours spent on Dev work vs. QA work. Would that satisfy your need? else how would you want to determine QA work vs Dev work?
Reply to this Comment
well im sure the feature would be usable in a few places my prefrence would be the test case and here are my thoughts as to why.
If I go through the process of estimating time on each ind test case and each tim eits run put an actual time I can over time get a good estimation of how long that test case takes. Then if I create a set of 15 test cases I know how long that set should take. If I add or remove test cases to the set I still know how long the set will take. If I estimate the set I have to re estimate each time I add or remove a test case. If I do not always run the same set. So say I add 15 test cases to release 1 testing in release 2 its the 15 test cases plus 8 new ones the original test set time is meaningless. but if its on the test cases it adds 15 accurate guesses and 8 new ones so its much better.
What you cannot do if its on the test case is estimate testing tasks that take time that may not be an actual test case. Those would still need to be added via a task on the story.
I have to say the more I use the quality module the more I find it almost unusable. So many small things it just wont work for me.
Reply to this Comment