When we are doing automation testing project (5000 Man days project), What are the metrics needs to capture in the agile scrum method other then
1. Number of user stories automation,
2. No of lines covered
3. Defects captured.
Here my tricky questions is When we need to start automation; Same user story in the same spirnt or in the next sprint? (After complete the development activity manual testing is part of the same sprint) One of my client want's to complete Automation scripts also needs to complete in the same sprint. Really its very challenging, How to handle this situation. Saving Changes...
Ideally, all test cases are automated such that the focus of your testing resources is on designing methods of maximizing quality vs doing manual testing.
I would focus more on code coverage and the percentage of acceptance criteria which were addressed through automation than defect counts as with the latter you will hit the law of unintended consequences.
As far as when automation should happen, if a story has good acceptance criteria, the test analyst should create the automation within the same sprint so that when it has been developed, it can be fully tested via automation.