Followup on priority mass-initialization project

        In case it helps you out, I can provide you with raw data, as well as
additional constraints to help you focus your prioritization efforts.
If nothing else it'll reinforce the sleepily absorbed discussion from
earlier.
        Not that we wouldn't like every bug either fixed or discretely
prioritized with perfect spanning, but hopefully by keeping thoroughly
guided we can get the most bang for your buck (or pound for sound, if
you find the prostitutory allusion offensive).
        Let me know if there's anything you don't quite understand (or if you
just don't care). I'm assuming (do correct me if I'm wrong) that you
don't know mysql, but it should be relatively straight forward:

mysql> SELECT profiles.login_name AS email,
        COUNT(bugs.bug_id) AS bugcount
        FROM bugs
        INNER JOIN profiles ON bugs.assigned_to = profiles.userid
        WHERE
        ((bugs.bug_status = "UNCONFIRMED" OR bugs.bug_status = "NEW" OR
bugs.bug_status = "ASSIGNED")
        # Of open bugs except for REOPENED
        AND bugs.target_milestone = "2.2.x"
        # Which are targetted at 2.2.x (only)
        AND (bugs.bug_severity="major" OR bugs.bug_severity="critical" OR
bugs.bug_severity="blocker" OR bugs.bug_severity="normal"))
        # And are of normal or greater severity
        GROUP BY email ORDER BY bugcount DESC LIMIT 10;
        # Show me who has the most, and hence, presumably needs the most help
prioritizing.

+---------------------------------+----------+
| email | bugcount |
+---------------------------------+----------+
| msevior@physics.unimelb.edu.au | 56 |
| hfiguiere@teaser.fr | 19 |
| cinamod@hotmail.com | 19 |
| jmas@softcatala.org | 19 |
| bugs-owner@abisource.com | 11 |
| F.J.Franklin@ncl.ac.uk | 10 |
| abisource-qa-spam@abisource.com | 5 |
| hippytrail@gmail.com | 4 |
| tomasfrydrych@yahoo.co.uk | 3 |
| j.m.maurer@student.utwente.nl | 3 |
+---------------------------------+----------+
10 rows in set (0.18 sec)

        Then we proceed to do a query (web works) for the bugs themselves.
Knowing an assignee to filter by helps keeps things manageable at this
stage, and the above query includes some filters from the below that
we'll want to maintain as they reflect our immediate aims. Open but not
reopened bugs, restrict to 2.2.x, don't worry about low-severity bugs,
assigned to one of the doomed (or use a boolean chart with the assignees
to flag all of the bugs cropped in the above set) &c &c...

Martin's query:
http://bugzilla.abisource.com/buglist.cgi?short_desc_type=allwordssubstr&short_desc=&target_milestone=2.2.x&long_desc_type=allwordssubstr&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=allwords&keywords=&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_severity=blocker&bug_severity=critical&bug_severity=major&bug_severity=normal&emailassigned_to1=1&emailtype1=substring&email1=msevior&emailtype2=substring&email2=&bugidtype=include&bug_id=&votes=&changedin=&chfieldfrom=&chfieldto=Now&chfieldvalue=&cmdtype=doit&namedcmd=1-0-STABLE&newqueryname=&order=Importance&field0-0-0=noop&type0-0-0=noop&value0-0-0=
(you can see by the numbers that the queries match - both currently 56)
Martin and dom, as an example of the charting:
http://bugzilla.abisource.com/buglist.cgi?short_desc_type=allwordssubstr&short_desc=&target_milestone=2.2.x&long_desc_type=allwordssubstr&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=allwords&keywords=&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_severity=blocker&bug_severity=critical&bug_severity=major&bug_severity=normal&emailtype1=substring&email1=&emailtype2=substring&email2=&bugidtype=include&bug_id=&votes=&changedin=&chfieldfrom=&chfieldto=Now&chfieldvalue=&cmdtype=doit&namedcmd=1-0-STABLE&newqueryname=&order=Importance&field0-0-0=assigned_to&type0-0-0=equals&value0-0-0=msevior%40physics.unimelb.edu.au&field0-0-1=assigned_to&type0-0-1=equals&value0-0-1=cinamod%40hotmail.com
(again, 56 + 19 = 75)

        The additional restriction of severity brings the number of bugs with
which we are concerned down further still to 133 (of the currently 303
bugs targetted at 2.2.x). Again, this is the easy part, the hard part
is actually prioritizing, but by narrowing our focus, we have a much
more reasonable goal and, if it's all we can do, gets us the most
important domain to be done for our time. Working in descending order
of most doomed also helps the assignees the most (ie, it probably
matters a lot more to martin the order in which he approaches 56 bugs
than to tomas the order in which he approaches 3 - Martin can't fix 56
bugs in the two weeks after 2.2.0, but he might be able to fix a dozen
(or more), so it's important that he know which to do).

NB: I chose for brevity's sake to use sql for part of this message, but
there isn't anything there that you can't accomplish via the web
interface.

        The actual criterion behind prioritization itself is beyond the scope
of this email, and has been explained numerous times already anyway. If
anyone (I've cced -qa, btw) has any questions about it or cases they
aren't certain about, you're welcome to ask me.

        Also, it should be noted that maintaining prioritization of incoming
bugs is absolutely key, possibly even more so than the initialization of
priorities for existing bugs. Otherwise it's like trying to swim up
river in heavy rain rather than swimming across a large but calm pond.

        Finally, a word about how much time this should take. Not much. While
we have to give careful thought to prioritization (commonality of
encounter, severity of impact on feature, severity of impact on product,
fixability, importance relative to the other bugs in the target, etc.),
if after a few dozen you're still spending more than a couple minutes
deciding on bugs (not just one specific tricky bug), something's wrong.
In general we don't want to spend more time prioritizing reports than we
spend filing them or the coders spend fixing them. The whole point of
bringing down our aim domain of prioritization (for this next step past
2.2) is to make this _not_ a big project. A few manhours at the most,
not more. So make sure to skip individual bugs that you can't decide on
or figure out, and always feel free to bounce bugs off of your fellow
qa-mates (if any exist). We've got the list, it remains to be really
used. Also don't forget lads that priorities can evolve.

        Oh, and there was a little confusion as to the differences btwn
milestone target based prioritization and branch target based
prioritization. I've tried to bring things down into realistic street
terms a bit:

2.2.x: P1 - Needs to be fixed ASAP. If not by 2.2.1, then by 2.2.2.
If not by 2.2.2, then by 2.2.3, and so on.... if the number of P1 bugs
doesn't diminish, btw, it could be justifiably used to support not
branching.
        P2 - Still very important, fix ASAP. Probably should not stop
branching, though. "Don't be spending most of your time working on
nutty 2.4 feature branches when there's still P1 and P2 bugs left,
they're marked important because they are."
        P3 - Yay, we've fixed the P1 and P2 bugs, those were the really urgent
ones that needed to be fixed ASAP. Now the big post-milestone rush is
over, we can relax the release schedule and encompass more P3 bugs in
each. "We won't be terribly upset at you if you reward yourself by
dividing your time with feature work on 2.4 either, as long as there's
still a steady stream of P3 fixes coming in."
        P4 - Dear developer, you've fixed the P1-P3 bugs, good job. Have a cup
of coffee, take a nap, visit your family who hasn't seen you since the
2.2 rush began, you deserve it. Then come back to these.
        P5 - It'd be quite nice for 2.2.x to eventually include these fixes,
but noone is going to be upset if it's 2.2.10 before you get to them.
"Have fun hacking away at 2.4 as long as you don't forget about these."
Not so much because they're hard to fix as because they're either
absolutely trivial or of easily survivable severity and likely not to be
encountered by more than one in thirty thousand users.

        Remember lads, coders are volunteers, too; QA's role is merely that of
an expert suggestor. If we say "this affects a significant number of
people in a significant manner, P2" and the assignee says "this is going
to be a really time consuming fix, P3", the latter is predominant. Our
goal is to _help_ the coders make the most of their time.

Hope everyone enjoyed spending the afternoon reading this email, sorry
it got so verbose (again), but I hope it helps those that wish to help
with the prioritization effort (or just understand QA processes a little
better).

Best regards
-MG
Received on Mon Nov 15 23:24:00 2004