Software testing at YouView
jimb
Member, Super User Posts: 1,425 ✭✭✭
in Discussion
Some of you might find this interesting...
http://www.softwaretestingnews.co.uk/head-of-test-at-youview-gives-tips-to-shipping-good-quality-code/
http://www.softwaretestingnews.co.uk/head-of-test-at-youview-gives-tips-to-shipping-good-quality-code/
0
Comments
YouView’s design, as regards the initial UI for NextGen, was surprisingly lacking. I suspect all the effort had gone into the wire-framing, without enough thought as to what was going to hook it all together.
Then development turned out to be no quicker than before, so the incremental improvement of the code was slow.
But YouView’s testing has always been very good; I don’t think I have ever seen a regression error (where an old problem resurfaces to bite you), and no matter how small or large, frequent or infrequent, releases pretty much do what they are intended to do. Even if, as happened in the early days of NextGen, they came with a pretty tragic UI; such issues were not in the testing teams’ gift to fix.
By and large, YouView releases hold pretty slim pickings for bughunters, which is greatly to the test teams’ credit.
(And also, you may say, pretty slim enhancements for users; but again, that is hardly the test teams’ fault).
I get a bit apprehensive about Agile though (again, not the testers’ fault); there are good things in it, which we see coming through in how easy it now is to release little and often, rather than large and infrequent.
But the model it has, of iterating from approximation towards perfection, fits better in a controlled work environment among collaborative user teams than it does in a consumer product where every release has to be as perfect as possible.
So we get a hybrid development model, part Agile, part traditional, where the Agile methodology of breaking down the requirements into little chunks - the sprints - can be operated well, but the releases can’t contain any approximations, and so, as in earlier development models, each added feature has to be perfected by the developers before release.
Which the test teams do their level best to ensure is indeed the case.
@ Roy - I don't agree with that, there are many releases that contain fixes (issues that were either not spotted in the original or have been brought in), off the top of my head I vaguely remember a HDMI handshake issue but there are many. Of course most updates do, so YouView is just as normal but I also think your picture paints them purer than pure.
If you look at Keith's list of bugs and faults you will see some that are brought in and fixed (some purely on the next gen).
Also I would suggest that if they have a team of 30 people with very little to test as you suggest (even if they do it well) they need to look at resource levels across their departments
Brought In bugs are the hard ones; every new feature demands a new test plan, and yes, may break something that worked before.
But that’s not regression; regression is when an old bug resurfaces, and YouViews methodology ensures that every new test plan goes into the overall testing armoury, as applied to each new release to prevent this.
I would guess that such testing will also reveal many cases where a new release breaks earlier functionality; but not every case, as there are so many paths to be walked.
So there are always a few stony paths for us to find
But I still hold to the view that given the complexity of the YouView software and the changing and unpredictable environment it has to operate in, the bug rate is gratifyingly low.
From what I've seen all YouViews releases are pretty much bug free.