Comments on: The art of beta testing-Part 1 http://tamspalm.tamoggemon.com/2005/11/09/the-art-of-betatesting-part-1/ Palm OS / web OS news and opinion source Thu, 09 Feb 2012 07:33:49 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: admin http://tamspalm.tamoggemon.com/2005/11/09/the-art-of-betatesting-part-1/comment-page-1/#comment-695 admin Wed, 15 Mar 2006 17:49:42 +0000 http://tamspalm.tamoggemon.com/?p=146#comment-695 Hi Paul, thank you very much for the comment! Actually, sth like that exists for the PalmOS too, sorta. It is called Gremlins, and is a UI-Stresstester... Best regards Tam Hanna Hi Paul,
thank you very much for the comment!

Actually, sth like that exists for the PalmOS too, sorta. It is called Gremlins, and is a UI-Stresstester…

Best regards
Tam Hanna

]]> By: Paul Dietz http://tamspalm.tamoggemon.com/2005/11/09/the-art-of-betatesting-part-1/comment-page-1/#comment-673 Paul Dietz Tue, 14 Mar 2006 13:30:04 +0000 http://tamspalm.tamoggemon.com/?p=146#comment-673 You should be careful to distinguish two meanings of 'random testing'. The first meaning -- unstructured, unplanned, arbitrary testing -- seems to be the one you mean. But there is another meaning: automated testing using very large numbers of test cases generated by a randomized test generator. This latter approach is most definitely NOT useless -- it will find bugs that structured testing with manually constructed test cases will have a very hard time finding. Indeed, experience is that any non-trivial program subjected to such testing for the first time will quickly break. This includes otherwise mature programs that have extensive test suites, such as gcc or Mozilla. Microsoft estimates that 15-20% of the bugs found during development of their products come from high volume randomized testing of this kind. You should be careful to distinguish two meanings of ‘random testing’. The first meaning — unstructured, unplanned, arbitrary testing — seems to be the one you mean. But there is another meaning: automated testing using very large numbers of test cases generated by a randomized test generator. This latter approach is most definitely NOT useless — it will find bugs that structured testing with manually constructed test cases will have a very hard time finding. Indeed, experience is that any non-trivial program subjected to such testing for the first time will quickly break. This includes otherwise mature programs that have extensive test suites, such as gcc or Mozilla. Microsoft estimates that 15-20% of the bugs found during development of their products come from high volume randomized testing of this kind.

]]> By: The art of betatesting-part 2 >> TamsPalm-the Palm OS Blog http://tamspalm.tamoggemon.com/2005/11/09/the-art-of-betatesting-part-1/comment-page-1/#comment-88 The art of betatesting-part 2 >> TamsPalm-the Palm OS Blog Fri, 18 Nov 2005 17:31:04 +0000 http://tamspalm.tamoggemon.com/?p=146#comment-88 [...] Welcome back to our art of betatesting column. Just in case you forgot what the first part was all about, feel free to read it up here: The art of betatesting-structured beta testing Completely outsourcing tests is dangerous Man, your old-style. Having five handhelds at home is stupid! I dont have a single one at home or in my office-Developer of Censored, on ICQ This impressive person apperently trusts his betatesters even though he does not give them test cases or anything. If he has extremely committed testers, this will work fine. However, if not, goodnight. A beta tester usually is not paid (much) and is a plain user interested in either the new program, getting a serial for free or showing off to his friends that he runs the latest and greatest apps! His interest in your well-beeing is rather low. Exceptions exist-guard them like a treasure, people! Anyways, a beta tester has a higher probability to overlook things than the original developer has. It doesn’t have to be lack of interest or typing effort, he can also treat this as o.k. as he discovered a workaround. Anyways, there is an old german proverb saying that control is better than trust. Keeping at least a mid-end box like a Tungsten T or E2 in your labs definitely wont bust your budget-but it would keep you at least theoretically capable to test your software yourself. If you are too lazy to do it though,… Got something to add? [...] [...] Welcome back to our art of betatesting column. Just in case you forgot what the first part was all about, feel free to read it up here: The art of betatesting-structured beta testing Completely outsourcing tests is dangerous Man, your old-style. Having five handhelds at home is stupid! I dont have a single one at home or in my office-Developer of Censored, on ICQ This impressive person apperently trusts his betatesters even though he does not give them test cases or anything. If he has extremely committed testers, this will work fine. However, if not, goodnight. A beta tester usually is not paid (much) and is a plain user interested in either the new program, getting a serial for free or showing off to his friends that he runs the latest and greatest apps! His interest in your well-beeing is rather low. Exceptions exist-guard them like a treasure, people! Anyways, a beta tester has a higher probability to overlook things than the original developer has. It doesn’t have to be lack of interest or typing effort, he can also treat this as o.k. as he discovered a workaround. Anyways, there is an old german proverb saying that control is better than trust. Keeping at least a mid-end box like a Tungsten T or E2 in your labs definitely wont bust your budget-but it would keep you at least theoretically capable to test your software yourself. If you are too lazy to do it though,… Got something to add? [...]

]]> By: admin http://tamspalm.tamoggemon.com/2005/11/09/the-art-of-betatesting-part-1/comment-page-1/#comment-59 admin Mon, 14 Nov 2005 21:30:38 +0000 http://tamspalm.tamoggemon.com/?p=146#comment-59 Hi, I started the external tests earkly in the alpha process. That and an occasional code sanity check were all my measures up to now. Now, I use a strictly planned betatesting that happens in-house on various machines and code sanity reviews on paper. How do you do it? Any info on other methods? Best regards and thanks for talking back Tam Hanna Hi,
I started the external tests earkly in the alpha process. That and an occasional code sanity check were all my measures up to now.
Now, I use a strictly planned betatesting that happens in-house on various machines and code sanity reviews on paper.
How do you do it? Any info on other methods?
Best regards and thanks for talking back
Tam Hanna

]]> By: Kossi http://tamspalm.tamoggemon.com/2005/11/09/the-art-of-betatesting-part-1/comment-page-1/#comment-57 Kossi Mon, 14 Nov 2005 13:04:29 +0000 http://tamspalm.tamoggemon.com/?p=146#comment-57 Hi Tam, ok, beta and frontend-testing is one form off testing. Which testing methods do you use in your development process? Do you implement unittests or some other kind of component test methods? Hi Tam,

ok, beta and frontend-testing is one form off testing. Which testing methods do you use in your development process? Do you implement unittests or some other kind of component test methods?

]]>