What is Crowdtesting anyway?
Crowdtesting is a test method that has the advantage that many users (with different experience) access the same application and look for vulnerabilities or errors during the beta phase, ie before the official launch, which may have been left over from previous cycles. Many eyes see more and therefore, this approach is always found remaining, rather low prioritized »bugs.
Many crowdtesting providers use amateur testers who do this as a part-time job, but still achieve acceptable results. Ultimately, it’s about taking the AUT (Application under Test) again from the user’s point of view under the magnifying glass and eradicating it before the market launch. Crowdtesting can be used for end user applications (desktop and mobile), but not in the b2b area or backend systems.
Proceed exhibit at Rocketlab
If we are commissioned with a crowd testing project, the following procedure applies:
1. Project setup (entry)
You will receive your personal contact person for each project, who will take care of all matters. Together, we first go through the application being tested, all facets (e.g., language versions, different accesses, if any, etc.). Furthermore, we define the media with which to test. These include desktop applications (operating system, OS version, browser, browser version, language, certain presets in the OS and / or browser, etc.), smartphones (iOS / Android, browser, version, language, extensions, …), tabs (iOS / Android, resolution, horizontal / vertical orientation, browser, version, language, extensions, …).
Last but not least, the tester group is an important factor, as it requires both experienced testers, the “normal user” or even subject matter experts or testers with specific knowledge.
Whether one or the other makes sense is discussed in the setup, but can also be added or subtracted during the course. Your contact person is always available to help you achieve valuable test results.
2. Setup test environment
The test setup covers u.a. the provision of the IT infrastructure in which the predefined test cases (and steps) are to be subsequently performed by the testers. This includes rights and roles, use of bug tracking tool (for example, Jira, ALM / ALM Octane, Bugzilla or your own tool). All required test equipment is prepared and checked, possibly adjusted if specific settings are necessary. We have a large number of common devices of different OS versions and browsers. Most of them are also in use in test automation. Not least, the selection of testers is part of this phase. It is very important to have reliable testers in use. We place the highest value on quality. This is also reflected in the tester selection.
3. Test procedure
Depending on the definition, the test procedure takes place within short cycles or the aim is to carry out the entire scope in different variants (browser, language, currencies, …). Despite the highest quality, you will soon reach the first results so that you can continue to work on the troubleshooting.
4. Test Analyses
Found bugs are still going through our test analysts before they are made available to you. It sometimes happens that certain behaviors only occur in certain persons, since there may still exist a session cookie from earlier or another presence and therefore the errors occur. We attach great importance to the fact that you receive our results comprehensible, with screenshot or screencasts and with understandable documentation. Finally, the test results are prepared for you. At this point you are already informed about all found bugs, as you always have the opportunity to look over our shoulders. We work transparently for our customers.
5. Test conclusion and presentation
Together we will discuss the test results and any other necessary measures. You will receive all the documentation in the form of a final test report, which you need to fix found bugs and vulnerabilities. In addition, you always have the option of accessing our repositories, unless we use your tools. Each job ends with a test exit and a satisfied customer.