Test factory as a part of continuous integration using Selenium 1

Brief

The factory is a bunch of most useful open source tools aimed to improve product quality. In conjunction with best testing practices it simplifies QA process and significantly reduces costs for maintenance phase.

Architecture

• functional/regression testing is powered by Hudson
• tests are front-end based and driven by Selenium
• possibility to run automated tests in simultaneous multithreading mode on different environments using Selenium Grid
• alarm notification when a build or tests fail
• ability to capture screenshots for failed tests
• testing can be performed not for all but only for certain functional areas; such option is available on start testing page
• test scripts are environment independent (same script can be performed on various environments like dev, staging, prod)
• test scripts are country independent (same script can be performed on various country specific instances; total 19 countries)
• test scripts are browser independent (different browsers are supported like FireFox, Chrome and IE)
• test report has human-readable format including error message, link to respective screenshot, stack trace, scenario to reproduce
• test is self-documented; every failure description contains scenario (steps) performed before the issue occurred
• test framework contains customized exceptions (exceptions are handled by test framework, every stack trace contains appropriate exception defining the reason of the failure)
• emailing feature is tested by using base mail protocols (SMTP, IMAP)
• tests are performed for both full and mobile sites
• ability to mark any test to run if certain condition is true (for instance the test is environment or browser specific)
• capability to run individual tests (dynamically create/update set of tests to be performed in the next test run)
• authorization stage is enabled for the test factory
• ability to perform tests using different branches (branches with test scripts are synchronized to release branches)
• regression testing is performed on a nightly basis and under various conditions
• core of the factory is common for certain projects
• ability to run tests for specific revision
• ability to specify different log levels for output