I am sure anyone running Selenium tests run into issues with long test execution times. Once we got to a thousand or so test cases it would take multiple hours to run the test suites. We wanted to speed it up. Some notes below:
- We were running all tests with one specific user login per DB. If we tried to log in at the same time the program would recognize it and bump the earlier log in out. Solution for that was to create separate users for each test suite. That would mean that the test suites could run in parallel as different users could be logged in to the same DB at once.
- Our regular login method pulled a user from a JSON file unique for tester’s DB. We had to create test suite users for all DBs. A small script helped with that, not much time lost. When someone runs the automation set up script for their DB it adds these now.
- We included a setting where the tester running the suites could specify if they wanted originally specified user to run these, or use the test suite based users.
- To run in parallel we could just start the tests in PyCharm (for example) one after the other and it would start separate phantomjs instances and execute test suites in parallel.
- From command line, it was even easier as we usually run these from Linux machines and we would run with ampersand at the end, something like : nosetests sometestshere.py &
- Sixteen test suites that used to take 3-4 hours now take 30-45 minutes.
- Since this is dependent on the tester’s machine power, we usually run 5-7 test suites at once,then run the other 5-7 and similar. Otherwise a noticeable slowdown is observed.
- The main goal for this was to allow testers to run tests quicker in the simplest possible way with minimal training and it totally worked.
- We also have docker set ups that can run all 16 test suites at once, but that’s another post.