This assignment was part of the Fundamentals of Software Testing study unit, therefore the main aim of this assignment was to explore and try out different testing techniques and tools which include:
- Unit Testing
- Selenium (Automated Web Testing)
- Cucumber
- Model-Based Performance Testing
I’ve worked on this assignment with my good friend Jake Dalli. Check out my GitHub for our code!
Overview
The main idea of this website was that we were working for a new online betting company which allows users to place blind bets on sports events without them needing to know what sport or team their money will be bet on. The company employs expert betting analysts to place bets on behalf of users. In effect, a user can place a bet of (for example) EUR 5 and a betting analyst might split that into three different bets to spread the risk across different events. We are required to develop an initial version of the website using a 3-tier approach with the following requirements:
- User Registration
- Certain rules did apply, including credit card number verification for American Express, Mastercard and Visa. This was done using the Luhn Algorithm.
- Log In
- If the user enters an incorrect password three times, the account is locked for 5 minutes.
- Placing bets
- certain betting restrictions applied to whether the user was free or premium.
- Log out
Site Implementation
The website was implemented using a combination of Java, Java Server Pages(JSP) and jQuery. We used MySQL for the database and Apache Tomcat for our web server.
The website was designed to work on two pages, one for users to login or register, and the other to place bets. All error messages are displayed on these pages. A concept diagram can be seen below:
Three-Tier Approach
A three-tier approach was taken:
- Data tier – intended to interact with the database.
- Presentation tier – intended to display and receive data.
- Logic tier – intended to commit to all the processing between the two outer tiers.
This organised structure allowed unit testing to solely deal with the logic tier, thus the application was designed with testing in mind.
Database schema
The database schema created for this web application is found below:
Implementation Results
The screenshots below show the resultant website:
Testing
The unit testing component of this assignment solely deals with the unit testing of the logic tier of the web application. This is due to the fact that tests for the data tier are not classified as unit tests, but as integration tests. Moreover, the presentation layer experiences testing within the Testing Automation section of this document.
Unit Testing
Unit tests were implemented using the JUnit Framework. Since we were using Java and the Eclipse IDE, we decided to use EclEmma for code coverage. We achieved a test coverage of 82.9%, which was considered a desirable result given that this metric includes DAO (Data Access Objects) which were not explicitly targeted within the test.
However, with the interest of generating a more honest metric which directly addresses the logic tier, an additional test coverage report was generated excluding selenium tests – rendering a 75.6% test coverage. This was also deemed to be a satisfactory result, since all major branches were covered and the targeted classes, being controllers, achieved a coverage of 89.6%.
Several test strategies were used including the use of Mock Objects, Test stubs and Test spies. Test patterns include Dependency, Constructor and parameter injections.
Automated Web Testing
The automated web test cases, found in src/java/test/selenium make use of Cucumber scenarios in order to run Junit tests which make use of the selenium framework. Moreover, a number of classes were developed to encapsulate the selenium framework – these are found under GeneralFunctions.java; since encapsulation is outside the scope of the testing, this document shall not list the exact method stubs used within this function.
Several test scenarios were defined, and can be found within the resources.feature file.
Model-Based Testing
The model-based test cases found under src/model/com/testmodel/site/bt implement an FSM model using ModelJUnit in order to build a model of user behaviour and measure performance by analysing response times. Below is a diagram of the implemented Model.
Several tests where made with the aim of determining which test case performed the best, either using AllRoundTester, LookAheadTester, GreedyTester or RandomTester. Each Test was run with 3 concurrent users. From our results it seems that the All Round Tester has the most coverage while Lookahead has the least. Lookahead took the least amount of time while Random Tester took over half an hour. The best performance however was with Look Ahead at around half a second. All Round Tester had the longest response time at 1.9 seconds. Deciding on which test case to choose ultimately depends on the model being used. If coverage is not important, yet the model will be given a lot of users (therefore performance is important), then Look Ahead would be ideal. However if we are required to obtain the most coverage, then All Round Tester would be the way to go.
Patrick Buhagiar 