Organisations want to build products that are not just functional but also scalable, performant and secure. To achieve performance goals, one must conduct the load tests as close as possible to the real-world requirements. There are many aspects which must be considered to mimic the real-time traffic like User Mix, Transaction Mix, User Think Times, Browser Mimicking (like, Caching, Cookies, and parallel loading of resources), Server Configurations etc. Let's discuss each of these.
Why Real-World Traffic Matters
As a performance tester, one must simulate the scenarios/workflows in scripting depicting the real-world use cases, otherwise the results of the load test may not be reliable to deploy the products. Missing implementation of user think times could simulate more load than expected with the same number of threads. Similarly, improper caching strategy may result in reporting higher response times than what an actual user will experience.
Factors to be Considered in Simulation
Following is a partial list of factors that act as the critical variables of a workload design in performance test simulation:
User Mix
We have several types of users for the majority of the applications. If we consider a web application representing a school, we have various types of users: Principal, Teachers, Students, Administrators, IT etc. Each user uses the application to perform different tasks.With respect to the number of users as well, the distribution is not even. There are more students than teachers, for example. User mix is to represent the relative distribution of thread count in a performance test simulation, corresponding to different roles. Identifying a realistic user mix is very important. A user mix of 70% students, 20% teachers, 10% admin users, is not right or wrong in itself. Its correctness can be evaluated only in the context of a given application in a given time frame of usage.
Transaction Mix
Today's modern applications have a lot to offer. Tens or hundreds of workflows are available. For Load testing, simulating all the workflows available is a tedious task and not a feasible thing to do as well. A tester will need to identify the most used transactions like, in a school, marking attendance, giving homework, conducting tests, giving grades, travel on school buses etc, as the most used transactions while going on a school trip, marking immunisation, applying leaves are less used transactions though important. Considering the most used transactions for load testing would help in understanding how the product responds to day-to-day activities. These workflows must be spread across different types of users as we discussed in User Mix.
Browser Mimicking
Modern browsers do a lot in the background in order to give the best end user experience like load resources in parallel, caching etc to reduce load times. While most of the tools provide these features by default, one must still know how to enable/disable/configure to use them.
- Parallel Loading of Resources: Browsers load static resources in parallel so the overall load time of the pages decreases. The number of parallel threads used varies across browsers. Chrome and Firefox support 6 threads. Many performance testing tools provide the option to configure this number.
- Caching: Browsers save the static resources (like css, image, js files etc.) locally. When the request for the same static resource arises, they use local copy rather than requesting the server. This reduces the load on the server as well as the network traffic. This arrangement leads to lower latency/response times in loading pages. Majority of the tools provide a Caching feature. One must configure it as per their requirements.
- Cookies: This is an obvious feature to implement without which the functionality itself may not work. Cookies often take care of maintaining the authenticated session of the users, so only authenticated users access the resources on the server. We need to configure Cookie management in the performance testing tools.
User Think Times
While using any application, end users fill forms, or take time to read the page, or to decide what step to do next etc. These actions take time and vary for each user. In Performance testing terms, we call this as “User Think Time”. Filling login form might take 4-8 seconds (again depending on user’s speed of typing and mood), reading a small paragraph may take 15-60 seconds depending on content size, which is a must in order to take the next step like Accept/Agree etc. During Scripting, one must implement User Think Times, otherwise, we may end up simulating more load than anticipated for a specific user load, as more requests are generated in the same unit of time. Many tools allow the use of random sleep times in a range, to achieve the realistic behaviour of different end users using the application.
Server Configurations
While we discussed client-side concepts, equally important is the correct server configuration. The server configuration must be as close as possible to the production environment. It could be resources like CPU, RAM, Storage, Network Bandwidth etc. Or configurations like threads, timeouts, load balancing etc. All these configurations must be as per production requirements, otherwise, the load test results may not be reliable.
Concluding Thoughts
The motto of any performance tester is to simulate the load and conduct the test as close as possible to the real-time traffic. This enables accurate results which can be trusted to use in deployment of the production environment. Majority of the performance testing tools support all the configurations discussed above. These concepts and factors are tool-agnostic and can be followed by testers across applications irrespective of their functional context.
If you would like a workshop on performance testing for you and your team, you are welcome to contact Naveen and our engineering colleagues.
