Performance testing plays an important role in software quality assurance, and it includes a rich variety of test content. The China Software Evaluation Center summarizes the performance test into three aspects: the test of the performance of the application on the client, the test of the performance of the application on the network and the test of the performance of the application on the server. Under normal circumstances, an effective and reasonable combination of the three aspects can achieve a comprehensive analysis of system performance and prediction of bottlenecks.
The purpose of the application performance test on the client is to examine the performance of the client application, and the entrance of the test is the client. It mainly includes concurrent performance testing, fatigue strength testing, large data volume testing and speed testing, among which concurrent performance testing is the focus.
Concurrent performance testing is the key point
The process of concurrent performance testing is a process of load testing and stress testing, that is, gradually increasing the load until the system bottleneck or unacceptable performance points. The process of determining the concurrent performance of the system by comprehensively analyzing transaction execution indicators and resource monitoring indicators. Load testing (LoadTesting) is to determine the performance of the system under various workloads. The goal is to test the corresponding output items of the system components when the load gradually increases, such as throughput, response time, CPU load, memory usage, etc. to determine the system Performance. Load testing is a process of analyzing software applications and supporting structures and simulating the use of real environments to determine acceptable performance. Stress Testing is a test to obtain the maximum service level that the system can provide by determining the bottleneck or unacceptable performance points of a system.
The purpose of concurrent performance testing is mainly reflected in three aspects: based on real business, select representative and key business operation design test cases to evaluate the current performance of the system; when expanding the application When the function of the program or a new application is about to be deployed, load testing will help determine whether the system can still handle the expected user load to predict the future performance of the system; by simulating hundreds of users, repeating and running the test, Can confirm the performance bottleneck and optimize and adjust the application, the purpose is to find the bottleneck problem.
When an enterprise organizes its own strength or entrusts a software company to develop an application system on its behalf, especially when it is actually used in a production environment in the future, users often have questions about whether this system can withstand a large number of applications. Concurrent users access at the same time? This type of problem is most common in systems such as online transaction processing (OLTP) database applications, Web browsing, and video-on-demand. The solution of this kind of problem requires the help of scientific software testing methods and advanced testing tools.
Illustration: Telecom billing software
As we all know, around the 20th of each month is the peak period for local calls, and thousands of toll outlets in the city are activated at the same time. The charging process is generally divided into two steps. First, according to the user's phone number, the user needs to inquire about the expenses incurred in the current month, and then collect the cash and modify the user to the paid state. A user seems to have two simple steps, but when hundreds of thousands of terminals perform such operations at the same time, the situation is very different. So many transactions happen at the same time, which is very important for the application itself, the operating system, and the central database server. The endurance of middleware servers and network equipment is a severe test. It is impossible for decision makers to consider the endurance of the system after a problem occurs, and foresee the concurrent endurance of the software. This is a problem that should be solved in the software testing stage.
Most companies and enterprises need to support hundreds of users, various application environments and complex products assembled from components provided by different suppliers, unpredictable user loads and increasingly complex applications The program has caused the company to worry about problems such as poor delivery performance, slow user response, and system failure. The result is a loss of company revenue.
How to simulate the actual situation? Find several computers and the same number of operators to operate at the same time, and then use a stopwatch to record the reaction time? Such hand-workshop-style testing methods are impractical and cannot capture the internal changes of the program. This requires the assistance of stress testing tools.
The basic strategy of testing is automatic load testing. It tests the application by simulating hundreds or thousands of virtual users on one or several PCs to execute the business at the same time. A transaction processing time, middleware server peak data, database status, etc. Repeatable and realistic testing can thoroughly measure the scalability and performance of the application, identify the problem and optimize system performance. Knowing the endurance of the system in advance provides a strong basis for the end user to plan the configuration of the entire operating environment.
Preparation work before concurrent performance test
Test environment: Configuring the test environment is an important stage of test implementation. The suitability of the test environment will seriously affect the authenticity of the test results and Correctness. The test environment includes the hardware environment and the software environment. The hardware environment refers to the environment constituted by the servers, clients, network connection devices, and auxiliary hardware devices such as printers/scanners necessary for the test; the software environment refers to the operating system and database when the software under test is running And other application software.
A well-prepared test environment has three advantages: a stable and repeatable test environment that can ensure the correct test results; ensure that the technical requirements for test execution are met; ensure that the correct and repeatable And easy-to-understand test results.
Test tool: Concurrent performance test is a black box test performed on the client side. Generally, it does not use manual methods, but uses tools to conduct automated methods. There are many mature concurrent performance testing tools, and the selection is mainly based on testing requirements and performance-price ratio. Well-known concurrent performance testing tools include QALoad, LoadRunner, BenchmarkFactory and Webstress. These test tools are all automated load test tools. Through repeatable and real tests, they can thoroughly measure the scalability and performance of the application. They can perform test tasks automatically throughout the development life cycle, across multiple platforms, and can be simulated. Hundreds or thousands of users concurrently execute key services to complete the test of the application.
Test data: In the initial test environment, you need to enter some appropriate test data. The purpose is to identify the state of the data and verify the test case used for testing. The test case is debugged before the formal test starts. Minimize errors at the beginning of the formal test. When the test progresses to the key process link, it is very necessary to back up the data state. Manufacturing initial data means storing the appropriate data and restoring it when needed. The initial data provides a baseline to evaluate the results of test execution.
When the test is formally executed, business test data is also required, such as testing concurrent query business, then the corresponding database and table are required to have a considerable amount of data and the type of data should be able to cover all businesses.
Imitate the real environment test, some software, especially the commercial software for the general public, often need to examine the performance in the real environment when testing. For example, when testing the scanning speed of anti-virus software, the proportion of different types of files placed on the hard disk should be as close as possible to the real environment, so that the tested data has practical meaning.
Types and indicators of concurrent performance testing
Using automated load testing tools to perform concurrent performance testing, the basic testing process followed are: test requirements and test content, test case formulation, test environment preparation, test script recording, writing and debugging, script distribution, Playback configuration and loading strategy, test execution tracking, result analysis and location of problems, test report and test evaluation.
The objects of concurrent performance testing and monitoring are different, and the main indicators of the test are also different. The main test indicators include transaction processing performance indicators and UNIX resource monitoring. Among them, transaction processing performance indicators include transaction results, number of transactions per minute, transaction response time (Min: minimum server response time; Mean: average server response time; Max: maximum server response time; StdDev: transaction server response deviation, value The greater the value, the greater the deviation; Median: median response time; 90%: server response time for 90% transaction processing), the number of virtual concurrent users.
Application example: "Xinhua News Agency Multimedia Database V1.0" Performance Test
China Software Testing Center (CSTC) according to the "Multimedia Database (Phase I) Performance" proposed by Xinhua News Agency Technology Bureau "Test Requirements" and GB/T17544 "Package Quality Requirements and Testing" national standards, using industry standard load testing tools to perform performance tests on the "Xinhua News Agency Multimedia Database V1.0" used by Xinhua News Agency.
The purpose of the performance test is to simulate multiple users concurrently accessing the Xinhua News Agency multimedia database, perform key retrieval services, and analyze system performance.
The focus of performance testing is to conduct concurrent testing and fatigue testing for the main retrieval business that has a large concurrent pressure load. The system adopts the B/S operation mode. Concurrent testing is designed to perform concurrent test cases in the Chinese library, English library, and image library in a specific time period, such as single search terms, multiple search terms, variable search formulas, and mixed search services. The fatigue test case is 200 concurrent users in the Chinese database, and a single search term search with a test period of about 8 hours. While conducting concurrency and fatigue tests, the monitored test indicators include transaction processing performance and UNIX (Linux), Oracle, Apache resources, etc.
Test conclusion: In the Xinhua News Agency computer room test environment and intranet test environment, under the condition of 100M bandwidth, for each specified concurrent test case, the system can withstand the load pressure of 200 concurrent users, and the maximum transaction The number/minute reaches 78.73, the operation is basically stable, but as the load pressure increases, the system performance is attenuated.
The system can withstand the fatigue stress of 200 concurrent users for approximately 8 hours in a cycle, and can basically operate stably.
Through the monitoring of system UNIX (Linux), Oracle and Apache resources, system resources can meet the above-mentioned concurrency and fatigue performance requirements, and system hardware resources still have a lot of room for utilization.
When the number of concurrent users exceeds 200, HTTP500, connect, and timeout errors are monitored, and the Web server reports a memory overflow error. The system should further improve the performance to support a larger number of concurrent users.
It is recommended to further optimize the software system, make full use of hardware resources, and shorten the transaction response time.
Fatigue strength and large data volume test
Fatigue test is to use the maximum number of concurrent users that the system can support under stable operation of the system, continue to execute business for a period of time, and comprehensively analyze transaction execution indicators And resource monitoring indicators to determine the process of the system to handle the maximum workload intensity performance.
Fatigue strength testing can be carried out in a tool-automated way, or it can be manually programmed for testing, of which the latter accounts for a larger proportion.
Under normal circumstances, the maximum number of concurrent users that the server can normally and stably respond to requests is used to perform a fatigue test for a certain period of time to obtain transaction execution index data and system resource monitoring data. If an error occurs and the test cannot be successfully executed, the test indicators should be adjusted in time, such as reducing the number of users and shortening the test cycle. In another case, the fatigue test is to evaluate the performance of the current system, using the number of concurrent users under normal business conditions of the system as the basis for a fatigue test for a certain period of time.
Big data volume tests can be divided into two types: independent large data volume tests for certain system storage, transmission, statistics, and query services; and stress performance testing, load performance testing, A comprehensive data volume testing program combined with fatigue performance testing. The key to large-data-volume testing is the preparation of test data, and you can rely on tools to prepare test data.
Speed test is mainly to manually measure the speed for the key business with speed requirements. The average value can be calculated on the basis of multiple tests, and it can be compared and analyzed with the response time measured by the tool.
The focus of the application on the network performance test is to use mature and advanced automation technology for network application performance monitoring, network application performance analysis and network prediction.
Network application performance analysis
The purpose of network application performance analysis is to accurately show how changes in network bandwidth, latency, load, and TCP ports affect user response time. Using network application performance analysis tools, such as ApplicationExpert, can find application bottlenecks. We can know the application behavior that occurs at each stage when the application is running on the network, and analyze application problems at the application thread level. Many problems can be solved: Does the client run unnecessary requests to the database server? When the server receives a query from the client, does the application server spend unacceptable time contacting the database server? Predict the response time of the application before it is put into production; use ApplicationExpert to adjust the performance of the application on the WAN; ApplicationExpert allows you to quickly and easily simulate the performance of the application. According to the response time of the end user in different network configuration environments, users can follow their own conditions Decide the network environment in which the application will be put into production.
Network application performance monitoring
After the system is commissioned, it is necessary to know in time and accurately what is happening on the network; what applications are running and how to run; how many PCs are accessing the LAN or WAN: Which applications cause system bottlenecks or resource competition? At this time, network application performance monitoring and network resource management are critical to the normal and stable operation of the system. Using network application performance monitoring tools, you can achieve twice the result with half the effort. In this regard, the tool we can provide is NetworkVantage. In layman's terms, it is mainly used to analyze the performance of key applications and locate the source of the problem on the client, server, application or network. In most cases, users are more concerned about which applications take up a lot of bandwidth and which users generate the largest network traffic. This tool can also meet the requirements.
Considering the future expansion of the system, it is very important to predict the impact of changes in network traffic and changes in network structure on user systems. Make predictions based on planning data and provide network performance prediction data in a timely manner. We use the network predictive analysis capacity planning tool PREDICTOR to: set service levels, complete daily network capacity planning, offline test network, network failure and capacity limit analysis, complete daily fault diagnosis, predict network equipment migration and network equipment upgrades to the entire network Impact.
Get the network topology structure from the network management software, and obtain the flow information from the existing flow monitoring software (if there is no such software, the flow data can be manually generated), so that the basic structure of the existing network can be obtained. Based on the basic structure, reports and graphs can be generated according to changes in network structure and changes in network traffic to illustrate how these changes affect network performance. PREDICTOR provides the following information: According to the predicted results, it helps users to upgrade the network in time to avoid system performance degradation caused by key equipment exceeding the utilization threshold; which network equipment needs to be upgraded, which can reduce network delays and avoid network bottlenecks; Necessary network upgrades.
For testing the performance of the application on the server, you can use tool monitoring, or you can use the monitoring commands of the system itself. For example, in Tuxedo, you can use the Top command to monitor resource usage . The purpose of the implementation test is to achieve a comprehensive monitoring of the server equipment, server operating system, database system, and application performance on the server. The test principle is as shown in the figure below.
UNIX resource monitoring indicators and descriptions
Monitoring indicators description
The average number of synchronization processes in the last 60 seconds in the normal state of the load average system
Conflict rate The number of conflicts per second monitored on the Ethernet
Process/thread exchange rate Process and thread exchanges per second
CPU utilization CPU occupancy rate ( %)
Disk exchange rate Disk exchange rate
Receive packet error rate The number of errors per second when receiving Ethernet data packets
Packet input rate Input per second Number of Ethernet packets
Interrupt rate The number of interrupts processed by the CPU per second
Output packet error rate The number of errors per second when sending Ethernet packets
Packets Input rate The number of Ethernet packets output per second
The rate of reading memory pages per second in the physical memory The number of memory pages read per second in the physical memory
The rate of writing out memory pages per second from the physical memory The number of memory pages written to the page file in the memory
The number of memory pages deleted from physical memory
The exchange rate of memory pages written to memory pages and slaves per second The number of read pages in the physical memory
The number of processes that enter the exchange rate exchange area.
The number of processes that process out of the exchange rate exchange area
System CPU Utilization System CPU Occupancy Rate (%)
User CPU Utilization CPU Occupancy Rate in User Mode (%)
Disk blocking The number of bytes per second blocked by disk
The purpose is to verify whether the software system can reach the performance indicators proposed by the user, and to find the performance bottlenecks in the software system, optimize the software, and finally achieve the purpose of optimizing the system.
Include the following aspects
1. To evaluate the capabilities of the system, the load and response time data obtained in the test can be used to verify the capabilities of the planned model and help make decisions.
2. Identify weaknesses in the system: The controlled load can be increased to an extreme level and break through it, thereby repairing bottlenecks or weaknesses in the system.
3. System tuning: Repeatedly run the test to verify that the activities of tuning the system get the expected results, thereby improving performance.
Detect problems in the software: Long-term test execution can cause the program to fail due to memory leaks, revealing hidden problems or conflicts in the program.
4. Verify the stability (resilience) reliability (reliability): Performing a test for a certain period of time under a production load is the only way to assess whether the system stability and reliability meet the requirements.
The performance test types include load test, strength test, capacity test, etc.
Load testing (LoadTesting): Load testing is mainly to test whether the software system meets the goals of the requirements document design, such as the maximum number of concurrent users supported by the software in a certain period of time, the error rate of software requests, etc. , The main test is the performance of the software system.
Stress Testing: Stress testing is also stress testing. Stress testing is mainly to test whether the hardware system meets the performance goals of the requirements document design, such as the system's cpu utilization and memory in a certain period of time. Utilization rate, disk I/O throughput rate, network throughput, etc. The biggest difference between stress test and load test is the test purpose.
Volume Testing: Determine the maximum capacity of the system, such as the maximum number of users of the system, the maximum storage capacity, and the most processed data flow.
The performance test includes the following test types:
Benchmark test-compares the performance of new or unknown test objects with known reference standards (such as existing software or evaluation standards).
Contention test:-Verify whether the test object can accept multiple protagonists' requests for the same resources (data records, memory, etc.).
Performance configuration-to verify the acceptability of the performance behavior of the test subject when using different configurations under the condition that the operating conditions remain unchanged.
Load test-to verify the acceptability of the performance behavior of the test object under different operating conditions (such as different number of users, number of transactions, etc.) while keeping the configuration unchanged.
Strength test-to verify the acceptability of the test subject's performance behavior under abnormal or extreme conditions (such as reduced resources or excessive number of users).
Capacity test-to verify the maximum number of software programs used by test users at the same time.
Performance evaluation is usually performed in collaboration with user representatives and in a multi-level method.
The first level of performance analysis involves the evaluation of the results of a single protagonist/use case instance and the comparison of the results of multiple test executions. For example, when there is no other activity on the test object, record the performance behavior of a single actor executing a single use case, and compare the results with several other test executions of the same actor/use case. The first level of analysis helps to identify trends that can indicate contention in system resources that will affect the validity of conclusions drawn from other performance test results.
The second level of analysis examines the summary statistics and actual data values executed by a specific protagonist/use case, as well as the performance behavior of the test object. Summary statistics include the standard deviation and percentile distribution of response time. This information shows how the system's response has changed, just as each protagonist sees it.
The third level of analysis helps to understand the causes and weights of performance problems. This detailed analysis uses low-level data and uses statistical methods to help testers draw correct conclusions from the data. Detailed analysis provides objective and quantitative criteria for decision-making, but it takes a long time and requires a basic understanding of statistics.
When differences in performance and behavior do exist, or are caused by some random events related to the collection of test data, detailed analysis uses the concept of statistical weighting to help understand. That is to say, at the basic level, any event is random. Statistical testing determines whether there are systematic differences that cannot be explained by random events.
Performance testing mainly uses automated testing tools to simulate a variety of normal, peak and abnormal load conditions to test various performance indicators of the system. Load testing and stress testing are both performance testing, and the two can be combined. Through load testing, determine the performance of the system under various workloads. The goal is to test the changes in various performance indicators of the system when the load gradually increases. Stress testing is a test to obtain the maximum service level that the system can provide by determining the bottleneck or unacceptable performance points of a system.
In actual work, we often test two types of software: bs and cs. What are the performance indicators of these two aspects?
The general indicators that Bs structure programs generally pay attention to are as follows (abbreviated):
Web server indicators:
*AvgRps: average number of responses per second= Total request time/seconds;
*Avgtimetolastbyteperterstion(mstes): The average number of iterations of business scripts per second. Some people will confuse the two;
*SuccessfulRounds: Successful requests ;
*FailedRounds: failed requests;
*SuccessfulHits: successful clicks;
*FailedHits: failed clicks;
*HitsPerSecond: the number of clicks per second;
*SuccessfulHitsPerSecond: the number of successful clicks per second;
*FailedHitsPerSecond: the number of failed clicks per second;
*AttemptedConnections: the number of attempts to connect;
CS structure program, because the general software background is usually a database, we pay more attention to the test indicators of the database:
*User0Connections: the number of user connections, That is, the number of database connections;
*Numberofdeadlocks: database deadlock;
*BufferCachehit: database cache hits
Of course, in practice, we still Will observe the memory, CPU, and system resource usage under the multi-user test. These indicators are actually one of the extended performance tests: competition tests. What is a competitive test? The software competes to use various resources (data records, memory, etc.), depending on its ability to compete for resources with other related systems.
We know that software architecture restricts the choice of testing strategies and tools in actual testing. How to choose a performance test strategy is what we need to understand in actual work. General software can be divided into several types according to the system architecture:
client/Server client/server architecture
Based on the client/server three Layer architecture
Client/server-based distributed architecture
Browser/Web server-based three-tier architecture
Three-tier architecture based on middleware application serverl
Multi-tier architecture based on Web server and middlewarel
In the implementation of the system architecture, developers may choose different implementation methods, resulting in complex and complicated actual situations. It is impossible for us to explain each technology in detail. This is just to introduce a method to provide you with how to choose a test strategy, so as to help analyze the performance indicators of different parts of the software, and then analyze the performance indicators and performance bottlenecks of the overall architecture.
Because of the differences between projects and projects, the selected metrics and evaluation methods are also different. But there are still some general steps to help us complete a performance test project. The steps are as follows
1. Setting goals and analysis system
2. Choose the method of test measurement
3. Related technologies and tools for learning
4. Develop evaluation criteria
5. Design test cases
6. Run test cases
7. Analyze test results
Develop goals and analyze system
The first step in every performance test plan is to formulate goals and analyze system composition. Only by clarifying the goal and understanding the structure of the system can the scope of the test be clarified and know what kind of technology to master in the test.
1. Determine customer needs and expectations
2. Actual business requirements
3. System requirements
System composition includes several meanings: system category, system composition, system function, etc. Understanding the nature of these contents actually helps us to clarify the scope of the test and choose the appropriate test method to perform the test.
System category: distinguishing the system category is the prerequisite for what kind of technology we have mastered, and the performance test can be successful if we master the corresponding technology. For example: the system category is a bs structure, and you need to master http protocol, java, html and other technologies. Or the cs structure, you may need to understand the operating system, winsock, com, etc. Therefore, the classification of the system is very important for us.
System composition: Hardware settings and operating system settings are the constraints of performance testing. Generally, performance testing uses testing tools to imitate a large number of actual user operations, and the system operates under overload conditions. Different system composition performance tests will get different results.
System function: System function refers to the different subsystems provided by the system, the official document subsystem in the office management system, the conference subsystem, etc. The system function is the link to be simulated in the performance test. It is necessary to understand these .
Choose the method of test measurement
After the first step, you will have a clear understanding of the system. Next, we will focus on software metrics and collect system-related data.
Related aspects of measurement:
*Develop relevant processes, roles, responsibilities
*Develop improvement strategiesp>
*Develop results comparison standards
Related technologies and tools learned
Performance testing is to use tools to simulate a large number of user operations and increase the load on the system. Therefore, a certain knowledge of tools is required to perform performance testing. Everyone knows that performance testing tools generally record user operations through winsock, http and other protocols. The protocol selection is based on the implementation of the software system architecture (web generally chooses http protocol, cs chooses winsock protocol), and different performance testing tools have different scripting languages. For example, the vu script in rationalrobot is implemented in c-like language.
To carry out performance testing, various performance testing tools need to be evaluated, because each performance testing tool has its own characteristics. Only after tool evaluation, can performance testing tools that conform to the existing software architecture be selected. After determining the test tools, you need to organize testers to learn about the tools and train related technologies.
Develop evaluation criteria
The purpose of any test is to ensure that the software meets the pre-specified goals and requirements. Performance testing is no exception. Therefore, a set of standards must be developed.
Generally, there are four model technologies for performance testing:
*Linear projection: use a large amount of past, extended or future data to form a scatter diagram, use this The chart is constantly compared with the current state of the system.
*Analysis model: Use queuing theory formulas and algorithms to predict response time, and use data describing workloads to correlate with the nature of the system
*Imitation: imitate actual users’ usage methods to test you System
*Benchmark: Define the test and your initial test as a standard, and use it to compare with all subsequent test results
Design test cases
Designing test cases is based on understanding the software business process. The principle of designing test cases is to provide the most test information with the least impact. The goal of designing test cases is to include as many test elements as possible at a time. These test cases must be achievable by the test tool, and different test scenarios will test different functions. Because performance testing is different from the usual test cases, it is possible to find the performance bottleneck of the software by designing the performance test cases as complex as possible.
Run test cases
Run test cases through performance testing tools. The test results obtained from the performance test under the same environment are inaccurate, so when running these test cases, you need to use different test environments and different machine configurations to run.
Analyze the test results
After running the test case, collect relevant information, perform statistical analysis of the data, and find the performance bottleneck. By eliminating errors and other factors, the test results are close to the real situation. Different architectures have different methods of analyzing test results. For the bs structure, we will analyze the network bandwidth and the impact of traffic on user operation response, while for the cs structure, we may be more concerned about the impact of the overall system configuration on user operations.
For enterprise applications, there are many methods for performance testing, some of which are more difficult to implement than others. The type of performance test to be performed depends on the desired result. For example, for reproducibility, benchmarking is the best method. To test the upper limit of the system from the perspective of current user load, capacity planning testing should be used. This article will introduce several methods for setting up and running performance tests, and discuss the differences between these methods.
Without reasonable planning, performance testing of J2EE applications will be a daunting and somewhat confusing task. Because for any software development process, it is necessary to collect requirements, understand business needs, and design a formal schedule before actual testing. The requirements for performance testing are driven by business needs and clarified by a set of use cases. These use cases can be based on historical data (for example, the load pattern of the server for a week) or predicted approximations. After you figure out what needs to be tested, you need to know how to test.
In the early stages of the development phase, benchmark testing should be used to determine whether performance regressions occur in the application. Benchmarking can collect repeatable results in a relatively short period of time. The best way to benchmark is to change one and only one parameter per test. For example, if you want to know whether increasing the JVM memory will affect the performance of the application, you can increase the JVM memory gradually (for example, from 1024MB to 1224MB, then 1524MB, and finally 2024MB), collect the results and environmental data at each stage, and record Information, and then go to the next stage. In this way, there are traces to follow when analyzing the test results. In the next section, I will introduce what a benchmark is and the best parameters for running a benchmark.
In the late development phase, bugs in the application have been resolved. After the application reaches a stable state, more complex tests can be run to determine the performance of the system under different load patterns. These tests are called capacity planning tests, penetration tests (soaktest), peak-rest tests (peak-resttest), and they aim to test scenarios close to the real world by testing the reliability, robustness, and scalability of the application.对于下面的描述应该从抽象的意义上理解，因为每个应用程序的使用模式都是不同的。例如，容量规划测试通常都使用较缓慢的ramp-up（下文有定义），但是如果应用程序在一天之中的某个时段中有快速突发的流量，那么自然应该修改测试以反映这种情况。但是，要记住，因为更改了测试参数（比如ramp-up周期或用户的考虑时间（think-time)），测试的结果肯定也会改变。一个不错的方法是，运行一系列的基准测试，确立一个已知的可控环境，然后再对变化进行比较。