Load testing results of s_aturn
Performance rating of complex IT-systems according to DIN 66273 / ISO 14756:
Advantages of using the DIN process model:
- Standardisation of the loadtest measurement
- Comparability of computer systems and applications
- Independent evaluation of the test product using a theoretical reference model
- Validation of the measurement results by a procedure for performance rating, defined in the DIN/ISO, which leads to objective ratios
- Statements about the efficiency of IT systems
- Establishing transparency
Compact results by rating of the DIN values:
|Actual load||B||The actual load B describes the relationship between the number of orders given to a task and the duration of a measurement|
|Reference load||beta||Theoretical reference load defined by the customer|
|Average throughput time||t_AN||Average of the measured throughput time|
|Reference throughput time||t_Ref||Throughput time defined before measurement|
|Throughput of orders that are done on time||E||Number of orders to a task that are done timely in a certain time unit|
|Throughput rating number||L1||The value L1 is calculated from the ratio between the actual load (B) and the reference load defined (beta).|
|Response time rating number||L2||The value L2 is calculated from the ratio between t_Ref and t_AN.|
|On schedule behaviour||L3||The value L3 describes the on schedule behaviour, that means the ratio between the orders that are done on time and the number of given tasks in the measurement time. The value 1.0 means, that throughput time demands are reached up to 100%|
Demonstration of exemplary results from different s_aturn measurements:
- Evaluation of cycles
- Demonstration of the results of an evaluation
- Results of evaluation in table-format
- Graphical results of evaluation
- Response time courses
- Response time histograms
- Response time sum histograms
- Series of measurement
- Distributed measurements
Evaluation of cycles: Task view in different colours in the measurement range
The evaluation of cycles shows the lifelines of the individual users and the use of the cyclic tasks processed by the users.
Course of response time: response times of the finished transactions in the measurement interval
To the abscissa the point of time in the measurement interval is assiged, to the ordinate the achieved response time in seconds. In the diagram of the course bottlenecks through response time summits can be evinced.
Response time histogram: Response times as lines divided in classes
To the abscissa the class division of the response times in seconds is assiged, at the ordinate the number of executed tasks in the respective class is read.
Response time sum histogram: Prozentual number of settled tasks per task unit
To the abscissa the response time in seconds is assiged, to the ordinate the percentage of the total number of tasks of this task type.
Overview of results of a measurement in table form
Detailled tables and most diverse graphs for the particular measurements can be selected.
Measurement results, mean values can also be seen from the mentioned tables as scattering of thinktimes and chain shares, and their confidence intervals.
Table of results of a s_aturn measurement
Evaluation of the measurements:
The color highlighting of the table shows, whether the configured demands are met.
Evaluation of a series of measurement
Selecting a plurality of measurement results in the GUI s_qusi you can bring measurements into comparison with each other.
Thereby overview diagrams are created along with the tables, which include the results of all single measurements already evaluated:
- DIN graphics: Graphical overviews of the DIN/ISO performance indicators of the single measurements
- The histograms of all measurement runs are displayed in one diagram side by side
- The sum histograms of all single measurements are stacked in one graphic, so that a clear overview is given for the objective evaluation of the quality of the response times.
s_aturn can generate, run, execute and evaluate distributed measurements.
Therefore a driver system is defined as master, that takes over the complete control of the measurement. The load is generated and evaluated centrally. The load is started on the slaves. The distribution of the users to be simulated on the slaves is calculated automatically in accordance with the percentages.
Automatic generation of a measurement report
The report describes configuration and components of the system at the time of execution of the measurements. The hardware and software of the systems to be audited including operating system and network infrastructure are described as test object. The figures and tabulars contained in the text document the results of the measurements.
The layout-files are stored in LaTeX -format and can be updated if desired. Thus the report can be designed according to own wishes and can receive new measurement results by an update.
A measuring report includes:
- Definition of measurements: Overview on the structural composition of the load by task-types, task-chains, user-types, quantities about users and timetables. In addition the flow of dynamical data is assigned to the task-types and an overview of the measurements executed is listed. Definition of a theoretical reference model.
- Results of the DIN-measurements: By the color highlighting of the measurement values in the tables you receive a rating of your measurement, whether the demands of the theoretical system are met or a departure from the directives of the model for the reference maschine occurred. This shows trouble spots, that should be analyzed absolutely. If the measurement is based on an adequate reference model, the monitoring data provide indications to the causes in the system, like e.g. bottlenecks in system ressources, processor load and memory.
- Results of the system-monitoring: During the measurements in 5-sec-intervals CPU load, RAM usage and network data are logged. These data help to interpret important associations between the load of the test object and its behaviour.
- Evaluation of the measurement: In the evaluation the results are shown in tables and diagrams as response time course, response time histograms, sum histograms and comparison evaluations between different measurement runs. The evaluation according to DIN is shown graphically in view of the theoretical reference model.
- Attachments for explanations of a measurement according to DIN 66273/ISO 14756 and glossar for the most important terms.