testRTC collects and analyzes a lot of different data points and metrics. To manage that information, testRTC also offers various scoring values for tests and collected monitors data. When opening test results or monitor information, you can find the score values at the top ribbon bar of the results:
If you are using testingRTC or upRTC, then this information block includes Score and MOS. For watchRTC this information block includes also User.
The Score value is a number between 0-10 giving the overall scoring of the scenario.
It looks at both audio and video, across all channels, penalizing them from a score of 10 based on the media metrics values as well as the variance of these values over time.
It is an objective score that is mainly useful for baselining service performance. Deciphering if a score value is good or bad is left to the judgement of the testRTC user based on his understanding and experience with his own application.
Learn more about test scoring in testingRTC.
MOS score stands for Mean Opinion Score. It looks only at the audio channels in a scenario. It is a widely used metric.
MOS is a value between 1-5 indicating a subjective quality measurement.
testRTC derives its MOS score calculations indirectly from RTCP reports that are exposed in WebRTC via the getStats() APIs. We rely on jitter, packet loss, round trip time and codec type to calculate it (we figure out the R-factor and from there derive the MOS score value).
User scoring is available in watchRTC. It enables developers to collect user rating and store them in watchRTC directly.
You can use the watchRTC SDK‘s watchRTC.setUserRating() API to report user rating information.
Quality of a scenario
When more than a single user or device is part of the scenario, the scoring information of the whole scenario is calculated as the average score across the relevant devices.
- Average score relates to Score, MOS and User values
- Relevant devices are all the devices in the scenario that have that score value. For example, if a single User in a meeting of 10 participants provided a user rating, then his rating will be the User rating given to the whole scenario