Mobile Network Report Methodology

Data collection

The data collected by Weplan is crowdsourced through an SDK introduced in smartphone apps. Currently, at Weplan Analytics we measure anonymously information on the quality of the network and the coverage of all mobile operators, through hundreds of apps and millions of devices worldwide. We reach partnership agreements with these apps to implement our SDK in their code, we pay them for active users and we, therefore, get anonymous network data provided by Smartphone users.

Our Key Performance Indicators

Connectivity: Disconnection time and 4G and 5G time

A mobile phone user may have three possible connection status: Connected, Limited Connection and No Coverage.

  • Connected includes all the time that a user has proper connection to its MNO’s network. It can be further decomposed into 2G, 3G, 4G and 5G, depending on the technology.
  • Limited Connection includes all the time that a user has the connectivity enough to make emergency calls, either under its own MNO or under another MNO’s coverage.
  • No Coverage includes all the time that a user does not have connectivity at all.

The KPI Disconnection time provides the average of daily minutes that the users of an MNO have Limited Connection or No Coverage, that is, out of the 24 hours of the day, how many total minutes on average were spent without coverage by the users of an MNO.

Similarly, the KPI 4G and 5G time provides the percentage of time (on average) that the users of an MNO have 4G or 5G connectivity. In this case, 5G includes both 5G SA and 5G NSA.

5G Penetration and Coverage

This KPI provides two insights: first, the percentage of clients by MNO that have actual 5G coverage. Second, the percentage of time that a client with 5G access is under 5G coverage. Both 5G SA and 5G NSA are considered for these figures. When a country does not have 5G coverage, this KPI is not provided.

It must be kept in mind that the ability of a customer to use the 5G network depends on several factors. First, their device must be 5G compatible. Second, the MNO must have a 5G network. Finally, the client must contract a mobile plan that allows the 5G to be used, as not all MNOs offer full access to the 5G network as part of the regular plans. If a client has connected to the 5G network (either SA or NSA) at least once it is included in the statistic.

A client may detect a 5G NSA network without connecting (due to it being a test network, the client not having a 5G rate but a 5G-capable phone, or other circumstance). Although internally the device may report a 5G NSA connection (more precisely, a 5G NSA RESTRICTED connection), the customer will not see it reflected in its phone (it will mark a 4G connection) or its usage. This type of 5G NSA connections have not been considered when determining if a customer is a 5G client or the 5G coverage time.

Mobile network situation

Determining the performance of a network is a multivariate and nuanced problem. But checking the signal strength and quality of a network is a great approach to get a solid insight of the network status for an MNO. Unless otherwise stated, this analysis is performed only in 4G and 5G (both SA and NSA) networks, as they are the most used networks overall.

Analyzed signal strength and quality indicators

Each technology has its own parameters and signal strength and quality indicators. In the case of 4G, the signal strength is provided by the RSRP, and the signal quality by the RSRQ. For 5G, the signal strength is provided by the NR-RSRP and the signal quality by the NR-RSRQ. Both networks have approximately the same value range for their indicators, so they can be compared directly.

Although there are some standard thresholds for these values when determining what is a good or bad signal strength or quality, in practice each MNO or TowerCo has its own parameters, based on their analysis and experiences. Furthermore, different countries exhibit different distributions of these indicators. To ensure a fair analysis, the thresholds for determining wether a signal strength or signal quality indicator is better or worse have been determined country by country using percentiles:

  • If the indicator is under the 5th national percentile, the indicator is Very Degraded.
  • If the indicator is between the 5th and the 25th national percentile, the indicator is Degraded.
  • If the indicator is between the 25th and the 75th national percentile, the indicator is Fair.
  • If the indicator is between the 75th and the 95th national percentile, the indicator is Good.
  • If the indicator is above the 95th national percentile, the indicator is Very Good.

Mobile Network Situation KPI calculation

The Mobile Network Situation KPI value has the five possible values stated above. It is determined by the values of the two indicators. Depending on the combination of the signal strength and the signal quality indicator, a final result is given.

In layman terms, the KPI values (not the quality or signal strength indicator value) have the following meanings:

  • Very Good: the performance for all network usages will be excellent.
  • Good: the performance for all network usages may present occasional difficulties but is overall good.
  • Fair: most network-dependent usages (such as calls or data usage) will have a decent or better performance.
  • Degraded: network usage will be unstable and unreliable but allows for basic usage such as calls with acceptable quality and very slow data transfer.
  • Very Degraded: network usage different from emergency calls is near impossible.

Area-wise and Measurement-wise Mobile Network Situation

Finally, the network situation can be analyzed in two ways: area-wise and measurement wise. For the area-wise approach, Earth's surface is split into rectangular tiles called geohashes, and the average of each indicator is calculated. Then, those averages are used in each geohash to determine the Mobile Network Situation for the geohash. Afterwards, the percentage of geohashes on each category is calculated. To ensure a fair comparison, only geohashes on the Common Coverage Area (that is, the area where all analyzed MNOs have data) are considered. For the measurement-wise approach, each measurement is given a Mobile Network Situation value and then the percentage of measurements on each category is calculated.

Call types percentage

Although 2G and 3G are capable of data usage, the advent of 4G and more recently 5G technologies have relegated them to mostly call usage. The absence of 4G-based call technology ensured that 2G and 3G, as the only way to make a phone call, were still relevant. The development of VoLTE (Voice over LTE) calls during the 2010s marked the beginning of the end for these legacy technologies.

However, not all customers can benefit from VoLTE calls. For a customer to use this technology several conditions must be met:

  1. Their MNO must provide 4G and VoLTE.
  2. They must have a phone capable of performing VoLTE calls.
  3. They must have a phone rate that includes VoLTE calls.
  4. Their phone must be homologated by the MNO and the manufacturer to perform VoLTE calls.

The fourth condition means that a customer, without changing their phone, can have VoLTE with one MNO but lack it with a different MNO. When a customer connected to 4G without VoLTE capabilities (for any reason) tries to perform a call, a hand-off process to the 2G or 3G network takes place. This process, depending on the method applied, is called CSFB (the most common one) or SRVCC. Aside from those technologies, VoWiFi (Voice over Wi-Fi) is also used when a Wi-Fi network is available. Sometimes, a combination of VoWiFi and VoLTE may be applied. 5G does not have its own call technology, so 5G users will resort to VoLTE or older technologies for their calls.

The calculation of this KPI is straightforward: it is just the percentage of calls made under each technology.

Latency status

Latency is the measurement of how much time it takes for the information to be transmitted between the user and the network. Latency is measured in miliseconds (ms). In our analysis, only mobile latency has been considered, as WiFi latencies are usually lower and does not reflect the real situation of the MNOs.

A lower latency means a faster, smoother network experience, whereas a higher latency means that the network experience will not be as good, or even unusable. This makes latency a good indicator of user experience, although it is not a guarantee of usability. We have divided latency in four great groups, based on the latency requirements of some of the most used applications in the market.

  • Excellent latency: latencies under 50 ms. This range of latency translates to a very smooth user experience, even with the most data-intensive usage, such as gaming or 4K streaming.
  • Good latency: latencies between 50 ms and 100 ms. This range of latency translates to a good user experience, although gaming may not be as fluid and 4K may present occasional problems.
  • Degraded latency: latencies between 100 ms and 300 ms. This range of latency translates to a mediocre user experience. Gaming and 4K are either uncomfortable or impossible, videochat may present noticeable lag and messaging with multimedia may take a long time to load.
  • Bad latency: latencies over 300 ms. This range of latency translates to an essentially unusable network. Only the lightest usage, such as sending messages without multimedia works decently.

2G and 3G networks have, on average, far higher latencies than 4G and 5G, but they have been included in the analysis, as they are technologies still in use.

Data aggregation

For all KPIs, the following process is performed:

  1. The data is collected from the users, and cleaned before storing it in our systems.
  2. After collection, the daily value of the KPI is calculated for all days in the report timeframe.
  3. Finally, the average of all daily values is calculated, along with the standard deviation for determining the confidence interval. As this provides an estimate of the actual value, a confidence interval is provided. The confidence interval is determined assuming a confidence level of 95%, that is, there is a 95% chance that the actual value of the KPI is in the provided confidence interval.

Glossary

  • 5G SA: 5G Stand-Alone, a type of 5G connectivity that does not require a 4G network to work.
  • 5G NSA: 5G Non Stand-Alone, a type of 5G connectivity that relies on the existing 4G network.
  • KPI: Key Performance Indicator, an indicator of performance based on a measurable fact.
  • MNO: Mobile Network Operator, a company that provides mobile services to customers deploying a network and having its own frequency spectrum.
  • NR-RSRP: New Radio Reference Signal Received Power, a signal strength indicator for 5G networks.
  • NR-RSRQ: New Radio Reference Signal Received Quality, a quality indicator for 5G networks.
  • RSRP: Reference Signal Received Power, a signal strength indicator for 4G networks.
  • RSRQ: Reference Signal Received Quality, a quality indicator for 4G networks.
  • VMNO: Virtual Mobile Network Operator, a company that provides mobile services to customers without its own frequency spectrum, thus using that of another MNO.