Methodology

Index

Data collection

The data collected by Weplan is crowdsourced through an SDK introduced in smartphone apps. Currently, at Weplan Analytics we measure anonymously information on the quality of the network and the coverage of all mobile operators, through hundreds of apps and millions of devices worldwide. We reach partnership agreements with these apps to implement our SDK in their code, we pay them for active users and we, therefore, get anonymous network data provided by smartphone users.

Methodology of the analysis

Connectivity

A mobile phone user may have three possible connection status: Connected, Limited Connection and No Coverage.

• Connected includes all the time that a user has proper connection to its MNO’s network. It can be further decomposed into 2G, 3G, 4G and 5G, depending on the technology.
• Limited Connection includes all the time that a user has the connectivity enough to make emergency calls, either under its own MNO or under another MNO’s coverage.
• No Coverage includes all the time that a user does not have connectivity at all.

The Disconnection time provides the average of daily minutes that the users of an MNO have Limited Connection or No Coverage, that is, out of the 24 hours of the day, how many total minutes on average were spent without coverage by the users of an MNO.

Similarly, the 4G and 5G time provides the percentage of time (on average) that the users of an MNO have 4G or 5G connectivity. In this case, 5G includes both 5G SA and 5G NSA.

5G Penetration and Usage

Firstly, the percentage of customers per operator who have 5G coverage is provided. Secondly, the percentage of time that customers with 5G access have 5G coverage. Both 5G SA and 5G NSA are taken into account for this calculation. When a country does not have 5G coverage, these values are not provided.

It must be kept in mind that the ability of a customer to use the 5G network depends on several factors. First, their device must be 5G compatible. Second, the MNO must have a 5G network. Finally, the client must contract a mobile plan that allows the 5G to be used, as not all MNOs offer full access to the 5G network as part of the regular plans. If a client has connected to the 5G network (either SA or NSA) at least once it is included in the statistic.

A client may detect a 5G NSA network without connecting (due to it being a test network, the client not having a 5G rate but a 5G-capable phone, or other circumstance). Although internally the device may report a 5G NSA connection (more precisely, a 5G NSA RESTRICTED connection), the customer will not see it reflected in its phone (it will mark a 4G connection) or its usage. This type of 5G NSA connections have not been considered when determining if a customer is a 5G client or the 5G coverage time.

4G and 5G Network Status

Determining the performance of a network is a multivariate and nuanced problem. But checking the signal strength and quality of a network is a great approach to get a solid insight of the network status for an MNO. Unless otherwise stated, this analysis is performed only in 4G and 5G (both SA and NSA) networks, as they are the most used networks overall.

Signal strength and quality indicators analysed

Each technology has its own parameters and signal strength and quality indicators. In the case of 4G, the signal strength is provided by the RSRP, and the signal quality by the RSRQ. For 5G, the signal strength is provided by the NR-RSRP and the signal quality by the NR-RSRQ. Both networks have approximately the same value range for their indicators, so they can be compared directly.

Although there are some standard thresholds for these values when determining what is a good or bad signal strength or quality, in practice each MNO or TowerCo has its own parameters, based on their analysis and experiences. Furthermore, different countries exhibit different distributions of these indicators. To ensure a fair analysis, the thresholds for determining wether a signal strength or signal quality indicator is better or worse have been determined country by country using percentiles:

• If the indicator is under the 5th national percentile, the indicator is Very Degraded.
• If the indicator is between the 5th and the 25th national percentile, the indicator is Degraded.
• If the indicator is between the 25th and the 75th national percentile, the indicator is Fair.
• If the indicator is between the 75th and the 95th national percentile, the indicator is Good.
• If the indicator is above the 95th national percentile, the indicator is Very Good.

Network Status Calculation

The network status calculation has the five values indicated above. Its value is determined using both the signal strength indicator and the signal quality indicator. Depending on the combination of signal strength and signal quality, one result or another is provided.

In summary, the results can be interpreted as follows:

• Very Good: the performance for all network usages will be excellent.
• Good: the performance for all network usages may present occasional difficulties but is overall good.
• Fair: most network-dependent usages (such as calls or data usage) will have a decent or better performance.
• Degraded: network usage will be unstable and unreliable but allows for basic usage such as calls with acceptable quality and very slow data transfer.
• Very Degraded: network usage different from emergency calls is near impossible.

Network Status by Area and by Measurements

The network situation can be analyzed in two ways: area-wise and measurement wise.

For the area-wise approach, Earth's surface is split into rectangular tiles called geohashes, and the average of each indicator is calculated. Then, those averages are used in each geohash to determine the Mobile Network Situation for the geohash. Afterwards, the percentage of geohashes on each category is calculated. To ensure a fair comparison, only geohashes on the Common Coverage Area (that is, the area where all analyzed MNOs have data) are considered.

For the measurement-wise approach, each measurement is given a Mobile Network Situation value and then the percentage of measurements on each category is calculated.

Call Technologies

Although 2G and 3G are capable of data usage, the advent of 4G and more recently 5G technologies have relegated them to mostly call usage. The absence of 4G-based call technology ensured that 2G and 3G, as the only way to make a phone call, were still relevant. The development of VoLTE (Voice over LTE) calls during the 2010s marked the beginning of the end for these legacy technologies.

However, not all customers can benefit from VoLTE calls. For a customer to use this technology several conditions must be met:

1. Their MNO must provide 4G and VoLTE.
2. They must have a phone capable of performing VoLTE calls.
3. They must have a phone rate that includes VoLTE calls.
4. Their phone must be homologated by the MNO and the manufacturer to perform VoLTE calls.

The fourth condition means that a customer, without changing their phone, can have VoLTE with one MNO but lack it with a different MNO. When a customer connected to 4G without VoLTE capabilities (for any reason) tries to perform a call, a hand-off process to the 2G or 3G network takes place. This process, depending on the method applied, is called CSFB (the most common one) or SRVCC. Aside from those technologies, VoWiFi (Voice over Wi-Fi) is also used when a Wi-Fi network is available. Sometimes, a combination of VoWiFi and VoLTE may be applied. 5G does not have its own call technology, so 5G users will resort to VoLTE or older technologies for their calls.

The calculation of this value is straightforward: it is simply the percentage of calls made using each technology.

Network Latency

Latency is the measurement of how much time it takes for the information to be transmitted between the user and the network. Latency is measured in miliseconds (ms). In our analysis, only mobile latency has been considered, as WiFi latencies are usually lower and does not reflect the real situation of the MNOs.

A lower latency means a faster, smoother network experience, whereas a higher latency means that the network experience will not be as good, or even unusable. This makes latency a good indicator of user experience, although it is not a guarantee of usability.

For the analysis, latency is divided into four main groups, based on the latency requirements of some of the most widely used applications on the market:

• Excellent: Latencies under 50 ms. This range of latency translates to a very smooth user experience, even with the most data-intensive usage, such as gaming or 4K streaming.
• Good: Latencies between 50 ms and 100 ms. This range of latency translates to a good user experience, although gaming may not be as fluid and 4K may present occasional problems.
• Degraded: Latencies between 100 ms and 300 ms. This range of latency translates to a mediocre user experience. Gaming and 4K are either uncomfortable or impossible, videochat may present noticeable lag and messaging with multimedia may take a long time to load.
• Bad: Latencies over 300 ms. This range of latency translates to an essentially unusable network. Only the lightest usage, such as sending messages without multimedia works decently.

2G and 3G networks have, on average, far higher latencies than 4G and 5G, but they have been included in the analysis, as they are technologies still in use.

Data aggregation

The following process is applied to calculate all values:

1. Data is obtained from users and cleaned before being included in our systems.
2. After data ingestion, daily results for each value are calculated for the dates included in the report.
3. Finally, the average of each daily result is calculated. The standard deviation is also calculated to determine the confidence interval. This method provides an estimate of the actual value, thus offering a confidence interval. The confidence interval is determined by assuming a confidence level of 95%, i.e. there is a 95% probability that the true value lies within the given interval.

Glossary

5G SA
5G Stand-Alone, a type of 5G connectivity that does not require a 4G network to work.

5G NSA
5G Non Stand-Alone, a type of 5G connectivity that relies on the existing 4G network.

NR-RSRP
New Radio Reference Signal Received Power, a signal strength indicator for 5G networks.

NR-RSRQ
New Radio Reference Signal Received Quality, a quality indicator for 5G networks.

RSRP
Reference Signal Received Power, a signal strength indicator for 4G networks.

RSRQ
Reference Signal Received Quality, a quality indicator for 4G networks.