Website synthetic testing is a process of testing a website by simulating user interactions with the website. This type of testing allows developers to identify and fix issues with the website before users encounter them.
As we will discuss throughout this article, synthetic tests are handy in markets like China, where various problems can arise from one region to another due to the Great Firewall of China (GFW) and different local ordinances.
Why Are Synthetic Tests Necessary?
There are many reasons why website synthetic tests are necessary. As mentioned, synthetic tests can help identify issues with the website that may not be apparent during manual testing. Additionally, synthetic tests can help identify issues that only occur under certain conditions, such as high traffic or specific user interactions. For this, developers can run different tests and compare the results in the tool dashboard.
Moreover, website synthetic tests can be used to simulate real-world conditions and users. This is important because it gives developers a rough idea of how the website might perform under real-world conditions. Finally, website synthetic tests can be run automatically, helping speed up website development.
In other words, we could summarize the benefits of synthetic tests as follows:
- Help identify performance and other website issues proactively
- Reduce mean time to resolution (MTTR) on live websites
- Facilitate deploying code regularly
- Are invaluable for identifying major issues facing websites hosted outside of China
Now that you know why synthetic tests are necessary, let's review some specific use cases.
Synthetic Testing Tools
No matter which synthetic testing platform your organization chooses, make sure it has at least the following features:
1. Code-free tests
Code-free tests are synthetic tests that do not require any coding or scripting. These tests can be created and run using a visual interface, making them easy to set up and use. This empowers team members with no coding experience to create tests that can be very useful.
2. Managed and private locations
When running synthetic tests, it is usual for the platform to offer you different locations for them to run. Running synthetic tests in different locations helps to detect latency problems and other performance issues that may affect some regions more than others.
This type of situation is very common, especially in China, since there are notable differences in infrastructure and service quality between regions. It is also useful to have private test locations since they allow developers to test their site or application in a controlled environment. This can be useful for testing new features or changes or for troubleshooting issues.
3. Alert system
Alerting systems notify developers when a synthetic test detects an error or performance issue. This allows developers to identify and fix problems quickly. In this sense, the platform must provide easy access to traces, logs, and metrics from synthetic tests so developers can easily troubleshoot and identify potential problems.
4. Easy access to different troubleshooting contexts
A sometimes undervalued aspect is the ability of the platform to provide information in different contexts. In that regard, favor platforms that provide easy access to traces, logs, and metrics from synthetic tests so developers can quickly troubleshoot and identify potential problems.
5. Built-in CI/CD integration
CI/CD integration allows developers to automatically run synthetic tests as part of their Continuous Integration and Continuous Delivery process. This helps ensure any issue is detected and fixed early before deployment to production. This makes this feature a must-have when considering your synthetic testing tool.
6. Visualization tools
Today, having full observability of your system is a necessity. With so many moving parts, it is essential to have dashboards that provide an overview of your application or website and more specialized dashboards with relevant information on performance, uptime, failures, etc. In this sense, graphical dashboards provide a powerful tool for developers since they make it easy to identify patterns and trends, which helps optimize your website's performance.
Moreover, a graphical view of synthetic test results also makes it easy to identify issues that may not necessarily trigger an alarm, for example, which parts of the application can be optimized to withstand peak traffic hours.
Examples of Synthetic Testing Use Cases
Synthetic testing can be used in several use cases, including:
1. Websites Having Little to No Traffic
Synthetic testing can be helpful in cases where website traffic is low or nonexistent. By creating artificial traffic, testers can simulate real-world conditions and identify potential bottlenecks or performance issues. These simulations are also useful for evaluating key aspects of the network such as ping round trip (latency), base response time, packet loss, and more.
Moreover, in some cases, synthetic testing may be the only option available, for example, when testing a new website in a unique market like China or testing an app that has yet to be released. By creating a realistic simulation of traffic and activity, developers can ensure that the site or app can handle real-world conditions when it is launched.
2. Simulating Specific Segments or New Markets
Simulating specific segments or even new markets can be a powerful use case for synthetic testing. By creating realistic simulations of customer engagement, marketers can validate proposed changes to their product or service before rolling them out to the entire customer base.
Additionally, by understanding how new segments or markets react to their offerings, companies can make informed decisions about whether or not to pursue these opportunities. Finally, synthetic testing can provide valuable insights into customer behavior that can be used to assess key points to consider while building your organization's Chinese web presence.
3. Monitoring Uptime and Third-Party Apis
Synthetic testing can be used to monitor the uptime of an API or service. This is invaluable for detecting problems in an API that can affect the user experience and allows for proactively solving these problems. For instance, your team can get an overview of your service uptime globally and compare it to a specific region as shown below.
With a simple movement of the mouse, the same information can be obtained from any other region of interest. Moreover, synthetic testing can be used to monitor the performance of third-party APIs, which is essential to ensure that the API is performing as expected.
4. Improving Continuous Integration and Delivery (CI/CD)
The use of synthetic testing in a continuous integration and delivery (CI/CD) pipeline can speed up the feedback loop and improve the quality of the delivered software. By automating the execution of synthetic tests as part of the CI/CD process, it is possible to detect issues early and prevent them from being introduced into the production website.
One synthetic testing strategy that can be used in a CI/CD pipeline is to create a baseline set of tests that are run automatically on every build. These tests can be used to check for regression issues and ensure that the site is functioning as expected. If any issues are found, the team can investigate and fix them before the website is live.
Another use for synthetic testing in a CI/CD pipeline is to create tests that specifically target new features or changes. These tests ensure that the new code works as expected and does not introduce any new issues.
Challenges of Synthetic Testing in China
The Chinese market presents unique challenges for synthetic testing. The most notable difference is the sheer size and diversity of the market. With over 1.3 billion people and thousands of cultures and dialects, China is a complex market to target.
Another challenge is the fact that the Chinese government heavily regulates the internet. This includes censorship of certain websites and keywords and the requirement that all internet companies register with the government. This makes it difficult to create and maintain a synthetic testing environment that accurately reflects the real-world internet for Chinese users.
Finally, the Chinese market is constantly changing. New technologies like 5G and increasing demand for streaming video mean that the landscape is in a state of flux. This makes it challenging to keep synthetic testing environments up-to-date and accurate.
Despite these challenges, synthetic testing is essential for companies looking to enter the Chinese market. By carefully designing synthetic testing environments and constantly monitoring for changes, companies can gain valuable insights into how their products will perform in the real world.
Conclusion
Throughout this article, we have explored the multiple advantages offered by synthetic tests. That being said, combining synthetic tests with manual tests in a market like China is essential to ensure that your website or application behaves as expected. You may wonder if this is necessary. The short answer is yes, it is highly recommended to do so. Manual tests allow you to do last-mile tests, ad-hoc tests, comprehensive usability tests, browser compatibility tests, and more. Since real people perform these tests, their insights are valuable in confirming the conclusions of synthetic tests and even detecting problems that were missed during them.
If you want to go deeper into the topic of website performance tuning, we suggest you read "A short guide to China website testing in 7 mins," where we summarize the different types of tests that can be carried out in China, including synthetic tests, manual tests, and Real User Monitoring tests (RUM).
On the other hand, if you just want to learn more about manual testing and why it is vitally important to combine it with synthetic tests, you can read our article "How does manual testing work in China to help with website performance?" and also "6 things that manual testing can do but synthetic testing can't in China"
For more information on which is the best strategy to enter the burgeoning Chinese market, contact our GoClick China specialists, who will gladly help you design a personalized strategy that fits your needs and market niche.
