How to make push notifications more effective with A / B testing


Push notifications are actually the best and powerful marketing tools on the market to interact with your customers. However, we can not use the same push notification strategy and expect to have maximum user engagement and improve conversions. This is where A / B testing of push notifications will help you experiment and provide insight into push notifications campaigns.

In this article, we will focus on A / B testing in push notifications. You track user behavior to check which strategy works best for your business.

What is Push Notification A / B Test?

Why waste your time on assumptions about effective strategy when you have the opportunity to test examples of push notifications.

With A / B testing, also called split testing, you can experiment with the best competent strategies and find optimal ones to achieve your goals. Marketers can try different designs, content, CTAs and more. You can compare one or more versions of the same push notifications and see which one performs best.

Marketers can divide audiences into small groups and send test-push messages. They can determine the impact of test-push notifications to know optimized one and push winning version to most users.

For example, your website has 500 subscribers that are evenly divided into half, which means that each segment has 250 subscribers. Now divide one of the segments in two again with 125 members in each. We have three different segments with 250, 125, 125 subscribers. Send push message samples A and B to the smaller subscriber number segments, respectively, for analysis. You can check the campaign results with statistics like impression rate, opening rate or clickthrough rate. The push message with the good statistical rates will be adopted and sent to the section with higher number of subscribers.

  • You can test variants of web-push notifications when you are unsure of campaign results.
  • Marketers can know the best timings and frequency of push notifications for high user engagement.
  • Keep experimenting with different combinations of variables until you find an optimized push notification strategy.
  • You can divide the target groups based on the various parameters such as location, browser, last visit, subscription day, etc.

Steps in A / B testing of push messages

Here is the optimized process for starting A / B test push notifications. Be sure to follow these steps that will lead you to marketing growth.

Set goals for Push Notification A / B testing

Before starting A / B testing, set goals you want to achieve, such as user engagement, retention, or conversion. Also determine the hypothetical results for the campaigns. So you can work towards them while exchanging the variables for testing push messages. Select the goals of the test push messages and variables to achieve them.

Run A / B test

Write two different push notification examples with variations in text, images, CTAs, emojis, etc., and send them to the audience. In popular free push tools like Truepush, you can split subscribers and send push messages, respectively. Users can check the campaign statistics in the dashboard to observe the effectiveness of push notifications.


Here you can see the push notification campaign statistics. It provides details on reach, delivered, clicks, impressions, subscriptions and more.

Write down results

Document every detail of the test-push message, such as message content, timings, frequency, and their results. Make sure you note small changes in the push notifications variables and their impact on the success of the push notifications campaign.

Apply results

Carefully observe the A / B test results for the push message by considering all statistics such as opening rates, conversion rates, traffic increase, engagement rate and more. Choose the best sample and send it to your subscribers for better benefits of push notifications.

Campaign goals and user needs often change, so you can always experiment with different examples for an effective push notification strategy. In the next section, you can find the various push message variables that you should consider for A / B testing of push examples.

What variables should you test with A / B testing?

Notification words and phrases

The contents of push notifications include copy of notifications, images and links. You can observe the user reaction to changes in message content such as using emojis, humor, force words, etc. We have put together the tips for writing a perfect web push message copy. You can experiment with the same campaign with different copies of push notifications and choose the one that converts.

For example, here are two push notification samples posted by a food blogger about their new recipes.

push notification A / B test

The above push message is a simple message without emojis and a direct message.

push notification

Here you can see emojis, humor touch in the push messages.

You can experiment and control which message has caused more users to click and see the push message.

Timing of push notifications

It is necessary to send relevant push notifications, but your work does not end there. Users who receive push notifications at oblique times will cause them to click “Clear notification” and you can never reach your users. So you need to find the optimal timing and frequency for sending push notifications.

The push message A / B test technique gives you the advantage of testing the push message examples at different times. You can know the details of what time of day users are interacting with your app or site. We have analyzed thousands of push notifications sent by our customers and listed the best push notification times for all industries.

For example, if a user checks his notifications panel after work hours and notices that there are several push notifications from websites and apps. There is a good chance that your subscriber will clear all notifications where you lose a conversion. So the timing of push notifications is very crucial.


Check if creating urgency among customers can engage more customers or a friendly push. For this you can use force words like “Hurry up”, “The offer ends now”, “Only today” in push message content.

For example, let’s observe two variants of the same push message.

Example 1: Discount expires today, shop and redeem now !!!

Example 2: You can save $ 10 with coupon for today.

The first web push message may have higher clicks due to the addition of force words and create FOMO feeling among the users. Keep evolving and try adding new words to the messages to get users’ attention.

Location-based messaging

The push notifications may not work the same globally, especially when there are different occasions, seasons and holiday sales. For this push message A / B test, you can group subscribers by their country, state, or city. It can be helpful for advertisers to plan promotional campaigns around the events.

Let’s say a football game only takes place in a few states in the United States. An event booking site can test the effectiveness of push notifications in different cities by segmentation. According to the results, advertisers can only concentrate on a few regions to get more conversions. You can collect subscriber location information using push notification tools and can easily segment audiences.

Tips for A / B testing push messages

We’ve put together some tips to follow and get accurate results while testing your push notifications.

  • You need to test one variable at a time. It is helpful to know which variable has contributed the most to the success of the campaign.
  • Always divide subscribers into larger segments so you can accurately read the campaign’s output.
  • Try to place similar audiences in one segment according to parameters like location, browser, platform, last visit and others. It will help you send similar push message samples and observe user behavior.

For example, you can not send push notifications about abandoned shopping cart to new users and welcome messages to regular customers. So dividing into similar segments will help you gain more clear insight into the effectiveness of push notification.

  • We can never hold a single test with push notifications with great competition and changes in customer expectations.
  • Check for overall analysis of push notification campaigns such as sign-ups, traffic, increase, sales, customer engagement and more.
  • Follow best practices for push notifications while writing examples for your A / B test.

Hopefully we believe this article has helped you perform push notifications A / B testing for better customer engagement. Once you get to know this process, you can easily sort the push notification strategy for your business growth. Keep evolving and try new things in your push notification examples that work best for your campaigns.

Leave a Comment