New Site Case Studies
Dec 07, 2021
Understand the differences between multivariate testing and A/B tests
The modern digital landscape is founded on one critical element — data. From content creation to site design, there’s no reason to take chances on what will resonate with your audiences. Adopting a data-driven mindset means you can take the guesswork out of your business strategy and focus on the methods that are actually moving the needle.
And one of the best ways to figure out what strategies are moving the needle for your website is through multivariate testing.
Multivariate testing is the process of testing one or more components on a website in a live environment. These components can be anything from a CTA button, headline formatting, or even an entire page design. The beauty of multivariate testing is that you can test each one of these individual features on a page to see what performs well among your users.
Think about it for a moment. Creative teams with great ideas are most successful when they have an environment where ideas can easily be tested against each other instead of trying to find total agreement on one idea. Multivariate testing allows teams to cherry-pick each idea to create an end result that works best, backed by the data to prove it.
See which elements and layout designs attract the most readers with multivariate testing. Graphic from Invesp.
Traditional A/B testing is the process of creating two different layouts and splitting the traffic between the two to see which one performs better. It’s possible to test more than just two layouts, of course, and there’s no issue with creating A/B/C/D/etc. tests depending on how many layouts you have to try.
A/B tests can produce great results, but they are limited since they test an entire layout at once. Remember, multivariate testing allows you to test the different components of a layout individually. Think of multivariate testing as running multiple A/B tests at one time. Here’s a good illustration of the differences between A/B testing and multivariate testing from HubSpot:
An illustration of the more complex testing available through multivariate testing. From HubSpot.
Multivariate testing is a great way to help creative environments stay focused. However, it’s vital that all ideas get measured, because one idea might sound awesome to the group or a team member, but it may not always perform.
If you are looking for fast results, it’s best to use A/B testing. However, multivariate testing is the preferred choice if you have the time to analyze and review multiple data points. Combined, the testing on each one of your site elements will help you curate the highest-performing page possible. It’s also recommended that you use multivariate testing on your pages with the highest traffic because there will be more data to analyze to determine which site elements are garnering the most engagement.
Multivariate testing is a more complex process that can be hard for the average marketing team to take on. Luckily at RebelMouse, we’ve solved this in a very elegant way. At the core level of our platform, we can natively render different layouts and track the exact differences in performance when comparing a test to your other layouts.Within our Layout & Design Tool, any idea can easily be built out from the base of an existing one. You can be knee-deep in the middle of redesign work and then quickly pause for a moment to test a variant of an idea with users or colleagues.
A look at layout comparisons available on the RebelMouse platform.
All of the variant testing we enable is measured against the core KPIs that were established at site launch by default. So pages per session, bounce rates, total engagement, and conversion goals are all easy to measure for your exact mission. When teams know every test can and will be tested against core KPIs, it keeps everyone’s creative energy focused on actually moving the needle.
Request a proposal today to learn more about multivariate testing that works.
Make sure your site is set up for success in 2022.
In the spring of 2020, Google let the world know that its Core Web Vitals would become the new benchmark for measuring a site's performance in its search results, known as the page experience update. Fast forward to more than a year later in August 2021 when, after much anticipation, Google's page experience update became official.
Since its rollout, developers have felt the impact of how their publishing platforms stack up against the new standard. Important decisions around the architecture of your site can now make or break your site's performance in the eyes of Google.
HTTP Archive, a tracking platform that crawls the web to identify trends and record historical patterns, frequently reports on how top content management systems (CMS) have weathered the page experience update through the creation of its Core Web Vitals Technology Report. RebelMouse has consistently outperformed major CMS platforms on Google's most critical metrics throughout the year:
Getting superior scores on Google's performance benchmarks isn't easy, either. The Ahrefs blog analyzed Core Web Vitals data from the Chrome User Experience Report (CrUX), which is data from actual Chrome users, to see how the web stacks up against Core Web Vitals. Their study found that only 33% of sites on the web are passing Core Web Vitals.
Luckily, performing well on Core Web Vitals is possible with thoughtful, strategic changes to your site’s codebase. Here's what you need to know and how we can help.
Google’s benchmarks for Core Web Vitals.
First, it’s important to understand the metrics that make up Google’s Core Web Vitals:
The other existing search signals are:
Together, all of these metrics determine Google's new page experience signals.
Google says it will always consider informative, quality content as its number one search signal. However, if two websites both have quality content, but one site has better Core Web Vitals, the site with a better page experience will always outrank any site that isn’t optimized for performance.
"By adding page experience to the hundreds of signals that Google considers when ranking search results, we aim to help people more easily access the information and web pages they’re looking for, and support site owners in providing an experience users enjoy.” —From Google’s original page experience announcement in May 2020.
Click here to learn more about what it takes to improve your site’s Core Web Vitals.
Having a quick load time and passing Core Web Vitals are important factors in Google Search’s ranking and results. But there are multiple ways of testing your speed and vitals, and it can get very confusing to try and understand the results since different measuring tools can result in different scores.
There are generally three ways to get your site's performance measurements:
Each of those three methods can give you important details on how your site is performing, but their results are derived using different methodologies.
Click here to learn more about how to accurately determine your site’s performance.
The saying "Content is King" is still true in today's publishing landscape, but there's no kingdom without high-performing sites. While page speed may have begun as a luxury for savvy webmasters and lucky readers, it's now a make-or-break component that deeply impacts a site's longevity in a highly competitive and global space.
A high-performance site is so critical in today's digital ecosystem that a poor-loading site could be a fatal blow to any publisher or brand. Here are just a few ways poor page speed can impact your site’s bottom line.
Mobile Page Speed Impacts Overall SEO Ranking: Mobile devices account for more than 50% of web traffic. If a site's mobile page speed is slow, this means half of the users trying to access the site are not only suffering a poor experience, but they're likely abandoning the site visit completely. This puts the site in danger of losing positions in its Google Search rankings.
Poor Page Speed Makes Ads More Expensive: Much like with SEO, if your page performance is slow and prompting lost site visits, the ads being served on your site will receive lower impressions. Lower impressions mean the ads are more expensive to deliver, which costs revenue and users in a matter of seconds.
Poor Page Speed Tanks Usability and Loyalty: The health of your site will always be dependent on the experience you deliver to your readers. Usability is the core reason why Google decided to prioritize page speed. Slow load times are a sure-fire way to give your readers a reason to abandon your content. To make matters worse, thanks to the massive amount of content being created every day, users have plenty of other options to choose from and may be wary of clicking a link or CTA associated with your site in the future.
The RebelMouse platform evolves alongside Google’s algorithm, and our team of growth experts spans all time zones to ensure our sites are optimized for page experience with every new article.
To do this correctly, it takes a lot of optimization to perfect every moment of your site’s load time. We’ve mastered Core Web Vitals already, and our performance scores are drastically outperforming every other CMS on the market.
Our jaw-dropping page speeds have allowed us to power the fastest sites on the web. We’ve done this through a simplified version of code that still allows for ads, videos, and third-party applications to load quickly.
Now that you’re up to speed (pun intended) on how Google measures and scrutinizes site experience, let's take a look at some of the benefits RebelMouse-powered clients enjoy as a result of their excellent performance scores:
Here’s how those scores compare to traditional CMS solutions that don’t prioritize page performance:
Currently, publishers are trying to build optimized websites that translate easily across devices and platforms, but are failing to also deliver an experience that checks all their boxes and prioritizes their readers. It takes less than a second of delayed load time to turn away a user.
If you are publishing to a site with poor performance, it’s going to take a mammoth effort to overhaul your site’s entire architecture so that it scores high against Google’s Core Web Vitals. In most cases, it’s simply easier and more cost-effective to move your data to a platform that is already high performing with page speed woven into the foundation of its technology versus reinventing the wheel on your own.
At RebelMouse, performance is a pillar of our company’s culture. We’ve taken site performance seriously for years, and have leveraged them to ensure our sites are constantly high performing. It's why we outperform some of the biggest sites on the web. And like with any pillar of a company's culture, optimizing for high performance is never a one-time effort. Our engineers have been crafting and tuning our platform to address these new standards long before they surfaced.
If you want to publish alongside the speed of the web, request a proposal today and let’s start working together.