Performance Split A-B Testing with PageSpeed
To run an experiment you must set several options in the configuration file. First, you turn on this feature and tell it your Web Property ID:
pagespeed RunExperiment on pagespeed AnalyticsID UA-XXXXXXX-Y
This will enable several filters needed for experiments including the Insert Google Analytics filter to insert a tracking snippet into each page. If you already have a tracking snippet on your pages, you should remove it so that the one PageSpeed inserts is the only one.
To disable experiments and experiment tracking you can set RunExperiment to 'off':
pagespeed RunExperiment off
If you still want PageSpeed to automatically insert the Google Analytics tracking snippet, you need to enable the insert_ga filter:
pagespeed EnableFilters insert_ga
Once you have turned on RunExperiment and set your Google Analytics id you can set up an experiment. For example, to test how much PageSpeed is speeding up your site you can apply optimizations for only half your visitors:
pagespeed ExperimentSpec "id=1;percent=50;default" pagespeed ExperimentSpec "id=2;percent=50"
ExperimentSpec defines an experimental treatment or specification with a semicolon-separated list of settings. Some settings are required:
- A positive integer, unique across all experiments. You can't reuse ids from one experiment to the next; every id must be unique. If you stop running an experiment that had three experiment specs 1, 2, and 3, then your next experiment should start with id 4 or higher.
- An integer between 0 and 100 indicating what fraction of users should receive this treatment. If the sum of percent over all treatments is less than 100 then the remaining users will be assigned to 'no experiment' and continue seeing what they usually would. If the sum exceeds 100 it will report an error when parsing the config file.
ExperimentSpec 1 above adds the optional setting
default, described below, which tells PageSpeed to apply just the filters and settings it would normally apply. ExperimentSpec 2 specifies nothing and so turns off all filters.
_gaq.push(['_setCustomVar', 1, 'ExperimentState', 'Experiment: 1']); _gaq.push(['_setCustomVar', 1, 'ExperimentState', 'Experiment: 2']);
In addition to
percent described above, there are additional optional settings intended to give you a way to test many settings you could set in the configuration file:
- Apply the filters and settings that would normally apply. Warning: prior to 220.127.116.11 this setting could not be used in combination with any others. It would override any other configuration specified in the experiment spec, and the other settings would be silently ignored. This meant that in
enabled=remove_commentswould have no effect. As of 18.104.22.168, however, it is now safe to combine
options=, and it indicates that the current non-experiment options should be used as a starting point. For example, in
default,enabled=remove_comments, you would now get the expected result of server default filters and options, plus
remove_comments. Note that
level=may still not be used with
default. While all the other settings are in the form
key=value, as in
percent=42, this setting has no 'value' and is just
- Set the
- A comma-separated list of filters explicitly enabled. For example,
enabled=rewrite_images,inline_js. Equivalent to
- A comma-separated list of filters explicitly disabled. Equivalent to
- A comma-separated list of options to set. For example,
- The Google Analytics ID you would like the data of this experiment reported to. If not specified, this experiment will use the Analytics ID specified with
- Google Analytics provides five slots for custom variables. Here you specify which custom variable slot you would like to use. The default is slot 1, but you can change this setting globally with:
Visitors are assigned to ExperimentSpecs on a week-to-week basis. This means that when you finish one experiment and start another, it will take up to a week for repeat visitors to be assigned to the new experiment.
Starting in PageSpeed 22.214.171.124 you can test experiment configurations on a live site before assigning anyone to them by creating them initially with a percentage of '0%' and then visiting the page with
?PageSpeedEnrollExperiment=<experiment_id>. This will set a cookie that assigns you to the group you're trying to test. You can stop running this test by deleting the
PageSpeedExperiment cookie or loading the page with