Polus - Getting Started

Installation

The Spiny team will include the Polus library in the Pandora bundle you put on page. Please let the Spiny team know that you would like to use Polus and we will enable it for you.

How are tests ran

Polus is a multivariate testing library that allows you to test multiple variations of client side code. It is designed to be used in conjunction with the Spiny library Orion. Orion handles session persistence and reporting.

One thing to note is that Polus runs tests by session not page view. This means a single user will only be assigned their experiments until their Orion session expires at which point they will be assigned a new experiment(or the same if they fall into the same bucket).

The primary issue with page view level testing is that it can lead to inconsistent results when it comes to SPAs and cross-site test experiences. If a user recieves a test on page load, but then navigates to a different page via a non-SPA navigation, the test will not be re-evaluated but rather the same test will be served to the user. When navigating via an SPA, the complexity of undo-ing the test and re-evaluating it can be difficult if not done correctly can lead to inconsistent results.

Usage

Initialization

Similar to other Spiny libraries, Polus requires bootstrapping to begin using. Since Polus leverages return values more so than BidRoll and Orion, we will need to wrap the Polus methods in a function and push them to the window.polus.queue array. Below is an example of how to use Polus.

window.polus = window.polus || {};
window.polus.queue = window.polus.queue || [];

window.polus.queue.push(function () {
    // Polus is now available
});

Polus Methods

Below is a list of the methods available to you when using Polus.

console.log(polus);
// Polus API
{
    // The createExperiment method creates an experiment within Polus.
    createExperiment: ƒ (experimentConfig | experimentConfig[]), // => Promise<ExperimentApi | ExperimentApi[]>
    // Some more advanced usage of Polus may use asynchronous
    // methods internally. If not leveraging these capabilities
    // you can use the createExperimentSync method.
    createExperimentSync: ƒ (experimentConfig | experimentConfig[]), // => ExperimentApi | ExperimentApi[]
    // The getExperiment method returns an experiment object by id.
    getExperiment: ƒ (id), // => ExperimentApi
    // The getExperiments method returns an array of all experiments.
    getExperiments: ƒ (), // => Array<ExperimentApi>
    // Queue callback to run when a specific experiment's test is active.
    onTest: ƒ (id, callback),
    // Queue callback to run when a specific experiment's control is active.
    onControl: ƒ (id, callback),
    // Force a specific experiment to run its test. Refresh may be necessary.
    forceTest: ƒ (id),
    // Force a specific experiment to run its control. Refresh may be necessary.
    forceControl: ƒ (id),
    // Get currently running tests.
    getRunningTests: ƒ (), // => Array<string>
    // Get currenlty running controls.
    getRunningControls: ƒ (), // => Array<string>
    // Get currently running experiments IDs.
    getRunningIDs: ƒ (applyControlTest = false), // => Array<string>
    // Reprocesses HTML elements, GA Dimension setting, and GPT targeting setting for a given experiment id.
    processBuiltIns: ƒ (id),
    // Explicitly loads and experiment. Can be ran if the experiment's config contained disableInitialLoad=true as an alternative to the experiment.load method.
    loadExperiment: ƒ (id),
    // Set the index time. Client side evaluated times are inaccurate so please leverage this method to ensure the date time matches your server time. An internal library Kronus will use this value to determine if the experiment should be run given the experiment's timeframe.
    setIndexTime: ƒ (now)
}

Defining an Experiment

const experimentConfig = {
    // The category of the experiment
    category: "cta", // required
    // The subcategory of the experiment
    subcategory: "primary", // required
    // The timeframe of the experiment
    timeframe: ["2024-01-01", "2024-01-02"], // required
    // The traffic percentage of the experiment
    trafficPercentage: 5, // Optional, default is 10
    // This can also be a range. End users are assigned a specific percentage number if the user falls within the trafficPercentage range the test will run given all other qualifiers are met. If running multiple experiments on the same page you can use this range option to ensure the user is only assigned one experiment out of a series of experiments.
    trafficPercentage: [5, 10],
    // Whether to disable the experiment on load
    disableInitialLoad: false, // optional, default is false
    // Metadata about the experiment
    attributes: {
        // metadata about the experiment
        someAttribute: "attributeValue",
    },
    // Control run settings (optional)
    runControlOn: {
        // Whether to run the control on inactive pages
        inactive: true, // Default=true,
        // Whether to run the control on expired pages
        expired: true, // Default=true,
        // Whether to run control on half of the users in the traffic percentage
        half: false, // Default=false,
    },
};

Setting an Index time

Client side timestamps can vary greatly in accuracy. The Polus library supports setting a time to index of which all time ranges will be tested against. We recommend passing a server derived timestamp if possible to ensure the test is ran in relation to the server time.

// Server derived timestamp
const SERVER_TIMESTAMP = "2024-01-01T00:00:00Z";

window.polus.setIndexTime(new Date(SERVER_TIMESTAMP).getTime());

Creating an Experiment

const experiment = await window.polus.createExperiment(experimentConfig);
// or
const experiment = window.polus.createExperimentSync(experimentConfig);

Creating Multiple Experiments

createExperiment can also accept an array of experiment configs.

const experimentConfigs = [
    experimentConfig1,
    experimentConfig2,
    experimentConfig3,
];
const experiments = await window.polus.createExperiment(experimentConfigs);
// or
const experiments = window.polus.createExperimentSync(experimentConfigs);

Experiment API

console.log(experiment);

// Experiment API
{
    // The id of the experiment
    id: "cta/primary",
    // The status of the experiment
    status: "qualified" | "disqualified" | "running-test" | "running-control" | "not-running" | "not-in-timeframe",
    // Whether the experiment is running
    isRunning: true | false,
    // Whether the experiment is in the test group
    isTest: true | false,
    // Whether the experiment is in the control group
    isControl: true | false,
    // Whether the experiment is reporting
    isReporting: true | false,
  // Queue callback to run when the experiment's test is active.
    onTest: ƒ (id, callback),
    // Queue callback to run when the experiment's control is active.
    onControl: ƒ (id, callback),
    // Explicitly loads and experiment. Must be ran if the experiment's config contained disableInitialLoad=true.
    load: ƒ (id)
}

Full Examples

Single Experiment

(async () => {
    const experiment = await window.polus.createExperiment({
        category: "cta",
        subcategory: "primary",
        timeframe: ["2022-01-01", "2030-01-02"],
    });

    experiment.onTest(() => {
        console.log("Running onTest callbacks");
    });

    experiment.onControl(() => {
        console.log("Running onControl callbacks");
    });
})();

Multiple Experiments

(async () => {
    const experimentConfigs = [
        experimentConfig1,
        experimentConfig2,
        experimentConfig3,
    ];
    const experiments = await window.polus.createExperiment(experimentConfigs);

    experiments.forEach((experiment) => {
        experiment.onTest(() => {
            console.log(experiment.id, "Running onTest callbacks");
        });

        experiment.onControl(() => {
            console.log(experiment.id, "Running onControl callbacks");
        });
})();

Multivariate Experiments

(async () => {
    const multiVariateConfigs = [
        {
            category: "cta",
            subcategory: "primary",
            timeframe: ["2022-01-01", "2030-01-02"],
            trafficPercentage: [0, 5], // Using incrementing ranges is key to multivariate testing
            runControlOn: {
                inactive: false,
                expired: false,
                half: false,
            },
        },
        {
            category: "cta",
            subcategory: "secondary",
            timeframe: ["2022-01-01", "2030-01-02"],
            trafficPercentage: [5, 10],
            runControlOn: {
                inactive: false,
                expired: false,
                half: false,
            },
        },
        {
            category: "cta",
            subcategory: "tertiary",
            timeframe: ["2022-01-01", "2030-01-02"],
            trafficPercentage: [10, 15],
            runControlOn: {
                inactive: false,
                expired: false,
                half: false,
            },
        },
    ];

    const experiments = await window.polus.createExperiment(
        multiVariateConfigs
    );

    experiments.forEach((experiment) => {
        experiment.onTest(() => {
            console.log(experiment.id, "Running onTest callbacks");
        });

        experiment.onControl(() => {
            console.log(experiment.id, "Running onControl callbacks");
        });
    });
})();

Forcing an Experiment

If you would like to force an experiment to run its test or control, you can use the forceTest or forceControl methods.

// via Polus API
polus.forceTest("cta/primary");
// or 
polus.forceControl("cta/primary");
// via Experiment API
experiment.forceTest();
// or
experiment.forceControl();

Query Param

For forcing a test via a query param you can use ?plForce=cta/primary this will force the user into either the test or control bucket. If you would like to further force a test/control you can add those to the end of the param value ?plForce=cta/primary/test or ?plForce=cta/primary/control.

Verification

If you would like the Spiny team to verify your test is reporting as expected please reach out to our team with a test url.