Are we spending more on Netflix subscriptions than we plan to? Does the design of local parks and streets encourage littering? Do games on investment platforms influence trading behaviour?

These are all empirical questions that can inform regulatory reform. To ensure regulations remain effective, they should be based on recent, reliable evidence. But what happens when that evidence does not exist? Regulators must experiment.

Regulatory experimentation involves modifying a regulatory product, process or policy and evaluating its impact. In a rapidly evolving world, using robust evidence is essential to adapting to new challenges.

In a recent project with the NSW Productivity Commission, the Behavioural Insights Team created twelve case studies to share tips on running successful experiments. We interviewed regulators from around the world who are leading the way on experimentation.

Drawing on these practical experiences, here are five valuable lessons for improving regulatory experimentation.

1. Build expertise by running small exploratory trials

Regulators have different levels of experience and resources for running experiments. Randomised controlled trials (RCTs) are one way to generate actionable insights, but small, exploratory experiments can also help build expertise and inform decision-making.

Example: Australian Securities & Investments Commission (ASIC)

ASIC conducted a proof-of-concept trial to explore the potential of generative AI in summarising public submissions to a parliamentary inquiry. Over five weeks, they tested AI-generated summaries against human-written ones, using blind assessments to evaluate quality and accuracy. AI-generated summaries struggled to capture nuance and context, performing worse on all key criteria, providing insights into the technology’s current limitations.

2. Run online experiments in fast-moving regulatory environments

Online experiments offer cost-efficiency, speed and scalability. They allow regulators to rapidly test interventions in simulated environments, particularly in fast-moving regulatory spaces where emerging risks and market behaviours evolve quickly.

Example: Ontario Securities Commission (OSC)

With the rapid rise of digital trading platforms, the OSC wanted to quickly assess the impact of gamification techniques – such as points and leaderboards – on investor behaviour. Using a RCT in a simulated trading platform, they found that points increased trading activity by 39 per cent, while leader boards made investors 14 per cent more likely to hold featured stocks. These insights highlighted the risks of gamification in financial markets.

3. Run field experiments to address real-world challenges

While online experiments are useful, field trials provide insights into real-world behaviour, ensuring that interventions work outside controlled environments.

Example: NSW Environment Protection Authority (EPA)

Cigarette butts are the most commonly littered item in NSW. The EPA partnered with 16 councils in NSW to test four interventions: creating pathways to bins, instilling a sense of ownership by establishing clean smokers’ areas, using social norms messaging and emphasising enforcement. All interventions reduced littering, but the “sense of ownership” intervention was the most effective. This experiment informed the state’s anti-littering guidelines and led to the development of the “Butt Litter Check” assessment tool.

4. Build internal research capability and a culture of experimentation

Having an internal research unit and a strong experimental culture, supported by leadership, is crucial for driving successful regulatory experiments. Agencies that normalise experimentation are better positioned to continuously improve regulation.

Example: UK Office of Gas and Electricity Markets (Ofgem)

Ofgem’s embedded research unit and strong culture of experimentation enable it to run experiments in-house. The Office’s trial on energy debt communications found harsh debt communications resulted in poorer comprehension and reduced desire to contact an energy supplier when compared to more positive framing. These findings informed cross-sector regulatory statements on debt communications.

5. Form strategic partnerships to expand reach and expertise

Partnering with industry provides access to real-world data and operational insights. Collaborating with external research experts also enhances experimental capacity, especially when internal resources are limited.

Example: Danish Competition and Consumer Authority (DCCA)

DCCA partnered with a subscription payments provider to analyse consumer behaviour in subscription markets. Together, they identified a natural experiment that found consumers were 70 per cent more likely to cancel subscriptions after a card payment was declined, highlighting passivity in consumer decision-making. This finding informed EU-wide policy proposals and strengthened regulatory enforcement strategies to improve consumer protection.

Making regulatory experimentation work

Regulatory experimentation is a powerful tool for enhancing policy effectiveness in a fast-changing world. These case studies provide practical insights from regulators who have successfully tested and refined their approaches. They complement existing resources and “how-to” guides, helping regulators build their capability to run effective experiments. This work also builds on previous research by the Behavioural Insights Team for the NSW Productivity Commission on the barriers and enablers of regulatory experimentation.

Saul Wodak is an advisor at the Behavioural Insights Team.

Image credit: Fotogestoeber/Getty Images