8 Steps to Experiment & Achieve Long-term Growth

Growth hacks can only go so far if they’re not part of a global strategy.

“Over time, all marketing strategies result in shitty clickthrough rates.” Andrew Chen’s Law of Shitty Clickthrough

Achieving growth in the long-term is only possible when you have a well-defined growth process. It’s all about having big goals that you break down and contribute to via experiments. Experiments fail, experiments succeed and in the end, it’s a matter of creativity, optimization, scalability. Growth is a sum of little things that all in one move the needle for your business.

1. Analytics

One of the main advantages of being online is that you’re able to track everything. Without that, how will you know if an experiment succeeded of failed?

So first of all, make sure your website, app, emails are tracked in a way that:

• You can measure their success regarding what you’re testing
• You can measure high-level impact (# users, revenue, etc)

2. Determine High-level Goals & Brainstorm

Trying to optimize everything at the same time doesn’t work. It indicates a lack of focus and it simply doesn’t work:  for instance if you change a website design, how will you know if changing the button color increased clicks?

What are the high-level goals for your business? Do you want to focus on acquisition, activation, retention, revenue of referral?How does it make money? By answering this question, you’ll know what to focus on when you brainstorm about test ideas.

Regularly, have some brainstorms with your team and come up with test ideas for each of your high-level goals. Those tests will constitute your experiment backlog. If you’ve brainstormed properly, you’ll have come up with dozens of test ideas. Great, but where to begin?

Also, you will come up with ideas that you don’t need to test as they are no-brainers: just do them!

3. Prioritize your Experiment Backlog

Growth hacking is becoming popular for the huge potential impact possible to attain in a short span of time. That’s how we’ll prioritize: by potential impact & difficulty.

What better tool for this than a good spreadsheet? (Feel free to create a copy to your google drive folder)

This will also help you keep track of your experiments and your status for each one of them.

4. Hypothesis

Following up on the priority and order of your experiments, what does your gut tell you?

5, Goal (starting point & target)

With the risk of sounding cliché, each experiment should have a clear main goal you want to achieve, and it need to be measurable. Also, you’ll need a starting metric to compare your results to: what was you click rate before doing the email experiment?

6. How & How Long to Test

Now that you know what you should test and what’s the priority order of your backlog, it’s time to focus on how to test each idea. This is when you will fill the second part “Analyze and Plan Your Experiment” of your experiment spreadsheet.

A/B/C/etc… tests are usually how growth hackers test their hypothesis: by having different versions of the same page/message/content with little difference between each. This means each user will see one version of your experiment and you will be able to track how each perform. In practice, your different versions are often called variations. It’s important that you want to track one main goal for each experiment.

If you want your experiments to be statistically significant (know if you can really trust your result), you will need a minimum of visitors. You can use Optimizely’s sample size calculator for this. In other case and if you don’t have that many users, you will just want to see a trend.

As a side note, you should also always track engagement/retention additionally to your main goal so that you make sure you didn’t kill your user experience. Sometimes an experiment will contribute to your goal but will also negatively impact another one. In this case use your best judgment but it’s usually not worth it.

When it comes to how long you should run your experiments, you will want to wait until you have a significant number of participants to your contest. Generally, you will want to run your experiments for a few weeks so that external factors don’t impact your results. External factors could be:

• a bug
• an event outside of your control
• your startup got a surprise article in a publication

7. Results & data-crunching

When you have enough data or you’ve run your experiment long enough, you will stop your experiment and analyze. It’s tempting to start analyzing during the experiment but results can change quickly and it’s often a waste of time.

If you tracked your experiment correctly and set a clear goal for your experiments, results should be very easy to interpret. This is when you will get to use the last section “Analyze the Results” of your experiment spreadsheet.

Usually you will:

• kill the experiment (which can be hard if you’ve spent a lot of time on it)
• implement it & optimize

You will now be able to enter the results of your experiment and the way the spreadsheet is built, it will force you to dig a bit deeper to find how to go on with the experiment or get ideas for new ones.

8. Build Upon That & Iterate

You’ve got it by now, sustainable growth is the sum of experiments, optimization & iterations.

An example I often use (the one in the spreadsheet) because it’s easy to understand and usually drives great results is the HTML vs. text email experiment. Here is how it breaks down:

• Goal: increase email click-through rate % (% CTR)
• Hypothesis: Text email will get more clicks as it looks less spammy
• Baseline (current state): CTR=5%
• Target: CTR=10%
• Results: CRT=12% with text variation
• Ideas of optimization: test different copy & text structure
• Ideas of new test: Now that I’ve increased conversions, A/B test subject lines to find our what type of subject line drives more email open rate

With this example, we’ve increased the emails CTR from 5% to 12%. Let’s say you have a 1,000 people email list and that the value of each click to your website is 15$. If you run the subject line experiment and increase your open rate from 10% to 20%. Here is how much money those two tests made you:

• Initial setup: 1000*10%*5%*15= 75$ 
• HTML vs. text email: 1000*10%*12%*15$= 180$
• HTML vs. text email + subject line email: 1000*20%*12%*15$= 360$ 

Now, with those two improvements you’ve made thanks to your tests, each email blast you send will get your 360$ which is a 480% increase.

Summary

You should never stop experimenting, that’s how top performers continue to perform better and better as well as growth hackers to achieve long-term growth. Experiments are core to hacking growth and should be exectuted with both the vision in mind and steps you cannot miss:

1. Get your analytics right
2. Get experiments ideas from your long-term vision
3. Prioritize your experiments according to potential impact & effort required
4. For each, formulate your hypothesis
5. Have one main Goal
6. Determine How & How long to test
7. Analyze the outcome
8. Optimize and repeat.

How did this process work for you? Let me know in the comments or on Twitter.

[fbcomments]