Moodlepreneur Monday: Bootstrapping Market Research, Part One

Moodlepreneur Monday Bootstrapping Market Research, Part One
"Thomas Bayes" by Simon Harriyott is licensed under CC BY 2.0

Moodle is, for the most part, a great ally of the open source educational entrepreneur. A glaring and paradoxical exception is market research data. You can get an up-to-date number of registered sites, which is a clear and methodologically reliable data point, but its decision making value is limited. You can also filter this number by country, that’s how we know, among other things, that the US is the unambiguous top Moodle nation in the world. But following the ranking there is an obvious correlation with economic size. It is impossible to dig just a little deeper. We don’t know how many of these sites are the primary LMS of an organization. We know these are active sites as of a couple of months at most, but we do not know their purpose or level of activity. We cannot tell whether a site is for primary, secondary or tertiary education, or other kind of institutional environment. We get a sum of total users, but it is impossible to know how many repeat accounts there are. Even Moodle partners ignore how the stack up against each other. In summary, data provided by Moodle HQ is not apt for market research purposes.

What’s an enthusiastic, ambitious Moodler to do? Perhaps the best way is to dust up the conventional wisdom on market sizing for startups. More than a tested approach, is a process of ongoing refinement that benefits from diligent attention to the build-test-learn loop and the hints it offers. If you are running a lean ship, implementing data keeping habits for market estimation could be trivial and offer invaluable benefits.

There are two common approaches to market sizing in contexts lacking historical data, as it is the case of technological innovations: Bottom-up and Top-down. But before getting into them, we will focus first on some important ground rules.

The crash lesson on Bayesian inference you never knew you needed

When a product is getting on the hands of customers for the first time, there are unknowns left and right. Thanks to Bayes theorem, it is possible to follow a process of small sampling to hint at the bigger picture. A proficient market analyst knows that “truth-seeking” Bayesian techniques help manage —never kill— uncertainty. Here is the basic form of Bayes conditional probability equation, where M = “Market” and S = “Success”:

P(MS) = P(S|M)×P(M)

It literally reads: The probability of “Market Success” is equal to the probability of Success in a given Market, times the probability of the Market. Easy, right? Well, the mathematical implications and caveats are varied. Hopefully we will be able to explore them in later parts.

For now, suffice it to say that this equation helps define a set of ground rules upon which relatively small (but increasing) set of data can be used for productive market research:

      • A product must face its users as soon as possible and request as much feedback as possible, both in quantity and quality.
      • User feedback deserves limited attention. But each new round of feedback deserves more credit than the previous ones.
      • After a round of feedback, the product must change to at least satisfy the most repeated complaints. In fact, from a hypothesis testing point of view, it is not advisable to make any other kinds of changes.
      • Each new round of feedback must involve a larger and hopefully more diverse customer sample, UNLESS the negative feedback grows compared to the previous round.

There is a key assumption at play: Startup equals growth. From a lean startup perspective, a new product is supposed to always be on its way to find “market fitness,” therefore it can only expect to earn better feedback. Simultaneously, a picture of the market must be getting clearer. If both processes do not show progress, it should be read that a product is not following its best possible development path towards market share maximization. If there is a first takeaway in here is the following: For agile startups, product feedback and market sizing are two sides of the same coin.

Interested in Part Two? Let us know on the comments or social media. Tell your friends!

eThink LogoThis Moodle Practice related post is made possible by: eThink Education, a Certified Moodle Partner that provides a fully-managed Moodle experience including implementation, integration, cloud-hosting, and management services. To learn more about eThink, click here.