I once approved a production release to one of the U.S.’s most trafficked websites that randomized the color of the [Search] button between a dozen or so pre-selected options in order to see which performed best. When the experiment (I believe it was nicknamed, “Skittles”) completed, one color outperformed the others by only a fraction of a percent; however, the sheer volume was so high that the difference carried weight (and revenue).
Building a company from scratch last year, I found myself facing the challenge of creating a compelling and valuable experience with a user base of 0; however, even in the absence of scale, user behavior insights were already at the top of my mind.
Attn.: Startups. Resist your instinct to put off collecting this data until a later stage. This instinct is forgivable, especially as you race to market, but I want to openly share our story to illustrate why it should already be a priority.
At NoPlex, we’ve built an ecosystem that empowers individuals with ADHD and anxiety to stay productive and organize their lives in ways that circumvent their unique challenges. NoPlex’s entire leadership team has roots deep in the world of enterprise-scale customer experience personalization (in fact, we first met and began working together in this world years ago), and our approach to iterative product development draws deeply from our experience.
While NoPlex was still fresh out of the oven — and I mean ugly, buggy, confusing, and half-baked fresh — we were collecting almost 300 anonymized data points relating to nearly every transaction. Premature? Over-engineered? Not at all.
Unpopular opinion: There’s little value in a quick-and-dirty concept validation MVP in 2025. There’s no shortage of wisdom claiming that the key to a successful SaaS is to get an experience in front of some users within hours of coming up with an idea. I disagree.
I’m glad that no- or low-code tools exist, but digital consumers are so used to rich experiences that simulating an app or platform via a glorified Google form isn’t going to provide anything remotely close to the insights that it would have a few years ago. Don’t get me wrong, you should loosely validate the problem space and value of even proceeding to R&D step 1, but you need to release something significantly more mature to truly understand the validity of your proposed solution. This is because while problems may be somewhat unidimensional, even simple solutions are always multidimensional, and concept validation MVPs fail to capture said dimensions. Skip this step.
Dimensions is purposely open-ended here. Brand, aesthetic, cost, competition, market, timing, time-to-value, complexity, voice, copy, discoverability…
While it requires significantly more effort, time, and — in many cases — capital than the type of dead-simple glorified test described above, you should instead focus on creating an MVP that encompasses as many relevant dimensions of the proposed solution as possible… and then take a deep breath and make it publicly discoverable with a single goal: Collecting initial user behavior insights. Not revenue. Not adoption.
This will hurt. Your name is going to be attached to something kind of bad, and you won’t be able to hide behind the “it’s just an initial concept validation” excuse.
I recently took my own advice, as NoPlex released versions of our B2C app for Android, iOS, and the web a few months ago. They were expensive and far from pretty, but they did their job: As our user base grew through organic discovery, we began to collect very valuable anonymized behavior insights.
Oh, and don’t get attached.
Even though it cost a lot of effort, time, and money to launch our initial versions, we were careful not to get attached to them. While we were confident in the problem space that we sought to solve, we were well aware that our solution (up until that point) was essentially a collection of best guesses reinforced only by our collective history building user experiences.
We didn’t expect to validate our instincts, but rather to objectively audit all nomenclature and brand voice. We didn’t expect to adjust the UI, but rather to embark on a ground-up redesign — not just in relation to how the app looked but also how it was navigated and engaged with.
If you want to go into extreme detail on this, you’re in luck. While every product is unique, for the sake of illustration, I’ll share a few specific examples insights that NoPlex would likely never have uncovered if not for prioritizing diligent early data collection.
With a nod to Miles Davis’ quote, “it’s not the notes you play, it’s the notes you don’t play” — consider that at low volume, identifying the features that users fail to engage with (and why) is often more valuable than identifying those that they do engage with, especially if the latter is self-reported.
NoPlex’s earliest user behavior data volume was low, but interpreted properly, it told us things that users would’t (and often couldn’t) articulate in interviews or surveys. Things somewhat immune to bias. Things that users simply weren’t aware that they were thinking.
1. Orientation needed adjustment.
As far as I can recall, not a single interviewed user reported that they were having trouble orienting themselves in the app. However, users were behaving in ways that told us otherwise; for example, regular users (even those with 1.5k+ sessions) were beginning the process of creating a new item, but aborting before successfully doing so. This led us to hypothesize that the views specific to each “artifact type” (tasks, routines, notes, etc.) were visually similar enough to cause confusion. The MVP relied heavily on a shared new artifact creation interface and exclusively featured NoPlex’s primary brand color for structural UI accents.
To solve this, we leveraged NoPlex’s bold brand palette and built a ground-up relationship between each artifact type and a unique brand color, effectively adding an additional dimension to each type’s identity without introducing significant additional cognitive load. Doing so has seemingly improved orientation…and, as a bonus, introduced a good deal of character to the overall experience.
2. We used too many new words.
NoPlex’s experience was different from what most users were used to, and in order to relieve them of the burden inherent in translating deeply ingrained concepts, we leaned in on custom cultural language. For instance, in NoPlex, you can assemble collections of tasks that can be deployed on a schedule (e.g. daily self-care or spring clean-up) or ad-hoc (e.g. things to do before leaving a rental property). We called these scripts. The term seemed appropriate and self-explanatory, yet unique to the space.
We might as well have called them Aufgabenprozess. People simply didn’t make the connection and were wildly confused by the term. We received some minor feedback on this through interviews, but the confusion became clear only through observing how users interacted with the feature (see below).
We tested different terminology and ended up with routines. Less flare, more clarity. But again, we didn’t let ourselves become attached.
3. Our assumptions about which features were intuitively “related” were incorrect.
In NoPlex, the concept of routines (see above) was somewhat siloed. Separately, the user could create tasks (one-off) and recurring tasks (those that repeat on a schedule). We launched our first publicly available version with the assumption that users would mentally group tasks and recurring tasks together, despite the fact that recurring tasks were — in essence — routines containing only a single task. To illustrate this, take the garbage to the curb could be a recurring task that comes to life every Tuesday, whereas weekend chores could be a routine containing five tasks that come to life every Saturday.
We didn’t receive much feedback about how we were presenting/batching these concepts, but the data told us that we weren’t doing it right, and subsequent targeted interviews showed us that users were retrofitting routines to act like recurring tasks (creating single-task routines) and vice-versa (creating several recurring tasks that effectively acted as a routine). To address this, we completely reorganized the app to batch recurring tasks and routines together.
4. We were educating users incorrectly.
Some of NoPlex’s more unique features require education; for example, the way you can connect with and interact with another user is unique to the platform, as are the ways in which task creation/curation is separated from one’s daily commitments. Following recent trends that lean into more robust initial onboarding experiences, we built an interactive captive flow that engaged the user upon first launch.
If we were to rely entirely on customer verbatims to determine this experience’s efficacy, we would have pulled our shoulder muscles patting ourselves on the back. But the data — small as the set may have been — told a different story: The onboarding knowledge dump wasn’t sticking.
To address this, we drastically simplified the initial walkthrough and spread all but the most critical education out between a few bite-sized experiences that trigger at specific moments early in each user’s lifecycle.
5. Mobile store product summaries spoke the right language, but detail pages didn’t.
NoPlex’s Apple and Google Play store impression-to-download conversion rates were overwhelmingly positive — in fact, both were over double the industry average. However, when prospects found themselves moving beyond the store search results view to see details, browse screenshots, and read through feature descriptions, we saw sub-average download rates.
This told us a story about how compelling our “first impression” icon/title/30-character slogan/category content was relative to the detailed view, prompting us to focus our optimization efforts on presenting product pages that are better customized to the prospect’s specific segmentation attributes.
6. Our ICPs were too narrow.
For confused juggalos and those unfamiliar with product jargon, ICP stands for ideal customer profile. We built NoPlex with a specific set of ICPs in mind — adults and older children with ADHD, the parents or partners of those with ADHD, etc. — but we knew that the profile would evolve as data became available (remember: don’t get attached). Being open to growth here resulted in some unexpected insights (e.g. the fact that NoPlex resonated well with those struggling with anxiety), which led us to refine our messaging and reassess how our features would/wouldn’t likely be leveraged by these users.
7. We needed more targeted engagement moments.
Anyone who’s released a freemium app knows that not every new download means landing a customer for life. At the risk of obscenely oversimplifying a complex topic: If you separate attrition behaviors into three archetypes defined by their orientation in an end-to-end customer journey, you’ll have:
- Immediate abandoners: These are users who downloaded the app, perhaps resolve to check it out later, and move on with life.
- “Meh” abandoners: These are users who use the app for a short while and then lose interest.
- High-potential abandoners: These are users who used the app heavily and consistently…but then effectively churn (e.g. use daily for 1+ month followed by a complete drop-off). These are the ones who saw and understood your product’s value and could have been loyal longterm users, but something stopped them.
We used our data insights to understand each of these three archetype’s patterns/timing and determined their likely platform abandonment motivations. This gave us guidance on when and how to engage each.
For immediate abandoners: It became clear that — despite it being a familiar pattern in 2025 — users weren’t expecting to face a login page that required signup or social auth…or perhaps they simply didn’t trust the app in the absence of familiarity or didn’t feel like dealing with signing up at that time. Either way, the account creation step proved to be a drop-off point. This resulted in roadmap items to leverage NoPlex’s offline-friendly architecture and on-device capabilities to allow for initial app usage without account creation.
For “meh” abandoners: we began delivering behavior-specific native device notifications to these users at defined intervals to encourage reengagement.
For high-potential abandoners: As a life-management tool, NoPlex requires regular engagement to provide value. In this way, it’s more like a language learning or calorie tracking app than it is an airline or food delivery app (the latter of which you can love yet use infrequently), so we theorized that consistency is a disproportionately linked to retention.
As for the data: In most cases, the data told us that high-potential abandoners didn’t “fade away” so much as they “vanished.” This pointed to a behavioral pattern that we were familiar with: these users stumbled and never got back up before short-term habits became long-term. This typically happens when a user goes on a trip, addresses a family health crisis, experiences a career change, or anything else similarly routine-disruptive. To combat this, we’re introducing a light gamification dimension that rewards longer-term consistency.
Despite the promise of Starbucks gift cards, very few of the high-potential abandoners we contacted responded. When we asked a “meh” abandoner questions to determine whether or not they understood the purpose of a feature that they failed to engage with during their tenure, they confidently responded that they did understand it, but that it wasn’t relevant to them. It turned out that they didn’t understand it and therefore missed out on something that would have in fact been useful to them.
I say this to illustrate a point: Yes, you should talk to your users; however, Your users aren’t infallible insight machines, and early on in a product’s life, relying solely on feedback can skew your perspective in ways that lead you down the wrong paths. Outliers can provide dangerous misinformation; If you have 100 users, interview three of them, and one suggests that you change your fintech app name to “Doinky-Doink,” that doesn’t mean that 33% of your users believe this. You know this because it’s common sense, but let’s be honest: No matter how seasoned you are, it can be tempting to overvalue individual snippets of early feedback. So talk to your users, but remember that it’s not too early to collect and learn from user behavior data.
If you want to talk about this, or you’d like to talk about specific tools (or leveraging early customer data to shape your own product decisions) — reach out and let’s geek out.
NoPlex: iOS | Android | Web App | Website
Matt: LinkedIn
Additional tools and resource ➡️ Visit StartupStash
Zendesk is giving $75,000 in credits and perks for startups! ➡️ Apply Now!
