A step by step way to check demand, competitors and merchant pain before you write code, plus what 1,250+ real 1 to 3 star App Store reviews say about where the big apps fail.
To validate a Shopify app idea, prove four things before you build: merchants describe the problem in public, apps that solve it already hold hundreds of reviews, those apps' unhappy reviewers name a gap you can fill, and a few merchants will commit to using or paying for yours. Most guides stop at "talk to merchants". The App Store itself is the cheapest, fastest evidence, and almost nobody shows what is in it. So we read it. On November 2, 2026 we pulled 1,250+ one, two and three star reviews from the 30 biggest apps in two Shopify categories and counted what they complain about.
The result contradicts the usual advice. Founders are told to read competitors' bad reviews to find missing features. In our sample only 5.2% of written low-star reviews were about a missing feature. Support was mentioned in 29.6%, billing and price in 21.7%, and bugs in 21.9%. The opening in these categories is mostly operational, not functional.
Check demand, read the competitors' low-star reviews, then get commitments from 10 to 15 merchants before writing code. Shopify shows ratings and review counts but no install counts, so review count is your demand proxy. In a November 2, 2026 BigIdeasDB sweep of 30 Shopify apps, support (29.6%), billing and price (21.7%) and bugs (21.9%) were the most common complaints in 1,090+ written low-star reviews. Missing features were 5.2%. Run the same sweep on any category with BigIdeasDB Desktop, on your own computer, with your own AI agent.
Validation means evidence that a specific group of merchants has a recurring problem, already pays to solve it, and would switch to or adopt your fix. An idea is not validated because it sounds useful or because a friend said they would use it. A Shopify app also has one structural twist: you are selling inside a store where the buyer compares you with the apps next to you in search. So you validate four things, not one.
One platform fact shapes the plan. Shopify's developer documentation says you choose a distribution method, custom for one merchant or public for many, and you cannot change it afterwards (source: shopify.dev, "About app distribution"). A merchant-specific custom app is a fine way to test a workflow, but it will not become your public listing, so treat it as an experiment, not the first version of the product.
Run six steps in order, and stop the moment one fails. The steps get more expensive as you go, so the cheap ones come first.
| Check | Go | Stop or rethink |
|---|---|---|
| Problem in public | 5 or more separate merchants describe it in their own words | One thread, or only developers talking about it |
| Category demand | Top 5 apps each have 500 or more reviews | Top app under 100 reviews |
| Gap in low-star reviews | Same complaint in 5 or more reviews on 3 or more apps | Complaints scatter, no theme above a few percent |
| Commitment | 3 or more of your first 10 to 15 merchants join a waitlist or pre-pay | Warm words, no commitment |
The threads that rank for this question agree on one thing: validation is a conversation problem, and most builders have no merchants to talk to. The top organic result for the query is a forum thread, not a guide. Here is what the replies, read on November 2, 2026, actually say.
"look at shopify app store apps that already do download delivery and upsell emails. their review counts show if people actually use this and the 1 star reviews show what's missing."
– r/SaaS
"Cold DMs have a terrible response rate for validation. Merchants get pitched constantly and your message looks like every other one."
– r/shopifyDev
"If you ask for opinions, you'll get polite answers. If you ask what they actually did the last time they faced the problem, you get something usable."
– r/shopifyDev
"one thing that worked for me was finding people already paying for competitors, they have the strongest opinions."
– r/shopifyDev
The first quote is the method this article tests. It sounds obvious, and no one in the thread showed a result. The other three set the rules for step 5: ask about past behavior, go to people who already pay, and warm up before you pitch. For the Reddit side of that work, see our guide to finding customers on Reddit without getting banned.
Two posts show what happens after validation. A founder who reached 80 paying merchants in six months wrote that paying for App Store ads before about 10 reviews "felt like a waste of money", that around 33% of merchants left a review when sent a direct link after a support fix, and that personal, value-first outreach converted around 5%. Another founder with zero installs after two weeks and 130 cold emails got one reply, and commenters called that normal. In a third thread, a developer with a long career wrote that his app made 60 dollars a month after 3 hard rejections and that "doing business is a different skills". Validation lowers that risk. It does not remove it.
We listed the first five result pages of two App Store categories, took the 15 biggest non-Shopify-built apps by review count in each, and pulled their 1, 2 and 3 star reviews, newest first. The categories were shipping solutions (100+ apps listed) and store management operations (110+ apps listed, covering bulk editing, import and export, reporting and session recording). We chose them because they hold many apps with large review counts, which makes a complaint count meaningful.
Shopify's review pages can be filtered by star rating, so this needs no login. We fetched them politely, about one page a second, around 200 requests in total, and read only the public review text. Reviewer names were never stored and no individual app is named below. For each app we pulled up to 12 pages of 10 reviews, so 120 per app at most. Four apps hit that cap, so for those four the sample is the newest 120, not every low-star review. The sweep was run with a script. BigIdeasDB Desktop does the same job without code, as the last sections show.
We then counted nine complaint themes with keyword patterns, no AI model. A review can match several themes. We checked each pattern by reading random matches: about 9 in 10 matches for support, hard to use, billing surprise, price and bugs were real mentions of that theme, about 3 in 4 for sync and data accuracy, and roughly 6 in 10 for setup and for missing feature, which are the noisiest. We tightened the patterns once after that check. The first looser pass put "hard to use" at 4.4% instead of 3.1%, so the conclusion does not depend on the choice.
| Theme | All (n=1,090+) | Shipping (n=720+) | Operations (n=370+) | Apps with 5+ | Limitation |
|---|---|---|---|---|---|
| Support (slow, absent, unhelpful) | 29.6% | 36.8% | 15.6% | 16 | Counts any support mention, so a few are praise. About 2 in 24 sampled were positive |
| Bugs and broken features | 21.9% | 18.7% | 28.2% | 17 | Generic words like slow and error can match mild complaints |
| Price and billing | 21.7% | 20.6% | 23.7% | 13 | Includes any mention of a charge or plan, not only price complaints |
| Sync and data accuracy | 10.5% | 12.3% | 7.0% | 8 | Some delivery delay complaints match by mistake |
| Billing surprise | 5.4% | 5.1% | 5.9% | 4 | Subset of price and billing: charged after cancelling, doubled price, hidden fee |
| Left or switched away | 5.2% | 4.4% | 6.7% | 5 | Only counts reviewers who say so; most do not |
| Missing feature | 5.2% | 5.4% | 4.8% | 2 | About 6 in 10 matches were true feature gaps. Others were bugs or billing |
| Setup and onboarding | 4.6% | 4.6% | 4.6% | 4 | Noisy. Includes neutral and positive mentions of setup |
| Too complicated, hard to use | 3.1% | 2.5% | 4.3% | 1 | Narrow phrases, so a floor. The looser first pass gave 4.4% |
65% of written reviews matched at least one theme. The rest were short, in other languages, or about something else, such as a fee dispute with a carrier. Of the 1,250+ reviews, 150+ had no usable text and were left out of the theme counts.
Support was mentioned in 36.8% of shipping reviews and 15.6% of operations reviews. In shipping, a failed label or a lost parcel is urgent, so slow support turns a bug into an angry review. In reporting and editing tools, a stuck user can wait a day. Sixteen of 30 apps had at least five support reviews. Reviewers do not write that the product lacks a feature. They write that nobody answered.
Only 3.1% of written low-star reviews called the app hard to use or confusing, and only one app had five or more. That is the opposite of the claim that merchants leave because apps are too complicated. The caveat is real: our phrases are narrow, and a merchant who gives up during setup may leave no review. The reviews that did say it were blunt, as in "Overly complicated for what should be the simplest of tasks." If your wedge is simplicity, test it with merchants, because the reviews do not prove it.
21.7% mention price or billing, and 5.4% describe a specific surprise: charged after cancelling, a price that doubled, or a hidden fee. The surprises repeat across apps with different features. "Canceled and uninstalled app May 6th and still charged renewal May 9th for next month" is the whole pattern in one line. Plain pricing, a visible cancel button and a no-questions refund are cheap to build and unusually visible in reviews.
Missing-feature complaints were 5.2%, and only two apps had five or more. The ones that were true gaps fall into a few groups: compliance and data controls, such as a session recording tool that gave "No option to remove customers personal data collection", cross-border paperwork in shipping, and partial fulfillment or bulk-edit edge cases. A feature-led wedge is possible, but in these categories it is a narrow one.
Low-star reviews were a median 2.3% of an app's reviews, but ranged from 0.3% to 18%. Across the 210+ apps we listed, 110+ had 100 or more reviews and the median rating was 4.8 in shipping and 4.9 in operations. Only 14 of those 110+ were rated below 4.5. A "popular but poorly rated" app is rare, so do not wait for one. Look for the same complaint across several well-rated apps, such as support or billing, where a 4.8 average still carries dozens of angry reviews.
| Low-star share of an app's reviews | Apps | What it tells you |
|---|---|---|
| Under 1% | 5 | Few complaints. Hard to find a gap in these apps |
| 1% to under 2.5% | 9 | Typical for a well-run app |
| 2.5% to under 5% | 7 | Complaints exist, usually one or two themes |
| 5% to under 10% | 4 | A visible opening. Read these first |
| 10% or more | 1 (plus 4 capped apps at 4.9% or more) | Large, unhappy customer base |
Counts show the pattern. Quotes show whether you can build against it. Here are lines from the sample, one per theme, with app names removed.
"We have sent 2 emails and a support ticket but have received no response on an issue with inventory syncing. Very frustrating."
– Shopify App Store review, 1 star, shipping category
"I cancelled this and I've still been charged. I will update my review if they fix it, otherwise be warned they've doubled the cost I originally agreed to and now I can't unsubscribe and stop paying."
– Shopify App Store review, 1 star, operations category
"60 percent of tracking numbers are not updated.... The system does not work well for dropshipping. Look for another solution."
– Shopify App Store review, 1 star, shipping category
"Support is non-existent.. a full week has gone by with multiple emails being ignored. UTM reports were loaded with duplicates and they were unable to help me figure out a solution."
– Shopify App Store review, 1 star, operations category
"Having no way to upload customs documents for our CUSMA/USCMA compliant items is a total deal breaker."
– Shopify App Store review, 1 star, shipping category
"No option to remove customers personal data collection. This just adds problems complying with Privacy Policy regulations."
– Shopify App Store review, 1 star, operations category
Two details matter for a North American or European builder. Cross-border and compliance gaps, customs paperwork under trade agreements and data collection under privacy rules, show up as the kind of missing feature merchants name specifically. And the billing complaints often turn on a renewal date or a currency conversion, which a clearer pricing page can fix.
Turn each repeated complaint into one testable promise, then check it with merchants who pay for the incumbent. From this sample, three patterns are worth testing, in order of how often they appeared.
Competition is not a reason to stop. Sixteen shipping apps with 500 or more reviews each means merchants pay for this job. The question is the wedge. We make the same argument for software generally in our guide to finding SaaS ideas from negative reviews on G2 and the App Store, and our startup idea validation guide covers the commitment tests in more depth.
You can do it by hand in an afternoon for one category, or use BigIdeasDB Desktop to sweep one in minutes. The manual route works for one or two competitors.
That is 300 to 500 reviews to read for 10 apps. It breaks at scale, and it is a one-off: nothing tells you when a new app arrives or when a rating moves.
BigIdeasDB Desktop is a local-first research app for Mac and Windows that connects to the AI agent you already use (Claude, Claude Code, ChatGPT desktop, Codex or Cursor) through a local MCP server. There is no API key to manage and no AI bill from us. Its App Marketplaces Lab (Beta) covers the Shopify, WordPress.org and Atlassian stores, one category at a time:
Our sweep above used a script, not the Lab, so its themes and the Lab's will not match exactly. Desktop's own "too complicated" lens is one of the pain themes, and in these two categories it was a small one, which is a useful result in itself. A sweep shows its request count and time first, asks you to confirm above 200 requests, and caches pages for 24 hours. Shopify has no install counts, so the Lab says so and uses review count as the proxy, as we did.
For software outside the App Store, the Software Reviews Lab reads G2, Capterra, TrustRadius and Trustpilot through your own signed-in browser, so you can sweep a product such as QuickBooks the same way, and the Reddit Lab covers the merchant threads in step 1. Connecting the agent is in our MCP setup guide, and the founder MCP roundup compares the options. We also explain what an agent can and cannot read in can Claude read App Store, G2 and Capterra reviews. Desktop is included with Basic and Pro, or sold as a one-time add-on, and the live price is on the pricing page.
BigIdeasDB Desktop sweeps a Shopify category, reads the 1 to 3 star reviews and quotes the pain, on your own computer. Try it on the category you are thinking of building in.
Our data is thin on Shopify apps specifically, which is why we ran the sweep. The 1M+ complaint corpus is built from G2, Capterra, Reddit and mobile app stores, and Shopify's App Store was not part of it. Inside it, only 8 Reddit pain points mention Shopify and just 6 Capterra pain points belong to Shopify's own products. Treat those as anecdotes.
The revenue side is thin too. In TrustMRR, our revenue database, 8 startups with revenue describe themselves as a Shopify app or for Shopify stores. That is below the 30 we need to publish a median, so we show the count and withhold the figure. For the economics of building and selling one, use our guides on how to sell a Shopify app, which has real exit data, and Shopify app versus WordPress plugin, which compares the two platforms on revenue.
The gap is the point. Generic validation tools and checklists have the same blind spot: they cannot tell you what the paying customers of a specific store category are angry about. Only the store can.
Plan the first 10 reviews before the first line of code. A founder who reached 80 paying merchants wrote that most of his reviews came after support interactions, which fits what our sweep found: support is where reviews are won and lost. Ask for a review right after you fix someone's problem, send a direct link, and do not offer anything in exchange. One commenter noted that Shopify does not allow asking for a positive review or offering a reward, so check the current store rules in the developer documentation before you design the prompt.
Then keep watching. Re-run the sweep on your category every quarter, because new apps and rating changes move the opening. Pin the quotes behind your wedge on the evidence board so your listing copy uses the merchants' own words, and use the idea validation tool and Reddit market research help pages for the surrounding work.
| Source | What we used it for | Limitation |
|---|---|---|
| Shopify App Store category pages (shipping solutions, store management operations) | First 5 result pages of each category, sponsored cards removed: 100+ and 110+ apps with rating and review count | Two categories only. First 5 pages, not the full category. Review count is a demand proxy, not installs |
| Shopify App Store review pages, filtered to 1 to 3 stars | 1,250+ reviews from the 30 largest non-Shopify-built apps in the two categories, newest first | Up to 12 pages of 10 per app. 4 apps capped at 120. 150+ reviews had no usable text. Reviewers who leave a review are not all merchants who churned |
| Keyword pattern counts, 9 themes | Share of 1,090+ written reviews matching each theme | Regex, not AI. A review can match several themes. Precision checked on random matches: about 90% for support, hard to use, billing surprise, price and bugs, about 75% for sync, about 60% for setup and missing feature. Non-English reviews match poorly |
| Reddit threads (r/shopifyDev, r/SaaS, r/ShopifyAppDev, r/ShopifyAppMarketing) | Founder advice and quotes on validating, outreach and early reviews | Seven threads, top comments only. Quotes are anonymized and some are lightly trimmed. Founder claims are self-reported |
| BigIdeasDB corpus and TrustMRR (read-only SQL) | Count of Shopify-specific complaint and revenue rows | 8 Reddit pain points, 6 Capterra pain points, 8 revenue-earning TrustMRR startups. Too few for any median, so none is shown |
| Shopify developer documentation (shopify.dev) | Custom versus public distribution cannot be changed after you choose | Rules change. Check the current page before you decide |
| Google SERP and People Also Ask (US, Nov 2, 2026) | What ranks for the head query and which questions people ask | One market, one day |
What this data cannot tell you: how many installs any app has, how many merchants left without reviewing, why a reviewer was unhappy beyond what they wrote, or whether a gap is big enough to pay for. Reviews skew toward unhappy and delighted customers. A support complaint is evidence of a gap, not of a market. Treat the sample as evidence about two categories and 30 apps, not the whole App Store.
Do four checks in order. Find the problem in merchants' own public posts. Count reviews on the apps that already solve it, because Shopify shows no install counts. Read those apps' 1 to 3 star reviews for what is failing. Then get 10 to 15 merchants, ideally paying competitor users, to commit to a waitlist spot or a small payment. Build only after all four pass.
BigIdeasDB Desktop is the best fit because its App Marketplaces Lab sweeps a whole Shopify category on your own computer, plots review count against rating, shows the pricing ladder and quotes the pain in 1 to 3 star reviews. Your own AI agent reads it over local MCP, with no API key. It does the check that thin validation checklists skip: reading what paying merchants complain about.
Pick one category, list its apps by review count, then read the 1 to 3 star reviews of the biggest ones and count the repeating complaints. BigIdeasDB Desktop automates that sweep for Shopify, WordPress.org and Atlassian stores. In our own run of 30 apps, support, billing and reliability complaints repeated far more often than missing features, so look for operational gaps, not only feature gaps.
There is no official threshold, and the store reports no installs, so review count is a proxy. As a rule of thumb, a category where the top five apps each hold 500 or more reviews has real demand. In the two categories we swept, 110+ of 210+ listed apps had 100 or more reviews, and 34 had 500 or more.
Sometimes, but badly at first. One founder in r/ShopifyAppDev sent 130 cold emails and got one reply, and commenters called that normal. Another reported about a 5 percent conversion from personal, value-first outreach. Warm context works better: answer merchants' public questions first, then ask for a short chat, and ask for a commitment, not an opinion.
Commenters in r/shopifyDev suggest 10 to 15 real conversations, and another in r/ShopifyAppMarketing suggests starting with 30 stores where the problem is clearly painful. Both are rules of thumb, not data. Stop when the same problem, in the same words, comes up from most of the last five merchants, and at least a few will pre-pay or join a waitlist.
They can structure the work, but on their own they cannot read a store category's reviews reliably. Connect them to BigIdeasDB Desktop's local MCP server and the agent can sweep a Shopify category, count pain themes in 1 to 3 star reviews and quote them, using the plan you already pay for. The merchant conversations still have to be yours.
No. Many apps with hundreds of reviews mean merchants already pay to solve the problem. The question is whether the big apps leave an opening. In our sweep, low-star reviews were 0.3 percent to 18 percent of an app's reviews, and the loudest complaints were about support and billing, which a small focused team can beat without a new feature.
BigIdeasDB Research. (2026). How to Validate a Shopify App Idea: 1,250+ Reviews Read. BigIdeasDB. Retrieved from https://bigideasdb.com/how-to-validate-a-shopify-app-idea