Research Methodology and Data Sources
Last updated July 16, 2026
Every figure in BigIdeasDB research traces to one of the datasets below. This page documents how the data is collected, deduplicated, and scored, what the severity and market-gap scores mean, where models are used versus humans, and the known limitations. It is the reference that every study in the research hub links back to.
Data sources
Record counts are current to the July 16, 2026 snapshot. The canonical corpus figure, "1M+ complaints, reviews and discussions," describes the historical cross-source total analyzed across pipelines. It is not the sum of the structured tables below, which are smaller, AI-extracted subsets. When precision matters, we cite the exact structured count.
| Source | Records | Evidence type | Limitation |
|---|---|---|---|
| Capterra reviews | 270,000+ | Software-review records | A review is not a structured pain point |
| Capterra structured pain points | 39,000+ | AI-extracted, severity-scored complaints | Structured subset of the review corpus |
| Capterra feature gaps | 40,000+ | Documented feature requests | A request is not proof of willingness to pay |
| Capterra category pain points | 5,000+ | Aggregated systemic complaints | Affected companies are tracked vendors, not end users |
| G2 reviews / insights | 129,000+ / 7,900+ | Review records / structured insights | Current snapshot, not a lifetime total |
| App Store and Google Play reviews | 136,000+ (99,000+ rated 1-3) | Mobile review records | Collection over-samples negative reviews; not an organic negative rate |
| Reddit structured pain points | 2,070 | Community complaints | Directional, not payment validation |
| Upwork jobs analyzed | 5,351 | Paid-demand signals (pain-point frequency) | Budget fields are not published; frequency only |
| TrustMRR startups | 8,000+ (3,787 with revenue) | Revenue and growth records | Present as ranges; mix of sources |
| Funded companies | 17,000+ | Funding-announcement records, AI-scored | No dollar amounts stored; momentum proxy only |
| Stripe Index companies | 30,000+ | Public Stripe directory, AI-scored | Not revenue data |
| Swiper validation cards | 71,000+ | Human validation signals | A swipe is interest, not purchase intent |
For product-specific technical detail on how these sources are queried, see the data sources overview.
Collection, sampling, and deduplication
Reviews and discussions are collected from each source and normalized. Because collection is targeted at software feedback, some datasets over-sample negative sentiment by design. The clearest example is the app-store corpus: the 99,000+ reviews rated 1-3 stars are collected for complaint analysis and must never be presented as an organic negative rate for a category.
Aggregated cluster and category analyses sample within token limits (roughly several hundred records per cluster) rather than reading every row, so cluster-level figures are representative estimates, not exhaustive counts.
Severity and market-gap definitions
Severity (scored out of 5) measures how frustrated users are in the underlying complaints. A score above 4.0 means people describe the problem in urgent terms, name the money or hours it costs them, and are actively seeking a replacement rather than venting.
Market gap (scored out of 10) measures how poorly the existing tools in a category serve the complaint. A 9.0 to 10.0 means users have tried what is on the market and it still fails them; a 7.0 means partial solutions exist but leave a clear wedge open.
Affected-company count refers to tracked software vendors exhibiting a systemic complaint, not the number of end users. The full definitions also appear on the how BigIdeasDB works page.
Model usage and human validation
Structured pain points, feature gaps, and opportunity scores are extracted by large language models, then spot-checked by humans. BigIdeasDB's own LLM pain-point-extraction benchmark measures how reliably models perform this task against a 900-record human gold standard, and informs which models are used in the pipeline. Build-difficulty ratings and editorial judgments are labeled as such and kept separate from measured scores.
Known limitations and bias
- Complaint data is a demand signal, not a business plan; it does not count competitors or estimate acquisition cost.
- Negative sentiment is over-sampled by design in review datasets.
- Regulated categories (health, finance, legal) carry compliance costs the scores do not capture.
- Self-reported revenue figures (for example, from Reddit) are labeled and not audited.
- Funded-company and Stripe datasets show presence and momentum, not revenue.
Refresh cadence, licensing, and corrections
Structured datasets are refreshed on a rolling basis and every published figure is tied to a dated snapshot so readers can see how current it is. Data is collected from publicly available sources and presented in aggregate; individual reviews and listings are anonymized and never republished with identifying detail.
Corrections: if a figure looks wrong, email ompatel@om.bigideasdb.com. Confirmed errors are corrected or the analysis is re-run, with the snapshot date updated. Research is authored by Om Patel.
Worked research example
From software complaints to a SaaS idea
Analysis verified October 10, 2026. Historical Capterra-derived records.
In BigIdeasDB's historical software-review data, 39.4% of product records with a reporting-labelled feature gap also have a reporting-labelled complaint. A requested feature and a documented problem produce different research shortlists. Inspect both before choosing what to build.
This is a label-matching test across 13,300+ product/company records with both analysis types. It counts stored software products, not customers. Both signals derive from Capterra feedback; their overlap is not corroboration from independent sources.
See the worked example and counts
Suppose you want to investigate reporting software. Searching feature-gap labels alone finds 5,100+ product records. Requiring a reporting match in both the feature-gap and pain-point categories reduces that set to 2,000+. This narrower set gives you a place to inspect requests alongside complaints. It does not establish that these products currently lack reporting, or that their users would buy your solution.
| Label screen | Feature-gap match | Complaint match | Both | Both / gap matches |
|---|---|---|---|---|
| Reporting | 5,100+ | 2,300+ | 2,000+ | 39.4% |
| Integration | 2,900+ | 2,300+ | 1,300+ | 45.8% |
| Customization | 860+ | 890+ | 330+ | 38.1% |
The screens match category labels containing reporting, integration or customiz, after trimming and lowercasing. Labels can match more than one screen. These are three illustrative themes, not an exhaustive taxonomy or a ranking of the best SaaS markets.
A product is counted once per theme and analysis type, even if it has several matching records. The two matches need not come from the same review. A missing match can reflect the way a model labelled the complaint, rather than absence of the underlying problem. Read the source text before rejecting it.
What to investigate next
- Choose one buyer and task, such as agencies assembling client reports. This is an illustrative hypothesis, not a result established by the aggregate table.
- Inspect reviews for that task. Record the obstacle, consequence and current workaround. Exclude generic complaints about a different workflow.
- Check current product documentation and talk to users about the last time the problem happened. This feedback predates the query date.
- Test a specific offer with those buyers. A repeated complaint helps choose the interview; it does not establish willingness to pay.
Sample and limitations
- Population
- 40,900+ processed feature gaps and 39,900+ processed pain points, joined by their common product/company records. Two feature-gap-only products and four gap records are excluded.
- Historical dates
- Derived records were created December 4–9, 2025. The company-level latest-review dates range from April 25, 2011 to December 2, 2025. This range is not the date coverage of every underlying review.
- Missingness
- 42 feature-gap labels are blank. Company-level latest-review dates are unavailable for 382 records. They remain in the population; a blank label matches no screen.
- Interpretation
- No current vendor capability, customer count, market size, market-wide complaint rate or purchase intent can be inferred. AI-derived labels and targeted review collection limit the result.
The percentage is calculated as 100 × products with both matching labels / products with a matching feature-gap label, using unrounded counts.
Choose review sources for your idea
Start with the customer task. Use another source to check whether the same task and consequence appear there, rather than treating keyword matches as proof.
| Source | Question to investigate | Limitation |
|---|---|---|
| Capterra research | What fails in an existing software workflow, and which features are requested? | Processed complaints and feature gaps can describe the same underlying reviews. They are not two independent sources. |
| G2 insights | Does a related B2B software complaint appear in another review corpus? | Insights are model-derived summaries. Verify sample and source context before comparing counts. |
| Reddit research | How do people describe the task and the workaround in discussions? | Posts and replies are not unique buyers. Agreement does not demonstrate a payment commitment. |
| App reviews | If the workflow is mobile, what fails in comparable apps? | BigIdeasDB's collection over-samples negative reviews. Its rating mix is not a store-wide complaint rate. |
Keep the sources separate until you have checked dates, units and duplicate evidence. Do not add review counts, extracted pain points and product counts into one demand number.
Run this research in an AI assistant
Connect the BigIdeasDB MCP server to a compatible assistant, then adapt the prompt below to your buyer and task. Tool names and inputs are documented in the MCP tools reference. The prompt retrieves a bounded sample to investigate; it does not reproduce the full-corpus aggregate study above.
I am investigating client reporting for small agencies, not deciding to build yet. Use the connected BigIdeasDB tools to retrieve evidence: 1. search_capterra_feature_gaps: query "reporting", limit 25. 2. search_pain_points: query "reporting", source "capterra", limit 25. 3. search_capterra_reviews: query "reporting", max_rating 3, limit 25. 4. search_reddit: query "client reporting spreadsheet", time_filter "year", limit 25. For each useful result, show the source, source URL if returned, observation date if known, product, customer role, task, complaint and workaround. Mark unavailable fields as unknown. Separate original source text from AI-derived summaries. Keep only evidence relevant to agencies and explain each exclusion. Group matching tasks, not just matching keywords. Separate repeated product records from independent reviewers. Do not calculate market-wide percentages from these bounded search results. Return three hypotheses with supporting evidence, contradictions, current alternatives to check, and one interview question each. If a result does not support a hypothesis, say so. Do not invent quotes, URLs, customer counts, current features or willingness to pay.
Use web search to check current alternatives and product documentation. Use connected research tools to retrieve the relevant feedback. If a tool returns no source URL or observation date, keep that field unknown and verify it separately.
For the decision after research, see how to test willingness to pay with a paid pilot.