Research Methodology and Data Sources
Last updated July 16, 2026
Every figure in BigIdeasDB research traces to one of the datasets below. This page documents how the data is collected, deduplicated, and scored, what the severity and market-gap scores mean, where models are used versus humans, and the known limitations. It is the reference that every study in the research hub links back to.
Data sources
Record counts are current to the July 16, 2026 snapshot. The canonical corpus figure, "1M+ complaints, reviews and discussions," describes the historical cross-source total analyzed across pipelines. It is not the sum of the structured tables below, which are smaller, AI-extracted subsets. When precision matters, we cite the exact structured count.
| Source | Records | Evidence type | Limitation |
|---|---|---|---|
| Capterra reviews | 273,727 | Software-review records | A review is not a structured pain point |
| Capterra structured pain points | 39,935 | AI-extracted, severity-scored complaints | Structured subset of the review corpus |
| Capterra feature gaps | 40,937 | Documented feature requests | A request is not proof of willingness to pay |
| Capterra category pain points | 5,040 | Aggregated systemic complaints | Affected companies are tracked vendors, not end users |
| G2 reviews / insights | 129,883 / 7,989 | Review records / structured insights | Current snapshot, not a lifetime total |
| App Store and Google Play reviews | 136,898 (99,501 rated 1-3) | Mobile review records | Collection over-samples negative reviews; not an organic negative rate |
| Reddit structured pain points | 2,070 | Community complaints | Directional, not payment validation |
| Upwork jobs analyzed | 5,351 | Paid-demand signals (pain-point frequency) | Budget fields are not published; frequency only |
| TrustMRR startups | 8,699 (3,787 with revenue) | Revenue and growth records | Present as ranges; mix of sources |
| Funded companies | 17,611 | Funding-announcement records, AI-scored | No dollar amounts stored; momentum proxy only |
| Stripe Index companies | 30,322 | Public Stripe directory, AI-scored | Not revenue data |
| Swiper validation cards | 71,214 | Human validation signals | A swipe is interest, not purchase intent |
For product-specific technical detail on how these sources are queried, see the data sources overview.
Collection, sampling, and deduplication
Reviews and discussions are collected from each source and normalized. Because collection is targeted at software feedback, some datasets over-sample negative sentiment by design. The clearest example is the app-store corpus: the 99,501 reviews rated 1-3 stars are collected for complaint analysis and must never be presented as an organic negative rate for a category.
Aggregated cluster and category analyses sample within token limits (roughly several hundred records per cluster) rather than reading every row, so cluster-level figures are representative estimates, not exhaustive counts.
Severity and market-gap definitions
Severity (scored out of 5) measures how frustrated users are in the underlying complaints. A score above 4.0 means people describe the problem in urgent terms, name the money or hours it costs them, and are actively seeking a replacement rather than venting.
Market gap (scored out of 10) measures how poorly the existing tools in a category serve the complaint. A 9.0 to 10.0 means users have tried what is on the market and it still fails them; a 7.0 means partial solutions exist but leave a clear wedge open.
Affected-company count refers to tracked software vendors exhibiting a systemic complaint, not the number of end users. The full definitions also appear on the how BigIdeasDB works page.
Model usage and human validation
Structured pain points, feature gaps, and opportunity scores are extracted by large language models, then spot-checked by humans. BigIdeasDB's own LLM pain-point-extraction benchmark measures how reliably models perform this task against a 900-record human gold standard, and informs which models are used in the pipeline. Build-difficulty ratings and editorial judgments are labeled as such and kept separate from measured scores.
Known limitations and bias
- Complaint data is a demand signal, not a business plan; it does not count competitors or estimate acquisition cost.
- Negative sentiment is over-sampled by design in review datasets.
- Regulated categories (health, finance, legal) carry compliance costs the scores do not capture.
- Self-reported revenue figures (for example, from Reddit) are labeled and not audited.
- Funded-company and Stripe datasets show presence and momentum, not revenue.
Refresh cadence, licensing, and corrections
Structured datasets are refreshed on a rolling basis and every published figure is tied to a dated snapshot so readers can see how current it is. Data is collected from publicly available sources and presented in aggregate; individual reviews and listings are anonymized and never republished with identifying detail.
Corrections: if a figure looks wrong, email ompatel@om.bigideasdb.com. Confirmed errors are corrected or the analysis is re-run, with the snapshot date updated. Research is authored by Om Patel.