How to Scrape Reddit in 2026 (We Tested What Still Works)

To scrape Reddit in 2026, you can no longer append .json to a URL from a script. On October 18, 2026 we sent 11 plain requests to Reddit with no login, and none of the seven JSON requests returned data. Three routes work today, and two are on a clock. Anonymous RSS feeds end on November 13, 2026 by Reddit’s own announcement, and new public API requests stop on October 31, 2026. Fetching through your own logged-in browser at a normal pace is the route Reddit has not announced an end for.

The top results are forum threads from people stuck on 403 errors, a deprecated marketplace listing, and 2024 and 2025 tutorials that start with “create a script app and copy your client ID.” Google’s answer box still says the same, and that step is no longer self-service.

We run Reddit research through the BigIdeasDB MCP daily, so we hit this wall early. The BigIdeasDB Reddit Connector is our answer, and we say below what it is for and what it is not for.

How do you scrape Reddit in 2026? The short answer

The seven steps below follow that order. If you only want the working setup, jump to step 4 and step 5. If you came for the rules, jump to is scraping Reddit allowed.

What changed in 2026: the dated timeline

Reddit’s changes arrive in sequence, and most now carry a date. Here is the order, with the source for each line.

Read the table in date order. New API requests close October 31, RSS goes November 13, removal begins January 12 and public API access ends in March 2027. The Developer Platform that apps are meant to move to is not ready to absorb them yet. Reddit’s admin post on the September 30 deadline says that porting comes later and “None of this is changing today. It honestly won’t happen this year.” Reddit’s own words about RSS are blunter:

“If you use RSS for feeds outside of a community you moderate, there is no replacement.” – r/modnews, Reddit moderator newsletter

The stated reason is that RSS is a common surface for large-scale scraping and automated abuse. Both the anonymous door and the approved-app door are closing on dates Reddit has published.

Two older changes sit under that timeline. Reddit’s Responsible Builder Policy states that you must request access and get explicit approval before accessing any Reddit data through the API. And the site’s robots.txt, which links to the Public Content Policy, blocks every user agent with User-agent: * followed by Disallow: /. Developers describe the approval side:

“I applied twice for Reddit Data API access since the new approval process started.” – r/redditdev

On the .json side, people try the old trick, get a 403 and assume their script is broken:

“Tried using Reddit's Public JSON endpoints- Python got blocked(403 error)” – r/learnpython

“Pretty much every self-hosted app that used Reddit for me is now returning 403. I couldn't find any news or announcement this anywhere” – r/selfhosted

The script is not broken, and low volume does not fix it. One developer wrote that they were scraping “pretty conservatively maybe 30-60 requests per hour” before the 403s started (r/webscraping). The full fix guide is why a Reddit .json URL returns 403.

We tested every endpoint, logged out and logged in

On October 18, 2026 at about 18:59 UTC we sent one request to each endpoint below from a residential IP address, with a current Chrome User-Agent, no cookies, two seconds apart. Then we compared against the same kinds of requests made through a logged-in browser session by the Reddit Connector.

Read the table in three parts. Every JSON route returned 403 or a login redirect to a logged-out client, so the answer to “does .json still work from a script” is no. The page you see without logging in is a shell with no posts in the HTML, so parsing the public page does not help either. And RSS still serves listings anonymously, which is the one anonymous door we found open.

Two details matter for troubleshooting. A real browser User-Agent did not change the result, so the common advice to set a user agent is not the fix. And the block is the same on every host we tried. One scraper author described the cause as Reddit having “restricted the JSON endpoints to logged in cookies only” (r/webscraping). We cannot verify why Reddit did it, but a logged-in session getting JSON while a logged-out request does not matches our test.

Step 1: Decide what you need

Sort the job into one of three sizes before you pick a method, because each size has a different legitimate route.

• A research question. You want to read 10 to 100 threads on a topic and summarise them. Your own logged-in session at a human pace fits this (steps 4 to 6).
• A small recurring read. You want the latest posts from a few subreddits on a schedule. RSS fits this (step 2).
• A dataset or a product. You want thousands of posts, or you plan to sell, license or train on the data. This needs Reddit’s approval first (step 3). No method in this guide is a substitute for it.

Step 2: Use RSS for light listing reads (until November 13, 2026)

Append .rss to a subreddit listing URL and you get an Atom feed with no login. In our test, the top-posts-of-the-week feed for one subreddit returned status 200 with a content type of application/atom+xml. A request for five posts returned five entries, and each entry carried an HTML content field with the post body, between roughly 1.5K and 3K characters in our run.

We only tested the top listing with a week window. A single thread has a feed too: we fetched one thread’s feed once and it returned 200 with 120 entries, which looks like the post plus its comments. That is one observation, so check it on your own thread before you build on it.

The limits are real, and they are why RSS is step 2 and not the whole answer.

• Search rate limits at once. The one search feed we requested returned 429 on the first call. Do not plan a keyword monitor on search RSS.
• It is listings and single threads only. There is no score filter and no cross-subreddit search.
• It has an end date. Reddit’s stated RSS shutdown is November 13, 2026. Anything built on feeds needs a plan for that day.

Use RSS now when you want the newest or top posts of a few communities delivered to a script or a reader. If you want an AI client to search and read threads on demand, or you need anything after November 13, move to step 4.

Step 3: Apply for an approved app if you qualify

The official route is the Reddit Data API, and it now starts with an application. The Responsible Builder Policy sets three conditions: get explicit approval before accessing Reddit data through the API, do not misrepresent or mask how or why you access it, and respect the limits. Commercial use needs explicit written approval. Approval is not quick. One developer in r/redditdev wrote “I applied twice for Reddit Data API access since the new approval process started,” and a thread titled “Academic API Access Rejected” shows that rejections happen.

If you qualify and you need programmatic access, apply, and use a Python wrapper library or plain HTTP once you have credentials. Our guide to the Python wrapper library alternative covers when an AI client is a simpler fit than writing client code, and the Reddit API alternative for AI page covers the approval-versus-browser trade-off in more depth.

Timing is the problem. New requests for public API access stop after October 31, 2026, removal begins January 12, 2027 for apps that have not registered, and remaining public access ends in March 2027. Approval is neither instant nor guaranteed, so do not tie a launch date to it.

Step 4: Fetch through your own logged-in browser

The logged-in column of our table is the whole idea. A request from your own browser with your own Reddit login returned JSON for listings, search and threads in our run. So instead of fighting the block from a server, you ask your browser to fetch the page and hand the result to your AI client. Reddit has announced no end date for this route, and it can change it at any time.

That is what the Reddit Connector does. It is a switch inside the Reddit scraper Chrome extension from BigIdeasDB. When your AI client calls a Reddit tool, BigIdeasDB sends a wake-up message to Clipper. Clipper fetches that reddit.com page in your Chrome, using your login, and returns the result so the client can answer. Fetches are paced at about one per second, and on October 18 one request measured about 1.2 seconds from our push to the browser fetch to the result. Results are passed back and deleted from BigIdeasDB servers once delivered. Nothing runs between your requests, and it is off until you turn it on.

What you need:

• Chrome, running on your computer (it can sit in the background).
• Clipper version 1.2.0 or later, with the Reddit Connector turned on.
• A Reddit login in that same Chrome profile.
• BigIdeasDB Pro, which includes the MCP and the Reddit tools.

What you do not need: Reddit API keys, a developer account, an approval wait, proxies or a server. The Reddit Connector setup page has the click-by-click version, and the Clipper extension docs cover the rest of the extension.

Step 5: Connect your AI client

The Reddit tools live inside the BigIdeasDB MCP, so any MCP client can call them. Connect the MCP to the client you use. The Claude connection guide walks through Claude, and the MCP overview explains what the hosted server exposes. Cursor and ChatGPT connect the same way through their own MCP settings.

Four Reddit tools appear once it is connected:

• search_reddit searches posts by keyword, optionally inside one subreddit.
• fetch_subreddit reads posts from a subreddit by sort and time window.
• fetch_post_comments loads a full comment thread.
• fetch_reddit_json reads any Reddit JSON path for edge cases.

The full parameter list is in the MCP tools reference, and rate and client notes are in MCP rate limits and clients. Run Test connection in the Reddit tab of Clipper. It should show that you are connected to your Reddit account. If it says you are not logged in, log in to reddit.com in that Chrome profile and test again.

Reddit is also the most used part of this MCP. In the last 30 days the four Reddit tools made up 17,000+ of 32,000+ tool calls (53%) across 50 tools, and 30 of the 57 users who called any tool called a Reddit one. Searching was the biggest single use: search_reddit alone logged 7,000+ calls, ahead of fetch_post_comments at 5,000+. The client side is covered in Reddit data for Claude and Cursor.

Step 6: Ask narrow questions and keep the evidence

A vague prompt wastes fetches. A narrow prompt gets you a usable answer in a few calls. Give the client one subreddit or one topic, one time window and one output.

• “Search r/smallbusiness for threads about invoicing from the last month. Read the top 10 and list the five most repeated complaints with one short quote each.”
• “Load the comments on this thread and separate the answers into things people paid for and things they built themselves.”
• “Read the top posts of the week in r/Entrepreneur and tell me which three describe a problem a small product could solve.”

Save the thread excerpt behind each finding, with the subreddit as the source and no username. A claim without a saved excerpt is a guess.

If the goal is business ideas, our guide to finding business ideas on Reddit shows the full workflow, and the Reddit market research help page covers the prompts we use. For a research pipeline that stays inside a human pace, see the Reddit research MCP.

Step 7: Keep a human pace and stay inside the rules

The last step is a set of habits, not a tool. Reddit’s Responsible Builder Policy asks everyone to respect limits, to be transparent about how and why they access data, and not to cause disruption through excessive use. Turn that into a short checklist.

• One browser, one session, one person. No parallel sessions and no extra accounts.
• Read only. Do not post, vote or message through any automated path.
• Pull what the question needs. If a question needs 40 threads, do not pull thousands.
• Do not mask what you are doing or why.
• Keep copies short and sourced. Do not republish threads in bulk.
• Apply for approval before any commercial use, resale, licensing or model training.

If a project outgrows that list, it has outgrown this guide. Ask Reddit for approval or a data license rather than scaling a workaround.

How to scrape Reddit comments

Comments are the hardest part to get anonymously. A comment thread .json returned 403 logged out in our test. A thread feed returned 200 with 120 entries on the one thread we tried, the only anonymous route we found. Logged in, the thread JSON returned normally.

For an AI workflow, hand fetch_post_comments a thread URL and ask for structure, for example “group the replies into complaints, workarounds and tools people named.” That beats “summarise the thread.” Quote narrowly, name the subreddit and drop usernames.

How to scrape Reddit with Python

Python can still read Reddit, but not the way older tutorials show. Appending .json from a script returned 403 on every host we tried, and the API route now starts with approval. What works without approval is the feed. This snippet uses only the standard library. We fetched a live feed with a browser User-Agent on October 18, 2026 and parsed it with this code: five entries for a limit of five. The descriptive User-Agent above is untested and Reddit may treat it differently, so check the response status.

import urllib.request
import xml.etree.ElementTree as ET

URL = "https://www.reddit.com/r/smallbusiness/top/.rss?t=week&limit=5"
UA = "research-script/1.0 (contact: you@example.com)"

req = urllib.request.Request(URL, headers={"User-Agent": UA})
with urllib.request.urlopen(req, timeout=20) as r:
    root = ET.fromstring(r.read())

ns = {"a": "http://www.w3.org/2005/Atom"}
for e in root.findall("a:entry", ns):
    title = e.find("a:title", ns).text
    link = e.find("a:link", ns).attrib["href"]
    body = e.find("a:content", ns).text or ""
    print(title, link, len(body))

Run it a few times a day, not a few times a second. It stops working when Reddit ends RSS support on November 13, 2026, it can return 429 or 403 before then, and it returns listings only. For a job that needs search and comments, the Python wrapper library alternative compares the routes.

How to scrape Reddit for free

“Free” has a few honest answers, and each has a catch.

One of the top forum threads for this query asked for exactly this and got a blunt answer from a commenter:

“No free options for sure.” – r/learnpython

That is too pessimistic, since RSS and approved access exist. The Reddit Connector needs BigIdeasDB Pro. Our free Reddit keyword research tool covers the free end for keyword work, and the best Reddit research tools for founders list compares the rest.

Is scraping Reddit allowed?

Reddit’s published rules say you can use Reddit content for non-commercial purposes such as learning and community, and that you should talk to Reddit if you have commercial purposes in mind. The same Public Content Policy says Reddit is seeing more entities using unauthorized access, for example by scraping or using data brokers, to collect public data in bulk, and that Reddit does not believe third parties have a right to misuse public content just because it is public.

Read that as three tiers. Non-commercial, small-scale use is the tier Reddit describes as open. Commercial use needs Reddit’s written approval. Bulk collection through unauthorized access is the behavior Reddit says it is acting against. The robots.txt file, which disallows all automated crawling, points the same way.

We are not lawyers, and nothing here is legal advice. Policies change, so read the Public Content Policy and the Responsible Builder Policy yourself before you start anything beyond personal research.

Does Reddit allow AI scraping?

Not without approval. The Responsible Builder Policy says you must not sell, license, share or otherwise commercialize Reddit data without express written approval, and that the restriction extends to mining, scraping or using data to train machine learning or AI models. Reddit has licensing arrangements with some AI model makers, researchers and brand-monitoring companies, which is how it describes giving access under contract.

The policies are about collecting and reusing data at scale. They do not describe one person asking an assistant to read a few threads for their own research. We do not claim Reddit has approved that either. Anything that looks like model training, resale, a public page of pasted threads or a dataset needs Reddit’s approval first. See the Reddit API alternative for AI page for the trade-offs.

How to scrape Reddit without getting banned

Reports say speed is not the only trigger. The r/webscraping author above saw 403s at 30 to 60 requests per hour, and a thread author said browser automation ended in a ban from anti-bot detection (both self-reported, neither tested by us). Masking what you do is against the Responsible Builder Policy, and its enforcement section lets Reddit revoke tokens and suspend apps and accounts. So read at a human pace through your own session and use the step 7 checklist. The connector’s one-per-second pacing follows that logic. A job that needs more needs approval, not a bigger script.

The methods, ranked

Ranked by fit for one job, a research question answered at a human pace. The ranking is ours, from our October 18 test and Reddit’s published rules.

The Reddit Connector is first because it needs no approval wait and Reddit has announced no end date for logged-in browsing. We did not test rows 4 and 5 and have not ranked them on evidence.

What to do with the data once you have it

A thread with 300 upvotes tells you people agree, not that they would pay. Complaints are a pain signal, not a payment signal, so we check a Reddit finding against the other sources in our 1M+ complaint corpus (review sites and app stores) before trusting it. The complaint analysis help page shows how, and customer pain point analysis shows the method on real data. For ideas, see how to validate a startup idea, the Reddit pipeline builder and Reddit SaaS business ideas. For keyword work, try the free Reddit keyword research tool.

Methodology and limits

Every number on this page traces to one of the rows below. Where a source cannot support a claim, the limitation column says so.

This guide does not claim that the block will last, that Reddit will keep every date above, that any route is approved by Reddit for your use, or that any third-party service works or fails.

Frequently asked questions

BigIdeasDB Research. (2026). How to Scrape Reddit in 2026 (We Tested What Still Works). BigIdeasDB. Retrieved from https://bigideasdb.com/how-to-scrape-reddit