> For the complete documentation index, see [llms.txt](https://evidano.gitbook.io/ailyze-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://evidano.gitbook.io/ailyze-docs/collect-research-data/collect-social-media-and-web-data.md).

# Collect Social Media & Web Data

Collect posts, comments, reviews, and web page content from social media and websites into files you can analyze, on demand or on a schedule.

This page shows you how to turn a list of links into ready-to-analyze data. Paste or upload your URLs, let Evidano match each one to its platform, and download the comments, posts, reviews, or page content as spreadsheets and documents. It is for anyone who needs public online discussion or reviews as a dataset. Every control, including the platform catalog, is listed in [Web Data Extraction Settings](/ailyze-docs/reference/settings/web-data-extraction.md).

## Before you start

* An Evidano account with a real email address. The first URL extractions are free; see [Plans, Limits & Credits](/ailyze-docs/reference/plans-limits-and-credits.md).
* The public URLs you want to collect: posts, videos, profiles, review pages, or ordinary web pages.
* A clear idea of what you want from each link, such as comments, reviews, or posts, because each URL is exported in one mode.

## Step 1: Choose Extraction

From your account page, click **Get Social Media Web Data** to open the **Extract Data from URLs** page. It has three tabs: **New Extraction**, **History**, and **Scheduled**. A new extraction runs through **Choose Extraction**, **Add URLs**, and **Download Results**.

<figure><img src="https://1178311500-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fi4GBXPMUDxszIpUVOARU%2Fuploads%2Fgit-blob-ff97f49f4fb4914e3d20c6f36462048e0bdcf8a3%2Fcollect-social-media-and-web-data-01-choose-extraction.png?alt=media" alt="Extract Data from URLs page showing the three tabs, the step indicator, and the Auto-detect Platform card"><figcaption><p>Auto-detect Platform is the recommended starting point; the platform catalog below it is for forcing one mode.</p></figcaption></figure>

Pick one of three options:

* **Auto-detect Platform** (recommended). Paste links from any mix of platforms. Each one is matched to its platform and best-supported export mode, short and share links are expanded, ordinary web pages are saved as documents, and you review every match before anything runs. Click **Continue with Auto-detect**.
* **Web pages (documents)**. Saves each page, including a social-media link, as a Word document of its content.
* **Or pick a platform and export mode**. Search or browse the catalog by **Social**, **Video**, **Reviews & shopping**, **Apps & games**, and **Community & news**. Each card shows a support badge, the **URL form** it expects, an **Export mode** menu (for example comments, reviews, posts, search results, followers), the **Data fields you'll get**, and **Preview sample output**. Click **Continue to Step 2** on the card you want.

The badges tell you how a platform held up in live tests: **Verified** completed at least one live test, **Conditional** worked for some URLs or modes only, and **Try it** is recognised but not confirmed. Platforms marked **Unavailable** cannot be collected.

## Step 2: Add URLs

Under **Add Your URLs**, choose an **Input Method**: **Paste URLs (one per line)**, or drag and drop a CSV or XLSX file with the URLs in a single column (any column name is fine). Lines that do not look like URLs are listed under "These lines will be skipped unless fixed:" so you can correct them. A batch is capped at a number of URLs the page tells you; split a longer list into batches.

<figure><img src="https://1178311500-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fi4GBXPMUDxszIpUVOARU%2Fuploads%2Fgit-blob-d149b1fe1eafb5fa84a81bdcae034b40156dae0b%2Fcollect-social-media-and-web-data-02-add-urls.png?alt=media" alt="Add URLs step with the paste box, file upload area, and the Advanced options panel expanded"><figcaption><p>Advanced options apply to the whole batch: item cap, replies, and a date window where the platform supports one.</p></figcaption></figure>

Open **Advanced options** to tune the batch:

* **Max items per URL** caps how many rows a link returns. You are charged per URL, not per row, and the cap is best effort.
* **Include replies** adds replies to comments where the platform returns them.
* **From date** and **To date** limit the window; some platforms ignore date filters.
* **Advanced mode** switches on deeper collection for platforms that offer it.

Click **Review URLs to Extract**. The **Review URLs** table shows the **Platform** and **Export mode** chosen for each link and what **You will receive**; change the mode on any row to override it. Then click **Start Extraction**. If a charge applies, a dialog shows the amount and explains that only URLs that return data are billed; confirm with **Extract Data**.

{% hint style="warning" %}
**Costs money:** Beyond your free URLs, extraction is a pay-as-you-go add-on drawn from your wallet. Failed URLs and URLs that return no data are not billed. See [Plans, Limits & Credits](/ailyze-docs/reference/plans-limits-and-credits.md).
{% endhint %}

## Step 3: Download Results

The last step, **Processing & Download**, runs in the background. You can leave the page: results are emailed to you and saved under **History**. When it finishes, an **Extraction Complete!** card shows **Total URLs**, **Completed**, **Failed**, and **No data** counts.

* **Download ZIP file** (or **Download All Results (ZIP)**) saves `extracted_data.zip`. It holds an XLSX, JSON, and CSV file for every social or structured URL, with columns specific to that source, and one Word document per general web page.
* **Preview data** opens the preview page described below.
* **Analyze now** carries the data straight into an analysis.

## Step 4: Send your data to analysis

Click **Analyze now**, or **Analyze this extraction** in **History**, to open **Select Analysis**. Pick a workflow and Evidano carries the extracted data into it without a re-upload. For spreadsheets, a **Themes & sub-themes** card lets you **Use AI to generate themes** or **Enter themes manually** before you click **Create Project**. For web pages, the Word documents go to document analysis.

## Step 5: Review past extractions and preview the data

The **History** tab lists every extraction with **Download all results**, **Preview data**, **Analyze this extraction**, **Cancel extraction** for one still running, and **Delete extraction**. Failed URLs show a per-URL **Retry**, and anything that was not billed carries a **Not charged** chip.

<figure><img src="https://1178311500-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fi4GBXPMUDxszIpUVOARU%2Fuploads%2Fgit-blob-df24b405fd4fa02bcc89b3a4adc953d63973da88%2Fcollect-social-media-and-web-data-03-history.png?alt=media" alt="History tab listing past extractions with their status and the download, preview, and analyze actions"><figcaption><p>History keeps every run, so a closed browser tab never loses a result.</p></figcaption></figure>

**Preview data** opens a **Data preview** for one extraction: the **Data source**, **All Authors** and **Unique Authors** counts, the **Average Rating** where reviews were collected, and a **Volume over time** chart you can filter to the last days, with a ZIP export of what you are looking at.

## Step 6: Schedule a recurring extraction

The **Scheduled** tab repeats an extraction without you: pick a **Frequency** of **Hourly**, **Daily**, **Weekly**, or **Monthly** with a time zone, and each run appears in **History**. A schedule shows as **Active**, **Paused**, or paused for insufficient funds, with **Run now**, **Pause schedule**, **Resume schedule**, **Preview latest run**, **Edit schedule**, and **Delete schedule** actions. Each run consumes URL extractions like a normal extraction, so keep the wallet topped up for long-running schedules.

## Responsible use

Only collect **publicly available** data, and use it in line with each platform's terms of service and the privacy laws that apply to you (for example, GDPR, Europe's data-protection law, or CCPA, California's). Avoid collecting personal data you don't have a lawful reason to process.

## Next steps

* Find the themes in what you collected: [Analyze social media discussions](/ailyze-docs/common-research-workflows/analyze-social-media-discussions.md).
* Working with reviews instead? [Analyze product reviews](/ailyze-docs/common-research-workflows/analyze-product-reviews.md).
* A link rejected, an empty result, or an unexpected charge: [Social media & web data issues](/ailyze-docs/troubleshooting/social-media-web-data-issues.md).