Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/README.skills.md
Original file line number Diff line number Diff line change
Expand Up @@ -417,4 +417,4 @@ See [CONTRIBUTING.md](../CONTRIBUTING.md#adding-skills) for guidelines on how to
| [workiq-copilot](../skills/workiq-copilot/SKILL.md)<br />`gh skills install github/awesome-copilot workiq-copilot` | Guides the Copilot CLI on how to use the WorkIQ CLI/MCP server to query Microsoft 365 Copilot data (emails, meetings, docs, Teams, people) for live context, summaries, and recommendations. | None |
| [workshop-create](../skills/workshop-create/SKILL.md)<br />`gh skills install github/awesome-copilot workshop-create` | Create a new workshop or use an existing directory as one. Handles two paths: (A) use an existing local directory the operator points at, or (B) create a new private GitHub repo in the signed-in account. Never creates a repo inside another repo. | None |
| [write-coding-standards-from-file](../skills/write-coding-standards-from-file/SKILL.md)<br />`gh skills install github/awesome-copilot write-coding-standards-from-file` | Write a coding standards document for a project using the coding styles from the file(s) and/or folder(s) passed as arguments in the prompt. | None |
| [x-twitter-scraper](../skills/x-twitter-scraper/SKILL.md)<br />`gh skills install github/awesome-copilot x-twitter-scraper` | Build GitHub Copilot workflows with Xquik X API SDKs, REST endpoints, MCP tools, TweetClaw OpenClaw plugin installs, signed webhooks, tweet search, user lookup, follower exports, media actions, and agent automation. | None |
| [x-twitter-scraper](../skills/x-twitter-scraper/SKILL.md)<br />`gh skills install github/awesome-copilot x-twitter-scraper` | Build GitHub Copilot workflows with Xquik X API SDKs, REST endpoints, hosted Apify Actor runs, MCP tools, TweetClaw OpenClaw plugin installs, signed webhooks, tweet search, user lookup, follower exports, media actions, and agent automation. | None |
47 changes: 45 additions & 2 deletions skills/x-twitter-scraper/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: x-twitter-scraper
description: 'Build GitHub Copilot workflows with Xquik X API SDKs, REST endpoints, MCP tools, TweetClaw OpenClaw plugin installs, signed webhooks, tweet search, user lookup, follower exports, media actions, and agent automation.'
description: 'Build GitHub Copilot workflows with Xquik X API SDKs, REST endpoints, hosted Apify Actor runs, MCP tools, TweetClaw OpenClaw plugin installs, signed webhooks, tweet search, user lookup, follower exports, media actions, and agent automation.'
---

# X Twitter Scraper
Expand All @@ -14,6 +14,7 @@ Use this skill when a user wants to integrate Xquik into an app, script, data pi
- Start extraction jobs for replies, reposts, quotes, likes, lists, communities, articles, and search results.
- Create account monitors and verify HMAC-signed webhook events.
- Add TypeScript, Python, Go, Java, Kotlin, C#, Ruby, PHP, CLI, or Terraform clients.
- Run hosted tweet and audience collection through Apify Actors.
- Connect agent runtimes through the Xquik MCP server.
- Install TweetClaw when the workflow belongs inside OpenClaw and needs plugin-managed approvals for X account actions.

Expand All @@ -28,13 +29,15 @@ Before writing code, inspect the current Xquik source material:
- Skill repo: https://github.com/Xquik-dev/x-twitter-scraper
- TweetClaw OpenClaw plugin: https://github.com/Xquik-dev/tweetclaw
- TweetClaw npm registry metadata: https://registry.npmjs.org/@xquik%2Ftweetclaw
- X Tweet Scraper Actor: https://apify.com/xquik/x-tweet-scraper
- X Follower Scraper Actor: https://apify.com/xquik/x-follower-scraper

Do not invent endpoint names, request fields, response fields, scopes, pricing, limits, or package names. Read the relevant SDK README and API reference page first.

## Implementation Flow

1. Identify the workflow: search, lookup, extraction, monitor, webhook, media, write action, billing, or MCP.
2. Choose the integration surface: generated SDK for application code, REST for custom clients, MCP for agents, TweetClaw for OpenClaw plugin workflows, or webhooks for event delivery.
2. Choose the integration surface: generated SDK for application code, REST for custom clients, Apify Actors for hosted collection, MCP for agents, TweetClaw for OpenClaw plugin workflows, or webhooks for event delivery.
3. Confirm authentication requirements from the docs and use environment variables for API keys.
4. Use typed request and response models when an SDK exists for the user's language.
5. Add retries and pagination according to the SDK or API docs.
Expand All @@ -54,6 +57,43 @@ When application code is involved, match the SDK to the user's project language:

Use project-native typed request and response models. Keep network calls in server-side code unless the SDK docs explicitly support browser use.

## Apify Actor Pattern

Use the Apify path when a workflow needs hosted runs, datasets, schedules, or Apify-native orchestration.

| Need | Actor | REST ID |
|---|---|---|
| Tweets, search, timelines, lists, articles, replies, quotes, threads, retweeters, or best-effort favoriters | `xquik/x-tweet-scraper` | `xquik~x-tweet-scraper` |
| Followers, following, verified followers, list members, list subscribers, or community members | `xquik/x-follower-scraper` | `xquik~x-follower-scraper` |

Authenticate with an Apify API token. Keep it in `APIFY_API_TOKEN`. Fetch the current input schema from the relevant Actor page before selecting fields.

Start a bounded tweet run:

```bash
curl --fail --silent --show-error --request POST \
"https://api.apify.com/v2/actors/xquik~x-tweet-scraper/runs" \
--header "Authorization: Bearer ${APIFY_API_TOKEN}" \
--header "Content-Type: application/json" \
--data '{"twitterHandles":["apify"],"outputVariant":"rich","maxItems":25}'
```

Start a bounded follower run:

```bash
curl --fail --silent --show-error --request POST \
"https://api.apify.com/v2/actors/xquik~x-follower-scraper/runs" \
--header "Authorization: Bearer ${APIFY_API_TOKEN}" \
--header "Content-Type: application/json" \
--data '{"twitterHandles":["apify"],"relation":"followers","outputMode":"compact","maxItems":50}'
```

Record the returned run ID. Poll the Actor run with a bounded retry loop. Stop on `SUCCEEDED`, `FAILED`, `ABORTED`, or `TIMED-OUT`. On success, read `defaultDatasetId`, then fetch its dataset items.

Treat `maxItems` as the cap for the entire tweet run, including runs with several search terms. Keep follower target metadata when attribution matters. Treat rows with `resultType: "diagnostic"` as status information, not scraped records. Inspect any run-report row before trusting an incomplete result.

Review each Actor's live Apify pricing box before every paid run. Apify platform usage may apply separately. Start with a small `maxItems` value and ask before raising the cap.

## Webhook Pattern

When adding webhook handlers:
Expand Down Expand Up @@ -84,5 +124,8 @@ Treat create, reply, quote, like, bookmark, retweet, follow, delete, media, and
- Do not bypass access controls or platform policies.
- Do not expose API keys, webhook secrets, account cookies, tokens, or raw signatures.
- Do not hard-code credentials in examples or tests.
- Never put Apify API tokens in URL query parameters.
- Do not document private infrastructure details.
- Prefer official Xquik docs, SDK READMEs, and the OpenAPI spec over memory.

Xquik is an independent third-party service. Not affiliated with X Corp. "Twitter" and "X" are trademarks of X Corp.
Loading