Skip to content

A Python crawler tool that can automatically simulate browser operations to crawl all users' tweet content and save all static resources (videos, pictures) locally without calling the Twitter API. Use the selenium library.

License

Notifications You must be signed in to change notification settings

kaixinol/twitter_user_tweet_crawler

Repository files navigation

Another better project of mine: offline-twitter-cyber-fortune-teller-py

Warning

Do not leak your cookie.json, this will lead to your Twitter account being stolen.

Introduction

  • This tool can automatically simulate browser operations to crawl all users' tweets and save all static resources (videos, pictures) locally without calling the Twitter API.
  • At the same time, sqlite3 is used to save the crawled data as an index file for easy query.

Installation & Configuration

  • Install Python3.10+
  • Install Poetry
  • Install Chrome 119.0+
  • Run the command poetry install in the directory with pyproject.toml
  • Configure config.yaml
  • Change the display language of your Twitter account to English (More>Settings and Support>Account information>Languages>Display language)

Run

  1. Run the command in the upper-level directory with pyproject.toml
poetry run python3 -m twitter_user_tweet_crawler
  1. Log in to Twitter
  2. Press the Enter key
  3. Done.

About

A Python crawler tool that can automatically simulate browser operations to crawl all users' tweet content and save all static resources (videos, pictures) locally without calling the Twitter API. Use the selenium library.

Topics

Resources

License

Stars

Watchers

Forks

Languages