Skip to content

Latest commit

 

History

History
98 lines (66 loc) · 2.7 KB

README.md

File metadata and controls

98 lines (66 loc) · 2.7 KB
Raw

Tweet Scraper API

This API allows you to scrape tweets from the CoinDesk Twitter channel, save them to a database, and retrieve them through a RESTful interface.

Getting Started

Follow these steps to set up and run the Tweet Scraper API using yarn:

  1. Clone the Repository: Clone this repository to your local machine.

    git clone https://github.com/ezedinff/tweet-scraper-api

  2. Install Dependencies: Navigate to the project directory and install the dependencies using yarn.

    cd tweet-scraper-api
yarn install

  3. Create Resend Account and Generate API Key: Visit the Resend website and sign up for an account if you haven't already. Once logged in, generate an API key for your account.

  4. Set Up Environment Variables: Create a .env file in the root directory and configure the following environment variables:

    DB_HOST=127.0.0.1
DB_PORT=5432
DB_USERNAME=postgres
DB_PASSWORD=postgres
DB_DATABASE=scrapper_db
PORT=3000
FROM_EMAIL=<sender-email>
TO_EMAIL=<recipient-email>
RESEND_API_KEY=<resend_api_key>

  5. Run the Application: Start the application.

    yarn start

  6. Access the API: Once the application is running, you can access the API endpoints using the base URL http://localhost:3000.

Endpoints

1. GET /api/tweets

Retrieve scraped tweets from the CoinDesk Twitter channel.

Request:

  • Method: GET
  • URL: /api/tweets?page=1&pageSize=10

Response:

{
  "items": [
    {
      "username": "CoinDesk",
      "text": "Explore the trajectory of finance in the digital age with one of the industry’s key figures. Join @bullish CEO @ThomasFarley at #Consensus2024 in Austin where insightful perspectives await this May. Discover all the speakers.",
      "image": null
    },
    // Additional tweets...
  ],
  "totalCount": 22,
  "page": "1",
  "pageSize": "10"
}

Screenshots

You can find screenshots of the request and response of the /api/tweets endpoint below:

Request Screenshot

Get Tweets Request

Response Screenshot

Get Tweets Response

Testing

You can test the API endpoints using tools like cURL or Postman. Here's an example cURL command to retrieve tweets:

curl -X 'GET' 'http://localhost:3000/api/tweets?page=1&pageSize=10' -H 'accept: */*' -H 'Content-Type: application/json'

License

This project is licensed under the MIT License. You are free to use, modify, and distribute the code for both commercial and non-commercial purposes, as long as you include the original copyright notice and license terms.