Compute Bully

• Bully that uses Groq to generate a narrative response to a caption and then uses Play.ht to convert the narrative response to speech and play it back. See Screen and Webcam for the images that are used to generate the captions.

Tech Stack

• Python
• OpenCV
• LangChain
• Sentence Transformers
• Groq API
• Play.ht API
• Pydub
• Pyaudio
• dotenv

Setup

1. Clone the repository:

2. Create and activate a virtual environment:

   python -m venv venv
   source venv/bin/activate  # On Windows use `venv\Scripts\activate`

3. Install the required dependencies:

   pip install -r requirements.txt

4. Set up the environment variables:

   • Create a .env file in the root directory of the project.
   • Add the following environment variables to the .env file:

     GROQ_API_KEY=<your-groq-api-key>
     PLAYHT_API_KEY=<your-playht-api-key>
     PLAYHT_USER_ID=<your-playht-user-id>
     

5. Install the Ollama model for VLLM:

   ollama pull 0ssamaak0/xtuner-llava:phi3-mini-int4

6. Get API keys:

   • Groq API: (FREE)
   • Play.ht API: (5 dollars)
     • Clone the voice of your choosing.

Usage

1. Run the main script:

   python bullyAI.py

2. Interact with the application:

   • The application will capture images from your webcam periodically.
   • It will generate captions for the images using the Ollama model.
   • The captions will be processed by the Groq API to generate a narrative response.
   • The narrative response will be converted to speech using the Play.ht API and played back.

3. Stop the application:

   • Press q in the webcam preview window to stop the application.

Contributing

1. Fork the repository.
2. Create a new branch (git checkout -b feature-branch).
3. Make your changes.
4. Commit your changes (git commit -m 'Add some feature').
5. Push to the branch (git push origin feature-branch).
6. Open a pull request.

Acknowledgements

• LangChain
• Sentence Transformers
• Groq
• Play.ht
• Pydub
• OpenCV