Skip to content

Commit

Permalink
Merge branch 'main' of https://github.com/kaiban-ai/KaibanJS into str…
Browse files Browse the repository at this point in the history
…ucture-task-output
  • Loading branch information
anthonydevs17 committed Dec 20, 2024
2 parents 4cca258 + a52073d commit 73a5c0c
Show file tree
Hide file tree
Showing 8 changed files with 574 additions and 137 deletions.
57 changes: 46 additions & 11 deletions packages/tools/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,53 @@
# Changelog

## [0.1.0]
All notable changes to the `@kaibanjs/tools` package will be documented in this file.

Initial experimental release of the tools package.
## [0.4.1] - 2024-12-19

### Features
### Documentation
- Added missing README files for:
- Exa Search Tool
- Firecrawl Tool
- Updated main package README with improved tool listing
- Standardized documentation format across all tools

- Basic tool package implementation
- Firecrawl tool integration
- Tavily tool integration
- Storybook playground for tools
- Basic testing suite per tool
- CI workflows for releasing and testing
## [0.4.0] - 2024-12-19

### Notes
### Added
- Zapier Webhook Tool for workflow automation integration
- Make Webhook Tool for Make (formerly Integromat) integration

- This is an experimental release
## [0.3.0] - 2024-12-14

### Added
- Simple RAG Tool for basic RAG implementations
- Website Search Tool for semantic website content search
- PDF Search Tool for document analysis
- Text File Search Tool for plain text processing

### Enhanced
- Added support for custom vector stores
- Improved documentation for all tools
- Added comprehensive examples in tool READMEs

## [0.2.0] - 2024-11-17

### Added
- Serper Tool for Google Search API integration
- WolframAlpha Tool for computational queries
- Exa Search Tool for neural search capabilities
- GitHub Issues Tool for repository management

### Improved
- Enhanced error handling across all tools
- Better type definitions and input validation
- Updated documentation with more examples

## [0.1.0] - Initial Release

### Added
- Initial package setup
- Basic tool implementation structure
- Core utility functions
- Testing framework setup
- Documentation foundation
144 changes: 18 additions & 126 deletions packages/tools/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,130 +21,22 @@ npm install @kaibanjs/tools

## Available Tools

### 1. Firecrawl

Firecrawl is a tool that allows agents to interact with the Firecrawl web scraping service, enabling them to extract clean, structured data from websites.

Learn more: https://www.firecrawl.dev/

### 2. Tavily Search

Tavily Search is a tool that provides AI-optimized search capabilities, delivering comprehensive and accurate results. It's particularly useful for retrieving current information and answering questions about recent events.

Learn more: https://tavily.com/

### 3. Serper

Serper is a tool that integrates with Google Search API service, providing access to various types of search results including web search, news, images, and more. It's particularly useful for retrieving real-time information from Google's search engine.

Learn more: https://serper.dev/

### 4. Exa Search

Exa Search is a tool that integrates with Exa (https://exa.ai/), a search engine for AI that organizes the web using embeddings. It provides high-quality web data specifically optimized for AI applications.

Key features:

- Neural Search: Meaning-based search using embeddings
- Keyword Search: Traditional search capabilities
- Auto Search: Dynamically chooses between neural and keyword
- Category-focused search (company, research paper, news, github, tweet, etc.)
- Domain and text filtering
- Date-based filtering

Learn more: https://exa.ai/

### 5. WolframAlpha

WolframAlpha is a tool that integrates with WolframAlpha's computational intelligence engine, providing detailed answers to complex queries across various domains. It's particularly powerful for mathematical, scientific, and technical computations.

Key features:

- Advanced computations and data analysis
- Scientific and mathematical calculations
- Real-time data processing
- Domain-specific knowledge in:
- Mathematics
- Physics
- Chemistry
- Engineering
- Earth Sciences
- Life Sciences
- Units & Measures
- Financial calculations
- And more

Learn more: https://www.wolframalpha.com/

### 6. Github Issues

Github Issues is a tool that allows agents to interact with the Github API, enabling them to fetch issues from a repository.

Learn more: https://docs.github.com/en/rest/issues/issues

### 7. Simple RAG

The Simple RAG Tool integrates with various components from the langchain library to provide a simple interface for asking questions and retrieving answers using the Retrieval-Augmented Generation (RAG) approach. By default use MemoryVectorStore for storing vectors, OpenAIEmbeddings for embeddings, and OpenAI LLM for generating answers.

#### Features

- Integrates with the RAG Toolkit for handling the RAG process
- Uses a Loader instance to load and process documents
- Utilizes a Chunker instance to chunk and process text
- Employs an Embeddings instance for handling embeddings
- Stores vectors using a VectorStore instance
- Leverages an LLM instance for the language model
- Defines a promptQuestionTemplate for asking questions
- Requires an OpenAI API key for interacting with the OpenAI API

#### Usage

```js
const tool = new SimpleRAG({
OPENAI_API_KEY: 'your-openai-api-key',
content: 'Large content to process',
});
const result = await tool._call({ query: 'question to ask' });
```

or

```js
const tool = new SimpleRAG({
OPENAI_API_KEY: 'your-openai-api-key',
});
const result = await tool._call({
content: 'large content to process',
query: 'question to ask',
});
```

### 8. Website Search

The Website Search Tool is specifically crafted for conducting semantic searches within the content of a particular website. Leveraging a Retrieval-Augmented Generation (RAG) model, it navigates through the information provided on a given URL. Users have the flexibility to either initiate a search across any website known or discovered during its usage or to concentrate the search on a predefined, specific website.

#### Features

Conduct semantic searches within the content of a specific website.
Utilize a RAG model to navigate and retrieve relevant information.
Flexibility to search across any website or focus on a predefined one.
Usage

#### Usage

```js
import { WebsiteSearch } from '@kaibanjs/website-search';

const tool = new WebsiteSearch({
OPENAI_API_KEY: 'your-openai-api-key',
url: 'https://example.com',
});

const result = await tool._call({
query: 'What is the main topic of this website?',
});
console.log(result);
```
Here's a list of all available tools. Click on the tool names to view their detailed documentation.

| Tool | Description | Documentation |
|------|-------------|---------------|
| Exa | AI-focused search engine using embeddings to organize web data | [README](src/exa/README.md) |
| Firecrawl | Web scraping service for extracting structured data | [README](src/firecrawl/README.md) |
| GitHub Issues | GitHub API integration for fetching and analyzing repository issues | [README](src/github-issues/README.md) |
| PDF Search | Extract and search content from PDF documents | [README](src/pdf-search/README.md) |
| Serper | Google Search API integration with support for multiple search types | [README](src/serper/README.md) |
| Simple RAG | Basic Retrieval-Augmented Generation implementation for Q&A | [README](src/simple-rag/README.md) |
| Tavily Search | AI-optimized search engine for comprehensive and accurate results | [README](src/tavily/README.md) |
| Text File Search | Search and analyze content within text files | [README](src/textfile-search/README.md) |
| Website Search | Semantic search within website content using RAG models | [README](src/website-search/README.md) |
| WolframAlpha | Computational intelligence engine for complex queries and calculations | [README](src/wolfram-alpha/README.md) |
| Zapier Webhook | Integration with Zapier for workflow automation | [README](src/zapier-webhook/README.md) |
| Make Webhook | Integration with Make (formerly Integromat) for workflow automation | [README](src/make-webhook/README.md) |

## Development

Expand Down Expand Up @@ -186,13 +78,13 @@ VITE_WOLFRAM_APP_ID=your_wolfram_app_id
npm run storybook
```

5. Build the package:
6. Build the package:

```bash
npm run build
```

6. Run tests:
7. Run tests:

```bash
npm run test
Expand Down
94 changes: 94 additions & 0 deletions packages/tools/src/exa/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# Exa Search Tool

This tool integrates with Exa (https://exa.ai/), a search engine for AI that organizes the web using embeddings. It provides high-quality web data specifically optimized for AI applications, offering advanced search capabilities through neural and traditional keyword approaches.

## Components

The tool uses the following components:

- An Exa API client instance
- An API Key for authentication
- A custom HTTP client (ky) for making API requests
- Input validation using Zod schema
- Configurable search parameters
- Multiple search type options

## Key Features

- Neural Search: Meaning-based search using embeddings
- Keyword Search: Traditional search capabilities
- Auto Search: Dynamically chooses between neural and keyword
- Category-focused search (company, research paper, news, github, tweet, etc.)
- Domain and text filtering
- Date-based filtering
- Configurable content retrieval options
- Support for autoprompt query enhancement

## Input

The input should be a JSON object with a "query" field containing the search query to process.

## Output

The output is the response from Exa's API containing search results based on the configured parameters and search type.

## Configuration Options

- `type`: Search type ('neural', 'keyword', or 'auto')
- `useAutoprompt`: Enable query enhancement (for neural search)
- `numResults`: Number of results to return
- `category`: Focus on specific category
- `startPublishedDate`: ISO 8601 date for earliest publish date
- `endPublishedDate`: ISO 8601 date for latest publish date
- `includeDomains`: List of domains to include
- `excludeDomains`: List of domains to exclude
- `includeText`: Text/phrase to include in results
- `excludeText`: Text/phrase to exclude from results
- `startCrawlDate`: ISO 8601 date for earliest crawl date
- `endCrawlDate`: ISO 8601 date for latest crawl date
- `contents`: Configuration for content retrieval

## Example

```javascript
const tool = new ExaSearch({
apiKey: 'your-api-key',
type: 'neural',
useAutoprompt: false,
numResults: 10,
category: 'company'
});

const result = await tool._call({
query: 'AI companies focusing on natural language processing'
});
```

## Advanced Example with Filters

```javascript
const tool = new ExaSearch({
apiKey: process.env.EXA_API_KEY,
type: 'neural',
numResults: 20,
includeDomains: ['techcrunch.com', 'wired.com'],
startPublishedDate: '2023-01-01',
contents: {
text: { maxCharacters: 1000, includeHtmlTags: false },
highlights: { numSentences: 3, highlightsPerUrl: 2 }
}
});

try {
const result = await tool._call({
query: 'recent developments in quantum computing'
});
console.log(result);
} catch (error) {
console.error('Error performing Exa search:', error);
}
```

### Disclaimer

Ensure you have proper API credentials and respect Exa's usage terms and rate limits. Some features may require specific subscription tiers.
Loading

0 comments on commit 73a5c0c

Please sign in to comment.