Open Deep Research: generating AI research based on web search content

General Introduction

Open Deep Research is an open source AI-driven research report generation tool that serves as an open source alternative to Google Gemini's deep research capabilities. Developed in TypeScript and built on the Next.js 15 framework, the project integrates the Azure Bing search API and Google Gemini AI Model. It automates the extraction of valuable information from web search results and generates in-depth analysis reports. The system works through three core steps: first using the Bing Search API to obtain comprehensive search results, then using JinaAI to extract and process content from selected web pages, and finally applying AI technology to generate customized research reports. The project supports multiple export formats and implements request rate limiting to ensure stability, making it a full-featured open source research assistant.

Related Recommendations:STORM: Search web data based on Topic to generate papers with citations, long paper reports

Function List

• Supports web search with time filtering
• Intelligent extraction of web content and key information
• AI-driven automated report generation
• Multi-format export support (PDF, Word, Text)
• Built-in request rate limiting protection
• Responsive design for multiple devices
• Customizable number of search results and selection limits
• Support for secure search level adjustment
• Flexible market/region setting options

Using Help

1. Preparation for deployment

Before you start using Open Deep Research, you need to prepare the following environment and resources:

• Node.js version 18 or higher
• Package manager (npm, yarn, pnpm or bun)
• Azure subscription key (for Bing search)
• Google Gemini Pro API key
• Upstash Redis account (for rate limiting)

2. Installation steps

1. Clone the code repository:

git clone https://github.com/btahir/open-deep-research
cd open-deep-research

2. Install the dependencies:

npm install

3. Configure environment variables:
   establish.env.localfile and fill in the necessary API keys:

AZURE_SUB_KEY=Your Azure Subscription Key
GEMINI_API_KEY=your Gemini API key
UPSTASH_REDIS_REST_URL=Your Redis URL
UPSTASH_REDIS_REST_TOKEN=Your Redis Token

4. Start the development server:

npm run dev

3. System configuration

You can do this by modifying thelib/config.tsfile to customize the following settings:

Rate Limit Setting:

rateLimits: {
enabled: true, // enable/disable rate limiting
search: 5, // number of search requests per minute
contentFetch: 20, // number of content fetch requests per minute
reportGeneration: 5, // number of report generation requests per minute
}

Search Settings:

search: {
resultsPerPage: 10, // number of results per page
maxSelectableResults: 3, // Maximum number of results that can be selected by the user
safeSearch: 'Moderate', // safe search settings
market: 'en-US', // search market/region
}

4. Utilization process

1. Visit the application homepage (http://localhost:3000 for local development environment)
2. Enter the topic or keyword to be researched
3. The system will get the search results via Bing API
4. Select the search results to be analyzed (up to 3)
5. The system will automatically extract the content of the selected page
6. AI will process the extracted content and generate a research report
7. Select the desired export format (PDF, Word or Text) to download the report

5. Cautions

• Ensure that the API key is configured correctly and is valid
• Be aware of API usage limits and avoid exceeding quotas
• It is recommended that rate limiting be enabled in production environments
• Configuration parameters can be adjusted as required
• Periodically check the validity of the API key