Free-Search: API tool for free access to real-time Google search results

General Introduction

Free-Search is an open source API tool developed by Hanzla Javaid and hosted on GitHub. Its main function is to provide real-time Google search results through a custom search engine and crawl web content to return structured data. This tool is designed for AI agents and large-scale language modeling (LLM) applications to help improve the accessibility of internet data. Users can use it for free by simply downloading the code and deploying it locally. The project is developed in Python and supports custom search parameters for developers and technical teams. It is currently available with public demo examples for easy experience.

Function List

• Supports real-time Google search and returns the latest results after entering keywords.
• Automatically crawls search result pages and extracts the core content of each page.
• Returns structured data, including sources, links, and content snippets.
• Adjustable number of results (1-5) and length of each piece of content (100-5000 characters).
• Open source code, users are free to modify and optimize.
• Public demo examples are provided and can be tried without installation.

Using Help

Free-Search is a developer-oriented API tool that requires a certain technical foundation to install and use. Here is a detailed guide to help you get started.

How to install

Free-Search requires that you download the source code from GitHub and configure the runtime environment. Here are the steps:

1. clone warehouse
   Open your browser and visit https://github.com/HanzlaJavaid/Free-SearchClick on the green Code button, select Download ZIP Download the zip archive. Or enter it on the command line:

git clone https://github.com/HanzlaJavaid/Free-Search.git

Once the download is complete, go to the project folder.

2. Installation of dependencies
   The project requires Python 3.7 or later, and several key libraries. Run it from the command line:

pip install fastapi uvicorn pydantic playwright beautifulsoup4

This will install FastAPI (building the API), Uvicorn (running the server), Pydantic (data validation), Playwright (web crawling), and BeautifulSoup4 (parsing HTML).

3. Install Playwright Browser
   The project relies on Playwright's Chromium browser, which is installed by running the following command:

playwright install chromium

After the installation, the tool will be able to crawl the web page properly.

4. Starting services
   Run it in the project folder:

xvfb-run python main.py

If you don't have a graphical interface on your Linux system, thexvfb-run is required; Windows or Mac users can simply run the python main.pyThe API service will run locally after startup. Once started, the API service runs locally on default port 11235.

How to use the main features

The core of Free-Search is the search API, which is run to get results via HTTP requests. Here's how it works:

• Accessing API Endpoints
  Once the service has started, open a browser or command line tool and access the http://localhost:11235/search. You need to provide search parameters, such as keywords.
• Send search request
  utilization curl or other tools to send requests. Example:

curl "http://localhost:11235/search?query=fastapi+tutorial&max_results=2&max_content=1000"

• query: Search term, required, e.g. "fastapi tutorial".
• max_results: Number of results to return, optional, default 3, range 1-5.
• max_content: The length of the content of each result, optional, default 2000 characters, range 100-5000.
• View Return Results
  The API returns structured data in JSON format, for example:

[
{
"source": "fastapi官方文档",
"link": "https://fastapi.tiangolo.com/tutorial/",
"context": "FastAPI 是一个现代、快速的 Web 框架..."
},
{
"source": "教程博客",
"link": "https://example.com/fastapi-guide",
"context": "本教程介绍如何使用 FastAPI 构建 API..."
}
]

Each result includes the name of the source, the link and the extracted content.

Featured Function Operation

• Real-time Google search
  Free-Search connects to Google through a custom search engine for the latest results. No need to develop your own crawler, just enter keywords and use it.
• Content Crawling and Structured Output
  The tool automatically visits each search results page, extracts the main content and organizes it into an easy-to-read format. This is particularly useful for AI applications, where it can be fed directly to models for processing.
• Customized parameters
  You can adjust the max_results cap (a poem) max_content, which controls the amount and level of detail of the returned data. For example, to get more results, you can use:

curl "http://localhost:11235/search?query=python+学习&max_results=5"

• Using public instances
  If you don't want to install it locally, you can directly visit the official demo examples provided:https://freesearch.replit.app/. Simply type into your browser something like https://freesearch.replit.app/search?query=测试 address, you can try it out.

caveat

• When running locally, make sure the internet is open, as the tool requires access to Google and other web pages.
• If you encounter Playwright errors, check that Chromium is installed correctly.
• The project is open source and you can modify main.py or other documents, add new features such as support for other search engines.

With these steps, you can easily deploy and use Free-Search to get real-time search data.

application scenario

1. AI Agent Development
   Developers can use Free-Search to acquire real-time web data to train or enhance the AI agent's knowledge base.

Scene DescriptionFor example, if you're developing a chatbot that needs to answer questions about the latest tech news, Free-Search can provide the relevant web content for the bot to give accurate answers.

2. LLM Data Input
   Large language models require a lot of external data support, and Free-Search can be used as a data source.

Scene DescriptionYou are building a model to analyze papers, enter the keyword "machine learning" and Free-Search returns links and abstracts of the latest papers for the model to process.

3. Individual research tools
   Students or researchers can use it to quickly gather information from the web.

Scene Description: When writing a paper, type in "climate change impacts" to get real-time data from multiple sources and save time searching.

QA

1. What is the difference between Free-Search and Google Search?
   Free-Search is an API tool that automatically crawls Google search results and returns structured data, while Google Search only provides a web interface.
2. What is the difference between a public instance and a local deployment?
   Public instances can be used without installation, but may have access limitations; local deployments are fully controllable and can also be customized for functionality.
3. Does it support Chinese search?
   Yes, just type in Chinese keywords and the API will return relevant results, provided Google can find a match.