ClickClickClickClick: Enable Any LLM to Automate Android and PC Operations

General Introduction

ClickClickClick is a framework developed by BandarLabs that aims to automate Android and PC operations by using any local or remote Large Language Model (LLM). The project is currently in a highly experimental phase and supports multiple models such as Ollama, Gemini, and GPT 4o. Users can use ClickClickClick to perform a variety of tasks such as sending emails, browsing maps, launching games, and more by configuring different models and API keys. The framework requires ADB to be installed on the local machine and USB debugging mode to be enabled on the Android phone.

Function List

• Support for multiple LLM models (Ollama, Gemini, GPT 4o)
• Automated Android and PC operations
• Configuring and Using API Keys
• Perform a variety of tasks (send emails, browse maps, launch games, etc.)
• Highly configurable task execution platform

Using Help

Installation process

1. Clone the repository and go to the project directory:

   git clone https://github.com/BandarLabs/clickclickclick
cd clickclickclick

2. Create a virtual environment and activate it:

   python3 -m venv venv
source venv/bin/activate # Windows users use `venv\Scripts\activate`.

3. Install the dependencies:

   pip install -r requirements.txt

Usage

1. Configure model settings: In theconfig/models.yamlfile to set the model-related configuration and export the corresponding API key in the environment.
2. To use as a CLI tool: Install the tool and make sure you have theOPENAI_API_KEYcap (a poem)GEMINI_API_KEY::

   pip install 
click3 run open uber app

3. Used as a script: By default, planner is openai and finder is gemini. can be found in theconfig/models.yamlto change the default configuration.
4. Run the task: Useruncommand to perform a task. The basic usage is as follows:

   python main.py run ""

Options:

• --platform: Specify the platform, the default is Android. Example: bash
python main.py run "example task" --platform=osx

• --planner-model: Specifies the planner model. the default is openai. example: bash
python main.py run "example task" --planner-model=gemini

• --finder-model: Specifies the finder model, which defaults to gemini. example: bash
python main.py run "example task" --finder-model=ollama


sample command (computing)

The full sample command is shown below:

python main.py run "Open Google news" --platform=android --planner-model=openai --finder-model=gemini