Ai2 OLMoE: An Open Source iOS AI App Based on OLMoE Models Running Offline

General Introduction

Ai2 OLMoE is an open source iOS app developed by the Allen Institute for AI (Ai2, Allen Institute for Artificial Intelligence) to provide AI models that run entirely on the device. The app utilizes Ai2's open-source OLMoE models, which are able to run offline without a cloud connection, ensuring user privacy and data security.Ai2 OLMoE is not only suitable for researchers exploring how to improve local AI models, but also provides developers with tools to rapidly prototype new AI experiences.

Online experience: https://playground.allenai.org/?model=olmoe-0125

Function List

• Completely open source: The model and application code of Ai2 OLMoE are completely open source, which facilitates researchers and developers to experiment and improve.
• efficient operation: High-performance AI computation on the device using an efficient Mixture-of-Experts model.
• Privacy: All operations are done on the device, ensuring that user data does not leave the device.
• multifunctional integration: Support for integrating OLMoE models into other iOS apps to extend their functionality.
• Community Support: Join Ai2's Discord server to communicate with other researchers and developers and share projects and experiences.

Using Help

Installation process

1. Open the App Store and search for "Ai2 OLMoE".
2. Click the "Get" button to download and install the app.
3. Once installation is complete, open the app and follow the prompts for initial setup.

Guidelines for use

Experience the AI model

1. Open the Ai2 OLMoE application.
2. Select the "Experience Model" option to start interacting with the model.
3. Enter or select a task and observe how the model behaves.

Researching and improving AI models

1. Download and install the Ai2 OLMoE application.
2. Open the application and select "Research Mode".
3. Model training and improvement using the tools and datasets provided within the application.
4. Upload the improved model to the application and test its performance.

Integration into other iOS apps

1. Access to Ai2 OLMoE's GitHub open source repository (link provided within the app).
2. Download the code base and configure it according to the documentation instructions.
3. Integrate OLMoE models into your iOS app to extend its functionality.

Detailed Operation Procedure

1. Download and Installation: Search for and download the Ai2 OLMoE app from the App Store and follow the prompts to complete the installation.
2. initial setup: Open the app for initial setup, including choosing a language and privacy settings.
3. experiential modelSelect "Experience Mode", enter or select a task, and observe how the model behaves.
4. research model: Select "Research Mode" and use the provided tools and datasets for model training and refinement.
5. Integrated Mode: Visit the GitHub code repository to download and configure code to integrate the model into other iOS apps.

More about Ai2 OLMoE

The Allen Institute for Artificial Intelligence's (AI2) OLMoE model is now officially released as a cutting-edge, open-source, device-side model.

The Allen Institute for Artificial Intelligence (AI) has long held a vision of building superior, fully open source language models. Now they have taken a significant step towards that goal - the AIIM has redefined the boundaries of what is considered fully open source with the release of a fully open source iOS app. The app is designed to allow users to experience their models securely and privately on their own devices, demonstrating the AIII's profound expansion of the open source philosophy.

This fully open source app will help researchers explore ways to improve device-side models and empower developers to prototype innovative AI experiences. Its open-source nature not only provides a valuable platform for researchers, but also inspires unlimited creativity in the developer community to drive device-side AI technology advancements and applications.

Users can download the app now from the Apple App Store or choose to build it themselves by obtaining the source code from the code repository. It's important to note that due to hardware considerations, the first version of the OLMoE app will only work on an iPhone 15 Pro or newer, or any iPad with an M-series chip.

Expanding the depth and breadth of open ecosystems

The Allen Institute for Artificial Intelligence (AI) has adhered to the principle of complete openness since the inception of the OLMo series of models. They have not only disclosed the final model weights, but also open-sourced the software, data and related technical details involved in the model construction process. Allen AI Institute believes that true openness is not only limited to the model itself, but should also be extended to the user experience level, to build an open AI ecosystem that is truly accessible. The following are some examples of open AI technologies vLLM and SGLang are excellent open-source projects that have given a strong boost to developers deploying Large Language Models (LLMs) on разнообразных cloud servers; and Ollama and the emergence of tools such as LM Studio, on the other hand, allow users to experience open weighting models directly on their personal computers.

In recent years, the capabilities of small models have improved by leaps and bounds. In particular, by the end of 2024, the 7B parametric model has easily surpassed the performance of the previous year's state-of-the-art model. At the same time, the performance of the mobile processing units has also been increasing, signaling that the Device-side AI is poised for broader adoptionThe

To better embrace this trend, the Allen Institute for Artificial Intelligence has launched OLMoE, a Fully open source toolkit, designed to provide researchers and developers with Device-side AI experimentation platformOLMoE has a wide range of application scenarios, including:

• Real World Mission Exploration: Hands-on experience with state-of-the-art device-side models in real-world application scenarios and discover what they can do for real-world tasks.
• A Study on Improvement of Efficient Local AI Models: Delve into how to further improve the efficiency of local AI models and provide ideas for device-side AI optimization.
• Local Model Testing: Conveniently test and validate user собственные models in local environments using the open source code base provided by OLMoE.
• iOS Application Integration: Seamlessly integrate OLMoE into other iOS apps to expand the boundaries of AI technology.

The most significant advantages of device-side models such as OLMoE over cloud-based models are Complete privacyThe user's prompts and model responses are always processed locally on the device. User prompts and model responses are always processed locally on the device and do not need to be uploaded to a cloud server, thus maximizing the security of user data. In addition, because No Internet connection requiredOLMoE is able to operate stably and reliably in any location and any network environment, providing users with AI services anytime, anywhere.

The Leap from Models to Applications

To successfully build the Ai2 OLMoE application, the Allen Institute for Artificial Intelligence has skillfully integrated their state-of-the-art, fully open technology solution. The cornerstone of the application is the OLMoE model, which is the most efficient and fully open-source language model ever developed by the Allen Institute for Artificial Intelligence. Building on this foundation, the Allen Institute for Artificial Intelligence has released a new version of the model, allenai/OLMoE-1B-7B-0125-Instruct, which cleverly incorporates the Dolmino hybrid training strategy introduced in OLMo 2 (used in the middle of model training) and the Tülu 3 model training. and a post-training optimization scheme for Tülu 3 models. With these innovations, the new version of OLMoE achieves an average performance increase of 35% on the Allen Institute for Artificial Intelligence's evaluation suite, while still maintaining a similar level of efficiency as the original version.

*AlpacaEval refers to Alpaca Eval 2 Length Controlled.

To ensure that OLMoE runs efficiently on the device side, the Allen Institute of Artificial Intelligence used Q4_K_M quantization to reduce the size of the model. This quantization has had a minimal impact on model performance, e.g., the IFEval score has dropped only slightly from 66.4 to 63.6. If you are interested in experiencing the performance of the new OLMoE model before quantization, go to the Ai2 Playground Online testing. In addition, the quantitative model in GGUF format has been released on HuggingFace, and users can choose the base version or instruct version to download according to their needs.

The Allen Institute for Artificial Intelligence worked closely with GenUI in the development of the Ai2 OLMoE app. The application is built on top of several outstanding open source projects, such as the Swift bindings to Llama.cpp. The Allen Institute of Artificial Intelligence deeply optimized the entire technology stack, resulting in an average processing speed of 41 Tokens/s on an iPhone 16 Pro device, which is a very impressive performance.

It is worth noting that the code for the Ai2 OLMoE application has been Completely open source (https://github.com/allenai/OLMoE.swift), which provides valuable references and lessons for a wide range of AI researchers and developers. Developers can use the Ai2 OLMoE application codebase as a scaffold for evaluating more efficient device-side AI models, or seamlessly integrate Ai2 OLMoE's model implementations into other applications to empower them with powerful AI capabilities.

The Allen Institute for Artificial Intelligence believes that the release of the Ai2 OLMoE app is a critical step towards the future of device-based intelligence. As the processing power and performance of mobile devices continue to increase, AIIR expects the OLMoE app to help researchers and developers stay on the cutting edge of technology and explore the infinite possibilities of device-side AI technology.

Experience the OLMoE app on your iPhone today!