General Introduction
Transformers.js is a JavaScript library developed by Hugging Face to enable users to run state-of-the-art machine learning models directly in the browser without server support. The library is functionally equivalent to Hugging Face's transformers library for Python, and supports a wide range of pre-trained models covering tasks such as natural language processing, computer vision, and speech recognition. transformers.js uses the ONNX Runtime to run models, supports execution on both CPUs and WebGPUs, and provides efficient It provides efficient model transformation and quantization tools for users to convert PyTorch, TensorFlow or JAX models to ONNX format and run them in the browser.
Function List
- natural language processing (NLP): text categorization, named entity recognition, question-and-answer systems, language modeling, summary generation, translation, multiple choice, and text generation.
- computer vision: image classification, target detection, image segmentation and depth estimation.
- speech recognition: automatic speech recognition, audio classification and text-to-speech.
- multimodal task: embedding, zero-sample audio classification, zero-sample image classification, and zero-sample target detection.
- Model transformation and quantification: Support for converting PyTorch, TensorFlow, or JAX models to ONNX format and quantizing them to optimize performance.
- WebGPU Support: Increase computational efficiency by running models with WebGPU in supported browsers.
Using Help
mounting
Transformers.js can be installed via NPM or used in the browser via CDN or static hosting. Here are the installation steps:
NPM Installation
npm i @huggingface/transformers
CDN Usage
Add the following script tags to the HTML file:
import { pipeline } from 'https://cdn.jsdelivr.net/npm/@huggingface/transformers@3.1.0';
</script
usage example
Transformers.js provides a pipeline API similar to the Python version of the transformers library, which makes using the model very easy. Here are some examples of common tasks:
emotional analysis
import { pipeline } from '@huggingface/transformers';
// Assign a pipeline for sentiment analysis
const pipe = await pipeline('sentiment-analysis');
const out = await pipe('I love transformers!'); // Assign a sentiment analysis pipeline.
// Output: [{'label': 'POSITIVE', 'score': 0.999817686}]]
Use of different models
You can use a different model by specifying a model ID or path:
const pipe = await pipeline('sentiment-analysis', 'Xenova/bert-base-multilingual-uncased-sentiment');
Running models on WebGPUs
const pipe = await pipeline('sentiment-analysis', 'Xenova/distilbert-base-uncased-finetuned-sst-2-english', { device: 'webgpu' });
Model transformation and quantification
Transformers.js supports converting PyTorch, TensorFlow, or JAX models to ONNX format and quantizing them to optimize performance. Below are examples of conversion and quantization:
python -m scripts.convert --quantize --model_id bert-base-uncased
The converted file will be saved in the . /models/ directory, including model.onnx cap (a poem) model_quantized.onnx Documentation.
sample application
Transformers.js provides several sample applications to help users get started quickly:
- Whisper Web: Speech Recognition
- Doodle Dash: Real-time sketch recognition game
- Code Playground: In-browser code completion
- Semantic Image Search: Text Search Images
- Text to Speech: Text-to-Speech
