Higress: Delivering an Efficient AI Gateway Solution to Simplify Microservice Management and Enhance Security

General Introduction

Higress is a cloud-native API gateway developed by Alibaba, built on Istio and Envoy, designed to provide efficient traffic scheduling, service governance and security solutions. It supports Wasm plug-in extensions for multiple programming languages, and is suitable for a variety of scenarios such as AI business, microservices, and Kubernetes Ingress, with high usability and ease of use.

Function List

• AI Gateway: Supports multi-model load balancing, AI token flow control, AI caching and other features.
• Kubernetes Ingress Gateway: Compatible with K8s Nginx Ingress annotations, supports smooth migration from Ingress API to Gateway API.
• Microservices Gateway: Support for a variety of registries such as Nacos, ZooKeeper, Consul, Eureka , etc. , integrated Dubbo, Nacos, Sentinel and other technology stack .
• Safety Gateway: Provides WAF capability and supports multiple authentication and authentication policies such as key-auth, hmac-auth, jwt-auth, etc.
• Plug-in extensions: Support for Wasm, Lua and other plug-in extension mechanism, plug-ins can be hot updated in real time.
• streaming: Supports fully streaming request/response Body for large bandwidth scenarios.
• consoles: Provides out-of-the-box UI console with support for Grafana and Prometheus observability integration.

Using Help

Installation process

1. Docker Installation::
   • Create a working catalog:mkdir higress; cd higress
   • Start Higress:docker run -d --rm --name higress-ai -v ${PWD}:/data -p 8001:8001 -p 8080:8080 -p 8443:8443 higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/all-in-one:latest
   • Port Description:
     • Port 8001: Higress UI console portal
     • Port 8080: gateway HTTP protocol portal
     • Port 8443: Gateway HTTPS protocol portal
2. Kubernetes Installation::
   • Deployment with Helm: Refer to the Quick Start documentation on the official website.

Function Operation Guide

1. AI Gateway Features::
   • Interfaces with all domestic and international LLM model vendors through a unified protocol.
   • Configure AI token flow control and AI caching to improve the stability and efficiency of AI services.
2. Kubernetes Ingress Gateway::
   • Compatible with K8s Nginx Ingress annotations, supports smooth migration from Ingress API to Gateway API.
   • Configure K8s Service service discovery to support registries such as Nacos/ZooKeeper.
3. Microservices Gateway::
   • Docking multiple registries and configuring routing rules for service discovery and load balancing.
   • Integrate Dubbo, Nacos, Sentinel and other technology stacks to improve the efficiency of microservice management.
4. Safety Gateway::
   • Configure WAF rules to protect applications from attacks.
   • Supports multiple authentication and authentication policies to ensure access security.
5. Plug-in extensions::
   • Write plug-ins using Wasm, Lua, etc. to extend the gateway functionality.
   • Real-time hot updating of plug-ins to ensure uninterrupted business.
6. streaming::
   • Support streaming processing request/response Body, suitable for large bandwidth scenarios.
   • Customize the processing of messages for streaming protocols such as SSE.
7. Console operation::
   • Access the Higress UI console for gateway configuration and management.
   • Integration with Grafana and Prometheus for observability monitoring.