glhf.chat: running almost (all) open source big models, free access to GPU resources and API services (beta period)

General Introduction

good luck have fun (glhf.chat) is a website that provides an open source big model chat service. The platform allows users to run almost any open source big model using vLLM and a custom auto-scaling GPU scheduler. Users can simply paste a link to the Hugging Face repository and interact using the chat interface or OpenAI-compatible APIs. The platform is being offered for free during the beta period and will be available in the future at a lower price than major cloud GPU providers.

Function List

• Support for a variety of open source big models, including Meta Llama, Qwen, Mixtral, etc.
• Provides access to up to eight Nvidia A100 80Gb GPUs
• Reasoning services for automated agent popularity models
• On-demand startup and shutdown of clusters to optimize resource usage
• Provides OpenAI-compatible APIs for easy integration

Using Help

Installation and use

1. Register & Login: Accessglhf.chatand register for an account, and log in when you're done.
2. Select Model: Select the desired macromodel on the platform's homepage. Supported models include Meta Llama, Qwen, Mixtral, and more.
3. Paste the link: Paste the Hugging Face repository link into the specified location and the platform will automatically load the model.
4. Using the Chat Interface: Interact with the model through the chat interface provided on the website, enter questions or commands, and the model will generate responses in real time.
5. API integration: Integrate the platform's features into your own applications using the OpenAI-compatible APIs; see the website's Help Center for API documentation.

Detailed function operation flow

1. Model Selection and Loading::
   • After logging in, you will be taken to the model selection page.
   • Browse the list of supported models and click on the desired model.
   • Paste the link to the Hugging Face repository in the pop-up dialog box and click the "Load Model" button.
   • Wait for the model to finish loading, the loading time depends on the model size and network conditions.
2. Chat Interface Usage::
   • Once the model is loaded, enter the chat screen.
   • Enter a question or instruction in the input box and click Send.
   • The model generates a reply based on the input and the reply is displayed in the chat window.
   • Multiple questions or commands can be entered in succession, and the model will process and respond to them one by one.
3. API Usage::
   • Visit the API documentation page for API keys and instructions.
   • Integrate the API in your application and follow the sample code provided in the documentation to call it.
   • Send a request through the API to get a model-generated response.
   • The API supports a variety of programming languages, see the documentation for specific sample code.

Resource management and optimization

• Automatic Expansion: The platform uses a customized GPU scheduler that automatically scales up and down GPU resources based on user demand to ensure efficient utilization.
• on-demand activation: For infrequently used models, the platform starts clusters on demand and automatically shuts them down when they are used up, saving resources.
• Free Test: During the beta test period, users will have free access to all the features offered by the platform, and a discounted pricing plan will be available at the end of the test.

The layman's guide to getting started with AI

Help you learn how to utilize AI tools at a low cost and from a zero base.AI, like office software, is an essential skill for everyone. Mastering AI will give you an edge in your job search and half the effort in your future work and studies.

View Details>