LongWriter: Wisdom Spectrum launched a powerful model for generating ultra-long text, supporting the generation of 10,000+ words in a single conversation.

General Introduction

LongWriter is a powerful long text generation tool developed by Tsinghua University Data Mining Research Group (THUDM). The tool is based on the advanced Long Contextual Language Model (LLM) and is capable of generating long texts of more than 10,000 words.The core models of LongWriter include LongWriter-glm4-9b and LongWriter-llama3.1-8b, which are based on the GLM-4-9B and Meta-Llama-3.1-8B training, respectively. . These models perform well in generating long text and are suitable for various application scenarios that require the generation of large text, such as novel creation, academic paper writing and detailed report generation, etc. LongWriter is not only open source, but also provides a detailed deployment and usage guide to facilitate users to get started quickly.

Function List

• Long Text Generation: Support for generating long texts of more than 10,000 words for a wide range of application scenarios.
• Multi-model support: Two models based on GLM-4-9B and Meta-Llama-3.1-8B are provided and users can choose according to their needs.
• Rapid deployment: Provides detailed deployment guides and sample code to support rapid deployment and use.
• open source: All code and models are open source and can be freely downloaded and modified by users.
• Automated Data Building: Provide AgentWrite tool to support automated ultra-long output data construction.

Using Help

Installation and deployment

1. environmental preparation: Ensure that Python 3.8 or above is installed and install the necessary dependency libraries, e.g. transformers>=4.43.0.
2. Download model: Download the required models from Hugging Face, such as LongWriter-glm4-9b or LongWriter-llama3.1-8b.
3. Installation of dependencies: Run the following command to install the required Python packages:

   pip install transformers torch

4. Loading Models: Use the following code to load the model and perform inference:

   from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
tokenizer = AutoTokenizer.from_pretrained("THUDM/LongWriter-glm4-9b", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("THUDM/LongWriter-glm4-9b", torch_dtype=torch.bfloat16, trust_remote_code=True, device_map = "auto")
model = model.eval()
query = "Write a 10000-word China travel guide"
response, history = model.chat(tokenizer, query, history=[], max_new_tokens=32768, temperature=0.5)
print(response)

Using the AgentWrite tool

1. Configuring the API Key: inplan.pycap (a poem)write.pyfile to configure your API key.
2. Operational Data Construction: Run the following commands in sequence to generate the final data:

   python plan.py
python write.py

Main function operation flow

1. Long Text Generation: Enter the subject of the text to be generated, and the model will generate a long text of more than 10,000 words based on the input.
2. Model Selection: Select the appropriate model (e.g. LongWriter-glm4-9b or LongWriter-llama3.1-8b) according to specific needs.
3. parameterization: Optimize generation by adjusting generation parameters (e.g., temperature, maximum number of new words, etc.).

Detailed operating instructions

• Text Generation: Enter a theme or prompt word in the input box, click the Generate button, and the model will automatically generate long text.
• model training: Download and train the model using a mix of the LongWriter-6k dataset and other SFT data, following the training code provided.
• Deployment Example: Referencevllm_inference.pyDocumented code to quickly deploy and experience ultra-long text generation.