DeepCoder-14B-Preview: an open source model that specializes in code generation

General Introduction

DeepCoder-14B-Preview is an open source code generation model developed by Agentica team and released on Hugging Face platform. It is based on DeepSeek-R1-Distilled-Qwen-14B, optimized by Distributed Reinforcement Learning (RL) techniques, and is capable of handling up to 64K token of very long contexts. With 14 billion parameters, this model achieved a Pass@1 accuracy of 60.6% in LiveCodeBench v5 testing (August 1, 2024-February 1, 2025), which is an 8% improvement over the base model, and a performance close to that of OpenAI's o3-mini. it's completely open source, including the model weights, training data, and scripts. The goal of DeepCoder is to help developers write complex code efficiently, especially for programming competitions and large-scale projects.

Function List

• Generate long code: supports contexts up to 64K tokens, and can generate and handle very long code.
• Highly accurate output: 60.6% Pass@1 in LiveCodeBench v5 for reliable code quality.
• Open source available: model files, datasets and training scripts are available for free download and customization.
• Supports a wide range of programming tasks: suitable for competition question answering, code debugging and project development.
• Long contextual reasoning: optimized by GRPO+ and DAPO techniques to ensure long code generation capabilities.

Using Help

DeepCoder-14B-Preview is a powerful tool that can help you generate code or handle complex programming tasks. Below is a detailed installation and usage guide.

Installation process

To use DeepCoder-14B-Preview locally, you need to prepare the environment and download the model. The steps are as follows:

1. Prepare hardware and software
   • A computer with a GPU is required, NVIDIA H100 or a graphics card with at least 24GB of RAM is recommended.
   • Installing Python 3.10: Run conda create -n deepcoder python=3.10 -yThen activate the environment conda activate deepcoderThe
   • Install dependent libraries: run pip install transformers torch huggingface_hub vllmThe
2. Download model
   • Visit the official page at https://huggingface.co/agentica-org/DeepCoder-14B-Preview.
   • In "Files and versions" find the model files (e.g. model-00001-of-00012.safetensors).
   • Use the command download:

     huggingface-cli download agentica-org/DeepCoder-14B-Preview --local-dir ./DeepCoder-14B
     

   • After downloading, the model files are saved locally ./DeepCoder-14B Folder.
3. Loading Models
   • Load the model in Python:

     from transformers import AutoModelForCausalLM, AutoTokenizer
     model_path = "./DeepCoder-14B"
     model = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype="auto", device_map="auto")
     tokenizer = AutoTokenizer.from_pretrained(model_path)
     

   • This will load the model onto the GPU ready for use.

How to use the main features

The core of DeepCoder is generating code and handling long contexts. Here's how it works:

Generate Code

1. Input Programming Requirements
   • Prepare a problem, such as "Write a Python function that finds the maximum value in an array".
   • Convert requirements to text:

     prompt = "写一个 Python 函数，找出数组中的最大值"
     

2. Generate Code
   • Use the following code to generate the answer:

     inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
     outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.6, top_p=0.95)
     result = tokenizer.decode(outputs[0], skip_special_tokens=True)
     print(result)
     

   • Possible output:

     def find_max(arr):
     if not arr:
     return None
     max_value = arr[0]
     for num in arr:
     if num > max_value:
     max_value = num
     return max_value
     

3. Optimized generation
   • If longer code is needed, adjust the max_new_tokens is 1024 or higher.
   • set up max_tokens=64000 Optimal long context performance is achieved.

Handling long contexts

1. Enter the long code
   • Let's say you have a code that is 32K tokens long and you want the model to renew it:

     long_code = "def process_data(data):\n    # 几千行代码...\n    return processed_data"
     prompt = long_code + "\n请为这个函数添加异常处理"
     

2. Generate a continuation
   • Enter and generate:

     inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
     outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.6, top_p=0.95)
     result = tokenizer.decode(outputs[0], skip_special_tokens=True)
     print(result)
     

   • The output may be:

     def process_data(data):
     try:
     # 几千行代码...
     return processed_data
     except Exception as e:
     print(f"错误: {e}")
     return None
     

3. Verification results
   • Check that the code meets the requirements. If it doesn't, describe the requirements more clearly.

Featured Functions Operation Procedure

DeepCoder's long code generation capability is its highlight, suitable for competitions and large projects.

Solving contests

1. Get Title
   • Find a topic from Codeforces, such as "Given an array, return all possible subsets".
   • Enter a description of the topic:

     prompt = "给定一个数组，返回所有可能的子集。例如，输入 [1,2,3]，输出 [[],[1],[2],[3],[1,2],[1,3],[2,3],[1,2,3]]"
     

2. Generate Code
   • Run the generate command:

     inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
     outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.6, top_p=0.95)
     result = tokenizer.decode(outputs[0], skip_special_tokens=True)
     print(result)
     

   • The output may be:

     def subsets(nums):
     result = [[]]
     for num in nums:
     result += [curr + [num] for curr in result]
     return result
     

3. Test results
   • To run the code in Python, type [1,2,3], check that the output is correct.

debugging code

1. Enter the question code
   • Suppose there is a piece of buggy code:

     buggy_code = "def sum_numbers(n):\n    total = 0\n    for i in range(n)\n        total += i\n    return total"
     prompt = buggy_code + "\n这段代码有语法错误，请修复"
     

2. Generate fixes
   • Enter and generate:

     inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
     outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.6, top_p=0.95)
     result = tokenizer.decode(outputs[0], skip_special_tokens=True)
     print(result)
     

   • The output may be:

     def sum_numbers(n):
     total = 0
     for i in range(n):
     total += i
     return total
     

3. Verification Fixes
   • Check the syntax for correctness and run the code to confirm the result.

Recommendations for use

• Do not add a system prompt, state the requirement directly in the user prompt.
• set up temperature=0.6 cap (a poem) top_p=0.95 for optimal results.
• commander-in-chief (military) max_tokens Set to 64000 to take advantage of long contexts.

application scenario

1. Programming Contest
   DeepCoder generates answers to contest questions quickly and is suitable for complex tasks in LiveCodeBench or Codeforces.
2. Large-scale project development
   It can generate long code modules to help developers complete large projects.
3. Education and learning
   Students can use it to generate sample code, learn algorithms or debug assignments.

QA

1. Is DeepCoder-14B-Preview free?
   Yes, it's licensed under the MIT license, completely open source, and free for anyone to use.
2. What hardware is required to run it?
   It is recommended to use a computer with a GPU and at least 24GB of video memory. If you use a CPU, it will be much slower.
3. What programming languages does it support?
   It specializes mainly in Python, but can also generate code in Java, C++, and other languages, depending on the clarity of the prompt.