AI knowledge Page 3

I found an interesting paper "Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs", which analyzes the frequent switching of thinking paths and the lack of focus in o1-like reasoning models, or "underthinking" for short. The topic is to analyze the o1-like reasoning model's frequent switching of thinking paths and lack of focused thinking, which is referred to as "underthinking", and at the same time to give a solution to alleviate ...

Introduction In the vast starry sky of AI technology, deep learning models drive the innovative development of many fields with their excellent performance. However, the continuous expansion of model scale, like a double-edged sword, brings about a dramatic increase in arithmetic demand and storage pressure while improving performance. Especially in resource-constrained applications ...

Abstract Although Large Language Models (LLMs) perform well, they are prone to hallucinating and generating factually inaccurate information. This challenge has motivated efforts in attribute text generation, prompting LLMs to generate content that contains supporting evidence. In this paper, we present a new approach called Think&Cite ...

Introduction The purpose of this document is to help readers quickly understand and grasp the core concepts and applications of Prompt Engineering through a series of prompt examples (in part). These examples are all derived from an academic paper on a systematic review of prompt engineering techniques ("The Prompt Report: A Systematic Survey of Pr...

Titans: Learning to Memorize at Test Time Original text: https://arxiv.org/pdf/2501.00663v1 Titans architecture Unofficial implementation: https://github.com/lucidrains/titans- pytorch I. Research Background and Motivation: Transformer of ...

For any application that requires Retrieval Augmented Generation (RAG) system, the massive PDF documents into machine-readable blocks of text (also known as "PDF chunking") are a big headache. There are both open-source programs and commercialized products on the market, but honestly, there is no program that can really...

DeepSeek R1 official jailbreaks are great experimental environments for triggering basically all types of censorship mechanisms, and you can learn a lot of defense techniques, so this is a big model censorship mechanism learning article that will take you through examples of big model jailbreaks over the years. Large model censorship mechanisms are usually used...

Original: https://cdn.openai.com/o3-mini-system-card.pdf 1 Introduction The OpenAI o model family is trained using large-scale reinforcement learning to reason using chains of thought. These advanced reasoning capabilities provide new ways to improve the security and robustness of our models. In particular, ...

Quick Reads A comprehensive and in-depth look at the past and present of the Scaling Law of Large Language Models (LLMs) and the future direction of AI research. With clear logic and rich examples, author Cameron R. Wolfe takes the reader from the basic concepts to the...

Abstract Large-scale language models (LLMs), such as OpenAI's GPT-4, Google's PaLM, and Meta's LLaMA, have dramatically transformed Artificial Intelligence (AI) by enabling human-like text generation and natural language understanding. However, their reliance on static training data limits their ability to respond to dynamic, real-time queries...

Artificial Intelligence (AI) is a rapidly growing field. Language models have evolved to enable AI Agents to perform complex tasks and make complex decisions. However, as the skills of these Agents continue to grow, the infrastructure to support them struggles to keep up. LangGraph, a revolutionary library designed to revolutionize...

Introduction Like many others, over the past few days my news tweets have been filled with news, praise, complaints, and speculation about the Chinese-made DeepSeek-R1 large language model, which was released last week. The model itself is being brought up against some of the best inference models from OpenAI, Meta, and other...

  Summary of Key Contributions of CORAG CORAG (Cost-Constrained Retrieval Optimization for Retrieval-Augmented Generation) is an innovative retrieval-augmented generation (RAG) system designed to address key challenges in existing RAG approaches. The following CORAG ...

Knowledge distillation is a machine learning technique that aims to transfer learning from a large pre-trained model (i.e., a "teacher model") to a smaller "student model". Distillation techniques can help us develop lighter weight generative models for intelligent conversations, content creation, and other areas. Recently Distil...

Recently, many people engaged in large model training and inference have been discussing the relationship between the number of model parameters and model size. For example, the famous alpaca series LLaMA large model contains four versions with different parameter sizes, LLaMA-7B, LLaMA-13B, LLaMA-33B and LLaMA-65B. Here "...

Original article: https://arxiv.org/pdf/2412.15479 INTERPRETATION: This article itself is not very innovative and has little application. However, it reminds me of three highly informative articles I read a long, long time ago. Reading this article in conjunction with the three previous articles will hopefully bring you more inspiration. Recommended reading: the...

In the field of artificial intelligence and machine learning, especially when building applications such as RAG (Retrieval Augmented Generation) systems and semantic search, efficiently processing and retrieving massive unstructured data becomes crucial. Vector databases have emerged as a core technology to address this challenge. They are not only for storing high-dimensional ...

Xiaohongshu, a hot social e-commerce platform in China and even in Asia, has long gone beyond a simple shopping app to become a weathervane for young people's lifestyles and a new position for brand marketing. For overseas brands and individuals wishing to enter the Chinese market or reach young consumers, mastering Xiaohongshu...

Unexpectedly, AI has set off a half-changing sky in the programming field. From v0, bolt.new to various Agant programming tools Cursor, Windsurf, AI Coding has a huge potential of idea MVP. From the traditional AI-assisted coding, to today's direct project generation behind, in the end is...