OpenSPG: Open Source Knowledge Graph Engine

General Introduction

OpenSPG is an open source knowledge graph engine developed by Ant Group in collaboration with OpenKG, based on the SPG (Semantic Augmented Programmable Graph) framework. The engine is designed to support the construction and management of domain knowledge graphs by providing features such as explicit semantic representation, logical rule definitions and operational frameworks.OpenSPG combines the simplicity of the LPG structure with the complexity of RDF semantics, and is suitable for a wide range of business scenarios such as finance.

Function List

• Knowledge modeling: support for knowledge modeling of domain model constraints
• Semantic Representation: Provide explicit semantic representation and logic rule definitions
• Operational frameworks: support for construction, reasoning and other operational frameworks
• Plug-in Adaptation: Supports plug-in adaptation of the underlying engine and algorithmic services
• Data visualization: provides an intuitive interface for data exploration and analysis

Using Help

Installation process

Server Installation

The server side is deployed based on Docker Compose and contains 4 main images:

• openspg-server: Provide schema services
• openspg-mysql: Storing schema data
• tugraph: Storage of mapping data
• elasticsearch: Indexed mapping data

1. downloading docker-compose.yml file, and execute the following commands in the current directory, wait for the completion of the command execution, that is, to complete the server startup:

   docker-compose -f docker-compose.yml up -d
   

2. (coll.) fail (a student) openspg-server After local startup, you can open the http://127.0.0.1:8887 port to see the front-end visualization page. If openspg-server Deployed remotely, the corresponding IP address needs to be modified accordingly.

Client Installation

The client also provides a Docker image, and executing the following command directly will pull that image:

docker pull --platform linux/x86_64 spg-registry.cn-hangzhou.cr.aliyuncs.com/spg/openspg-python:latest

After downloading the image, you can clone the OpenSPG source code:

git clone --depth=1 https://github.com/OpenSPG/openspg.git

Once the source clone is complete, you can experience the cases that come with the source:

# Start the container and replace ${project_dir} in it with the source directory
docker run --rm --net=host -v ${project_dir}:/code \
  -it spg-registry.cn-hangzhou.cr.aliyuncs.com/spg/openspg-python:latest \
  "/bin/bash"

After the # container starts, go to the /code directory, the openspg project source directory
cd /code

# Follow-up can install case tutorials, such as going into the diskmining directory
cd python/knext/knext/examples/riskmining

# Refer to the case tutorials and execute the appropriate knext commands, such as
knext project create --prj_path .
knext schema commit

knext builder execute ...
knext reasoner execute ...

Alternatively, when going to write a mapping project locally based on an IDE, you can install knext by executing the following command:

pip install openspg-knext

Detailed Operation Procedure

server-side startup

1. Clone the OpenSPG source code and open it in the IDE:

   git clone git@github.com:OpenSPG/openspg.git
   

   The following commands are all assumed to be executed in the root directory of the openspg code base.

2. Execute the mvn compile command:

   mvn clean install -Dmaven.test.skip=true -Dspotless.check.skip -Dspotless.apply.skip
   

3. Start MySQL, ElasticSearch, TuGraph, and other containers locally:

   sh dev/test/docker-compose.sh
   

4. The startup server entry is located:

   com.antgroup.openspg.server.arks.sofaboot.Application
   

Client Startup

1. Prepare a Python virtual environment, version 3.8+.
2. Copy the mvn-compiled reasoner and builder packages to the knext directory:

   cp dev/release/python/lib/builder* python/knext/knext/builder/lib
   cp dev/release/python/lib/reasoner* python/knext/knext/reasoner/lib
   

3. Local installation of nn4k:

   cd python/nn4k
   python setup.py develop
   

4. Local installation of knext:

   cd python/knext
   python setup.py develop
   

5. Once you're done, you can execute the knext command in that Python virtual environment.

Core Competency Model

OpenSPG's core competency model includes:

• SPG-Schema semantic modeling: Responsible for the design of Schema frameworks for semantic enhancement of attribute graphs, e.g., subject models, evolutionary models, predicate models, etc.
• SPG-Builder Knowledge Building: Supports both structured and unstructured knowledge import, is compatible and articulated with big data architectures, and provides a knowledge construction operator framework to realize the conversion from data to knowledge. Abstracts the knowledge processing SDK framework, provides the ability of entity chaining finger, concept labeling and entity normalization operators, combines natural language processing (NLP) and deep learning algorithms, improves the uniqueness level of different instances in a single type, and supports the continuous and iterative evolution of the domain mapping.
• SPG-Reasoner Logic Rule Reasoning: Abstracts KGDSL (Knowledge Graph Domain Specific Language) to provide programmable symbolic representation of logic rules. Support downstream rule inference, neural/symbolic fusion learning, KG2Prompt linked LLM knowledge extraction/knowledge inference, etc. with machine-understandable symbolic representations. Define dependencies and transfer of knowledge through predicate semantics and logic rules, and support modeling and analysis of complex business scenarios.
• Programmable Framework KNext: KNext, as a graph programmable framework, provides a set of scalable, process-oriented, user-friendly componentized capabilities; abstracts the core capabilities of graphs and precipitates them into componentized, frameworked, and engine-built capabilities; achieves the isolation of the engine from the business logic and the domain model, which facilitates the business to quickly define the graph solution; and builds knowledge-driven and controllable AI technology based on the OpenSPG engine stack, linking deep learning capabilities such as LLM and GraphLearning.
• Cloud Adaptation Layer CloudextBusiness systems can dock the open engine via SDK to build their own characteristic business front-end; expandable/adaptable customized graph storage/graph computing engine; expandable/adaptable machine learning framework suitable for their business characteristics.

The layman's guide to getting started with AI

Help you learn how to utilize AI tools at a low cost and from a zero base.AI, like office software, is an essential skill for everyone. Mastering AI will give you an edge in your job search and half the effort in your future work and studies.

View Details>