Skip to content

Commit

Permalink
Merge pull request #3 from OpenSPG/jifan_dev1
Browse files Browse the repository at this point in the history
refactor: adjust document and fix some direct link
  • Loading branch information
Aarebecca authored Oct 26, 2023
2 parents d606e2a + 1cac521 commit 322c510
Show file tree
Hide file tree
Showing 21 changed files with 681 additions and 594 deletions.
2 changes: 1 addition & 1 deletion .dumi/pages/download.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ function Download() {
const ableToDownload = !isExpired && allow && url;

useEffect(() => {
fetch('release.wasm')
fetch('/release.wasm')
.then((response) => response.arrayBuffer())
.then((bytes) => instantiate(bytes))
.then(({ validate }: { validate: ValidateFn }) => {
Expand Down
15 changes: 11 additions & 4 deletions .dumi/pages/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@
* title: 语义增强可编程知识图谱
*/

import { useEffect } from 'react';
import { history, usePrefersColor } from 'dumi';
import { Button, Descriptions, Tabs } from 'antd';
import { history, useLocale, usePrefersColor } from 'dumi';
import { useEffect } from 'react';
import styled from 'styled-components';
import AboutUs from '../components/AboutUs';
import CustomItem from '../components/CustomItem';
Expand Down Expand Up @@ -292,8 +292,15 @@ const MainPage = () => {
SPG_POINT_CONFIG,
},
} = useIntl();
const locale = useLocale();

const [prefersColor] = usePrefersColor();

const push = (link: string) => {
// @ts-ignore
history.push(`${locale.base}/${link}`);
};

useEffect(() => {
const main = document.querySelector('body .dumi-default-doc-layout > main');

Expand All @@ -313,7 +320,7 @@ const MainPage = () => {
}, []);

const handleDownload = () => {
history.push('/download');
push('download');
};

return (
Expand All @@ -333,7 +340,7 @@ const MainPage = () => {
<Button
size="large"
onClick={() => {
history.push('/quick-start');
push('quick-start');
}}
>
{PARAGRAPH.QuickStart}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/gh-pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,5 +21,5 @@ jobs:
- name: Deploy
uses: peaceiris/actions-gh-pages@v3
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
deploy_key: ${{ secrets.ACTIONS_DEPLOY_KEY }}
publish_dir: ./dist
12 changes: 7 additions & 5 deletions docs/example/enterprise-supply-chain/index.en-US.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
---
title: Enterprise Supply Chain
order: 1
nav:
second:
title: Enterprise Supply Chain
order: 2
---

## 1 Background
Expand All @@ -23,7 +26,6 @@ Concept knowledge maintains industry chain-related data, including hierarchical

<center style="font-size:14px;color:#c0c0c0;text-decoration:underline">Figure 2:Inductive Deduction of Industrial Chain Events </center>


Within the industrial chain, categories of product and company events are established. These categories are a
combination of indicators and trends. For example, an increase in price consists of the indicator "price" and the trend "upward". Causal knowledge sets the events of a company's profit decrease and cost increase due to a rise in product prices. When a specific event occurs, such as a significant increase in rubber prices, it is categorized under the event of a price increase. As per the causal knowledge, a price increase in a product leads to two event types: a decrease in company profits and an increase in company costs. Consequently, new events are generated:"Triangle** Tire Company cost increase" and "Triangle** Tire Company profit decrease".

Expand Down Expand Up @@ -103,9 +105,9 @@ knext reasoner query --dsl "
MATCH
(s:SupplyChain.Company)
RETURN
s.id, s.fundTrans1Month, s.fundTrans3Month,
s.fundTrans6Month, s.fundTrans1MonthIn, s.fundTrans3MonthIn,
s.fundTrans6MonthIn, s.cashflowDiff1Month, s.cashflowDiff3Month,
s.id, s.fundTrans1Month, s.fundTrans3Month,
s.fundTrans6Month, s.fundTrans1MonthIn, s.fundTrans3MonthIn,
s.fundTrans6MonthIn, s.cashflowDiff1Month, s.cashflowDiff3Month,
s.cashflowDiff6Month
"
```
Expand Down
15 changes: 8 additions & 7 deletions docs/example/enterprise-supply-chain/index.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
---
title: 案例演示
order: 1
title: 企业供应链
nav:
second:
title: 企业供应链
order: 2
---

## 1 背景
Expand All @@ -17,14 +20,12 @@ order: 1

<center style="font-size:14px;color:#C0C0C0;text-decoration:underline">图1:产业链企业图谱深度语义关联 </center>


概念知识维护着产业链相关数据,包括上下位层级、供应关系;实体实例仅有法人代表、转账信息,公司实例通过生产的产品属性和概念中的产品节点挂载,实现了公司实例之间的深度信息挖掘,例如供应商、同行业、同法人代表等关系。基于深度上下文信息,可提供更多的信用评估因子。

![产业链事件归纳演绎 ](https://mdn.alipayobjects.com/huamei_xgb3qj/afts/img/A*X2TES7hf9ycAAAAAAAAAAAAADtmcAQ/original)

<center style="font-size:14px;color:#c0c0c0;text-decoration:underline">图2:产业链事件归纳演绎 </center>


产业链中建立了产品和公司事件类别,该类别属于指标和趋势的一种组合,例如价格上涨,是由指标:价格,趋势:上涨两部分构成。
事理知识设定了产品价格上涨引起公司利润下降及公司成本上涨事件,当发生某个具体事件时,例如“橡胶价格大涨事件”,被归类在产品价格上涨,由于事理知识中定义产品价格上涨会引起公司利润下降/公司成本上涨两个事件类型,会产出新事件:“三角\*\*轮胎公司成本上涨事件”、“三角\*\*轮胎公司利润下跌”。

Expand Down Expand Up @@ -117,9 +118,9 @@ knext reasoner query --dsl "
MATCH
(s:SupplyChain.Company)
RETURN
s.id, s.name, s.fundTrans1Month, s.fundTrans3Month,
s.fundTrans6Month, s.fundTrans1MonthIn, s.fundTrans3MonthIn,
s.fundTrans6MonthIn, s.cashflowDiff1Month, s.cashflowDiff3Month,
s.id, s.name, s.fundTrans1Month, s.fundTrans3Month,
s.fundTrans6Month, s.fundTrans1MonthIn, s.fundTrans3MonthIn,
s.fundTrans6MonthIn, s.cashflowDiff1Month, s.cashflowDiff3Month,
s.cashflowDiff6Month
"
```
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
---
title: Medical
order: 2
nav:
second:
title: Medical
order: 3
---

## Building a medical knowledge graph from text
Expand Down Expand Up @@ -56,7 +59,7 @@ For the ChatGLM2 model, you need to prepare the structured samples. You can use

```shell
python builder/model/dataset/convert_util.py \
--entity_type Medical.Disease \
--entity_type Medical.Disease \
--task_type RE \
--src_path builder/model/dataset/RE/sample.json \
--tgt_path builder/model/dataset/RE/processed.json \
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
---
title: 医疗领域
order: 2
nav:
second:
title: 医疗领域
order: 3
---

## 从文本构建医疗图谱
Expand Down Expand Up @@ -53,7 +56,7 @@ knext operator publish DiseaseExtractor

```shell
python builder/model/dataset/convert_util.py \
--entity_type Medical.Disease \
--entity_type Medical.Disease \
--task_type RE \
--src_path builder/model/dataset/RE/sample.json \
--tgt_path builder/model/dataset/RE/processed.json \
Expand All @@ -63,13 +66,15 @@ python builder/model/dataset/convert_util.py \
**2、模型微调**

执行以下命令,使用 [p-tuning-v2](https://github.com/THUDM/ChatGLM2-6B/tree/main/ptuning#p-tuning-v2) 对模型参数进行微调

```shell
sh builder/model/train.sh
```

**第三步:部署大模型(ChatGLM2)推理服务**

将p-tuning的结果参数文件覆盖原模型参数文件,并执行以下命令部署模型服务

```bash
sh builder/model/deploy.sh
```
Expand Down
8 changes: 5 additions & 3 deletions docs/example/risk-mining/index.en-US.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
---
title: Risk Mining
order: 3
nav:
second:
title: Risk Mining
order: 1
---

## Risk Mining Knowledge Graph
Expand Down Expand Up @@ -66,7 +69,6 @@ knext reasoner query --dsl "${ql}"

![image.png](https://mdn.alipayobjects.com/huamei_xgb3qj/afts/img/A*uKcjRqTdy7cAAAAAAAAAAAAADtmcAQ/original)


MobilePhone: "standard attribute" compare to "text attribute".

Edit the `dsl_task.txt` file and enter the following query:
Expand Down Expand Up @@ -115,7 +117,7 @@ Define (s:Person)-[p:developed]->(o:App) {
(s)-[:hasDevice]->(d:Device)-[:install]->(o)
}
Constraint {
deviceNum = group(s,o).count(d)
deviceNum = group(s,o).count(d)
R1("设备超过5"): deviceNum > 5
}
}
Expand Down
8 changes: 5 additions & 3 deletions docs/example/risk-mining/index.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
---
title: 黑产挖掘
order: 3
nav:
second:
title: 黑产挖掘
order: 1
---

## 黑产挖掘图谱
Expand Down Expand Up @@ -66,7 +69,6 @@ knext reasoner query --dsl "${ql}"

![image.png](https://mdn.alipayobjects.com/huamei_xgb3qj/afts/img/A*uKcjRqTdy7cAAAAAAAAAAAAADtmcAQ/original)


电话号码:标准属性 vs 文本属性。

编辑dsl_task.txt,输入如下内容:
Expand Down Expand Up @@ -115,7 +117,7 @@ Define (s:Person)-[p:developed]->(o:App) {
(s)-[:hasDevice]->(d:Device)-[:install]->(o)
}
Constraint {
deviceNum = group(s,o).count(d)
deviceNum = group(s,o).count(d)
R1("设备超过5"): deviceNum > 5
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,64 +5,61 @@ order: 1

## OpenSPG

[中文版文档](../../quick-start/quick-start)
[中文版文档](../../quick-start)

<br/>

[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](./LICENSE)


OpenSPG is a knowledge graph engine developed by Ant Group in collaboration with OpenKG, based on the SPG (Semantic-enhanced Programmable Graph) framework, which is a summary of Ant Group's years of experience in constructing and applying diverse domain knowledge graphs in the financial sector.


![OpenSPG Architecture](https://mdn.alipayobjects.com/huamei_xgb3qj/afts/img/A*DmdvRKo5pyYAAAAAAAAAAAAADtmcAQ/original)

## Background

SPG (Semantic-enhanced Programmable Graph): semantic-enhanced programmable framework is a set of semantic representation framework based on attribute graph precipitated by Ant Knowledge Graph platform after years of supporting business in the financial field. It creatively integrates LPG structural and RDF semantic, which overcomes the problem that RDF/OWL semantic complexity cannot be industrially landed, and fully inherits the advantages of LPG structural simplicity and compatibility with big data system. The framework defines and represents knowledge semantics from three aspects. First, SPG explicitly defines the formal representation and programmable framework of "knowledge", so that it can be defined, programmed, understood and processed by machines. Secondly, SPG achieves compatibility and progressive advancement between knowledge levels, supporting the construction of knowledge graphs and the continuous iterative evolution of incomplete data states in industrial-level scenarios. Finally, SPG serves as an effective bridge between big data and AI technology systems, facilitating the efficient transformation of massive data into knowledge-based insights. By doing so, it enhances the value and application potential of the data. With the SPG framework, we can construct and manage graph data more efficiently, and at the same time, we can better support business requirements and application scenarios. Since SPG framework has good scalability and flexibility, new business scenarios can quickly build their domain models and solutions by extending the domain knowledge model and developing new
operators.

For a detailed introduction to SPG, please refer to **the [《SPG White Paper》](https://spg.openkg.cn/ "SPG White Paper") jointly released by Ant Group and OpenKG**.

For a detailed introduction to SPG, please refer to **the [《SPG White Paper》](download) jointly released by Ant Group and OpenKG**.

## Capabilities

OpenSPG is an open engine for knowledge graph designed and implemented on the basis of SPG framework, which provides explicit semantic representations, logical rule definitions, operator frameworks (construction, inference) and other capabilities for the domain knowledge graphs, and supports pluggable adaptation of basic engines and algorithmic services by various vendors to build customized solutions.

OpenSPG Core Capabilities:

* SPG-Schema semantic modeling
* Schema framework responsible for semantic enhancement of attribute graphs, such as subject models, evolutionary models, predicate models, etc.
* SPG-Builder knowledge construction
* Supports the construction of both structured and unstructured knowledge.
* Compatible and articulated with big data architecture, provides a knowledge construction operator framework to realize the conversion from data to knowledge.
* Abstracts the knowledge processing SDK framework, provides the ability of entity linking, concept standardization and entity normalization operators, combines Natural Language Processing (NLP) and deep learning algorithms, improves the uniqueness level of different instances within a single type. Furthermore, it supports the continuous iterative evolution of the domain knowledge graphs.
* SPG-Reasoner logical rule reasoning
* Abstracts KGDSL (Knowledge Graph Domain Specific Language) to provide programmable symbolic representation of logic rules.
* Supports downstream tasks, such as rule inference, neural/symbolic fusion learning, KG2Prompt linked LLM knowledge extraction/knowledge reasoning, represented in machine-understandable symbolic form.
* Define dependency and transfer between knowledge through predicate semantics and logic rules, and support modeling and analysis of complex business scenarios.
* Extensible SDK framework
* Business systems build their own characteristic business front-end by interfacing with open SDKs
* Extensible/adaptable customized graph storage/graph calculation engine
* Extensible/adaptable machine learning framework suitable for their own business characteristics
- SPG-Schema semantic modeling
- Schema framework responsible for semantic enhancement of attribute graphs, such as subject models, evolutionary models, predicate models, etc.
- SPG-Builder knowledge construction
- Supports the construction of both structured and unstructured knowledge.
- Compatible and articulated with big data architecture, provides a knowledge construction operator framework to realize the conversion from data to knowledge.
- Abstracts the knowledge processing SDK framework, provides the ability of entity linking, concept standardization and entity normalization operators, combines Natural Language Processing (NLP) and deep learning algorithms, improves the uniqueness level of different instances within a single type. Furthermore, it supports the continuous iterative evolution of the domain knowledge graphs.
- SPG-Reasoner logical rule reasoning
- Abstracts KGDSL (Knowledge Graph Domain Specific Language) to provide programmable symbolic representation of logic rules.
- Supports downstream tasks, such as rule inference, neural/symbolic fusion learning, KG2Prompt linked LLM knowledge extraction/knowledge reasoning, represented in machine-understandable symbolic form.
- Define dependency and transfer between knowledge through predicate semantics and logic rules, and support modeling and analysis of complex business scenarios.
- Extensible SDK framework
- Business systems build their own characteristic business front-end by interfacing with open SDKs
- Extensible/adaptable customized graph storage/graph calculation engine
- Extensible/adaptable machine learning framework suitable for their own business characteristics

## How to use

### Get Started

* [Install OpenSPG](./install.en-US.md)
* Quick start with examples:
* [Enterprise Supply Chain Knowledge Graph](../example/enterprise-supply-chain/index.en-US.md)
* [Risk Mining Knowledge Graph](../example/risk-mining/index.en-US.md)
* [Medical Knowledge Graph](../example/Medical/index.en-US.md)
- [Install OpenSPG](./install.en-US.md)
- Quick start with examples:
- [Enterprise Supply Chain Knowledge Graph](../example/enterprise-supply-chain/index.en-US.md)
- [Risk Mining Knowledge Graph](../example/risk-mining/index.en-US.md)
- [Medical Knowledge Graph](../example/medical/index.en-US.md)

### Advanced Tutorials

* [KNext command tool and SDK Tutorial](../tutorial/knext/index.en-US.md)
* [SPG-Schema tutorial](../tutorial/spgschema/index.en-US.md)
* [SPG-Reasoner Tutorial](../tutorial/spgreasoner/index.en-US.md)
* [Adapting a new graph database tutorial](../tutorial/spg2lpg/index.en-US.md)
- [KNext command tool and SDK Tutorial](../tutorial/knext/index.en-US.md)
- [SPG-Schema tutorial](../tutorial/spgschema/index.en-US.md)
- [SPG-Reasoner Tutorial](../tutorial/spgreasoner/index.en-US.md)
- [Adapting a new graph database tutorial](../tutorial/spg2lpg/index.en-US.md)

## How to contribute

* [Contribution Guidelines](./contribution.en-US.md)
- [Contribution Guidelines](./contribution.en-US.md)
Loading

0 comments on commit 322c510

Please sign in to comment.