Merge pull request #12 from VisActor/sync/main-1.2.2

[Auto Sync] Sync the code from branch main to branch develop after release 1.2.2

common/config/rush/version-policies.json

@@ -2,7 +2,7 @@
   {
     "definitionName": "lockStepVersion",
     "policyName": "vmindMin",
-    "version": "1.2.1",
+    "version": "1.2.2",
     "mainProject": "@visactor/vmind",
     "nextBump": "patch"
   }

packages/calculator/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@visactor/calculator",
-  "version": "0.0.0",
+  "version": "1.2.2",
   "description": "SQL-like query executor with DSL",
   "main": "lib",
   "module": "es",

packages/vmind/CHANGELOG.json

@@ -0,0 +1,11 @@
+{
+  "name": "@visactor/vmind",
+  "entries": [
+    {
+      "version": "1.2.2",
+      "tag": "@visactor/vmind_v1.2.2",
+      "date": "Tue, 30 Jan 2024 07:40:01 GMT",
+      "comments": {}
+    }
+  ]
+}

packages/vmind/CHANGELOG.md

@@ -0,0 +1,9 @@
+# Change Log - @visactor/vmind
+
+This log was last generated on Tue, 30 Jan 2024 07:40:01 GMT and should not be manually modified.
+
+## 1.2.2
+Tue, 30 Jan 2024 07:40:01 GMT
+
+_Initial release_
+

packages/vmind/README.md

@@ -138,7 +138,7 @@ We want to show "the changes in sales rankings of various car brands". Call the
 ```typescript
 const describe = 'show me the changes in sales rankings of various car brand';
 //Call the chart generation interface to get spec and chart animation duration
-const { spec, time } = await vmind.generateChart(userInput, fieldInfo, dataset);
+const { spec, time } = await vmind.generateChart(describe, fieldInfo, dataset);
 ```
 
 In this way, we get the VChart spec of the corresponding dynamic chart. We can render the chart based on this spec:
@@ -164,7 +164,7 @@ Users can specify different theme styles (currently only gpt chart generation su
 //describe can be in both Chinese and English
 //Specify to generate a tech-style chart
 const describe = 'show me the changes in sales rankings of various car brand,tech style';
-const { spec, time } = await vmind.generateChart(userInput, fieldInfo, dataset);
+const { spec, time } = await vmind.generateChart(describe, fieldInfo, dataset);
 ```
 
 You can also specify the chart type, field mapping, etc. supported by VMind. For example:
@@ -173,7 +173,7 @@ You can also specify the chart type, field mapping, etc. supported by VMind. For
 //Specify to generate a line chart, with car manufacturers as the x-axis
 const describe =
   'show me the changes in sales rankings of various car brands,tech style.Using a line chart, Manufacturer makes the x-axis';
-const { spec, time } = await(vmind.generateChart(csvData, describe));
+const { spec, time } = await(vmind.generateChart(csvData, describe, dataset));
 ```
 
 #### Customizing LLM Request Method
@@ -196,6 +196,35 @@ temperature?: number;//recommended to set to 0
 Specify your LLM service url in url (default is https://api.openai.com/v1/chat/completions)
 In subsequent calls, VMind will use the parameters in params to request the LLM service url.
 
+
+
+#### Data Aggregation
+📢 Note: The data aggregation function only supports GPT series models, more models will come soon.
+
+When using the chart library to draw bar charts, line charts, etc., if the data is not aggregated, it will affect the visualization effect. At the same time, because no filtering and sorting of fields has been done, some visualization intentions cannot be met, for example: show me the top 10 departments with the most cost, show me the sales of various products in the north, etc.
+
+VMind supports intelligent data aggregation since version 1.2.2. This function uses the data input by the user as a data table, uses a LLM to generate SQL queries according to the user's command, queries data from the data table, and uses GROUP BY and SQL aggregation methods to group, aggregate, sort, and filter data. Supported SQL statements include: SELECT, GROUP BY, WHERE, HAVING, ORDER BY, LIMIT. Supported aggregation methods are: MAX(), MIN(), SUM(), COUNT(), AVG(). Complex SQL operations such as subqueries, JOIN, and conditional statements are not supported.
+
+
+Use the `dataQuery` function of the VMind object to aggregate data. This method has three parameters:
+- userInput: user input. You can use the same input as generateChart
+- fieldInfo: Dataset field information. The same as generateChart, it can be obtained by parseCSVData, or built by the user.
+- dataset: Dataset. The same as generateChart, it can be obtained by parseCSVData, or built by the user.
+
+
+```typescript
+const { fieldInfo, dataset } = await vmind?.dataQuery(userInput, fieldInfo, dataset);
+```
+
+
+The fieldInfo and dataset returned by this method are the field information and dataset after data aggregation, which can be used for chart generation.
+By default, the `generateChart` function will perform a data aggregation using the same user input before generating the chart. You can disable data aggregation by passing in the fourth parameter:
+```typescript
+const userInput = 'show me the changes in sales rankings of various car brand';
+const { spec, time } = await vmind.generateChart(userInput, fieldInfo, dataset, false); //pass false as the forth parameter to disable data aggregation before generating a chart.
+```
+
+
 #### Dialog-based editing
 
 Under development, stay tuned
@@ -213,3 +242,4 @@ Under development, stay tuned
 #### Pie chart
 
 ![Alt text](https://lf9-dp-fe-cms-tos.byteorg.com/obj/bit-cloud/VChart-Video-3.gif)
+

packages/vmind/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@visactor/vmind",
-  "version": "1.2.1",
+  "version": "1.2.2",
   "main": "cjs/index.js",
   "module": "esm/index.js",
   "types": "esm/index.d.ts",
@@ -11,6 +11,27 @@
     "esm",
     "build"
   ],
+  "keywords": [
+    "charts",
+    "visualization",
+    "VMind",
+    "LLM",
+    "storytelling",
+    "VisActor",
+    "graphics",
+    "AIGC"
+  ],
+  "homepage": "https://www.visactor.io/vmind",
+  "bugs": "https://github.com/VisActor/VMind/issues",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/VisActor/VMind.git",
+    "directory": "packages/vmind"
+  },
+  "author": {
+    "name": "VisActor",
+    "url": "https://www.visactor.io/"
+  },
   "scripts": {
     "start": "vite ./__tests__/browser",
     "build": "bundle --clean",

packages/vmind/readme-zh.md

@@ -138,7 +138,7 @@ const { fieldInfo, dataset } = await vmind.parseCSVDataWithLLM(csv, userInput);
 ```typescript
 const describe = 'show me the changes in sales rankings of various car brand';
 //调用图表生成接口，获得 spec 和图表动画时长
-const { spec, time } = await vmind.generateChart(userInput, fieldInfo, dataset);
+const { spec, time } = await vmind.generateChart(describe, fieldInfo, dataset);
 ```
 
 这样我们就得到了对应动态图表的 VChart spec。我们可以基于该 spec 渲染图表：
@@ -164,7 +164,7 @@ vchart.renderAsync();
 //describe使用中英文均可
 //指定生成科技感风格的图表
 const describe = 'show me the changes in sales rankings of various car brand,tech style';
-const { spec, time } = await vmind.generateChart(userInput, fieldInfo, dataset);
+const { spec, time } = await vmind.generateChart(describe, fieldInfo, dataset);
 ```
 
 也可以指定 VMind 支持的图表类型，字段映射等等。比如：
@@ -173,7 +173,7 @@ const { spec, time } = await vmind.generateChart(userInput, fieldInfo, dataset);
 //指定生成折线图，汽车厂商做 x 轴
 const describe =
   'show me the changes in sales rankings of various car brands,tech style.Using a line chart, Manufacturer makes the x-axis';
-const { spec, time } = await(vmind.generateChart(csvData, describe));
+const { spec, time } = await(vmind.generateChart(csvData, describe, dataset));
 ```
 
 #### 自定义大模型调用方式
@@ -196,6 +196,33 @@ const vmind = new VMind(openAIKey:string, params:{
 在 url 中指定您的大模型服务 url（默认为https://api.openai.com/v1/chat/completions）
 在随后的调用中，VMind 会使用 params 中的参数请求大模型服务 url
 
+#### 数据聚合
+📢 Note: 数据聚合功能只支持GPT系列模型，更多模型正在接入中。
+
+在使用图表库绘制柱状图、折线图等图表时，若传入的数据不是聚合后的数据，会影响可视化效果。同时由于没有对字段进行筛选和排序，某些图表展示意图无法满足，例如：帮我展示使用量最多的10个部门，帮我展示北方各商品的销售额等。
+
+VMind 1.2.2版本开始支持智能数据聚合功能。该功能会将用户传入的数据作为一张数据表，使用大模型根据用户的指令生成SQL查询，从数据表中查询数据，并通过GROUP BY和SQL聚合函数对数据进行分组聚合、排序、筛选。目前支持的SQL语句：SELECT, GROUP BY, WHERE, HAVING, ORDER BY, LIMIT。目前支持的聚合函数：MAX(), MIN(), SUM(), COUNT(), AVG()。不支持子查询、JOIN、条件语句等复杂的SQL操作。
+
+
+使用VMind对象的`dataQuery`函数对数据进行聚合。该方法有3个参数：
+- userInput：用户输入。使用与generateChart相同的输入即可
+- fieldInfo：数据集字段信息。与generateChart相同，可使用parseCSVData获得，或者由用户自己构建。
+- dataset：数据集。与generateChart相同，可使用parseCSVData获得，或者由用户自己构建。
+
+
+```typescript
+const { fieldInfo, dataset } = await vmind?.dataQuery(userInput, fieldInfo, dataset);
+```
+
+
+该方法返回的fieldInfo和dataset是数据聚合后的字段信息和数据集，可用于后续的图表生成。
+`generateChart`函数默认会在生成图表之前，使用相同的用户输入进行一次数据聚合。可通过传入第四个参数来禁用数据聚合：
+```typescript
+const userInput = 'show me the changes in sales rankings of various car brand';
+const { spec, time } = await vmind.generateChart(userInput, fieldInfo, dataset, false); //pass false as the forth parameter to disable data aggregation before generating a chart.
+```
+
+
 #### 对话式编辑
 
 开发中，敬请期待

packages/vmind/src/core/VMind.ts

@@ -89,6 +89,7 @@ class VMind {
       return queryDatasetWithGPT(userPrompt, fieldInfo, dataset, this._options);
     }
     if ([Model.SKYLARK, Model.SKYLARK2].includes(this._model)) {
+      console.error('Please user GPT model');
       return { fieldInfo: [], dataset };
     }
     console.error('unsupported model in data query!');

rush.json

@@ -60,6 +60,7 @@
       "packageName": "@visactor/vmind",
       "projectFolder": "packages/vmind",
       "shouldPublish": true,
+      "versionPolicyName": "vmindMin",
       "tags": [
         "package"
       ]
@@ -68,6 +69,7 @@
       "packageName": "@visactor/calculator",
       "projectFolder": "packages/calculator",
       "shouldPublish": true,
+      "versionPolicyName": "vmindMin",
       "tags": [
         "package"
       ]