Skip to content

Latest commit

 

History

History
311 lines (236 loc) · 15.9 KB

README_en.md

File metadata and controls

311 lines (236 loc) · 15.9 KB
Raw

GitHub release GitHub Stars GitHub Forks GitHub Contributors GitHub issues JDK support LICENSE Downloads

Github地址 | Gitee地址 | 中文 | English

📖 Introduction

Frostmourne is an open source version of the Autohome Dealer Tech monitoring system to help monitor almost all database data (including Elasticsearch, Prometheus, SkyWalking, MySql, etc.) . If you have established a logging system, metric system, but worry about not having a supporting monitoring system, maybe it can help you.

Purpose

After building a logging system with ELK, we found that there is no other solution for application log monitoring except ElastAlert. We initially used ElastAlert to solve the problem of log monitoring, However, with the increase of configuration, not only the management cost and usage cost are high, but also the stability cannot satisfy us. Therefore, for better ease of use and stability, we decided to build a simple monitoring system by ourselves. to solve the problem of log monitoring.

The project is not limited to Elasticsearch data, but also supports HTTP, PING, Telnet monitoring, Prometheus, SkyWalking, InfluxDB, MySQL, ClickHouse, IoTDB, SqlServer data monitoring, Later, more common data sources (such as Loki, MongoDB Redis, Oracle, etc.) will be added to the monitoring category. There are still many things that need to be done, and more related development needs to be added. Welcome to join us to make the project more powerful.

✨ Features

  • You only need to write a data query to easily get monitoring

  • Support multiple data source support：Elasticsearch, HTTP, SkyWalking, Prometheus, InfluxDB, MySQL/TiDb, ClickHouse, SqlServer, PING, IotDB, Telnet

  • Support number aggregation：count, min, max, avg, sum, unique count, percentiles, standard deviation; and Elasticsearch data bucket.

  • Support multiple message way: dingtalk(robot)、wechat(robot)、feishu robot、Email、sms、HTTP

  • Support multiple message type: text, markdown

  • Flexible alarm message Freemarker template customization, support for variable placeholders; message template management

  • Distributed scheduling implementation, each monitoring is independently scheduled without affecting each other

  • The alarm message comes with a short link for log query, directly to the cause of the alarm

  • Value year-on-year, month-on-month monitoring

  • HTTP data monitor, judge with Javascript expression; support PING monitor, Telnet monitor

  • The front end is simple and easy to use: monitoring management, testing, saving, execution logs and historical messages

  • Elasticsearch data query、share and download

  • Alarm message suppression function to prevent message bombardment; there is also an alarm upgrade function to prevent fault-related parties from not being notified for a long time.

  • Account, team, department information management module, you can also realize internal docking by yourself

  • Integrated LDAP login authentication

  • Permission control, data isolation, each team does not affect each other

    wechat robot message example： （View full function screenshots）

Table of contents

💻 Online demo

In order to understand the role of this project more quickly，we provide a static web with mock data for preview: online demo The online demo is not updated in time, please refer to the actual running effect of the project. The demo is only for quick browsing.

📦 Quick start

We provides the docker-compose method to make it run faster and better understand the role of the project。 Please see the documentation for details：Quick-Start

🧰 Deployment

Prepare：MySQL scripts

Frostmourne's all sql scripts in file: frostmourne.sql 。

1、k8s deployment

k8s way deployment refers to the following three configuration files:

Related parameters are configured in the frostmourne-monitor-deployment.yaml file. It should be noted that the external mapping port is specified in frostmourne-monitor-service.yaml，default nodePort=30054

kubectl applt -f frostmourne-monitor-namespace.yaml
kubectl applt -f frostmourne-monitor-deployment.yaml
kubectl apply -f frostmourne-monitor-service.yaml

2、zip package deployment

requirements

  • JDK 1.8
  • MySQL 5.7.8+

The latest Release zip package，please go to releases for download. After decompression, modify the application configuration file application.properties and environment configuration file env according to your own environment，then execute the following command to start：

./scripts/startup.sh

execute the following command to stop:

./scripts/shutdown.sh

3、Self-build deployment

requirements

  • JDK 1.8
  • Maven 3.2.x+
  • MySQL 5.7.8+

In frostmourne main folder, execute maven build command：

mvn -U clean package -DskipTests=true

The frontend module frostmourne-vue will copy static resource to frostmourne-monitor module's resources/dist folder，so you only need deploy frostmourne-monitor.

frostmourne-monitor config assembly package，target folder has zip package，then you can refer zip package deployment.

📚 Usage guide

Elasticsearch data monitor guide HTTP monitor guide Prometheus data monitor guide
SkyWalking data monitor guide MySQL data monitor guide ClickHouse data monitor guide
InfluxDB data monitor guide PING guide Same time data compare guide
Message template Send message Alarm suppression
User management and login authentication Precautions Other
Ring time data compare guide Iotdb data monitor guide Telnet guide

🛠 Develop

Requirements

  • JDK 8 ~ 14
  • Node 16.14.2 (recommand)
  • Yarn 1.22.10 (recommand) or Npm 8.7.0
  • MySQL 5.7.8+
  • Elasticsearch 6.3.2+

start up frostmourne-monitor, add arguments:

-Dmysql.host=localhost -Dmysql.user=root -Dmysql.password=example -Dlog.console.level=INFO

Modify MySQL related parameters to those of your own environment，active profile set local, test url: http://localhost:10054 , Open frostmourne-vue with VS Code，execute the following command:

# install dependency
yarn install

# develop
yarn dev

will automatically open：http://localhost:9528

Contact us if you got problems.

⚙Follow-up planning

  • 【0.9】发布0.8-RELEASE, 进入0.9开发 [2022-06-09]
  • 【0.9】增加telnet端口连通监控 [2022-06-09]
  • 【0.9】修复飞书消息发送两条的问题 [2022-06-17]
  • 【0.9】增加SqlServer数据监控报警 [2022-06-30]
  • 【0.9】监控列表增加监控调度时间查看 [2022-08-04]
  • 【0.9】优化启动脚本，支持java11, java13 [2022-08-04]
  • 【0.9】告警列表组件新增日志查询 [2022-08-13]
  • 【0.9】数据源及数据名页面数据类型新增icon [2022-08-13]
  • prometheus增加/api/v1/query_range查询支持
  • 增加本项目内程序日志采集至MySQL并提供查询页面，方便排查问题和监控
  • Elasticsearch数据名增加kibana链接配置，在数据查询页面增加kibana地址跳转链接，方便将数据查询切换至kibana
  • 短信报警方式实现，默认用阿里云短信实现
  • 完成英语国际化
  • 增加 loki 数据监控报警
  • 数据源列表页面增加数据源图标列，方便区分
  • 增加邮箱在线配置页面功能
  • 增加企业微信在线配置页面功能
  • 将短链接id以16进制格式展示，解决id数字很大的时候较长的问题
  • 增加邮箱在线配置页面功能
  • pom优化
  • 增加消息内容长度配置，超过长度配置部分将被截掉
  • 员工换组增加是否迁移监控至新组的选项，如果勾选将该员工创建的监控也转移至新组
  • 增加报警组支持
  • 增加监控转组功能
  • Elasticsearch数据名增加traceid字段配置，可以配置跳转链接。例如: 配置skywalking的链接将跳转到skywalking对应的调用链
  • 监控列表增加"执行日志"操作按钮，点击跳转到对应监控执行日志列表页
  • 增加时序数据历史数据比较规则
  • 监控增加报警消息允许发送时间段设置，非允许发送时间段内消息将只记录不发送，发送状态为FORBID
  • 增加企业钉钉发消息默认实现(本地没有环境，需要帮助，欢迎PR，或者提供示例代码，先行谢过)
  • 更多报警方式补充（欢迎PR）
  • 后端接口增加数据校验并返回合适的提示信息
  • 增加 skywalking Database Layer 数据监控报警支持
  • 监控列表增加一个开关选项，只显示我的监控
  • 监控调度配置后显示预计调度时间
  • Elasticsearch数据名配置时自动提示索引名称
  • 引入对象迭代器，迭代器代表从数据源中得到一个List数据列表，或者自定义输入；用于动态生成监控查询语句，例如: Level: ERROR AND Project: ${ITEM_VALUE}；达到数据遍历监控的效果；
  • 国际化
  • 发布1.0-RELEASE
  • 3-sigma离群点检测报警规则
  • 加入时序数据异常检测算法规则(需要实验可行性 基于时间序列的异常检测)
  • 总结项目用到的知识点

🗓 ReleaseNotes

👍 Thanks to

👷 Contributors

@menong-chen @fox2zz @xyzj91 @wxmclub @Aping

GitHub Contributors

💡️ How to contribute

If you think this project is helpful to you and want to give something back, you are very welcome to contribute. can be done as follows：

  • Select the appropriate task from the follow-up plan to submit PR
  • Make necessary additions to the documentation
  • Deploy this project and use it, inform us via github or gitee
  • Help spread the word
  • Submit your valuable suggestions in issue
  • Join the exchange group and answer exchange questions. The group will publish project updates from time to time
  • Open source is not easy, thank you for your one-click three-link encouragement
  • code conduct

💬 Contact us

If you have any questions and need help or communication, you can add the WeChat group or QQ group below. Please give priority to issue, which is convenient for the discussion and record tracking of the problem, and it is also convenient to have A buddy search solution for similar problems. Colleagues who are interested in the project are also welcome to join the group to communicate. Special mention: If you feel that the documentation is not smooth, it is difficult to understand, or there is something missing, you are welcome to raise issue。

👥 Registered user

via github or gitee registered user

汽车之家 最右APP 家家悦 阳光电源 ETCP

⭐️ Stargazers over time

Stargazers over time