Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add realtime inference consumer #90

Open
wants to merge 50 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
b5191c6
Merge pull request #77 from RADAR-base/release-0.3.0
yatharthranjan Jul 17, 2018
36e8ae2
Merge pull request #83 from RADAR-base/release-0.4.0
nivemaham Sep 13, 2018
d549524
Initial impl for realtime consumers
yatharthranjan Apr 20, 2021
850c0e6
Fix dependencies
yatharthranjan Oct 18, 2021
3109e31
First complete set of conditions and actions
yatharthranjan Oct 20, 2021
70f1594
better use of enums
yatharthranjan Oct 20, 2021
1d5915a
add basic javadocs
yatharthranjan Oct 20, 2021
be852c8
simplify defaults and enum
yatharthranjan Oct 21, 2021
bc8cbc8
add support for filtering conditions and actions based on projects.
yatharthranjan Nov 30, 2021
9b08b8a
remove unintended check
yatharthranjan Nov 30, 2021
57c246f
add appserver related components to docker-compose.yml file
nivemaham Jan 12, 2022
72ac511
Merge remote-tracking branch 'origin/master' into add_realtime_consumer
yatharthranjan Jan 26, 2022
03f3762
minor fixes to make build successful
yatharthranjan Jan 26, 2022
9fb26af
Merge branch 'add_realtime_consumer' into intervention-trigger
nivemaham Feb 3, 2022
2b53ffc
resolve conflicts
nivemaham Feb 3, 2022
50af14f
move monitor configs to separate package
nivemaham Feb 3, 2022
a93afd8
rename NotifyConfig.java as EmailNotifyConfig
nivemaham Feb 3, 2022
e1a3b93
restructure email server config.
nivemaham Feb 3, 2022
25435cb
new intervention config model
nivemaham Feb 7, 2022
426f4e6
Commit InterventionMonitor
nivemaham Feb 7, 2022
077837b
Commit config format and object creation
nivemaham Feb 7, 2022
fac723b
Implemented intervention monitor and updated build config
blootsvoets Feb 15, 2022
5e61dd3
Simplify docker build to remove rest proxy
blootsvoets Feb 15, 2022
a37f5a7
Fix tests
blootsvoets Feb 15, 2022
628d17a
Fix docker setup
blootsvoets Feb 15, 2022
6522391
Use sh in docker
blootsvoets Feb 16, 2022
5811215
Move email senders and simplify config
blootsvoets Feb 16, 2022
3e199ef
Tested and fixed tests
blootsvoets Feb 16, 2022
9c25663
Tested full flow
blootsvoets Feb 17, 2022
618c9e7
Rename .java to .kt
yatharthranjan Feb 21, 2022
06e5482
Initial kotlin convert of added classes
yatharthranjan Feb 21, 2022
31af146
various fixes and improvements
yatharthranjan Feb 21, 2022
2e07eb5
Final improvements and fixes
yatharthranjan Feb 22, 2022
e7a8b9b
Minor updates
yatharthranjan Feb 22, 2022
4e92616
Add localisation support for AppServerIntervention.kt
yatharthranjan Feb 23, 2022
a24adcf
fix referenceTimestamp
yatharthranjan Feb 23, 2022
ed04048
exclude clinicalProtocol if null
yatharthranjan Feb 23, 2022
77a8082
fix eval
yatharthranjan Feb 23, 2022
0a745ce
fix and test email action
yatharthranjan Feb 24, 2022
17308f3
Notify admin in case of action, condition or consumer failure
yatharthranjan Feb 24, 2022
3d808c6
Add docs on realtime inference consumers
yatharthranjan Feb 24, 2022
bfa015c
Merge pull request #92 from RADAR-base/finalise_realtime_inference
yatharthranjan Feb 24, 2022
1fc45c6
JsonPathCondition.kt: return false if record value is null
yatharthranjan Feb 28, 2022
3c5b488
Add support for sending both notification and data message simultaneo…
yatharthranjan Mar 3, 2022
76ab324
send data messages with HIGH priority
yatharthranjan Mar 3, 2022
28a5874
make jitter configurable and null by default
yatharthranjan Mar 3, 2022
0b306f3
Fix ttl
yatharthranjan Mar 21, 2022
497f1ed
add explicit "type" to protocol
yatharthranjan May 17, 2022
41f3582
Add action for adding triggered task to user's schedule on appserver
yatharthranjan Nov 7, 2022
dde1b4a
Merge branch 'dev' into add_realtime_consumer
yatharthranjan Sep 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 9 additions & 6 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -37,13 +37,14 @@ ext {
findbugVersion = '3.0.2'
commonsCliVersion = '1.4'
mockitoVersion = '3.5.11'
radarCommonsVersion = '0.13.0'
radarSchemasVersion = '0.5.14'
radarCommonsVersion = '0.13.2'
radarSchemasVersion = '0.7.4'
subethamailVersion = '3.1.7'
jsoupVersion = '1.13.1'
slf4jVersion = '1.7.30'
log4jVersion = '1.2.17'
avroVersion = '1.9.2'
jsonPathVersion = '2.6.0'
}

//---------------------------------------------------------------------------//
Expand All @@ -53,9 +54,6 @@ ext {
// In this section you declare where to find the dependencies of your project
repositories {
mavenCentral()
// Non-jcenter radar releases
maven { url 'http://dl.bintray.com/radar-cns/org.radarcns' }
maven { url 'http://dl.bintray.com/radar-base/org.radarbase' }
// Kafka/confluent releases
maven { url 'http://packages.confluent.io/maven/' }
// For working with dev-branches
Expand All @@ -69,7 +67,9 @@ dependencies {
implementation group: 'org.radarbase', name: 'radar-commons', version: radarCommonsVersion
implementation group: 'org.apache.avro', name: 'avro', version: avroVersion
implementation group: 'org.radarbase', name: 'radar-commons-testing', version: radarCommonsVersion
implementation group: 'org.radarcns', name: 'radar-schemas-commons', version: radarSchemasVersion
implementation group: 'org.radarbase', name: 'radar-schemas-commons', version:
radarSchemasVersion
implementation group: 'org.radarbase', name: 'oauth-client-util', version: '0.8.0'

// Kafka streaming API
implementation group: 'org.apache.kafka', name: 'kafka-streams', version: kafkaVersion
Expand All @@ -92,6 +92,9 @@ dependencies {
// JSoup html parser
implementation group: 'org.jsoup', name: 'jsoup', version: jsoupVersion

// JsonPath for evaluating json conditions dynamically.
implementation group: 'com.jayway.jsonpath', name: 'json-path', version: jsonPathVersion

runtimeOnly group: 'org.radarbase', name: 'radar-commons-unsafe', version: radarCommonsVersion

runtimeOnly group: 'log4j', name: 'log4j', version: log4jVersion
Expand Down
3 changes: 1 addition & 2 deletions gradle/wrapper/gradle-wrapper.properties
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
#Fri Oct 04 14:28:19 BST 2019
distributionUrl=https\://services.gradle.org/distributions/gradle-6.6.1-bin.zip
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
3 changes: 3 additions & 0 deletions src/main/java/org/radarcns/RadarBackend.java
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
import org.radarcns.config.RadarBackendOptions;
import org.radarcns.config.RadarPropertyHandler;
import org.radarcns.config.SubCommand;
import org.radarcns.consumer.realtime.RealtimeInferenceConsumerFactory;
import org.radarcns.monitor.KafkaMonitorFactory;
import org.radarcns.producer.MockProducerCommand;
import org.radarcns.stream.KafkaStreamFactory;
Expand Down Expand Up @@ -76,6 +77,8 @@ public SubCommand createCommand() throws IOException {
return new KafkaMonitorFactory(options, radarPropertyHandler).createMonitor();
case "mock":
return new MockProducerCommand(options, radarPropertyHandler);
case "realtime_consumers":
return RealtimeInferenceConsumerFactory.createConsumersFor(radarPropertyHandler);
default:
throw new IllegalArgumentException("Unknown subcommand "
+ options.getSubCommand());
Expand Down
11 changes: 11 additions & 0 deletions src/main/java/org/radarcns/config/ConfigRadar.java
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@

import org.radarbase.config.ServerConfig;
import org.radarbase.config.YamlConfigLoader;
import org.radarcns.config.realtime.RealtimeConsumerConfig;

/**
* POJO representing the yml file
Expand All @@ -42,6 +43,8 @@ public class ConfigRadar {
private DisconnectMonitorConfig disconnectMonitor;
@JsonProperty("statistics_monitors")
private List<SourceStatisticsStreamConfig> statisticsMonitors;
@JsonProperty("realtime_consumers")
private List<RealtimeConsumerConfig> consumerConfigs;
@JsonProperty("stream")
private StreamConfig stream;
@JsonProperty("persistence_path")
Expand Down Expand Up @@ -182,6 +185,14 @@ public void setStatisticsMonitors(List<SourceStatisticsStreamConfig> statisticsM
this.statisticsMonitors = statisticsMonitors;
}

public List<RealtimeConsumerConfig> getConsumerConfigs() {
return consumerConfigs;
}

public void setConsumerConfigs(List<RealtimeConsumerConfig> consumerConfigs) {
this.consumerConfigs = consumerConfigs;
}

public StreamConfig getStream() {
return stream;
}
Expand Down
24 changes: 24 additions & 0 deletions src/main/java/org/radarcns/config/realtime/ActionConfig.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
package org.radarcns.config.realtime;

import java.util.Map;

public class ActionConfig {
private String name;
private Map<String, Object> properties;

public String getName() {
return name;
}

public void setName(String name) {
this.name = name;
}

public Map<String, Object> getProperties() {
return properties;
}

public void setProperties(Map<String, Object> properties) {
this.properties = properties;
}
}
24 changes: 24 additions & 0 deletions src/main/java/org/radarcns/config/realtime/ConditionConfig.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
package org.radarcns.config.realtime;

import java.util.Map;

public class ConditionConfig {
private String name;
private Map<String, Object> properties;

public String getName() {
return name;
}

public void setName(String name) {
this.name = name;
}

public Map<String, Object> getProperties() {
return properties;
}

public void setProperties(Map<String, Object> properties) {
this.properties = properties;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
package org.radarcns.config.realtime;

import com.fasterxml.jackson.annotation.JsonProperty;
import java.util.List;
import java.util.Map;

public class RealtimeConsumerConfig {

private String name;

private String topic;

@JsonProperty("conditions")
private List<ConditionConfig> conditionConfigs;

@JsonProperty("actions")
private List<ActionConfig> actionConfigs;

@JsonProperty("consumer_properties")
private Map<String, String> consumerProperties;

public String getName() {
return name;
}

public void setName(String name) {
this.name = name;
}

public String getTopic() {
return topic;
}

public void setTopic(String topic) {
this.topic = topic;
}

public List<ConditionConfig> getConditionConfigs() {
return conditionConfigs;
}

public void setConditionConfigs(List<ConditionConfig> conditionConfigs) {
this.conditionConfigs = conditionConfigs;
}

public List<ActionConfig> getActionConfigs() {
return actionConfigs;
}

public void setActionConfigs(List<ActionConfig> actionConfigs) {
this.actionConfigs = actionConfigs;
}

public Map<String, String> getConsumerProperties() {
return consumerProperties;
}

public void setConsumerProperties(Map<String, String> consumerProperties) {
this.consumerProperties = consumerProperties;
}
}
Loading