Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add LLM-Based Recommendations for Broken Pipeline Runs #57

Open
wants to merge 113 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 109 commits
Commits
Show all changes
113 commits
Select commit Hold shift + click to select a range
bb52c88
feat(secrets): add commands to register, update, and remove OpenAI AP…
sklarfox Aug 21, 2024
5fc0003
feat(secrets): add function to retrieve secrets from VSCode
sklarfox Aug 21, 2024
a0de734
feat: remove unnecessary deleteOpenAIKey function
sklarfox Aug 22, 2024
a22ee0e
feat(genai): scaffold command to send OpenAI API request
sklarfox Aug 22, 2024
97bb892
feat: add basic OpenAI API request with hardcoded prompt
sklarfox Aug 22, 2024
a64b86f
feat: send hardcoded message to OpenAI
sklarfox Aug 23, 2024
f3deec4
feat: Added basic custom context menu to DAG
Morgan-Summer-Davis Aug 23, 2024
03d8e07
feat: Implemented hardcoded OpenAI calls from the DAG context menu
Morgan-Summer-Davis Aug 23, 2024
62c7fdd
fix: Fixed error in sendOpenAIRequest from user error during merge
Morgan-Summer-Davis Aug 23, 2024
6824174
feat: generalize LLM API Key registry command, support for Anthropic,…
sklarfox Aug 23, 2024
e1a363f
chore: remove todo comments
sklarfox Aug 23, 2024
5035dc0
feat: Context menu now stays open and indicates it is loading when Su…
Morgan-Summer-Davis Aug 23, 2024
c193356
chore: address coderabbitai suggestions
sklarfox Aug 23, 2024
29a93ee
feat: Suggest Fix only appears in context menu if the step resulted i…
Morgan-Summer-Davis Aug 23, 2024
fd94ec0
fix: DAG context menu now correctly retrieves component id
Morgan-Summer-Davis Aug 26, 2024
eb420c4
feat: Add initial fixMyPipelineRequest function to send code/error to…
sklarfox Aug 27, 2024
948c2c7
feat: read log file from disk, send source code / log to OpenAI
sklarfox Aug 27, 2024
0ca54d0
feat: added editStepFile function to allow gen AI features to make co…
Morgan-Summer-Davis Aug 27, 2024
0d3acb6
feat: modify LLM prompts to highlight typos and include edited code i…
sklarfox Aug 27, 2024
068ffc9
feat: suggest fix now opens a hard-coded file, finds where the step i…
Morgan-Summer-Davis Aug 27, 2024
38d76b3
fix: adjusted virtual document rendering
Morgan-Summer-Davis Aug 27, 2024
88fb58f
feat: add searchWorkspaceByFileContent function
sklarfox Aug 28, 2024
04501cb
fix: create StepData type declaration
sklarfox Aug 28, 2024
bf8648c
fix: replace JsonObject type with StepData
sklarfox Aug 28, 2024
dec0d91
fix: remove console log
sklarfox Aug 28, 2024
3777ce7
feat: refactored editStepFile to use vsCode's native difference manag…
Morgan-Summer-Davis Aug 28, 2024
fce9137
Merge branch 'exp/source-to-chatgpt' into feature/openai-api-requests
sklarfox Aug 28, 2024
7838108
fix: adjusted the 'suggest fix' text document to correctly display ma…
Morgan-Summer-Davis Aug 28, 2024
9ede811
refactor: add ArtifactData type and replace JsonObject types
sklarfox Aug 29, 2024
d1b5720
Merge branch 'stepdata-type' into feature/openai-api-requests
sklarfox Aug 29, 2024
10c1ee8
feat: dag context menu will no longer extend off screen
Morgan-Summer-Davis Aug 29, 2024
f045b45
feat: create AIService singleton class
sklarfox Aug 29, 2024
3c186ef
refactor: searchWorksapceByFileContent returns an array of 0 or more …
sklarfox Aug 29, 2024
aff9c40
refactor: split context menu logic into more encapsulated functions
Morgan-Summer-Davis Aug 29, 2024
54b3e74
refactor: modify apikey secret name
sklarfox Aug 29, 2024
8f0a4fe
refactor: change api key naming convention to `zenml.{provider}.key`
sklarfox Aug 29, 2024
d9fffce
Merge remote-tracking branch 'origin/feature/openai-api-key' into fea…
sklarfox Aug 29, 2024
a58e05c
feat: 'suggest fix' AI response now opens as markdown preview, rather…
Morgan-Summer-Davis Aug 29, 2024
f42ee33
feat: add FixMyPipeline interface
sklarfox Aug 29, 2024
3749483
refactor: editStepFile accepts a vscode.Uri type directly instead of …
sklarfox Aug 30, 2024
bc63e1f
refactor: searchWOrkspaceByFileContent returns array of matches
sklarfox Aug 30, 2024
d274e4d
feat: aiService.fixMyPipelineRequest returns fixMyPipelineResponse ty…
sklarfox Aug 30, 2024
00fb0d7
feat: migrate openai API call to structured response
sklarfox Aug 30, 2024
2ade4dd
feat: implemented functionality to cycle between 'suggest fix' code …
Morgan-Summer-Davis Aug 30, 2024
d772e2b
refactor: removed some unnecessary comments / logic
Morgan-Summer-Davis Aug 30, 2024
473dd16
feat: added command palette command to cycle through code recommendat…
Morgan-Summer-Davis Aug 30, 2024
ab965be
feat: Add OpenAI structured response format
sklarfox Aug 30, 2024
9b245b1
Merge branch 'feature/openai-api-requests' into aiService-singleton
sklarfox Aug 30, 2024
157c88b
feat: Added a command to the command bar for files edited by Suggest …
Morgan-Summer-Davis Aug 30, 2024
7d1f6a3
Merge pull request #1 from sklarfox/aiService-singleton
sklarfox Aug 30, 2024
c176a82
fix: fixed logic in createCodeRecommendation
Morgan-Summer-Davis Sep 2, 2024
2f7f9b8
fix: code recommendation should now always open in the proper viewColumn
Morgan-Summer-Davis Sep 2, 2024
2a4fd27
feat: code recommendations for files no longer persist after being cl…
Morgan-Summer-Davis Sep 2, 2024
515c575
fix: suggest fix files now open relative to the DAG that spawned them
Morgan-Summer-Davis Sep 2, 2024
2b3b85a
fix: suggest fix no longer prompts the user to see the next code sugg…
Morgan-Summer-Davis Sep 2, 2024
0ccbf10
fix: removed hardcoded values for ChatGPT response in suggest fix
Morgan-Summer-Davis Sep 3, 2024
2192fdb
refactor: moved fixBrokenStep to AIStepFixer
Morgan-Summer-Davis Sep 3, 2024
cc1bb36
fix: removed unnecessary imports
Morgan-Summer-Davis Sep 3, 2024
fab0fc0
Merge branch 'feature/ui-improvements' into feature/openai-api-requests
Morgan-Summer-Davis Sep 3, 2024
18ab0a5
fix: fixed several merge-related issues
Morgan-Summer-Davis Sep 3, 2024
11632b6
feat: migrate to provider agnostic, replace openai with token.js
sklarfox Sep 4, 2024
5f38d20
chore: remove comment
sklarfox Sep 4, 2024
89b5842
chore: token.js ts-expect-error annotation
sklarfox Sep 4, 2024
196a325
fix: add check for existing API key variable
sklarfox Sep 4, 2024
9f17914
fix: implemented Suggest Fix with vscode.diff and a virtual document …
Morgan-Summer-Davis Sep 4, 2024
23afa52
feat: implemented Suggest Fix with a virtual file system to allow liv…
Morgan-Summer-Davis Sep 5, 2024
5c48e9a
fix: fixed updateCodeRecommendation functionality with new virtual fi…
Morgan-Summer-Davis Sep 5, 2024
40d609a
fix: adjusted several uri comparisons to string comparisons
Morgan-Summer-Davis Sep 5, 2024
0877593
feat: implemented git grep as a fallback in cases where manual file s…
Morgan-Summer-Davis Sep 9, 2024
9f15f60
feat: searchWorkspaceByFileContent now returns both uri and the file'…
Morgan-Summer-Davis Sep 10, 2024
090b5d8
feat: implemented an Accept Change option in suggest fix
Morgan-Summer-Davis Sep 11, 2024
495b8d0
fix: suggest fix will now search for the location of the relevant fil…
Morgan-Summer-Davis Sep 11, 2024
be2ac96
feat: added notifications on unexpected outcomes when searching for f…
Morgan-Summer-Davis Sep 11, 2024
09c64c4
fix: suggest fix will not attempt to create a code recommendation if …
Morgan-Summer-Davis Sep 11, 2024
cddc78e
feat: implemented UI for selecting default LLM for suggest fix
Morgan-Summer-Davis Sep 16, 2024
0b5c677
feat: implemented frontend and dummy backend code for setting a defau…
Morgan-Summer-Davis Sep 16, 2024
338c356
refactor: split secondary functionality in AIStepFixer.ts into modules
Morgan-Summer-Davis Sep 18, 2024
e85a3cb
fix: change order of conditionals when checking for an existing API k…
sklarfox Sep 4, 2024
c205c74
feat: migrate python snippets to non-regex method, map response
sklarfox Sep 17, 2024
189fa44
test: add mockResponse
sklarfox Sep 17, 2024
4b4d639
feat: expand configuration options to include Anthropic and Gemini mo…
sklarfox Sep 18, 2024
79d1bc9
chore: remove old config file
sklarfox Sep 18, 2024
f23ff51
feat: add llm configuration options / getter methods for frontend UI
sklarfox Sep 18, 2024
f37d385
chore: fix linting curly warnings
sklarfox Sep 18, 2024
0e61c98
fix: change order of conditionals when checking for an existing API k…
sklarfox Sep 4, 2024
51cd458
feat: migrate python snippets to non-regex method, map response
sklarfox Sep 17, 2024
2bbd7e2
test: add mockResponse
sklarfox Sep 17, 2024
3292518
feat: expand configuration options to include Anthropic and Gemini mo…
sklarfox Sep 18, 2024
31d2b1f
chore: remove old config file
sklarfox Sep 18, 2024
596c1ae
feat: add llm configuration options / getter methods for frontend UI
sklarfox Sep 18, 2024
63df37a
chore: fix linting curly warnings
sklarfox Sep 18, 2024
6cb4c5b
Merge branch 'prompt-engineering' of https://github.com/sklarfox/vsco…
sklarfox Sep 19, 2024
2daee5b
refactor: rename zenml config setting from llm-provider to llm-model
sklarfox Sep 19, 2024
88a66bb
fix: SaveAIChangeEmitter import path
sklarfox Sep 19, 2024
320c6c2
refactor(AIStepFixer): add Provider/Model typing from AIService, adju…
sklarfox Sep 19, 2024
6c29ce7
chore: remove mock data and deprecated test command
sklarfox Sep 20, 2024
c69cdfe
Merge pull request #2 from sklarfox/prompt-engineering
sklarfox Sep 20, 2024
5819e4f
fix: resolved bug that only allowed users to change model by resettin…
Morgan-Summer-Davis Sep 24, 2024
02d8207
feat: added indicator to DAG context menu to show the current LLM
Morgan-Summer-Davis Sep 24, 2024
40d9e9e
fix: fixed error which would incorrectly set API key to stringified u…
Morgan-Summer-Davis Sep 24, 2024
62cf143
refactor: reestablished front-end refactor that was accidentally remo…
Morgan-Summer-Davis Sep 24, 2024
a45b6c7
fix: fixed secret storage so that storage of LLM names with periods w…
Morgan-Summer-Davis Sep 24, 2024
2f2d092
feat: added message when user attempts to suggest a fix with no LLM s…
Morgan-Summer-Davis Sep 24, 2024
b353d33
feat: added error-handling when LLM sends back an error response
Morgan-Summer-Davis Sep 24, 2024
fcc1eab
fix: re-registering an LLM API key now updates the corresponding envi…
sklarfox Sep 25, 2024
ed4f7df
Merge branch 'feature/llm-fix-my-pipeline' into develop
Morgan-Summer-Davis Sep 25, 2024
3601132
fix: addresses most of coderabbitai's suggestions
Morgan-Summer-Davis Sep 26, 2024
3fa49bf
fix: code file will no longer open when there are no code fixes to su…
Morgan-Summer-Davis Sep 26, 2024
d920b32
refactor: remove process.env calls throughout AIService, make secretS…
sklarfox Sep 26, 2024
754c0f1
refactor: removed console.log
Morgan-Summer-Davis Sep 27, 2024
00a1f26
feat: added notice to top of markdown document
Morgan-Summer-Davis Sep 27, 2024
aa43052
fix: fixed broken 'Accept Changes' functionality
Morgan-Summer-Davis Sep 27, 2024
5ea8113
fix: codeblocks that are indented in the response are now de-indented…
Morgan-Summer-Davis Sep 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2,838 changes: 2,534 additions & 304 deletions package-lock.json

Large diffs are not rendered by default.

200 changes: 129 additions & 71 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -63,83 +63,104 @@
"watch-tests": "tsc -p . -w --outDir out"
},
"contributes": {
"configuration": {
"title": "ZenML",
"properties": {
"zenml-python.args": {
"default": [],
"description": "Arguments passed in. Each argument is a separate item in the array.",
"items": {
"type": "string"
"configuration": [
{
"title": "ZenML",
"properties": {
"zenml-python.args": {
"default": [],
"description": "Arguments passed in. Each argument is a separate item in the array.",
"items": {
"type": "string"
},
"scope": "resource",
"type": "array"
},
"scope": "resource",
"type": "array"
},
"zenml-python.path": {
"default": [],
"scope": "resource",
"items": {
"zenml-python.path": {
"default": [],
"scope": "resource",
"items": {
"type": "string"
},
"type": "array"
},
"zenml-python.importStrategy": {
"default": "useBundled",
"enum": [
"useBundled",
"fromEnvironment"
],
"enumDescriptions": [
"Always use the bundled version of `<pytool-module>`.",
"Use `<pytool-module>` from environment, fallback to bundled version only if `<pytool-module>` not available in the environment."
],
"scope": "window",
"type": "string"
},
"type": "array"
},
"zenml-python.importStrategy": {
"default": "useBundled",
"enum": [
"useBundled",
"fromEnvironment"
],
"enumDescriptions": [
"Always use the bundled version of `<pytool-module>`.",
"Use `<pytool-module>` from environment, fallback to bundled version only if `<pytool-module>` not available in the environment."
],
"scope": "window",
"type": "string"
},
"zenml-python.interpreter": {
"default": [],
"description": "When set to a path to python executable, extension will use that to launch the server and any subprocess.",
"scope": "resource",
"items": {
"zenml-python.interpreter": {
"default": [],
"description": "When set to a path to python executable, extension will use that to launch the server and any subprocess.",
"scope": "resource",
"items": {
"type": "string"
},
"type": "array"
},
"zenml-python.showNotifications": {
"default": "off",
"description": "Controls when notifications are shown by this extension.",
"enum": [
"off",
"onError",
"onWarning",
"always"
],
"enumDescriptions": [
"All notifications are turned off, any errors or warning are still available in the logs.",
"Notifications are shown only in the case of an error.",
"Notifications are shown for errors and warnings.",
"Notifications are show for anything that the server chooses to show."
],
"scope": "machine",
"type": "string"
},
"type": "array"
},
"zenml-python.showNotifications": {
"default": "off",
"description": "Controls when notifications are shown by this extension.",
"enum": [
"off",
"onError",
"onWarning",
"always"
],
"enumDescriptions": [
"All notifications are turned off, any errors or warning are still available in the logs.",
"Notifications are shown only in the case of an error.",
"Notifications are shown for errors and warnings.",
"Notifications are show for anything that the server chooses to show."
],
"scope": "machine",
"type": "string"
},
"zenml.serverUrl": {
"type": "string",
"default": "",
"description": "ZenML Server URL"
},
"zenml.accessToken": {
"type": "string",
"default": "",
"description": "Access token for the ZenML server"
},
"zenml.activeStackId": {
"type": "string",
"default": "",
"description": "Active stack id for the ZenML server"
"zenml.serverUrl": {
"type": "string",
"default": "",
"description": "ZenML Server URL"
},
"zenml.accessToken": {
"type": "string",
"default": "",
"description": "Access token for the ZenML server"
},
"zenml.activeStackId": {
"type": "string",
"default": "",
"description": "Active stack id for the ZenML server"
}
}
},
{
"title": "ZenML AI",
"properties": {
"zenml.llm-model": {
"enum": [
"anthropic.claude-3-5-sonnet-20240620",
"anthropic.claude-3-opus-20240229",
"anthropic.claude-3-haiku-20240307",
"gemini.gemini-1.5-pro",
"gemini.gemini-1.5-flash",
"gemini.gemini-1.0-pro",
"openai.gpt-4o",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"openai.gpt-4o",
"openai.gpt-4o",
"openai.gpt-4",

"openai.gpt-4o-mini",
"openai.gpt-4-turbo",
"openai.gpt-3.5-turbo"
]
}
}
}
},
],
"commands": [
{
"command": "zenml.promptForInterpreter",
Expand Down Expand Up @@ -297,6 +318,29 @@
"title": "Restart LSP Server",
"icon": "$(debug-restart)",
"category": "ZenML Environment"
},
{
"command": "zenml.registerLLMAPIKey",
"title": "Register LLM API Key",
"icon": "$(add)",
"category": "ZenML Secrets"
},
{
"command": "zenml.sendOpenAIMessage",
"title": "Send OpenAI API Message",
"category": "ZenML GenAI"
},
{
"command": "zenml.displayNextCodeRecommendation",
"title": "Next Recommendation",
"category": "ZenML GenAI",
"when": "resourceFilename in zenml.aiCodeRecommendations && resourceScheme == zenml-stepfixer"
},
{
"command": "zenml.acceptCodeRecommendation",
"title": "Accept Change",
"category": "ZenML GenAI",
"when": "resourceFilename in zenml.aiCodeRecommendations && resourceScheme == zenml-stepfixer"
}
],
"viewsContainers": {
Expand Down Expand Up @@ -469,6 +513,18 @@
"command": "zenml.setPythonInterpreter",
"group": "inline"
}
],
"editor/title": [
{
"command": "zenml.displayNextCodeRecommendation",
"group": "navigation",
"when": "resourceFilename in zenml.aiCodeRecommendations && resourceScheme == zenml-stepfixer"
},
{
"command": "zenml.acceptCodeRecommendation",
"group": "navigation",
"when": "resourceFilename in zenml.aiCodeRecommendations && resourceScheme == zenml-stepfixer"
}
]
}
},
Expand All @@ -493,7 +549,7 @@
"sinon": "^17.0.1",
"ts-loader": "^9.5.1",
"ts-node": "^10.9.2",
"typescript": "^5.3.3",
"typescript": "^5.6.2",
"webpack": "^5.94.0",
"webpack-cli": "^5.1.4"
},
Expand All @@ -504,8 +560,10 @@
"dagre": "^0.8.5",
"fs-extra": "^11.2.0",
"hbs": "^4.2.0",
"simple-git": "^3.26.0",
"svg-pan-zoom": "github:bumbu/svg-pan-zoom",
"svgdom": "^0.1.19",
"token.js": "^0.4.4",
"vscode-languageclient": "^9.0.1"
}
}
33 changes: 33 additions & 0 deletions resources/dag-view/dag.css
Original file line number Diff line number Diff line change
Expand Up @@ -155,3 +155,36 @@ g.svg-pan-zoom-control {
width: 100%;
height: 100%;
}

#context-menu {
position: absolute;
background: white;
box-shadow: 0 4px 5px 3px rgba(0, 0, 0, 0.2);
border-radius: 5px;
}

#context-menu ul {
list-style: none;
padding: 0px;
}

#context-menu li {
font-weight: 500;
font-size: 14px;
padding: 10px 40px 10px 20px;
cursor: pointer;
}

#context-menu li:hover {
color: #333;
background: #e0e0e0;
}

#loading-ai {
display: none;
}

.context-menu-subitem-indicator {
font-size: 0.7rem;
vertical-align: text-top;
}
Loading