-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow parsing just the vault name without requiring the secret path #305
Conversation
I think I prefer null in those scenarios? Undefined is usually meant to mean void.
|
37a9a58
to
aac9445
Compare
Undefined usually means it wasn't provided. Null would mean it was intentionally not provided. We always use undefined if it's optional. |
I mean in TS, |
It depends, you need to review other places in the code where this has occurred. |
Look at the other parsers. |
91a4f12
to
616df30
Compare
While working on this PR, I actually discovered a bug in https://github.com/MatrixAI/Polykey-CLI/issues/311 There could have been other issues like this which might have slipped past testing and reviewing. I will go through all the currently implemented commands and ensure that the tests properly test everything so similar bugs can't catch us by surprise. |
I just realised a potential issue that would crop up in the future. Lets say that I have a vault called This same issue applies to basically all secrets commands which are expected to operate on the user's local file system. This needs to be addressed before we go about merging this PR. |
I have observed an interesting behaviour with [aryanj@matrix-34xx:~]$ pk secrets write vault:test
some content
more content
i will press ctrl d on the next line
this line will not be shown
[aryanj@matrix-34xx:~]$ pk secrets cat vault:test
some content
more content
i will press ctrl d on the next line
[aryanj@matrix-34xx:~]$ pk secrets cat vault:test > tempfile
[aryanj@matrix-34xx:~]$ cat tempfile
some content
more content
i will press ctrl d on the next line
this line will not be shown% Terminal emulators add a This is not supported by Node's As a workaround, currently we are manually writing a This still needs to be changed to be more intelligent, and detect if the last character was a newline, and do this only if it wasn't. Another interesting discovery was made in regards to We should also consider writing a message to stderr informing the user that the message will now be written to the file, as writing to the file takes about a second, and the users won't be sure if the program needs another |
I think firstly there are only some commands that operate on both local and vault namespaces. That's the So in the case of The solutions:
Do note that you don't currently have PCC nor OCC with vault operations via RPC, until you've developed the stream-lifecycle representing an RPC conversation where the stream lifetime means 1 PCC lock. OCC requires MVCC and that's not possible without snapshots of the vault state which we cannot do at the moment. Please ensure that: pk secrets cp test test # this is an error
pk secrets cp -r testA testB # this means copying testA into testB, so `testA:/testB`
pk secrets mv test ./test # means local path
pk secrets mv test test/test # means moving test into test/test Note that There's actually alot of constraints here. @tegefaulkes you should be working with @aryanjassal to ensure that all scenarios are being fast checked here and clearly in one big list. I want to have a clear list here in the spec. |
The |
This is a bad idea. |
I've never seen this with other unix commands, you should check how you're taking in STDIN here. |
This is definitely a bug and not something terminal emulators fail at doing. The reason for this is due to line buffering. You just need to flush the buffer explicitly at the end when the |
In fact you SHOULD ALWAYS be flushing the buffer explicitly at the end of every CLI command. |
This a key constraint. You need to fast check this well. The point is But Let's get some variants: Here’s the updated table based on the clarified rules that vault paths must follow the
Key Rules Recap:
This table and rule set should now provide a clear, unambiguous interpretation of paths in the context of vault and local operations. |
This would mean that vault names cannot have a Also, what about Should we allow vault names to start with |
Yes Under the rules then it will need to be Well we have to make some rules on a vault name - it would be more constrained than any path. At least 2 symbols are not allowed here: We may also want to disallow non-printable characters and control characters but utf-8 symbols are ok. Use a regex rule - ask chatgpt to help generate and use fastcheck and regex101 to check. |
Previously, and in this PR for now, only ASCII characters are allowed as the vault name. The previous regex followed this: Polykey-CLI/src/utils/parsers.ts Line 11 in fd5bd17
Right now, this PR is following I will work on stricter rules and parsing for vault names in this PR. #312 will see a simple change to additionally allow dots in vault names, as that is required urgently. More extensive changes will be made in this PR. |
2848e25
to
fc7bb0b
Compare
https://unix.stackexchange.com/a/701319 It does appear to be consistent across unix commands. This behaviour is also seen on unix's |
The rules for local paths aren't relevant for this PR, as its scope is vault paths. I will make a new issue for implementation of local paths, and input all this information there. |
I tried to add a I'll make a separate issue for this, as I don't think that this can be done in this PR. |
As ENG-431 (#310) is a small fix, I will address that in this PR too. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This PR is more or less complete. I've put down some of my thoughts in my review, which would be useful for reference later.
expect(result.exitCode).toBe(0); | ||
expect(result.stdout).toBe('MySecret1\nMySecret2\nMySecret3\n'); | ||
}, | ||
globalThis.defaultTimeout * 2, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When I ran the tests, I saw no need for additional timeout, so I removed this line. This applies to all secrets subcommand tests, as most of them completed within 3 seconds, with longer ones taking up to 5, which is well under the 20 second timeout limit.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Usually we preserve the globalThis.defaultTimeout
because that's the default.
const vaultNameArb = fc.stringOf( | ||
fc.char().filter((c) => binParsers.vaultNameRegex.test(c)), | ||
{ minLength: 1, maxLength: 100 }, | ||
); | ||
const singleSecretPathArb = fc.stringOf( | ||
fc.char().filter((c) => binParsers.secretPathRegex.test(c)), | ||
{ minLength: 1, maxLength: 25 }, | ||
); | ||
const secretPathArb = fc | ||
.array(singleSecretPathArb, { minLength: 1, maxLength: 5 }) | ||
.map((segments) => path.join(...segments)); | ||
const valueFirstCharArb = fc.char().filter((c) => /^[a-zA-Z_]$/.test(c)); | ||
const valueRestCharArb = fc.stringOf( | ||
fc.char().filter((c) => /^[\w]$/.test(c)), | ||
{ minLength: 1, maxLength: 100 }, | ||
); | ||
const valueDataArb = fc | ||
.tuple(valueFirstCharArb, valueRestCharArb) | ||
.map((components) => components.join('')); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This probably isn't the best way to generate testing data, but this is the only way I found to ensure the tested data will be valid at all times. Instead of using the actual regex, I might need to use some other arb rule, but it would become pretty complex pretty fast.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Regex is fine. What's wrong with regex?
1fed621
to
602a546
Compare
chore: vault commands are now using vaultNameParser chore: writing fastcheck tests for parsers chore: jestified outputFormatter tests feat: updated tests for every secret command to allow undefined secret path fix: consistently using command instead of [...command] in secret tests chore: stream consumption must be inside retryAuthentication chore: tests consistently use command fix: lint fix: mkdir fix: variable name
602a546
to
8a59d6e
Compare
All the tasks for this PR have been done. Brian and I have sanity checked the commands and ensured that they all work as expected. I will merge this PR and make a new one to address all the issues that cropped up during the development of this PR. |
Description
When specifying a secret path, currently the format needs to be
<vaultName>:<secretPath>
, otherwise a parser error is thrown. This should not be the case, and instead paths like<vaultName>
should also be allowed, which would point to the root of the directory.To do this, a parser for
vaultName
will also be implemented, ensuring consistency in vault's allowed characters. This is important as currently, there are no parsers for vault name, but there is a regex validation for vault names when they are being used in the secret commands. This means that inadvertently vaults can be created which cannot be actually used in any commands likematrix.ai
.Issues Fixed
Tasks
'/'
if the secret path is undefinedout of scope for this PR[ ]
7. Split tests forvaults/scanNode.test.ts
cat
create
dir
edit
env
list
mkdir
remove
rename
stat
write
vaults list
tovaults ls
Final checklist