Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

meltano config tap-smoke-test set --interactive doesnt work well #95

Open
jfo8001 opened this issue Oct 13, 2023 · 1 comment
Open

meltano config tap-smoke-test set --interactive doesnt work well #95

jfo8001 opened this issue Oct 13, 2023 · 1 comment

Comments

@jfo8001
Copy link

jfo8001 commented Oct 13, 2023

I was trying to use this for some sample data and it was nearly impossible to choose a stream inside of the interactive configurator - I had to fall back to using meltano to copy paste.

meltano config tap-smoke-test set --interactive
Choose option 10 (streams)

image

[ {"stream_name": "animals"}, {"input_filename": "https://raw.githubusercontent.com/meltano/tap-smoke-test/main/demo-data/animals-data.jsonl"}]

image

When you meltano invoke tap-smoke-test it blows chunks because there's an extra dash in front of input_filename in meltano.yml

In the end I had to scrape the config out of the getting started guide
https://docs.meltano.com/cloud/getting_started/#new-to-meltano

Given this is in the getting started guide - perhaps some love and attention to the interactive config would be nice?

@edgarrmondragon
Copy link
Collaborator

Thanks for reporting @jfo8001!

I think the error here is the input format, since you're inputting two array elements each with one key, instead of a single element with two keys:

- [ {"stream_name": "animals"}, {"input_filename": "https://raw.githubusercontent.com/meltano/tap-smoke-test/main/demo-data/animals-data.jsonl"}]
+ [ {"stream_name": "animals", "input_filename": "https://raw.githubusercontent.com/meltano/tap-smoke-test/main/demo-data/animals-data.jsonl"}]

This produces what I think is the correct setting value:

extractors:
  - name: tap-smoke-test
    variant: meltano
    pip_url: git+https://github.com/meltano/tap-smoke-test.git
    config:
      streams:
        - stream_name: animals
          input_filename: https://raw.githubusercontent.com/meltano/tap-smoke-test/main/demo-data/animals-data.jsonl

The "Getting Started" guide could certainly be improved, PRs welcome!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants