-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[CT-2931] TypeError: > not supported between str and int when attempting to build a dbt project #8312
Comments
I am trying to finish a project with a deadline so any help would be appreciated. |
Could you provide any more log output? Specifically, we'd be looking a filename and line number for the underlying Python exception. Your Alternatively, setting the log level to dbt --log-level debug build |
It doesn't generate a file or line listing. It just displays the error which is a python type error. I have attached the log of the console.
Thanks,
Brian Custer
206-661-2674
…________________________________
From: Doug Beatty ***@***.***>
Sent: Thursday, August 3, 2023 1:56 PM
To: dbt-labs/dbt-core ***@***.***>
Cc: Brian Custer ***@***.***>; Author ***@***.***>
Subject: EXTERNAL - Re: [dbt-labs/dbt-core] [CT-2931] TypeError: > not supported between str and int when attempting to build a dbt project (Issue #8312)
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
Could you provide any more log output? Specifically, we'd be looking a filename and line number for the underlying Python exception.
Your logs/dbt.log may have more details.
Alternatively, setting the log level<https://docs.getdbt.com/reference/global-configs/logs#log-level> to debug should provide more verbose output:
dbt --log-level debug build
—
Reply to this email directly, view it on GitHub<#8312 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A3FCQ5EWHABSJ6JNUUPSGHTXTQF6RANCNFSM6AAAAAA3DHHX3A>.
You are receiving this because you authored the thread.Message ID: ***@***.***>
[0m20:50:41 Registered adapter: databricks=1.5.5
[0m20:50:41 checksum: 529671a0b29ebfc974e67a4b34212002b33abafa4e7571c4d11f04a63b60249a, vars: {}, profile: user, target: , version: 1.5.4
[0m20:50:41 Unable to do partial parsing because saved manifest not found. Starting full parse.
[0m20:50:44 1605: jinja rendering because of STATIC_PARSER flag. file: staging/d365/stg_d365__productnames.sql
[0m20:50:44 1605: jinja rendering because of STATIC_PARSER flag. file: staging/d365/stg_d365__customers.sql
[0m20:50:44 1605: jinja rendering because of STATIC_PARSER flag. file: staging/d365/stg_d365__partnerpayments.sql
[0m20:50:44 1605: jinja rendering because of STATIC_PARSER flag. file: staging/d365/stg_d365__partnercontracts.sql
[0m20:50:44 1605: jinja rendering because of STATIC_PARSER flag. file: staging/d365/stg_d365__paymentterms.sql
[0m20:50:44 1605: jinja rendering because of STATIC_PARSER flag. file: staging/d365/stg_d365__partners.sql
[0m20:50:44 1605: jinja rendering because of STATIC_PARSER flag. file: staging/d365/stg_d365__subscriptionscheduleheader.sql
[0m20:50:44 1605: jinja rendering because of STATIC_PARSER flag. file: staging/d365/stg_d365__subscriptionsdetailline.sql
[0m20:50:44 1605: jinja rendering because of STATIC_PARSER flag. file: staging/d365/stg_d365__customerpayments.sql
[0m20:50:44 1605: jinja rendering because of STATIC_PARSER flag. file: staging/d365/stg_d365__partnerpaymenttransactions.sql
[0m20:50:44 1605: jinja rendering because of STATIC_PARSER flag. file: staging/d365/stg_d365__customernames.sql
[0m20:50:44 1605: jinja rendering because of STATIC_PARSER flag. file: staging/d365/stg_d365__customercreditcards.sql
[0m20:50:44 1605: jinja rendering because of STATIC_PARSER flag. file: staging/d365/stg_d365__customerinvoicelines.sql
[0m20:50:44 1605: jinja rendering because of STATIC_PARSER flag. file: staging/d365/stg_d365__subscriptionscheduleline.sql
[0m20:50:44 1605: jinja rendering because of STATIC_PARSER flag. file: staging/d365/stg_d365__customergroups.sql
[0m20:50:44 1605: jinja rendering because of STATIC_PARSER flag. file: staging/d365/stg_d365__customerinvoiceheader.sql
[0m20:50:44 1605: jinja rendering because of STATIC_PARSER flag. file: staging/d365/stg_d365__customeraddresses.sql
[0m20:50:44 1605: jinja rendering because of STATIC_PARSER flag. file: staging/d365/stg_d365__products.sql
[0m20:50:44 1605: jinja rendering because of STATIC_PARSER flag. file: staging/dates/dates.sql
[0m20:50:44 1605: jinja rendering because of STATIC_PARSER flag. file: marts/snow_mart/dim_dates.sql
[0m20:50:44 [WARNING]: Did not find matching node for patch with name 'stg_d365__partnertransactions' in the 'models' section of file 'models/staging/d365/_d365_models.yml'
[CT-2931]: https://dbtlabs.atlassian.net/browse/CT-2931?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
|
Thanks for including the log from the console. I didn't see the Do you have access to |
I have pasted the stack trace here:
Traceback (most recent call last):
File "C:\Users\Brian.Custer\source\repos\ds_snow\snow_venv\lib\site-packages\dbt\cli\requires.py", line 87, in wrapper
result, success = func(*args, **kwargs)
File "C:\Users\Brian.Custer\source\repos\ds_snow\snow_venv\lib\site-packages\dbt\cli\requires.py", line 72, in wrapper
return func(*args, **kwargs)
File "C:\Users\Brian.Custer\source\repos\ds_snow\snow_venv\lib\site-packages\dbt\cli\requires.py", line 143, in wrapper
return func(*args, **kwargs)
File "C:\Users\Brian.Custer\source\repos\ds_snow\snow_venv\lib\site-packages\dbt\cli\requires.py", line 172, in wrapper
return func(*args, **kwargs)
File "C:\Users\Brian.Custer\source\repos\ds_snow\snow_venv\lib\site-packages\dbt\cli\requires.py", line 219, in wrapper
return func(*args, **kwargs)
File "C:\Users\Brian.Custer\source\repos\ds_snow\snow_venv\lib\site-packages\dbt\cli\requires.py", line 246, in wrapper
manifest = ManifestLoader.get_full_manifest(
File "C:\Users\Brian.Custer\source\repos\ds_snow\snow_venv\lib\site-packages\dbt\parser\manifest.py", line 306, in get_full_manifest
manifest = loader.load()
File "C:\Users\Brian.Custer\source\repos\ds_snow\snow_venv\lib\site-packages\dbt\parser\manifest.py", line 482, in load
self.parse_project(
File "C:\Users\Brian.Custer\source\repos\ds_snow\snow_venv\lib\site-packages\dbt\parser\manifest.py", line 656, in parse_project
parser.parse_file(block, dct=dct)
File "C:\Users\Brian.Custer\source\repos\ds_snow\snow_venv\lib\site-packages\dbt\parser\schemas.py", line 170, in parse_file
self.generic_test_parser.parse_versioned_tests(versioned_test_block)
File "C:\Users\Brian.Custer\source\repos\ds_snow\snow_venv\lib\site-packages\dbt\parser\schema_generic_tests.py", line 395, in parse_versioned_tests
self.parse_tests(block)
File "C:\Users\Brian.Custer\source\repos\ds_snow\snow_venv\lib\site-packages\dbt\parser\schema_generic_tests.py", line 388, in parse_tests
self.parse_column_tests(block, column, None)
File "C:\Users\Brian.Custer\source\repos\ds_snow\snow_venv\lib\site-packages\dbt\parser\schema_generic_tests.py", line 79, in parse_column_tests
self.parse_test(block, test, column, version)
File "C:\Users\Brian.Custer\source\repos\ds_snow\snow_venv\lib\site-packages\dbt\parser\schema_generic_tests.py", line 384, in parse_test
self.parse_node(block)
File "C:\Users\Brian.Custer\source\repos\ds_snow\snow_venv\lib\site-packages\dbt\parser\schema_generic_tests.py", line 322, in parse_node
node = self.parse_generic_test(
File "C:\Users\Brian.Custer\source\repos\ds_snow\snow_venv\lib\site-packages\dbt\parser\schema_generic_tests.py", line 158, in parse_generic_test
builder = TestBuilder(
File "C:\Users\Brian.Custer\source\repos\ds_snow\snow_venv\lib\site-packages\dbt\parser\generic_test_builders.py", line 186, in __init__
short_name, full_name = self.get_synthetic_test_names()
File "C:\Users\Brian.Custer\source\repos\ds_snow\snow_venv\lib\site-packages\dbt\parser\generic_test_builders.py", line 337, in get_synthetic_test_names
return synthesize_generic_test_names(name, target_name, self.args)
File "C:\Users\Brian.Custer\source\repos\ds_snow\snow_venv\lib\site-packages\dbt\parser\generic_test_builders.py", line 44, in synthesize_generic_test_names
for arg_name in sorted(args):
TypeError: '<' not supported between instances of 'str' and 'int'
Thanks,
Brian Custer
206-661-2674
…________________________________
From: Doug Beatty ***@***.***>
Sent: Thursday, August 3, 2023 2:18 PM
To: dbt-labs/dbt-core ***@***.***>
Cc: Brian Custer ***@***.***>; Author ***@***.***>
Subject: EXTERNAL - Re: [dbt-labs/dbt-core] [CT-2931] TypeError: > not supported between str and int when attempting to build a dbt project (Issue #8312)
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
Thanks for including the log from the console. I didn't see the TypeError in that output though -- was it missing once you used the dbt --log-level debug?
Do you have access to logs/dbt.log? If so, could you search for TypeError within that file to see if there's a Python stacktrace with file path or line numbers?
—
Reply to this email directly, view it on GitHub<#8312 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A3FCQ5BWMNAJ6XNTAODI4ELXTQIUHANCNFSM6AAAAAA3DHHX3A>.
You are receiving this because you authored the thread.Message ID: ***@***.***>
[CT-2931]: https://dbtlabs.atlassian.net/browse/CT-2931?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
|
Thanks for getting that stacetrace @brian-custer 👍 It looks like the Do you have any custom generic tests defined? Or are you using any of the four out-of-the-box tests ( |
@brian-custer Could you check your YAML files for any lines that start with an integer key? If so, then I think we've found the source of the issue within your project. Reprex ("reproducible example")I was able to get a
select 1 as id
models:
- name: my_model
columns:
- name: id
tests:
- accepted_values:
values: [1]
1: asdf Note the very last line in the YAML file that Could you see if you have anything that looks similar and try to remove it? |
No. I am using the standard tests as far as I know. I'll check and remove any unordinary tests.
Brian Custer
206-661-2674
…________________________________
From: Doug Beatty ***@***.***>
Sent: Thursday, August 3, 2023 3:11:43 PM
To: dbt-labs/dbt-core ***@***.***>
Cc: Brian Custer ***@***.***>; Mention ***@***.***>
Subject: EXTERNAL - Re: [dbt-labs/dbt-core] [CT-2931] TypeError: > not supported between str and int when attempting to build a dbt project (Issue #8312)
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
@brian-custer<https://github.com/brian-custer> Could you check your YAML files for any lines that start with an integer key? If so, then I think we've found the source of the issue within your project.
Reprex ("reproducible example")
I was able to get a TypeError: '<' not supported between instances of 'int' and 'str' error message when I with a simple setup like below and then running dbt test:
models/my_model.sql
select 1 as id
models/_models.yml
models:
- name: my_model
columns:
- name: id
tests:
- accepted_values:
values: [1]
1: asdf
Note the very last line in the YAML file that 1: asdf: it is completely extraneous, and everything works correctly if I remove it.
Could you see if you have anything that looks similar and try to remove it?
—
Reply to this email directly, view it on GitHub<#8312 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A3FCQ5BOVWYHHYNQQLPDSWDXTQOZ7ANCNFSM6AAAAAA3DHHX3A>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
[CT-2931]: https://dbtlabs.atlassian.net/browse/CT-2931?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
|
Whenever I replace the lists with a dictionary I get the error message I gave you. If I replace with a list for accepted_values I get the error Invalid test config given in models\staging\d365\_d365_models.yml:
test arguments must be a dict, got <class 'list'> (value [0, 1])
@: UnparsedModelUpdate(original_file_path='mode...ne)
What am I supposed to use for the tests?
Thanks,
Brian Custer
206-661-2674
…________________________________
From: Doug Beatty ***@***.***>
Sent: Thursday, August 3, 2023 3:11 PM
To: dbt-labs/dbt-core ***@***.***>
Cc: Brian Custer ***@***.***>; Mention ***@***.***>
Subject: EXTERNAL - Re: [dbt-labs/dbt-core] [CT-2931] TypeError: > not supported between str and int when attempting to build a dbt project (Issue #8312)
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
@brian-custer<https://github.com/brian-custer> Could you check your YAML files for any lines that start with an integer key? If so, then I think we've found the source of the issue within your project.
Reprex ("reproducible example")
I was able to get a TypeError: '<' not supported between instances of 'int' and 'str' error message when I with a simple setup like below and then running dbt test:
models/my_model.sql
select 1 as id
models/_models.yml
models:
- name: my_model
columns:
- name: id
tests:
- accepted_values:
values: [1]
1: asdf
Note the very last line in the YAML file that 1: asdf: it is completely extraneous, and everything works correctly if I remove it.
Could you see if you have anything that looks similar and try to remove it?
—
Reply to this email directly, view it on GitHub<#8312 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A3FCQ5BOVWYHHYNQQLPDSWDXTQOZ7ANCNFSM6AAAAAA3DHHX3A>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
[CT-2931]: https://dbtlabs.atlassian.net/browse/CT-2931?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
|
Could you share what you are using for your It should look something like this: models:
- name: orders
columns:
- name: status
tests:
- accepted_values:
values: ['placed', 'shipped', 'completed', 'returned']
- name: status_id
tests:
- accepted_values:
values: [1, 2, 3, 4]
# To test non-strings (like integers or boolean values) explicitly set the quote config to false.
quote: false |
I did that and now I am getting this error:
Parsing Error
Invalid test config given in models\staging\d365\_d365_models.yml:
test definition dictionary must have exactly one key, got [('accepted_values', None), ('values', [0, 1]), ('quote', False)] instead (3 keys)
@: UnparsedModelUpdate(original_file_path='mode...ne)
Thanks,
Brian Custer
206-661-2674
…________________________________
From: Doug Beatty ***@***.***>
Sent: Thursday, August 3, 2023 6:26 PM
To: dbt-labs/dbt-core ***@***.***>
Cc: Brian Custer ***@***.***>; Mention ***@***.***>
Subject: EXTERNAL - Re: [dbt-labs/dbt-core] [CT-2931] TypeError: > not supported between str and int when attempting to build a dbt project (Issue #8312)
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
Could you share what you are using for your accepted_values<https://docs.getdbt.com/reference/resource-properties/tests#accepted_values> configuration?
It should look something like this:
models:
- name: orders
columns:
- name: status
tests:
- accepted_values:
values: ['placed', 'shipped', 'completed', 'returned']
- name: status_id
tests:
- accepted_values:
values: [1, 2, 3, 4]
# To test non-strings (like integers or boolean values) explicitly set the quote config to false.
quote: false
—
Reply to this email directly, view it on GitHub<#8312 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A3FCQ5EE2TMQIRWMYXPDA3LXTRFTVANCNFSM6AAAAAA3DHHX3A>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
[CT-2931]: https://dbtlabs.atlassian.net/browse/CT-2931?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
|
Could you share what you are using for your accepted_values configuration? I'll be able to help you troubleshoot better if I can see your exact configuration rather than only an error message. I'm guessing you need to add some indentation in front of your e.g., This works: # This works:
tests:
- accepted_values:
values: [1, 2]
quote: false But this will give an error like you described: # This doesn't:
tests:
- accepted_values:
values: [1, 2]
quote: false |
I'm not encountering any more testing errors I'm just encountering the error:
SQLConnectionManager.get_result_from_cursor() missing 1 required positional argument: 'limit'
It appears to be coming from Databricks. What does this error mean and how do I fix it?
Thanks,
Brian Custer
206-661-2674
…________________________________
From: Doug Beatty ***@***.***>
Sent: Friday, August 4, 2023 6:51 AM
To: dbt-labs/dbt-core ***@***.***>
Cc: Brian Custer ***@***.***>; Mention ***@***.***>
Subject: EXTERNAL - Re: [dbt-labs/dbt-core] [CT-2931] TypeError: > not supported between str and int when attempting to build a dbt project (Issue #8312)
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
Could you share what you are using for your accepted_values<https://docs.getdbt.com/reference/resource-properties/tests#accepted_values> configuration? I'll be able to help you troubleshoot better if I can see your exact configuration rather than only an error message.
I'm guessing you need to add some indentation in front of your values and quote keys.
e.g., This works:
# This works:
tests:
- accepted_values:
values: [1, 2]
quote: false
But this will give an error like you described:
# This doesn't:
tests:
- accepted_values:
values: [1, 2]
quote: false
—
Reply to this email directly, view it on GitHub<#8312 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A3FCQ5GNVTCP64JOYHKEE4LXTT443ANCNFSM6AAAAAA3DHHX3A>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
[CT-2931]: https://dbtlabs.atlassian.net/browse/CT-2931?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
|
That's good news @brian-custer ! I'm going to close out this original bug report as resolved then. We use GitHub issues to focus on bug reports and feature requests, so definitely open up new issues as needed!
We use the dbt Community Forum for questions like "I've hit this error and am stuck", so I'm going to redirect you there instead for figuring out the new error you are running into. Providing code examples along with the error message will help others reproduce the error and spot what the cause might be. This guide on creating reproducible examples is specific to the R community, but has great general advice too. 🧠 |
Is this a new bug in dbt-core?
Current Behavior
I have a dbt project with numerous staging files and whenever I attempt to build the project I get the sam error as described. It appears to be a bug because the specific error is a python error and not a dbt error.
Expected Behavior
dbt builds without issue or the issue is described in the error message enabling me to respond.
Steps To Reproduce
Relevant log output
Environment
Which database adapter are you using with dbt?
other (mention it in "Additional Context")
Additional Context
dbt-databricks
The text was updated successfully, but these errors were encountered: