You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The GoogleCloudStorageDownloadOperator (used by the CompareXComMapsOperator in wordcount_test.py) works with Python 2.7 but raises a TypeError: Object of type 'bytes' is not JSON serializable when running in Python 3.6.
The error is easily reproducible in the below Cloud Composer Environment:
Composer version:1.16.8
Airflow version:1.10.15
Python version:3.6
Error details:
Traceback (most recent call last)
File "/usr/local/lib/airflow/airflow/models/taskinstance.py", line 985, in _run_raw_tas
result = task_copy.execute(context=context
File "/usr/local/lib/airflow/airflow/contrib/operators/gcs_download_operator.py", line 96, in execut
context['ti'].xcom_push(key=self.store_to_xcom_key, value=file_bytes
File "/usr/local/lib/airflow/airflow/models/taskinstance.py", line 1516, in xcom_pus
execution_date=execution_date or self.execution_date
File "/usr/local/lib/airflow/airflow/utils/db.py", line 74, in wrappe
return func(*args, **kwargs
File "/usr/local/lib/airflow/airflow/models/xcom.py", line 101, in se
value = XCom.serialize_value(value
File "/usr/local/lib/airflow/airflow/models/xcom.py", line 224, in serialize_valu
return json.dumps(value).encode('UTF-8'
File "/opt/python3.6/lib/python3.6/json/__init__.py", line 231, in dump
return _default_encoder.encode(obj
File "/opt/python3.6/lib/python3.6/json/encoder.py", line 199, in encod
chunks = self.iterencode(o, _one_shot=True
File "/opt/python3.6/lib/python3.6/json/encoder.py", line 257, in iterencod
return _iterencode(o, 0
File "/opt/python3.6/lib/python3.6/json/encoder.py", line 180, in defaul
o.__class__.__name__
TypeError: Object of type 'bytes' is not JSON serializabl
GoogleCloudStorageDownloadOperator is deprecated and probably needs to be replaced in order to fix the test workflow.
The text was updated successfully, but these errors were encountered:
The
GoogleCloudStorageDownloadOperator
(used by theCompareXComMapsOperator
inwordcount_test.py
) works withPython 2.7
but raises aTypeError: Object of type 'bytes' is not JSON serializable
when running inPython 3.6
.The error is easily reproducible in the below Cloud Composer Environment:
1.16.8
1.10.15
3.6
Error details:
GoogleCloudStorageDownloadOperator
is deprecated and probably needs to be replaced in order to fix the test workflow.The text was updated successfully, but these errors were encountered: