File upload editor is failing while uploading a file

[ImranSk3]

Hi Team

I had done set up for Sunbird File upload Editor following below link on my local Ubuntu machine
https://knowlg.sunbird.org/use/installation-guide/editors/file-upload-editor

While uploading a file it throws the below error

Please help me to solve this error

[pallakartheekreddy]

@ImranSk3 Can you provide more details like Share the config what you're passing to the editor and host what you are pointing to create the contnet?

[ImranSk3]

@pallakartheekreddy please find the following configuration details


So we are using sunbird knowlg BB to upload the content .By using the content service API from postman we can able to create,upload and read the content successfully. so we are using file upload editor to upload the content but I am getting below issue.

please help me to solve this error

[ImranSk3]

@pallakartheekreddy I got the issue for metadata channel not set ,we need to send the X-Channel-Id header in the request headers currently no such header is passing in the request header. So, please guide us how to add that required header in the request.

[pallakartheekreddy]

@ImranSk3 first, please create a channel using the channel API. Then pass that channel ID like below. Please add below line in app.js
proxyReqOpts.headers['X-Channel-Id'] = "<<CHANNEL_ID>>";

Channel API:

curl --location 'https://dev.sunbirded.org/channel/v3/create' \
--header 'Content-Type: application/json' \
--header 'Authorization: Beare {{api_key}}' \
--data '{
    "request": {
        "channel": {
            "name": "ChannelName",
            "description": "Channel Description",
            "code": "UUID"
        }
    }
}'

[ImranSk3]

@pallakartheekreddy ,I had added the X-channel-id header in app.js ,now the following request is failing



I tried with the postman by copying same curl by changing knowledge platform content service port :9000 ,I am getting response with artifact url .So, please guide me how to solve this error.

[ImranSk3]

@pallakartheekreddy could you please help us on this issue.

[ImranSk3]

@pallakartheekreddy could you please help us on this issue.

[pallakartheekreddy]

@ImranSk3 Can we have a call to check this? Are you available today 3PM, if yes, please use the below link to connect
File upload editor issue
Monday, April 3 · 3:00 – 3:30pm
Google Meet joining info
Video call link: https://meet.google.com/kzm-egnj-zrv
Or dial: ‪(US) +1 314-474-2304‬ PIN: ‪531 599 409‬#
More phone numbers: https://tel.meet/kzm-egnj-zrv?pin=9596314215757

[ImranSk3]

@pallakartheekreddy Thank you, yes I am available,we can have a call today.

[pallakartheekreddy]

@ImranSk3 As discussed in the call, Please make your S3 bucket publicly accessible and check Because the upload API tries to download files to compute file size, and it is breaking there as the file is not accessible.

[ImranSk3]

@pallakartheekreddy Thank you so much for helping us out! After changing the S3 bucket to publicly accessible, content upload API works with editor. now I am facing following error to preview the content as shown in the below image. Also I'm am not aware what should we pass the build_number.

Branches we cloned:
Knowledge platform : release-4.8.0
File upload editor : release-5.0.0

attaching the logs from knowlg local instance(upload editor has no logs other than telemetry)

create...........Schema path: ../../schemas//content/1.0/schema.json
Config path: ../../schemas//content/1.0/config.json
Schema path: ../../schemas//objectcategorydefinition/1.0/schema.json
Config path: ../../schemas//objectcategorydefinition/1.0/config.json
2023-04-04 17:27:25,220 DataStax Java driver 3.7.0 for Apache Cassandra
2023-04-04 17:27:25,509 Detected Guava < 19 in the classpath, using legacy compatibility layer
2023-04-04 17:27:26,102 Using native clock to generate timestamps.
2023-04-04 17:27:26,318 Found Netty's native epoll transport in the classpath, using it
2023-04-04 17:27:27,270 Using data-center name 'datacenter1' for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct datacenter name with DCAwareRoundRobinPolicy constructor)
2023-04-04 17:27:27,273 New Cassandra host /127.0.0.1:9042 added
2023-04-04 17:27:27,613 {"eid":"LOG","ets":1680609447518,"ver":"3.0","mid":"LP.1680609447518.e33e311e-851c-4427-8c29-0995614e92bc","actor":{"id":"org.sunbird.learning.platform","type":"User"},"context":{"channel":"in.ekstep","pdata":{"id":"dev.ekstep.learning.platform","pid":"","ver":"1.0"},"env":"system"},"edata":{"level":"ERROR","type":"system","message":"Entry is not found in cassandra for content with identifier: obj-cat:learning-resource_content_tibil"},"syncts":1680609447518}
2023-04-04 17:27:28,310 Direct driver instance 1396327919 created for server address localhost:7687
2023-04-04 17:27:30,175 Direct driver instance 950378030 created for server address localhost:7687
2023-04-04 17:27:30,426 {"eid":"LOG","ets":1680609450426,"ver":"3.0","mid":"LP.1680609450426.2dc1bebc-56c2-4973-add1-a4614454e022","actor":{"id":"org.sunbird.learning.platform","type":"System"},"context":{"channel":"in.ekstep","pdata":{"id":"dev.ekstep.learning.platform","pid":"","ver":"1.0"},"env":"content"},"edata":{"level":"INFO","type":"api_access","message":"","params":[{"duration":7906},{"protocol":"http"},{"size":329},{"method":"POST"},{"uip":"127.0.0.1"},{"url":"/content/v3/create"},{"status":200}]},"syncts":1680609450426}
uploadPreSigned....................2023-04-04 17:27:36,723 {"eid":"LOG","ets":1680609456722,"ver":"3.0","mid":"LP.1680609456722.273fa663-8899-407f-b43b-40efaf6a791e","actor":{"id":"org.sunbird.learning.platform","type":"System"},"context":{"channel":"in.ekstep","pdata":{"id":"dev.ekstep.learning.platform","pid":"","ver":"1.0"},"env":"content"},"edata":{"level":"INFO","type":"api_access","message":"","params":[{"duration":5860},{"protocol":"http"},{"size":670},{"method":"POST"},{"uip":"127.0.0.1"},{"url":"/content/v3/upload/url/do_11376755259665612811"},{"status":200}]},"syncts":1680609456722}
upload....................2023-04-04 17:27:38,561 ProducerConfig values:
acks = 1
batch.size = 16384
bootstrap.servers = [localhost:9092]
buffer.memory = 33554432
client.id = KafkaClientProducer
compression.type = none
connections.max.idle.ms = 540000
enable.idempotence = false
interceptor.classes = null
key.serializer = class org.apache.kafka.common.serialization.LongSerializer
linger.ms = 0
max.block.ms = 60000
max.in.flight.requests.per.connection = 5
max.request.size = 1048576
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
receive.buffer.bytes = 32768
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 30000
retries = 0
retry.backoff.ms = 100
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
send.buffer.bytes = 131072
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
transaction.timeout.ms = 60000
transactional.id = nullthese endpoint proxies w
value.serializer = class org.apache.kafka.common.serialization.StringSerializer

2023-04-04 17:27:38,819 Kafka version : 1.0.0
2023-04-04 17:27:38,819 Kafka commitId : aaa7af6d4a11b29d
2023-04-04 17:27:38,905 ConsumerConfig values:
auto.commit.interval.ms = 5000
auto.offset.reset = latest
bootstrap.servers = [localhost:9092]
check.crcs = true
client.id = KafkaClientConsumer
connections.max.idle.ms = 540000
enable.auto.commit = true
exclude.internal.topics = true
fetch.max.bytes = 52428800
fetch.max.wait.ms = 500
fetch.min.bytes = 1
group.id =
heartbeat.interval.ms = 3000
interceptor.classes = null
internal.leave.group.on.close = true
isolation.level = read_uncommitted
key.deserializer = class org.apache.kafka.common.serialization.LongDeserializer
max.partition.fetch.bytes = 1048576
max.poll.interval.ms = 300000
max.poll.records = 500
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partition.assignment.strategy = [class org.apache.kafka.clients.consumer.RangeAssignor]
receive.buffer.bytes = 65536
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 305000
retry.backoff.ms = 100
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
send.buffer.bytes = 131072
session.timeout.ms = 10000
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
value.deserializer = class org.apache.kafka.common.serialization.StringDeserializer

2023-04-04 17:27:39,211 Kafka version : 1.0.0
2023-04-04 17:27:39,211 Kafka commitId : aaa7af6d4a11b29d
2023-04-04 17:27:41,797 {"eid":"LOG","ets":1680609461796,"ver":"3.0","mid":"LP.1680609461796.0fb059f6-bb49-4b2b-bda4-d2a643f40ba0","actor":{"id":"org.sunbird.learning.platform","type":"System"},"context":{"channel":"in.ekstep","pdata":{"id":"dev.ekstep.learning.platform","pid":"","ver":"1.0"},"env":"content"},"edata":{"level":"INFO","type":"api_access","message":"","params":[{"duration":3780},{"protocol":"http"},{"size":571},{"method":"POST"},{"uip":"127.0.0.1"},{"url":"/content/v3/upload/do_11376755259665612811"},{"status":200}]},"syncts":1680609461796}
2023-04-04 17:27:42,312 {"eid":"LOG","ets":1680609462312,"ver":"3.0","mid":"LP.1680609462312.c317ff64-4b0b-463c-bc3d-f431d790a7ee","actor":{"id":"org.sunbird.learning.platform","type":"User"},"context":{"channel":"in.ekstep","pdata":{"id":"dev.ekstep.learning.platform","pid":"","ver":"1.0"},"env":"system"},"edata":{"level":"ERROR","type":"system","message":"Entry is not found in cassandra for content with identifier: do_11376755259665612811"},"syncts":1680609462312}
2023-04-04 17:27:42,732 {"eid":"LOG","ets":1680609462732,"ver":"3.0","mid":"LP.1680609462732.4641087b-9bca-4642-8aed-bdaa86a5fe43","actor":{"id":"org.sunbird.learning.platform","type":"System"},"context":{"channel":"in.ekstep","pdata":{"id":"dev.ekstep.learning.platform","pid":"","ver":"1.0"},"env":"content"},"edata":{"level":"INFO","type":"api_access","message":"","params":[{"duration":656},{"protocol":"http"},{"size":964},{"method":"GET"},{"uip":"127.0.0.1"},{"url":"/content/v3/read/do_11376755259665612811?mode=edit&fields=body,collaborators,editorState,stageIcons,templateId,languageCode,template,gradeLevel,status,concepts,versionKey,name,appIcon,contentType,owner,domain,code,visibility,createdBy,description,language,mediaType,mimeType,osId,languageCode,createdOn,lastUpdatedOn,audience,ageGroup,attributions,artifactUrl,board,subject,keywords,config,resourceType,medium,publisher,year,pkgVersion,framework,rejectReasons,rejectComment,topic,ownedBy,ownershipType,creators,contributors,reservedDialcodes,qrCodeProcessId,channel,purpose,assets,assetsMap,copyright,author,copyrightYear,origin,license,displayScore,courseType,licenseterms,primaryCategory,additionalCategories,maxAttempts,verticals,programs"},{"status":200}]},"syncts":1680609462732}

Please help me how to resolve this error! Thanks in advance!

[ImranSk3]

@pallakartheekreddy could you please help us on this issue.

[pallakartheekreddy]

@ImranSk3 Can we have a call to discuss this issue?

[ImranSk3]

@pallakartheekreddy Sure, can u please suggest a time to connect!

[sravani-64]

@pallakartheekreddy Hi, we too got a similar error while saving and publishing the content. Please find the content-service logs as below

2023-06-06 09:24:46,953 {"eid":"LOG","ets":1686043486953,"ver":"3.0","mid":"LP.1686043486953.fb48e9e2-2d6d-41f4-a4cf-cd09b11f27d2","actor":{"id":"org.sunbird.learning.platform","type":"User"},"context":{"channel":"in.ekstep","pdata":{"id":"dev.ekstep.learning.platform","pid":"","ver":"1.0"},"env":"system"},"edata":{"level":"ERROR","type":"system","message":"Entry is not found in cassandra for content with identifier: do_1137837685170585601162"},"syncts":1686043486953}
2023-06-06 09:24:46,955 {"eid":"LOG","ets":1686043486955,"ver":"3.0","mid":"LP.1686043486955.c975f57d-b623-49fb-b10c-a9a978b76f66","actor":{"id":"org.sunbird.learning.platform","type":"System"},"context":{"channel":"in.ekstep","pdata":{"id":"dev.nulp.portal","pid":"","ver":"1.0"},"env":"content","sid":"kE_snajcmuS_erDwa5U68u0k-xMr1lbH","did":"228f5c78842b8eab28a9943a3caf5fe4"},"edata":{"level":"INFO","type":"api_access","message":"","params":[{"duration":45},{"protocol":"http"},{"size":1517},{"method":"GET"},{"uip":"10.10.2.186"},{"url":"/content/v3/read/do_1137837685170585601162?fields=ageGroup%2CappIcon%2CartifactUrl%2Cattributions%2Cattributions%2Caudience%2Cauthor%2CbadgeAssertions%2Cboard%2Cbody%2Cchannel%2Ccode%2Cconcepts%2CcontentCredits%2CcontentType%2Ccontributors%2Ccopyright%2CcopyrightYear%2CcreatedBy%2CcreatedOn%2Ccreator%2Ccreators%2Cdescription%2CdisplayScore%2Cdomain%2CeditorState%2CflagReasons%2CflaggedBy%2Cflags%2Cframework%2CgradeLevel%2Cidentifier%2CitemSetPreviewUrl%2Ckeywords%2Clanguage%2ClanguageCode%2ClastUpdatedOn%2Clicense%2CmediaType%2Cmedium%2CmimeType%2Cname%2CoriginData%2CosId%2Cowner%2CpkgVersion%2Cpublisher%2Cquestions%2CresourceType%2CscoreDisplayConfig%2Cstatus%2CstreamingUrl%2Csubject%2Ctemplate%2CtemplateId%2CtotalQuestions%2CtotalScore%2CversionKey%2Cvisibility%2Cyear%2CprimaryCategory%2CadditionalCategories%2CinterceptionPoints%2CinterceptionType&mode=edit&licenseDetails=name%2Cdescription%2Curl"},{"status":200}]},"syncts":1686043486955}
2023-06-06 09:24:53,599 {"eid":"LOG","ets":1686043493599,"ver":"3.0","mid":"LP.1686043493599.895c6df8-0985-4e76-bbff-2e37f36f9fd1","actor":{"id":"org.sunbird.learning.platform","type":"User"},"context":{"channel":"in.ekstep","pdata":{"id":"dev.ekstep.learning.platform","pid":"","ver":"1.0"},"env":"system"},"edata":{"level":"ERROR","type":"system","message":"Entry is not found in cassandra for content with identifier: do_1137837951656017921186"},"syncts":1686043493599}
2023-06-06 09:24:53,601 {"eid":"LOG","ets":1686043493601,"ver":"3.0","mid":"LP.1686043493601.fc448f7c-400d-4fd1-b1c8-9e8122fa5a13","actor":{"id":"org.sunbird.learning.platform","type":"System"},"context":{"channel":"in.ekstep","pdata":{"id":"dev.nulp.portal","pid":"","ver":"1.0"},"env":"content","sid":"YBWWPGbTihmwKfBOTz8dZHx4Lf-eCiUe","did":"daad42c70954e9eb1d320b48072436b2"},"edata":{"level":"INFO","type":"api_access","message":"","params":[{"duration":44},{"protocol":"http"},{"size":1467},{"method":"GET"},{"uip":"10.10.2.186"},{"url":"/content/v3/read/do_1137837951656017921186?fields=ageGroup%2CappIcon%2CartifactUrl%2Cattributions%2Cattributions%2Caudience%2Cauthor%2CbadgeAssertions%2Cboard%2Cbody%2Cchannel%2Ccode%2Cconcepts%2CcontentCredits%2CcontentType%2Ccontributors%2Ccopyright%2CcopyrightYear%2CcreatedBy%2CcreatedOn%2Ccreator%2Ccreators%2Cdescription%2CdisplayScore%2Cdomain%2CeditorState%2CflagReasons%2CflaggedBy%2Cflags%2Cframework%2CgradeLevel%2Cidentifier%2CitemSetPreviewUrl%2Ckeywords%2Clanguage%2ClanguageCode%2ClastUpdatedOn%2Clicense%2CmediaType%2Cmedium%2CmimeType%2Cname%2CoriginData%2CosId%2Cowner%2CpkgVersion%2Cpublisher%2Cquestions%2CresourceType%2CscoreDisplayConfig%2Cstatus%2CstreamingUrl%2Csubject%2Ctemplate%2CtemplateId%2CtotalQuestions%2CtotalScore%2CversionKey%2Cvisibility%2Cyear%2CprimaryCategory%2CadditionalCategories%2CinterceptionPoints%2CinterceptionType&mode=edit&licenseDetails=name%2Cdescription%2Curl"},{"status":200}]},"syncts":1686043493601}
2023-06-06 09:24:55,875 {"eid":"LOG","ets":1686043495875,"ver":"3.0","mid":"LP.1686043495875.d0854a29-ed2c-468d-8103-f40b76c9197d","actor":{"id":"org.sunbird.learning.platform","type":"System"},"context":{"channel":"in.ekstep","pdata":{"id":"dev.nulp.portal","pid":"","ver":"1.0"},"env":"content","sid":"P3Y9h9Jqlz6NVKpHfDXCU9YMOsWNiZ39","did":"256400c3859c78e0730d0359e8c47c0e"},"edata":{"level":"INFO","type":"api_access","message":"","params":[{"duration":42},{"protocol":"http"},{"size":583},{"method":"GET"},{"uip":"10.10.2.186"},{"url":"/content/v3/read/do_11381201846435840012?fields=createdBy%2Cstatus%2CmimeType%2CcontentType%2CresourceType%2Ccollaborators%2CcontentDisposition%2CprimaryCategory%2Cframework%2CtargetFWIds&mode=edit"},{"status":200}]},"syncts":1686043495875}
2023-06-06 09:24:56,246 {"eid":"LOG","ets":1686043496246,"ver":"3.0","mid":"LP.1686043496246.bb61bc08-96fb-4027-b682-7101c3ba5c19","actor":{"id":"org.sunbird.learning.platform","type":"System"},"context":{"channel":"in.ekstep","pdata":{"id":"dev.nulp.portal","pid":"","ver":"1.0"},"env":"content","sid":"P3Y9h9Jqlz6NVKpHfDXCU9YMOsWNiZ39","did":"256400c3859c78e0730d0359e8c47c0e"},"edata":{"level":"INFO","type":"api_access","message":"","params":[{"duration":45},{"protocol":"http"},{"size":3375},{"method":"GET"},{"uip":"10.10.2.186"},{"url":"/content/v3/hierarchy/do_11381201846435840012?mode=edit"},{"status":200}]},"syncts":1686043496246}
2023-06-06 09:24:58,522 {"eid":"LOG","ets":1686043498522,"ver":"3.0","mid":"LP.1686043498522.952f5b09-865c-46a7-af69-75809d07b579","actor":{"id":"org.sunbird.learning.platform","type":"User"},"context":{"channel":"in.ekstep","pdata":{"id":"dev.ekstep.learning.platform","pid":"","ver":"1.0"},"env":"system"},"edata":{"level":"ERROR","type":"system","message":"Entry is not found in cassandra for content with identifier: do_1137837685170585601162"},"syncts":1686043498522}
2023-06-06 09:24:58,524 {"eid":"LOG","ets":1686043498524,"ver":"3.0","mid":"LP.1686043498524.7aa162e0-49ef-4c13-a2c7-401c84daef3f","actor":{"id":"org.sunbird.learning.platform","type":"System"},"context":{"channel":"in.ekstep","pdata":{"id":"dev.nulp.portal","pid":"","ver":"1.0"},"env":"content","sid":"btdwQfMij7kvBF0XzZpi-MBm60ko-uO8","did":"a728032d93283c004c8c1ee831f20ddb"},"edata":{"level":"INFO","type":"api_access","message":"","params":[{"duration":46},{"protocol":"http"},{"size":1517},{"method":"GET"},{"uip":"10.10.2.186"},{"url":"/content/v3/read/do_1137837685170585601162?fields=ageGroup%2CappIcon%2CartifactUrl%2Cattributions%2Cattributions%2Caudience%2Cauthor%2CbadgeAssertions%2Cboard%2Cbody%2Cchannel%2Ccode%2Cconcepts%2CcontentCredits%2CcontentType%2Ccontributors%2Ccopyright%2CcopyrightYear%2CcreatedBy%2CcreatedOn%2Ccreator%2Ccreators%2Cdescription%2CdisplayScore%2Cdomain%2CeditorState%2CflagReasons%2CflaggedBy%2Cflags%2Cframework%2CgradeLevel%2Cidentifier%2CitemSetPreviewUrl%2Ckeywords%2Clanguage%2ClanguageCode%2ClastUpdatedOn%2Clicense%2CmediaType%2Cmedium%2CmimeType%2Cname%2CoriginData%2CosId%2Cowner%2CpkgVersion%2Cpublisher%2Cquestions%2CresourceType%2CscoreDisplayConfig%2Cstatus%2CstreamingUrl%2Csubject%2Ctemplate%2CtemplateId%2CtotalQuestions%2CtotalScore%2CversionKey%2Cvisibility%2Cyear%2CprimaryCategory%2CadditionalCategories%2CinterceptionPoints%2CinterceptionType&mode=edit&licenseDetails=name%2Cdescription%2Curl"},{"status":200}]},"syncts":1686043498524}

No API failures are found. Could You pls help us to fix this? Thank you

[pallakartheekreddy]

@sravani-64 Which version of Knowlg you are using?

I see some error in above log Entry is not found in cassandra for content with identifier: do_1137837685170585601162. Are you trying to publish content or collection?

[sravani-64]

Hi @pallakartheekreddy,
As per our discussion in call, Prasath has shared an email with Samza node-manager, resource-manager and content-service logs. Please let us know how can we proceed further.
Thank you

[sravani-64]

@pallakartheekreddy , the issue is resolved by deleting /tmp folder and samza-jobs folder in Yarn VM and redeploying samza-jobs. Publish is working fine now. Thank you