[BUG] XQSuite test passes locally but fails in CI #649

joewiz · 2021-03-24T15:39:30Z

Describe the bug

The XQSuite test passes locally for me but is failing in CI. The mocha test runner files were generated by generator-exist and otherwise completely unchanged. Original: xqSuite.js from generator-exist package. My app's version: xqSuite.js. Here’s the failure in CI. From the CI logs, it looks like the request to /exist/rest/db/apps/airlock/test/xqs/test-runner.xq is returning invalid JSON, whereas on my machine this request is returning valid JSON and passing as expected.

Expected behavior

I'd expect a test that passes locally to also pass in CI.

To Reproduce

See the mocha test at https://github.com/joewiz/airlock/blob/master/test/xqs/xqSuite.js.

Running this locally, I get the following successful results:

joe@choskimac-iii airlock % npm test

> airlock@2.0.2 test
> mocha test/mocha/ --recursive --exit && mocha test/xqs/*.js



  file system checks
    markup files are well-formed
      ✓ *.html is xhtml
      ✓ *.xml
      ✓ *.xconf
      ✓ *.odd
    Consistent data in aux files
      ✓ should contain identical descriptions
      ✓ should contain identical versions
      ✓ should contain identical licenses
      ✓ should contain identical titles
      ✓ Readme should have latest meta-data

  rest api returns
    ✓ 404 from random page
    ✓ 200 from default rest endpoint
    - application root is available from rest endpoint


  11 passing (59ms)
  1 pending



  0 passing (1ms)



  Xqsuite tests for http://joewiz.org/ns/app/airlock/tests
    ✓ Test: one-is-one


  1 passing (2ms)

See the logs showing the CI failure at https://github.com/joewiz/airlock/runs/2185512671?check_suite_focus=true:

  npm test
  shell: /usr/bin/bash -e {0}
  env:
    JAVA_HOME_8.0.282_x64: /opt/hostedtoolcache/jdk/8.0.282/x64
    JAVA_HOME: /opt/hostedtoolcache/jdk/8.0.282/x64
    JAVA_HOME_8_0_282_X64: /opt/hostedtoolcache/jdk/8.0.282/x64

> airlock@2.0.2 test /home/runner/work/airlock/airlock
> mocha test/mocha/ --recursive --exit && mocha test/xqs/*.js



  file system checks
    markup files are well-formed
      ✓ *.html is xhtml
      ✓ *.xml
      ✓ *.xconf
      ✓ *.odd
    Consistent data in aux files
      ✓ should contain identical descriptions
      ✓ should contain identical versions
      ✓ should contain identical licenses
      ✓ should contain identical titles
      ✓ Readme should have latest meta-data

  rest api returns
    ✓ 404 from random page (139ms)
    ✓ 200 from default rest endpoint (75ms)
    - application root is available from rest endpoint


  11 passing (297ms)
  1 pending



  0 passing (1ms)

/home/runner/work/airlock/airlock/node_modules/mocha/lib/runner.js:962
    throw err;
    ^

SyntaxError: Unexpected token < in JSON at position 0
    at JSON.parse (<anonymous>)
    at IncomingMessage.<anonymous> (/home/runner/work/airlock/airlock/test/xqs/xqSuite.js:22:26)
    at IncomingMessage.emit (events.js:327:22)
    at endReadableNT (internal/streams/readable.js:1327:12)
    at processTicksAndRejections (internal/process/task_queues.js:80:21)
npm ERR! Test failed.  See above for more details.

Context (please always complete the following information):

OS: macOS 11.2.3
eXist-db Version: eXist 5.3.0-SNAPSHOT 987e49277047e13b696cffb3bc4d8a214ca11b67 20210322230617
Java Version: openjdk version "1.8.0_282" (Liberica OpenJDK)
App Version: I generated my app's scaffolding with generator-exist on 11 Feb 2021.

Additional context

How is eXist-db installed? built from source
Any custom changes in e.g. conf.xml? none

The text was updated successfully, but these errors were encountered:

problem reported at eXist-db/generator-exist#649

joewiz · 2021-03-24T21:20:26Z

I think I'm CI-cursed. After commenting out the xqsuite test, CI is now showing a new failure (though no such error appears on my local system):

  npm run cypress
  shell: /usr/bin/bash -e {0}
  env:
    JAVA_HOME_8.0.282_x64: /opt/hostedtoolcache/jdk/8.0.282/x64
    JAVA_HOME: /opt/hostedtoolcache/jdk/8.0.282/x64
    JAVA_HOME_8_0_282_X64: /opt/hostedtoolcache/jdk/8.0.282/x64

> airlock@2.0.2 cypress /home/runner/work/airlock/airlock
> cypress run

It looks like this is your first time using Cypress: 6.4.0

25l[20:58:53]  Verifying Cypress can run /home/runner/.cache/Cypress/6.4.0/Cypress [started]
[20:58:55]  Verifying Cypress can run /home/runner/.cache/Cypress/6.4.0/Cypress [completed]
25h
Opening Cypress...

====================================================================================================
tput: No value for $TERM and no -T specified

  (Run Starting)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Cypress:    6.4.0                                                                              │
  │ Browser:    Electron 87 (headless)                                                             │
  │ Specs:      1 found (landing_spec.js)                                                          │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘


────────────────────────────────────────────────────────────────────────────────────────────────────
                                                                                                    
  Running:  landing_spec.js                                                                 (1 of 1)


  The landing page
    1) should load 


  0 passing (246ms)
  1 failing

  1) The landing page
       should load :
     CypressError: `cy.visit()` failed trying to load:

http://localhost:8080/exist/apps/airlock/

The response we received from your web server was:

  > 404: Not Found

This was considered a failure because the status code was not `2xx`.

If you do not want status codes to cause failures pass the option: `failOnStatusCode: false`
      at http://localhost:8080/__cypress/runner/cypress_runner.js:158226:25
      at visitFailedByErr (http://localhost:8080/__cypress/runner/cypress_runner.js:157602:12)
      at http://localhost:8080/__cypress/runner/cypress_runner.js:158206:13
      at tryCatcher (http://localhost:8080/__cypress/runner/cypress_runner.js:10609:23)
      at Promise._settlePromiseFromHandler (http://localhost:8080/__cypress/runner/cypress_runner.js:8544:31)
      at Promise._settlePromise (http://localhost:8080/__cypress/runner/cypress_runner.js:8601:18)
      at Promise._settlePromise0 (http://localhost:8080/__cypress/runner/cypress_runner.js:8646:10)
      at Promise._settlePromises (http://localhost:8080/__cypress/runner/cypress_runner.js:8722:18)
      at _drainQueueStep (http://localhost:8080/__cypress/runner/cypress_runner.js:5316:12)
      at _drainQueue (http://localhost:8080/__cypress/runner/cypress_runner.js:5309:9)
      at Async.../../node_modules/bluebird/js/release/async.js.Async._drainQueues (http://localhost:8080/__cypress/runner/cypress_runner.js:5325:5)
      at Async.drainQueues (http://localhost:8080/__cypress/runner/cypress_runner.js:5195:14)
  From Your Spec Code:
      at Context.eval (http://localhost:8080/__cypress/tests?p=test/cypress/integration/landing_spec.js:102:8)




  (Results)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Tests:        1                                                                                │
  │ Passing:      0                                                                                │
  │ Failing:      1                                                                                │
  │ Pending:      0                                                                                │
  │ Skipped:      0                                                                                │
  │ Screenshots:  1                                                                                │
  │ Video:        true                                                                             │
  │ Duration:     0 seconds                                                                        │
  │ Spec Ran:     landing_spec.js                                                                  │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘


  (Screenshots)

  -  /home/runner/work/airlock/airlock/reports/screenshots/landing_spec.js/The landin     (1280x720)
     g page -- should load (failed).png                                                             


  (Video)

  -  Started processing:  Compressing to 32 CRF                                                     
  -  Finished processing: /home/runner/work/airlock/airlock/reports/videos/landing_sp    (0 seconds)
                          ec.js.mp4                                                                 


tput: No value for $TERM and no -T specified
====================================================================================================

  (Run Finished)


       Spec                                              Tests  Passing  Failing  Pending  Skipped  
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ✖  landing_spec.js                          234ms        1        -        1        -        - │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ✖  1 of 1 failed (100%)                     234ms        1        -        1        -        -  

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! airlock@2.0.2 cypress: `cypress run`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the airlock@2.0.2 cypress script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/runner/.npm/_logs/2021-03-24T20_59_07_630Z-debug.log
25h
Error: Process completed with exit code 1.

For cypress to get a 404 for http://localhost:8080/exist/apps/airlock/, the package being tested must not be installed, right? Why would that be? This is very puzzling.

In contrast, here's the passing test on my local machine:

joe@choskimac-iii airlock % npm run cypress

> airlock@2.0.2 cypress
> cypress run


====================================================================================================

  (Run Starting)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Cypress:    6.4.0                                                                              │
  │ Browser:    Electron 87 (headless)                                                             │
  │ Specs:      1 found (landing_spec.js)                                                          │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘


────────────────────────────────────────────────────────────────────────────────────────────────────
                                                                                                    
  Running:  landing_spec.js                                                                 (1 of 1)


  The landing page
    ✓ should load  (324ms)


  1 passing (338ms)


  (Results)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Tests:        1                                                                                │
  │ Passing:      1                                                                                │
  │ Failing:      0                                                                                │
  │ Pending:      0                                                                                │
  │ Skipped:      0                                                                                │
  │ Screenshots:  0                                                                                │
  │ Video:        true                                                                             │
  │ Duration:     0 seconds                                                                        │
  │ Spec Ran:     landing_spec.js                                                                  │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘


  (Video)

  -  Started processing:  Compressing to 32 CRF                                                     
  -  Finished processing: /Users/joe/workspace/hsg-project/repos/airlock/reports/vide    (0 seconds)
                          os/landing_spec.js.mp4                                                    


====================================================================================================

  (Run Finished)


       Spec                                              Tests  Passing  Failing  Pending  Skipped  
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ✔  landing_spec.js                          335ms        1        1        -        -        - │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ✔  All specs passed!                        335ms        1        1        -        -        -

Reported at eXist-db/generator-exist#649 (comment)

eXist-db/generator-exist#649 (comment)

joewiz added a commit to joewiz/airlock that referenced this issue Mar 24, 2021

disable xqsuite test; it’s a dummy & fails in CI

1985a8f

problem reported at eXist-db/generator-exist#649

joewiz added a commit to joewiz/airlock that referenced this issue Mar 24, 2021

disable cypress; it passes locally but fails in CI

440d6d7

Reported at eXist-db/generator-exist#649 (comment)

joewiz added a commit to joewiz/airlock that referenced this issue Mar 24, 2021

really disable cypress

0bc0788

eXist-db/generator-exist#649 (comment)

duncdrum mentioned this issue Jul 2, 2021

Next major version release #701

Merged

duncdrum mentioned this issue Apr 26, 2022

minor enhancements #801

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG] XQSuite test passes locally but fails in CI #649

[BUG] XQSuite test passes locally but fails in CI #649

joewiz commented Mar 24, 2021

joewiz commented Mar 24, 2021 •

edited

Loading

[BUG] XQSuite test passes locally but fails in CI #649

[BUG] XQSuite test passes locally but fails in CI #649

Comments

joewiz commented Mar 24, 2021

joewiz commented Mar 24, 2021 • edited Loading

joewiz commented Mar 24, 2021 •

edited

Loading