Cannot connect to externally running webdriver #519
Comments
|
Hi @DonCziken, thanks for raising an issue, and also thanks for being so detailed. |
|
Ok, here's me trying this out. I happen to be on Pop!_OS, which based on Ubuntu, so similar to your OS. Personally, I use sdkman, so I'll use it to switch to a Java version that matches yours: > sdk use java 8.0.352-temI'll fire up that docker image, like you did: > docker run -d -p 4444:4444 -p 7900:7900 --shm-size="2g" selenium/standalone-chrome:4.8.0-20230131From a new empty directory, I'll create a {:deps {org.clojure/clojure {:mvn/version "1.9.0"}
etaoin/etaoin {:mvn/version "1.0.39"}}}Now I'll fire up a REPL and test. ❯ clj
Clojure 1.9.0
user=> (require '[etaoin.api :as e])
nil
user=> (def driver (e/chrome {:webdriver-url "http://localhost:4444" :capabilities {"chromeOptions" {"args" ["--no-sandbox"]}}}))
#'user/driver
user=> (e/go driver "https://en.wikipedia.org/")
{:state "success", :sessionId nil, :class "org.openqa.selenium.remote.Response", :value nil, :status 0}
user=> (e/get-url driver)
"https://en.wikipedia.org/wiki/Main_Page"
user=> (e/get-title driver)
"Wikipedia, the free encyclopedia"
user=> (e/quit driver)
{:type :chrome, :host "127.0.0.1", :port 42615, :url "http://127.0.0.1:42615", :locator "xpath", :webdriver-url "http://localhost:4444", :capabilities {"chromeOptions" {"args" ["--no-sandbox"]}}}
user=>So from an initial look-see, seems to be ok. |
I think this is a good approach. |
|
I've tested as you suggested using REPL and it have worked like charm (same as for you). However to try to do it in REPL I had to I had to install clojure locally following: https://clojure.org/guides/install_clojure. The reason is that the build for the project is based on boot and it seems I didn't need local clojure to be able to build it with it. I guess thats something ot replace as well, as it seems the tool haven't been developed since mid 2019 :[. Anyway, my next guess was maybe its issue with Boot indeed, however you've asksed:
and it indeed it was a right question to ask, as following is how the driver is managed (just learned that mount is a global state manager) (ns coretime-sync.sage.driver
(:require [mount.core :refer [defstate]]
[etaoin.api :as e]))
(defstate driver
:start (e/chrome {:webdriver-url "http://localhost:4444" :capabilities {"chromeOptions" {"args" ["--no-sandbox"]}}})
:stop (e/quit driver))So I decided to keep playing with REPL further, to use the mount as maybe it was the problem, so I've added following into deps with same version as is in the project and run following with REPL (took me some time, as was doing :require instead of require... and couldn't figure out why it doesn't work ;)) Clojure 1.9.0
user=> (require '[mount.core :refer [defstate]] '[etaoin.api :as e] '[mount.core :as m])
nil
user=> (defstate driver :start (e/chrome {:webdriver-url "http://localhost:4444" :capabilities {"chromeOptions" {"args" ["--no-sandbox"]}}}) :stop (e/quit driver))
#'user/driver
user=> (m/start)
SocketTimeoutException Read timed out java.net.SocketInputStream.socketRead0 (SocketInputStream.java:-2)So as you can see, boom when using mount I am receiving Socket Ex, but I've checked that there is a newer version of it, so I've updated the deps for REPL: and BINGO, it worked! Clojure 1.9.0
user=> (require '[mount.core :refer [defstate]] '[etaoin.api :as e] '[mount.core :as m])
nil
user=> (defstate driver :start (e/chrome {:webdriver-url "http://localhost:4444" :capabilities {"chromeOptions" {"args" ["--no-sandbox"]}}}) :stop (e/quit driver))
#'user/driver
user=> (m/start)
{:started ["#'user/driver"]}
user=> (e/go driver "https://en.wikipedia.org/")
{:state "success", :sessionId nil, :class "org.openqa.selenium.remote.Response", :value nil, :status 0}The final test was with the project itself, I've bumped mount to latest version and yeah indeed manage to connect, however found further issue: #520 However in regards to this thread there is one more bit. I've tested connecting to webdriver without providing cziken@cziken-ThinkPad-T540p:~/Documents/repositories/iterative/clj-test$ clj
Clojure 1.9.0
user=> (require '[mount.core :refer [defstate]] '[etaoin.api :as e] '[mount.core :as m])
nil
user=> (defstate driver :start (e/chrome {:webdriver-url "http://localhost:4444"}) :stop (e/quit driver))
#'user/driver
user=> (m/start)
SocketTimeoutException Read timed out java.net.SocketInputStream.socketRead0 (SocketInputStream.java:-2) |
|
Well good for you @DonCziken, it seems you have solved some mysteries!
Yes, these days boot is not a popular choice. I don't see an Etaoin issue here, so I'm going to close this one and move on to #520. |
|
Just noticed:
Different WebDrivers have different options. |
|
Ok, but I would say it is not at all intuitive, that without that option lib fails with socket timeout exception - I would consider possibly some improvement on handling the error, especially that python library (from my 1st message) has connected without any issue (I only specified that its a chrome driver like I wanted to do here). |
Agreed, but Etaoin is just a very thin wrapper over WebDriver implementations. The Python example uses Selenium which is a thicker wrapper. But.. like you, I am kinda curious how the python example worked without the extra args. |
|
Ok. Sooo... here's what I've learned. To get juicy details of what is going on when interacting with the Selenium Grid docker container, I start it like so in its own terminal window: > docker run -it -p 4444:4444 -p 7900:7900 --shm-size="2g" selenium/standalone-chrome:4.8.0-20230131This allows me to see what is being logged by the container. Now when I try your python script, I see the following logged: Now I'll plunk our REPL session into a (require '[etaoin.api :as e])
(def driver (e/chrome {:webdriver-url "http://localhost:4444"
:capabilities {"chromeOptions" {"args" ["--no-sandbox"]}}}))
(e/go driver "https://en.wikipedia.org/")
(println "url:"(e/get-url driver))
(println "title:" (e/get-title driver))
(e/quit driver)Let's run that: From the container terminal window I see: Ok. So the fact that we are passing Let's edit our (require '[etaoin.api :as e])
(def driver (e/chrome {:webdriver-url "http://localhost:4444"
:capabilities {:browserName "chrome"}}))
(e/go driver "https://en.wikipedia.org/")
(println "url:"(e/get-url driver))
(println "title:" (e/get-title driver))
(e/quit driver)When I rerun, it works as above, and we see the following in the container window: Conclusion/guesses:
This is all perhaps interesting info for #378. Gonna re-close this one, can re-open if we want to explore more. |
Version
[etaoin "1.0.39"]
Platform
Operating System: Ubuntu 18.04.6 LTS
Clojure version: org.clojure/clojure "1.9.0"
JDK vendor and version: OpenJDK Runtime Environment (build 1.8.0_352-8u352-ga-1~18.04-b08)
Browser vendor: chrome
WebDriver version: https://github.com/SeleniumHQ/docker-selenium / selenium/standalone-chrome:4.8.0-20230131
Symptom
Not able to connect to running webdriver via:
:start (e/chrome {:webdriver-url "http://localhost:4444" :capabilities {"chromeOptions" {"args" ["--no-sandbox"]}}})exiting with following stacktrace:
Reproduction
run:
docker run -d -p 4444:4444 -p 7900:7900 --shm-size="2g" selenium/standalone-chrome:4.8.0-20230131then
Expected behavior
To connect to wb (as when used without params works as charm and spawns new local wd process)
Diagnosis
Not sure. The only clue I have is that maybe there is issue within JDK and http-client (as its quite old), but its component which was there for much longer and worked ok...
Action
Looking for help how to approach that.
DISCLAIMER
I am not proficient with Clojure. This is an old piece of code which we use to do some recurring task. It's been developed by different developer which is no longer with us and I am just maintaining it. Recently I had upgraded etaoin lib, as there were some issues with execution due to changes of UI. It solved the problem, but now decided to trying to convert from using 'local' chromdriver towards web one, which would help out with using the tool for us greatly.
I could try bumping JDK or clojure version. However I expect this would mean quite a bit of rewriting and fixing of dependencies - so before I do that (which maybe the cause) I wanted to check if the above issue isn't related strictly with etaoin lib.
Note that I've tested webdriver with following python code and it worked ok:
The text was updated successfully, but these errors were encountered: