when using selenium(firefox) to connect to pywebview app, how to inject the pywebview JS part?

[justincui]

I can use the following code to pop up a selenium firefox windows and display the web page, but when I click button to show the file selection dialog, the dialog doesn't display (the file selection dialog is displayed from python side, which works properly when using pywebview).

From firefox debug console:

Uncaught (in promise) ReferenceError: pywebview is not defined
    browseFile http://127.0.0.1:8088/js/main.js:178
    browseYAMLFile http://127.0.0.1:8088/js/main.js:169
    onclick http://127.0.0.1:8088/app.html:1

The selenium code that starts a firfox window and connect to pywebview's internal http server:

from time import sleep
from selenium import webdriver

# Set up Firefox WebDriver
driver = webdriver.Firefox()

# Open the pywebview window (replace the URL with the correct address)
driver.get("http://127.0.0.1:8088/app.html")

# Perform your Selenium actions (e.g., find elements, interact with them)
sleep(100)
# Close the WebDriver
driver.quit()

The python code that starts pywebview:

window = webview.create_window('My App',
				url=start_page,
				js_api=api,
				confirm_close=True,
				text_select=True,
				width=1280,
				height=1060,
				min_size=(640, 530))
webview.start(prepare_window, window,
			  gui='qt',
			  http_server=True,
			  http_port=8088,
			  debug=True)

The js+python code that pops up the file selection dialog:

async function browseFile(target_elm_id, suffix){
    var target_elm = document.getElementById(target_elm_id);
    const response = await pywebview.api.open_file_dialog(suffix, hint=target_elm.value);
    var msg = response.message;
    if(msg){
        target_elm.value = msg;
        target_elm.scrollLeft = target_elm.scrollWidth;
    }
    return msg;
}

def open_file_dialog(self, suffix, hint=''):
        hint = str(hint).strip()
        status = self.check_file_stat(hint)
        init_dir = os.path.dirname(hint) if status['is_file'] else hint if status['is_dir'] else ''

        file_types = [suffix, 'All files (*)'] if suffix else ['All files (*)']
        result = self.window.create_file_dialog(webview.OPEN_DIALOG, allow_multiple=False, file_types=file_types, directory=init_dir)
        response = {
            'message': str(result[0]) if result else ''
        }
        return response

[r0x0r]

pywebview object is injected ahem by the pywebview code after page is loaded and is visible only to pywebview itself.
What is your end goal here? What are you trying to achieve with selenium?

[justincui]

I want to create automatic GUI regression tests. I plan to use selenium to operate the webkit and use cv2/mss/pyautogui to operate GUI components popped from python side like the file selector dialog. 99% tests are just in webkit. In fact, cv2/mss/pyautogui can be used to create any kind of GUI automatic scripts, but it's very cumbersome compared to selenium and not as stable.

[r0x0r]

Unfortunately I do not see any way to expose window.pywebview to Selenium. Technically you could mock the pywebview object, but the problem is that it uses renderer's specific functions that are only available inside renderer's webview. pyautogui seems like the only way at the moment.
5.0 will introduce DOM traversal and manipulation functionality. I have been considering adding functions like element.click() and element.text() too, so in the future pywebview may be a step closer to selenium.

[justincui]

It seems that selenium is an overkill for pywebview. I have an idea that may be simpler: add a WebSocket listener at python side for GUI testing code to access. The listener uses evaluate_js() to access the JS world. Is window.evaluate_js() thread-safe?

# the internal http server of pywebview   
        server.thread = threading.Thread(
            target=lambda: bottle.run(
                app=app, server=server_adapter, port=server.port, quiet=not _settings['debug']
            ),
            daemon=True,
        )
        server.thread.start()

According to "Thread Mode" in docs/guide/usage.md, it seems that "window.some_function()" is thread safe:

import webview

def custom_logic(window):
    window.toggle_fullscreen()
    window.evaluate_js('alert("Nice one brother")')

window = webview.create_window('Woah dude!', html='<h1>Woah dude!<h1>')
webview.start(custom_logic, window)
# anything below this line will be executed after program is finished executing
pass

[r0x0r]

This could work. What architecture/API would look like?

[justincui]

The follow are the websocket related code in my app. It's working for some simple test.

# python 3.6.8
import asyncio
import websockets
# (other imports are skipped)

##############################
# websocket server support in my app 
##############################

def on_main_window_closing(ws_server_thread):
    print('closing the window...')
    if ws_server_thread and ws_server_thread.is_alive():
        # Signal the server to stop when the web view window is closed
        ws_server_thread._loop.call_soon_threadsafe(ws_server_thread._stop_future.set_result, True)
        ws_server_thread.join()
        ws_server_thread._loop.close()

def prepare_window(window, ws_server_thread):
    window.events.shown += on_main_window_shown
    window.events.closing += lambda: on_main_window_closing(ws_server_thread)

def start_ws_server(ws_port, window):
    # Set up the asyncio event loop
    loop = asyncio.new_event_loop()
    asyncio.set_event_loop(loop)

    # Create a future to signal the server to stop
    stop = loop.create_future()

    def _ws_server_main():
        async def _ws_job(websocket, path):
            try:
                async for message in websocket:
                    print(f"WS_JOB received: {message}")
                    result = json.dumps(window.evaluate_js(message))
                    print(f"WS_JOB send: {result}")
                    await websocket.send(result)
            except websockets.exceptions.ConnectionClosed:
                print("WS_JOB: Connection closed")
                pass

        async def _server():
            async with websockets.serve(_ws_job, "localhost", ws_port):
                print("WS Server: Server started")
                await stop
                print("WS Server: Server stopped")
        loop.run_until_complete(_server())

    # Start the server in a separate thread
    ws_server_thread = threading.Thread(target=_ws_server_main)
    ws_server_thread.start()
    ws_server_thread._loop = loop
    ws_server_thread._stop_future = stop
    print('WebSocket server started on port', wss_port)
    return ws_server_thread

#########################################
# in the main thread
#########################################
wss_port = os.environ.get('APP_WEBSOCKET_PORT')
websocket_thread = start_ws_server(wss_port, window) if wss_port else None

webview.start(prepare_window, 
              args=(window, websocket_thread),
              gui='qt',
              http_server=True,
              debug=False)