Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SQLAlchemy: Fix handling URL parameters timeout and pool_size #571

Merged
merged 1 commit into from
Sep 5, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGES.txt
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ Unreleased
==========

- Properly handle Python-native UUID types in SQL parameters
- SQLAlchemy: Fix handling URL parameters ``timeout`` and ``pool_size``


2023/07/17 0.33.0
=================
Expand Down
41 changes: 38 additions & 3 deletions docs/by-example/sqlalchemy/getting-started.rst
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,8 @@ Create an SQLAlchemy :doc:`Session <sa:orm/session_basics>`:
>>> Base = declarative_base()


Connection string
=================
Connect
=======

In SQLAlchemy, a connection is established using the ``create_engine`` function.
This function takes a connection string, actually an `URL`_, that varies from
Expand All @@ -65,7 +65,9 @@ to a different server the following syntax can be used:
>>> sa.create_engine('crate://otherserver:4200')
Engine(crate://otherserver:4200)

Since CrateDB is a clustered database running on multiple servers, it is
Multiple Hosts
--------------
Because CrateDB is a clustered database running on multiple servers, it is
recommended to connect to all of them. This enables the DB-API layer to
use round-robin to distribute the load and skip a server if it becomes
unavailable. In order to make the driver aware of multiple servers, use
Expand All @@ -76,6 +78,8 @@ the ``connect_args`` parameter like so:
... })
Engine(crate://)

TLS Options
-----------
As defined in :ref:`https_connection`, the client validates SSL server
certificates by default. To configure this further, use e.g. the ``ca_cert``
attribute within the ``connect_args``, like:
Expand All @@ -96,6 +100,37 @@ In order to disable SSL verification, use ``verify_ssl_cert = False``, like:
... 'verify_ssl_cert': False,
... })

Timeout Options
---------------
In order to configure TCP timeout options, use the ``timeout`` parameter within
``connect_args``,

>>> timeout_engine = sa.create_engine('crate://localhost/', connect_args={'timeout': 42.42})
>>> timeout_engine.raw_connection().driver_connection.client._pool_kw["timeout"]
42.42

or use the ``timeout`` URL parameter within the database connection URL.

>>> timeout_engine = sa.create_engine('crate://localhost/?timeout=42.42')
>>> timeout_engine.raw_connection().driver_connection.client._pool_kw["timeout"]
42.42

Pool Size
---------

In order to configure the database connection pool size, use the ``pool_size``
parameter within ``connect_args``,

>>> timeout_engine = sa.create_engine('crate://localhost/', connect_args={'pool_size': 20})
>>> timeout_engine.raw_connection().driver_connection.client._pool_kw["maxsize"]
20

or use the ``pool_size`` URL parameter within the database connection URL.

>>> timeout_engine = sa.create_engine('crate://localhost/?pool_size=20')
>>> timeout_engine.raw_connection().driver_connection.client._pool_kw["maxsize"]
20


Basic DDL operations
====================
Expand Down
5 changes: 3 additions & 2 deletions src/crate/client/http.py
Original file line number Diff line number Diff line change
Expand Up @@ -255,10 +255,11 @@ def _pool_kw_args(verify_ssl_cert, ca_cert, client_cert, client_key,
'cert_reqs': ssl.CERT_REQUIRED if verify_ssl_cert else ssl.CERT_NONE,
'cert_file': client_cert,
'key_file': client_key,
'timeout': timeout,
}
if timeout is not None:
kw['timeout'] = float(timeout)
if pool_size is not None:
kw['maxsize'] = pool_size
kw['maxsize'] = int(pool_size)
return kw


Expand Down
16 changes: 16 additions & 0 deletions src/crate/client/sqlalchemy/tests/connection_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,22 @@ def test_connection_server_uri_https_with_credentials(self):
conn.close()
engine.dispose()

def test_connection_server_uri_parameter_timeout(self):
engine = sa.create_engine(
"crate://otherhost:19201/?timeout=42.42")
conn = engine.raw_connection()
self.assertEqual(conn.driver_connection.client._pool_kw["timeout"], 42.42)
conn.close()
engine.dispose()

def test_connection_server_uri_parameter_pool_size(self):
engine = sa.create_engine(
"crate://otherhost:19201/?pool_size=20")
conn = engine.raw_connection()
self.assertEqual(conn.driver_connection.client._pool_kw["maxsize"], 20)
conn.close()
engine.dispose()

def test_connection_multiple_server_http(self):
engine = sa.create_engine(
"crate://", connect_args={
Expand Down