Skip to content

Commit

Permalink
SQLAlchemy: Fix handling URL parameters timeout and pool_size
Browse files Browse the repository at this point in the history
  • Loading branch information
amotl committed Sep 4, 2023
1 parent d2d44a5 commit d00a5fe
Show file tree
Hide file tree
Showing 4 changed files with 59 additions and 5 deletions.
2 changes: 2 additions & 0 deletions CHANGES.txt
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ Unreleased
==========

- Properly handle Python-native UUID types in SQL parameters
- SQLAlchemy: Fix handling URL parameters ``timeout`` and ``pool_size``


2023/07/17 0.33.0
=================
Expand Down
41 changes: 38 additions & 3 deletions docs/by-example/sqlalchemy/getting-started.rst
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,8 @@ Create an SQLAlchemy :doc:`Session <sa:orm/session_basics>`:
>>> Base = declarative_base()


Connection string
=================
Connect
=======

In SQLAlchemy, a connection is established using the ``create_engine`` function.
This function takes a connection string, actually an `URL`_, that varies from
Expand All @@ -65,7 +65,9 @@ to a different server the following syntax can be used:
>>> sa.create_engine('crate://otherserver:4200')
Engine(crate://otherserver:4200)

Since CrateDB is a clustered database running on multiple servers, it is
Multiple Hosts
--------------
Because CrateDB is a clustered database running on multiple servers, it is
recommended to connect to all of them. This enables the DB-API layer to
use round-robin to distribute the load and skip a server if it becomes
unavailable. In order to make the driver aware of multiple servers, use
Expand All @@ -76,6 +78,8 @@ the ``connect_args`` parameter like so:
... })
Engine(crate://)

TLS Options
-----------
As defined in :ref:`https_connection`, the client validates SSL server
certificates by default. To configure this further, use e.g. the ``ca_cert``
attribute within the ``connect_args``, like:
Expand All @@ -96,6 +100,37 @@ In order to disable SSL verification, use ``verify_ssl_cert = False``, like:
... 'verify_ssl_cert': False,
... })

Timeout Options
---------------
In order to configure TCP timeout options, use the ``timeout`` parameter within
``connect_args``,

>>> timeout_engine = sa.create_engine('crate://localhost/', connect_args={'timeout': 42.42})
>>> timeout_engine.raw_connection().driver_connection.client._pool_kw["timeout"]
42.42

or use the ``timeout`` URL parameter within the database connection URL.

>>> timeout_engine = sa.create_engine('crate://localhost/?timeout=42.42')
>>> timeout_engine.raw_connection().driver_connection.client._pool_kw["timeout"]
42.42

Pool Size
---------

In order to configure the database pool size, use the ``pool_size`` parameter within
``connect_args``,

>>> timeout_engine = sa.create_engine('crate://localhost/', connect_args={'pool_size': 20})
>>> timeout_engine.raw_connection().driver_connection.client._pool_kw["maxsize"]
20

or use the ``pool_size`` URL parameter within the database connection URL.

>>> timeout_engine = sa.create_engine('crate://localhost/?pool_size=20')
>>> timeout_engine.raw_connection().driver_connection.client._pool_kw["maxsize"]
20


Basic DDL operations
====================
Expand Down
5 changes: 3 additions & 2 deletions src/crate/client/http.py
Original file line number Diff line number Diff line change
Expand Up @@ -255,10 +255,11 @@ def _pool_kw_args(verify_ssl_cert, ca_cert, client_cert, client_key,
'cert_reqs': ssl.CERT_REQUIRED if verify_ssl_cert else ssl.CERT_NONE,
'cert_file': client_cert,
'key_file': client_key,
'timeout': timeout,
}
if timeout is not None:
kw['timeout'] = float(timeout)
if pool_size is not None:
kw['maxsize'] = pool_size
kw['maxsize'] = int(pool_size)
return kw


Expand Down
16 changes: 16 additions & 0 deletions src/crate/client/sqlalchemy/tests/connection_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,22 @@ def test_connection_server_uri_https_with_credentials(self):
conn.close()
engine.dispose()

def test_connection_server_uri_parameter_timeout(self):
engine = sa.create_engine(
"crate://otherhost:19201?timeout=42.42")
conn = engine.raw_connection()
self.assertEqual(conn.driver_connection.client._pool_kw["timeout"], 42.42)
conn.close()
engine.dispose()

def test_connection_server_uri_parameter_pool_size(self):
engine = sa.create_engine(
"crate://otherhost:19201?pool_size=20")
conn = engine.raw_connection()
self.assertEqual(conn.driver_connection.client._pool_kw["maxsize"], 20)
conn.close()
engine.dispose()

def test_connection_multiple_server_http(self):
engine = sa.create_engine(
"crate://", connect_args={
Expand Down

0 comments on commit d00a5fe

Please sign in to comment.