From 8fbb7030d356a624ff9bb0dda018996433eea5b9 Mon Sep 17 00:00:00 2001 From: Konstantin Semenov Date: Sun, 20 Nov 2016 21:48:32 +0000 Subject: [PATCH] Updated README with generic test case description [#126213923] --- README.md | 49 +++++++++++++++++++++++++++++++++++-------------- 1 file changed, 35 insertions(+), 14 deletions(-) diff --git a/README.md b/README.md index d3d33bc..a2e6755 100644 --- a/README.md +++ b/README.md @@ -1,12 +1,12 @@ # Trilogy -Trilogy is a tool for testing SQL stored procedures and functions. Test cases are represented by plain text files utilizing markdown format, which makes them easily readable and editable. +Trilogy is a tool for testing server-side database logic. Test cases are represented by plain text files utilizing markdown format, which makes them easy to read and edit. ## Requirements - JRE 1.7 or later installed - JDBC driver for the database under test available on the classpath. ##Latest version -0.1-pre +0.2 ## Command-line options - Single test case run: @@ -21,13 +21,30 @@ Trilogy is a tool for testing SQL stored procedures and functions. Test cases ar ``` add the `skip_schema_load` flag to indicate that the `tests/fixtures/schema.sql` should not be loaded if it is present in the project. -The `db_url` should point to the database under test. Make sure that the appropriate JDBC driver is available on the classpath. +The `db_url` [1] [2] should point to the database under test. Make sure that the appropriate JDBC driver is available on the classpath. Also, the `db_url`, `db_user` and `db_password` can be specified as environment variables. ## Standalone test case file format -Standalone test cases are good for small ad-hoc tests that do not require loading of the schema, procedure code or fixtures. +Standalone test cases are good for small ad-hoc tests that do not require loading of the schema, procedure code or fixtures. There are two types of test case files - generic and procedural. + +### Generic +The generic test case allows the developer to use a set of plain SQL statements as the test body. A generic test will fail if an error occurs during test or assertion execution. The layout of a generic test case is shown below: ``` -# TEST CASE +# TEST CASE + +## TEST + +```< SQL statements >``` +### ASSERTIONS +#### SQL + +```< SQL Assertions >``` +``` + +### Procedural +As the name suggests, procedural test cases can be run against stored procedures. Unfortunately, stored functions are not supported at this time. The generic layout of a procedural test case is shown below: +``` +# TEST CASE ## TEST @@ -41,11 +58,13 @@ Standalone test cases are good for small ad-hoc tests that do not require loadin ``` ``` ``` +#### Testing for errors +An optional special column `=ERROR=` can be used in the `DATA` section for testing error conditions. Leave it blank in order to specify that no error is expected during the call. Alternatively, an error substring or a wildcard keyword `ANY` can be used. For example, when a database throws an error with text `ORA-20111: User specified error is thrown`, any of the following values will ensure that the test will pass: `ANY`, `aNy`, `ORA-20111`, `user`, `Error`, `Error is thrown`. -The `TEST` and `SQL` sections can be repeated multiple times. The `SQL` assertion can span multiple lines. +For both test case types, the `TEST` and `SQL` sections can be repeated multiple times, and the `SQL` assertion, as well as the generic `TEST` SQL statement can span multiple lines. ## Test projects -Test projects are more powerful, they consist of a number of files that have to be organized in a structure outlined below. A project must have at least one test case. Additionally it can have a database schema, any number SQL procedure scripts and fixtures. +Test projects are more powerful, they consist of a number of files that have to be organized in a structure outlined below. A project can have any number of generic and/or procedural test cases, but at least one is required. Additionally it can have a database schema, any number SQL procedure scripts and fixtures. ### Project structure [project root] ┝[src]┐ @@ -73,6 +92,7 @@ Test projects are more powerful, they consist of a number of files that have to └ billpay_sender.stt ### Project test case format +The project test case format extends the standalone test case formats by adding the fixture references. All test cases can have any number of fixtures in the `BEFORE ALL`, `BEFORE EACH TEST`, `AFTER ALL` and `AFTER EACH TEST` sections at the top of the test case file. In addition to that, procedural test cases can also have `BEFORE EACH ROW` and `AFTER EACH ROW` sections, as shown in the example below: ``` # TEST CASE @@ -105,21 +125,22 @@ Test projects are more powerful, they consist of a number of files that have to ``` ### Referencing fixtures -When a fixture is referenced in the test case, regular words can be used. The reference will be then transforms the words to locate the file. For example, given the project structure above any "BEFORE *" section can reference the fixtures as "Messages", "Some folder / prepay user with LOW balance" or "some folder/user with low balance". Similarly, any "AFTER *" section can reference the fixtures as "Messages", "TransActions" or "users". At the moment only alpha-numeric filenames with underscores are supported. -For example: +When a fixture is referenced in the test case, regular words can be used. The reference will be then transformed in order to locate the file. For example, given the project structure above, any "BEFORE *" section can reference the fixtures as "Messages", "Some folder / prepay user with LOW balance" or "some folder/user with low balance". Similarly, any "AFTER *" section can reference the fixtures as "Messages", "TransActions" or "users". At the moment only alpha-numeric filenames with underscores are supported. For example: ``` ### BEFORE ALL - Messages - Balance / Prepay user with low balance ``` will instruct the framework to load the `tests/fixtures/setup/messages.sql` and `tests/fixtures/setup/balance/prepay_user_with_low_balance.sql` before running the test case in the order specified. -When a referenced fixture is not found, the whole test suite fails. - -### Testing for errors -An optional special column `=ERROR=` can be used in the test `DATA` section for testing error conditions. Leave it blank in order to specify that no error is expected during the call. Alternatively, an error substring or a wildcard keyword `ANY` can be used. For example, when a database throws an error with text `ORA-20111: User specified error is thrown`, any of the following values will ensure that the test will pass: `ANY`, `aNy`, `ORA-20111`, `user`, `Error`, `Error is thrown`. +When a referenced fixture is not found, the whole test suite fails. Notice that the locations for setup and teardown fixtures are separated. ### Automatic schema load When a `schema.sql` is placed into the `tests/fixtures` directory, it would be automatically executed before the project runs. ### Loading source files -The source files are executed in alphabetical order. \ No newline at end of file +All the `*.sql` files in the `src` directory will be loaded in alphabetical order before the project execution. This can be useful when working on a number of definitions that depend on each other. + +--- +[1] - [Reference to JDBC URLs: DB/2, Informix, MySQL, Oracle, Microsoft and Teradata](https://www.ibm.com/support/knowledgecenter/SSEP7J_10.2.0/com.ibm.swg.ba.cognos.vvm_ag_guide.10.2.0.doc/c_ag_samjdcurlform.html) + +[2] - [Reference to JDBC URLs: MySQL, Postgres and Microsoft](https://www.petefreitag.com/articles/jdbc_urls/) \ No newline at end of file