Tristen Raab: Putting Postgres to the Test: How to Create and Run Regression and TAP Tests

Introduction This blog is aimed at beginners trying to learn the basics of PostgreSQL but already have some experience under their belt. For this tutorial, we will assume you have PostgreSQL correctly installed on Ubuntu. All of these steps were done using PostgreSQL 16 (development version) and Ubuntu 23.04. We’ll go over 3 different but related testing frameworks: regression, TAP, and pgTap. First, we’ll go over regression testing. For the uninitiated, regression tests refer to tests that should be run after any changes to make sure that no new bugs are introduced into the system. This test suite should be the most comprehensive and cover all the base functionality. As a result, we can determine the correctness of our changes by running them against this test suite. Regression Tests The regression test suite in PostgreSQL can be run in one of two ways: On a temporary server that only exists for the tests On an installed server that is running before the tests start Running $ make check will execute the “core” tests on a temporary server. Furthermore, we can run $ make check-world to run the entire test suite on the temporary server and not just the “core” tests. Alternatively, we can issue the following command to run the tests on an existing server: $ make checkinstall Similar to the temporary server, there exists a “world” equivalent to run the entire test suite on an existing server: $ make checkinstall-world Make sure you actually have a server running on the default port otherwise, this test suite will fail. Overall, these tests are to ensure that PostgreSQL itself is operating correctly and without error to any of the internals. So it is a good idea to run these tests after you make any changes to the Postgres source code to ensure your changes meet these baselines. The following tests have more to do with testing a PostgreSQL implementation. That is, testing to see if your schema, functions, views, etc. work correctly when analyzed at a higher le[...]