maintenance: upgrade to postgres:17.7-trixie in docs and for Testcontainers (#222)
Co-authored-by: Michael Hoennig <michael@hoennig.de> Reviewed-on: http://dev.hostsharing.net/hostsharing/hs.hsadmin.ng/pulls/222 Reviewed-by: Stefan Begerad <stefan.begerad@hostsharing.net>
This commit is contained in:
+2
-2
@@ -1,7 +1,7 @@
|
|||||||
# build using:
|
# build using:
|
||||||
# docker build -t postgres-with-contrib:15.5-bookworm .
|
# docker build -t postgres-with-contrib:17.7-trixie .
|
||||||
|
|
||||||
FROM postgres:15.5-bookworm
|
FROM postgres:17.7-trixie
|
||||||
|
|
||||||
RUN apt-get update && \
|
RUN apt-get update && \
|
||||||
apt-get install -y postgresql-contrib && \
|
apt-get install -y postgresql-contrib && \
|
||||||
|
|||||||
@@ -60,7 +60,7 @@ Everything is tested on _Ubuntu Linux 22.04_ and _MacOS Monterey (12.4)_.
|
|||||||
To be able to build and run the Java Spring Boot application, you need the following tools:
|
To be able to build and run the Java Spring Boot application, you need the following tools:
|
||||||
|
|
||||||
- Docker 20.x (on MacOS you also need *Docker Desktop* or similar) or Podman
|
- Docker 20.x (on MacOS you also need *Docker Desktop* or similar) or Podman
|
||||||
- optionally: PostgreSQL Server 15.5-bookworm, if you want to use the database directly, not just via Docker
|
- optionally: PostgreSQL Server 17.7-trixie, if you want to use the database directly, not just via Docker
|
||||||
(see instructions below to install and run in Docker)
|
(see instructions below to install and run in Docker)
|
||||||
- The matching Java JDK at will be automatically installed by Gradle toolchain support to `~/.gradle/jdks/`.
|
- The matching Java JDK at will be automatically installed by Gradle toolchain support to `~/.gradle/jdks/`.
|
||||||
- You also might need an IDE (e.g. *IntelliJ IDEA* or *Eclipse* or *VS Code* with *[STS](https://spring.io/tools)* and a GUI Frontend for *PostgreSQL* like *Postbird*.
|
- You also might need an IDE (e.g. *IntelliJ IDEA* or *Eclipse* or *VS Code* with *[STS](https://spring.io/tools)* and a GUI Frontend for *PostgreSQL* like *Postbird*.
|
||||||
|
|||||||
@@ -0,0 +1,40 @@
|
|||||||
|
# PR#222: Maintenance Upgrade to PostgreSQL 17.7 Trixie
|
||||||
|
|
||||||
|
## The Problem
|
||||||
|
|
||||||
|
Even though I had tested the application with PostgreSQL `17.7-trixie` a while ago,
|
||||||
|
the project still used PostgreSQL `15.5-bookworm` in the local Docker setup, in Testcontainers-based test
|
||||||
|
and in documentation.
|
||||||
|
|
||||||
|
After switching the tests to PostgreSQL `17.7-trixie`, the migration tests which generate reference SQL dumps failed on developer machines with an older host `pg_dump`.
|
||||||
|
PostgreSQL requires `pg_dump` to be at least as new as the database server.
|
||||||
|
Therefore, a host-side `pg_dump` 16 could not dump a PostgreSQL 17 Testcontainer database.
|
||||||
|
|
||||||
|
The failing error looked like this:
|
||||||
|
|
||||||
|
```text
|
||||||
|
pg_dump: error: aborting because of server version mismatch
|
||||||
|
pg_dump: detail: server version: 17.7 (Debian 17.7-3.pgdg13+1); pg_dump version: 16.13 (Ubuntu 16.13-0ubuntu0.24.04.1)
|
||||||
|
```
|
||||||
|
|
||||||
|
## The Solution
|
||||||
|
|
||||||
|
The PostgreSQL Docker image references were upgraded from `15.5-bookworm` to `17.7-trixie`.
|
||||||
|
This includes the custom PostgreSQL image, the Docker Compose setup, developer documentation, and the Testcontainers JDBC URLs used by integration and migration tests.
|
||||||
|
|
||||||
|
The SQL dump helper used by the migration tests was changed to run `pg_dump` inside the PostgreSQL Testcontainer.
|
||||||
|
This ensures that `pg_dump` always has the same major version as the PostgreSQL server used by the test.
|
||||||
|
|
||||||
|
## Additional Changes
|
||||||
|
|
||||||
|
The RBAC performance analysis documentation was updated to use the new PostgreSQL image tag in its examples.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
The Liquibase compatibility test was run successfully with the required PostgreSQL role environment variables:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
env HSADMINNG_POSTGRES_ADMIN_USERNAME=admin \
|
||||||
|
HSADMINNG_POSTGRES_RESTRICTED_USERNAME=restricted \
|
||||||
|
./gradlew test --tests net.hostsharing.hsadminng.hs.migration.LiquibaseCompatibilityIntegrationTest
|
||||||
|
```
|
||||||
@@ -24,7 +24,7 @@ The module auto_explain can be used to automatically run EXPLAIN on long-running
|
|||||||
To use this extension and module, we extended the PostgreSQL-Docker-image:
|
To use this extension and module, we extended the PostgreSQL-Docker-image:
|
||||||
|
|
||||||
```Dockerfile
|
```Dockerfile
|
||||||
FROM postgres:15.5-bookworm
|
FROM postgres:17.7-trixie
|
||||||
|
|
||||||
RUN apt-get update && \
|
RUN apt-get update && \
|
||||||
apt-get install -y postgresql-contrib && \
|
apt-get install -y postgresql-contrib && \
|
||||||
@@ -36,7 +36,7 @@ COPY etc/postgresql-log-slow-queries.conf /etc/postgresql/postgresql.conf
|
|||||||
And create an image from it:
|
And create an image from it:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
docker build -t postgres-with-contrib:15.5-bookworm .
|
docker build -t postgres-with-contrib:17.7-trixie .
|
||||||
```
|
```
|
||||||
|
|
||||||
Then we created a config file for PostgreSQL in `etc/postgresql-log-slow-queries.conf`:
|
Then we created a config file for PostgreSQL in `etc/postgresql-log-slow-queries.conf`:
|
||||||
@@ -61,7 +61,7 @@ version: '3.8'
|
|||||||
|
|
||||||
services:
|
services:
|
||||||
postgres:
|
postgres:
|
||||||
image: postgres-with-contrib:15.5-bookworm
|
image: postgres-with-contrib:17.7-trixie
|
||||||
container_name: custom-postgres
|
container_name: custom-postgres
|
||||||
environment:
|
environment:
|
||||||
POSTGRES_PASSWORD: password
|
POSTGRES_PASSWORD: password
|
||||||
|
|||||||
@@ -2,7 +2,7 @@ version: '3.8'
|
|||||||
|
|
||||||
services:
|
services:
|
||||||
postgres:
|
postgres:
|
||||||
image: postgres-with-contrib:15.5-bookworm
|
image: postgres-with-contrib:17.7-trixie
|
||||||
container_name: custom-postgres
|
container_name: custom-postgres
|
||||||
environment:
|
environment:
|
||||||
POSTGRES_PASSWORD: password
|
POSTGRES_PASSWORD: password
|
||||||
|
|||||||
@@ -95,7 +95,7 @@ import static org.springframework.util.FileCopyUtils.copyToByteArray;
|
|||||||
@Tag("migrationTest")
|
@Tag("migrationTest")
|
||||||
@Tag("importHostingAssets")
|
@Tag("importHostingAssets")
|
||||||
@DataJpaTest(properties = {
|
@DataJpaTest(properties = {
|
||||||
"spring.datasource.url=${HSADMINNG_POSTGRES_JDBC_URL:jdbc:tc:postgresql:15.5-bookworm:///importHostingAssetsTC}",
|
"spring.datasource.url=${HSADMINNG_POSTGRES_JDBC_URL:jdbc:tc:postgresql:17.7-trixie:///importHostingAssetsTC}",
|
||||||
"spring.datasource.username=${HSADMINNG_POSTGRES_ADMIN_USERNAME:ADMIN}",
|
"spring.datasource.username=${HSADMINNG_POSTGRES_ADMIN_USERNAME:ADMIN}",
|
||||||
"spring.datasource.password=${HSADMINNG_POSTGRES_ADMIN_PASSWORD:password}",
|
"spring.datasource.password=${HSADMINNG_POSTGRES_ADMIN_PASSWORD:password}",
|
||||||
"hsadminng.superuser=${HSADMINNG_SUPERUSER:import-superuser@hostsharing.net}",
|
"hsadminng.superuser=${HSADMINNG_SUPERUSER:import-superuser@hostsharing.net}",
|
||||||
|
|||||||
+1
-1
@@ -37,7 +37,7 @@ import static org.springframework.test.context.jdbc.Sql.ExecutionPhase.BEFORE_TE
|
|||||||
*/
|
*/
|
||||||
@Tag("migrationTest")
|
@Tag("migrationTest")
|
||||||
@DataJpaTest(properties = {
|
@DataJpaTest(properties = {
|
||||||
"spring.datasource.url=jdbc:tc:postgresql:15.5-bookworm:///liquibaseMigrationTestTC",
|
"spring.datasource.url=jdbc:tc:postgresql:17.7-trixie:///liquibaseMigrationTestTC",
|
||||||
"hsadminng.superuser=${HSADMINNG_SUPERUSER:import-superuser@hostsharing.net}",
|
"hsadminng.superuser=${HSADMINNG_SUPERUSER:import-superuser@hostsharing.net}",
|
||||||
"spring.liquibase.enabled=false" // @Sql should go first, Liquibase will be initialized programmatically
|
"spring.liquibase.enabled=false" // @Sql should go first, Liquibase will be initialized programmatically
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -4,10 +4,7 @@ import lombok.SneakyThrows;
|
|||||||
import org.testcontainers.containers.JdbcDatabaseContainer;
|
import org.testcontainers.containers.JdbcDatabaseContainer;
|
||||||
import org.testcontainers.jdbc.ContainerDatabaseDriver;
|
import org.testcontainers.jdbc.ContainerDatabaseDriver;
|
||||||
|
|
||||||
import java.io.BufferedReader;
|
|
||||||
import java.io.File;
|
import java.io.File;
|
||||||
import java.io.InputStreamReader;
|
|
||||||
import java.util.stream.Collectors;
|
|
||||||
|
|
||||||
import static java.nio.charset.StandardCharsets.UTF_8;
|
import static java.nio.charset.StandardCharsets.UTF_8;
|
||||||
import static org.apache.commons.io.FileUtils.readFileToString;
|
import static org.apache.commons.io.FileUtils.readFileToString;
|
||||||
@@ -23,22 +20,20 @@ public class PostgresTestcontainer {
|
|||||||
|
|
||||||
final var jdbcDatabaseContainer = getJdbcDatabaseContainer(jdbcUrl);
|
final var jdbcDatabaseContainer = getJdbcDatabaseContainer(jdbcUrl);
|
||||||
|
|
||||||
|
// use the pg_dump from within the container to get the same version
|
||||||
final var sqlDumpFile = new File(targetFileName.getParent(), "." + targetFileName.getName());
|
final var sqlDumpFile = new File(targetFileName.getParent(), "." + targetFileName.getName());
|
||||||
final var pb = new ProcessBuilder(
|
final var containerDumpFile = "/tmp/" + sqlDumpFile.getName();
|
||||||
|
final var result = jdbcDatabaseContainer.execInContainer(
|
||||||
|
"env", "PGPASSWORD=" + jdbcDatabaseContainer.getPassword(),
|
||||||
"pg_dump", "--column-inserts", "--disable-dollar-quoting",
|
"pg_dump", "--column-inserts", "--disable-dollar-quoting",
|
||||||
"--host=" + jdbcDatabaseContainer.getHost(),
|
"--host=localhost",
|
||||||
"--port=" + jdbcDatabaseContainer.getFirstMappedPort(),
|
"--port=5432",
|
||||||
"--username=" + jdbcDatabaseContainer.getUsername() ,
|
"--username=" + jdbcDatabaseContainer.getUsername() ,
|
||||||
"--dbname=" + jdbcDatabaseContainer.getDatabaseName(),
|
"--dbname=" + jdbcDatabaseContainer.getDatabaseName(),
|
||||||
"--file=" + sqlDumpFile.getCanonicalPath()
|
"--file=" + containerDumpFile
|
||||||
);
|
);
|
||||||
pb.environment().put("PGPASSWORD", jdbcDatabaseContainer.getPassword());
|
assertThat(result.getExitCode()).describedAs(result.getStderr()).isEqualTo(0);
|
||||||
|
jdbcDatabaseContainer.copyFileFromContainer(containerDumpFile, sqlDumpFile.getCanonicalPath());
|
||||||
final var process = pb.start();
|
|
||||||
int exitCode = process.waitFor();
|
|
||||||
final var stderr = new BufferedReader(new InputStreamReader(process.getErrorStream()))
|
|
||||||
.lines().collect(Collectors.joining("\n"));
|
|
||||||
assertThat(exitCode).describedAs(stderr).isEqualTo(0);
|
|
||||||
|
|
||||||
final var header = """
|
final var header = """
|
||||||
-- =================================================================================
|
-- =================================================================================
|
||||||
|
|||||||
@@ -53,7 +53,7 @@ import static org.assertj.core.api.Assertions.assertThat;
|
|||||||
classes = { HsadminNgApplication.class,
|
classes = { HsadminNgApplication.class,
|
||||||
RbacGrantsDiagramService.class },
|
RbacGrantsDiagramService.class },
|
||||||
properties = {
|
properties = {
|
||||||
"spring.datasource.url=${HSADMINNG_POSTGRES_JDBC_URL:jdbc:tc:postgresql:15.5-bookworm:///scenariosTC}",
|
"spring.datasource.url=${HSADMINNG_POSTGRES_JDBC_URL:jdbc:tc:postgresql:17.7-trixie:///scenariosTC}",
|
||||||
"spring.datasource.username=${HSADMINNG_POSTGRES_ADMIN_USERNAME:ADMIN}",
|
"spring.datasource.username=${HSADMINNG_POSTGRES_ADMIN_USERNAME:ADMIN}",
|
||||||
"spring.datasource.password=${HSADMINNG_POSTGRES_ADMIN_PASSWORD:password}",
|
"spring.datasource.password=${HSADMINNG_POSTGRES_ADMIN_PASSWORD:password}",
|
||||||
"hsadminng.superuser=${HSADMINNG_SUPERUSER:superuser-alex@hostsharing.net}"
|
"hsadminng.superuser=${HSADMINNG_SUPERUSER:superuser-alex@hostsharing.net}"
|
||||||
|
|||||||
@@ -14,8 +14,8 @@ spring:
|
|||||||
platform: postgres
|
platform: postgres
|
||||||
|
|
||||||
datasource:
|
datasource:
|
||||||
url-tc: jdbc:tc:postgresql:15.5-bookworm:///spring_boot_testcontainers
|
url-tc: jdbc:tc:postgresql:17.7-trixie:///spring_boot_testcontainers
|
||||||
url-tcx: jdbc:tc:postgresql:15.5-bookworm:///spring_boot_testcontainers?TC_REUSABLE=true&TC_DAEMON=true
|
url-tcx: jdbc:tc:postgresql:17.7-trixie:///spring_boot_testcontainers?TC_REUSABLE=true&TC_DAEMON=true
|
||||||
url-local: jdbc:postgresql://localhost:5432/postgres
|
url-local: jdbc:postgresql://localhost:5432/postgres
|
||||||
url: ${spring.datasource.url-tc}
|
url: ${spring.datasource.url-tc}
|
||||||
username: postgres
|
username: postgres
|
||||||
|
|||||||
Reference in New Issue
Block a user