Skip to content

feat: migrate to rapidfort container #587

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
RobHooper wants to merge 4 commits into
base: main
Choose a base branch
from
Open

feat: migrate to rapidfort container #587

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
94d110d
feat: migrate to rapidfort container
RobHooper Oct 28, 2025
cbeb9cb
chore: Simplify redis config
RobHooper Oct 28, 2025
de667b1
chore: Order Docker Compose fields consistently
jpmckinney Dec 1, 2025
fea7b14
chore: Use consistent whitespace around | within the same file
jpmckinney Dec 1, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion pillar/cove.sls
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@ x-volumes: &volumes
- db
- media
- redis/data
- redis/tmp

x-env: &env
DJANGO_PROXY: True
Expand Down
1 change: 0 additions & 1 deletion pillar/digitalbuying.sls
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ docker_apps:
volumes:
- media
- redis/data
- redis/tmp
env:
DJANGO_PROXY: True
ALLOWED_HOSTS: digitalbuying.open-contracting.org
Expand Down
1 change: 0 additions & 1 deletion pillar/dreambi.sls
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,6 @@ docker_apps:
site: bi.dream.gov.ua
volumes:
- redis/data
- redis/tmp
env:
# Must end with a "/".
QLIK_PROXY_SERVICE: https://ocp15.open-contracting.org:4243/qps/prod/
1 change: 0 additions & 1 deletion pillar/registry.sls
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,6 @@ docker_apps:
- media
- tmp
- redis/data
- redis/tmp
env:
DJANGO_PROXY: True
ALLOWED_HOSTS: flatten.open-contracting.org
Expand Down
101 changes: 101 additions & 0 deletions salt/docker_apps/files/conf/redis.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# Network
Copy link
Member

@jpmckinney jpmckinney Dec 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should minimize this file to only the options that change the default and that are truly necessary. This will make it easier to upgrade if options are deprecated in future. For example, *-ziplist-* became *-listpack-*, lua-time-limit is deprecated. I did:

curl https://raw.githubusercontent.com/redis/redis/6.2/redis.conf | grep -v '^#' | grep . | sort > a
curl https://raw.githubusercontent.com/open-contracting/deploy/de667b17e3eae5a245570df3a0a1fff4d29b9623/salt/docker_apps/files/conf/redis.conf | grep -v '^#' | grep . | sort > b
diff -u a b

And I get (defaults in comments just for clarity in this comment):

bind 0.0.0.0 :: # 127.0.0.1 -::1
appendonly yes # no
daemonize yes # no
dir /data # ./
pidfile /tmp/redis.pid # /var/run/redis_6379.pid
protected-mode no # yes
save ""  # not present in default redis.conf
  • RapidFort must be doing something special for daemonize yes to work. If we use the default image, I think we want to leave it as daemonize no.
  • If we use the default image, protected mode is off by default (protected-mode no): https://hub.docker.com/_/redis#security

So I think that leaves us with just:

bind 0.0.0.0 ::
dir /data
pidfile /tmp/redis.pid
# https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/#append-only-file
appendonly yes
# https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/#how-i-can-switch-to-aof-if-im-currently-using-dumprdb-snapshots
save ""

I repeated the diff comparing to 8.4, and, besides new options from later versions and the differences already noted, the default of repl-diskless-sync changed from no to yes. However, we don't use replicas, so it's not relevant.

bind 0.0.0.0 ::
protected-mode no

# TLS/SSL
port 6379
tcp-backlog 511
timeout 0
tcp-keepalive 300

# General
daemonize yes
pidfile /tmp/redis.pid
loglevel notice
logfile ""
databases 16
always-show-logo no
set-proc-title yes
proc-title-template "{title} {listen-addr} {server-mode}"

# Snapshotting
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
rdb-del-sync-files no
dir /data
save ""

# Replication
replica-serve-stale-data yes
replica-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-diskless-load disabled
repl-disable-tcp-nodelay no
replica-priority 100

# Security
acllog-max-len 128

# Lazy Freeing
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
replica-lazy-flush no
lazyfree-lazy-user-del no
lazyfree-lazy-user-flush no

# Kernel OOM Control
oom-score-adj no
oom-score-adj-values 0 200 800

# Kernel transparent huge page Control
disable-thp yes

# Append Only Mode
appendonly yes
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
aof-use-rdb-preamble yes

# Non-Deterministic Long Blocking Commands
lua-time-limit 5000

# Slow Log
slowlog-log-slower-than 10000
slowlog-max-len 128

# Latency Monitor
latency-monitor-threshold 0

# Event Notification
notify-keyspace-events ""

# Advanced Config
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
stream-node-max-bytes 4096
stream-node-max-entries 100
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit replica 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
dynamic-hz yes
aof-rewrite-incremental-fsync yes
rdb-save-incremental-fsync yes

# Active Defragmentation
jemalloc-bg-thread yes
8 changes: 4 additions & 4 deletions salt/docker_apps/files/cove.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,18 +21,18 @@ x-django: &django
services:
{%- if 'REDIS_URL' in entry.env %}
redis:
image: bitnamilegacy/redis:6.2
image: rapidfort/redis-official:6.2-bookworm
Copy link
Member

@jpmckinney jpmckinney Dec 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think best to use the official image, rather than risk the same issue again. I don't think the official image is especially vulnerable.

command: ["redis-server", "/redis.conf"]
user: "{{ pillar.docker.uid }}:{{ pillar.docker.uid }}"
Copy link
Member

@jpmckinney jpmckinney Dec 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the same as running Docekr with --user? See "Process User and Privileges" at https://hub.docker.com/_/redis#process-user-and-privileges

restart: always
environment:
ALLOW_EMPTY_PASSWORD: "yes"
healthcheck:
test: redis-cli -h 127.0.0.1 ping | grep -q PONG
interval: 10s
timeout: 5s
retries: 5
volumes:
- {{ directory }}/redis/data:/bitnami/redis/data
Copy link
Member

@jpmckinney jpmckinney Dec 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bitnami still referenced in 3 files.

- {{ directory }}/redis/tmp:/opt/bitnami/redis/tmp
Copy link
Member

@jpmckinney jpmckinney Dec 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we want the /tmp volumes for the pidfile configuration?

- {{ directory }}/redis/redis.conf:/redis.conf
{%- endif %}
web:
<<: *django
Expand Down
10 changes: 5 additions & 5 deletions salt/docker_apps/files/digitalbuying.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,18 +16,18 @@ x-django: &django

services:
redis:
image: bitnamilegacy/redis:6.2
image: rapidfort/redis-official:6.2-bookworm
command: ["redis-server", "/redis.conf"]
user: "{{ pillar.docker.uid }}:{{ pillar.docker.uid }}"
restart: always
environment:
ALLOW_EMPTY_PASSWORD: "yes"
healthcheck:
test: redis-cli -h 127.0.0.1 ping | grep -q PONG
interval: 10s
timeout: 5s
retries: 5
volumes:
- {{ directory }}/redis/data:/bitnami/redis/data
- {{ directory }}/redis/tmp:/opt/bitnami/redis/tmp
- {{ directory }}/redis/data:/data
- {{ directory }}/redis/redis.conf:/redis.conf
web:
<<: *django
ports:
Expand Down
8 changes: 4 additions & 4 deletions salt/docker_apps/files/qlikauth.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,18 @@ x-shared: &shared

services:
redis:
image: bitnamilegacy/redis:6.2
image: rapidfort/redis-official:6.2-bookworm
command: ["redis-server", "/redis.conf"]
user: "{{ pillar.docker.uid }}:{{ pillar.docker.uid }}"
restart: always
environment:
ALLOW_EMPTY_PASSWORD: "yes"
healthcheck:
test: redis-cli -h 127.0.0.1 ping | grep -q PONG
interval: 10s
timeout: 5s
retries: 5
volumes:
- {{ directory }}/redis/data:/bitnami/redis/data
- {{ directory }}/redis/tmp:/opt/bitnami/redis/tmp
- {{ directory }}/redis/redis.conf:/redis.conf
web:
<<: *shared
image: "ghcr.io/open-contracting/bi.dream.gov.ua-qlikauth:latest"
Expand Down
8 changes: 4 additions & 4 deletions salt/docker_apps/files/spoonbill.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,18 +20,18 @@ x-django: &django

services:
redis:
image: bitnamilegacy/redis:6.2
image: rapidfort/redis-official:6.2-bookworm
command: ["redis-server", "/redis.conf"]
user: "{{ pillar.docker.uid }}:{{ pillar.docker.uid }}"
restart: always
environment:
ALLOW_EMPTY_PASSWORD: "yes"
healthcheck:
test: redis-cli -h 127.0.0.1 ping | grep -q PONG
interval: 10s
timeout: 5s
retries: 5
volumes:
- {{ entry.host_dir }}/redis/data:/bitnami/redis/data
- {{ entry.host_dir }}/redis/tmp:/opt/bitnami/redis/tmp
- {{ directory }}/redis/redis.conf:/redis.conf
web:
<<: *django
ports:
Expand Down
19 changes: 13 additions & 6 deletions salt/docker_apps/init.sls
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,16 +41,23 @@ include:
- user: {{ pillar.docker.user }}_user_exists

{% for volume in entry.volumes|default([]) %}
# "NOTE: As this is a non-root container, the mounted files and directories must have the proper permissions for the UID 1001."
# https://github.com/bitnami/containers/blob/main/bitnami/redis/README.md#persisting-your-database
{% set volume_user_group = 1001 if volume.startswith('redis/') else pillar.docker.user %}
Copy link
Contributor Author

@RobHooper RobHooper Oct 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Setting UID 1001 is no longer needed as the new container respects the user setting in docker compose.


{{ entry.host_dir|default(directory) }}/{{ volume }}:
file.directory:
- user: {{ volume_user_group }}
- group: {{ volume_user_group }}
- user: {{ pillar.docker.user }}
- group: {{ pillar.docker.user }}
- makedirs: True
- require:
- user: {{ pillar.docker.user }}_user_exists
{% endfor %}

{% if 'redis' in entry.volumes|join(' ') %}
{{ directory }}/redis/redis.conf:
file.managed:
- source: salt://docker_apps/files/conf/redis.conf
- user: {{ pillar.docker.user }}
- group: {{ pillar.docker.user }}
- makedirs: True
- require:
- user: {{ pillar.docker.user }}_user_exists
{% endif %}
{% endfor %}