[Questions] what is the best version of rabbitmq for consuming parallel tasks

[NevertherlessSolution]

Community Support Policy

• I have read RabbitMQ's Community Support Policy
• I agree to provide all relevant information (versions, logs, rabbitmq-diagnostics output, detailed reproduction steps)

RabbitMQ version used

3.13.7 or older

Erlang version used

26.2.x

Operating system (distribution) used

Ubuntu

How is RabbitMQ deployed?

Community Docker image

rabbitmq-diagnostics status output

See https://www.rabbitmq.com/docs/cli to learn how to use rabbitmq-diagnostics

Details

# PASTE OUTPUT HERE, BETWEEN BACKTICKS

Logs from node 1 (with sensitive values edited out)

See https://www.rabbitmq.com/docs/logging to learn how to collect logs

Details

# PASTE LOG HERE, BETWEEN BACKTICKS

Logs from node 2 (if applicable, with sensitive values edited out)

See https://www.rabbitmq.com/docs/logging to learn how to collect logs

Details

# PASTE LOG HERE, BETWEEN BACKTICKS

Logs from node 3 (if applicable, with sensitive values edited out)

See https://www.rabbitmq.com/docs/logging to learn how to collect logs

Details

# PASTE LOG HERE, BETWEEN BACKTICKS

rabbitmq.conf

See https://www.rabbitmq.com/docs/configure#config-location to learn how to find rabbitmq.conf file location

Details

# PASTE rabbitmq.conf HERE, BETWEEN BACKTICKS

Steps to deploy RabbitMQ cluster

version: '3.8'
name: "rabbit"

services:
rabbitmq:
build:
context: .
dockerfile: Dockerfile.rabbitmq
container_name: "rabbitmq-server"
environment:
- RABBITMQ_DEFAULT_USER=${RABBITMQ_DEFAULT_USER}
- RABBITMQ_DEFAULT_PASS=${RABBITMQ_DEFAULT_PASS}
- RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS=-rabbit heartbeat 100000
ports:
- "15672:15672"
- "5672:5672"
volumes:
- "rabbitmq_data:/var/lib/rabbitmq"
restart: always
networks:
- rabbitmq_network
healthcheck:
test: [ "CMD-SHELL", "rabbitmq-diagnostics", "-q", "ping"]
interval: 30s
timeout: 20s
retries: 5
deploy:
resources:
limits:
cpus: '0.50'
memory: 1.5GB
reservations:
cpus: '0.25'
memory: 500M

networks:
rabbitmq_network:
name: rabbitmq_network

volumes:
rabbitmq_data:
rabbitconsumer_data:

Steps to reproduce the behavior in question

My application has to consume as many tasks as possible.
I am converting it in asynchronous mode using asynio and aio_pika.
But I found that it got idle sometimes.

advanced.config

See https://www.rabbitmq.com/docs/configure#config-location to learn how to find advanced.config file location

Details

# PASTE advanced.config HERE, BETWEEN BACKTICKS

Application code

Details

# PASTE CODE HERE, BETWEEN BACKTICKS

Kubernetes deployment file

Details

# Relevant parts of K8S deployment that demonstrate how RabbitMQ is deployed
# PASTE YAML HERE, BETWEEN BACKTICKS

What problem are you trying to solve?

I am converting it in asynchronous mode using asynio and aio_pika.

[michaelklishin]

Based on a very brief description (virtual no details) on what you are trying to accomplish or why, but also in general, 4.0.x is "the best version" because it is the only version that is maintained for everyone.

For "embarrassingly parallel" problems you'd likely benefit from using streams and/or superstreams, which are relatively recent and have matured by 4.x.