PgQueuer vs Celery

Celery and PgQueuer solve the same core problem (reliable background job processing) with different trade-offs. This page compares them side by side so you can pick the right tool for your situation.

Feature Comparison

Feature	Celery	PgQueuer
Message broker	Redis, RabbitMQ, SQS, etc.	PostgreSQL
Recurring tasks	Separate celery-beat process	Built-in @pgq.schedule decorator
Architecture	App + broker + optional beat	App + PostgreSQL
Job delivery	Broker-dependent	Real-time via LISTEN/NOTIFY
Transactional enqueue	Separate from app data	Same PostgreSQL transaction
Async model	Sync-first with async support	Built on asyncio
Complex workflows	Chains, chords, groups, canvas	Basic job queue

Example: Running a Worker

Celery needs a broker and a separate worker process:

# celery_app.py
from celery import Celery

celery_app = Celery("tasks", broker="redis://localhost:6379/0")

@celery_app.task
def add(x, y):
    return x + y

celery -A celery_app worker -l info

PgQueuer connects directly to PostgreSQL:

# pgqueuer_app.py
import json
import asyncpg
from contextlib import asynccontextmanager
from pgqueuer import PgQueuer
from pgqueuer.db import AsyncpgDriver
from pgqueuer.models import Job

@asynccontextmanager
async def create_pgq():
    conn = await asyncpg.connect()
    driver = AsyncpgDriver(conn)
    pgq = PgQueuer(driver)

    @pgq.entrypoint("add")
    async def add(job: Job):
        data = json.loads(job.payload)
        return data["x"] + data["y"]

    yield pgq

pgq run pgqueuer_app:create_pgq

Example: Enqueuing a Task

Celery uses delay():

from celery_app import add
result = add.delay(2, 3)
print(result.id)

PgQueuer enqueues directly into PostgreSQL:

import asyncpg
from pgqueuer.db import AsyncpgDriver
from pgqueuer.queries import Queries

async def main() -> None:
    conn = await asyncpg.connect()
    driver = AsyncpgDriver(conn)
    job_ids = await Queries(driver).enqueue("add", b'{"x": 2, "y": 3}')
    print(job_ids)

Example: Scheduling a Recurring Task

Celery requires the celery-beat service:

from celery import Celery
from celery.schedules import crontab

app = Celery("tasks", broker="redis://localhost:6379/0")

@app.task
def cleanup():
    print("Cleaning up...")

app.conf.beat_schedule = {
    "cleanup-every-hour": {
        "task": "cleanup",
        "schedule": crontab(minute=0, hour="*"),
    }
}

celery -A celery_scheduled worker -B -l info

PgQueuer has the scheduler built in:

import asyncpg
from contextlib import asynccontextmanager
from pgqueuer import PgQueuer
from pgqueuer.db import AsyncpgDriver
from pgqueuer.models import Schedule

@asynccontextmanager
async def create_pgq():
    conn = await asyncpg.connect()
    driver = AsyncpgDriver(conn)
    pgq = PgQueuer(driver)

    @pgq.schedule("cleanup", "0 * * * *")
    async def cleanup(schedule: Schedule):
        print("Cleaning up...")

    yield pgq

pgq run pgqueuer_scheduled:create_pgq

Example: Waiting for Job Completion

Celery provides AsyncResult.get():

result = add.delay(2, 3)
print(result.get())

PgQueuer uses CompletionWatcher, which streams status updates via LISTEN/NOTIFY:

import asyncpg
from pgqueuer import PgQueuer
from pgqueuer.db import AsyncpgDriver
from pgqueuer.queries import Queries
from pgqueuer.core.completion import CompletionWatcher

async def wait_for_job() -> None:
    conn = await asyncpg.connect()
    driver = AsyncpgDriver(conn)
    pgq = PgQueuer(driver)

    queries = Queries(driver)
    job_ids = await queries.enqueue("add", b'{"x": 2, "y": 3}')
    async with CompletionWatcher(driver, queries=queries) as watcher:
        status = await watcher.wait_for(job_ids[0])
        print(status)

Choosing Between Them

Celery is a mature project with a long history. Its advantages over PgQueuer are canvas primitives for task composition, brokers that are not tied to one language, and throughput beyond a single PostgreSQL instance. PgQueuer's advantages are the ones visible in the examples above: no broker, no beat process, and enqueuing inside the transaction that writes your application data.

When to use PgQueuer works through the fit question in full, and Benchmarks covers throughput.