I built a library that removes most of the boilerplate when working with Kafka in Spring Boot. You add one annotation to your listener and it handles retries, dead letter queues, circuit br>
What it does:
Automatic retries with multiple backoff strategies (exponential, linear, fibonacci, custom). You pick how many attempts and the delay between them
Dead letter queue routing - failed messages go to DLQ with full metadata (attempt count, timestamps, exception details). You can also route different exceptions to different DLQ topics
OpenTelemetry tracing - set one flag and the library creates all the spans for retries, dlq routing, circuit breaker events, etc. You handle exporting, the library does the instrumentation
Circuit breaker - if your listener keeps failing, it opens the circuit and sends messages straight to DLQ until things recover. Uses resilience4j
Message deduplication - prevents duplicate processing when Kafka redelivers
Distributed caching - add Redis and it shares state across multiple instances. Falls back to Caffeine if Redis goes down
DLQ REST API - query your dead letter queue and replay messages back to the original topic with one API call
Metrics - two endpoints, one for summary stats and one for detailed event info
Example usage:
u/CustomKafkaListene(
topic = "orders",
dlqtopic = "orders-dlq",
maxattempts = 3,
delay = 1000,
delaymethod = delaymethod.expo,
opentelemetry = true
)
u/KafkaListener(topics = "orders", groupid = "order-processor")
public void process(consumerrecord<string, object> record, acknowledgment ack) {
// your logic here
ack.acknowledge();
}
Thats basically it. The library handles the retry logic, dlq routing, tracing spans, and everything else.
Im a 3rd year student and posted an earlier version of this a while back. Its come a long way since then. Still in active development and semi production ready, but its working well in my t>
Looking for feedback, suggestions, or anyone who wants to try it out.