Open source Confluent Cloud alternatives

Why do we need Flask, Celery, and Redis? (2019)

“This is what I have used in the past. I think they're very convenient if you are just running one instance of uwsgi and want to share some state quickly without going through a persistent database. If you're worrying about tweaking Celery for performance, then I suspect your…”

Why do we need Flask, Celery, and Redis? (2019)

“I suspect they are using a single server. I found uwsgi's mules and cache2 very useful in that situation. If someone finds that Redis and Celery are more complication than they need for a given task, then I think they're probably not using an orchestration framework.”

NSQ – A realtime distributed messaging platform

“I would also like to see a queue system with better guarantees. We've been running RabbitMQ for a couple of years and are fed up with its lack of safety in the face of minor cluster failures. I don't know about NSQ — it looks nice, but the lack of true persistence and ordering…”

NSQ – A realtime distributed messaging platform

“For the applications I've dealt with, it's an upside that a workers distributed across a number of servers can process messages from producers distributed across a number of servers, with no user-defined partitioning. Kafka, I think, requires you to define partitions to have…”

JavaScript Temporal is coming

“As the comment mentions there, the idea is that you can do those sorts of transformations on the duration and then emit them: >> d = Temporal.Duration.from({milliseconds: 60000}) >> d.seconds 0 >> d.milliseconds 60000 >> d = d.round({largestUnit: 'second'}) >> d.seconds 60 >>…”

JavaScript Temporal is coming

“Passing Dates to a React component caused re-rendering issues in some of my applications due to Date being an object. I had to resort to numbers (Unix time) to not have to add memoization everywhere. Seems like this will continue with Temporal. Or React-Compiler will solve this.”

Jepsen: NATS 2.12.1

“> What level of technical due diligence was performed by organizations like Mastercard, Volvo, PayPal, Baidu, Alibaba, or AT&T before adopting this system? I have to note the following as a NATS fan: - I am horrified at Jespen's reliability findings, however they do vindicate…”

Jepsen: NATS 2.12.1

“The only post in this thread that actually summarized the core findings of the study, namely: - ACKed messages can be silently lost due to minority-node corruption. - A single-bit corruption can cause some replicas to lose up to 78% of stored messages - Snapshot corruption can…”

Streaming Databases in Real-Time with Kafka, Debezium, and MySQL

“The "traditional" Kafka infrastructure (i.e. LinkedIn) uses Kafka as the source of truth. Everything gets written directly into Kafka and makes its way from there. This is a powerful concept, but involves giving up a lot of advantages of a RDBMS. This is the first time I've…”

Streaming Databases in Real-Time with Kafka, Debezium, and MySQL

“Debezium implements Kafka Connectors, so the serialization is pluggable. Debezium can work with thrift, you just need a thrift converter for Kafka Connect.”

Suggestion: multi-job dependency

“I suggest extending the current dependency feature to allow a job to depend on multiple jobs, not just one, so that it can be run only when all those jobs it depends on have succeeded.”

Job queued but not run

“I have a job not run, in the log of worker: 2016-11-15 23:18:34,617 11894 {q}default: [job details] (e47b96dc-d0f8-48b4-bbc9-090e6443080a) 2016-11-15 23:18:34,618 11894 Sent heartbeat to prevent worker timeout. Next one should arrive within 420 seconds. 2016-11-15 23:18:34,621…”

Job Scheduling in Node.js with BullMQ

“Great article, with one caveat: The big upfront question should be; do I need BullMQ (and Redis). I think in most cases a simple DB queue table will suffice, reducing complexity allowing for greater customization.”

[Bug]: Worker stopped processing jobs, and mostly delayed Jobs

“### Version v5.4.2 ### Platform NodeJS ### What happened? We have a service, where a worker runs, and processes jobs. After the processing is done, it will create another job, which is delayed (around 64 minutes). Today, I noticed that the service and worker stopped processing…”

RisingWave: An Open‑Source Stream‑Processing and Management Platform

“I've been running this in prod self hosted for around 6 months (podman with docker compose, minio for s3, streaming with pulsar). We have built position calculations for risk monitoring and booking enrichment pipelines. Risingwave is a much better alternative to Kafka Streams:…”

RisingWave: An Open‑Source Stream‑Processing and Management Platform

“The use case section also mentions Event-driven applications which is quite broad. Like other comment on the thread I'd be curious to hear about anyone having experience with RisingWave in this use case area.”

Upgrade to log4j 2.15.0 to address CVE-2021-44228

“Could log4j be upgraded to 2.15.0 to address CVE-2021-44228? adding below in pom.xml seems to do it ... org.apache.logging.log4j log4j-bom 2.15.0 import pom ...”

Looking for collaborators/maintainers

“Hello! I'm looking for additional collaborators/maintainers, to help me triage issues, do small fixes, and coordinate the activities for Kafdrop. I don't think any particular commitment is required. We mainly need 2/3 maintainers (or more!) that using Github "reviewers" feature…”

VerneMQ – An Erlang powered MQTT message broker for the IoT

“>>> which is similar to VerneMQ (I guess) but way less complex I quick read of both VerneMQ and your queue service indicates that your guess is wrong: - The whole MQTT spec (compact on-the-air messages, hierarchical topics, wildcard subscriptions, QoS level, message persistence)…”

VerneMQ – An Erlang powered MQTT message broker for the IoT

“I wrote a very simple queue for embedded devices to talk to (in Go, so you just copy the single binary), which is similar to VerneMQ (I guess) but way less complex: It's basically an HTTP API for polling pub/sub, along with a streaming endpoint for push.”

Infra: Upload images to a public AWS ECR

“We currently publish to github (and mirror to DockerHub), but I'll raise an issue and discuss the possibility of uploading the images to a public ECR as well. _Originally posted by @Haarolean in TODO: - [x] public gallery ( - [x] workflow”

[MINOR][Test][Subtask] Validate and fix robust-ctl MQTT commands

“## Description Validate the functional `robust-ctl mqtt` command families against an isolated test broker, then fix only confirmed discrepancies. ## Coverage - Command discovery and help output for all MQTT subcommands. - Representative read and write actions across topics,…”

[MINOR][Test][Umbrella] Establish unit coverage baseline and PR coverage policy

“## Objective Establish a trustworthy LLVM-based coverage baseline for the repository's configured Rust unit-test scope, then add useful workspace, crate-level, and pull-request coverage views. This issue covers unit-test coverage only. Broker-process coverage and protocol-level…”