Open source Stitch alternatives

Amazon's exabyte-scale migration from Apache Spark to Ray on EC2

“I remember when everyone shifted from Apache Hadoop to Apache Spark. This seems like a possibly similar sea change. I am not sure if many other users will embrace Ray over Spark, but this is a sign that people are looking to either improve Spark on some fundamental levels, or…”

BlazingSQL – GPU SQL Engine Now Runs Over 20X Faster Than Apache Spark

“Further down in the article: In our original demo, we ran a V100 GPU against 8 CPU nodes. The new T4 GPUs cut our cost in half, which meant we reduced the Apache Spark cluster to 4 CPU nodes in order to maintain price parity. Even with the reduced GPU memory, the whole workload…”

Introducing Complex Event Processing (CEP) with Apache Flink

“For anyone wondering what CEP is... (from the blog) "in contrast to traditional DBMSs where a query is executed on stored data, CEP executes data on a stored query." And "Apache Flink with its true streaming nature and its capabilities for low latency as well as high throughput…”

Introducing Complex Event Processing (CEP) with Apache Flink

“I'm watching closely Apache Flink recently and it seems to me that it's more universal that Apache Spark, and with less overhead. Today if somebody asked me for general purpose stream processor engine I would recommend Flink. Of course Spark has many advantages too, but unless…”

Luigi Mangione's account has been renamed on Stack Overflow

“>causing users to publicly support the murder and voice support for future murders too. Sounds like society's problem, not SO. Put it another way, how many people would be checking on Luigi's account On SO in this way without this streisand effect it caused by banning the feww…”

Luigi Mangione's account has been renamed on Stack Overflow

“I agree that there's like a true crime fangirl sort that probably goes for Luigi either way just on looks maybe, but I don't think that would really be more than a marginal difference. There are still a fair variety Christopher Dorner shirts in circulation, for example, and he's…”

Show HN: CocoIndex – Open-Source Data framework for AI, built for data freshness

“Absolutely! CocoIndex is an ETL framework not limited to RAG. You can use it to do transformation as you need, including parsing documents and extracting information you need using your model or code. We have a similar example, and welcome to checkout:”

Show HN: CocoIndex – Open-Source Data framework for AI, built for data freshness

“Appreciate the feedback! we are adding more native built-ins, if there's anything i can help please feel free to open a ticket at our repo or shoot me a email [email protected]”

Show HN: ElectricSQL, Postgres to SQLite active-active sync for local-first apps

“Congratulations to James and the team! I reached out to James when they were searching for early adopters for their V2 version, which focuses on PostgreSQL. Our primary business involves developing Line of Business (LOB) applications. We decided to transform our LOB applications…”

Show HN: ElectricSQL, Postgres to SQLite active-active sync for local-first apps

“ElectricSQL stores additional data in the source PostgreSQL database, while PowerSync stores this in a separate data store in its sync service. Because of this, ElectricSQL requires schema changes and has some schema restrictions on the source database. ElectricSQL uses CRDTs…”

SeaTunnel Will Support CDC as a Feature Soon

“CDC is crucial to data synchronization, and we need SeaTunnel to support it as a feature. Here below is the reason why SeaTunnel needs this feature and my overall design for it.”

[Umbrella][Improve] Migrate Connector/Transform Validation to Declarative OptionRule — Claim & Tracking

“We need community help to migrate connector and transform validation from imperative `if/throw` checks to declarative `optionRule()` + `Conditions.*`. If the `Issue` column shows an existing issue number, please claim in that issue. If the `Issue` column shows `This issue`,…”

Michigan DOT Snowplow Names

“I read a dozen of these names to a friend just now and she said, "This is genius! They must have let the snowplow crews name their own plows. After all, who is going to take better care of their machine, the crew who drives Snowplow 437 or the crew who drives Clearopathra?" This…”

Michigan DOT Snowplow Names

“And rightly so - just about every year there's a news article about another pedestrian or cyclist killed by a snowplow here in Montreal.”

DataJet – Fully managed Apache NiFi as a service. Visual Data flows without code

“Use the best in class open source solution for managing and integrating data without the hassle of maintaining a secured, scalable cluster. Using DataJet you can build data flows that are as simple or complex and you like with no code. Join millions of data engineers and…”

DataJet – Fully managed Apache NiFi as a service. Visual Data flows without code

“I'm curious what domain you are using Apache NiFi for? I have used it before for analytical graph based pipelines. It is very capable but it definitely takes a different mindset to architect problems with NiFi.”

Jepsen: Radix DLT 1.0-Beta.35.1

“But they wrote it in the present tense, said it was a world record even ! And how can you say an earlier iteration will come in the future, that seems like a crypto-truth bending like we see a lot :s Admit it, they exaggerated and removed the statement to instead put it in a…”

Show HN: Dlt – Python library to automate the creation of datasets

“All of this sounds very appealing, both as a standalone tool and as a complement to some thing like Dbt. However the following seems like kind of an anti-feature, which I at least would want the option to disable: > You do not need to worry about the structure of a database or…”

Daft Punk Break Up

“What if I told you that both things are bad, and that I want the perpetrators of both crimes to face proportionate punishment? There's no reason for one's critique to fall victim to a false dichotomy here, even if the American political environment tries to do that. Maybe it…”

Daft Punk Break Up

“What is it about Daft Punk that gets one of the largest threads I've seen on HN? I find them utterly unremarkable. There's an absolute wealth of electronic music, mostly from the 90s, that Daft Punk is derived from. If you like them then check out Ishkur's Guide and explore more!”

Launch HN: Jitsu (YC S20) – Open-Source Segment Alternative

“Snowplow CEO here. We haven't used Jitsu before but are very familiar with Segment. It looks like Jitsu sits in the Segment product family, along with Rudderstack: basically a Customer Data Platform bundle of simple JSON event tracking, Fivetran-style transactional/SaaS data…”

Launch HN: Jitsu (YC S20) – Open-Source Segment Alternative

“Hi! My name is Sergey, I’m a Jitsu product engineer. I’ll gladly answer your question! AsyncLogger works asynchronously by design. There is a go channel which writes JSON’s to the log file. Answering your question: the service doesn’t wait until data is written to the log prior…”

Show HN: WhoDB – Streamlined, Lightweight Database Management with a Sleek UI/UX

“I agree - generalist UIs definitely won't be able to make full use of all specialized features. WhoDB is majorly created for fast developer experience when working with multiple DBs (or even for a single one). I think each database company usually has a specialized tool that…”

Show HN: WhoDB – Streamlined, Lightweight Database Management with a Sleek UI/UX

“Thanks - I will give it a try! Found some typo: You can run WhoDB with as little as 50m core and 100Mi RAM. WhoDB is also only ~20Mi compressed size.”

Add Support for PDF Input with OCR Capabilities

“## Objective Extend DocETL to support PDF files as input, including the ability to extract text using OCR for scanned documents or images within PDFs. ## Background Currently, DocETL supports structured data formats like JSON. Adding PDF support with OCR will greatly expand our…”

Add flexible support for other LLM providers

“First of all, many thanks for sharing this project! It's a much better execution of an idea we (University of Groningen, Center for Information Technology) had floating internally. We are currently serving models through vLLM for our research staff and this would be a great tool…”

[Bug][chunjun-connection-oraclelogminer] oraclelogminer to mysql have error java.lang.Long cannot be cast to java.lang.I

“### Search before asking - [X] I had searched in the [issues]( and found no similar issues. ### What happened oracle to mysql real-time sync error: java.lang.Long cannot be cast to java.lang.Integer ### What you expected to happen sync success ### How to reproduce oracle ddl :…”