Ability to output data changes via logical decoding (ex. wal2json)
PostgreSQL, since 9.4, has the ability to provide the list of changes made to a database in a manner that is transaction-safe and lossless.
To enable this feature, the wallevel needs to be set to logical and there needs to be an available output plugin. Unfortunately, PostgreSQL only comes with the testdecoding plugin, which was only really meant as an example.
AWS for example, has included wal2json support:
The simple wal2json output plugin would be a good way to provide access to this powerful feature for a wide audience.
I'd also be fine with other output plugins. It may be worth noting that there has been some discussion about including wal2json in the standard set of contrib plugins.
Looking forward to having some way to get this CDC data out of PostgreSQL with this great feature.
Logical decoding and the wal2json output plugin are now available in preview on Azure Database for PostgreSQL – single server. Please visit the documentation for more information: https://docs.microsoft.com/azure/postgresql/concepts-logical
Karunakar Goud commented
yes, in our project , we have this requirement of collecting DMLs using Debezium. Why Azure has restricted this feature while AWS has already supporting and provides inbuilt logical decoding plugin wal2json.
It would be great if we have this as soon as possible
Can we have an indication whether this feature is under consideration?
I don't see any reason why not have the ability to use the built in CDC feature of PostgreSQL especially when this is available in AWS already.
Without support for custom plug-ins, it'd be great to at least allow to use logical replication by means of the wal_sender and related properties. Then, with Postgres 10+, the pgoutput plug-in, as e.g. supported in Debezium 0.10 can be used to stream changes out of managed Postgres on Azure.
Jelmer van Amen commented
Any indication whether this is under consideration.
As per author's update this is the plugin: https://github.com/eulerto/wal2json
Patrick Earl commented
That should be wal2json, not wol2json.