Skip to Navigation
Settings: ETL (Extract Transform Load)
Here you can edit the following ETL settings:
You can choose Skip index replication for all destinations to configure it globally instead of per destination.
Adding an ETL destination
Adding an ETL destination is quite similar to replication setup, with a few key differences:
- There is no Client failover behavior and Conflict resolution in ETL
- A collection (to replicate from) must be chosen
Simple ETL can be configured by choosing the source collection and providing the destination URL and Database.
Defining Transformation and Filtering scripts
If you want to manipulate the replicated documents in a custom way, you can easily define transformation and filtering. There is an option to write a JS function (examples) for each collection/destination, which means that the script will run on every replicated document from that collection.
Note
The deletion of documents (tombstones) will be replicated to their destination without applying the transformation script on them. It means that they won't be filtered out and all deletes will be processed as usual.
Advanced Options
-
Force replicate all indexes and transformers -
Skip Index Replication - for the current destination -
Transitive Replication - marks what document types should be replicated: - Changed only - locally
- Changed and replicated - from other sources
Warning: Failover behavior
An important consideration with ETL is that because the data is filtered and possibly transformed, a destination that is using this feature isn't a viable fall-back target, and it will not be considered as such by the client. If you want failover, you need to have multiple replicas, some with the full data set and some with the filtered data.
Related Articles