Settings: ETL (Extract Transform Load)

Here you can edit the following ETL settings:

You can choose Skip index replication for all destinations to configure it globally instead of per destination. Figure 1. Settings. ETL. First Screen.

Adding an ETL destination

Adding an ETL destination is quite similar to replication setup, with a few key differences:
- There is no Client failover behavior and Conflict resolution in ETL
- A collection (to replicate from) must be chosen

Simple ETL can be configured by choosing the source collection and providing the destination URL and Database.

Figure 2. Settings. ETL. Add Destionation. Figure 3. Settings. ETL. Choose Collection.

Defining Transformation and Filtering scripts

If you want to manipulate the replicated documents in a custom way, you can easily define transformation and filtering. There is an option to write a JS function (examples) for each collection/destination, which means that the script will run on every replicated document from that collection.

Figure 4. Settings. ETL. Defining Transformation and Filtering scripts.


The deletion of documents (tombstones) will be replicated to their destination without applying the transformation script on them. It means that they won't be filtered out and all deletes will be processed as usual.

Advanced Options

Some advanced options you can use are:
- Force replicate all indexes and transformers
- Skip Index Replication - for the current destination
- Transitive Replication - marks what document types should be replicated:
  - Changed only - locally
  - Changed and replicated - from other sources

Warning: Failover behavior

An important consideration with ETL is that because the data is filtered and possibly transformed, a destination that is using this feature isn't a viable fall-back target, and it will not be considered as such by the client. If you want failover, you need to have multiple replicas, some with the full data set and some with the filtered data.

Figure 5. Settings. ETL. Advanced Options.