RabbitMQ Queue Sink Task
-
RabbitMQ brokers are designed to disperse data to multiple queues, making for a flexible data channeling system that can easily handle complex message streaming scenarios.
-
RavenDB can collaborate with message brokers like RabbitMQ both as a producer, by running ETL tasks), and as a consumer, using a sink task to consume enqueued messages.
-
To use RavenDB as a consumer, define an ongoing Queue Sink Task. Sink tasks can read batches of JSON formatted messages from RabbitMQ queues, construct documents using user-defined scripts, and store the documents in RavenDB collections.
-
This page explains how to create a RabbitMQ sink task using the Studio.
Learn more about RavenDB queue sinks here.
Learn how to define a RabbitMQ queue sink using the API here. -
In this page:
Add a Database Task
To open the ongoing tasks view:
Ongoing Tasks
- Ongoing Tasks
Click to open the ongoing tasks view. - Add a Database Task
Click to create a new ongoing task. -
Info Hub
Click for usage and licensing assistance.Info Hub
Task Selection
- Click to create a RabbitMQ sink task.
Define a RabbitMQ Sink Task
New RabbitMQ Sink
-
Save to store the configuration and exit. If the task is enabled it will start running.
Cancel to revoke the creation of a new task or the changes made to an existing task. -
Task Name (Optional)
- Enter a name for your task
- If no name is provided, RavenDB will create a name based on the defined connection string,
e.g. Queue Sink to RabbitMqTaskConStr
-
Task State
Select the task state:
Enabled - The task runs in the background, reading, manipulating, and storing data as defined in this view.
Disabled - No documents are read or stored, and the task's script is inactive. -
Responsible Node (Optional)
- Select a node from the Database Group to be responsible for this task.
- If no node is selected, the cluster will assign a responsible node (see Members Duties).
-
Create a new RabbitMQ connection String
The connection string defines the source RabbitMQ brokers URLs.
Enable to create a new connection string, or leave disabled to select an existing string.RabbitMQ Connection String
- A. Name - Enter a name for the new connection string.
- B. Connection string (format:
amqp://guest:guest@localhost:5672/
)
-
Test Connection
Click after defining the connection string, to test the connection to the RabbitMQ broker.Successful Connection
-
Scripts Click Add Script to add the task a new script.
Edit or Delete any existing script from the list.
Define and Test Task Scripts
Task Script Editor
-
Name: Name the script, or leave it for the task to name it (e.g.
Script_1
). -
Syntax: Click for assistance and sample scripts.
-
Script: Edit the script.
-
Source Queues
Enter the name of at least one RabbitMQ queue that resides on the broker/s the task connects, and click Add Queue to add it to the queues list. -
Add a new script to the list of scripts that this task runs, or Update an existing script (a different button appears for existing and new scripts).
-
Cancel to revoke a new script or any changes made in an existing one.
-
Test Script
Click to test your script in a test area without actually loading data from queues or storing documents in RavenDB.Test Script
- A. Script - Edit your script here to find the version that produces the documents you want.
- B. Message - write here any message you want in JSON format. Your script will be applied to this message so you can check the outcome.
- C. Test to run the test, or Close Test Area to return to the task editing view.
- D. Documents - The documents created when the test is executed.
Task Statistics
Sink statistics are added to RavenDB's ongoing tasks stats view, where their performance can be examined from various aspects. To watch these statistics, enter Studio's ongoing tasks stats view.
Queue Sink Stats
- RabbitMQ sink task statistics
All statistics related to the sink task.
Click the bars to expand or collide statistics.
Hover over bar sections to expose statistics. -
Sink statistics
- Total duration
The time it took to get a batch of documents (in MS) - Currently allocated
The memory allocated for the task (in MB) - Number of processed messages
The number of messages that were recognized and processed - Number of read messages
The number of messages that were actually transferred to the database - Successfully processed
Has this batch of messages been fully processed (yes/no)
- Total duration
- Queue readings
Time duration of reading from RabbitMQ queues (in MS) - Script processing
Time duration of script processing (in MS)