Ongoing Tasks Stats: OLAP ETL Stats

OLAP ETL Stats Closed View

OLAP ETL Stats Closed View

1. Task Type
   Click the arrow or the task type to toggle Closed/Expanded View.
2. Task Name
3. Transform Script
   Click to display the ETL transform script.
4. Task Bar
   • Hover over the bar to display a tooltip with the task's information.
   • Click the bar for the expanded view.
   • Click and Drag the bar to slide the graph.
   • Zoom in & out using the mouse wheel.

OLAP ETL Stats Expanded View

OLAP ETL Stats Expanded View (1)

• OLAP ETL

  • Total Duration
    Overall ETL operation time, including the Extraction, Transformation, and Loading phases.
  • Batch Complete Reason
    The reason the operation ended, e.g. -
    "No items to process",
    "Stopping the batch because maximum batch size limit was reached",
    and others.
  • Currently Allocated
    The amount of memory allocated by the server to handle this task.
  • Batch Size
    Size of ETL batches.

• Extract Phase

  • Duration
    The time it took to extract the documents from the database.
  • Extracted Documents
    The number of documents extracted from the database.

• Transform Phase

  • Duration
    The time it took to process the documents using the transform script.
  • Transformed Documents
    The number of documents processed by the transform script.
  • Documents Processing Speed
    Average Doc/Sec transformation speed.
  • Last Transformed Etag for Document
    Last transformed document's identifier.

• Load Phase

  • Duration
    The time it took to transfer the documents to their destination.
  • Successfully Loaded
    Documents transfer verification.
  • Last Loaded Etag
    Last loaded document's identifier.

OLAP ETL Stats Expanded View (2)

• Load/Local

  • Duration
    The time it took to store all documents on the local storage.
    

    The documents are stored locally before they are transferred to a remote destination.

• Load/Local/X

  • Duration
    The time it took to store document X on the local storage.

• Load/Upload

  • Duration
    The time it took to upload all documents to the remote destination.
    

    The Load/Upload bars will not be displayed if the destination is set to local storage only.
    

• Load/Upload/X

  • Duration
    The time it took to upload document X to the remote destination.

  • Parquet File Name on the remote destination, constructed of -

    • Upload date and time
    • Document database name
    • OLAP ETL task name
    • ETL script name
  • File Upload Properties per Destintion Type, including -
    • Upload State
    • Upload Type
    • File Size
    • Upload Speed