What's new

Server

• [Backup] Added ability to cancel and delay server-wide backups by Operator
• [Certificates] Using UTC time when validating the certificate for developer license
• [Changes API] Added the timeout for sending a message and clearing messages in the queue when we're in low memory
• [Cluster] Fixed the maintenance of the order of nodes in a database group after a node gets online
• [Cluster] Using TCP compression for the cluster communication
• [Cluster] Fixed negotiation boundaries based on the negotiation PrevLogIndex
• [Configuration] Marking 'Logs.Microsoft.ConfigurationPath' as read only so it won't attempt to create a file in the default location
• [Corax] Fixed querying with order by
• [Corax] Memory usage and performance improvements
• [Debug] Fixed serialization of GC dump output
• [Debug] Fixed Debug Package created by for the whole cluster
• [ETL] Added getRevisionsCount() method available during execution of an ETL script
• [HTTP] Fixed HTTP response compression when performing lazy load
• [Indexing] Fixed edge cases in the Index Merger
• [Indexing] Fixed loading multiple attachments using LoadAttachments function with the usage of Select
• [JavaScript] Added support for spread operator JS indexes and Patching
• [Import] Added alerts when skipping configuration for a conflict solver, time series or revisions during the import because they already exist
• [Import] Fixed import of a CSV file with empty column names
• [Memory] Fixed an issue with allocating less than allocation unit
• [Memory] Purging the Lucene's field cache for order by when we are in low memory state
• [Memory] Fixed calculation of allocated memory in internal allocators
• [Memory] Added defragmentation mechanism to internal allocators to deal with specific memory allocation patterns
• [Performance] Improved Smuggler and Patching performance when handling documents with complex nested objects
• [Querying] Optimized order by queries when ordering by field which is a date
• [Querying] Added support for count(), key() and sum() methods in JavaScript projections when using dynamic group by queries
• [Querying] Fixed a bug where only identity RQL projection would work on document loaded from reference existing only in the index
• [Subscriptions] Sending noop ACK cluster command once per state
• [Smuggler] Added ability to import / export history of indexes
• [Time Series] Improved memory usage when processing large number of time series during ETL and indexing
• [Time Series] Fixed potential AccessViolationException on restoring time series from a backup file (regression introduced in 5.4.100)
• [Time Series] Improved memory usage during import of time series
• [.NET] Updated to .NET 7.0.2

Client

• [Cluster Transactions] Fixed throwing ClusterTransactionConcurrencyException after the failover when using cluster-wide transactions
• [Querying] Fixed 'declared function' chopped by IAsyncDocumentQueryBase.ToQueryable
• [Querying] Fixed comparison of field names for projections. We need to do it in case sensitive manner
• [Querying] Added AsAsyncEnumerable to RavenQueryableExtensions
• [Querying] Fixed direct Projection of Array from document
• [Serialization] Prevented ISerializable support by default in the serializer
• [Subscriptions] Fixed an issue that metadata was not tracked in the subscription session
• [Time Series] Deleting time series entry should affect the cache

Studio

• Added on/off switch to disable index optimize during compact operation
• Fixed column name on Index Errors view

Server

• [Backup] Fixed issue with a backup status after changing the type of a backup
• [Backup] Changed the order in which we perform the snapshot backup - first the indexes then the documents
• [Backup] Fixed issue with calculating the space required for snapshot backup if indexes were excluded
• [Backup] Fixed memory usage during backup with time series
• [Configuration] Added Indexing.MinimumTotalSizeOfJournalsToRunFlushAndSyncWhenReplacingSideBySideIndexInMb
• [Configuration] Added Http.KeepAlivePingTimeoutInSec and Http.KeepAlivePingTimeoutInSec
• [Corax] Query execution performance improvements
• [Corax] Support ticks in date queries
• [Corax] Introduced index-level boosting
• [Cluster] Fixed the format of atomic guard values used by Cluster Transactions and block using them via the session
• [Cluster] Improved handling of a database shutdown
• [Debug]Added info regarding the GC operation to /admin/memory/gc handler
• [ETL] Added ability to enable compatibility mode when sending documents to Elasticsearch
• [ETL] Fixed handling of scripts containing getMetadata() and delete behavior functions
• [ETL] Fixed high unmanaged memory usage when filtering out docs during ETL
• [Indexing] Fixed handling of Distinct() with mixed values of string and LazyStringValue types
• [Indexing] Improved handling of replacement of side-by-side index by waiting for journals to be flushed
• [Indexing] Fixing unhandled case in the index suggestions merger
• [Indexing] Fixing potential problems with JS or multi map-reduce indexes not preserving the order of hashing of reduce key values
• [Logging] Enhancements in the cluster logging to improve logs readability
• [Logging] Added schema upgrades progress to the init log
• [Logging] Added ability to get internal Microsoft logs (in particular Kestrel logs)
• [Logging] Added details to the PutIndexCommand logging
• [Monitoring] Fixed the issue that average request time stays on last value if traffic moves to an another node
• [Patching] Reduced JsString allocations in patching
• [Performance] Improved performance of putting a document
• [Performance] Improved performance of deletion of a document
• [Performance] Improved performance of scenario with document writes getting ConcurrenyException in the merged transaction
• [PostgreSQL] Fixed handling of fields with large values
• [PostgreSQL] Better schema generation if first item has null values
• [Querying] Fixed regression issue that we stopped detecting enums as server known type during indexing
• [Querying] Fixing WaitForNonStaleResults for Time Series and Counters
• [Revisions] Revert revision per collection
• [Replication] Fixed very high memory usage when replicating a big transaction and using encryption
• [Replication] Ensure replication is working from a node in rehabilitation
• [Smuggler] Added ability to import just one collection
• [Subscriptions] Subscription worder will continue to retry on DatabaseDisabledException and AllNodesTopologyDownException
• [TimeSeries] Improved memory usage when importing .ravendump with high number of time series to an encrypted database
• [TimeSeries] Fixed clone document with a rollup time series
• [.NET] Updated to .NET 7.0.1

Client API

• [Session] Making sure that the HiLo number we got is used with the server node tag it came from to avoid duplicate ids
• [Indexing] Prevent transforming additional method into extended method in index definition rewriter
• Added AggressivelyCacheForAync to avoid sync usage in async manner

Studio

• Fixed visibility of time series requests on studio dashboard
• Allow to cleanup tombstones from the studio
• Fixed showing of guids in related documents list
• Endpoint /databases became lighter to improve performance of databases view

Server

• [Attachments] prevent saving attachments inside a batch for documents with server-side generated IDs
• [Backup] optimized cluster transaction restore performance
• [Backup] added the ability to exclude indexes for Snapshot backup
• [Backup] after restore database will be automatically loaded to apply the journals
• [Backup] using 'Backup.TempPath' for one time backups
• [Backup] optimized memory during restore/import of Time Series to an encrypted database
• [Batch] fixed issue with waiting for indexes not working when batch is a cluster transaction
• [Corax] added support for Range facets
• [Cluster] fixed possibility of Cluster Transaction application hanging after Database is being initialized
• [Cluster] introduced dedicated commands for processing client/studio/settings configurations to avoid using database record updates all the time, which need much more processing than other commands
• [Certificates] allow to start the server with expired certificate to allow certificate refresh to take place
• [ETL] added support for providing SQL provider specific parameter types in SQL ETL scripts
• [Indexes] added support for file-scoped namespace declarations in additional sources
• [Indexes] fixed issue with optimize hanging during index compaction
• [Licensing] fixed issue throwing a 'LicenseLimitException' when asserting PostgreSQL license limit, even if license has this feature enabled
• [Multi Get] fixed issue with multi-get requests not respecting client timeout
• [PostgreSQL] disposing documents transaction properly after handling query
• [Querying] fixed IndexOutOfBoundsException when skip is greater than number of results in index after Distinct is used
• [Querying] expose @spatial metadata for JavaScript projections
• [Replication] delay replication on missing attachments
• [Voron] do not change valid pages if the page checksum or number is invalid
• [.NET] updated to .NET 6.0.11

Client API

• [Bulk Insert] throw NotSupportedException on attempt to insert Incremental Time Series
• [Events] invoke 'OnAfterConversionToEntity' event after entity has been tracked so methods like 'GetMatadataFor' can be used inside the event
• [Querying] added support for case-sensitive string comparisons in LINQ
• [Querying] fixed issue with SELECT clause containing unnecessary aliases for methods
• [Session] fixed possible NRE when updating compare exchange metadata in the session
• [Session] allow incrementing Time Series multiple times in the same session
• [Session] added Typed Incremental Time Series API
• [Subscriptions] forbid opening more than one session per each subscription batch

Studio

• [Cluster Dashboard] added average request time per database
• [Index Merge Suggestions] added view
• [Querying] removed dynamic fields from RQL code assistance
• [Thread Info] added websockets for live updates

Server

• [Backup] fixed issue with OLAP ETL not being disabled when 'Disable Ongoing Task' is set during restore
• [Backup] fixed issue with Elasticsearch not being disabled when 'Disable Ongoing Task' is set during restore
• [Backup] fixed issue with creating new server-wide backup with name identical to existing backup
• [Backup] restore of compare exchange should not only be count-based but also size based
• [Backup] restore of cluster transactions should not only be count-based but also size based
• [Certificates] using EphemeralKeySet and UserKeySet as a default key set instead of MachineKeySet with MachineKeySet as a failover
• [Cluster] take into account the state of compare exchange replication when calculating database group node state
• [Configuration] added 'Databases.DeepCleanupThresholdInMin'
• [Configuration] added 'Databases.RegularCleanupThresholdInMin'
• [Configuration] added 'Indexing.Lucene.Analyzers.NGram.MinGram' alias to 'Indexing.Analyzers.NGram.MinGram'
• [Configuration] added 'Indexing.Lucene.Analyzers.NGram.MaxGram' alias to 'Indexing.Analyzers.NGram.MaxGram'
• [Configuration] added 'Indexing.Lucene.MaximumSizePerSegmentInMb' alias to 'Indexing.MaximumSizePerSegmentInMb'
• [Configuration] added 'Indexing.Lucene.MergeFactor' alias to 'Indexing.MergeFactor'
• [Configuration] added 'Indexing.Lucene.LargeSegmentSizeToMergeInMb' alias to 'Indexing.LargeSegmentSizeToMergeInMb'
• [Configuration] added 'Indexing.Lucene.NumberOfLargeSegmentsToMergeInSingleBatch' alias to 'Indexing.NumberOfLargeSegmentsToMergeInSingleBatch'
• [Configuration] added 'Indexing.Lucene.MaxTimeForMergesToKeepRunningInSec' alias to 'Indexing.MaxTimeForMergesToKeepRunningInSec'
• [Configuration] added 'Indexing.OrderByScoreAutomaticallyWhenBoostingIsInvolved'
• [Configuration] added 'Indexing.Lucene.UseCompoundFileInMerging' and 'Indexing.UseCompoundFileInMerging'
• [Configuration] added 'Indexing.Lucene.IndexInputType'
• [ETL] added the ability to define document ID postfix
• [Indexes] fixed race condition when handling nested references
• [Indexes] adjustments in index merge suggestions mechanism
• [Memory] fixed memory leak that could happen for large allocations, especially when compression is used (e.g. Revisions). This should reduce the memory footprint needed for processing compressed documents e.g. during backup operation
• [Monitoring] added 'AvailableMemoryForProcessing' to SNMP
• [PAL] fixed issue with writing large transactions to journals on Posix (over 2GB)
• [Querying] reduced Lucene processing time by buffering the input resulting in reducing the overall querying time
• [Querying] fixed issue with CSV export throwing when JavaScript projection is used
• [Time Series] fixed issue with start and end range being in different TZ when getting range stats
• [Voron] fixed faulty error handling during the recovery of storage environment that could happen if recyclable journal is being used
• [Voron] fixed issue with updating headers when partial recovery is being done
• [Voron] fixed issue with gathering data needed to display storage report in Studio
• [.NET] updated .NET to 6.0.9

Client API

• [Conventions] added 'HttpPooledConnectionLifetime' and 'HttpPooledConnectionIdleTimeout' conventions
• [HTTP] limit number of HttpClient recreations during error handling to 1 per 5 seconds
• [HTTP] immediately recreate HttpClient after getting 'SocketError.TryAgain'
• [Querying] fixed issue with LINQ 'As<>' not taking into account the projection used
• [Querying] added TimeOnly DateOnly support for single projections
• [Querying] better 'CompareTo' support on strings in LINQ
• [Session] fixed issue with Compare Exchange metadata not marking values as dirty so they are ignored during 'SaveChanges'

Studio

• [Certificates] Allow to renew client certificate 30 days before expiration
• [Configuration] added read-only view for server-level configuration
• [Dashboard] added disk metrics to Cluster Dashboard
• [Documents] fixed issue in collection tracker that was detecting changes in collection (an displaying 'Your data has changed' notification) despite nothing being changed
• [Notifications] Merging 'DeleteByQuery' notifications into one to avoid flooding the notification center
• [Traffic Watch] added average response size to stats
• [Traffic Watch] added the ability to save to file

Other

• [Test Driver] setting 'DisableTopologyCache' to 'true' by default for the DocumentStore created by the driver

Server

• [Corax] Performance improvements during indexing
• [Corax] Improved execution of range queries
• [Cluster] Fixed infinite leader/follower negotiation
• [Cluster] Fixed incapability to expand database group
• [Compression] Fix potential usage of disposed buffer in ZstdStream
• [Compression] Suppressing security check for Zstd native calls for better performance
• [Core] Fixed possible leak of an instance of a document database that could lead to data corruption or errors with accessing db.lock file
• [Debug] Fixed dump creation via /admin/debug/gcdump
• [Indexing] Support for creation of dynamic and spatial fields on arrays inside JavaScript indexes
• [Indexing] Added a performance hint when fanout index is outputting a source document
• [Indexing] Added a performance hint warning about too many usages of 'let' in index definition
• [Indexing] Fixed side-by-side index duplication after server restart
• [Indexing] Fixed handling of out of disk space errors
• [Indexing] Improved memory usage calculations on Linux for big indexes
• [Memory] Fixed native memory allocation when in specific case of fragmentation
• [Querying] Fixed performance of collection query with startsWith()
• [Setup] Allow to create a setup package for unsecured setup via Setup Wizard
• [Subscriptions] Fixed problem with a subscription not sending over new docs after restart of responsible node
• [.NET] updated to .NET 6.0.8

Client

• Improved error messages in case of broken responses
• [Session] Fixed marking document as dirty if only read operations or no-op operations were performed on metadata
• [Session] Added ability to include counters, time series when including revisions
• [Session] Fixed unexpected behavior for saving same entity with different id
• [Querying] Support for more complex scenarios when querying on a dictionary

Studio

• [Cluster Dashboard] Added details per database to Ongoing Tasks widget
• [Cluster Dashboard] Added Elasticsearch ETL to Ongoing Tasks widget
• [Cluster Dashboard] Added Kafka and RabbitMQ ETLs to Ongoing Tasks widget
• [Cluster Dashboard] Added configuration option to scale disk size based on its size
• [Cluster Dashboard] Added server and OS info to Cluster Overview widget
• [Ongoing tasks] Added Hub task details
• [Traffic Watch] Added a request size
• [Index Terms] Fixed querying on complex term objects
• [Backup Task] Fixed testing credentials functionality
• Fixed and enhancements

Tools

• Replay traffic watch tool for queries