What's new

Server

• [Backups] fixed possible desync of backup status between nodes
• [Backups] fixed issue with saving a server-wide backup when any of the database is idle
• [Backups] during Restore we should not hold read transaction (held by atomic guards) for a long time to decrease scratch file size
• [Backups] fixed potential NRE when importing old dump file without Indexes History
• [Changes API] created optimized channel for aggressive cache usage
• [Configuration] added 'Security.WellKnownIssuers.Admin' configuration
• [Debug] added the ability to collect debug package only for server or per database
• [ETL] added stop reason to OLAP ETL statistics
• [ETL] fixed issue with disabling an ETL task when in fallback mode
• [ETL] fixed 'Failed to parse' issue when registering behavior function in a script engine when script contains GlobalObject
• [Indexes] fixed regression and using a different approach to fix potential problems with JS or multi map-reduce indexes not preserving the order of hashing of reduce key values
• [Indexes] fixed regression with proper detection of dynamic fields in JavaScript indexes
• [Indexes] added certificate info to index history view
• [Indexes] added support for .NET 7 'TimeOnly' increased precision
• [Indexes] fixed issue with extraction of collection name for more index types
• [Logging] added the ability to enable Kestrel logs
• [Memory] releasing allocated compressed value when deleting a document with a lot of revisions
• [Memory] added Zswap and Zswapped to memory info reader on Linux due to 'top' displaying only first 15 chars
• [Memory] fixed possible memory leak when storage compaction is being used
• [Querying] fixed issue with timings not working when metadata comes from query metadata cache
• [Querying] fixed issue with accessing a disposed object when (id) when 'load' is used in RQL and value is used during projection more than once
• [Querying] fixed internal cache issue which could result in returning invalid query results for certain query types
• [Replication] when entire Time Series is being deleted, we should not update the document's metadata to avoid creating a conflict on the document
• [Replication] when document with Time Series is being deleted, we should create a DeletedRange to prevent possible inconsistency between Time Series on database nodes
• [Setup] added Ubuntu 22.04 DEB package
• [Threading] using shorter thread names on Linux
• [.NET] updated to 7.0.4

Client API

• [Conventions] added 'CreateHttpClient' and 'HttpClientType' conventions
• [Changes API] optimized changes API for aggressive cache usage (requires both server and client update)
• [JSON] added support for .NET 7 'TimeOnly' increased precision
• [JSON] added 'DefaultRavenSerializationBinder' which will throw for known .NET RCE Gadgets when they are used in '$type' field and deserialization needs to use value from this field to find a type
• [Querying] fixed issue with generation of JS projection when '.ToDictionary' is used in LINQ select
• [Querying] fixed regression that did not allow to select after project into
• [Session] added Refresh(IEnumerable<T> entities) overload
• [Subscriptions] fixed possible memory leak in subscription worker when we fail to process incoming batch

Studio

• [Backups] sending appropriate value when Delay is executed
• [Dashboard] fixed issue with graphs not displaying time when hovered
• [Replication] disabled sink to hub replication in the UI when server is unsecured
• [Memory] exposed total SWAP and SWAP used in the dashboard

Other

• [Docker] added Windows LTSC 2022 docker images

Server

• [Backup] Added ability to cancel and delay server-wide backups by Operator
• [Certificates] Using UTC time when validating the certificate for developer license
• [Changes API] Added the timeout for sending a message and clearing messages in the queue when we're in low memory
• [Cluster] Fixed the maintenance of the order of nodes in a database group after a node gets online
• [Cluster] Using TCP compression for the cluster communication
• [Cluster] Fixed negotiation boundaries based on the negotiation PrevLogIndex
• [Configuration] Marking 'Logs.Microsoft.ConfigurationPath' as read only so it won't attempt to create a file in the default location
• [Corax] Fixed querying with order by
• [Corax] Memory usage and performance improvements
• [Debug] Fixed serialization of GC dump output
• [Debug] Fixed Debug Package created by for the whole cluster
• [ETL] Added getRevisionsCount() method available during execution of an ETL script
• [HTTP] Fixed HTTP response compression when performing lazy load
• [Indexing] Fixed edge cases in the Index Merger
• [Indexing] Fixed loading multiple attachments using LoadAttachments function with the usage of Select
• [JavaScript] Added support for spread operator JS indexes and Patching
• [Import] Added alerts when skipping configuration for a conflict solver, time series or revisions during the import because they already exist
• [Import] Fixed import of a CSV file with empty column names
• [Memory] Fixed an issue with allocating less than allocation unit
• [Memory] Purging the Lucene's field cache for order by when we are in low memory state
• [Memory] Fixed calculation of allocated memory in internal allocators
• [Memory] Added defragmentation mechanism to internal allocators to deal with specific memory allocation patterns
• [Performance] Improved Smuggler and Patching performance when handling documents with complex nested objects
• [Querying] Optimized order by queries when ordering by field which is a date
• [Querying] Added support for count(), key() and sum() methods in JavaScript projections when using dynamic group by queries
• [Querying] Fixed a bug where only identity RQL projection would work on document loaded from reference existing only in the index
• [Subscriptions] Sending noop ACK cluster command once per state
• [Smuggler] Added ability to import / export history of indexes
• [Time Series] Improved memory usage when processing large number of time series during ETL and indexing
• [Time Series] Fixed potential AccessViolationException on restoring time series from a backup file (regression introduced in 5.4.100)
• [Time Series] Improved memory usage during import of time series
• [.NET] Updated to .NET 7.0.2

Client

• [Cluster Transactions] Fixed throwing ClusterTransactionConcurrencyException after the failover when using cluster-wide transactions
• [Querying] Fixed 'declared function' chopped by IAsyncDocumentQueryBase.ToQueryable
• [Querying] Fixed comparison of field names for projections. We need to do it in case sensitive manner
• [Querying] Added AsAsyncEnumerable to RavenQueryableExtensions
• [Querying] Fixed direct Projection of Array from document
• [Serialization] Prevented ISerializable support by default in the serializer
• [Subscriptions] Fixed an issue that metadata was not tracked in the subscription session
• [Time Series] Deleting time series entry should affect the cache

Studio

• Added on/off switch to disable index optimize during compact operation
• Fixed column name on Index Errors view

Server

• [Backup] Fixed issue with a backup status after changing the type of a backup
• [Backup] Changed the order in which we perform the snapshot backup - first the indexes then the documents
• [Backup] Fixed issue with calculating the space required for snapshot backup if indexes were excluded
• [Backup] Fixed memory usage during backup with time series
• [Configuration] Added Indexing.MinimumTotalSizeOfJournalsToRunFlushAndSyncWhenReplacingSideBySideIndexInMb
• [Configuration] Added Http.KeepAlivePingTimeoutInSec and Http.KeepAlivePingTimeoutInSec
• [Corax] Query execution performance improvements
• [Corax] Support ticks in date queries
• [Corax] Introduced index-level boosting
• [Cluster] Fixed the format of atomic guard values used by Cluster Transactions and block using them via the session
• [Cluster] Improved handling of a database shutdown
• [Debug]Added info regarding the GC operation to /admin/memory/gc handler
• [ETL] Added ability to enable compatibility mode when sending documents to Elasticsearch
• [ETL] Fixed handling of scripts containing getMetadata() and delete behavior functions
• [ETL] Fixed high unmanaged memory usage when filtering out docs during ETL
• [Indexing] Fixed handling of Distinct() with mixed values of string and LazyStringValue types
• [Indexing] Improved handling of replacement of side-by-side index by waiting for journals to be flushed
• [Indexing] Fixing unhandled case in the index suggestions merger
• [Indexing] Fixing potential problems with JS or multi map-reduce indexes not preserving the order of hashing of reduce key values
• [Logging] Enhancements in the cluster logging to improve logs readability
• [Logging] Added schema upgrades progress to the init log
• [Logging] Added ability to get internal Microsoft logs (in particular Kestrel logs)
• [Logging] Added details to the PutIndexCommand logging
• [Monitoring] Fixed the issue that average request time stays on last value if traffic moves to an another node
• [Patching] Reduced JsString allocations in patching
• [Performance] Improved performance of putting a document
• [Performance] Improved performance of deletion of a document
• [Performance] Improved performance of scenario with document writes getting ConcurrenyException in the merged transaction
• [PostgreSQL] Fixed handling of fields with large values
• [PostgreSQL] Better schema generation if first item has null values
• [Querying] Fixed regression issue that we stopped detecting enums as server known type during indexing
• [Querying] Fixing WaitForNonStaleResults for Time Series and Counters
• [Revisions] Revert revision per collection
• [Replication] Fixed very high memory usage when replicating a big transaction and using encryption
• [Replication] Ensure replication is working from a node in rehabilitation
• [Smuggler] Added ability to import just one collection
• [Subscriptions] Subscription worder will continue to retry on DatabaseDisabledException and AllNodesTopologyDownException
• [TimeSeries] Improved memory usage when importing .ravendump with high number of time series to an encrypted database
• [TimeSeries] Fixed clone document with a rollup time series
• [.NET] Updated to .NET 7.0.1

Client API

• [Session] Making sure that the HiLo number we got is used with the server node tag it came from to avoid duplicate ids
• [Indexing] Prevent transforming additional method into extended method in index definition rewriter
• Added AggressivelyCacheForAync to avoid sync usage in async manner

Studio

• Fixed visibility of time series requests on studio dashboard
• Allow to cleanup tombstones from the studio
• Fixed showing of guids in related documents list
• Endpoint /databases became lighter to improve performance of databases view

Server

• [Attachments] prevent saving attachments inside a batch for documents with server-side generated IDs
• [Backup] optimized cluster transaction restore performance
• [Backup] added the ability to exclude indexes for Snapshot backup
• [Backup] after restore database will be automatically loaded to apply the journals
• [Backup] using 'Backup.TempPath' for one time backups
• [Backup] optimized memory during restore/import of Time Series to an encrypted database
• [Batch] fixed issue with waiting for indexes not working when batch is a cluster transaction
• [Corax] added support for Range facets
• [Cluster] fixed possibility of Cluster Transaction application hanging after Database is being initialized
• [Cluster] introduced dedicated commands for processing client/studio/settings configurations to avoid using database record updates all the time, which need much more processing than other commands
• [Certificates] allow to start the server with expired certificate to allow certificate refresh to take place
• [ETL] added support for providing SQL provider specific parameter types in SQL ETL scripts
• [Indexes] added support for file-scoped namespace declarations in additional sources
• [Indexes] fixed issue with optimize hanging during index compaction
• [Licensing] fixed issue throwing a 'LicenseLimitException' when asserting PostgreSQL license limit, even if license has this feature enabled
• [Multi Get] fixed issue with multi-get requests not respecting client timeout
• [PostgreSQL] disposing documents transaction properly after handling query
• [Querying] fixed IndexOutOfBoundsException when skip is greater than number of results in index after Distinct is used
• [Querying] expose @spatial metadata for JavaScript projections
• [Replication] delay replication on missing attachments
• [Voron] do not change valid pages if the page checksum or number is invalid
• [.NET] updated to .NET 6.0.11

Client API

• [Bulk Insert] throw NotSupportedException on attempt to insert Incremental Time Series
• [Events] invoke 'OnAfterConversionToEntity' event after entity has been tracked so methods like 'GetMatadataFor' can be used inside the event
• [Querying] added support for case-sensitive string comparisons in LINQ
• [Querying] fixed issue with SELECT clause containing unnecessary aliases for methods
• [Session] fixed possible NRE when updating compare exchange metadata in the session
• [Session] allow incrementing Time Series multiple times in the same session
• [Session] added Typed Incremental Time Series API
• [Subscriptions] forbid opening more than one session per each subscription batch

Studio

• [Cluster Dashboard] added average request time per database
• [Index Merge Suggestions] added view
• [Querying] removed dynamic fields from RQL code assistance
• [Thread Info] added websockets for live updates

Server

• [Backup] fixed issue with OLAP ETL not being disabled when 'Disable Ongoing Task' is set during restore
• [Backup] fixed issue with Elasticsearch not being disabled when 'Disable Ongoing Task' is set during restore
• [Backup] fixed issue with creating new server-wide backup with name identical to existing backup
• [Backup] restore of compare exchange should not only be count-based but also size based
• [Backup] restore of cluster transactions should not only be count-based but also size based
• [Certificates] using EphemeralKeySet and UserKeySet as a default key set instead of MachineKeySet with MachineKeySet as a failover
• [Cluster] take into account the state of compare exchange replication when calculating database group node state
• [Configuration] added 'Databases.DeepCleanupThresholdInMin'
• [Configuration] added 'Databases.RegularCleanupThresholdInMin'
• [Configuration] added 'Indexing.Lucene.Analyzers.NGram.MinGram' alias to 'Indexing.Analyzers.NGram.MinGram'
• [Configuration] added 'Indexing.Lucene.Analyzers.NGram.MaxGram' alias to 'Indexing.Analyzers.NGram.MaxGram'
• [Configuration] added 'Indexing.Lucene.MaximumSizePerSegmentInMb' alias to 'Indexing.MaximumSizePerSegmentInMb'
• [Configuration] added 'Indexing.Lucene.MergeFactor' alias to 'Indexing.MergeFactor'
• [Configuration] added 'Indexing.Lucene.LargeSegmentSizeToMergeInMb' alias to 'Indexing.LargeSegmentSizeToMergeInMb'
• [Configuration] added 'Indexing.Lucene.NumberOfLargeSegmentsToMergeInSingleBatch' alias to 'Indexing.NumberOfLargeSegmentsToMergeInSingleBatch'
• [Configuration] added 'Indexing.Lucene.MaxTimeForMergesToKeepRunningInSec' alias to 'Indexing.MaxTimeForMergesToKeepRunningInSec'
• [Configuration] added 'Indexing.OrderByScoreAutomaticallyWhenBoostingIsInvolved'
• [Configuration] added 'Indexing.Lucene.UseCompoundFileInMerging' and 'Indexing.UseCompoundFileInMerging'
• [Configuration] added 'Indexing.Lucene.IndexInputType'
• [ETL] added the ability to define document ID postfix
• [Indexes] fixed race condition when handling nested references
• [Indexes] adjustments in index merge suggestions mechanism
• [Memory] fixed memory leak that could happen for large allocations, especially when compression is used (e.g. Revisions). This should reduce the memory footprint needed for processing compressed documents e.g. during backup operation
• [Monitoring] added 'AvailableMemoryForProcessing' to SNMP
• [PAL] fixed issue with writing large transactions to journals on Posix (over 2GB)
• [Querying] reduced Lucene processing time by buffering the input resulting in reducing the overall querying time
• [Querying] fixed issue with CSV export throwing when JavaScript projection is used
• [Time Series] fixed issue with start and end range being in different TZ when getting range stats
• [Voron] fixed faulty error handling during the recovery of storage environment that could happen if recyclable journal is being used
• [Voron] fixed issue with updating headers when partial recovery is being done
• [Voron] fixed issue with gathering data needed to display storage report in Studio
• [.NET] updated .NET to 6.0.9

Client API

• [Conventions] added 'HttpPooledConnectionLifetime' and 'HttpPooledConnectionIdleTimeout' conventions
• [HTTP] limit number of HttpClient recreations during error handling to 1 per 5 seconds
• [HTTP] immediately recreate HttpClient after getting 'SocketError.TryAgain'
• [Querying] fixed issue with LINQ 'As<>' not taking into account the projection used
• [Querying] added TimeOnly DateOnly support for single projections
• [Querying] better 'CompareTo' support on strings in LINQ
• [Session] fixed issue with Compare Exchange metadata not marking values as dirty so they are ignored during 'SaveChanges'

Studio

• [Certificates] Allow to renew client certificate 30 days before expiration
• [Configuration] added read-only view for server-level configuration
• [Dashboard] added disk metrics to Cluster Dashboard
• [Documents] fixed issue in collection tracker that was detecting changes in collection (an displaying 'Your data has changed' notification) despite nothing being changed
• [Notifications] Merging 'DeleteByQuery' notifications into one to avoid flooding the notification center
• [Traffic Watch] added average response size to stats
• [Traffic Watch] added the ability to save to file

Other

• [Test Driver] setting 'DisableTopologyCache' to 'true' by default for the DocumentStore created by the driver