Read now
Introducing Quill: the live context layer for AI →
QuillNEW
Get Started
QuillNEW
Decoration background
Posts
Blog
Implementing a file pager in Zig: Reclaiming memory

Implementing a file pager in Zig: Reclaiming memory

January 12, 2022

Up to this point, we focused on reading data from the disk, we can do that up to a point. Eventually we’ll run out of memory (assuming that the database is bigger than memory, which is a pretty safe assumption). That means that we need to decide what to remove from memory. When we use mmap(), the OS gets to decide that for us. In fact, that is…

Blog
Implementing a file pager in Zig: Reading from the disk

Implementing a file pager in Zig: Reading from the disk

January 11, 2022

We have finally gotten to the point where we can ask the pager for a page of data (reminder, a page in this case is 8KB of data) and get it back. Here is what this looks like: There are a few things to note here. There is a new concept here, the transaction. We don’t want to have that concept at the pager level, so we just pass the relevant…

Blog
Implementing a file pager in Zig: Managing the list of files

Implementing a file pager in Zig: Managing the list of files

January 10, 2022

This is my 7th post in this series, and we are now starting to get into the really interesting bits. So far we worked with the individual components, each of them doing just one thing, which makes it hard to see how they are all put together. In this post, we are going to start plugging things together. As a reminder, the whole point of this…

Blog
Teaching, learning and tearing your hair out

Teaching, learning and tearing your hair out

January 6, 2022

One of my favorite activities is teaching. I love explaining how things work and passing on knowledge. Another good way to pass the time is to learn, which can be a source of great joy and incredible frustration. Recently I had a conversation with a university student in the Computer Science track talking about databases. As you might imagine,…

Use Cases
An Agile NoSQL Database for Startups

An Agile NoSQL Database for Startups

January 5, 2022

RavenDB Equips Zero to MVP with Extreme Agility. Unless a client agrees to use RavenDB, I won’t take the client. We are more efficient that way. Georgios Diamantopoulos, founder of Zero to MVP, Inc. What do a foodie site for vegans in Abu Dhabi, a social network for employees to communicate with management, and a toll system for a maritime…

document databaseautomatic indexingagile database
Blog
Implementing a file pager in Zig: Reading & Writing from the disk

Implementing a file pager in Zig: Reading & Writing from the disk

January 5, 2022

After implementing the memory management in the previous post, I set out to handle the actual I/O primitives that we need. As a reminder, we are separating the concerns here. We managed memory and reference counting in the previous post and now I want to focus on how we can read and write from the disk in as efficient a manner as possible. Before…

Blog
Implementing a file pager in Zig: Rethinking my approach

Implementing a file pager in Zig: Rethinking my approach

January 4, 2022

After writing the post about handling chunk metadata, I started thinking about the overall approach. Both the method using compressed pointers and the baseline computation felt… off to me. They were certainly workable, but it was too complex and felt fragile. I don’t like dealing with a high level of complexity, I would rather put a lot of effort…

Blog
Production postmortem: An error on the first act will lead to data corruption on the second act…

Production postmortem: An error on the first act will lead to data corruption on the second act…

January 3, 2022

The topic of this post is a bug in RavenDB, a pretty serious one. The end result is that a user reported that they got an error from RavenDB that they are unable to read a stored document. In some cases, RavenDB needs to read a document on startup, which means that it wasn’t able to start up if that document had this behavior. As you can imagine,…

Blog
Code review horror in 4 lines of code

Code review horror in 4 lines of code

December 30, 2021

I run into the following code during code review and had an immediate and visceral reaction. This is a (bad) attempt to add thread safety, because you are getting a value through a read only interface, but there is still the mutable instance to work with at the source, and now you have someone that observes the instance while it is being…