Publish the vdir specification as a standalone spec

content/_index.md

@@ -4,3 +4,11 @@ pimutils is a suite of programs for managing contacts, calendars and tasks on th
 * [todoman](https://github.com/pimutils/todoman) is a command line interface tasks manager
 * [vdirsyncer](https://github.com/pimutils/vdirsyncer) can synchronize calendar data, including data from khal and todoman
 * [pimsync](https://pimsync.whynothugo.nl/) is a re-implementation of vdirsyncer addressing outstanding issue and limitations.
+
+These tools are build around existing standards formats and protocols.
+The `vdir` storage format also builds atop existing standards:
+
+* [vdir] is a convention for storing calendars and address books in a
+  filesystem.
+
+[vdir]: http://localhost:1313/specs/vdir/

content/specs/vdir.md

@@ -0,0 +1,159 @@
+---
+layout: post
+title: The Vdir storage format
+date: 2025-03-31
+categories: specs
+---
+
+This specification describes a convention for storing calendars and address
+books in a filesystem, with the main goal of being ease of implementation, and
+compatibility with existing standards.
+
+# Basic Structure
+
+A main (root) directory contains an arbitrary number of subdirectories. Each
+subdirectory is a collection (an address book or a calendar). Each of these
+collections contains items, which may be contacts or calendar components
+(events, todos and/or journal entries) respectively.
+
+An item may only be one of the following two types:
+
+- A [vCard] file, in which case the file extension MUST be `.vcf`.
+- An [iCalendar] file, in which case the file extension MUST be `.ics`.
+
+[vCard]: https://tools.ietf.org/html/rfc6350
+[iCalendar]: https://tools.ietf.org/html/rfc5545
+
+An item SHOULD contain a `UID` property as described by the vCard and iCalendar
+specifications. If it contains more than one `UID` property, the values of
+those MUST NOT differ.
+
+The file MUST contain exactly one event, task or contact. In most cases this
+also implies only one `VEVENT`/`VTODO`/`VCARD` component per file, but
+e.g.  recurrence exceptions require multiple `VEVENT` components per event.
+
+## Filenames
+
+The filename should have similar properties as the `UID` of the file content.
+However, there is no requirement for these two to be the same. Programs may
+choose to store additional metadata in that filename, however, at the same time
+they MUST NOT assume that the metadata they included will be preserved by other
+programs.
+
+Filenames SHOULD be URL-safe, such that can be retained when synchronising to
+WebDAV collections.
+
+# Metadata
+
+Any following metadata files may optionally be present. None of the files
+listed below have any file extensions.
+
+- A file called `color` inside the vdir indicates that collection's color, a
+  property that is only relevant in UI design.
+
+  Its content is an ASCII-encoded hex-RGB value of the form `#RRGGBB`. For
+  example, a file content of `#FF0000` indicates that the vdir has a red
+  (user-visible) color. No short forms or informal values such as `red` (as
+  known from CSS, for example) are allowed. The prefixing `#` MUST be
+  present.
+
+- A file called `displayname` contain a UTF-8 encoded label description that
+  is suitable for presentation to the user. This property is a counterpart to
+  the `DAV:displayname` WebDAV property defined in [rfc4918 section 15.2].
+
+[rfc4918 section 15.2]: https://www.rfc-editor.org/rfc/rfc4918#section-15.2
+
+- A `description` contain a UTF-8 encoded description, which provides a
+  human-readable description of a calendar or an address book. This property is
+  a counterpart to the [`CALDAV:calendar-description`][ccd] and
+  [`CARDDAV:addressbook-description`][cad] properties in CalDAV and CardDAV
+  respectively.
+
+[ccd]: https://www.rfc-editor.org/rfc/rfc4791#section-5.2.1
+[cad]: https://www.rfc-editor.org/rfc/rfc6352#section-6.2.1
+
+- A file called `order` inside the vdir includes the relative order of a
+  collection. This property is only use for presentational purposes. Its value
+  shall be a numeric string, and collections shall be sorted relative to this
+  value.
+
+These properties and their names were chosen to match existing CalDAV and
+CardDAV properties, given that a `vdir` is commonly synchronised using one of
+these two protocols.
+
+Property files with an invalid content SHOULD be ignored. Application software
+SHOULD warn the user of the invalidity when appropriate.
+
+# Writing to vdirs
+
+Creating and modifying items or metadata files MUST happen [atomically]. That
+is, a file matching one of the extensions mentioned above must never be
+partially written; if it exists, it must be complete.
+
+[atomically]: https://en.wikipedia.org/wiki/Atomicity_%28programming%29
+
+This requirement has two purposes:
+
+- Ensure that one program will never read an item which has only partially been
+  written by another tool.
+- Ensure that a fatal system crash shall never result in a corrupt item.
+
+Writing to a temporary file on the same physical device, and then moving it to
+the appropriate location is usually an effective solution. For this purpose,
+files with the extension `.tmp` may be created inside collections.
+
+When changing an item, the original filename MUST be used.
+
+# Reading from vdirs
+
+- Any file ending with the `.tmp` or no file extension MUST NOT be treated as
+  an item.
+
+- The `ident` part of the filename SHOULD NOT be parsed to improve the speed
+  of item lookup.
+
+# Considerations
+
+The primary reason this format was chosen is due to its compatibility with the
+[CardDAV] and [CalDAV] standards.
+
+[CardDav]: https://www.rfc-editor.org/rfc/rfc6352
+[CalDAV]: https://www.rfc-editor.org/rfc/rfc4791
+
+## Performance
+
+Currently, a vdir suffer from a rather major performance problem, one which
+current implementations try to mitigate by building up indices of the
+collections for faster search and lookup.
+
+Item filenames convey not additional information within the scope of this
+specification. Any attempt to encode information in these results in
+duplication of data, where one duplicate might become out of date because of
+bad implementations. As it stands right now, an index format could be
+formalized separately though.
+
+Synchronisation tools such as [vdirsyncer] or [pimsync] don't really have to
+bother about efficient item lookup, because its synchronization algorithm needs
+to fetch the whole list of items anyway. Detecting changes is easily
+implemented by checking the files' modification time.
+
+[vdirsyncer]: https://vdirsyncer.pimutils.org/en/stable/
+[pimsync]: https://pimsync.whynothugo.nl/
+
+# History
+
+## Version 1.0.0
+
+The original version of this was published as part of [the vdirsyncer
+documentation]. However, the conventions described in this document are
+intended to be adopted by other command line and desktop tools operating on
+calendar and address book collections.
+
+[the vdirsyncer documentation]: https://vdirsyncer.pimutils.org/en/stable/vdir.html
+
+## Version 1.1.0 (2025-03-31)
+
+- Publish this specification separately.
+- Several minor clarifications and rewording.
+- Recommend for item filenames to be URL-safe.
+- Specify that properties with invalid data should be ignored.

layouts/_default/baseof.html

@@ -35,7 +35,7 @@ <h1>pimutils</h1>
     {{ block "main" . }} Content for each page will end up here. {{ end }}
 
     <footer>
-      Copyright &copy; 2017-2024
+      Copyright &copy; 2017–2025
       <a href="/contact/">the pimutils members</a>.
     </footer>
   </body>