[SE-0489] Better debugDescription for EncodingError and DecodingError (#80941)

Now accepted as
[SE-0489](https://github.com/ZevEisenberg/swift-evolution/blob/main/proposals/0489-codable-error-printing.md).

# To Do

- [x] confirm which version of Swift to use for the availability
annotations. Probably 6.3 at time of writing.

# Context

Re: [Swift forum
post](https://forums.swift.org/t/the-future-of-serialization-deserialization-apis/78585/77),
where a discussion about future serialization tools in Swift prompted
Kevin Perry to suggest that some proposed changes could actually be made
in today's stdlib.

# Summary of Changes

Conforms `EncodingError` and `DecodingError` to
`CustomDebugStringConvertible` and adds a more-readable
`debugDescription`.

# Future Directions

This is a pared-down version of some experiments I did in
[UsefulDecode](https://github.com/ZevEisenberg/UsefulDecode). The
changes in this PR are the best I could do without changing the public
interface of `DecodingError` and `EncodingError`, and without modifying
the way the `JSON`/`PropertyList` `Encoder`/`Decoder` in Foundation
generate their errors' debug descriptions.

In the above-linked
[UsefulDecode](https://github.com/ZevEisenberg/UsefulDecode) repo, when
JSON decoding fails, I go back and re-decode the JSON using
`JSONSerialization` in order to provide more context about what failed,
and why. I didn't attempt to make such a change here, but I'd like to
discuss what may be possible.

# Examples

To illustrate the effect of the changes in this PR, I removed my changes
to stdlib/public/core/Codable.swift and ran my new test cases again.
Here are the resulting diffs.

##
`test_encodingError_invalidValue_nonEmptyCodingPath_nilUnderlyingError`

### Before
`invalidValue(234, Swift.EncodingError.Context(codingPath:
[GenericCodingKey(stringValue: "first", intValue: nil),
GenericCodingKey(stringValue: "second", intValue: nil),
GenericCodingKey(stringValue: "2", intValue: 2)], debugDescription: "You
cannot do that!", underlyingError: nil))`

### After
`EncodingError.invalidValue: 234 (Int). Path: first.second[2]. Debug
description: You cannot do that!`

## `test_decodingError_valueNotFound_nilUnderlyingError`

### Before
`valueNotFound(Swift.String, Swift.DecodingError.Context(codingPath:
[GenericCodingKey(stringValue: "0", intValue: 0),
GenericCodingKey(stringValue: "firstName", intValue: nil)],
debugDescription: "Description for debugging purposes", underlyingError:
nil))`

### After
`DecodingError.valueNotFound: Expected value of type String but found
null instead. Path: [0].firstName. Debug description: Description for
debugging purposes`

## `test_decodingError_keyNotFound_nonNilUnderlyingError`

### Before
`keyNotFound(GenericCodingKey(stringValue: "name", intValue: nil),
Swift.DecodingError.Context(codingPath: [GenericCodingKey(stringValue:
"0", intValue: 0), GenericCodingKey(stringValue: "address", intValue:
nil), GenericCodingKey(stringValue: "city", intValue: nil)],
debugDescription: "Just some info to help you out", underlyingError:
Optional(main.GenericError(name: "hey, who turned out the lights?"))))`

### After
`DecodingError.keyNotFound: Key \'name\' not found in keyed decoding
container. Path: [0].address.city. Debug description: Just some info to
help you out. Underlying error: GenericError(name: "hey, who turned out
the lights?")`

## `test_decodingError_typeMismatch_nilUnderlyingError`

### Before
`typeMismatch(Swift.String, Swift.DecodingError.Context(codingPath:
[GenericCodingKey(stringValue: "0", intValue: 0),
GenericCodingKey(stringValue: "address", intValue: nil),
GenericCodingKey(stringValue: "city", intValue: nil),
GenericCodingKey(stringValue: "birds", intValue: nil),
GenericCodingKey(stringValue: "1", intValue: 1),
GenericCodingKey(stringValue: "name", intValue: nil)], debugDescription:
"This is where the debug description goes", underlyingError: nil))`

### After
`DecodingError.typeMismatch: expected value of type String. Path:
[0].address.city.birds[1].name. Debug description: This is where the
debug description goes`

## `test_decodingError_dataCorrupted_nonEmptyCodingPath`

### Before
`dataCorrupted(Swift.DecodingError.Context(codingPath:
[GenericCodingKey(stringValue: "first", intValue: nil),
GenericCodingKey(stringValue: "second", intValue: nil),
GenericCodingKey(stringValue: "2", intValue: 2)], debugDescription:
"There was apparently some data corruption!", underlyingError:
Optional(main.GenericError(name: "This data corruption is getting out of
hand"))))`

### After
`DecodingError.dataCorrupted: Data was corrupted. Path: first.second[2].
Debug description: There was apparently some data corruption!.
Underlying error: GenericError(name: "This data corruption is getting
out of hand")`

## `test_decodingError_valueNotFound_nonNilUnderlyingError`

### Before
`valueNotFound(Swift.Int, Swift.DecodingError.Context(codingPath:
[GenericCodingKey(stringValue: "0", intValue: 0),
GenericCodingKey(stringValue: "population", intValue: nil)],
debugDescription: "Here is the debug description for value-not-found",
underlyingError: Optional(main.GenericError(name: "these aren\\\'t the
droids you\\\'re looking for"))))`

### After
`DecodingError.valueNotFound: Expected value of type Int but found null
instead. Path: [0].population. Debug description: Here is the debug
description for value-not-found. Underlying error: GenericError(name:
"these aren\\\'t the droids you\\\'re looking for")`

##
`test_encodingError_invalidValue_emptyCodingPath_nonNilUnderlyingError`

### Before
`invalidValue(345, Swift.EncodingError.Context(codingPath: [],
debugDescription: "You cannot do that!", underlyingError:
Optional(main.GenericError(name: "You really cannot do that"))))`

### After
`EncodingError.invalidValue: 345 (Int). Debug description: You cannot do
that!. Underlying error: GenericError(name: "You really cannot do
that")`

## `test_decodingError_typeMismatch_nonNilUnderlyingError`

### Before
`typeMismatch(Swift.String, Swift.DecodingError.Context(codingPath:
[GenericCodingKey(stringValue: "0", intValue: 0),
GenericCodingKey(stringValue: "address", intValue: nil),
GenericCodingKey(stringValue: "1", intValue: 1),
GenericCodingKey(stringValue: "street", intValue: nil)],
debugDescription: "Some debug description", underlyingError:
Optional(main.GenericError(name: "some generic error goes here"))))`

### After
`DecodingError.typeMismatch: expected value of type String. Path:
[0].address[1].street. Debug description: Some debug description.
Underlying error: GenericError(name: "some generic error goes here")`

## `test_encodingError_invalidValue_emptyCodingPath_nilUnderlyingError`

### Before
`invalidValue(123, Swift.EncodingError.Context(codingPath: [],
debugDescription: "You cannot do that!", underlyingError: nil))`

### After
`EncodingError.invalidValue: 123 (Int). Debug description: You cannot do
that!`

## `test_decodingError_keyNotFound_nilUnderlyingError`

### Before
`keyNotFound(GenericCodingKey(stringValue: "name", intValue: nil),
Swift.DecodingError.Context(codingPath: [GenericCodingKey(stringValue:
"0", intValue: 0), GenericCodingKey(stringValue: "address", intValue:
nil), GenericCodingKey(stringValue: "city", intValue: nil)],
debugDescription: "How would you describe your relationship with your
debugger?", underlyingError: nil))`

### After
`DecodingError.keyNotFound: Key \'name\' not found in keyed decoding
container. Path: [0]address.city. Debug description: How would you
describe your relationship with your debugger?`

## `test_decodingError_dataCorrupted_emptyCodingPath`

### Before
`dataCorrupted(Swift.DecodingError.Context(codingPath: [],
debugDescription: "The given data was not valid JSON", underlyingError:
Optional(main.GenericError(name: "just some data corruption"))))`

### After
`DecodingError.dataCorrupted: Data was corrupted. Debug description: The
given data was not valid JSON. Underlying error: GenericError(name:
"just some data corruption")`

##
`test_encodingError_invalidValue_nonEmptyCodingPath_nonNilUnderlyingError`

### Before
`invalidValue(456, Swift.EncodingError.Context(codingPath:
[GenericCodingKey(stringValue: "first", intValue: nil),
GenericCodingKey(stringValue: "second", intValue: nil),
GenericCodingKey(stringValue: "2", intValue: 2)], debugDescription: "You
cannot do that!", underlyingError: Optional(main.GenericError(name: "You
really cannot do that"))))`

### After
`EncodingError.invalidValue: 456 (Int). Path: first.second[2]. Debug
description: You cannot do that!. Underlying error: GenericError(name:
"You really cannot do that")`

Author: ZevEisenberg

stdlib/public/core/Codable.swift

@@ -89,6 +89,58 @@ extension CodingKey {
   public var debugDescription: String {
     return description
   }
+
+  /// A simplified description: the int value, if present, in square brackets.
+  /// Otherwise, the string value by itself. Used when concatenating coding keys
+  /// to form a path when printing debug information.
+  /// - parameter isFirst: Whether this is the first key in a coding path, in
+  ///   which case we will omit the prepended '.' delimiter from string keys.
+  fileprivate func _errorPresentationDescription(isFirstInCodingPath isFirst: Bool = true) -> String {
+    if let intValue {
+      return "[\(intValue)]"
+    } else {
+      let delimiter = isFirst ? "" : "."
+      return "\(delimiter)\(stringValue._escapedForCodingKeyErrorPresentationDescription)"
+    }
+  }
+}
+
+extension [any CodingKey] {
+  /// Concatenates the elements of an array of coding keys and joins them with
+  /// "/" separators to make them read like a path.
+  fileprivate func _errorPresentationDescription() -> String {
+    return (
+      self.prefix(1).map { $0._errorPresentationDescription(isFirstInCodingPath: true) }
+      + self.dropFirst(1).map { $0._errorPresentationDescription(isFirstInCodingPath: false) }
+    ).joined(separator: "")
+  }
+}
+
+extension String {
+  /// When printing coding paths, delimit string keys with a '.' (period). If
+  /// the key contains a period, escape it with backticks so that it can be
+  /// distinguished from the delimiter. Also escape backslashes and backticks
+  /// (but *not* periods) to avoid confusion with delimiters.
+  internal var _escapedForCodingKeyErrorPresentationDescription: String {
+    let charactersThatNeedBackticks: Set<Character> = [".", "`", "\\"]
+    let charactersThatNeedEscaping: Set<Character> = ["`", "\\"]
+    assert(
+      charactersThatNeedEscaping.isSubset(of: charactersThatNeedBackticks),
+      "Only some characters in backticks will require further escaping to disambiguate them from the backticks"
+    )
+
+    var escaped = self
+    var needsBackticks = false
+    for (character, index) in zip(self, indices).reversed() {
+      if charactersThatNeedBackticks.contains(character) {
+        needsBackticks = true
+        if charactersThatNeedEscaping.contains(character) {
+          escaped.insert("\\", at: index)
+        }
+      }
+    }
+    return needsBackticks ? "`\(escaped)`" : self
+  }
 }
 
 //===----------------------------------------------------------------------===//
@@ -3724,6 +3776,80 @@ public enum DecodingError: Error {
   }
 }
 
+@available(SwiftStdlib 6.3, *)
+extension EncodingError: CustomDebugStringConvertible {
+  /// A textual representation of this encoding error, intended for debugging.
+  ///
+  /// - Important: The contents of the returned string are not guaranteed to
+  ///    remain stable: they may arbitrarily change in any Swift release.
+  @available(SwiftStdlib 6.3, *)
+  public var debugDescription: String {
+    let (message, context) = switch self {
+      case .invalidValue(let value, let context):
+        (
+          "EncodingError.invalidValue: \(String(reflecting: value)) (\(type(of: value)))",
+          context
+        )
+    }
+
+    var output = message
+
+    let contextDebugDescription = context.debugDescription
+
+    if !context.codingPath.isEmpty {
+      output.append(". Path: \(context.codingPath._errorPresentationDescription())")
+    }
+
+    if !contextDebugDescription.isEmpty {
+      output.append(". Debug description: \(context.debugDescription)")
+    }
+
+    if let underlyingError = context.underlyingError {
+      output.append(". Underlying error: \(underlyingError)")
+    }
+
+    return output
+  }
+}
+
+@available(SwiftStdlib 6.3, *)
+extension DecodingError: CustomDebugStringConvertible {
+  /// A textual representation of this decoding error, intended for debugging.
+  ///
+  /// - Important: The contents of the returned string are not guaranteed to
+  ///    remain stable: they may arbitrarily change in any Swift release.
+  @available(SwiftStdlib 6.3, *)
+  public var debugDescription: String {
+    let (message, context) = switch self {
+      case .typeMismatch(let expectedType, let context):
+        ("DecodingError.typeMismatch: expected value of type \(expectedType)", context)
+      case .valueNotFound(let expectedType, let context):
+        ("DecodingError.valueNotFound: Expected value of type \(expectedType) but found null instead", context)
+      case .keyNotFound(let expectedKey, let context):
+        ("DecodingError.keyNotFound: Key '\(expectedKey._errorPresentationDescription())' not found in keyed decoding container", context)
+      case .dataCorrupted(let context):
+        ("DecodingError.dataCorrupted: Data was corrupted", context)
+    }
+
+    var output = message
+
+    if !context.codingPath.isEmpty {
+      output.append(". Path: \(context.codingPath._errorPresentationDescription())")
+    }
+
+    let contextDebugDescription = context.debugDescription
+    if !contextDebugDescription.isEmpty {
+      output.append(". Debug description: \(contextDebugDescription)")
+    }
+
+    if let underlyingError = context.underlyingError {
+      output.append(". Underlying error: \(underlyingError)")
+    }
+
+    return output
+  }
+}
+
 // The following extensions allow for easier error construction.
 
 internal struct _GenericIndexKey: CodingKey, Sendable {

test/Macros/macro_plugin_error.swift

@@ -37,7 +37,7 @@ func test() {
   // FIXME: -module-abi-name ABI name is leaking.
 
   let _: String = #fooMacro(1)
-  // expected-error @-1 {{typeMismatch(_CompilerSwiftCompilerPluginMessageHandling.PluginToHostMessage}}
+  // expected-error @-1 {{typeMismatch}}
   let _: String = #fooMacro(2)
   // expected-error @-1 {{failed to receive result from plugin (from macro 'fooMacro')}}
   let _: String = #fooMacro(3)

test/abi/macOS/arm64/stdlib.swift

@@ -1151,3 +1151,13 @@ Added: _swift_retain_preservemost
 
 // New debug environment variable for the concurrency runtime.
 Added: _concurrencyEnableTaskSlabAllocator
+
+// SE-0489 Better debugDescription for EncodingError and DecodingError
+Added: _$ss13DecodingErrorO16debugDescriptionSSvg
+Added: _$ss13DecodingErrorO16debugDescriptionSSvpMV
+Added: _$ss13DecodingErrorOs28CustomDebugStringConvertiblesMc
+Added: _$ss13DecodingErrorOs28CustomDebugStringConvertiblesWP
+Added: _$ss13EncodingErrorO16debugDescriptionSSvg
+Added: _$ss13EncodingErrorO16debugDescriptionSSvpMV
+Added: _$ss13EncodingErrorOs28CustomDebugStringConvertiblesMc
+Added: _$ss13EncodingErrorOs28CustomDebugStringConvertiblesWP

test/abi/macOS/x86_64/stdlib.swift

@@ -1146,3 +1146,13 @@ Added: __swift_debug_metadataAllocatorPageSize
 
 // New debug environment variable for the concurrency runtime.
 Added: _concurrencyEnableTaskSlabAllocator
+
+// SE-0489 Better debugDescription for EncodingError and DecodingError
+Added: _$ss13DecodingErrorO16debugDescriptionSSvg
+Added: _$ss13DecodingErrorO16debugDescriptionSSvpMV
+Added: _$ss13DecodingErrorOs28CustomDebugStringConvertiblesMc
+Added: _$ss13DecodingErrorOs28CustomDebugStringConvertiblesWP
+Added: _$ss13EncodingErrorO16debugDescriptionSSvg
+Added: _$ss13EncodingErrorO16debugDescriptionSSvpMV
+Added: _$ss13EncodingErrorOs28CustomDebugStringConvertiblesMc
+Added: _$ss13EncodingErrorOs28CustomDebugStringConvertiblesWP

test/api-digester/stability-stdlib-abi-without-asserts.test

@@ -879,4 +879,8 @@ Func _float64ToStringImpl(_:_:_:_:) is a new API without '@available'
 Func _int64ToStringImpl(_:_:_:_:_:) is a new API without '@available'
 Func _uint64ToStringImpl(_:_:_:_:_:) is a new API without '@available'
 
+// New conformances from SE-0489: Better debugDescription for EncodingError and DecodingError
+Enum DecodingError has added a conformance to an existing protocol CustomDebugStringConvertible
+Enum EncodingError has added a conformance to an existing protocol CustomDebugStringConvertible
+
 // *** DO NOT DISABLE OR XFAIL THIS TEST. *** (See comment above.)