Skip to content

System.Text.EncoderFallbackException: Unable to translate Unicode character \uD83D at index 14 to specified code page. #5053

@hfrmobile

Description

@hfrmobile

Prerequisites

  • Write a descriptive title.
  • Make sure you are able to repro it on the latest released version
  • Search the existing issues, especially the pinned issues.

Exception report

Oops, something went wrong.  Please report this bug with the details below.
Report on GitHub: https://github.com/PowerShell/PSReadLine/issues/new
### Environment
PSReadLine: 2.0.0
PowerShell: 5.1.22621.6133
OS: Microsoft Windows 10.0.22631
Last 134 Keys

 m a r k d Tab Spacebar F i n Tab Backspace Backspace Backspace . \ Spacebar F i n d i n g Spacebar E m u l a t o r . m d " Ctrl+LeftArrow Ctrl+LeftArrow Ctrl+LeftArrow Ctrl+LeftArrow Ctrl+RightArrow LeftArrow LeftArrow LeftArrow " Enter
 UpArrow Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace Backspace l s Enter
 � Spacebar F i n d i n g Spacebar E m u l a t o r . m d " Home " Home m a r k d o w n l i n t Spacebar Enter



### Exception

System.Text.EncoderFallbackException: Unable to translate Unicode character \uD83D at index 14 to specified code page.
   at System.Text.EncoderExceptionFallbackBuffer.Fallback(Char charUnknown, Int32 index)
   at System.Text.EncoderFallbackBuffer.InternalFallback(Char ch, Char*& chars)
   at System.Text.UTF8Encoding.GetBytes(Char* chars, Int32 charCount, Byte* bytes, Int32 byteCount, EncoderNLS baseEncoder)
   at System.Text.EncoderNLS.GetBytes(Char[] chars, Int32 charIndex, Int32 charCount, Byte[] bytes, Int32 byteIndex, Boolean flush)   
   at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
   at System.IO.StreamWriter.Dispose(Boolean disposing)
   at System.IO.TextWriter.Dispose()
   at Microsoft.PowerShell.PSConsoleReadLine.<>c__DisplayClass83_0.<WriteHistoryRange>b__0()
   at Microsoft.PowerShell.PSConsoleReadLine.WithHistoryFileMutexDo(Int32 timeout, Action action)
   at Microsoft.PowerShell.PSConsoleReadLine.MaybeAddToHistory(String result, List`1 edits, Int32 undoEditIndex, Boolean fromDifferentSession, Boolean fromInitialRead)
   at Microsoft.PowerShell.PSConsoleReadLine.InputLoop()
   at Microsoft.PowerShell.PSConsoleReadLine.ReadLine(Runspace runspace, EngineIntrinsics engineIntrinsics, CancellationToken cancellationToken)

Screenshot

VSCode's folder view displays name correctly

Image

VSCode integrated PowerShell command (auto-complete)

Filename is not displayed correctly, exception occurred when executing the command:

Image

Outside PowerShell has no problem/issue

The issue only occurs when running PowerShell which is integrated to VS Code, "normal" PowerShell executed from File Explorer works even with the UNICODE character 🕮:

Image

Environment data

### Windows Terminal / PowerShell (the working one)

PS Version: 5.1.22621.6133
PS HostName: ConsoleHost (Windows Terminal)
PSReadLine Version: 2.0.0
PSReadLine EditMode: Windows
OS: 10.0.22621.1 (WinBuild.160101.0800)
BufferWidth: 120
BufferHeight: 30

### VS Code's integrated terminal (the buggy one)

PS Version: 5.1.22621.6133
PS HostName: ConsoleHost (Windows Terminal)
PSReadLine Version: 2.0.0
PSReadLine EditMode: Windows
OS: 10.0.22621.1 (WinBuild.160101.0800)
BufferWidth: 120
BufferHeight: 30

Since they're the same the issue seems to be in the integration of PS into VS Code.

Steps to reproduce

  1. Open VS Code
  2. Select/mark file (F2, Ctrl+A, Ctrl+C) 🕮 Finding Emulator.md in file tree view
  3. Run 🕮 Finding Emulator.md (INFO: Here the filename is correct but in Terminal window it looks like:
    Image

Expected behavior

No exception: Copy&Paste should work with terminal window / VS Code; looks like they're using different encoding

Workaround: Type markdownlint and press until ‘🕮 Finding Emulator.md’ appears. If the folder contains many files, you’ll need multiple tab presses because VS Code’s tree view sorts differently than the terminal’s completion order.

Actual behavior

Copy&Paste inserts wrong UNICODE char + when executing the exception occurs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Needs-Triage 🔍It's a new issue that core contributor team needs to triage.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions