chore(windows): this is a dummy PR to allow testing

[rc-swag]

TODO:
This is just to get ES a check list to do some extra Windows on Arm testing

Note I got these test cases from the wiki and not dokimion so they may need some updating.

User Testing

SUITE_CAPSLOCK:

• GROUP_WIN11_ARM
• GROUP_WIN11_X86

Caps Lock

The test keyboard layouts are found in the keyman repo at app/windows/src/test/manual-tests/caps-lock-stores. There is a project file for the 3 keyboards used in this test. The project file can be used to build the keyboard packages, but you can conveniently use the .kmp file zipped and included respectively below.

The test cases below expect the usage of the capslock.kmp.zip keyboard. That keyboard outputs pass or fail if following the test cases.

Prerequisites before each test

• System keyboard layout is en-US
• Install a keyboard that doesn't use any of the caps lock stores, e.g. capslock.kmp.
• CapsLock is currently on
• Currently active keyboard is the capslock.kmp keyboard

Test cases

click to expand

• TEST_CAPSLOCK-1: uppercase with virtual key

  • press and release a

  Expected result:

  • pass. (with other keyboards uppercase A)

• TEST_CAPSLOCK-2: lowercase with virtual key

  • press and hold 'Shift'
  • press and release b
  • release Shift

  Expected result:

  • pass. (with other keyboards lowercase b)

• TEST_CAPSLOCK-3: capslock ignored for numbers

  • press and hold 'Shift'
  • press and release 3
  • release Shift

  Expected result:

  • pass. (with other keyboards #)

• TEST_CAPSLOCK-4: uppercase

  • press and release c

  Expected result:

  • pass. (with other keyboards uppercase C)

• TEST_CAPSLOCK-5: lowercase

  • press and hold 'Shift'
  • press and release d
  • release Shift

  Expected result:

  • pass. (with other keyboards lowercase d)

CapsAlwaysOff

For these tests, use a keyboard with the caps_always_off.kmp.zip store set. We call this keyboard capsalwaysoff below.

Any keyboard with that store set will work; if you don't have one at hand you can use the caps_always_off.kmp keyboard. The caps_always_off.kmp keyboard will prevent switching caps lock on. As a sanity check to verify that Keyman is actually active, pressing the key a will output ncaps_little_a, and Shift+a will output ncaps_shift_A.

Note: When testing in a virtual machine, use an on-screen keyboard (in VirtualBox: Input/Keyboard/Soft Keyboard) and observe the caps lock indicator of the on-screen keyboard. Using the hardware keyboard might show side effects with caps lock.

Prerequisites before each test

• Install a keyboard that has CapsAlwaysOff store set, e.g. caps_always_off.kmp.
• CapsLock is currently off
• Currently active keyboard is a non-Keyman keyboard

Test cases

click to expand

• TEST_CAPSOFF-1: sanity check

  • switch to capsalwaysoff keyboard
  • press and release a

  Expected result:

  • output: ncaps_little_a

• TEST_CAPSOFF-2: caps lock stays off

  • switch to capsalwaysoff keyboard
  • press and release CapsLock key
  • press and release a

  Expected result:

  • caps lock indicator stays turned off
  • output: ncaps_little_a

• TEST_CAPSOFF-3: no caps lock while holding capslock key

  • switch to capsalwaysoff keyboard
  • press and hold CapsLock key
  • press and release a
  • release CapsLock key

  Expected result:

  • output: ncaps_little_a

• TEST_CAPSOFF-4: no caps lock while holding capslock key

  • switch to capsalwaysoff keyboard
  • press and hold CapsLock key
  • press and hold Shift key
  • press and release a
  • release CapsLock and Shift keys

  Expected result:

  • output: ncaps_shift_A

• TEST_CAPSOFF-5: switching turns off caps lock

  • turn on caps lock
  • switch to capsalwaysoff keyboard
  • press and release a

  Expected result:

  • caps lock indicator turned off
  • output: ncaps_little_a

SHIFT: CapsOnOnly/ShiftFreesCaps

For these tests, use a keyboard with the CapsOnOnly and ShiftFreesCaps stores set. We call this keyboard shift_frees_caps below.

Any keyboard with these stores set will work; if you don't have one at hand you can use the shift_frees_caps.kmp.zip keyboard.

The shift_frees_caps.kmp keyboard will enable caps lock by pressing the CapsLock key, and will turn capslock off by pressing the Shift key. The keyboard outputs pass or fail if following the test cases.

Note: When testing in a virtual machine, use an on-screen keyboard (in VirtualBox: Input/Keyboard/Soft Keyboard) and observe the caps lock indicator of the on-screen keyboard. Using the hardware keyboard might show side effects with caps lock. Except for TEST_CAPSONLY-5 which can only be reliably tested on a hardware keyboard on host OS (not a VM). For windows 10 and windows 11 with a virtual box vm-onscreen keyboard, the following happens. The VM soft keyboard does NOT actually send the Shift Shift Key Stroke through but rather will change the keys pressed for example if an a is pressed the soft keyboard itself will change that key to a A. This means we can't Test TEST_CAPONLY-5 on a soft keyboard.

Prerequisites before each test

• Install a keyboard that has the CapsOnOnly and ShiftFreesCaps stores set, e.g.
  shift_frees_caps.kmp.
• CapsLock is currently off
• Currently active keyboard is shift_frees_caps keyboard

Test cases

click to expand

• TEST_CAPSONLY-1: no caps

  • press and release 1

  Expected result:

  • output: pass.

• TEST_CAPSONLY-2: caps

  • press and release CapsLock
  • press and release 2

  Expected result:

  • caps lock indicator turned on
  • output: pass.

• TEST_CAPSONLY-3: caps doesn't toggle

  • press and release CapsLock
  • press and release CapsLock
  • press and release 6

  Expected result:

  • caps lock indicator turned on
  • output: pass.

• TEST_CAPSONLY-4: shift turns off

  • press and release CapsLock
  • press and hold Shift
  • press and release 3
  • release Shift

  Expected result:

  • caps lock indicator turned off
  • output: pass.

• TEST_CAPSONLY-5: shift by itself turns off
  Be aware of limitations when testing this on virtual machines as noted above.

  • press and release CapsLock
  • press and release Shift

  Expected result:

  • caps lock indicator turned off
  • (no output)

[keymanapp-test-bot]

User Test Results

Test specification and instructions

🟥 SUITE_CAPSLOCK:

• 🟥 GROUP_WIN11_ARM:

  • ⬜ TEST_CAPSLOCK-1 (OPEN): retest
  • ⬜ TEST_CAPSLOCK-2 (OPEN): retest
  • ⬜ TEST_CAPSLOCK-3 (OPEN): retest
  • ⬜ TEST_CAPSLOCK-4 (OPEN): retest
  • ⬜ TEST_CAPSLOCK-5 (OPEN): retest
  • ⬜ TEST_CAPSOFF-1 (OPEN): retest
  • ⬜ TEST_CAPSOFF-2 (OPEN): retest
  • ⬜ TEST_CAPSOFF-3 (OPEN): retest
  • ⬜ TEST_CAPSOFF-4 (OPEN): retest
  • ⬜ TEST_CAPSOFF-5 (OPEN): retest
  • 🟥 TEST_CAPSONLY-1 (FAILED): with Notepad output is 1, with editorarm64 output is pass. as expected
  • 🟥 TEST_CAPSONLY-2 (FAILED): with Notepad output is 2, with editorarm64 output is pass. as expected
  • 🟥 TEST_CAPSONLY-3 (FAILED): with Notepad output is 6, with editorarm64 output is pass. as expected
  • 🟥 TEST_CAPSONLY-4 (FAILED): with Notepad output is #, with editorarm64 output is pass. as expected
  • 🟥 TEST_CAPSONLY-5 (FAILED): with Notepad caps lock indicator is still turned on, with editorarm64 caps lock indicator is turned off as expected

• ⬜ GROUP_WIN11_X86:

  • ⬜ TEST_CAPSLOCK-1 (OPEN)
  • ⬜ TEST_CAPSLOCK-2 (OPEN)
  • ⬜ TEST_CAPSLOCK-3 (OPEN)
  • ⬜ TEST_CAPSLOCK-4 (OPEN)
  • ⬜ TEST_CAPSLOCK-5 (OPEN)
  • ⬜ TEST_CAPSOFF-1 (OPEN)
  • ⬜ TEST_CAPSOFF-2 (OPEN)
  • ⬜ TEST_CAPSOFF-3 (OPEN)
  • ⬜ TEST_CAPSOFF-4 (OPEN)
  • ⬜ TEST_CAPSOFF-5 (OPEN)
  • ⬜ TEST_CAPSONLY-1 (OPEN)
  • ⬜ TEST_CAPSONLY-2 (OPEN)
  • ⬜ TEST_CAPSONLY-3 (OPEN)
  • ⬜ TEST_CAPSONLY-4 (OPEN)
  • ⬜ TEST_CAPSONLY-5 (OPEN)

Results Template

# Test Results

## SUITE_CAPSLOCK: 

### GROUP_WIN11_ARM: 

* **TEST_CAPSLOCK-1 (OPEN):** notes
* **TEST_CAPSLOCK-2 (OPEN):** notes
* **TEST_CAPSLOCK-3 (OPEN):** notes
* **TEST_CAPSLOCK-4 (OPEN):** notes
* **TEST_CAPSLOCK-5 (OPEN):** notes
* **TEST_CAPSOFF-1 (OPEN):** notes
* **TEST_CAPSOFF-2 (OPEN):** notes
* **TEST_CAPSOFF-3 (OPEN):** notes
* **TEST_CAPSOFF-4 (OPEN):** notes
* **TEST_CAPSOFF-5 (OPEN):** notes

### GROUP_WIN11_X86: 

* **TEST_CAPSLOCK-1 (OPEN):** notes
* **TEST_CAPSLOCK-2 (OPEN):** notes
* **TEST_CAPSLOCK-3 (OPEN):** notes
* **TEST_CAPSLOCK-4 (OPEN):** notes
* **TEST_CAPSLOCK-5 (OPEN):** notes
* **TEST_CAPSOFF-1 (OPEN):** notes
* **TEST_CAPSOFF-2 (OPEN):** notes
* **TEST_CAPSOFF-3 (OPEN):** notes
* **TEST_CAPSOFF-4 (OPEN):** notes
* **TEST_CAPSOFF-5 (OPEN):** notes
* **TEST_CAPSONLY-1 (OPEN):** notes
* **TEST_CAPSONLY-2 (OPEN):** notes
* **TEST_CAPSONLY-3 (OPEN):** notes
* **TEST_CAPSONLY-4 (OPEN):** notes
* **TEST_CAPSONLY-5 (OPEN):** notes

Retesting Template

Test-bot: retest SUITE_CAPSLOCK GROUP_WIN11_ARM TEST_CAPSONLY-1 TEST_CAPSONLY-2 TEST_CAPSONLY-3 TEST_CAPSONLY-4 TEST_CAPSONLY-5

Test Artifacts

• Windows
  • Keyman for Windows
  • FirstVoices Keyboards for Windows
  • FirstVoices Keyboards for Windows (old PRs)
  • Keyman for Windows (old PRs)
  • Text Editor (ARM64)
  • Text Editor (x64)
  • Text Editor (x86)

[sze2st]

Test Results

Testing this PR in the "keyman-19.0.183-alpha-test-15383" build on Windows 11 arm with

SUITE_CAPSLOCK

GROUP_WIN11_ARM

• TEST_CAPSLOCK-1 (failed): with Notepad output is A, with editorarm64 output is pass. as expected

• TEST_CAPSLOCK-2 (failed): with Notepad output is b, with editorarm64 output is pass. as expected

• TEST_CAPSLOCK-3 (failed): with Notepad output is #, with editorarm64 output is pass. as expected

• TEST_CAPSLOCK-4 (failed): with Notepad output is C, with editorarm64 output is pass. as expected

• TEST_CAPSLOCK-5 (failed): with Notepad output is d, with editorarm64 output is pass. as expected

• TEST_CAPSOFF-1 (failed): with Notepad output is a, with editorarm64 output is ncaps_little_a which is not correct either as it ends with unexpected whitespace

• TEST_CAPSOFF-2 (failed): with Notepad output is A, with editorarm64 output is ncaps_little_a which is not correct either as it ends with unexpected whitespace

• TEST_CAPSOFF-3 (failed): with Notepad output is A, with editorarm64 output is ncaps_little_a which is not correct either as it ends with unexpected whitespace

• TEST_CAPSOFF-4 (failed): with Notepad output is a, with editorarm64 output is ncaps_shift_A which is not correct either as it ends with unexpected whitespace

• TEST_CAPSOFF-5 (failed): with Notepad output is A, with editorarm64 output is ncaps_little_a which is not correct either as it ends with unexpected whitespace

• TEST_CAPSONLY-1 (failed): with Notepad output is 1, with editorarm64 output is pass. as expected

• TEST_CAPSONLY-2 (failed): with Notepad output is 2, with editorarm64 output is pass. as expected

• TEST_CAPSONLY-3 (failed): with Notepad output is 6, with editorarm64 output is pass. as expected

• TEST_CAPSONLY-4 (failed): with Notepad output is #, with editorarm64 output is pass. as expected

• TEST_CAPSONLY-5 (failed): with Notepad caps lock indicator is still turned on, with editorarm64 caps lock indicator is turned off as expected

[mcdurdin]

(PR shouldn't be closed with failed tests)

[rc-swag]

I have tested GROUP_WIN11_ARM in my testing I found Notepad suffered from #15333. I had the same results initially as @sze2st, however restarting Notepad and the tests in TEST_CAPSLOCK and TEST_CAPSOFF all passed. So please restart Notepad and try again.

Test-bot: retest SUITE_CAPSLOCK GROUP_WIN11_ARM TEST_CAPSLOCK-1 TEST_CAPSLOCK-2 TEST_CAPSLOCK-3 TEST_CAPSLOCK-4 TEST_CAPSLOCK-5 TEST_CAPSOFF-1 TEST_CAPSOFF-2 TEST_CAPSOFF-3 TEST_CAPSOFF-4 TEST_CAPSOFF-5

Some tests in TEST_CAPSONLY failed in TSF applications. I then tested the on Windows 10 x86 and they failed there also. Win 11 x86 had failures however for me that is in a VM which has known issues for this tests. These will requre further investigation I will start with checking the latest regression tests in Dokimion in case these are out of date.