Feature: Resizable Window

[viatearz]

Summary

Allow users to resize the window by dragging its borders, just like when we removed PlayTools.

Changes

File	Explanation
NSObject+Swizzle.m	The most important part. Force [_UIApplicationInfoParser requiresFullScreen] to return false, then macOS will create a resizable window.
AKPlugin.swift	Set a fixed aspect ratio via NSWindow.contentAspectRatio.
PlayScreen.swift	1. Remove the default minSize restriction. 2. Force UIScreen.main.bounds to return the key window's bounds, game rendering and keymapping rely on this value.
TouchscreenMouseEventAdapter.swift	Since keymapping consumes all mouse click events, we need to add extra logic to allow dragging the window borders.
ControlMode.swift‎	Reactivate keymapping after the window size changes.

[viatearz]

With this feature, we can play games that switch between portrait and landscape orientations.
PlayCover/PlayCover#1496
(Alright, it still has a login issue due to PlayChain bugs)

[viatearz]

SwiftLint complains that this function’s body exceeds 50 lines. Hope someone can refactor it later.

PlayTools/PlayTools/Controls/Frontend/ControlMode.swift

Line 33 in e40b589

public func initialize() {

[JoseMoreville]

Resizing freely causes the app to crash otherwise this feature is highly useful, i was actually working on the same. but this is almost fine, just i would suggest some changes to match my own work to improve this a bit more.

[viatearz]

Resizing freely causes the app to crash otherwise this feature is highly useful, i was actually working on the same. but this is almost fine, just i would suggest some changes to match my own work to improve this a bit more.

That’s great. If you are already working on this feature, I can close this PR and wait for yours, since I might not have enough time to improve it.
I just wanted to share the idea — feel free to take any code you need.
Of course, I can still make a few small changes to this PR if you’d like.

[JoseMoreville]

Resizing freely causes the app to crash otherwise this feature is highly useful, i was actually working on the same. but this is almost fine, just i would suggest some changes to match my own work to improve this a bit more.

That’s great. If you are already working on this feature, I can close this PR and wait for yours, since I might not have enough time to improve it. I just wanted to share the idea — feel free to take any code you need. Of course, I can still make a few small changes to this PR if you’d like.

No need to close, this work is pretty good. if anything I'll just commit over this or just merge and add changes on other pr.

Besides you deserve your contributions to the project to be acknowledged.

[JoseMoreville]

viatearz#1 Swiftlit fix. @viatearz

[JoseMoreville]

Found a "workaround" for the resizing crash on unitygfxdeviceworker. Just open the binary or execute it on the terminal. I'm not sure if I'll handle this, but I'll leave it here in case someone can think of a fix while I sleep.

Just an idea, envs can be set to simulate the terminal env but that might lead to unexpected other issues. we'll see

[viatearz]

Found a "workaround" for the resizing crash on unitygfxdeviceworker. Just open the binary or execute it on the terminal. I'm not sure if I'll handle this, but I'll leave it here in case someone can think of a fix while I sleep.

Just an idea, envs can be set to simulate the terminal env but that might lead to unexpected other issues. we'll see

Which apps are crashing? I haven’t encountered any crashes, but I’m still on macOS 15.

[JoseMoreville]

Found a "workaround" for the resizing crash on unitygfxdeviceworker. Just open the binary or execute it on the terminal. I'm not sure if I'll handle this, but I'll leave it here in case someone can think of a fix while I sleep.
Just an idea, envs can be set to simulate the terminal env but that might lead to unexpected other issues. we'll see

Which apps are crashing? I haven’t encountered any crashes, but I’m still on macOS 15.

Genshin Impact, but I guess all Unity apps might crash if you resize the app by dragging the corner nonstop.

It might be a race condition or something related to how Unity handles rendering, and the speed of updating the window and scaling its content.

nvm i just found waht the issue is, i still dont know how to fix it but there you go.

-[MTLDebugRenderCommandEncoder setScissorRect:]:4129: failed assertion `Set Scissor Rect Validation
(rect.x(0) + rect.width(3414))(3414) must be <= render pass width(1450)
'
Process 88031 launched: '/Users/hades/Library/Containers/io.playcover.PlayCover/Applications/com.miHoYo.GenshinImpact.app/GenshinImpact' (arm64)
Process 88031 stopped

• thread 32, name = 'UnityGfxDeviceWorker', stop reason = signal SIGABRT
  frame #0: 0x000000019a0225b0 libsystem_kernel.dylib__pthread_kill + 8 libsystem_kernel.dylib__pthread_kill:
  -> 0x19a0225b0 <+8>: b.lo 0x19a0225d0 ; <+40>
  0x19a0225b4 <+12>: pacibsp
  0x19a0225b8 <+16>: stp x29, x30, [sp, #-0x10]!
  0x19a0225bc <+20>: mov x29, sp
  Target 0: (GenshinImpact) stopped.
  (lldb) exit

[viatearz]

You may try removing PlayTools to see if it still crashes.
And you can try commenting out all the changed code, but only keeping this line in NSObject+Swizzle.m:

[objc_getClass("_UIApplicationInfoParser")
     swizzleInstanceMethod:NSSelectorFromString(@"requiresFullScreen")
     withMethod:@selector(hook_requiresFullScreen)];

[JoseMoreville]

You may try removing PlayTools to see if it still crashes. And you can try commenting out all the changed code, but only keeping this line in NSObject+Swizzle.m:

[objc_getClass("_UIApplicationInfoParser")
     swizzleInstanceMethod:NSSelectorFromString(@"requiresFullScreen")
     withMethod:@selector(hook_requiresFullScreen)];

I've tried the first option removing playtools before, it doesnt crash since as far as i remember because it doesnt tirgger a rerendering of the content. i'll need to see again tomorrow once i can work on this again.

but the issue is with rerendering the content inside, maybe the scalling? -[MTLDebugRenderCommandEncoder setScissorRect:]:4129: failed assertion `Set Scissor Rect Validation
(rect.x(0) + rect.width(3414))(3414) must be <= render pass width(1450)

[viatearz]

You may try removing PlayTools to see if it still crashes. And you can try commenting out all the changed code, but only keeping this line in NSObject+Swizzle.m:

[objc_getClass("_UIApplicationInfoParser")
     swizzleInstanceMethod:NSSelectorFromString(@"requiresFullScreen")
     withMethod:@selector(hook_requiresFullScreen)];

I've tried the first option removing playtools before, it doesnt crash since as far as i remember because it doesnt tirgger a rerendering of the content. i'll need to see again tomorrow once i can work on this again.

but the issue is with rerendering the content inside, maybe the scalling? -[MTLDebugRenderCommandEncoder setScissorRect:]:4129: failed assertion `Set Scissor Rect Validation (rect.x(0) + rect.width(3414))(3414) must be <= render pass width(1450)

I can also reproduce this crash.
It seems that when running through PlayCover, MTLCreateSystemDefaultDevice() creates a MTLDebugDevice, which triggers some assertion failures and causes the crash.
When running in Terminal, MTLCreateSystemDefaultDevice() returns an object like AGXDevice, it does not invoke [MTLRenderCommandEncoder setScissorRect:].

You can use the following code to verify (put it in PlayLoader.m):

static id<MTLDevice> pt_MTLCreateSystemDefaultDevice(void) {
    id<MTLDevice> device = MTLCreateSystemDefaultDevice();
    NSLog(@"[PlayTools] MTLCreateSystemDefaultDevice: %@", NSStringFromClass(object_getClass(device)));
    return device;
}
DYLD_INTERPOSE(pt_MTLCreateSystemDefaultDevice, MTLCreateSystemDefaultDevice)

Running it in Terminal, or double-clicking in Finder, does not cause this issue. It only occurs when opening the app through PlayCover.
So it might be related to the environment that PlayCover sets when launching the app.

[JoseMoreville]

You may try removing PlayTools to see if it still crashes. And you can try commenting out all the changed code, but only keeping this line in NSObject+Swizzle.m:

[objc_getClass("_UIApplicationInfoParser")
     swizzleInstanceMethod:NSSelectorFromString(@"requiresFullScreen")
     withMethod:@selector(hook_requiresFullScreen)];

I've tried the first option removing playtools before, it doesnt crash since as far as i remember because it doesnt tirgger a rerendering of the content. i'll need to see again tomorrow once i can work on this again.
but the issue is with rerendering the content inside, maybe the scalling? -[MTLDebugRenderCommandEncoder setScissorRect:]:4129: failed assertion `Set Scissor Rect Validation (rect.x(0) + rect.width(3414))(3414) must be <= render pass width(1450)

I can also reproduce this crash. It seems that when running through PlayCover, MTLCreateSystemDefaultDevice() creates a MTLDebugDevice, which triggers some assertion failures and causes the crash. When running in Terminal, MTLCreateSystemDefaultDevice() returns an object like AGXDevice, it does not invoke [MTLRenderCommandEncoder setScissorRect:].

You can use the following code to verify (put it in PlayLoader.m):

static id<MTLDevice> pt_MTLCreateSystemDefaultDevice(void) {
    id<MTLDevice> device = MTLCreateSystemDefaultDevice();
    NSLog(@"[PlayTools] MTLCreateSystemDefaultDevice: %@", NSStringFromClass(object_getClass(device)));
    return device;
}
DYLD_INTERPOSE(pt_MTLCreateSystemDefaultDevice, MTLCreateSystemDefaultDevice)

Running it in Terminal, or double-clicking in Finder, does not cause this issue. It only occurs when opening the app through PlayCover. So it might be related to the environment that PlayCover sets when launching the app.

It is, i fixed it already. or it seems like it. I will push to your playcover branch once i test throughtly and can guarantee its fixed

[JoseMoreville]

good to go