VM snapshot deletion failure may put it indefinitely in expunging state

[yadvr]

VM snapshot deletion may put the state in Expunging state indefinitely, originally logged here: apache/cloudstack-primate#488

ISSUE TYPE

• Bug Report

COMPONENT NAME

VMware

CLOUDSTACK VERSION

4.14

[DaanHoogland]

the statemachine VMSnapshot.s_fsm, the transition s_fsm.addTransition(Expunging, Event.OperationSucceeded, Removed) is never executed for this to happen (or an exception occurs).
DefaultVMSnapshotStrategy.processAnswer(final VMSnapshotVO vmSnapshot, UserVm userVm, final Answer as, Long hostId) is were is probably should happen.

[shwstppr]

@rhtyd @DaanHoogland are there any known reproduction steps?

[DaanHoogland]

@shwstppr not reliably, that I know of. I happens regularly though. usually after a full smoke test run, there will be some remnant VMs in expunging state.

[yadvr]

@shwstppr - create a VM snapshot, delete the snapshot when it's in expunging state, (a) try to restart mgmt server, (b) don't restart mgtm server but see if the GC thread kicks in to purge the VM snapshot.

[yadvr]

Closing issue, have a PR - #4898