So, how did the problem manifest? Whenever someone added a new VM, either using a new virtual disk or a template, this exception would pop up:
'TestVm' could not initialize.
An attempt to initialize VM saved state failed.
'TestVm' could not initialize. (Virtual machine ID ....)
'TestVm' could not create or access saved state file 80F7C822-455B-4D70-9D73-B250196B36A9.vsv.
We had a fix/hack for solving this.
Run the following command in a cmd-prompt (not PowerShell!):
D:\Hyper-V\02250683-9FD8-4286-95CC-5C131F6A09DE>
icacls 02250683-9FD8-4286-95CC-5C131F6A09DE.vsv /grant "NT VIRTUAL MACHINE\02250683-9FD8-4286-95CC-5C131F6A09DE":(F)
However, remembering that for every new VM is not much fun.
The problem was, that the folder permissions on the folder with the Hyper-V VM disks and configurations lacked a permission for the "Hyper-V Administrators".
I changed the permission on the base folder holding the disks and the configs by adding the Hyper-V Administrators group of the local machine, and the problem was gone: