I encountered the following error when attempting to update a plug-in in WordPress, running on IIS:
An error occurred while updating Akismet: Could not remove the old plugin.
This appeared to be a permissions problem, but as far as I was concerned, file system permissions were correct: the application pool was configured with the identity ApplicationPoolIdentity and this account, IIS AppPool\WordPress, had been granted modify permissions in the file system. This problem was perplexing, as I could make other changes to WordPress, such as installing and deleting themes, which obviously involves writing to the file system.
The solution was to also grant the IUSR user account modify permissions in the file system.
This must mean that WordPress is be using this anonymous account for certain update actions, but not others. Sounds like a bug to me.