Inside driver side door handle

Recently, the chrome door handle snapped. I have the replacement unit (#511 8 226 049 OES). I was able to remove the old broken part. However, I cannot figure out how to install the new part. It is very awkward to access the hook that the new part connects to, because the hook is recessed.

Do I need to remove the door panel? It looks like the panel is held by flimsy foam clips that will certainly break if I pull out the panel.

Thanks in advance for your help!

The door panel should be removed