Tom steady

Hello, I'm wanting to remove the outer handle of the sliding door on my 06 nissan interstar. I've removed the two screws, but it wont shift. With the front door, you just slide it along and it comes out. Is it the same with this ? More force required perhaps? Or do I need to take off the interior panel ? Don't want to damage it any more than it already is. Thanks for any suggestions