I had done all these quests but he was still wasn't taking my items. I switched to ultimate difficulty and he did though, I also opened and closed the game doing this; so not sure which did the trick. I think I did a few of the quests and missions across multiple difficulties, so maybe that messed things up?