So it turns out it looks like the orientation of the mesh at the time of capture actually matters. I learned this from this thread:
Hope this helps someone else because this was driving me crazy for a few hours trying to figure out what was going on.