Michael Rubin sets record straight on Taylor Swift wedding invite after rescheduling White Party