James Taylor reveals why he turned down Taylor Swift and Travis Kelce’s wedding invitation