Some contacts aren't importing from my CSV export

Very likely yes — this is the same underlying issue as this earlier thread on CSV encoding, just showing up as silently skipped rows instead of a hard error this time (depends on exactly how the bad bytes land). Re-saving the file explicitly as UTF-8 from a plain text editor, rather than however it was originally exported, should bring those rows back in on re-import.