For me the main flaw is that it uses Google Contacts for the attendee names and emails. I keep my Contacts in Apple Contacts.
Turns out that if you include at least 1 local Calendar it will also use Apple Contacts. Birthdays works for this. So I included that.
This should be a setting, but I'm glad to to know of the workaround via Calendars tech support.