I use self-hosted Baikal for CalDAV (calendar and tasks) and CardDAV (contacts).

I have had positive experiences with both Radicale and Baikal. I am not sure about the Home Assistant integration, but they both use CalDAV, so I would be surprised if there wasn’t a way of connecting them. iOS has native support for CalDAV, but Android needs the davx5 app (free on F-Droid).