Well, it is slightly worse than that - you do not even have to invite them! If they know where you live they can just walk in all by themselves - given that you visit their website.
Scenario:
It is normal for people to make their webID public available on the web together with some kind of contact information, just like being listed in a phonebook (see for instance Rubenâs friend list https://ruben.verborgh.org/profile/#me or https://www.npmjs.com/package/solid-server#contributing).
Using the public information and some social engineering, an attacker can craft a message to those people and ask them to visit a certain web page.
Once they visit the webpage, it can use rdflib.js to access their data in the background - and if they happen to be logged in, the website can read/write/modify/delete any and all of their private data.
This is possible due to the fact that there is no application validation - I do not grant access to an application. This is very unlike Facebook and others where I explicitely have to grant each and every third party application access to my data before they can, well, access it.