The default ~/.ssh/authorized_keys and ~/.ssh/authorized_keys2 file lists and stores the public keys (DSA/ECDSA/RSA) that can be used for logging for any user using public key authentication. Each line of the file contains one key. You can simply delete the key from this file, and the user can not access the server using the ssh client. You also need to disable or lock an user account using passwd command.
A note about long term solution
If your setup has hundreds of users, tens of thousands, thousands of accounts for OpenSSH, try OpenSSH with LDAP. Use LDAP for key storage management. You can add, remove, and revoke keys. However, this approach adds levels of complexity to a solution, and learning curve can be very complex.
Another option is to store shared home directories on an NFS server so that one can easily add or delete the keys.