FreeIPA Command and Cheatsheet
common problems
All ipa commands return ipa: ERROR: did not receive Kerberos credentials when I'm root it's broken!
this is normal you need to authenticate as a user with permission to run these commands (e.g. an admin member) to fix run kinit <username> and enter the password. If you have configured 2FA on a user you WILL need to do that too.
the following example is in the freeipa podman container in the HPC stack
[root@freeipa-1 ~]# podman exec -it freeipa bash [root@freeipa-1 /]# ipa user-find ipa: ERROR: did not receive Kerberos credentials [root@freeipa-1 /]# kinit antony Password for antony@CLUSTER.INTERNAL: [root@freeipa-1 /]# ipa user-find antony -------------- 1 user matched -------------- User login: antony First name: Antony Last name: Cleave Home directory: /define/home/antony Login shell: /bin/bash Principal name: antony@CLUSTER.INTERNAL Principal alias: antony@CLUSTER.INTERNAL Email address: antony@cluster.internal UID: 2006 GID: 2006 SSH public key fingerprint: SHA256:/M1Hr8rxG+Im8OOiPeDAqJNEDlyvvpTpAa8hnPJvEaI (ssh-ed25519) Account disabled: False ---------------------------- Number of entries returned 1 ----------------------------
but I don't want to type it. That's a bit of a tough cookie this is how kerberos works if you have the password in a file however you can cat it like so:
[root@freeipa-1 /]# kdestroy -A [root@freeipa-1 /]# klist klist: No credentials cache found (filename: /tmp/krb5cc_0) [root@freeipa-1 /]# ipa user-find antony ipa: ERROR: did not receive Kerberos credentials Password for antony@CLUSTER.INTERNAL: [root@freeipa-1 /]# ipa user-find antony -------------- 1 user matched -------------- User login: antony First name: Antony Last name: Cleave Home directory: /define/home/antony Login shell: /bin/bash Principal name: antony@CLUSTER.INTERNAL Principal alias: antony@CLUSTER.INTERNAL Email address: antony@cluster.internal UID: 2006 GID: 2006 SSH public key fingerprint: SHA256:/M1Hr8rxG+Im8OOiPeDAqJNEDlyvvpTpAa8hnPJvEaI (ssh-ed25519) Account disabled: False ---------------------------- Number of entries returned 1 ---------------------------- [root@freeipa-1 /]#
I can't login to the new node I added
FreeIPA allows you to configure login auth to a whole cluster from one place. You are probably hitting a host based access control restriction. Check /var/log/secure or system journal for errors like this:
Nov 07 09:52:25 login-1.cluster.internal sshd[88168]: Authorized to antony, krb5 principal antony@CLUSTER.INTERNAL (ssh_gssapi_krb5_cmdok) Nov 07 09:52:25 login-1.cluster.internal sshd[88168]: pam_sss(sshd:account): Access denied for user antony: 6 (Permission denied) Nov 07 09:52:25 login-1.cluster.internal sshd[88168]: fatal: Access denied for user antony by PAM account configuration [preauth]
see that permission denied. . . that's the killer here
how to fix is system dependent but on the NUIG cluster I have modified the allow_all to allow access from the cluster.internal hostgroup. This is NOT the same as the DNS zone. We can check the hbac-rulse here
[root@login-1 ~]# ipa hbacrule-find -------------------- 4 HBAC rules matched -------------------- Rule name: allow_admins Service category: all Enabled: True Rule name: allow_all User category: all Service category: all Description: Allow all users to access any host from any host in cluster.internal hostgroup keep it up to date! Enabled: True Rule name: allow_jack Service category: all Enabled: True Rule name: allow_systemd-user User category: all Host category: all Description: Allow pam_systemd to run user@.service to create a system user session Enabled: True ---------------------------- Number of entries returned 4 ----------------------------
and you can see the settings for each of those like so:
[root@login-1 ~]# ipa hbacrule-show allow_all Rule name: allow_all User category: all Service category: all Description: Allow all users to access any host from any host in cluster.internal hostgroup keep it up to date! Enabled: True Host Groups: cluster.internal
see that host group! that's why antony can't login to login-1
[root@login-1 ~]# ipa hostgroup-show cluster.internal Host-group: cluster.internal Member hosts: head.cluster.internal, small-001.cluster.internal, small-003.cluster.internal, small-002.cluster.internal, small-004.cluster.internal, test-001.cluster.internal
lets fix that by adding it to the hostgroup
[root@login-1 ~]# ipa hostgroup-add-member cluster.internal --hosts=login-1.cluster.internal Host-group: cluster.internal Member hosts: head.cluster.internal, small-001.cluster.internal, small-003.cluster.internal, small-002.cluster.internal, small-004.cluster.internal, test-001.cluster.internal, login-1.cluster.internal Member of HBAC rule: allow_all, allow_admins ------------------------- Number of members added 1 -------------------------
I cant get my replica container to start it gets partway and silently fails
sometimes this happens however I spend hours bashing my head against a wall so you don't have to.
i found that no matter what options were passed to the container it would just get stuck. So I used the following command to wipe EVERYTHING and start again and jump into the container
DO NOT RUN THIS ON THE LAST FUNCTIONING IPA SERVER WITH CA
podman rm -f freeipa; rm -rf /var/lib/state ; mkdir /var/lib/state ; podman run --network=host --detach --name freeipa --replace --user root --volume /var/lib/state:/data:Z -it docker.io/freeipa/freeipa-server:rocky-8 ipa-replica-install --domain cluster.internal --server head.cluster.internal --admin-password=REDACTED podman exec -it freeipa bash
once inside I could see that the ipa client was functional so I used kinit to become an admin user and manually added the server to the ipaservers host group and complete the ipa replica install as per this guide
https://www.freeipa.org/page/V4/Replica_Setup
this is approx what I did (I lost the output)
kinit antony ipa hostgroup-add-member ipaservers --hosts freeipa-1.cluster.internal ipa-replica-install --setup-ca --realm CLUSTER.INTERNAL --domain cluster.internal --setup-dns --auto-reverse --auto-forwarders --no-ui-redirect
at this point there is a stupid prompt that it can't use itself to resolve dns. . . yep it aint setup yet so say yes to continue it should then eventually complete and you have a functioning container
ok so maybe you don't want that admin password in the commandline of the container for the world + dog to read lets fix that to fix we
- delete the container but not the state folder
- create a new container without the password (there is a keytab in the state folder that takes it's place)
- create a systemd unit file to start this on boot as it gives up more failure control that and always restart podman container (i.e. only retry 10 times and give up)
- start the service
podman stop freeipa podman create --network=host --name freeipa --user root --volume /var/lib/state:/data:Z docker.io/freeipa/freeipa-server:rocky-8 ipa-replica-install --domain cluster.internal --server head.cluster.internal --setup-ca --principal=admin --setup-dns --auto-reverse --auto-forwarders --no-ui-redirect [root@freeipa-1 ~]# cat >/usr/lib/systemd/system/ipa-podman.service <<EOF # ipa-podman.service [Unit] Description=Podman container-freeipa-replica.service Documentation=man:podman-generate-systemd(1) Wants=network.target After=network-online.target [Service] Environment=PODMAN_SYSTEMD_UNIT=%n Restart=on-failure RestartSec=10s ExecStart=/usr/bin/podman start "freeipa" ExecStop=/usr/bin/podman stop --ignore "freeipa" -t 30 KillMode=no User=root Group=root TimeoutStartSec=180 Type=forking [Install] WantedBy=multi-user.target default.target EOF systemctl start ipa-podman systemctl enable ipa-podman
and it's been runnning for an hour while I write this wiki page
[root@freeipa-1 ~]# systemctl status ipa-podman
● ipa-podman.service - Podman container-freeipa-replica.service
Loaded: loaded (/usr/lib/systemd/system/ipa-podman.service; enabled; vendor preset: disabled)
Active: active (running) since Wed 2023-11-01 22:07:11 GMT; 1h 3min ago
Docs: man:podman-generate-systemd(1)
Main PID: 31664 (conmon)
Tasks: 1 (limit: 49024)
Memory: 1.2M
CGroup: /system.slice/ipa-podman.service
└─31664 /usr/bin/conmon --api-version 1 -c 9e30b6ca044debd861c5f18ee8ca5f04b1b2c54eeafca39e4515048e4a89f7b4 -u 9e30b6ca044debd861c5f18ee8ca5f04b1b2c54eeafca39e4515048e4a89f7b4 -r /usr/bin/runc -b /var/lib/containers/storage/o>
Nov 01 22:07:10 freeipa-1.cluster.internal systemd[1]: Starting Podman container-freeipa-replica.service...
Nov 01 22:07:11 freeipa-1.cluster.internal podman[31624]: freeipa
Nov 01 22:07:11 freeipa-1.cluster.internal systemd[1]: Started Podman container-freeipa-replica.service.