Hello,
I am trying to setup a keycloak instance on a kubarnetes cluster with user federation to LDAP, However the intial sync takes too long that it gets timed out.
When running the same sync locally on a standalone server it works fine.
My suspicion based on the logs is a too low timeout setting on arjuna transaction reaper.
I have tried adding the following:
<subsystem xmlns="urn:jboss:domain:transactions:5.0"> <core-environment node-identifier="${jboss.tx.node.id:1}"> <process-id> <uuid/> </process-id> </core-environment> <recovery-environment socket-binding="txn-recovery-environment" status-socket-binding="txn-status-manager"/> <coordinator-environment default-timeout="60000" statistics-enabled="${wildfly.transactions.statistics-enabled:${wildfly.statistics-enabled:false}}"/> <object-store path="tx-object-store" relative-to="jboss.server.data.dir"/> </subsystem>
But it still gets timed out after 5 minutes.
Any advice on how to get the sync completed? (presumably by increasing the timeout)
This is the start of the log. (Due to limits I can’t post the full log)
10:21:32,326 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffffac1d0003:-64730a0e:5ea01035:61375 in state RUN
10:21:32,330 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffffac1d0003:-64730a0e:5ea01035:61378 in state RUN
10:21:32,330 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012121: TransactionReaper::doCancellations worker Thread[Transaction Reaper Worker 0,5,main] successfully canceled TX 0:ffffac1d0003:-64730a0e:5ea01035:61375
10:21:32,339 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012121: TransactionReaper::doCancellations worker Thread[Transaction Reaper Worker 0,5,main] successfully canceled TX 0:ffffac1d0003:-64730a0e:5ea01035:61378
10:21:32,353 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffffac1d0003:-64730a0e:5ea01035:61381 in state RUN
10:21:32,357 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012095: Abort of action id 0:ffffac1d0003:-64730a0e:5ea01035:61381 invoked while multiple threads active within it.
10:21:32,358 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012381: Action id 0:ffffac1d0003:-64730a0e:5ea01035:61381 completed with multiple threads - thread default task-36 was in progress with
Edit:
We have actually been able to fix it now.
We added a script change-transaction-default-timeout.cli.
embed-server --server-config=standalone-ha.xml --std-out=echo
echo Changing transaction default timeout
/subsystem=transactions/:write-attribute(name=default-timeout,value=3600)
echo Done
stop-embedded-server
Mounted in a volume under /opt/jboss/startup-scripts