Data missing when importing realm

Configuring the server
1

After testing keycloak exploratory, I want to setup an installation script. First for local development and later for the test servers. I exported my manually configured realm into a json file (tried UI export and during startup, but seems to be the same except for passwords, like the doc says). But the import of my realm gives me a headache. It seems that a lot of configuration is missing. Like my User Storage Provider. I tried import on startup. This deletes the realm first if it already exists and then imports my realm. Result is a completely naked realm. After that I tried import via admin CLI. Update and partial import. But there are also a lot of configurations missing. Most values are default though, I think.

Commands I used:
For startup:
./standalone.sh -Djboss.socket.binding.port-offset=100 -Dkeycloak.migration.action=import -Dkeycloak.migration.provider=singleFile -Dkeycloak.migration.file=/vagrant/export.json

With admin CLI:
First created the realm
./kcadm.sh create realms -s realm=myrealm -s enabled=true

First try:
./kcadm.sh create partialImport -r myrealm -s ifResourceExists=OVERWRITE -o -f /vagrant/export.json

Next try:
./kcdadm.sh update realms/myrealm -f /vagrant/export.json

Would be nice to understand the differences of the different approaches. Especially because the import via startup keeps the server running. Is there an advantage?

edit
I used keycloak 9.0.3

2

I found one way the import works. Deleting the realm and creating a new one via UI. Add realm → select file → Create. That puzzles me even more, cause the json file seems to be good. All other ways do not import the UserStorageProvider. Can anyone tell me what happens on the server if I do the import via “Add realm”?
It’s possible to setup my environment now, but it is really bad for automation, getting environments fast and reproducible.

What I also discovered is, that after deleting the realm and importing it via startup, the realm doesn’t appear in the UI. But I can’t create a new one.

Logs while importing via startup:

07:04:12,984 INFO [org.keycloak.exportimport.singlefile.SingleFileImportProvider] (ServerService Thread Pool – 68) Full importing from file /vagrant/myrealm.json
07:04:15,318 INFO [org.keycloak.exportimport.util.ImportUtils] (ServerService Thread Pool – 68) Realm ‘myrealm’ imported
07:04:15,396 INFO [org.keycloak.services] (ServerService Thread Pool – 68) KC-SERVICES0032: Import finished successfully

server.log while creating via UI:

2020-04-23 07:06:31,998 ERROR [org.keycloak.services.resources.admin.RealmsAdminResource] (default task-13) Conflict detected: org.keycloak.models.ModelDuplicateException: javax.persistence.PersistenceException: org.hibernate.exception.ConstraintViolationException: could not execute statement
…
Caused by: org.h2.jdbc.JdbcSQLException: Unique index or primary key violation: “PRIMARY_KEY_624 ON PUBLIC.REALM(ID) VALUES (‘myrealm’, 49)”; SQL statement:

Is it better if I downgrade to keycloak 8?

3

Search the Keycloak issue register for open “import” issues, for example:

4

I searched the issue tracker, but didn’t find any issue that fits my problem. I don’t get any error message (except for the case I mentioned when I try to import a realm again after a faulty realm import).
And there was one way the import worked. It seems there are a lot of ways to import json files and each works different.
And because there is one that works, it would be good to know the differences to identify the problem. There must be people where the import works, right?
Or is it better to just open an issue?

5

Like most people I struggled to get import/export to work :frowning:

I wrote a blog post about the approaches I tried: Getting started with Keycloak

Search the Forum for “import / export” as see what you find.

6

@lrxw

After some more trial and error the following is working for me:

docker container run -d --name keycloak \
  -p 10001:8080 \
  -v $(pwd):/tmp \
  -e KEYCLOAK_USER=admin \
  -e KEYCLOAK_PASSWORD=secret \
  jboss/keycloak:9.0.3

Export:

docker exec -it keycloak /opt/jboss/keycloak/bin/standalone.sh \
  -Djboss.socket.binding.port-offset=100 \
  -Dkeycloak.migration.action=export \
  -Dkeycloak.migration.provider=singleFile \
  -Dkeycloak.migration.realmName=development \
  -Dkeycloak.migration.usersExportStrategy=REALM_FILE \
  -Dkeycloak.migration.file=/tmp/development-realm.json

When the export is complete use Ctrl-C to exit the session.

Import the exported file into my Keycloak container.

Dockerfile:

FROM jboss/keycloak:9.0.3

ENV THEME_VERSION 1.0

USER root

RUN microdnf install -y unzip

COPY json /tmp

COPY ./serendipity-keycloak-theme-$THEME_VERSION.zip /opt/jboss/keycloak/themes
RUN cd /opt/jboss/keycloak/themes && \
    unzip serendipity-keycloak-theme-$THEME_VERSION.zip && \
    rm serendipity-keycloak-theme-$THEME_VERSION.zip

docker-compose.yml:

version: '3.7'

services:

  openldap:
    container_name: openldap
    build:
      context: ./services/openldap
      dockerfile: Dockerfile
    env_file: ./services/openldap/openldap.env
    ports:
      - "389:389"

  keycloak:
    container_name: keycloak
    build:
      context: ./services/keycloak
      dockerfile: Dockerfile
    ports:
      - "10001:8080"
    volumes:
      - .:/export
    env_file: ./services/keycloak/keycloak.env

  serendipity-api:
    container_name: serendipity-api
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - "3001:3001"
    volumes:
      - ./h2:/h2
    environment:
      SPRING_DATASOURCE_URL: jdbc:h2:/h2/serendipity
      AUTO_SERVER: 1
      AUTO_SERVER_PORT: 9091
      DB_CLOSE_DELAY: -1

keycloak.env:

KEYCLOAK_USER=admin
KEYCLOAK_PASSWORD=secret
KEYCLOAK_DEFAULT_THEME=serendipity
KEYCLOAK_IMPORT=/tmp/development-realm.json
KEYCLOAK_MIGRATION_STRATEGY=OVERWRITE_EXISTING
1 Like
7

Hi,

we also had a lot of trouble with keycloak import. Initial creation of the resource might worked fine but update existing configurations won’t work great.

Thats the reason why we create a tool https://github.com/adorsys/keycloak-config-cli to gain a expected configuration based on a keycloak export. It’s useful if you want to stage your keycloak configuration.