REST API? Add users via CLI + Acquire an access token

[polarathene]

Main goal is to identify a config file + CLI commands for supporting a test:

• Add an identity with an associated email address for authenticating via Rauthy.
• Retrieve an access token for that identity (like client credentials flow, but without a browser interaction involved).
• Query /userinfo OIDC endpoint for a JSON response containing an email claim with that user email address.

Presumably a mocked /userinfo endpoint that sends that JSON response when the auth header matches an expected hard-coded access token is simpler (current approach). I was just curious about automating integration with a proper auth server.

I don't have much time presently to give Rauthy a try, just raising this Q in advance :)

---

Admin API Keys with fine-grained access rights | Source: README

Recently I was advised about using an API key to approach this:

since Rauthy introduced the API Keys, this should be pretty straight forward.

What you could do as well, is to create a Test / Dev SQLite once (with GUI or whatever) and then just re-use that DB, either by copying the file itself or use the MIGRATE_DB_FROM config var.
This way you would always have access with your API Key and depending on the access rights, you can do almost anything without a GUI or browser at all.

I had a quick browse/search on the docs but I didn't see anything regarding an API key?

There's not much context about the API key, is there a REST API to use it with?

With Ory Hydra their API has routes under admin/ that are advised to deployed with restricted access (eg via some auth header checks), I assume it's similar to that but Rauthy may be securing the endpoints via an API key by default?

Use-case scenario

Technical reference

Initial interest was for replacing a mocked auth service for this test case (shell script based):

function __should_login_successfully_with() {
  local AUTH_METHOD=${1}
  # These values are the auth credentials checked against the Caddy `/userinfo` endpoint:
  local USER_ACCOUNT='user1@localhost.localdomain'
  local ACCESS_TOKEN='DMS_YWNjZXNzX3Rva2Vu'

  __verify_auth_with_imap
  __verify_auth_with_smtp
}

# Dovecot direct auth verification via IMAP:
function __verify_auth_with_imap() {
  # NOTE: Include the `--verbose` option if you're troubleshooting and want to see the protocol exchange messages
  # NOTE: `--user username:password` is valid for testing `PLAIN` auth mechanism, but you should prefer swaks instead.
  _run_in_container curl --silent \
    --login-options "AUTH=${AUTH_METHOD}" --oauth2-bearer "${ACCESS_TOKEN}" --user "${USER_ACCOUNT}" \
    --url 'imap://localhost:143' -X 'LOGOUT'

  __dovecot_logs_should_verify_success
}

which is presently uses a hard-coded access token as shown above against this mocked /userinfo endpoint:

# /userinfo
(route-userinfo) {
  vars token "DMS_YWNjZXNzX3Rva2Vu"

  # Expects to match an authorization header with a specific bearer token:
  # https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication#authentication_schemes
  @auth header Authorization "Bearer {vars.token}"

  # If the provided authorization header has the expected value (bearer token), respond with this JSON payload:
  handle @auth {
    # JSON inlined via HereDoc string feature:
    # Dovecot OAuth2 defaults to `username_attribute = email`, which must be returned in the response to match
    # with the `user` credentials field that Dovecot received via base64 encoded IMAP `AUTHENTICATE` value.
    respond <<EOF
    {
      "email": "user1@localhost.localdomain",
      "email_verified": true
    }
    EOF
  }

  # Failed to authorize, close connection and send a 401 status (unauthorized):
  respond 401 {
    close
  }
}

---

The software makes a request to the auth service /userinfo endpoint with an access token:

curl http://auth.example.test/userinfo -H 'Authorization: Bearer DMS_YWNjZXNzX3Rva2Vu'

and expects the JSON response to have an email attribute with a value that matches the login username to authenticate successfully:

{
  "email": "user@example.test",
  "email_verified": true
}

I was curious if Rauthy can be configured with shell commands for the test-suite to replace that mocked service?:

• Is there an HTTP API or config file that can add a user/identity? (Ory Kratos has this via OpenAPI or a separate CLI client)
• Is there an HTTP API for retrieving an access token for that identity, as if the user had been provided one from a proper OIDC flow? (typically this requires a browser interaction for consent IIRC, along with CSRF cookie that I had trouble with using Ory Hydra via CLI)

The 2nd concern for acquiring an access token probably doesn't make much sense outside of a test? So the mocked endpoint with static access token is probably the right approach (and less burden to other project maintainers that don't really grok the actual auth process which is more complicated).

---

Earlier attempt

When I explored what was out there (such as Rauthy and Ory Hydra) this was all new to me and docs were not always straight-forward 😅

I recall having success with registering a client to Hydra where I could easily get the access token with credentials for that client, but that was only useful for a service to authenticate to another with, rather than the next step of getting the /userinfo endpoint to respond with user specific data.

Rauthy from what I recall was focused on a DB + GUI interaction with an admin user, so I think I was more swayed towards Ory Kratos + Hydra due to their broader docs + CLI/API and config. I had also looked into Authelia and Authentik, but recall some friction on both of those (Authelia was a bit interesting as I think it could configure users via a file or LDAP source).

Ory Hydra did offer some alternative flows that did not require a browser interaction, but I ran out of time before having success with those via curl (I think I hit some issues there and their docs are a bit messy / outdated or oriented towards their paid SaaS).

[sebadob]

There are a few ways, how you can achieve all of this, but I need some more information.

I guess you want to run tests against Rauthy in automated test pipelines and stuff, right?
Do you have an environment, where you could just keep a DEV / Test instance of Rauthy running all the time, like some docker / kubernetes, or do you need / want to start it inside the same pipeline, do you tests, and then shut it down again?

I had a quick browse/search on the docs but I didn't see anything regarding an API key?

Unfortunately, that's correct, they are not in the book yet. As I am getting closer to v1.0.0, one of my next TODO's is to update the docs with all the latest features that I added recently. API Keys are not yet in the book, but you should have no trouble getting them up and running by just using the Admin UI. Everything is documented and you even get curl examples after you created a Key.
They provide almost full access to the REST API (if configured so). The only thing they cannot be granted access to is to create / modify and other API Keys.

Create a new API Key

1. log in to the Admin UI and navigate to the API Keys section
2. click on New Key
3. give it a name like dev-test
4. specify access rights for the REST API
5. click Save

6. After saving, expand the new entry and go to SECRET
7. click on GENERATE NEW

8. You will get the API Key itself, which needs to be appended in the Authorization header as well as some curl examples to check the current key you just created

9. copy a curl example to test your key

10. Access the API. In this example, I granted full access to all Users related actions. Usually, it should be pretty clear, what this includes, but if this is not the case, you will get an answer from the REST API, that you don't have access with your current key and which access rights are missing. You could then just modify your key accordingly.
    The OpenAPI documentation is, by default, available only on the internal port, where the prometheus metrics are exported as well, just to reduce the possible attack surface a little bit. This make the link to the documentation in the UI now work, like mentioned in the description:

So, either access it via the internal port, or just expose it with the config:

If exposed, you can just click the link in the UI, if not, the URL would be something like

https://iam.example.com/docs/v1/swagger-ui/

but on the METRICS_PORT, for instance

https://iam.example.com:9090/docs/v1/swagger-ui/

So, now to your goals.
If you can keep Rauthy running all the time for your tests, this would be the easiest scenario. You could modify everything to your liking and adopt your tests and pipelines easily with all the necessary data like API Keys. If you need to start / stop with each pipeline, I would do it differently:

1. Solution

Create your whole instance like you need it, with everything set up. The version, again, depends on your environment. If you have some test postgres running somewhere, just connect to this instance inside the pipelines, so you data never changes. Otherwise, use the SQLite version of course. You could then provide your database file, which should be < 1mb actually, in whatever way you like, so you get it into your container. Then I would do the following:

1. Have the file inside your container but don't use it directly

2. For your testing, just so you can start "clean" each time again, use an in-memory SQLite
   DATABASE_URL=sqlite::memory:

3. Use Rauthy's migration feature to migrate your test file into the in-memory database with each start: MIGRATE_DB_FROM=sqlite:data/rauthy.db

4. Add an identity with an associated email address for authenticating via Rauthy.

You could do this dynamically with the API Key, or when you use the DB migration procedure from above, you can work with a static user as well, once it has been set up. Via API, you can do a POST on /auth/v1/users to do this like documented in the Swagger UI.

2. Retrieve an access token for that identity (like client credentials flow, but without a browser interaction involved).

That's actually the "tricky" part. You could of course just use the password flow, which is just a simple POST request, but that would not include the email claim you need in the next step. This means you need to do a code flow using a client, that actually is allowed to use the email scope. That one you would also pre-configure and set up in the above step. As soon as email is in the scope, you will get email and email_verified claims in the /userinfo.

The code flow is done via a browser, like you mentioned already. To prevent Login CSRF attacks, you actually need a session in the backend in the Init state. Getting this is easy, you just do a GET request on the /authorize endpoint first to actually get the HTML. With it, you get a Set-Cookie header with your session cookie. You also need to extract the CSRF token. I am doing this in the integration tests without any headless browser as well:

rauthy/rauthy-main/tests/handler_auth.rs

Line 72 in eec43bf

async fn test_authorization_code_flow() -> Result<(), Box<dyn Error>> {

You can extract the cookies like so:

rauthy/rauthy-main/tests/common.rs

Line 141 in eec43bf

pub async fn cookie_csrf_headers_from_res(res: Response) -> Result<HeaderMap, Box<dyn Error>> {

This is the only tricky part about it. Afterwards, you can do the POST, which will "redirect" you to your callback endpoint. You can then use the code to retrieve the tokens, like shown in the test.

3. Query /userinfo OIDC endpoint for a JSON response containing an email claim with that user email address.

That is simple. Just provide the access_token as Bearer in the Authorization header. You will get the response you need.

Is there an HTTP API or config file that can add a user/identity? (Ory Kratos has this via OpenAPI or a separate CLI client)

API yes, like mentioned above. Adding things by file is not supported. You would use the above approach with creating the test database just like you need it in advance and then just use that file inside your pipelines.

Is there an HTTP API for retrieving an access token for that identity, as if the user had been provided one from a proper OIDC flow? (typically this requires a browser interaction for consent IIRC, along with CSRF cookie that I had trouble with using Ory Hydra via CLI)

No. I planned to add something to inspect the token responses after you configured everything, so you can get a preview of what the final token for a given client + user would look like, but even when that comes, these tokens will not be valid and expire immediately for security reasons.

2. Solution

If you are just using this for testing and nothing else, you could actually do the same that I do with Rauthy itself for its integration tests.
I am starting the backend in test-mode, which will apply DEV migrations, data, clients, ...
This is kind of the same approach like the one with the pre-configured database above, but programmatically specifically tailored to the integration tests.
You would always have these accress credentials:

pub const CLIENT_ID: &str = "init_client";
pub const CLIENT_SECRET: &str = "LjERi0WSEz1E9OY9KFJaMjlwV1Uf3nuIuOUnJnoJQNm2i7YMjTDMy4PbAKnYRgFy";
pub const USERNAME: &str = "init_admin@localhost.de";
pub const PASSWORD: &str = "123SuperSafe";

Also, the admin@localhost.de will exist with the 123SuperSafe password. If you would need to add users dynamically, or with a specific client id or config, you would need to then add these each time via API beforehand. But that's actually what I do in all the integration tests as well, because they don't use any headless browser.

To start in test mode, you would however need to start the binary directly, or modify the docker run cmd and append test as the startup argument. For this case, I would actually build a new container image, because the test mode expects the config file rauthy.test.cfg and will panic, if it does not exist. If you like that approach, you can just copy https://github.com/sebadob/rauthy/blob/main/rauthy.test.cfg by using a new Dockerfile + use another start command CMD ["/app/rauthy", "test"].

3. Solution

I need to double check something, but it might be possible with the default scopes to get the same result by just using the password flow, without the need to extract CSRF and Session tokens and so on, but I need some time later to verify this.

[sebadob]

3. Solution

This is actually working. I just checked it while testing the latest release.
If you just need the email and email_verififed from the userinfo, there is an even easier way.
You should not do this in your pipeline, but I am using the

1. allow the password flow for a client
2. set at least email in the Allowed and Default Scopes

The default scopes is the important one here. Since you cannot provide the requested scope during the password flow, Rauthy will always use the set default. This will make the login a lot easier! This flow does not use the default login CSRF protection.
You can simply do a POST request only on the /token endpoint with Content-Type: application/x-www-form-urlencoded.
In this example, I used the rauthy client for my tests. Don't do this in your pipeline. The rauthy client uses anti-lockout mechanisms and things will get overridden with the next restart. But just as an example from Postman in this case:

This is way simpler than using the code flow.