CouchDB — How to design and implement a couch per user when using JWT-Auth

Summary

The PouchDB/CouchDB combination is a powerful tool for building offline-first apps and modules.

Often, we have different users using our app, wanting to keep their data private and secured. This can quickly become a challenge when Frontend-initiated data sync is in the mix.

This article illustrates the challenges faced and possible solutions for the task of replicating user-owned data from the IndexDB to a remote CouchDB database using JWT-Authorisation.

Prerequisites

• Basic understanding of what PouchDB/CouchDB is
• Familiarity with the general concepts of JWT-Authentication

The database design

The access to Databases in CouchDB can be defined through a role-based configuration.

There are admins and members, which can either be specified by username or by a role a CouchDb user needs to have.

This is an example configuration of a CouchDb database:

Since our data sync happens directly between frontend and database, there are some problems and risks to consider.

And you may find yourself wanting to implement a rather odd sounding pattern to people like me who come from a SQL background.

To keep this article hopefully rather short, check out these articles that describe the pattern well:

https://titrias.com/how-to-pouchdb-couchdb-database-per-user-made-simple/

https://www.joshmorony.com/creating-a-multiple-user-app-with-pouchdb-couchdb/

Note: Even if you use client credentials and not admin credentials in the frontend, privacy protection is pretty lacking here when using one database, as traditional “user-ids” can be manipulated in REST calls. At least without adding extra products on top.

So “one database per user” it is.

It has even made it into CouchDBs implementation and can actually be configured quite easily: https://docs.couchdb.org/en/stable/config/couch-peruser.html

Authentication: Pick a suitable option

CouchDb offers multiple authentication methods. Basic Auth, Cookie auth and JWT auth. Cookie auth also requires a username+password login.

Since in our use-case, we already have an established authentication and authorisation process in place, I did not want to add another user including credentials for CouchDB.

It would have been a cumbersome process to make that happen behind the scenes, since we offer SSO and do — in some cases — not have any credentials to “just pass” to CouchDB to migrate our users to that DB. Creating and managing additional user credentials under the hood only leads to a lot of custom effort we can skip if there is another way.

CouchDB only powers a certain feature(set) of our application, as it can be common in distributed systems.

That’s why I chose JWT authentication for easier authorised user access. Also, Basic-Auth is not how we authorise REST-calls anymore and Cookie-Auth is not stateless, making the REST-API kind of stateful as it relies on a login session. Just to be a little nitpicky here. Also, we already use JWTs to interact with several other services and I wanted to stick to our standard rather than introducing a custom workflow for CouchDB.

This also means that we have an existing backend that I am going to use for this setup to create JWT tokens for a user who is logged into our app.

If you have a setup where creating new users fits greatly, go for the Cookie-Auth option. This is a fair warning that the JWT thing is — unfortunately — going to be a bit of an effort here.

Still, I want to include a https://security.stackexchange.com/questions/248195/what-are-the-advantages-of-using-jwt-over-basic-auth-with-https pretty neat wall of text why Basic-Auth should be avoided in favor of JWTs whenever possible.

Implementing JWT-Auth

Let’s get into the implementation. For dev purposes, I chose to use the official docker image of CouchDB:

Give it some very nice, very secure admin credentials. CouchDB will create a server admin user from them. You can later use it to access Fauxton UI, CouchDBs Web-UI admin panel. Depending on your desired config, you will need to modify one or multiple .ini files and mount them into the docker container. .ini files are where configuration takes place in the CouchDB realm.

You can find these under /opt/couchdb/etc and save them on your local device for modification. Or modify the files directly, but be cautious as the config will be lost once the docker container is deleted.

Wherever you want to place your config ([default, local, staging, whatever].ini), the following alterations are required:

The basic-auth and cookie-auth handlers are worth keeping around for testing purposes and for accessing the Fauxton admin panel. Just configuring the JWT-handler is going to cause problems when trying to log into the UI. Additional JWT related settings:

Here comes the first lesson I learned by debugging expired tokens being accepted by CouchDB. This is, multiple times actually, mentioned in the documentation and the .ini-file comments. So basically a case of rtfm.

But while I agree on being responsible for carefully reading and testing myself, I consider this choice a dangerous default configuration.

If one forgot to configure exp in the required_claims, it will not be validated. Expiration is one of the key benefits of a JWT as a credential. If you wanted to configure never expiring JWTs, you should do that as a conscious choice by e.g. setting exp to some date-time in 2999, at which point you should really question what you are doing and why.

CouchDB will change this in a future release (found issue in 3.3.3 (2023–12–5)). Until then, please don’t forget to configure this.

If you wonder why it is important to keep your tokens short lived, why this is crucial and not just a minor detail that can be swept under the carpet for “convenience” reasons, look into this post: https://security.stackexchange.com/questions/219138/having-a-jwt-that-doesnt-expire

Access scopes are configured using the JWT payload. The payload-property pointing to the configuration must be configured in the .ini file as seen in the picture above. From the documentation:

Next, let’s configure our public key (or key if you are using symmetric encryption) that will be used for signing a key.

Here is an example using asymmetric keys.

Next, we have to generate the JWTs. How this is done depends on your Stack, I am using a SpringBoot/Kotlin backend for this. As there are tons of great resources out there on how to create and sign JWTs, I consider this out of scope for this article. Still, some things might be worth noting here.

Our JWT tokens are provided to logged-in users only via a protected token endpoint. This is why I didn’t use Refresh Tokens, as our established auth-session acts as the mechanism to verify the identity of someone requesting a new token.

Be wary of couch_per_user using JWT auth

So now we have our tokens ready to provide our users access to their database. So is all we have left to do to enable CouchDBs builtin couch_per_user feature and we are good to go?

Unfortunately, no. At least I decided against it because of a small but important detail about how that would work when using JWT-authorisation:

Have a look at this discussion: https://github.com/apache/couchdb/issues/4663

Especially this:

So you’d have to make every user a server admin to make this work. Apart from the required ability to programmatically create a database, all other privileges associated with the server admin role are waaay to extensive for us to want to grant them to every normal user.

Imagine implementing this in combination with the by default non-expiring JWTs, resulting in some pretty overpowered tokens being handed to each and every (self-registered) regular user. OuchDB! (I had to include this one, badum tss…) Depending on the place and role of CouchDB in your application, think about this carefully. Imho, just don’t use it.

This privilege problem is not present when using couch_per_user with Basic-Auth/Cookie-Auth, since users in the _users table are not required to have the _admin role for CouchDB to create databases for them.

The problem seems to lie in the implementation of couch_per_user via JWT auth.

A contributor offered a code snipped in the previously mentioned discussion that might be used to tackle the _admin problem, but I did not bother checking what it actually does and if it solves the concerns regarding this use case. I would not want to have to rely on patching my CouchDB instance with some custom code trying to somewhat limit the _admin role despite regular users having it.

I did not find any sign of a solution for this being merged or planned to be integrated in future releases.

So what do we do now? Give in to the admin party and go on with our day? That would certainly not be very responsible. The risk of this configuration causing unwanted side effects such as potentially enabling our users to change cluster configs or perform various other server admin tasks available via REST API is definitely there.

You would have to do potentially rather extensive request blocking and network traffic configuration to be somewhat confident that this is not going to happen.

I’d rather not have to worry about that by finding a way of not having to give our users the _admin role in the first place. It’s simply too error-prone to give users such a powerful role and then try to limit the impact of it with various band-aids thrown at the problem.

So let’s move on by changing our plan a bit.

Implementing couch per user

A bit may be an understatement, as we are basically implementing couch per user ourselves instead. Don’t fret! It’s not that big of an effort and also brings us some benefits of greater control over the implementation. The following code exists in the context of an offline-first time-tracking tool, hence the occasionally present domain specific variable names. The required CouchDB config metadata will be associated with a PlexUserDocument, which is simply our general user entity. So what we are going to do is to change our user registration process a little bit. On registration, additionally to the tasks we perform anyways, we are going to create a CouchDB database for our newly registered user.

With CouchDBs built-in implementation, we are required to follow a certain naming convention that contains a hash of the username. Depending on the hash-function, this can quite easily be reversed. Others have pointed out this can be used for leaking user lists if not prevented by blocking the list dbs endpoint (/_all_dbs). We can further minimize that risk by not including any user identifiers in the name, but use a random UUID. We do this via backend REST calls, authorizing them using a JWT for the actual server admin user we configured.

First, we create the DB itself, then we populate the DBs security object.

Note that we configure the admin member to be our server admin user. Since we’ll only grant that role to, well, our server admin, this is actually what we want to do.

A user is only a basic member of their own database. No admin rights whatsoever, all actions requiring admin privileges are performed through our backend only if and when it makes sense to do so. Our admin tokens are never exposed to the frontend and only used for background tasks.

With this approach, we will also be able to give other users access quite simply by assigning them the appropriate member-role as a guest (although we will need to make sure that only the owner has write access — by default, all members can read and write). This will eventually be a requirement for our app in the future, so for us it’s useful to keep that in mind. For now, this simple config is sufficient. For now, the member role is basically a one-to-one mapping between user and role. To round things up in the backend, we’ll need to extend our token endpoint from earlier a bit:

We’ll pass along the name of the DB, as the frontend could not determine that on its own. Also we add the value contained in the “exp” claim of the JWT to tackle a problem we’re about to encounter.

But first — so far so good:

When a user registers, a new database is created. During an active login session, a user can query the database using the metadata and token provided by a backend service.

The basic PouchDB/CouchDB config might look like this:

This can now be used to sync changes between the browsers IndexDB and the users remote DB.

Use the token endpoint to get required DB config data. Only works if our own Cookie-Auth is active during a valid session. This also only works when the user is online. That’s ok since for our offline-first approach, it is primarily important that the data gets persisted on the local device and can later be synchronised, when a connection is (re)gained.

Initially, this already works like a charm. That is until the token expires. Normally, we’re quite used to receiving 401 at some point and simply refreshing the token once that happens, usually by using a refresh token. CouchDb however, adds a little spice to its 401 responses: the WWW-Authenticate header. This little bad boy causes the browser to open a good old basic auth popup asking our users for a username/password. This is not only ugly but makes absolutely no sense for our users, since they do not even have any credentials to pass in there.

There are several opportunities to get rid of it. Some are illustrated here: https://stackoverflow.com/questions/32670580/prevent-authentication-popup-401-with-couchdb-pouchdb

First workaround:

While this config leads to the REST API returning the desired 401 response for us to work with in the frontend, it also leads to this

when trying to access the Fauxton admin panel.

You can keep it this way if you don’t mind being locked out of the admin UI. But that’s certainly not optimal, especially if you plan on using Fauxton, which is a handy tool for database admins.

The other workaround would be to use a nginx reverse-proxy to remove the header from (Fauxton related) requests. That could leave you with a solution that is working for both scenarios, but it’s a little overkill to host an extra product for the sole purpose of hiding a single header. Another option is to handle the situation in the frontend, which is what I’m going to do.

Handling token and sync refresh in the Frontend

With this setup, we basically need to prevent the 401 from occurring. Similar to when you need to proactively check token expiration when using Access- and Refresh-Tokens that have the same expiration time (token rotation with short-lived, non-reusable Refresh-Tokens), receiving the 401 will be too late for us to act. That’s why we passed the expiration time to the Frontend earlier. We’re going to have to periodically check it.

These are the properties we are going to work with. We use Redux as our state management framework.

DbConf contains the token and metadata like expiration time and database name we fetch from the backend.

IsDbTokenExpired is a simple flag we can set when our check fails.

DbSync is a ref that stores the PouchDb sync object. It’s not included in the state because it’s non-serializable.

IntervalRef is going to be used for the periodic checking of the expiration time.

IsBookingDbError is a simple component state variable that can be used for general error handling like displaying errors.

ERR_TOKEN_EXPIRED is part of the response of the 401 we’re hopefully no longer going to receive.

Canceling the sync will lead to a state change, triggering a re-initialization using the effect shown in the snipped above.

This definitely needs some real-world testing and hardening, but seems to be working fine. Shortly before a token expires, it is invalidated, the sync using it is canceled, a new one is fetched and finally a new sync is created with the new token. And that’s it.

That took us a little longer, but we finally got to where we wanted to go. Every user has their own database secured by a proper (non-admin) token-based authorisation using short-lived access tokens and narrow-scoped role-based access.

Conclusion

And that’s how I would implement couch per user with JWT-Auth. This is basically an advice against using the most simplistic tutorial (and even parts of the currently online official documentation) alone for anything other than toy projects. Basically common sense. You can build some pretty cool stuff quickly with CouchDB. But beware. The problems we discovered could have easily been overlooked in the name of blazingly fast development times. It would’ve saved us some time to give in to the _admin thing and call it a day. There would have been some hacky workarounds we could have chosen but all of them stay exactly that: a workaround instead of a solution to the actual problem. I like a saying that goes something like

“If you pull in an external lib/product/etc that code becomes your problem and your responsibility.”

And with that, all of its flaws and pitfalls. So RTFM carefully, nothing is all sunshine and rainbows. Sometimes, you can read it carefully and get presented with questionable practices documented there. It can be wise to temper enthusiasm and take irritation seriously. CouchDB is still a very respectable product but that doesn’t free me from using it responsibly. This of course applies to all external dependencies and tooling, so I thought some general thoughts on that would fit in here quite nicely.

I’m a dependency minimalist. There must be a very good reason and need for me to want to pull in some 3rd party stuff. Quality is extremely important here and the time I’d need to build it myself must be a whole damn lot more than the time required when using a shortcut like this. I’m talking months to years here. And/or the 3rd parties expertise in the field must greatly exceed mine. Also, I like the total number of dependencies of a product to be limited.

Dependencies must be monitored, updated and managed. You’d have to stay up to date with all of these codebases and the changes they make, introducing a whole lot of workload if one took that seriously. The absence of often all of these efforts in Frontend projects where people just npm install half of the internet seemingly without any second thought is one of the reasons why Frontend is my least favorite part of software development. No that’s not DRY that’s just madness. I digress, you get the point.

https://www.reddit.com/r/ProgrammerHumor/comments/992u1p/dependencies_101/

In this case, our alternative would have been to implement offline-online data sync between a client and a (not Couch) DB from scratch, which would’ve completely butchered our time frame due to its definitely not trivial nature. Taking some time to programmatically create the Databases ourselves instead of using out-of-the box features was a good middle ground for our use case.

Last but not least, if you don’t like something about the implementation of a certain aspect of an Open Source project like CouchDB, give them feedback. The ability to do so is one of the great powers of OS. We can all help to make these projects better. I pointed out the issues with the default JWT validation config and that will be changed in future releases. That’s what the community is for.