To be Reviewed By:

Authors:  Jinmei Liao/Dan Smith/Jacob Barrett/Joris Melchior

Status: Draft | Discussion | Active | Dropped | Superseded

Superseds: N/A 

Related: N/A

Problem

Currently, when a user is authenticated into Geode, the credentials remain active in the server until user logs out or the client's cache is closed. However, in the token-based authentication scenario, credentials can expire when a client is still actively connected to the server, so we need a way to have the server reject further operations from the client when credential expires and notify the client to re-authenticate. 

Anti-Goals

The first cut of the solution is to focus only on client-server connections. Pulse/Gfsh/Rest connections are outside of the scope of this RFC.

Solution

The Solution should have those effects

  • The server should reject further operations once the authentication is deemed to have expired.
  • The server should notify the client that the operation failed because the credential has expired. 
  • The server, once a user expired, should clean the user attributes from the server.
  • The client, once received the notification, should gather the new credentials and re-authenticate (only applies to java client for now, and only for the newer version of java clients).
  • The client, if multiple operations are in flight, should not flood the authentication server with re-authentication requests.

Server Changes

  1. Add a new type of exception, AuthenticationExpiredException, that can be thrown by the SecurityManager authorize method. SecurityManager authorize method is implemented by the third-party. We don't need to care how they determine user expiration. We just need to handle the AuthenticationExpiredExceptionIf it's thrown.
  2. The exception will need to bubble all the way back to the client
  3. When a client is authenticated, an entry with the client's longID as the key is added to map maintained by the ClientUserAuths. The entry is removed and the related shiro subject is logged out  to prevent resource leaks when client's cache is closed. When an authentication expires, we need to do the same with the current authenticated subject and clean that entry out of the map as well. This logic needs to be added to the code path where the AuthenticationExpiredException is thrown.
  4. For event driven operations, such as CQ and registered interests, we will need to check if the credential has expired when dispatching the event. 
    1. Just before sending an event to the client, call SecurityManager.authorize  (in MessageDispatcher.runDispatcher or thereabouts).
    2. If an AuthenticationExpiredException is thrown, catch it, and if the client is the new version client, send some sort of AuthorizationExpired token to the client. If the client is old, close the connection.
    3. After sending an AuthorizationExpired token, wait for a re-authorization request. Maybe only for some period of time before going back to 1 and trying again with the same event.

Java Client Changes

  1. There is logic on the java client to do re-authentication when user attributes somehow "disappeared" from the server, we can piggyback on this logic to handle the AuthenticationExpiredException.
  2. For the older versions of the client, it just needs to be notified of the exception, no re-authentication is required on older clients.
  3. When credentials expired, if there are multiple operations from the client, we need to do something to prevent the client from sending out multiple re-authentication requests to the authentication server.
  4. For event driven clients (CQ/registered interests clients):
    1. handle the AuthorizationExpired token when it is received from the server by re-authenticating (in CacheClientUpdater.processMessages or thereabouts).
    2. For older the clients, connection to the server will be lost after credential expired.

Things to verify:

We need to make sure in the following scenarios, the behavior will be acceptable and shouldn't introduce any security leaks:

  1. multi-user mode
    1. Make sure in this mode, expiring one user doesn't interrupt the other user's connections. 
  2. register interests and CQ
    1. In these cases, when the user making the CQ and registered interests has expired, no further data will be sent back to the user unless re-authenticated.
  3. function execution on servers
    1. we will need to verify that when user expires, function executions on all servers will be stopped and re-authenticated.
  4. WAN
    1. WAN automatically re-authenticate itself since the gateway-sender uses a client to send messages. We should verify that re-authentication happens and no messages are lost.

Performance Impact

For on-demand re-authentication, credential expiration is discovered by the server, the client gets the notification then initiate re-authentication. Client operations will be interrupted at some period, the client will try re-authenticate only once for this operation again, hopefully the 2nd time, it will succeed.

Backwards Compatibility and Upgrade Path

The exception thrown by the server will be passed down to the older clients. But in order to re-authenticate automatically, users will need to use the newer-version java client.

Prior Art

N/A

FAQ

N/A

Errata

N/A

  • No labels

24 Comments

  1. What will the older clients do in response to the AuthenticationExpiredException? Will they disconnect?

    1. I believe if the client didn't do anything to handle that exception, no further operations will succeed.

  2. What versions of the client will gain the capability to re-connect?

    1. The next released version will gain the capability of re-authenticate

  3. Regarding client re-authentication (Java Client point 1) - should the AuthenticationExpiredException  not bubble up to user/application code so that any relevant action can be taken at that level? If we simply try and re-authenticate it could just cause an endless loop if the SecurityManager keeps rejecting.

    1. The client will only try re-authenticate once, if it still get an exception, it will just throw that exception.

      1. I think this touches on something that is not quite spelled out in the proposal, but maybe sort of implied. The re-authentication will only work if the client gets some new  credentials when it re-authenticates itself. I think the way do that is to for the user to provide a AuthInitialize class on the client and for the client to call the user's AuthInitialize implementation as part of that automatic re-authentication.

        1. Correct, when client get the AuthExpiredException at the firs time, it will call AuthInitialize's "getCredentials" method again to get a NEW pair of credentials (hopefully) to re-authenticate.

  4. when server detected one authentication metadata for a client is expired. The server will send an exception back to client? via which channel? client notification?

    1. the client will receive this exception (wrapped in ServerOperationException) instead of the normal result of the operation it's doing (e.g. put, get etc).

  5. So when the authentication metadata for a client is expired, the server will NOT notify the client immediately. Instead, it will wait for this client's NEXT operation to throw back  ServerOperationException with cause is AuthenticationExpiredException. Is my understanding correct?

  6. Wanted to make few topics clear here:

    • Event notification (from server to client)

    Can this proposal be only for client-server operation and not for event communication from server to client. And if there is any further ask from community, it will be implemented in future.

    • WAN communication

    Again, can this proposal be only for client-server re-authentication and not for WAN event delivery.

    • Behavior with Older client

    Option 1: As the idea is to re-authenticate, can existing exception like "AuthenticationFailedException" be used/thrown with the message "Authentication expired"; both old and new client supports this exception. And if the server side security mechnism is going to adopt re-authentication with their authorization (as proposed here); they will/have-to be updating their old clients to re-authenticate. Because with this proposal there is no way the old client can get around the exception thrown during authorization. 

    Option 2: Since the "AuthenticationExpiredException" is new, for older client some known exception has to be returned, in that case, we could throw "AuthenticationFailedException" with the cause. And again we need to call-out document that if they are adopting this feature, there older client needs to be updated to handle this exception...Please correct me if there is other way you are thinking to address this.




    1. I think if we are still sending CQ/register interest responses indefinitely, that seems like a security issue if a users credentials have expired. Similarly with WAN. So I'm not sure deferring CQ and WAN support makes sense.

      WAN at least will probably work with the above proposal since I think WAN messages still go through authorization? We'd have to double check that.

    2. I must admit that I find this RFC too narrow to make any real determination on if it is a good approach.

      I think the  RFCs usecase is too restrictive, are there any other RFCs for the other uses cases that users/clients require? REST, gfsh, CQ, WAN? I think without ANY knowledge on what it would take to have a complete end-to-end solution, this RFC might be proven to be completely insufficient.

      I suggest another approach where we obtain the relevant information about all the other connectivity usecases. Then evaluate how to resolve each of those and see if there is any overlap in the approaches. Knowing THAT might help me better form a complete picture of the approach and determine if it is acceptable.

      Without it, I think the area of risk is too large. I believe without a suggestion of HOW and WHEN the out-of-scope usecases are going to be addressed, this RFC (although sufficient for this singular usecase) might not be enough for me to vote for sign-off.

      I would be more interested in the final, end-to-end, solution that will resolve ALL the connectivity usecases that currently suffer this problem.

      1. Agreed that we need to at least consider these other protocols and explain what will happen to them.

        Since they all go through the same security manager, I would expect that most of them will start failing when the token expires (EG, a long lived gfsh connection?). I think that would be totally acceptable and ok to not worry about reauthentication. Same probably goes for REST.

        But I do think that it is important that when a token is expired, all access is cut off within some reasonable time window.

  7. I don't know if anyone has called this out yet, but if there is an introduction of another checked exception onto the `authenticate` method, it could be argued that the public API has been changed.

    Without a new API, Option 1 (as suggested by Anil) is close to the only option that one could acceptably satisfy both new and older client implementations. There needs to be guidance for both new and older clients on HOW to handle re-authentication. As I think even new clients would need to build in that logic vs it being provided out-of-the-box.

    1. It's an unchecked exception, so we are not breaking the api.

      1. Technically, I don't think the AuthenticationExpiredException ever makes it back to the user. We catch that, and then re-authenticate. AuthenticationExpiredException is thrown by the authorize  method, when we reauthenticate we call the authenticate  method. If that fails, whatever exception it throws goes back to the user (eg NotAuthorizedException).

        Practically speaking, I wouldn't be too concerned by an new unchecked exception that is thrown by the users own callback. If they are throwing it inside their callback they can be reasonably expected to handle it in their own code as well.

        1. Dan Smiththank you for pointing that out... Not sure where my head is/was at... the authorization needs to throw said exception once the token has expired.

          With that said, why would we not throw a `GemFireSecurityException` instead of a new exception? That way older clients would not have to upgrade to the latest version, they would merely have to handle the thrown Exception. (except for the fact that still have remnants of the original donor project)

          1. How do you suggest one distinguishes a expiration from any other security exception if all we throw is one expiation? Don't say by message since that is VERY bad practice. One should define unique exceptions where the message is mostly diagnostic information.

            The old clients are handled in this solution by use of our backwards compatibility framework. In this case it does send a specifically formatted message exception that the client is currently keying off of to work around a previously introduced bug that made it dependent on specific exception messages. That case is being deprecated out.

            1. Wanted to get things clear on few open items we are discussing (sorry getting little deeper into implementation):

              • What does the old java clients receive; I assume it can't receive new exception (if added)
              • What i am getting is there will be new logic in client; which on expired-exception invokes authentication check with server (retry) and that logic will throws back whatever exception it gets as part of authentication check (if there is); not the newly introduced expired-exception. In that case, user application won't be knowing the reason why it failed (say authentication has expired). Is this fine? or do we want to alter the exception-message to include the reason.
              • Native client - I believe the native client code on the server side uses the same path as java client code path; if that is the case, is the same exception handling done in native client side.
              • Wan messages on the receiver side is processed in GatewayReceiverCommand.java which does invoke authorize check; again as it is asynchronous delivery, what is the expectation? Since its connection/delivery between peer-to-peer, there is no need to check at this level.
                • The new exception will be converted to an existing exception type that's understood by the old client and there is logic in the older client to handle that exception with a specific string to re-authenticate, so old java clients will behave the same as the new client.
                • If the native client behaves the same as the java client, then it should handle the same exception.
                • For peer to peer connection, if there is authorization check on the receiving end, we should handle the exception on the sender side.
                1. >> there is logic in the older client to handle that exception

                  Then, the difference between old and new client will be auto re-authentication? which will be in new client?

                  Also, when this is implemented, need to make sure there are tests for operations that span multiple servers and the expiration is reported on only one server (timing related). E.g. putAll, functions, etc...