WSO2 Charon - Design

Getting started with WSO2 Charon 1.0.0

WSO2 Charon 1.0.0 is released... It is successfully integrated in WSO2 Identity Server 4.0.0 for identity provisioning.

WSO2 Charon is the open source implementation of SCIM specification, and it is made available under Apache 2.0 license.

You can check out my previous blogs which was written around its Milestone 1 release.

In this post, I will provide you step by step guide to play around with it by running the sampels.

Step 1: Obtaining binaries
Two jars are shipped with the distribution. They are:

1. Charon-Core-1.0.0 jar - This is the library that implements the specification and which can be used by any identity management solution to add provisioning capability.

2. charonDemoApp.war - This is the reference implementation of SCIM service provider which uses Charon-Core for SCIM support. It is a RESTful webapp exposing SCIM endpoint which you can host in a servlet container.

You can either obtain these jars from the release distribution or by building the source code.

Step 2: Setting up SCIM service provider
We need two parties to observe identity provisioning capability. i.e: Service Provider and the Consumer. You can use either Charon-Impl hosted in tomcat or WSO2 Identity Server 4.0.0 as the service provider.

My previous blogs explain WSO2 Identity Server's capability as a SCIM service provider. Therefore, here I will explain how to setup Charon-Impl as a SCIM SP.

1. Download tomcat 7.0.11
2. Replace server.xml and tomcat-users.xml [found in tomcat_home/conf] with the attached files here.
4. Place the attached keystore in your file system.
5. Open the server.xml and locate the HTTPS connector. Edit the keystore file location to point to the above keystore.
6. Run the server with sh catalina.sh jpda run.
7. Access http://localhost:8080/ and click on Manager APP.
8. Login with credentials: hasini@wso2.com, 7786htg
9. Upload the charonDemoApp.war which is obtained from step 1 and access http://localhost:8080/charonDemoApp/ - you should see the Charon home page.

Step 3: Running the samples
1. Compile the source of the charon-samples which is a maven project and found in the release distribution, using the command: maven clean install.
2. Open charon-samples from your IDE.
3 .Go to SampleConstants - here is where all the constants needed to run the samples are hard coded.
          i. Change the KEY_STORE_PATH to your file system location if the default one doesn't work. (This is only needed if you use https as the transport)
          ii. Change the User and Group resource endpoint urls according to your system.
4. Now access "CreateUserSample" class from your IDE. You will see the constants defined at the top - which are the values for the attributes of the user that we are going to create.
5. Run the client. You will see that the user created at server side is returned in JSON format and printed at client side. (You can observe the message on the wire using a tool like tcpmon, as I have shown in a previous post)
6. In the same way, try other samples as well, paying attention to instructions mentioned as comments in the sample code.

Note: You can also run the same set of samples against the SCIM endpoints of WSO2 Identity Server. The configuration that matches with Identity Server, is kept commented out in the SampleConstants file.

That's it.. Enjoy SCIM..! :)

SCIM Interop Event at IETF 83rd Meeting

First interop event organized by SCIM working group (or originally named as Cloud Directory WG), was held in the room: Corot of hotel Concorde La Fayette, Paris on 28th of March from 6-12 CET. 

Purpose: The purpose of this event was to bring together current working implementations of SCIM and test the level of interoperability between each other which in turn could be used as a valuable input to prove the interoperability of the SCIM spec itself for the Birds of Feathers Session  that was held on 29th of March, in order to form a SCIM working group in IETF.

SCIM (Simple Cloud Identity Management) is an emerging standard that is focused on identity provisioning. You may refer to my previous post to get an overview of SCIM. 

Participants: 
Erik & Samuel from Technology Nexus,
Kelly from Sailpoint,
Chuck from Salesforce,
Trey from UnboundID,
Travis from Ping Identity,
Morteza from Cisco,
Emmanuel from BCPSOFT,
Hasini from WSO2,
participated in person while Michael  from Gluu and James from Curion, participated remotely.

Following are some pics I took during the interop event:

Rest of the post is mainly about the interop experiences of WSO2 Charon when it was tested against SCIM service providers and SCIM Clients provided by other implementations.

WSO2 Charon:

Charon is the open source SCIM implementation offered by WSO2 under Apache 2.0 license. You may refer to my previous blog posts to get an idea about WSO2 Charon. Milestone 1 of WSO2 Charon was released in the time for first interop event. You may refer to a nice blog post written by Prabath on M1 release of Charon. We hosted a public SCIM endpoint for interop testing at people.wso2.com.

At the start of the interop event, Every one shared their server credentials with participants and started testing by picking one endpoint at a time and sending requests from their clients.

WSO2 Charon SCIM Client was tested against the SCIM endpoints provided by following SCIM service provider implementations:

• Technology Nexus
• UnboundID
• Curion
• Salesforce
• Ping Identity 

This list doesn't contain all in the participants list above, since Sailpoint offers only a SCIM Client implementation and other service provider endpoints were busy with fixing some of the issues encountered during the interop event.

Varying level of successes achieved when Charon client tested with each of the above endpoints among which highest percentage of success was achieved with Unbound Identity(8 out of 10 scenarios passed) and Technology Nexus(6 out of 10 scenarios passed) endpoints. 

WSO2 Charon-Samples module includes sample SCIM clients which cover all the SCIM operations supported by Charon as of its M1 release. These sample SCIM client programs made it easy to cover all the other working SCIM server endpoints during the 6 hours time period of the interop event.

Interop issues found:  Following are some of the issues found when testing WSO2 Charon client with other server endpoints which caused some operations to fail. We negotiated and discussed about how to align the implementations with the spec in order to overcome those issues.

1.Server expects ETag when update and delete requests are sent from client side. While it is a good feature to support resource versioning in server side, it should not be mandatory for client to set it according to the spec. Therefore agreed that server side should also allow handle the requests which doesn't contain ETag header.

2. Server returns an error when read-only attributes are contained in the payload of an update request. According to the spec: "Consumers must retrieve the entire Resource and PUT the desired modifications as the operation overwrites all previously stored data." The example payloads in the spec also contains read-only "id" attribute in the update request. Therefore, we agreed that spec needs more clarifications whether it should fail or ignore the read-only attributes in the update request payload and update only other attributes.

3. Server had an internal requirement to include a group attribute when creating a user and if a group is not provided, create user operation fails. But according to the spec, group attribute in User resource is read-only. Therefore, we agreed that servers should not mandate to expect group attribute in create User request payload, even if they have internal server requirements to do so.

4. Server replies with dateTime attributes formatted in .NET DateTime strings. WSO2 Charon client expects date time attributes be formatted in XML Schema Datatypes Specification (2008-01-23T04:56:22Z) which is mentioned in SCIM spec. Agreed to follow the same format for dateTime as specified in SCIM spec in order to avoid interop issues even on these minor areas.

5. Server doesn't add an id attribute rather considers external ID as the id of the resource. Still it is not made mandatory to make it unique which may cause issues in retrieving a particular resource.

Other SCIM Clients were tested against WSO2 Charon SCIM service provider endpoints:

• Curion
• Technology Nexus

This list too doesn't contain all in the participants list above, because it took participants quite some time to test against one endpoint and the duration of the interop was 6 hours.

Note: WSO2 Charon endpoint is available for public access and you may carry out interop testing with it anytime and please let us know if you find any issues.

Interop issues found: Following is the only issue reported by the ones who tested against WSO2 Charon endpoint:

1. List user operation returning "resource not found" error.
This operation returns proper response with the list of resources like below, when List user operation is performed with WSO2 Charon client. :

{
"schemas":["urn:scim:schemas:core:1.0"],
"totalResults":2,
"Resources":
 [
  {
   "id":"0f6fd995-38fb-4240-a5ce-961a7032427f",
   "externalId":"umesha",
   "meta"{
          "lastModified":"2012-03-28T05:56:32",
          "created":"2012-03-28T05:56:32",
          "location":"http://localhost:8080/charonDemoApp/scim/Users/0f6fd995-38fb-4240-a5ce-961a7032427f"
          }
  },
  {
   "id":"e942ac6d-476c-4c7a-add3-f4ecb068a2f6",
   "externalId":"hasini@gmail.com",
   "meta":{
          "lastModified":"2012-03-28T05:53:27",
          "created":"2012-03-28T05:53:27",
          "location":"http://localhost:8080/charonDemoApp/scim/Users/e942ac6d-476c-4c7a-add3-f4ecb068a2f6"
          }
  }
 ]
}

But due to an interop issue when the same was performed with other clients, it shows an unexpected behavior which needs to be reproduced and identified with that particular client.

Overall Comments:

I consider the whole interop event was an effective session where we were able to discuss and agree on interpretation of certain points of the spec wrt implementation aspects and identify areas of the spec which needs more clarification in order to overcome some of the interop issues mentioned above. 

It was also a good community meetup where the people who are communicating remotely over the mailing list could get together and meet in person and make their implementations communicate with each other. 

I would like to Thank all participants for collaborating effectively during the session to make the interop event a success.

Now the SCIM spec has a new beginning at IETF after the successful BOF session which was held 29th of March 2012, and the standard will have a long journey to go till it is published as a RFC in IETF.

WSO2 Charon road map in brief:

WSO2 Charon will be feature completed with its 1.0 release and will be integrated into WSO2 Identity Server 4.0.0 so that WSO2 product stack and WSO2 Stratos will be equipped with the standardized identity provisioning feacture based on SCIM.

Implementing SCIM with WSO2 Charon - Part III

In this post, lets look at how to implement few use cases of SCIM with WSO2 Charon.
You may need to refer to my previous posts on SCIM and Charon to get an overview of them.

Being the third post on Charon, lets continue looking at it in the top down approach.

Part II described the composition of Charon-Impl. which is the SCIM service provider reference implementation shipped with Charon. It is a Apache Wink based webapp that utilizes Charon-Core and Charon-Utils to expose a SCIM REST API for identity provisioning operations.

This post will demonstrate the following capabilities of Charon according to SCIM spec.

1). Create user
2). Create group
3). List Users

Setting up SCIM Service Provider with Charon-Impl:

Download CharonDemoApp from M1 Distribution and deploy it in an application server like Apache Tomcat and start the server.
As shown in the diagram of the part-II post, User Resource and Group Resource are now exposed based on Apache Wink JAX-RS implementation, via following URLS.

• User Resource: http://localhost:8080/charonDemoApp/scim/Users
• Group Resource: http://localhost:8080/charonDemoApp/scim/Groups

Sample use cases:

1. Creating a user.. 
This demonstrates how a SCIM consumer creates (provision) a User in SCIM service provider via a REST call in which User attributes are carried as a JSON encoded string.

Client code :  SCIM client uses the previously registered credentials to authenticate the request.

package org.wso2.charon.samples.user.sample01;

import org.apache.wink.client.ClientConfig;
import org.apache.wink.client.ClientWebException;
import org.apache.wink.client.Resource;
import org.apache.wink.client.RestClient;
import org.apache.wink.client.handlers.ClientHandler;
import org.wso2.charon.core.client.SCIMClient;
import org.wso2.charon.core.exceptions.CharonException;
import org.wso2.charon.core.objects.User;
import org.wso2.charon.core.schema.SCIMConstants;
import org.wso2.charon.samples.utils.CharonResponseHandler;
import org.wso2.charon.samples.utils.SampleConstants;
import org.wso2.charon.utils.authentication.BasicAuthHandler;
import org.wso2.charon.utils.authentication.BasicAuthInfo;

public class CreateUserSample {

    //user details
    public static final String USER_NAME = "hasinig";
    public static final String EXTERNAL_ID = "hasini@gmail.com";
    public static final String[] EMAILS = {"hasini@gmail.com", "hasini@wso2.com"};
    public static final String DISPLAY_NAME = "Hasini Gunasinghe";
    public static final String PASSWORD = "dummyPW";
    public static final String LANGUAGE = "Sinhala";
    public static final String PHONE_NUMBER = "9077657623";

    public static void main(String[] args) {

        try {
            //create SCIM client
            SCIMClient scimClient = new SCIMClient();
            //create a user according to SCIM User Schema
            User scimUser = scimClient.createUser();
            scimUser.setUserName(USER_NAME);
            scimUser.setExternalId(EXTERNAL_ID);
            scimUser.setEmails(EMAILS);
            scimUser.setDisplayName(DISPLAY_NAME);
            scimUser.setPassword(PASSWORD);
            scimUser.setPreferredLanguage(LANGUAGE);
            scimUser.setPhoneNumber(PHONE_NUMBER, null, false);
            //encode the user in JSON format
            String encodedUser = scimClient.encodeSCIMObject(scimUser, SCIMConstants.JSON);
            //create a apache wink ClientHandler to intercept and identify response messages
            CharonResponseHandler responseHandler = new CharonResponseHandler();
            responseHandler.setSCIMClient(scimClient);
            //set the handler in wink client config
            ClientConfig clientConfig = new ClientConfig();
            clientConfig.handlers(new ClientHandler[]{responseHandler});
            //create a wink rest client with the above config
            RestClient restClient = new RestClient(clientConfig);
            //create resource endpoint to access User resource
            Resource userResource = restClient.resource(SampleConstants.USER_ENDPOINT);

            BasicAuthInfo basicAuthInfo = new BasicAuthInfo();
            basicAuthInfo.setUserName(SampleConstants.CRED_USER_NAME);
            basicAuthInfo.setPassword(SampleConstants.CRED_PASSWORD);

            BasicAuthHandler basicAuthHandler = new BasicAuthHandler();
            BasicAuthInfo encodedBasicAuthInfo = (BasicAuthInfo) basicAuthHandler.getAuthenticationToken(basicAuthInfo);


            //TODO:enable, disable SSL. For the demo purpose, we make the calls over http
            //send previously registered SCIM consumer credentials in http headers.
            String response = userResource.
                    header(SCIMConstants.AUTHORIZATION_HEADER, encodedBasicAuthInfo.getAuthorizationHeader()).
                    contentType(SCIMConstants.APPLICATION_JSON).accept(SCIMConstants.APPLICATION_JSON).
                    post(String.class, encodedUser);

            //decode the response
            System.out.println(response);
        } catch (CharonException e) {
            e.printStackTrace();  //To change body of catch statement use File | Settings | File Templates.
        } catch (ClientWebException e) {
            System.out.println(e.getRequest().getEntity());
            System.out.println(e.getResponse().getMessage());
            e.printStackTrace();
        }
    }
}

Request-Response: Please click on the below image to see it in full size. After successful creation of the user;
- service provider responds with 201-Created response code and
- includes newly created user in response body.
- Representation in response contains an additional attributes 'id', 'created date', 'last modified date' etc which is assigned by the SP and
- the "Location" header contains the unique URI of the created user resource.

2. Create Group..(From SCIM spec)

Following request/reponse is captured when creating a group with two exisiing users in the system.

Request-Response: Please click on the below image to see it in full size.
Server responds with 200 OK including group resource in the body encoded in JSON.

3. List Users
Following request/response illustrates a SCIM consumer call to list all the users.
Server responds with two user entries existing in its user store.

Note:

1. Libraries for sample client code:
You can find the required libraries to run the WSO2 Charon samples from here.
2. Sample source code can be found in the M1 distribution linked above.

3. As illustrated by above sample use case, SCIM consumer will be able to perform identity provisioning operations with any SCIM service provider in a unified way by just changing the resource URLs - which avoids the need of developping multiple connectors to integrate with different cloud providers.

Implementing SCIM with Charon - Part II

We had a look at an overview of WSO2 Charon in my previous post.

Today lets go through a brief introduction of the module : Charon-Deplyment, which is the reference implementation of SCIM service provider that is shipped with Charon.

This will illustrate how any concrete implementation of a SCIM service provider can make use of Charon-Core (the SCIM API) with Charon-Utils (optional). So being the second post on Charon, this will continue the top down approach of looking at it.

As we got to know from the introductory post on SCIM, the protocol defines a REST API for user identity provisioning operations.

Hence SCIM service provider needs to be a RESTful web application. In an earlier post, I have noted down the characteristic of REST - which is an architectural style of building networked applications.

There are several ways to implement a REST style based applications - such as Servlets and JAX-RS based frameworks.

In the reference implementation of Charon-SCIM servervice provider, we have selected the latter approach since JAX-RS hides underlying HTTP handling and binds the servlets nicely to individual methods in the Java classes using annotations.  Annotations can also dynamically extract information from HTTP requests and map application-generated exceptions to HTTP response codes.

Out of the JAX-RS implementations, Apache-Wink was selected since it looks promising to cater our requirements.

The Charon-Impl module creates an Apache-Wink based web application which can be deployed in an application server like Tomcat and which acts as a SCIM service provider.

Following is a deployment diagram of Charon-SCIM service provider (the web application provided by Charon-Impl module). It also gives a high level idea on how Charon-Core and Charon-Utils modules will be utilized.

As this diagram of the reference implementation illustrates, a SCIM service provider can be developed using any REST implementation and SCIM-defined resources can be exposed utilizing the API provided by Charon-Core.

On the other hand, SCIM Consumers can also be implemented using the client API of Charon-Core.

More posts to be followed...

Implementing SCIM with Charon - Part 1

You may need to refer to my previous blog post in order to get an overall idea on SCIM - Simple Cloud Identity Management..

This is about WSO2 Charon - one of the SCIM implementations which will be made available under Apache 2.0 license. Lets look at it in a top down approach.

Following diagram will give an overview on the module break down of Charon along with purpose of each module and  planned tasks of them.

Following is a brief introduction on each of the modules:

• Charon-Core:

        This is the API that exposes an implementation of SCIM specification. It can be used by any SCIM service provider or client implementation to support SCIM operations/functionalities. In addition to that, it also allows room for extension points to be plugged in according to the particular server side/client side implementation, such as authentication handler, user storage, encoders/decoders etc.

• Charon-Utils:

          This contains a set of default implementations of the extension points mentioned above. For an example - Basic Auth, OAuth handlers, LDAP based user storage etc. A particular implementation that uses charon-core as SCIM API, can use these default implementations as building blocks.

• Charon-Deployment: (Note: this is renamed as Charon-Impl)

       A reference implementation of SCIM service provider will be shipped with this module. Currently it is a Apache Wink based webapp that can be deployed in any application server - such as Tomcat, and make the SCIM endpoints be exposed. This is based on the above two modules : charon-core and charon-utils, and illustrates how any SCIM implementation can utilize the API and supporting module provided by Charon.

• Charon-Samples:

          This contains samples illustrating the SCIM use cases. Samples contains mainly the SCIM client side implementations which can be run against a SCIM server, and hence can also be referenced to get to know how the API provided by Charon can be used to implement SCIM client side.

Well.. this is a brief overview of what Charon is, how it is structured and what each module is supposed to do, supposed to be used for.

 More posts on Charon to be followed...