Identity Provisioning from On-Premise to Cloud

Quoting from one of my initial posts on SCIM:

"Today the enterprise IT solutions adopt products and services from multiple cloud providers in order to accomplish various business requirements. Hence it is no longer sufficient to maintain user identities only in corporate LDAP.

In most cases, SaaS providers also need dedicated user accounts created for the cloud service users, which raises the need of proper identity provisioning mechanisms to be in place."

Identity Server(IS) 4.0.0 which is a 100% open source Enterprise Identity & Entitlement Management Server, supports the open standard SCIM for identity provisioning as I have mentioned in my previous posts as well.

With this, WSO2 Stratos Live next release will also be supporting SCIM for Identity Provisioning.

This post is about implementing a use case of identity provisioning from on-premise to cloud using Identity Server and Stratos (here, same IS distribution can be used to simulate Stratos IS with multi-tenancy aspects).

Following diagram gives an overview of the deployment:

Use case:
Two organizations called wso2.com and willpower.org have their on-premise enterprise Identity Management Solutions running with Identity Server.
Both these organizations use cloud services offered by WSO2 StratosLive and have created tenants in there.
Now, they want to provision the user account, identity management operations such as creating/deleting users and groups, updating user identity attributes etc which happens in their on-premise Identity Server to the respective tenants they have in StratosLive, as shown in the above diagram.

Implementation:
In this case, Identity Server running inside the organizational boundaries of each organization act as SCIM consumers and the Identity Server as a Service running in StratosLive acts as a SCIM Service Provider.

Each organization can register SCIM provider configurations pointing to their tenant space in SLive, within enterprise IS instances.

Following is a step by step guide for this.
Step1: setup
Download and unzip IS distribution into three different folders (to represent  instances at: 1.wso2, 2.willpower, 3.SLive)

Increment Ports->PortOffset element in carbon.xml s.t three instances are running in following ports:
IS of WSO2: 9443
IS of Willpower: 9444
SLive IS: 9445

You can find more details on how to do this step from the step1 of my previous post.

Step 2: creating tenants
Login as admin to the IS instance that simulates Stratos IS in our setup and create two tenants named "wso2.com" and "willpower.com".

Screen shots of the steps shown below:

Step3: registering SCIM providers

Now login to IS instances of WSO2 and WillPower organizations as admin user and register SCIM provider configurations pointing to their respective tenant spaces in SLive IS instance.
For a more detailed guide on how to register SCIM providers, please refer to step3 of my previous post.
Example configurations shown below:

Step 4: testing provisioning

Now you can test creating/deleting/updating users, groups in organizational IS instances and verify that they are provisioned to particular tenant space of each organization in SLive IS instance.

That's it... Thanks..!

Setting up cloud manager of Stratos with an external LDAP

Cloud manager is the service in WSO2 Stratos which provides the core functionalities of tenant management like creating new tenants by the super admin and self registration of tenants.

Stratos 1.5 is going to be released with in next couple of weeks. Then you will be able to download the distribution from wso2.org.

For now you need to check out the source from https://svn.wso2.org/repos/wso2/branches/carbon/3.2.0/ and build the stratos manager in products/manager/  location.

You can setup stratos on your machine using the setup script (stratos-setup.pl) which is available here.

In order to run that script, you need to: 

• have perl installed in your machine. In Linux, you can install "liblist-moreutils-perl".
• Have my-sql installed in the machine and have username and password root/root.
• set the following environment variables in the system.

export CARBON_DIR=/home/hasini/WSO2/branch_3.2.0/carbon
export STRATOS_DIR=/home/hasini/WSO2/stratos testing/1.5.0/setup/unpacks
export STRATOS_VERSION=1.5.0
export SSO_ENABLED=false
export CREATE_DB=true
export STARTUP_DELAY=30 
export PACKS_DIR=/home/hasini/WSO2/stratos testing/1.5.0/setup/packs
export PRODUCT_LIST="manager"

Following is an explanation on each of the above environment variable:
CARBON_DIR - root directory of your carbon check out. You need this if you build the stratos service distributions from source.
STRATOS_DIR - this is the directory to where the distributions zip files should be unpacked by the script.
STRATOS_VERSION - current stratos version
SSO_ENABLED - whether single sign on should be enabled across the different stratos services. In this case, we do not need it.
CREATE_DB - whether stratos related databased in my-sql should be created (if exists and this value is true, tables will be dropped and recreated)
STARTUP_DELAY - if several stratos services are going to be started, the delay between two startups
PACKS_DIR - if you are settting up stratos with downloaded packs, this is the folder where those packs are located.
PRODUCT_LIST - the list of stratos services that should be set up from this script.

Now all set, lets follow the following steps to start the cloud manager with an external ApacheDS LDAP server.

1. Create a new partition in ApacheDS LDAP where the separate user,group bases for each tenant will be created, when creating tenants through cloud manager. You may refer to my previous blog post for creating a new partition in ApacheDS. You may also create separate user base and group base for super tenant space. (i.e ou=Users,dc=wso2,dc=org & ou=groups,dc=wso2,dc=org)
2. Go to the above mentioned STRATOS_DIR where stratos manager has been unpacked, and change the following configuration files found in [carbon_home]/repository/conf
• tenant-mgt.xml: There you may notice that default tenant manager is JDBCTenantManager, but we now support tenant management with LDAP as well which we are going to demonstrate here. (if you didn't find this file in the above location, create a file named tenant-mgt.xml and copy the content shown below.)
• comment out the JDBCTenantManager and uncomment the CommonHybridLDAPTenantManager. 
• set the "RootPartition" as the partition name that we created in step 1 above.
• Following is the tenant-mgt.xml in our case:

    dc=wso2,dc=com
    organizationalUnit
    ou
    organizationalUnit
    ou

• user-mgt.xml:
• locate the property "MultiTenantRealmConfigBuilder". This is the class which is responsible for cloning the bootstrap user-mgt.xml and creating tenant-specific user-mgt.xml which contains the specific user and group bases of each tenant. Set it as follows: 

org.wso2.carbon.user.core.config.multitenancy.CommonLDAPRealmConfigBuilder

• Comment out the JDBCUserStoreManager which comes by default and uncomment the ApacheDSUserStoreManager for an external LDAP in read/write mode.

• In ApacheDSUserStoreManager configuration element, set ConnectionURL, ConnectionName, ConnectionPassword, UserSearchBase and GroupSearchBase for WSO2 user manager to connect to your external LDAP server.

            false
            ldap://localhost:10389
            uid=admin,ou=system
            secret
            SHA
            (objectClass=person)
            inetOrgPerson
            ou=Users,dc=wso2,dc=com
            (&(objectClass=person)(uid=?))
            uid
            [\\S]{5,30}
            true
            true
            false
            ou=Groups,dc=wso2,dc=com
            (objectClass=groupOfNames)
            groupOfNames
            (&(objectClass=groupOfNames)(cn=?))
            cn
            member
        

• embedded-ldap.xml:
• Disable the embedded-ldap server by setting "enable" property to false.

       3. Start the external ApacheDS server.

       4. Start the WSO2 Stratos Manager by running wso2server.sh/.bat scripts in  [stratos_manager_home] /bin directory.

       5. Access management console through https://localhost:9443/carbon.

       6. Login as super tenant with username: admin, password: secret (note that these are the credentials of admin user that you specified in user-mgt.xml)

       7.  Add a new tenant by providing tenant details as follows:

    8. After the tenant is successfully registered, you need to activate the tenant by checking the "activate" check box.
    9. Then log out from the super admin account.
   10. Login as the tenant admin of newly created tenant, by providing the above given credentials. For an example, in the above case, user name should be hasini@willpower.org.
You can create new users, new groups and assign users to those groups under your tenant.
If you look at the multi-tenanted LDAP structure from ApacheDirectory Studio, you will observer that there is a separate organizational unit is created for the tenant wso2.org and inside that, separate user base and a group base is created as shown in the following diagram.

That's it... You can create more tenants and experience the tenant management with a multi-tenanted LDAP.

Tenant management in WSO2 Carbon with a multi-tenanted embedded LDAP

You may refer to my previous blog post to get an idea what I meant by a multi-tenanted LDAP.

WSO2 stratos currently does tenant management with JDBC user store. And there is a requirement of achieving that functionality with a multi-tenanted LDAP too.

In achieving that requirement, the current solution that is there in WSO2 carbon takes a hybrid approach, where meta data related to tenants is stored in JDBC database and have the user stores of tenants in a LDAP server--which is an embedded Apache Directory Server.

Following is a high level design (as per now) related with tenant management with LDAP.

Here, apacheds-server component and apacheds orbit bundle are responsible in registering LDAPTenantManagementService (please refer the above diagram) and managing partitions in the embedded-ldap.

You can easily explore this functionality by installing multi-tenancy support on a WSO2 carbon based stand alone product. You may refer to post: Installing multi-tenancy support for WSO2 Identity Server.

In addition to the steps mentioned there, please take the following steps to connect a multi-tenanted IS to embedded ApacheDS LDAP and explore what happens in tenant management at the LDAP server's side:

Step 1: Place a file named: 'tenant-config.xml' in IS_HOME/repostory/conf directory with the following entry:

By default, it is the JDBC tenant manager that is being used and with this configuration file you can configure it to specify which tenant manager to be initialized at the initialization of the user manager.

Step 2: Start the server --> log in as admin and create few tenants as I  have described in the aforementioned post.

Step 3: Connect to the embedded-ApacheDS LDAP server through a LDAP browser like ApacheDS Studio. For that you need to:
      i. locate  user-mgt.xml file in IS_HOME/repository/conf 

     ii. obtain connection name and password to connect to the LDAP server from the browser. 

Step 4: View how separate partitions have been created for separate tenants as shown in the following image.

Few remarks:
      i. You can see the dc=cse,dc=com partition in the left panel of the above image and that is created for a tenant with the domain name: cse.com.

     ii. Inside that partition, two contexts have been created as ou=users and ou=groups to store the tenant user entries and group information.
     iii. At the creation of the partition, an entry of the tenant admin will also be created under the ou=users context. And that tenant admin can login to the system and create users for his tenant which will be stored in the particular tenant partition.

So above is one way you can manage users in LDAP in a multi-tenanted environment. But it is specific to ApacheDS and also to embedded-ApacheDS since we use ApacheDS API to manipulate storage.
We also need a solution where we can use any directory server that implements LDAP protocol to maintain the multi-tenanted user store. I will discuss about this in a future post.

Multi-tenant aware LDAP

This is my short note on the usage of  a LDAP storage in a multi-tenant environment which I intend to refer in a future post.

Well.. what is LDAP?
It is simply a protocol that basically defines how entries are stored and retrieved in a directory like storage (or directory server). If you want to learn more about LDAP, please find a comprehensive tutorial here. There are several implementations of this protocol out there. OpenLDAP and ApacheDS are two such popular LDAP servers.

How tenant management related with LDAP?

As I mentioned before, directory servers implemented according to LDAP, can be used as efficient and secure storages to maintain organizational user information. For an example, users will be authenticated into various applications against the user credentials stored there.

Usually LDAP serves as organizational user store. Storage is created and maintained as a tree structure and different contexts become branches of the tree. Generally when it is used as organizational user store, user entries are stored under one such context, where we call it a flat LDAP storage.

In cloud computing, we have this concept of multi-tenancy where the same application/service (customized for each organization/tenant) is used by  users from different organizations independent from one another. In this case, it would be good if we can store users of different tenants in separate directory trees or in different contexts of the same tree, rather than storing all user entries in a flat LDAP storage. 

What is a multi-tenant aware LDAP?
So..

• when we create a tenant in a cloud environment, if a separate directory structure or separate context is created in LDAP server for storing the users belonging to that particular tenant, 
• when we search a user of a particular tenant, if the search can only be directed to the relevant partition or context in the LDAP server,
• and if an admin of a particular tenant can connect to the LDAP server used in a multi-tenant application and view only the users of his tenant,

we can say it is a multi-tenant aware LDAP. In order to achieve the above requirements, we have to exploit the concepts of directory structure and access control mechanisms in LDAP.

Installing multi-tenancy support for WSO2 Identity Server

Multi-tenancy is a characteristic of a cloud native platform. WSO2 stratos provides multi-tenancy. In brief, multi-tenancy is to have one instance but customizable service for each tenant isolated from one another. If you want to know more about multi-tenancy, visit wso2.org which has many resources for learning.

Thanks to the modularity of OSGI; tenant management specific components in WSO2-stratos could be installed in WSO2-carbon based products-Identity Server in this case, and provide it with the multi-tenancy support.

In this blog post, I will explain how I could install multi-tenancy support in WSO2 Identity Server.

1. Get the Identity Server (IS) binary distribution. Either building from source or downloading binaries from here.
2. Get the stratos-tenant management specific bundles. We can either build from the source or extract them from the binary packages. Lets go for the latter option here for the convinience. Get the Stratos released packages of from here. Download wso2stratos-manager and wso2stratos-is.
3. Place following components from the above two packages, into Identity Server-home/repository/ components/dropins:

• org.wso2.stratos.tenant.mgt-1.0.0.jar
• org.wso2.stratos.tenant.mgt.ui-1.0.0.jar
• kaptcha-2.3.0.wso2v1.jar ---these three components will install tenant management feature. But we need other components as well, in getting it fully functioning.
• org.wso2.stratos.account.mgt.ui-1.0.0.jar
• org.wso2.stratos.account.mgt-1.0.0.jar ---these two are related to tenant's account management.
• org.wso2.carbon.email.verification.ui-3.1.0.jar
• org.wso2.carbon.email.verification-3.1.0.jar
• org.wso2.stratos.email.sender-1.0.0.jar ---these three will handle notifying tenant admin and e-mail verification after a tenant is registered.
• org.wso2.stratos.register.ui-1.0.0.jar ---is needed to show the registration complete page after e-mail verification. (Also, it is the ui component which is used in self-tenant registration.)
• org.wso2.stratos.identity.dashboard.ui-1.0.0.jar
• org.wso2.stratos.identity.login.ui-1.0.0.jar ---Identity Server's UI s are replaced by these.
• org.wso2.stratos.redirector.servlet.ui-1.0.0.jar
• org.wso2.stratos.redirector.servlet-1.0.0.jar ---each tenant will get a url with /t/domain name. In order to handle that redirection, these two are needed.
• org.wso2.stratos.manager.services.mgt-1.0.0.jar ---services are activated for tenants from stratos-manager. IS and G-Reg services are activated by default.
• org.wso2.stratos.common-1.0.0.jar
• org.wso2.stratos.activation-1.0.0.jar
• org.wso2.stratos.service.activation-1.0.0.jar
• org.wso2.stratos.theme.mgt.ui-1.0.0.jar
• org.wso2.stratos.theme.mgt-1.0.0.jar ---These handles tenant's theme management, you also need to copy stratos/services/manager/distribution/resources/allThemes to IS/resources.

            4.  Change the following in configuration files:

• add following two properties to the UserStoreManager element of user-mgt.xml in IS home/repository/conf folder:

false
default

• Change the carbon.xml in IS home/repository/conf to contain “WSO2 Stratos Identity” as the product name. This is because, in activating default identity service, service name is given as “WSO2 Stratos Identity”, And when checking whether a service is activated for a particular tenant, the service name is read from the carbon.xml. 
• Give smtp server information in smtp transport sender section of axis2.xml, found in IS home/repository/conf, in order to actually send e-mail notifications upon registration of tenants by the super admin.

        5.  Copy the following configuration files from stratos-manager home/repository/conf to IS-home/repository/conf:

• cloud-services-desc.xml
• email-registration.xml
• email-registration-complete.xml
• email-update.xml

That's it...now we are done. Start IS-->log in as super admin with "admin", "admin" credentials and create tenants. Tenant admins of those tenants will be able to log in to their tenants and create users and use other functionalities provided by IS independent of other tenants registered.

I will add some screenshots to illustrate the installed multi-tenancy support in IS stand alone product.
As shown circled in the left panel of following image, multi-tenancy feature is installed in IS.

Following image shows the list of sample tenants accounts that I created for testing purposes of the functionality.

Following image shows the console of IS when user 'hasini' of tenant: 'uom.org' has signed into the IS. Notice the url circled with the part '../t/uom.org..' which is specific to that particular tenant.

When multi-tenancy/tenant management feature is installed in IS, either JDBC or embedded-ldap user-store could be used. I will explain in a future post how to configure it to use tenant aware-embedded-ldap as the user store in tenant management.