Verifying entitlement caching with WSO2 API Manager 1.7.0

The API invocation flow with XACML is as follows

1 Request is received by the Gateway(APIM)
2. Token is validated by the Key Manager (APIM) and the validation results are sent back to the Gateway.
3. If the token is valid, the entitlement mediator will call the identity server for XACML policy evaluation.
4. If the result of the policy decision is 'permit' the actual back end endpoint will be invoked.
When you enable response caching for an API, the cache mediator will be engaged,before step 3 and 4 (before calling the Identity Server to get the XACML policy evaluated and the back end endpoint).
This will cache the response and the result of the XACML policy decision.
 
Setup:
       
1.Install the following features on API Manager and restart the server.
      Features
         -XACML
         -XACML Mediation
     Repository Location : http://dist.wso2.org/p2/carbon/releases/turing/
2. Create an API in the API Publisher and replace the content of the synapse configuration deployed with that of  this file 
3 In the above created API, I have used the following sample APIs as endpoints instead of calling external endpoints. Therefore copy these files to <AM_HOME>/repository/deployment/server/synapse-configs/default/api folder.
    acceptResponse_api.xml
    denyResponse_api.xml


4. The following sample xacml policy should be deployed in the Identity server used for entitlement validation.
    sample_xacml_policy.xml
  

Verification:
Step 1:
Enable debug logs for the package 'org.wso2.carbon.identity.entitlement' of the Identity server.
-For this add the following entry in the <IS_HOME>/repository/conf/log4j.properties file and restart the server.
 log4j.logger.org.wso2.carbon.identity.entitlement=DEBUG     

Step 2:
Create an API by enabling response caching
Subscribe to this API and invoke it.
- This will print the debug logs pertaining to the package enabled at step 1 on the identity server console.
- This implies the initial request made to the identity server for XACML policy evaluation.
- The policy decision and the back end response is cached at the APIM end at this point.
  

Step 3:
Invoke the API again (without changing the request).
- Since the request is identical to the request made at step 2, the response will be fetched from the response cache at APIM.
- No requests will be made to the Identity server for policy evaluation or to the actual back end.
- Therefore the debug logs observed on the Identity server console at step 2, will not be logged again.     

Step 4 (Optional):
Repeat step 1 and 2 after changing a request parameter.
- Since the request is different (due to the difference in request parameters) you will be able to observe the above mentioned debug log on the Identity Server for the first invocation.
- But not for the second invocation as the response will be fetched from the cache.

Guidelines for configuring WSO2 API-Manager workflows in a clustered environment

1) Work flow server URLs of site.json file should be updated with the correct port of the Business Process Server considering its port offset.(Work flow related configuration files by default contain port values assuming that the BPS port offset is 2.)

If BPS and API-Manager are required to be pointed to the same user store,  the workflow admin of the publisher node can be used, thus eliminating the need for a dedicated workflow node.
Since in a typical scenario the workflow admin will be from the same user store as APIM, we can use the workflow admin residing in the publisher node instead of having it separately.
Publisher node is recommended to be used here since it is an administrative level task and the publisher node is meant to reside within a private network.

In this case the URLs of <APIM_PUBLISHER_HOME>/repository/deployment/server/jaggeryapps/admin-dashboard/site/conf/site.json need to be updated.

If work flow admins are not from the API-Manager user store, have a seperate node for the workflow admin.
A dedicated node is only required if workflow admin users reside in a separate user store. In this case APIM and BPS will be pointed to different user store.

In this case the URLs of  <APIM_WORKFLOW_HOME>/repository/deployment/server/jaggeryapps/admin-dashboard/site/conf/site.json need to be updated.

If a workflow admin user role needs to be defined, add it under 'allowedRoles'.
eg:  "allowedRoles":"wfadmin"
Once this change is done, only users with the given role will be allowed to log in to the work flow admin dashboard.

2) Copy <APIM_HOME>/business-processes/epr folder to <BPS_HOME>/repository/conf

3) Make the following changes in the .epr files of <BPS_HOME>/repository/conf/epr

Change the following in case the default admin user has been changed.
<authorization-username>
<authorization-password>

The WorkFlowCallBackService endpoints of the following files should be pointed to the gateway :
<BPS_HOME>/repository/conf/epr/ApplicationCallbackService.epr  
<BPS_HOME>/repository/conf/epr/RegistrationCallbackService.epr  
<BPS_HOME>/repository/conf/epr/SubscriptionCallbackService.epr  
<BPS_HOME>/repository/conf/epr/UserSignupProcess.epr


The Service endpoints of the following files should be pointed to the Business Process Server :
<BPS_HOME>/repository/conf/epr/ApplicationService.epr          
<BPS_HOME>/repository/conf/epr/RegistrationService.epr          
<BPS_HOME>/repository/conf/epr/SubscriptionService.epr         
<BPS_HOME>/repository/conf/epr/ UserSignupService.epr

4) Update the port of the WSDL files of <API_MANAGER_HOME>/business-processes/<relevent workflow>/HumanTask with the correct port of the Business Process Server.

5) Upload the HumanTasks located in <API_MANAGER_HOME>/business-processes/<Relevent workflow>/HumanTask to the Business Process Server (Main -> Manage -> Human Tasks).
Alternatively you can copy it to <BPS_HOME>/repository/deployment/server/humantasks folder.

6) Upload the BPEL processes located in <API_MANAGER_HOME>/business-processes/<Relevent workflow>/BPEL to the Business Process Server. (Main -> Manage -> Processes).
Alternatively you can copy it to <BPS_HOME>/repository/deployment/server/bpel folder.

7) Point the endpoint of proxy service <APIM_GATEWAY_HOME>repository/deployment/server/synapse-configs/default/proxy-services/WorkflowCallbackService.xml of the gateway node to the 'Store' node of the cluster.

This proxy service is used to convert SOAP messages received by the Business Process Server  (which is unable to send JSON messages directly) to JSON, in order to call a rest endpoint.

8) Enable the executer relevant to the required workflow in the following file, by logging into the management console (the node does not matter since the  governance registry is shared)
Main -> Resources -> _system --> governance -> apimgt -> applicationdata -> workflow-extensions.xml.

Point the 'serviceEndpoint' to the Business Process Server.
Point the 'callbackURL' to the Gateway Node of the cluster.

The reason for pointing the callbackURL to the gateway node here is the fact that proxy service changes (eg: Security policy) are usually done in the gateway node. The store node does not get updated when such changes are concerned.

Enabling log4jdbc to verify key caching funcionality in WSO2 API-Manager

1.   Configure API-Manager database (WSO2AM_DB)  of WSO2 API-Manager with MYSQL        
      (http://docs.wso2.org/display/AM160/Setting+up+with+MySQL)

2.   Place the log4j driver in <KEY_MANAGER_HOME>/repository/components/lib
      (https://log4jdbc.googlecode.com/files/log4jdbc4-1.2beta2.jar)

3.   Append the following to the <KEY_MANAGER_HOME>/repository/conf/log4j.properties file.

      ! Log all JDBC calls except for ResultSet calls 
      ! Log timing information about the SQL that is executed. 
      log4j.logger.jdbc.sqltiming=DEBUG,sqltiming 
      log4j.additivity.jdbc.sqltiming=false 
      ! the appender used for the JDBC API layer call logging above, sql timing 
      log4j.appender.sqltiming=org.apache.log4j.FileAppender 
      log4j.appender.sqltiming.File=./repository/logs/sqltiming.log 
      log4j.appender.sqltiming.Append=false 
      log4j.appender.sqltiming.layout=org.apache.log4j.PatternLayout 
      log4j.appender.sqltiming.layout.ConversionPattern=-----> %d{yyyy-MM-dd HH:mm:ss.SSS} %m%n%n                                                                                                                                         
      This configures the following:

• Log level (DEBUG) : Captures all debug level logs of package jdbc.sqltiming
• Appender (sqltiming) : Uses a FileAppender where all logs are logged in the file specified in property 'log4j.appender.sqltiming.File'
• Layout : Format to use when logging

4.   Replace the following in the WSO2AM_DB datasource of located in file :
      <KEY_MANAGER_HOME>/repository/conf/datasources/master-datasources.xml

• Append 'jdbc:log4j' to the beginning of the content of <url> element.

         <url>jdbc:log4jdbc:mysql://localhost:3306/WSO2AM_DB?relaxAutoCommit=true</url>

• Change the driver class to the following:

         <driverClassName>net.sf.log4jdbc.DriverSpy</driverClassName>

       log4jdbc driver uses log4j to log messages. When 'jdbc:log4j' is appended to the url, the service calls pass through the log4jdbc driver where they are logged and passed over to the mysql driver.

      Once the above changes are done your WSO2AM_DB datasource should look like this.

<datasource>
      <name>WSO2AM_DB</name>
      <description>Datasource for AM database</description>
      <jndiConfig>
              <name>jdbc/WSO2AM_DB</name>
      </jndiConfig>
      <definition type="RDBMS">
           <configuration>
                   <url>jdbc:log4jdbc:mysql://localhost:3306/WSO2AM_DB?relaxAutoCommit=true</url>
                   <username>wso2carbon</username>
                   <password>wso2carbon</password>
                   <driverClassName>net.sf.log4jdbc.DriverSpy</driverClassName>
                   <maxActive>50</maxActive>
                   <maxWait>60000</maxWait>
                   <testOnBorrow>true</testOnBorrow>
                   <validationQuery>SELECT 1</validationQuery>
                   <validationInterval>30000</validationInterval>
       </configuration>
       </definition>
  </datasource>

      Now logging is enabled.

To verify key manager caching :

1.  Open <KEY_MANAGER_HOME>/repository/conf/api-manager.xml file and enable/disable <EnableKeyMgtValidationInfoCache> as required.
2.  Open <GATEWAY_HOME>/repository/conf/api-manager.xml file and disable  <EnableGatewayKeyCache>
3.  Re-start both Gateway & Key Manager nodes.
4.  Copy <KEY_MANAGER_HOME>/repository/logs/sqltiming.log & save under a different name.
     cp sqltiming.log sqltiming.log.1
5.  Invoke the desired API.
6.  Take another copy of sqltiming.log and save under a new name.
     cp sqltiming.log sqltiming.log.2
7.  Verify the database calls by checking  the difference between logs.
     diff sqltiming.log.1 sqltiming.log.2

If KM caching is enabled :  The access token related database call  should be logged only once until the cache expires.
If KM caching is disabled:   The access token related database call should be logged everytime the API is invoked.
   
To verify gateway caching :

Gateway caching can be tested by verifying whether database calls are logged on the Key Manager node when gateway caching is enabled/disabled.

1.  Open <KEY_MANAGER_HOME>/repository/conf/api-manager.xml file and disable <EnableKeyMgtValidationInfoCache>
2.  Open <GATEWAY_HOME>/repository/conf/api-manager.xml file and enable/disable  <EnableGatewayKeyCache> as required.
3.  Re-start both Gateway & Key Manager nodes.
4.  Copy <KEY_MANAGER_HOME>/repository/logs/sqltiming.log & save under a different name.
     cp sqltiming.log sqltiming.log.1
5.  Invoke the desired API.
6.  Take another copy of sqltiming.log and save under a new name.
     cp sqltiming.log sqltiming.log.2
7.  Verify the database calls by checking  the difference between logs.
     diff sqltiming.log.1 sqltiming.log.2

If GW caching is enabled:  The access token related database call should be logged only once until the cache expires. (In this scenario only the first invocation hits the key manager. The sub subsequent invocations use the token related data in the gateway cache)

If GW caching is disabled:  The access token related database call should be logged everytime the API is invoked.   (In this scenario every invocation hits the key manager as token related data are not cached on the gateway)

Eg:
2014-09-30 11:49:15.013  org.wso2.carbon.apimgt.impl.d
ao.ApiMgtDAO.validateKey(ApiMgtDAO.java:541)
3. SELECT IAT.VALIDITY_PERIOD, IAT.TIME_CREATED , IAT.TOKEN_STATE, IAT.USER_TYPE, IAT.AUTHZ_USER, 
IAT.TIME_CREATED, SUB.TIER_ID, SUBS.USER_ID, SUB.SUB_STATUS, APP.APPLICATION_ID, APP.NAME, 
APP.APPLICATION_TIER, AKM.KEY_TYPE, API.API_NAME, AKM.CONSUMER_KEY, API.API_PROVIDER FROM IDN_OAUTH2_ACCESS_TOKEN 
IAT, AM_SUBSCRIPTION SUB, AM_SUBSCRIBER SUBS, AM_APPLICATION APP, AM_APPLICATION_KEY_MAPPING 
AKM, AM_API API WHERE IAT.ACCESS_TOKEN = '6ef853c12bfebc66771baea147352860' AND API.CONTEXT 
= '/echo' AND API.API_VERSION = '1.0.0' AND IAT.CONSUMER_KEY=AKM.CONSUMER_KEY AND SUB.APPLICATION_ID 
= APP.APPLICATION_ID AND APP.SUBSCRIBER_ID = SUBS.SUBSCRIBER_ID AND API.API_ID = SUB.API_ID 
AND AKM.APPLICATION_ID=APP.APPLICATION_ID  {executed in 4 msec}