Bulk Importing of API’s to API Manager

In the last post we discussed on a tool that allows an admin to bulk export API’s from WSO2 API Manager. This is a useful tool if you have large number of API’s that needs to be exported. This blog looks at another tool that I have written to import API’s to an API Manager. Bulk importing of API’s may not be as common occurrence as bulk export, in many cases API’s are selectively imported to the API Manager rather than importing everything at once. However you can still identify which API’s to be imported and include them inside a folder where the tool will pick all the API’s inside the folder and import them to the WSO2 API Manager. The source code and instructions on how to run this tool can be found in the following Git Repo.

https://github.com/nadeesha5814/APIBulkImport

 

 

Advertisements

Bulk exporting of API’s in WSO2 API Manager

WSO2 API Manager is really powerful Opensource API Management tool that would allow you to expose, secure, manage and monitor API’s exposed via the API Manager. The product now comes with a nice feature to export/import API’s from and to the API Manager deployment. This feature comes in handy when you want to migrate API’s from one environment to another or when you want to copy an API from one API Manager to another. The feature is exposed as a RESTful API that can be invoked from a REST client, CuRL or a 3rd party script.

The limitation of the current implementation is that it only allows a single API to be exported at once. This is fine for majority of use-cases but there may be some instances where you may have 100’s of API’s and it might not be practical to export each and every API individually. To avoid this situation I have written the following tool that would allow you to do a Bulk export of API’s from the API Manager. The code and the instructions on how to use this tool can  be found in the following GIT repo.

https://github.com/nadeesha5814/APIBulkExport

 

 

Publish custom data from API Manager to WSO2 Business Activity Monitor

WSO2 API Manager provides out of the box integration with WSO2 Business Activity Monitor (WSO2 BAM), this integration would allow API Manager to publish a pre-defined event stream which would be stored, processed and summarized by WSO2 BAM to provide a meaningful set of information in the API Manager dashboard. A case may arise when an organization wants to publish custom data to the WSO2 BAM. For example a user may want to publish some values which are passed in the response message header to WSO2 BAM. In such a case the existing data Publisher needs to be customized to accommodate this custom value. API Manager’s extension mechanism allows you to write your own class to do this. In order to do this you can extend the ‘APIMgtUsageDataPublisher’ class and write your own data publisher. Given below is a custom data publisher written to pass a custom value to the WSO2 BAM. Please note that this is similar to the existing data publisher class with the difference of it using a custom DTO to publish response data.

This custom data publisher refers to a ‘CustomDataBridgeResponsePublisherDTO’ to publish data to WSO2 BAM. Given below is the custom DataBridgeResponsePublisherDTO.

As you could see here, we have add a ‘customValue’ field to the original datastream , and when creating the payload the custom value is included in the payload. In this example we are passing a string as the custom value, this logic can be replaced to pass any other value to WSO2 BAM.

 

Once you have done this changes build the source and add the jar file to the following folder.

<API_Manager_Home>\repository\components\lib

 

Java project file is attached for you to customize the values as required and build (through Maven).

Once this is done lets add the datapublisher class to APIManager.xml file. Lets modify the file to reflect the details of our custom data publisher. Given below is a sample snippet of the file. Notice that the custom data publisher is included as the ‘PublisherClass’ and the version of the response stream has been changed to’1.1.0′.

Now lets look at the changes needed to be done to WSO2 BAM, we need configure WSO2 BAM to expect this new data stream. You can do this by configuring the API Manage toolbox to include this value. The toolbox can be found in the following location.

<API_Manager_Home>\wso2am-1.9.1\statistics

The toolbox is a compressed file (.tbox format) that can be uncompressed, amended and re-compressed backed to a ‘.tbox’ file. I have attached a modified ‘.tbox’ file with this blog. This toolbox is configured to accept the field ‘customValue’ received in the response data stream.

Once you have a modified toolbox you enable statistics from the ‘admin-dashboard’ of API Manager which will automatically install this new toolbox to WSO2 BAM. Please make sure that you have uninstalled any existing API Manager toolbox before doing this.

 

You have now configured to publish a custom data value from WSO2 API Manager to WSO2 BAM

 

Git Project for the this example is given below. Toolbox is also available within this project.

https://github.com/nadeesha5814/Custom_Analytics

Role based API Throttling through WSO2 API Manager

If you are already familiar with the WSO2 API Manager you would know that the API Manager provides the capability to apply throttling tiers to an exposed API. A throttling tier is an access limit which is applied to a given API subscription to make sure that API’s are not used over and above the expected level. WSO2 API Manager’s subscription plan is such that an API Subscriber can choose from a set of throttling tiers available to him when subscribing for an API. By default all the throttling tiers are available for all users. 

In real world an organization would want to have control over how users can subscribe and access their API’s. Opening up all the tiers to all users may not be acceptable. In order to address this concern API Manager provides a capability to assign roles to different throttling tiers hence only users belonging to a given role would have access to that particular throttling tier. Lets take the following example. You would have multiple user groups such as internal developers, registered external partners, and guest external partners. You would want to open up your API’s for these users but want to have different plans to these users. For this example we will only be dealing with the existing throttling tiers, based on this I will create the following mapping of user roles to tiers.

Bronze Tier - No Restriction
Silver Tier - All users expect guest external partners
Gold Tier   - Only Internal developers and Admin users
Unlimited   - Only Admin users

These restrictions can be created from the API Publisher’s user interface. In order to do this, login into the Publisher console as a API Publisher. Once you are logged in you will see the ‘Tier Permissions’ link on the left hand navigation panel. Click on the this link. Once you are in the Tier Permission page set the roles to which you need to ‘Allow’ or ‘Deny’ to a given throttling tier.tierPermission

You can Create your own throttling tiers [1] and then define how these tiers should be made available to different application developers using the Tier Permission option provided available in the API Publisher.

[1] https://docs.wso2.com/display/AM191/Adding+new+Throttling+Tiers

WSO2 API Manager for message mediation

Even though the API Manager’s primary focus is to manage and expose API’s to different API consumers, it a common scenario to have the API Manager to perform some level of message mediation for both message requests and responses. The gateway component of the WSO2 API Manager is essentially a lightweight ESB that can perform almost all mediation activities that can be done by a regular ESB.

Newer releases of WSO2 API Manager provides the ability to attach mediation sequence to the request, response or the fault sequence of an API. The sequence gets executed when a request is made or a response is received to a request. Let’s look at how to add a message mediation to an API.

  1. Firstly you need to create message mediation sequence. Lets create the following sequence that would add a header to an out-going message. Save this sequence as setHeader.xml`
<sequence xmlns="http://ws.apache.org/ns/synapse"  name="SetHeader">
<header name="TempHeader" value="Test Header Value" action=set/>
</sequence>
  1. Log-in to the management console of the API Manager. You can access the management console via the following URL https://{IP}:9443/carbon You will find multiple options in the left hand navigation bar, select the ‘Browse’ icon in the Resources tab as shown below to access the internal registry.

Resources1

  1. Once inside the registry, navigate into ‘/_system/governance/apimgt/customsequences/in/’ folder in the registry and add the newly created setHeader.xml. Similarly if you have any sequence that needs to be added to the out sequence or the fault sequence you can do so by going into each of these folders under ‘customsequences’ folder. You can upload a sequence by selecting ‘Add Resource’ button and upload this file as shown below.

Resources2

  1. Once this is done, you can go to the publisher of the API Manager, and create a new API. Enter the required information in ‘Design’ and ‘Implement’ tabs and move to the ‘Manage’ tab. In the ‘Manage’ tab tick on the ‘message mediation policies’ and select the ‘SetHeader’ from the inflow dropdown as shown below. Once this is done Save & Publish the API to the API Store.

Mediation1

  1. You can now invoke the API from the API Store. You will see that the new header called ‘TempHeader’ is added to the message request before the backend API is invoked.

Throttling of API’s using WSO2 API Manager

When exposing API’s for consumption it is of paramount importance that the API access is controlled so that service consumers are not able to misuse these exposed API’s. WSO2 API Manager provides the capability to throttle API’s that are exposed via the API Manager. API Manager allows throttling at multiple levels.

1. Application Level throttling
2. API Level throttling
3. Resource level throttling

To understand the above levels of throttling we need to first understand the concept of API’s and applications in the API Manager.

An application inside the WSO2 API Manager is a logical group to which API’s are subscribed into. For example if you take a Weather Application, this will have 2 API calls. First one would make a call to a Location API to get GPS coordinates of the current location, after which a second API call would be made to a Weather API to get weather information of the current coordinates.So in this case the Application within the API Manager would be the ‘Weather Application’ and both the Location API and the Weather API would be subscribed under this application. This can be illustrated in the diagram below

IMG1_temp
Lets related the above diagram to different throttling tiers that are available with the API Manager, this can be illustrated as below.

IMG1_temp

1. Application Level Throttling

Each application can have a throttling policy defined that would limit the aggregated number of API calls that can be made to API’s subscribed under that application. In this case we can throttle the number of API calls made from the weather application irrespective of whether they are calling the Location or the Weather API. A throttling tier can be associated to an application from API Store. This can be done by navigating to ‘My application’ tab as shown below.

appView

2. API Level Throttling
Each API can be associated with a throttling tier. When an API is subscribed the subscriber would have a choice to choose which throttling plan to subscribe. This is shown in the screenshot below

APIView

The choice of throttling plans that needs to be associated to an API can be selected at the point of API creation. Given below is a screenshot which shows how to add a throttling plan to an API.

Tiers

3. Resource Level Throttling
WSO2 API Manager also allows you to define throttling tiers for each resource within an API. This allows you to have more control on how each resource is throttled in a given API. In the above example the Weather API would have two resources ‘getWeather’ and ‘getFullWeather’ which can have two different throttling tiers. A throttling tier can be associated to a resource at the point of API creation. This is shown in the screenshot below.

resource

If you wondering on how to set these tiers that you see on the screen, you can refer to the following API Manager documentation [1]

[1] https://docs.wso2.com/display/AM190/Adding+New+Throttling+Tiers

Creating your own profile in the API Manager

API Manager is shipped with the capability that allows the product to be started in a specific profile(for example a Gateway profile or a Key Manager profile). When the API Manager is started in a specific profile only the features specific to that profile and the common features as started with the server. For example if you start the API Manager with the Key Manager profile, only the features required for the key manager server and the set of features common to all profiles are started.

The feature bundling capability is provided using the underline OSGi architecture of the WSO2 API Manager. The API Manager is currently shipped with some server profiles which are documented in the following URL[1]. This blog looks at how we can create our own profile within the API Manager.

Features relating to a given API Manager profile can be found in the following location

\repository\components\\configuration\org.eclipse.equinox.simpleconfigurator\bundles.info

Bundles.info file contains the features that needs to be started with the given profile.

Lets create a new profile called ‘api-km-pub’. This profile will allow the API Manager to work as a Key Manager and a Publisher. For this profile I will merge the two bundle.info files in ‘api-key-manager’ and ‘api-km-pub’. The merged bundle file should be placed in the following directory structure below

\repository\components\api-km-pub\configuration\org.eclipse.equinox.simpleconfigurator\bundles.info

You can now start the API Manager server with the newly created profile by executing the following command

/bin/wso2server.sh -Dprofile=api-km-pub

[1] https://docs.wso2.com/display/AM180/Product+Profiles