Invoking a REST API with SFTP content through WSO2 EI

Let’s assume there are two systems which cannot directly communicate with each other. But the source system has the functionality to generate a file with the required data so that we can use the generated file to update the other systems. For this example, we assume the second system exposed a REST service to update the system but not a file upload. So we will use the WSO2 ESB to read the files from the remote secure FTP location and update the second system through the REST service.

Prerequisites

• JDK 1.8 (u141 or below)
• Apache Tomcat
• WSO2 Enterprise Integrator 6.3.0
• WSO2 Enterprise Integrator tools (A.K.A Developer Studio)

Setting up the environment

For this example, we will use a JAX-RS service, which is hosted in apache tomcat, as the backend JSON/REST service. WSO2 EI is running with default configurations.

Writing the code

Creating the project structure

We are going to develop this whole application using the WSO2 Developer Studio, which is based on Eclipse. Let’s create a Maven project structure for our scenario.

Parent project

• Create the Parent project by opening the project window (File > New > Project…)
• Select WSO2 > Maven Project > Maven Multi Module Project and follow the wizard (provide ParentProject as the Artifact ID)

Java web project for mock backend

• Right-click the ParentProject and select New > Project…
• Select Maven > Maven Module and follow the wizard
• Untick the create a simple project option and provide MockBE as the Module Name, then click next.
• Select the entry where Artifact id is maven-archetype-webapp from the list and click next.
• Enter com.wso2.poc as the package name and click Finish.

ESB Artifacts project

• Right-click the ParentProject and select New > Project…
• Select WSO2 > Message Mediation > Project Types > ESB Config Project and follow the wizard (provide SampleService as the ProjectName, update the Group ID as com.wso2.poc, Tick the Specify Parent From Workspace option and then select ParentProject from the list)

Capp export project

• Right-click the ParentProject and select New > Project…
• Select Composite Application Project and follow the wizard (provide SampleServiceCApp as the ProjectName, update the Group ID as com.wso2.poc, Tick the Specify Parent From Workspace option and then select ParentProject from the list)

Finally, you should have a project structure similar the below

Mock Backend

We are going to create an echo service as the mock backend. So let’s follow the below steps to create the service.

• Update the pom.xml to include the dependencies.

<dependencies>
    <dependency>
        <groupId>org.jboss.resteasy</groupId>
        <artifactId>resteasy-jaxrs</artifactId>
        <version>2.2.1.GA</version>
    </dependency>
    <dependency>
        <groupId>javax.xml</groupId>
        <artifactId>jaxb-api</artifactId>
        <version>2.1</version>
    </dependency>
    <dependency>
        <groupId>log4j</groupId>
        <artifactId>log4j</artifactId>
        <version>1.2.17</version>
    </dependency>
</dependencies>

• Create a new Class called MockBE and update the code as follows

package com.wso2.poc;

import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;

import org.apache.log4j.Logger;

@Path("/echo")
public class MockBE {
    final static Logger logger = Logger.getLogger(MockBE.class);

    @GET
    @Produces({ MediaType.APPLICATION_JSON })
    public Response getHandler() {
        return Response.status(200).entity("{\"message\":\"This is the mock WSO2 service.\"}").build();
    }

    @POST
    @Produces({ MediaType.APPLICATION_JSON })
    public Response postHandler(String jsonString) {
        logger.info("New incoming entry : " + jsonString);
        return Response.status(200).entity(jsonString).build();
    }
}

• Update the web.xml as follows.

<web-app>
    <display-name>SpeedLedger Mock Back-end</display-name>
    <context-param>
        <param-name>resteasy.scan</param-name>
        <param-value>true</param-value>
    </context-param>

    <listener>
        <listener-class>org.jboss.resteasy.plugins.server.servlet.ResteasyBootstrap</listener-class>
    </listener>
    <servlet>
        <servlet-name>resteasy-servlet</servlet-name>
        <servlet-class>org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher</servlet-class>
    </servlet>

    <servlet-mapping>
        <servlet-name>resteasy-servlet</servlet-name>
        <url-pattern>/*</url-pattern>
    </servlet-mapping>
</web-app>

That’s it on mock service. Let’s move to the real integration project development.

EI Artifacts

The first step is to create two sequence files. One to process the file content after reading the file. Another sequence is to handle errors while reading the files (Right click the SampleService project and select New > Sequence to create new Sequences)

• Create xmlHandler_sequence_1.0.0 sequence with following content.

<sequence name="xmlHandler_sequence_1.0.0" onError="errorHanlder_sequence_1.0.0" trace="disable" xmlns="http://ws.apache.org/ns/synapse">
   <!-- Adding a log to see the file content -->
   <log level="full"/>

   <!-- Telling the backend system that we are sending a json message Content-type header -->
   <property name="messageType" scope="axis2" type="STRING" value="application/json"/>

   <!-- Calling the backend system -->
   <call>
         <endpoint key="gov:speedledger/speedledger_mockbe_endpoint_1.0.0.xml"/>
   </call>

   <!-- Adding a log to see the response from the backend -->
   <log level="full"/>

   <!-- dropping the message -->
   <drop/>
</sequence>

• Create errorHanlder_sequence_1.0.0 with following content.

<sequence name="errorHanlder_sequence_1.0.0" trace="disable" xmlns="http://ws.apache.org/ns/synapse">
   <log level="full">
       <!-- logging the error message -->
       <property name="ERROR" value="ERROR OCCURRED"/>
   </log>
</sequence>

Now we can create the file inbound endpoint to listen for new files in our SFTP location.

• Right-click the SampleService project and select New > Inbound Endpoint.
• Choose the Create a New Inbound Endpoint option and enter sftp_inboundendpoint_1.0.0 as the Inbound Endpoint Name.
• Select File option from the Inbound Endpoint Creation Type and click Finish. (Select the appropriate Sequences if asked during the wizard in latest Developer studio versions)
• Update the file inbound properties as follows.

<inboundEndpoint name="sftp_inboundendpoint_1.0.0" onError="errorHanlder_sequence_1.0.0" protocol="file" sequence="xmlHandler_sequence_1.0.0" suspend="false" xmlns="http://ws.apache.org/ns/synapse">
   <parameters>
       <parameter name="interval">5000</parameter>
       <parameter name="sequential">true</parameter>
       <parameter name="coordination">true</parameter>
       <parameter name="transport.vfs.ContentType">application/xml</parameter>
       <parameter name="transport.vfs.LockReleaseSameNode">false</parameter>
       <parameter name="transport.vfs.AutoLockRelease">false</parameter>
       <parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
       <parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
       <!-- This is how the URL should look like -->
       <parameter name="transport.vfs.FileURI">sftp://myuser:mypassword@remotehost/fileshare/in</parameter>
       <!-- You can store the URL in the secure vault and use it here like below -->
       <parameter name="transport.vfs.MoveAfterFailure">{wso2:vault-lookup('sftp.failurelocation')}</parameter>
       <parameter name="transport.vfs.DistributedLock">false</parameter>
       <parameter name="transport.vfs.FileNamePattern">.*\.xml</parameter>
       <parameter name="transport.vfs.FileProcessInterval">5</parameter>
       <parameter name="transport.vfs.MoveAfterProcess">sftp://myuser:mypassword@remotehost/fileshare/out</parameter>
       <parameter name="transport.vfs.Locking">disable</parameter>
       <parameter name="transport.vfs.FileSortAttribute">none</parameter>
       <parameter name="transport.vfs.FileSortAscending">true</parameter>
       <parameter name="transport.vfs.CreateFolder">true</parameter>
       <parameter name="transport.vfs.Streaming">false</parameter>
       <parameter name="transport.vfs.Build">false</parameter>
   </parameters>
</inboundEndpoint>

Above configuration will tell the ESB to look for new files in FileURI location in every 5 seconds. Now we can ship the artifacts to ESB.

Deploying the artifacts

• Open the pom.xml file under the SampleServiceCApp project.
• Select all the artifacts available in the list and save the file.

• Right-click the SampleServiceCApp project > Export Composite Application Project and browse the <WSO2_SERVER_HOME>/repository/deployment/server/carbonapps/

Testing the application

• Export the MockBE to <TOMCAT_HOME>/web-apps directory.
• If the servers (both ESB and Tomcat) are not already started, you can start now.
• Testing this app is easy. You can copy an XML file to the given FileURI location. ESB will read the file and call the backend service. So you should be able to see the log entries in both ESB and tomcat servers.