The API Integration Process

Prologue

In recent years, API integration has taken on a life of its own. With so many new applications vying for the public’s attention and so many possibilities achievable by connecting these applications, API integration has become an important conversation to have. And an important part of that conversation is the API integration process – What will it take to “connect” these applications? What skills will we need? How long could it take? How do we know we are ready to “Go-Live?” This article will help you understand how to get the answers to these questions and shed some light on an increasingly important process.

At eMerge Technologies, we have integrated dozens of API’s over a period of 15 years. And in doing so, we have developed an “API integration process” that we feel is the best way to bridge two separate applications and have them work together in seamless harmony.

We feel that this information is helpful for software developers who want to an outline to the API integration process, SaaS administrators who want to get the best out of integration projects, and business analysts who have been drafted into working with the latest integration platforms that require less coding and more knowledge of the API integration process.

The API Integration Process!

Step 1. Research and Prep

When it comes to web API’s it’s important to understand that they are not all the same and they certainly haven’t all been created equal. Your research should be focused on finding out three things.

1. What type of API are we dealing with? There are three different types of web API’s: SOAP (Simple Object Access Protocol), REST (Representational State Transfer), and RPC (Remote Procedural Call).

2. What data payload are we dealing with? These API’s can usually either send data in XML or JSON or both payloads, but that is not always the case.

3. Do we have the appropriate documentation? If the applications are popular, the documentation should be robust and come complete with a WSDL (Web Services Definition Language) for SOAP applications or a WADL (Web Application Description Language) for REST applications. Documentation will let developers know how to format their requests and what data is available. Without proper documentation, an API integration is quite difficult.

Step 2. Data Flow Planning

After you have done the appropriate research and you determined an integration is possible. It is time to plan the data flows. Data flow planning is a critical part of the API integration process. Data flows are the objects or data you wish to transfer from one application to the other. Things to consider:

Authentication – Some API’s and most private web API’s are protected by some form of security. You simply cannot create a URL and send it to across the web and expect the data to come back unless you have met the required security measures. Popular security protocols include OAuth and SAML. Each of these protocols has their methods of satisfying access that should be accounted for. You should also consider how long you need to keep the authentication instance running. Will these data flows need to open continually or intermittently?

Timing/Volume/Speed – When, how often, and how much data needs to be transferred? An integration will only make sense if the data can get to the other system in a timely manner. The volume of data that needs to be transferred needs to be balanced with a rate of data transfer and the time that the data is needed to be in place. For example, if an organization will like to have all of its sales orders integrated by the end of each business day; then the number of orders and the rate of data transfer will determine if this goal is possible.

Data Discrepancies – When transferring data across applications, you’ll need to make sure that this data is in a format that is acceptable to the other end-point. This usually involves things like date/time formatting, and field size limitation. Although seemingly trivial, neglecting to translate this data will result in a failed integration.

There may also be cases where the data you wish to send is not an object the other end-point natively handles. Today, many SaaS applications have “custom objects” that will allow you add new types of data to an application. However, if you will do enough integrations, you will find scenarios where it is necessary to convert data in its native application to a completely different object in its end application. This will require some creativity on the part of the developer.

CRUDS – The intent of this data should be taken into account as well. CRUD stands for Create, Read, Update, Delete. When sending information to a new system, this system will need to know what you want to do with the information. It can either create a new record, read or select a record, Update a current record, or delete a record.

Newer integration platforms and developers have created a new data intention, the “Upsert.” Upsert allows an integration to create a new record if no relevant record is found or update the record with new information if a similar record is found. Regardless of the intention, it’s important to plan these out as to not create duplicate information and/or delete important records.

3. Data Map

Once you have the flows planned out, it’s best to create a master data map that puts all of this planning into a visual reference. This will make your life easier when it comes time for development. This guide is also the best way to make sure all the developers, consultants, and users are on the same page before the development begins and things become harder to change.

The data map will have the starting (Source) application on one side with all the objects you wish to transfer and all of the relevant information about those objects next to it. On the other (Target) side, you will list all the objects the starting objects will become once they are in the target system. Here is an example data map to give you a better idea of what we’re referring too. This is a screen shot of an invoice we moved over to NetSuite from another system.

4. Development

If you have done the first three steps, the development should flow easily. Think of it like writing an essay that you have spent the proper time planning out. The code should pretty much write itself.

5. Testing!

Testing is a critical step in the API development process. Testing makes sure that your integration will work exactly as intended under the specified conditions of the end user. It is ideal to have test environments for the applications you are connecting to best test the integration without disrupting production environments. Once you have your environments set up there are two main ways to test your integration. It is also important to note that the end user should have a lot of input in the testing process. They usually understand the data more than the developers and could easily spot small issues before they snowball into setbacks!

Development Testing – Development testing involves unit testing and systems testing to ensure two things are happening. The first being that authentication has properly been accounted for, and the second being that the data is making its way to the endpoint properly.

Use-Case Testing – This form of testing is something that the users should be provided to the developers before development, and even planning has begun. You can also think of them as stress tests for integration applications. Use-cases are scenarios that the integration will be expected to perform under. A good example would be making sure the integration can handle a spike in data related to a seasonal sales cycle. Integrations are usually designed to satisfy a need for a business or entity. Use-cases are a great way to make sure that the end product does exactly what the business intended it to do and do not break down at the worst possible times.

For more information on API integration and the API integration process, you can check out our article The Ultimate Guide to Understanding API Integration!