Das Skyvva Release Version 2.48 ist ab sofort verfügbar.
Release Winter´21 SKYVVA – Official Version 2.47
- Neue Version V4 des Inbound Processing
- Neue Version des Free Data Loader
- Weitere Adapter für den Integration Agent
- Der neue Integration Agent kann ein manuelles Processing durchführen
- Enhanced New agent support JSON and XML file format
- Enhanced New Agent to support V2 processing as old the Agent
- Auto Create SQL statement in Database Query based on do mapping
- Erweiterten Agent Datenbank Adapter für den neuen Integration Agent
- REST/SOAP Adapter unterstützt den Salesforce Authentication Provider
- Raw Logging Payload for inbound and outbound processing of SOAP adapter
- Outbound-Scheduler im neuen Agent Control Board verfügbar
- Several Bug fixes done
Share on facebook
Share on Facebook
Share on twitter
Share on Twitter
Share on linkedin
Share on LinkedIn
Release Summer ´20 SKYVVA – Official Version 2.46
- Added Ping Connection for Transport Route
- Enhanced mapping tool for "Copy Node"
- Agent Cache and SAP Cache
- Added several new adapters
- Added Adapter Monitor in new Agent Control Board
- Configuration logs of the new agent
- Added Agent Manual Processing API
- Added the import meta in New Agent Control Board
- Auto Create SQL statement in Database Query based on do mapping
- Added various message types format
- Several Bug fixes done
Share on facebook
Share on Facebook
Share on twitter
Share on Twitter
Share on linkedin
Share on LinkedIn
Webinar Anmeldung Summer Release 20
Wir laden Sie herzlich zur Teilnahme an unserem Webinar
– SKYVVA SUMMER RELEASE `20 –
ein.
In einer konzentrierten Form geben wir Ihnen einen Überblick über die Version 2.46.
Diese bietet Ihnen weitere Innovationen und funktionale Verbesserungen,
um Integrationsaufwände zu minimieren.
Registrieren Sie sich noch heute.
Wir freuen uns auf Ihre Teilnahme.
Ihr Aspara-Team!
- SAP - ERP
- SAP S/4 HANA
- SAP PI/PO
- SAP-CPI
- New Import Message Type
- New Export Message Type
- ANY Connect
Webinar Integration Agent – Neue Architektur am 24.09.2020
Wir laden Sie herzlich zu unserem Webinar – SKYVVA ANY Connect –
Eine neue schlanke, offene und erweiterbare Architektur
Wie wir alle wissen, ist Salesforce CRM eines der gefragtesten CRMs in der heutigen Zeit.
Die Integration von Drittsystemen in Salesforce ist meist nicht nur hilfreich, sondern geschäftliche Notwendigkeit. Die verschiedenen Komponenten der IT-Infrastruktur müssen miteinander kommunizieren können, ohne dass Daten verloren gehen.
Dies ist eine Voraussetzung für ein erfolgreich funktionierendes Salesforce CRM-System.
Vor diesem Hintergrund veranstalten wir ein Webinar zum Thema ” SKYVVA ANY Connect – Eine neue schlanke, offene und erweiterbare Architektur”.
Wir freuen uns auf Ihre Teilnahme.
Ihr Aspara-Team!
Erfahren Sie mehr in diesem Webinar über:
- Open and Extensible architecture for Salesforce Connectors
- New available connectors for event-driven based integration
- File connector enhancement to support file incoming event
- Lightweight connector framework e.g. not a overweight enterprise bus
- Q & A
Release Spring ´20 SKYVVA – Official Version 2.45
- New adapter for example SAP PI/PO to use v3 outbound processing and to replace the existing one such as SFDC2SAPPI adapter which uses v2
- Enhanced mapping tool to support hierarchical mapping
- Import different file format such as xsd, json schema, WSDL and openAPI to generate message type
- Enhance Message Monitoring to support Event- and API Monitoring
- Export xsd and wsdl from message type
- New Agent with Camel
- Old Agent using https
- Agent Control Board for old and new
- Adapt CDC to use v3 outbound processing
Share on facebook
Share on Facebook
Share on twitter
Share on Twitter
Share on linkedin
Share on LinkedIn
Release Winter ´20 SKYVVA – Official Version 2.44
- New version V3 of inbound and outbound processing
- Enhanced mapping tool to support hierarchical mapping
- Support Salesforce REST and Bulk Query to retrieve large volume data
- Enhance Message Monitoring to support Event- and API Monitoring
- Data Loader support hierarchical XML
- SKYVVA Bulk and Batch Mode support the new version V3 of Inbound Processing
- Workflow Support New Version V3 of Outbound Processing
- New Formula Functions
- Re-Order by Sequence Number for Repository and MessageType
- New Version of the SOAP and REST Adapter
- Acknowledgment Handling
- Export MetaData Provider are Missing Fields
- Get Error on Edit and Delete Repository
Share on facebook
Share on Facebook
Share on twitter
Share on Twitter
Share on linkedin
Share on LinkedIn
SAP Salesforce integration options and variants
SAP is the world-leading ERP software offering for all size of enterprise the right business solution. It is your digital back tier core. On the other side, Salesforce is your digital front tier core for your customer success. Business processes starting from your front tier e.g. Salesforce and ending at your back tier e.g. SAP create a unique business value chain which is essential for your company success. They can only work if the functional interaction and data synchronization smoothly work together. Deep and native integration is, therefore, the key success factor. But are you aware of how the solution suite can integrate into which way with Salesforce?
Before going into the various of SAP Salesforce integration options let us have a look at the business solution landscape of SAP as of today.
SAP business solution suite
Starting from R/2 in the earlier sixties SAP has launched the R/3 suite in the eighties which follows the Client/Server paradigm. In this range, we have SAP R/3 with the following version 3.1, 4.0 and 4.7 which is called R/3 Enterprise. Later there is ECC 6.0 or SAP ERP 6.0. These are the on-premise version of the SAP-ERP Business Suite. For small size business SAP has launched SAP Business One which now exists in on-premise as also in the cloud version.
Most used SAP ERP as of today is the ECC 6.0 or SAP ERP 6.0. New release and functionality enhancement are coming through SAP enhancement packs. Since 2015 SAP has launched a new suite called S/4HANA which come in two flavors e.g. on-premise or Cloud. It is based on the SAP innovative in-memory database engine and now become a populate migration path of an existing customer. Besides these main ERP suite branch SAP has other edition called SAP Business ByDesign which is a cloud offering and the SAP Business All-In-One which contain a reduced functional package as the full-blown SAP ERP 6.0 Suite.
Besides the ERP business suite SAP also provide integration product launched in earlier 2013 which call SAP-XI (Exchange Infrastructure) 2.0/3.0. The successor release is the SAP PI (Process Integration) and was renamed to PO (Process Orchestration). The latest version of SAP-PI/PO is as 7.50. Since the cloud paradigm become widely adopted SAP has launched a cloud version called SAP CPI (Cloud Platform Integration) and has now started the open initiative of SAP Cloud Platform Open Connectors. With the Cloud Open Platform Connectors SAP provides a huge number of different connectors like facebook, google, PayPal, etc.. to support mot than 100+ connectors.
You could have one of such SAP software stack and need to integrate with your Salesforce for a SAP Salesforce Integration. So how can you handle that many integration variants? Which technology to use in which case?
SAP Salesforce Integration variant
The integration of technology and possibility in SAP eco-system is huge and complex. To find the right technology you need to be an expert because there are too many modules, stack, and protocol to consider. You might don’t have time and the technical skill to implement such an integration tool and handle the different protocol conversion and data transformation yourself. With the SAP Connect of the SKYVVA integration service layer, you are able to connect to the different SAP component neither it is the middleware or the business suite application.
It makes sense to use specific and best-fit technology and protocol for each solution variants. For example, if you need to connect to your SAP-ERP ECC 6.0 only and you don’t have a middleware then it makes no sense first to buy a middleware. In this case, you can use the variant with the SAP-ERP Connect without using middleware in between as an integration mediator. This would be the fasted and optimized way to integrate Salesforce with your SAP-ERP ECC 6.0.
SKYVVA has built-in adapters on both platforms to leverage the best in place and natïve technology to handle the integration. It supports in case of SAP-ERP the native protocol such as BAPI/RFC, IDoc, and ABAP-Proxy. You can keep the investment, speed up your development by reusing the existing scenario where you have used BAPI and IDoc. If you want to use the latest technology such as Web service with REST and restful approach SKYVVA SAP connect can supports it as well.
The above picture gives you an overview of which option and variations you can choose to integrate Salesforce and SAP. SKYVVA offers different way based on the SAP business suite component used at your end. If you use on-premise middleware as the SAP-PI/PO then you can go with the SAP-PI/PO Connect. If you are using the cloud version e.g. SAP CPI then you can use the SAP CPI connect. SKYVVA provide on the SAP side the native adapter to utilize the most advanced and built-in technology available on the platform. Thus creating fast, reliable and optimized code for the integration demand.
Here are the available SAP Connect:
- SAP CPI Connect
- SAP PI/PO Connect
- SAP S/4HANA Connect
- SAP Business One Connect
- SAP Business ByDesign Connect
- SAP All-in-One Connect
- SAP ERP Connect
On-premise and Cloud integration technology
We have passed the day where we have only to deal with the on-premise system and installation. New disrupted business required definitely the cloud technology and thus we have to handle the different technology separately to consider and reflect their different particularity. Cloud technology is different from op-premise technology because the paradigm is totally changed.
SAP provides various interface technology from its first release which based on RFC and later evolved to become BAPI which follow the object encapsulation principle and follow the interface contractual design pattern. For asynchronous communication pattern often IDoc is used because it is integrated into the application as SD, MM, FI, etc. These technology based on the proprietary RFC which based on the CPIC protocol implemented in the C-Kernel of the underlying SAP basis stack.
With the emerging of the internet technology, the standard HTTP protocol was introduced to the abap kernel stack and thus new programing model like ABAP-Proxy was created to support open communication with the internet and using a lightweight protocol for better performance.
This technology is used by SKYVVA component sitting inside the abap stack of the SAP ERP 6.0, S/4HANA and SAP Business All-in-One. For integrating to the SAP Business One the HANA Service Layer is used to connect with the available API. These are the technology which is used to build a smooth and seamless integration to the on-premise SAP application suite.
On the other side thinking for the future SAP has launched the last few year’s different cloud-based products such as Ariba, SuccessFactors, Hybris, etc. and released new SAP business suite version such as S/4HANA Cloud, SAP Business One Cloud, and SAP BusinessByDesign which support the cloud deployment option.
Common to all cloud application is that they provide APIs based on the SOAP or RESTful technology and design principle. Swagger, openAPI 3.x and OData are emerging standard protocol to define and described the API while for soap-based web service, WSDL 1.1 or 2.0 are used. Knowing and building on this technology is crucial to conform to the emerging standard technology in the cloud to cloud integration. When we come to the RESTful web services OAuth 2.0 for the authentication are the standard protocol to use over the old-style username and password basic authentication.
Therefore merging the technology and using the same for any kind of integration to the SAP on-premise and cloud application suite will not make the most use of the strength and advantages of certain technology. SKYVVA provide for the on-premise world dedicated and specific connectors to leverage the SAP proprietary technology available. For the cloud world, you can use the SAP cloud connect to leverage the standard technology for cloud-based application integration.
Following we are going to discuss each integration variant in detail.
SAP ERP Business Suite
SAP ECC 6.0 or ERP 6.0 uses the abap stack. The interface technology is proprietary and is based on the CPIC library in the SAP kernel. Based on that the RFC protocol is used to build the foundation of the BAPI/RFC and IDoc. For each SAP module as SD, MM, CO, FI, etc. there are thousands of BAPI for almost any kind of business transaction. Furthermore, the customer can create their own RFC. Where BAPI/RFC is used for the synchronous communication IDoc is used for the asynchronous data transfer. IDoc is built-in into many SAP module and tightly integrated for example in an ALE and change point scenario.
Those interface technologies are proprietary and don’t not compatible with Salesforce interface technology which is based on the internet standard using web service based on the HTTP protocol. Salesforce understands non of the SAP communication protocol and thus incompatible to each other which means that they cannot connect to each other easily. To be able to do you need to convert the old and proprietary SAP RFC protocol to the internet standard.
SKYVVA provide a connector inside the SAP ECC as a bridge to convert the proprietary SAP protocol to the internet standard so that the Salesforce SOAP and REST API can be used. It uses the built-in ICM (Internet Communication Manager) component from SAP to implement the internet protocol. It is able to use the standard soap and rest technology provided by the SAP Gateway.
Note that to have an additional functionality needed for an enterprise-class integration SKYVVA has added a service layer into the Salesforce platform. To understand the need and benefits of the SKYVVA service layer on the Salesforce side refer to the following blog:
S/4HANA on-premise
Since its launch in February 2015, S/4HANA becomes a significant path for the customer who is still using SAP ERP 6.0 and thinks to move to the new technology supporting by the HANA database. In fact, the application layer is still based on the core abap stack but has been enhanced to support the new and latest technology in term of performance and cloud technology. From the function perspective, it offers a simple and lightweight module for some business area and in contrast to the SAP ERP business suite, it does not cover a monolith block of everything.
S/4HANA on-premise installation is from keeping the same technology stack as the SAP ERP business suite and thus the integration and interface technology remain but got some improvement. SKYVVA has adapted the SAP-ERP connect to support native element and enhancement of S/4HANA and thus provide a native component for the Salesforce integration.
S/4HANA is not only changing the underlying database to use the in-memory approach but from the deployment perspective, it has followed the cloud computing paradigm by providing it as the cloud option as well. This is the biggest update in SAP’s ERP strategy and platform and enables the customer to move into the new cloud computing or stay on the on-premise world.
Regarding the technology stack SAP S/4HANA covers different type of APIs such as web services based APIs (OData, REST and SOAP), traditional SAP APIs (BAPIs and IDOCs) and CDS views which can be exposed as OData services. The SKYVVA S/4HANA Connect uses the web API technology based on RESTful architecture. From the S/4HANA perspective, SKYVVA provides REST APIs to integrate seamlessly to the SKYVVA service layer.
SAP Business All-in-One
SAP Business All-in-One is a special offering which now is not promoted by SAP anymore. It flows into the product area of Small and Medium Enterprises (SME) and was intended to deliver a pre-packaged and industry-specific bundle of SAP-ERP. It is based on the SAP NetWeaver stack and thus leverages the same abap platform like the SAP-ERP 6.0. The integration technology SKYVVA provide here is similar to the SAP-ERP 6.0 Edition. Therefore refer to the integration option above at the SAP ERP Business Suite.
SAP Business ByDesign
SAP Business ByDesign is a cloud offering and now under the category of SAP Cloud ERP. It was developed by SAP for small and medium-sized businesses in mind which are based on SAP’s “best practices”.
For integration purpose to external system SAP Business ByDesign provide a set of APIs including OData endpoint to seamlessly execute the business operation on the given object. For example, the sales order API allows to create and change a sales order and its document flow. The SKYVVA ByDesign Connect use the web API technology to integrate Salesforce with the SAP Business ByDesign.
SAP Business One on-premise
For data exchange in SAP Business One provides these two interfaces, so-called Application Programming Interfaces (APIs) are provided:
- SAP HANA Service Layer API Technology
The next generation API for the digial business allowing you to create lightweight mobile apps consuming SAP Business One data and services using open core protocols such as HTTP and ODATA.
This technology is only available for SAP Business One, version for SAP HANA. - SAP Business One DI API
The DI API contains objects and methods that enable developers to read, write, update, and remove data objects on database level.
This DI API is available for SAP Business One on MS SQL, too.
SKYVVA support the latest technology using the Service Layer to connect from Salesforce through the API to the Business One Sever. All available API from SAP Business One can be used to create, read, update and delete application object. Business application function can be executed through the SOAP interfaces.
SAP Business One Cloud
SAP Business one can run on cloud which simplifies the hard- and software landscape within your company. There is two option which you can go with cloud deployment. You can either get your own license and your SAP partner host the service for you or you can rent on a monthly basis from SAP. Regardless for which deployment option you will go the underlying basis is the same which means that the integration technology is the same.
On the cloud layer, there is the same component and technology used for integration as in the on-premise deployment. Here the SAP HANA service layer can be used to provide REST, SOAP and OData services to access the business functionality in the form of APIs. SKYVVA Business One Connect can use to integrate to SAP Business One as a cloud-to-cloud approach.
SAP-PI/PO (Process Integration/Process Orchestration)
SAP-PI/PO provide you a middleware to handle all kind of integration which is mostly used to integrate to your SAP software landscape and other on-premise application. It provides a various adapter to connect to the business system and gives you the power to design, develop and run integration scenario within your on-premise landscape.
The integration to and from SAP-PI/PO is done use the SKYVVA connector component sitting directly on the J2EE-Engine of the SAP-PI/PO engine. This component handles all connection requirement to and from Salesforce and offers a specific Salesforce feature which is available as the additional layer on the Salesforce side. Additional value is for example to have the
- Message monitoring
- Message reprocessing
- Alerting
- Error handling
on Salesforce to give you more operational tool to handle your daily integration problem. From the technology standpoint, the native component is talking together to get the best possible integration speed you can ever thing. There is no something in between to impact the integration flow like protocol and format conversation because of the different techniques of the different platform. SKYVVA component uses the native technique and programming library on each side.
For the common integration use case like Account, Contact, Quote, etc. SKYVVA provide a ready to run so-called integration App on SAP-PI/PO which easily can be deployed and adapted to the business requirement quickly. This will reduce your development cycle and save cost and effort considerably.
SAP CPI (Cloud Platform Integration)
The SAP CPI is the cloud integration tool offering from SAP to follow the cloud computing paradigm and provide the latest cloud technology for their integration solution suite. As of SAP-PI/PO, it provides almost the same functionality and adapters and is compatible in term of for example the mapping engine. The SAP-PI/PO mapping can be reused in the CPI runtime.
SKYVVA provide in a similar way a native connector developed with the Adapter SDK from SAP and thus leverage most of the technology possible. It provides the inbound and outbound adapter to handle both communication direction, support streaming, and BULK API to handle the special requirement for Salesforce integration.
The adapter can be deployed to your CPI tenant as a third-party adapter and there is also pre-define integration app available like for the SAP-PI/PO edition.
SAP Cloud Platform Open Connectors
With the latest offering for the cloud, variant SAP has announced to have the open connectors which provide 100+ connectors for different use cases. One of the connectors available is the Salesforce connector which uses the Salesforce standard API to provide the integration services. The issues with any standard Salesforce adapter is the limitation which comes with the API connectivity approach. For some integration requirement, it is too basic and low level just to have the ability to do the CRUD operation. Refer to this blog https://apsara-consulting.com/de/beyond-api/ to overcome this issue.
Therefore SKYVVA has added the so-called API Connect to provide SKYVVA value-added API to the external system to be able to connect to the SKYVVA service layer to get all the value-added feature and capability missing with the standard Salesforce connector. With the additional service layer on the Salesforce side, you will add more quality and save significant cost in your daily interface operation.
Summary SAP Salesforce Integration
Now you have seen the possible solution landscape used within the SAP environment which is the most common one you will have yourself or will see it. The SAP Salesforce integration is based on the SAP common technology but has it’s different specific and thus SKYVVA provides for each option a dedicated connector to be able to handle their technology, protocol, and data format in an optimized way.
The connector is developed in SAP native technology to be able to use all available tool, technique, and protocol. Unlike the other solution on the market, the direct and native approach doesn’t create any friction between the connected system. It is the fastest and optimized way you can think for an SAP and Salesforce integration. If you need performance, reliability, directness, lightweight and cost-effective solution without having to buy another middleware then this is the option to go.
What is your SAP integration need? To which system do you want to connect? Is there something missing? This blog covers the most common SAP system used at the company of all size and nowadays with the cloud option there is a seamless way to connect Salesforce to the SAP cloud direct as the cloud-to-cloud connect without having a middleware in between.
There is simply no need for an intermediator like a middleware (middleware will cause unnecessary complexity) to have the deep and direct digital connected platform which is essential and crucial for your business. Go the new and direct cloud-to-cloud connect approach to disrupt your new business and be ahead of your competitor for Salesforce-centric solution-based processes.
Share on facebook
Share on Facebook
Share on twitter
Share on Twitter
Share on linkedin
Share on LinkedIn
Release Summer ´19 SKYVVA – Official Version 2.43
- Response mapping using message type for API integrate Synchronous
- Enhance workflow view because of the hierarchical interface
- Configure the whole integration to SAP (direct) from Salesforce
- Send Salesforce CDC Change Event to the receiver through our outbound interface
- The new version of integrating to speed up the processing
- Simplify Error handling for asynchronous outbound -BAPI to ABAP-Adapter
- Enhance operation query to use free define target structure for mapping
- CDC – Support to sent message in case of Undelete
- Conversion table as mapping formula
- Activate and Deactivate the adapter
- What to do with the failed message with the same external Id?
- Publish Event using SKYVVA interface
- Allow ping for soap, rest and all others adapter
- SOQL Statement for the outbound interface can be empty
- Display Columns Tree of the hierarchical XML in Manual Load
- Make Change on Stop By Admin with Push Data In Batch Processing
- Enhancing the MessageType validation for easier use
- By-passing message layer in any case
- New field External Message Id in the massage table
- Add new field ‘Comment’ on the message monitor to filter the comment field
- Option to exclude Interface name on the message search page
- Message monitoring- Comment field in a search filter to search message by comment
- Related To link on the message monitor
- Filter scheduler by its status
- Add new filter on the license tab key to select users
- Separate Batch and Bulk between Processing and Reprocessing on Functional Category
- Operation type = Apex Class to execute the post-processing after the mapping step
- Send Salesforce CDC data change to the external client without having the CometD support
- Using apex trigger to consume the CDC event or platform event to create CP records
- Add filter condition on a lookup field in Metadata
- Add the ID of parent interface into the parent interface field in child interface
- Show all messages from the basket in History Basket
- Create a link for the failed message in pending message in the message detail view
- Change section Message Processing Step to Custom Plug-in to SKYVVA Processing Block
- Create new fields in Agent Setting and Agent Property
- Create MsgType base on the Open API 3.0 file format
- Implement SKYVVA Trigger to support with the lightning version by using Quick Action
- Integration and Interface Overview tab on Agent Control Board with LWC
- Handling soap fault messages
- Message type which is constructed using Salesforce sObject is not shown correctly in the mapping editor
- IStructure Repository and MessageType’s child table should only show Child only
- Add additional property to the JSON schema and open API 2.0 parser to support the array and required node
- Got error on Apply Filters scheduler on Admin tab
- Fix bug on the XSD and JSON Schema parser where it should create Istructure instead of MessageType
- Long Comment sent from SOAP UI doesn’t work
- Named Metadata Provider file like its name in produce
- Unable to create SFDC2SAPPI adapter with full URL
- SKYVVA__PARENTID generate incorrect in Target Path
- Not allow show subinterface in Interface Name picklist
Share on facebook
Share on Facebook
Share on twitter
Share on Twitter
Share on linkedin
Share on LinkedIn
Why using a middleware only for Salesforce integration is not enough?
Salesforce is your digital front tier core for your customer success and you integrate ERP system such SAP, webshops such as Amazon or eBay, payment systems such as PayPal, social media such as Facebook or LinkedIn, legacy data pool such as database, file or FTP to and from your Salesforce to get a connected and non-disrupted customer experiences using a middleware. It works as design but one day you realize that some data are missing which has been sent from the middleware. You are not sure which data was arriving on your Salesforce side and you ask yourself “Where can I see the data which was sent by the middleware?”. You are not asking the question as “Which data was sent from the middleware?” because the middleware provides you a nice monitoring feature to see data leaving the middleware platform.
You’ve connected an HR and billing system to your Salesforce and need to keep data history for your auditing department to prove the evidence for a government security and auditing check happened regularly at your company. For such an auditing process data on one side e.g. sender (middleware) is not sufficient and do not fulfill the government auditing standard and they request you to provide the other side e.g. data that arrive on the Salesforce side. How are you going to provide such logging and tracing of the historical data on the Salesforce side?
Since you use a middleware you have a nice feature to alert in case the message get failed while get sent to Salesforce. You love this feature because you don’t need to actively look every five minutes on the monitoring at your middleware. One day you realize that some of the alerts contained strange technical error text as
“java.lang.StringIndexOutOfBoundsException: String index out of range: -1
at java.lang.AbstractStringBuilder.substring(AbstractStringBuilder.java:868)
at java.lang.StringBuilder.substring(StringBuilder.java:72)
at java.lang.AbstractStringBuilder.subSequence(AbstractStringBuilder.java:849)”
But you would expect a clear business error telling you something like “Quote xx cannot be posted due to invalid sales area code”. Which of the error text would you like to see?
Such situations can and will happen one day with your Salesforce integration and when it happens are you going to ask the same question instead of providing a solution to fix that? Read the following essay when you want to see the solution and answer.
How Middleware integration looks like
Let’s have a look and understand the Salesforce integration using middleware. To connect to Salesforce the middleware provides a technical connector which is able to invoke and consume the SOAP or REST API of Salesforce standard API. It uses the request/response pattern which is common to API principle and each call waits until the operation on the Salesforce side finish. This is the nature of synchronous API call which follows the tight coupling architecture pattern. With the standard Salesforce API, you can do great the basic four so-called CRUD operation meaning that you can create, read, update and delete a sObject.
Besides these four basic CRUD operation, there are no other additional capability and feature available on your Salesforce side. Some question you might ask is how you can do more than those four basic CRUD operation? Your integration, for example, needs to update a contact where it was changed on your ERP system such as SAP. But before the update, you need to check the assignment rule that the contact has the status ‘Released’ and is maintained by the sales agent group called ‘inside sales’. After you have updated the contact you want to do some cleanup operation to change the status to ‘Synchronized’. This means that you need more than only update the contact with some data which have been changed from SAP. You need pre- and post business logic to execute alongside the integration. How can you do such pre- and post additional business logic?
What is missing?
Because your Salesforce with all the apps from third-party vendors and your own app becomes such an important digital front tier layer for your custom success you want to have an enterprise-class integration which goes beyond the API connected approach. Refer to our other blog to understand the limitations and impact of the API only connected approach https://apsara-consulting.com/beyond-api/.
For enterprise-class integration, you need more than just the API connectivity. You might need to add business logic to your simple API integration. You might need monitoring, alerting and an easy way to handle errors on your Salesforce side. You might come to some urgent and emergency situation where the middleware is not alive and thus cannot resend the message to be processed on time. In this case, it would be a great help if you can modify data as you need on the Salesforce side and do a reprocessing. Furthermore, from the technical architectural standpoint, you need also a loose coupling pattern to handle mass data transfer which can only be done by using asynchronous API.
Why a decoupling services layer is needed
The additional value-added services mentioned above are needed for an enterprise-class integration which is not available on the Salesforce platform and which middleware cannot offer. To provide such services a decoupling layer is needed on the Salesforce platform which can stage the incoming data as messages for processing at any suitable time. Since no such a layer exists on the Salesforce platform SKYVVA has created such a layer to provide all the functionality like described in the above diagram.
The service layer is the technical basis for any functionality to allows the API processing to be executed in an asynchronous way. An asynchronous pattern is a crucial factor in the integration architecture because it provides the decoupling capability and independence of operation and thus is lose coupled. Lose coupled operation, on the other hand, has a big advantage and architectural benefits against the tight coupling pattern because it releases the client immediately from getting blocked. The client has not to wait until the invoked operation is finished. Less dependency and lose coupling create less cohesion and binding of interconnected operation which in term of resource consumption and performance is the best approach available for integration principle.
The loose coupling architectural style is just one of the outstanding feature resulted from having a service laver on the Salesforce side. Further benefits you have is, for example, the capability to shift the processing of the messages to a later time. Real-time and processing everything immediately sound great and fit best to some use cases where the user is waiting for a prompt response. But when coming to enterprise integration where the end-user is not involved you don’t need always a prompt response because it is useless and simply not needed.
Imagine that you have a webshop and you have to update on a daily basis millions of product prices. How can you do that at real-time? This is technically not possible doing in real-time because you will hit many Salesforce governor limits like the number of API calls per 24 hours, the available operating system thread to handle such a mass request and finally you will impact the performance of your Salesforce Org. and thus breaks the performance for your end-users in the business working hours.
In such a situation where it is sufficient to have the updated price of all product in the morning, you can use batch and bundling technique to send packages of for example 10k of price updates and process the messages in the night starting from 10:00 pm every day. Such scheduling is easy to configure using the scheduling functionality of SKYVVA. The benefit you get is shifting the processing of mass volume data into a time which is suitable for such a kind of data processing. Without a service layer like provided by SKYVVA, such a use case can simply not be handled by the Salesforce standard by just using the API connected approach only.
With a service layer on the Salesforce side, many additional value-added features like described above will be available to support you in your daily operation and maintenance of the integration scenario. The following description will show some important additional value-added feature in more detail.
Additional services you simply need
The various value-added feature will help and simplify the maintenance and operation of your daily integration with your Salesforce platform. Of course, you could say that you don’t need because everything works smoothly. But don’t forget the one day where some issues will arise and troubles start to appear sooner than you ever can think. I this situation you will not want to miss the additional value as the following examples below.
- Monitoring
Give you a second eye to see what happened on the Salesforce side. With the additional monitoring layer, you have a real end-2-end monitoring and have 100% view on both sides of the connected platform. - Reprocessing
Give you a possibility to independently do reprocessing of failed messages in case the middleware is not alive and can’t resend the data. This is to fix an emergency case where you have to make a quick fix inside Salesforce to reprocess the data which get failed due to some business data inconsistency. - Alerting
Give you real business error alert which deals with error happened on your Salesforce side and not an error which happened on the middleware. The alert is sent from the Salesforce side and therefore is a real and authentic alert cover the business data error on your Salesforce platform. - Workflow
Let you define a rule and pre-condition to process the message so that you can block wrong and dirty data from being mess up your business data on the Salesforce side. Complex business validation can run as a pre-condition before the data get posted and after the data get posted you can do a post-processing logic using the SKYVVA workflow. - Business logic
When you need to add business logic to the data posting Salesforce give you great and flexible tool like apex, trigger, process builder and flow. Use one of them to add business logic to your pure data processing linked to the SKYVVA interface. - Batch and Scheduling
When your data synchronization requirement comes to meet a mass volume data updates it is not correct and technically possible to use a real-time approach. In this case, you should move the processing of these mass data to a later free time window where you don’t break down the performance of the Salesforce Org. and thus impact the end-user while their business working hours. Instead, you should shift the processing to a suitable time for example in the night starting from 10:00 pm.
There are more additional value-added feature SKYVVA have added as integration services to the Salesforce platform. To keep this blog clear, short and comprehensible only these six value-added features will be explained in more detail.
Value-added #1 - Monitoring
To showcase the monitoring value-added feature we need to use a real interface example where the connected systems are SAP-ERP (sender), SAP-PI (middleware) and your Salesforce (receiver). The scenario is to change customer data in SAP and the changes will be synchronized immediately back to Salesforce. The diagram below shows the example flow.
In this screen, we can see the data changed in SAP for the customer 301557 and the old address in Salesforce. We expect to have the changed address from Salesforce to be updated on the Salesforce side.
Now have a look at the monitoring within the connected system as shown in the below picture. In this example, we are integrating to Salesforce using the standard API approach. In this screen, we can see the different monitoring on the sender e.g. SAP-ERP, on the middleware e.g. SAP-PI but on the Salesforce side there is no monitoring capability with the Standard Salesforce API. What you can do is going to check directly the sObject Account to see if the address data has been changed. When you check the address on the account you don’t see the changes getting updated in Salesforce. So what happened and what can you do?
What you can do in this case is to go to the debug trace and try to find the root cause why there the address has not been updated. Probably you will find something, probably not. As the Salesforce user or administrator, you cannot complain to the SAP-PI middleware team or the SAP-ERP team because all of them will prove you through their monitoring that the data was sent successfully where you can see it with the green flag. So what can you do to find the root cause in an easy way?
Unfortunately using the standard API approach with the middleware there is no monitoring capability inside Salesforce. The only way is to get your skilled apex developer team to try to find the root cause for this issue. Note that in this case we are using the asynchronous technique on SAP-ERP with Idoc and on SAP-PI the interface was designed as an asynchronous interface as well. There is no need to design this scenario using a synchronous interface because the SAP Idoc is an asynchronous technique. Therefore there will be no response of the API call telling the sender e.g. SAP-PI any error or exception. The message was sent successfully from SAP-PI to Salesforce and due to the nature of the asynchronous processing no further steps are needed on SAP-PI side.
So how to overcome this situation? Now let’s have a look at the same scenario which we rebuilt with SKYVVA. You are using, in this case, the SKYVVA connector on the SAP-PI/PO side. On the Salesforce side, you are using the SKYVVA service layer.
When the message arrives from SAP-PI/PO middleware it goes to the SKYVVA message layer where it gets processed immediately. From the message flow, there is no different to the scenario where SKYVVA was not involved. The significant differences are that now you have the SKYVVA layer which can provide you the monitoring service. So how does it looks like?
As you can see in the screen we have also monitoring in Salesforce where you can immediately see the root cause what was happened. The error is clear and understandable for the Salesforce user and administrator. Now you find the error in minutes instead of hours without involving your apex developer team. This is a real End-2-End monitoring enhancing your middleware integration scenario.
Value-added #2 - Reprocessing
Let’s stay with the above example which we used to showcase the monitoring. Let have a look at the red message in the SKYVVA monitoring. Let’s assume that you as the Salesforce user or administrator understand the business error clearly and you need to get the address change because you have to send the quote to that customer today before the end of the business day. You cannot wait until the next day to risk to lose the deal. You take your phone and ring your SAP-ERP colleague to resend the incorrect message again. You reach him but unfortunately, he left the office already because it was short to the office closing hour. Your SAP-ERP colleague cannot resend it today and promise to send tomorrow earlier in the morning. But this will not help you because tomorrow morning is too late for your emergency situation now. So what can you do?
Since you have rebuilt the interface with SKYVVA just use the reprocessing feature. Open the message, change and save the data and do a reprocessing.
Click on the button ‘Reprocess’ and confirm by clicking on the button ‘Yes’.
Now you can see a green message after the reprocessing.
Now you can navigate from here directly to the business record to see the change made for the address update.
The new address gets updated now after the reprocessing and you can use the new address to send the quote through your automatic quote process. Note that due to the simplicity the and clarity for the demonstration of the reprocessing feature we are not going to send the changes back to SAP. This is possible and makes sense but it would stretch the writing too long just for showing the reprocessing feature only. Also, note that the reprocessing you see here is the manual reprocessing. For this example, you have to do the manually reprocessing because the data is wrong and because you need to correct the data. But there is also others error where you don’t need to do a manual reprocessing but let the reprocessing job do it which you can schedule in an easy way. Have a look at the scheduling feature of the reprocessing job.
This is the message reprocessing job which you easily can start and stop by demand.
Value-added #3 - Alerting
The monitoring feature is great to see what happened on both sides. The annoying thing of the monitoring is that you always have to actively look to it. Wouldn’t it be great if you have some alert or notification when something gets failed and you get them as mail?
When looking to the scenario above SAP-ERP has an alerting, SAP PI PO has an alerting but on Salesforce there is no alert provided by the standard API. You would say that it is enough to have the alert on the middleware. Let’s have a detail look at the alert situation.
An alert should give you the real reason and root cause of an error which can happen in different places at any time. The error can be happened on SAP-ERP, on SAP PI PO, and on Salesforce and you need to have the original error text. In the above situation, the middleware alert the error happened on the middleware and not really what happened on Salesforce. It can reflect the error from Salesforce but there is a risk with this approach. The middleware has its own logic and code which can malfunction while transferring the message to Salesforce. In this case, it would alert its own error inside the middleware as for example “java.lang.StringIndexOutOfBoundsException: String index out of range: -1” which has nothing to do with the real error happened on Salesforce side.
What is the problem here? The alert is not authentic and it could be simply wrong because you get an alert which tells you the issue or error happened on the middleware but not happened on the Salesforce side. If the middleware stuck for some reason you could get alert which is delayed and was happened in Salesforce a few hours ago. This is crucial when you have a real-time scenario which also needs real-time alert from the source of truth. With the middleware in between, it is a kind of alert forwarding from Salesforce to you but not an alert delivering from Salesforce directly to you. From this example, you can see some drawback of the alerting capability.
- Inaccurate alert and sometimes simply the wrong alert
- Not real-time enough in case the middleware stuck or its performance gets degraded
- Not direct but indirect through forwarding by the middleware
How can we overcome these issues? The only accurate approach to deliver alert on time, correct and with its original root cause is to have the alerting component directly in Salesforce as well. As you can see in the picture above each component e.g. SAP-ERP and SAP PI PO has its own alert component. There is a good reason to have a native alert component inside each platform. Now let’s see how we can have such a native alert component inside Salesforce.
SKYVVA has added with its service layer the native alert component to be able to deliver the true alert. Alerts are directly sent real-time from Salesforce to you without any delay. You will get the original error text and never see something like:
“java.lang.StringIndexOutOfBoundsException: String index out of range: -1”.
Alerting can be defined in a granular way based on the integration or interface level. The integration is a logical group of interfaces. There are four different way to define alert delivery as below:
- Using external mail
- Using internal mail link to a user
- Using task
- Using chatter group
Here is an example, email alert.
You can click on the link and jump directly to the message detail and reprocess that failed message. With the option to use the chatter group you can use the social media feature to organize your alerting delivery. People following the group will get alert and there is no complex setup procedure to follow. There are also other alerting features for specific SKYVVA functionality like the Batch and CDC processing to observed the inbox of the basket or the change pointer table for data change.
Value-added #4 - Workflow
When you are using the Salesforce standard API to CRUD e.g. Create, Read, Update or Delete a sObject there is no validation or checks before create, update or delete a business record. It just above picture) which a service layer could bring to your integration need. It goes beyond the connectivity and provides you and your Salesforce team an easy way to develop, maintain and operate your daily integration need. This is what the SKYVVA service layer adds to the Salesforce Lightning platform which is missing for having the right and robust tool for doing the integration to and from Salesforce.executes it and modifies the sObject record immediately. But in most integration scenario you need to have some validation before modifying your business record otherwise you could mess up and create data inconsistency.
Have a look to the example which can be solved easily using the built-in SKYVVA workflow to have as a pre-condition check before an Asset gets modified. Here different condition consisting of the complex expression with formulas can be defined. A chain of ruled-based interface processing can be set up and provide you a powerful way to add complex message processing behavior than the simple CRUD operation. You can define any complex rule to validate first before you modify the sObject.
With the workflow, you cannot only define a rule or condition for interface processing. You can define a flow containing multiple steps involving in the message processing. Let’s assume you want to modify the account and the contact after each other. In this case, you can define a workflow which receives the account and contact data in a message and passing this message to the account and contact interface processing step. Thus you have defined a chain of processing step which can handle complex operation in one single API call.
You can define the different setting and run-time behavior for the workflow engine. For example, you can design a transaction-based processing flow and rollback all the operation within the flow when an error occurred. This guarantee transactional consistency.
Value-added #5 – Business logic
A message is transferred as the payload from the middleware to your Salesforce and through the Salesforce standard API, you can CRUD (Create, Read, Update, Delete) a sObject. The CRUD operation is the basic and elementary action you can think of handling with the business record in Salesforce. Sometimes it is enough just to CRUD sObject when your requirement is simple. Sometimes it is not enough when your integration scenario should handle some business requirement which goes over the CRUD operation.
With the SKYVVA workflow to define rules-based interface processing. Here you can use the built-in capability from Salesforce to add the business logic to the message processing. When your integration scenario goes beyond the CRUD operation then you need a powerful technique to add your business logic as you can do with a programing language. In this case, you can link an apex class to the interface and when the message arrives at the SKYVVA layer the processing will execute with your apex class logic where you can implement your business logic.
You can utilize the available programming and declarative tool for adding the business logic to your data integration. If you are a developer and can code in apex you can use apex trigger and apex class. If you are an admin and not an experienced developer you can use non-declarative tools such as the Process Builder and Flow. From the interface, you can link a Salesforce flow of type ‘Autolaunched’ and can build the business logic without any coding using the graphical flow.
Value-added #6 – Batch and Scheduling processing
Integration using middleware in between is mostly used to connect application to application scenario. This means that real-time scenario with end-user sitting and waiting on the other side is not common and thus not always a real-time and immediate data processing are needed. What sometimes need for mass and high-volume data integration between application system are batch and background processing to optimize the resource and performance and utilize the network, cpi and data storage.
Based on the integration scenario some use case makes perfect sense to use the real-time processing pattern. For example (1st scenario) when a user on the Salesforce side want to see the availability of a product in SAP-ERP then it makes sense to invoke an API and wait for the response. Such request is short running operation and will not hit the Salesforce time-out limit. When you want to update price by a nightly operation (2nd scenario) then it makes no sense to use a real-time processing pattern to update millions of prices. This would be technically not possible because of the Salesforce timeout limit and secondly make no sense because no user sits and wait to see the updated price immediately. It is enough to have the price update in the next day when the webshop will be open.
In fact, it is not a question whether the real-time (tight coupling) or batch/background driven (loose coupling) approach is better than the other. Both patterns fit perfectly when the given scenario is suitable. Do it real-time or batch based on your business requirement. The problem with the standard API is that it is not possible to delay the processing of the API or push it to run in the background job at a later time as for example in the night at 10:00 pm every day. The mechanism to do that is simply missing with the Salesforce standard.
SKYVVA service layer adds a scheduler and job processing component to the Salesforce platform which provides the batch and scheduling processing.
You can define the schedule in a different way to meet your business requirement. You can run every for example 15 minutes. You can run once at a certain time e.g. at 1:00 am or you can run every hour between for example 10:00 pm and 4:00 am.
You can select the days in the week and the month you want to exclude. This is a flexible way to define when message processing should start based on your requirement. You are not forced to process everything immediately if it would make more sense to process the data like a mass price update at the night time. Thus you can adjust and distribute the processing load and utilize your Salesforce resources and CPU time in a better way.
Different schedules are available for any kind of task you needed. For example with the batch processing and reprocessing scheduler you can push packages of message e.g. 10k per package and schedule the processing into the night time from 10:00 pm to 6:00 am for every hour. This utilizes the resources and distribute the load and keep the performance for of your Salesforce instance for the end-user while their working hours smoothly.
Starting and stoping schedule is easy by just clicking on the icon . Setting the schedule only need to click on the icon . All available schedule for the different task is generated automatically.
Summary
All six described value-added features will raise your integration quality to the next higher level giving you the operational instruments to ease and simplify the daily task. This method goes beyond the simple API integration approach where only pure connectivity is possible. When you want an enterprise-class integration then the only middleware approach is definitely not enough.
End-2-End monitoring and alerting increase the quality and improve error handling and daily operation tremendously. Message reprocessing give you a handy tool to handle emergency cases where the sender system is currently out of order. With workflow and business logic the simplicity and limitation of the CRUD operation can be overcome and complex business scenario can be easily implemented without much coding on the wrong side. Lastly, batch and scheduling feature enables to decouple the sender from the receiver and thus create less dependency which leads to optimized performance in the message processing for mass volume data.
Share on facebook
Share on Facebook
Share on twitter
Share on Twitter
Share on linkedin
Share on LinkedIn