As users we expect flexibility, customizability, and extensibility in standard software to meet our needs.
Whereas customizability measures the degree and ease with which we can modify existing features, extensibility measures the degree and ease with which we can add new ones.
Both customizability and extensibility express the openness of a platform and the freedom it provides to fit our needs – now and in the future. This freedom gives us confidence and comfort that we will not be locked into a rigidly inflexible system.
Creating freedom with extensibility:
One of the core guiding principles for the Workato platform has been to make it more open and extensible.
- Connector SDK enables you to extend the library of connectors for their ecosystem of apps.
- Platform APIs give you the ability to seamlessly integrate Workato into their existing provisioning, lifecycle management and CI/CD processes.
- RecipeOps APIs can be used to conveniently set up Job as a Service to work with 3rd party orchestration tools.
- Job history and audit log streaming enables you to extend the monitoring & auditing to a centralized SIEM application like Splunk or others.
Although Workato provides a rich set of formulas and in-memory databases for SQL transformations, occasionally both means to transform data are either inadequate or inefficient.
The ability to create custom transformations using a scripting language gives developers the freedom to extend the platform to fit their needs.
Until now, Workato supported builders creating custom transformations using AWS Lambda and Ruby.
Let’s look at some examples:
Cleaning and validating data to improve quality
Several business processes across marketing, finance, customer support, sales, and HR require reliable contact information of entities like leads, customers, partners, employees, and others. Typically data collection points like landing pages, lead generation forms, job submissions, and partner registration pages are designed for a frictionless experience. This often results in the values for emails being invalid, lack of standardized formats for phone numbers, name, zip codes, and other structural inconsistencies.
- Validity: The degree to which data conforms to business rules and constraints, e.g. email cannot use a personal domain like gmail.com
- Completeness: ensuring all required data is known, identifying and fixing missing values, e.g. business phone number is always present
- Consistency: Ensuring the data is represented the exact same way across datasets, e.g. country code or currency code uses the ISO values
- Uniformity: Converting values into standardized formats, e.g. ensuring first and last name is always capitalized in title case
- Correctness: Ensuring the accuracy of the data, e.g. phone numbers are 10 digits
The below example recipe shows a data pipeline that extracts contact information from Salesforce, validates the contact information vis-a-vis email, phone, zip-code to load good quality contacts. All valid contact data is loaded into a table in Snowflake and contacts with poor data quality are reported for review in Slack.
You can also look into creating an API endpoint to make this cleansing and validation, and make it available as a service to internal teams like Marketing, RevOps, Sales, Finance, HR, Support or others.
Converting Excel files into CSVs
If death and taxes are the two constants in life, spreadsheets is a constant for businesses. Everyone has used spreadsheets regardless of job function, role, or title. They are a powerful tool to organize, visualize, and manipulate data. Spreadsheets are used for tracking projects, campaign plans, campaign performance data, financial reports, forecasting models, deal details, lead data, order information and more.
Not surprisingly many business processes need access to the valuable and critical data stored in these documents. However, it is difficult to extract data from spreadsheets in their native form, especially Microsoft Excel documents. Add to that the complexity of multiple sheets, calculated fields using formulas, filters, and more, and the challenge only grows.
Consequently, accessing data from the Microsoft Excel documents requires manual conversion into a more friendly and accessible file format such as Comma Separated Values (CSV) or Tab Separated Values (TSV).
Got any other interesting ideas to share? Send your examples to [email protected]