Bulk API: Process large sets of records from Rails app to Salesforce

authors: Alex Borodai, Vlad Tupytskyi

Salesforce is one of the biggest Customer Relationship Management solutions on the market. We are Ruby on Rails developers, and with the ever-increasing popularity of Salesforce, we are often asked how to integrate Salesforce with Ruby on Rails applications.

One of our recent projects was integrating a Rails app with Salesforce CRM. The main challenge was to create the necessary integration conditions so that it would be possible to query, process, update and insert large sets of records from the Rails app to Salesforce. While we were solving this issue, we discovered a major lack of relevant information. Most of the articles we found provided outdated solutions, and many of the related questions on Stack Overflow remain unanswered.

So, we decided to write this article to provide the working solution we found to integrate Salesforce with Ruby on Rails, especially when you need to process large sets of data.

A Developer’s Opinion on Salesforce

Salesforce is a cloud-based customer relationship management platform that brings companies and customers together. Salesforce offers many useful functions in addition to CRM. And Salesforce leaves room for developers to exercise creativity, including write your own applications and integrating modern web frameworks into the platform. Plenty of interesting stuff.

Salesforce is also quite active in terms of innovation. For example, in 2010, one of the first Heroku cloud platforms was purchased, which allows you to run an application in different languages (like Ruby, Node.js, etc.). Salesforce is developing dynamically, so developers need to constantly refresh their knowledge with each update that optimizes existing solutions.

What is the Main Problem We Faced?

The central issue of our project was to grab data from Salesforce, process the data and then update Salesforce with the new or modified data. To upload data to Salesforce, we choose Bulk API. Bulk API is optimized to load or delete many records asynchronously. It’s faster than SOAP-based API due to parallel processing and fewer network roundtrips. Bulk API allows for batch sizes up to 10,000 records and gives the opportunity to avoid the standard limits of 200 records. But the first issue we faced was when we tried to connect the Rails App to Salesforce.

How to Connect Rails to Salesforce

To connect your app to Salesforce Online, the first thing you need to do is go to https://developer.salesforce.com/signup and create a developer account. This is how you obtain a client ID and client secret, which you will need to securely connect your Rails application to your salesforce account.

Setting Up in Salesforce

Once you log in to Developer Salesforce, press “Setup” in the dropdown Settings menu as shown on the screenshot.

On the left menu, select “Apps” then “App Manager” and then “New Connected App”

Fill in the required fields. The ‘Callback URL’ should be a HTTPS link (e.g. a ngrok link to your app on testing step) and choose OAuth scopes

After saving the connected app, “Consumer Key” and “Consumer Secret” will be available. You also need to generate an “Initial Access Token”.

Note: changes may take more than “2–10 minutes” they say it takes to apply. In some cases we had to wait for about 20 minutes for the changes to apply.

Don’t forget to write down the “Consumer Key” and “Consumer Secret” (Consumer Secret is hidden by default. You will have to click on the “Reveal secret” link)

Setting up a Rails app

Step 1. Add the gems below and run bundle install

Step 2. Create config/initializers/omniauth.rb

Step 3. Setup your routes

Step 4. Create model User

Step 5. Edit the migration you just created

Step 6. Edit /app/controllers/application_controller.rb (this is a helper file which can be used to find the current user data)

Step 7. Create a Sessions controller /app/controllers/sessions_controller.rb to manage user sessions

Step 8. Now create some pages to test this feature /app/controllers/welcome_controller.rb :

Step 9. Don’t forget to add the new route to your page

Step 10. Finally, create the sign in link for this page. It should look like this:

Your Ruby app is now integrated with Salesforce. Now you can just relax and enjoy Rails💎!

Useful links:

Originally published at https://activebridge.org.

IT company providing businesses with dedicated remote developers & teams from Ukraine. We talk about #IT #ITStaffing #Web&Software #RORdevelopment