Drupal 7 ServiceNow Integration

By November 8, 2018Uncategorized

My team at the University of Colorado Boulder is in charge of the Web Express product. Web Express is basically a free website offering to CU faculty, staff, and student groups. We’re currently running over 1000 Drupal 7 sites.

Looking at other Drupal 7 ServiceNow integrations

There are not a lot of documented ServiceNow (SN) integrations with Drupal 7 — at least as far as my team at CU Boulder have been able to find. It seems like the other instances we’ve heard of the developers create a form with the integration embedded, but nothing shows up as far as a D7 Webform to ServiceNow integration. And that’s what we needed.

We wanted to be able to enable this integration per-site and then allow the site owners to create a form on their own — complete with mapping to the correct fields within ServiceNow. The only piece we’d be dealing with would be the “incident” table.

We also wanted to be able to add a file upload field to the webform and upload that file to the incident record as well.

I joined the ServiceNow slack and found that they had an integration channel — fantastic! Unfortunately, there was no information about Drupal integrations.

One of the other teams at CU had been creating forms programmatically and tying them into SN on a per-form basis. They were kind enough to let us take a look at their code and that’s what we started working from.

I also went and did a lot of reading of the ServiceNow API documentation. I tried to blame some of my problems on the documentation, but as my co-workers pointed out, it wasn’t definitely not the documentation’s fault. Fair.

Our requirements for our integration

Very simply:

  • Users must be able to create the SN-integrated form themselves
  • Users must be able to map custom fields to SN fields themselves
  • An incident must be created upon form submission
  • Users should be able to attach a file to the incident
  • Incident creation only, no editing or checking incident via Drupal

Build Step One: Allow SN integration per-form

Obviously, we don’t want all forms on the site to create an incident on form submission. I started by creating a configuration page for the module as part of the instructions I’ve received is that there should always be a way to turn a feature or functionality off completely, if needed.

As you can see, we also wanted to be able to easily switch the environment to our test instance.

If the status is changed to “disabled” we want all of the other SN options to be removed from the site.

Now that we have a way to turn it off sitewide, we need a way to turn it on per-form.

To do this I added a field to the node/*/edit page called “field_cu_servicenow” that was a simple boolean.

This boolean would be hidden if the sitewide access check came back as “disabled.”

So if we check the sitewide and per-form access and they both return true, then we can allow the ServiceNow request to be sent.

The next step was to setup those fields that would create the data that would ultimately be sent to our ServiceNow instance to create a incident record.

Build Step Two: Creating the basic form field mapping assignments
The best way I could see to allow site owners to map their own fields was during the time of creating each field component. Basically, I wanted this to be another simple selection they could make as they go through the form creation process.

There were a handful of fields that our instance of ServiceNow required to create a full incident request. Some of them could be easily gathered from the active user — things like caller’s name and email.

There were two main fields that needed to be sent: Short Description, and Description. Which could be considered the “Subject Line” and “Email Body” of the ticket.

So I created a select list and added those options and placed it in the form component edit page:


$form['servicenow']['field_map'] = [
'#type' => 'select',
'#title' => t('Map this field to a ServiceNow Incident field below'),
'#options' => [
'N/A' => '-No selection-',
'short_description' => 'Short Description',
'description' => 'Description',
],

And then we’d need to do some fiddling to get those fields marked with this special value to send their form values in with the ServiceNow request. Without this marker, ServiceNow wouldn’t have a place to put them. The alternative would have been to train users to give the field a specific machine name. We didn’t want to have them keep track of things like that.


$form_values = $form_state['input']['submitted'];
$servicenow_field_map = variable_get('servicenow_field_mapping', '');

foreach($form_values as $key => $value) {
if(key_exists($key, $servicenow_field_map)) {
$sn[$servicenow_field_map[$key]] = $value;
}
}

We save each field mapping as its own variable that we can then refer to when building out the SN request. Now the data says that there is an array with both “Short Description” and “Description” with their respective values coming in from the form on submit.

At this point we had most of the required fields working correctly. We were still missing a value for two of the categories that essentially told SN where this incident was supposed to go once inside the SN system.

Build Step Three: Get the incident assignments from SN
This was a custom table — and something that we would need to query in order to use. It was also form-specific. So one form could have two specific values and the next would have another set. We needed this to be another selection for site owners as they created the form.

/**
* Available under "form settings" when creating/editing a form.
*
* This returns a list of options for incident assignments.
*/
function _servicenow_incident_assignments() {
global $conf;
$incident_assignment = variable_get('sn_environment', '') . '/api/...';

$ch = curl_init(); // initialize curl handle
curl_setopt($ch, CURLOPT_HTTPHEADER, Array("Accept:application/json","Content-Type:application/json"));
curl_setopt($ch, CURLOPT_USERPWD, $key);
curl_setopt($ch, CURLOPT_URL, $incident_assignment);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
$result = curl_exec($ch); // run the whole process
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
//$header = curl_exec($ch); // run the whole process
curl_close($ch); // close cURL handler

$result = json_decode($result);
watchdog('cu_servicenow', 'ServiceNow incident assignment list cURL request was successful.');

return $result;
}

The result of this could then be placed as “options” for another hook_form_alter that we placed in the “Form settings” tab of the webform edit page.

Once there, we would save the values as another unique variable in the database to be used later — linked by the NID of the webform.

We didn’t want to save or cache the list because we’d been told that the options available were subject to change. This way the request is only made when they visit the form settings tab. This could prove to be a misstep, but only testing on our actual infrastructure will show that one way or the other.

Now we had the last of our required fields figured out. Since the two fields we needed were related — if you chose one, it would have to be link to the other anyway — we placed them as pairs in the select list. A single option would yield both field values which we could parse out on the request itself.

Build Step Four: Creating an incident with a form submission
Now it was time to create an incident using a configured webform. Our other group had been using cURL, so I started off using their code.

Here’s the cURL request:

$ch = curl_init(); // initialize curl handle
curl_setopt($ch, CURLOPT_HTTPHEADER, Array("Accept:application/json","Content-Type:application/json"));
curl_setopt($ch, CURLOPT_USERPWD, $key);
curl_setopt($ch, CURLOPT_URL, $request);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
$result = curl_exec($ch); // run the whole process
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
//$header = curl_exec($ch); // run the whole process
curl_close($ch); // close cURL handler

All the field values that we had pieced together with our field mapping were placed inside an array called $data. This request was then placed inside a custom form submit that would be called on the form only if the global and per-form access checks passed.

The result was fantastic. If needed, we could concatenate data onto the “Description” field and send that in as one piece. This was just an MVP attempt though, so that piece is still in the works and won’t be too difficult to setup.

It was working — we could create ServiceNow incident records via an webform submission on our Drupal 7 sites.

Build Step Five: Attaching a file to the incident
ServiceNow has their Attachment API that allows you to attach files to records in the system. We wanted to be able to have a site owner add a file upload field to their form and send the file up along with the incident.

The major hurdle here — apart from my novice understanding of debugging PHP cURL requests — was that the incident must exist and have a sys_id connected to it in order to attach a file.

That means that in the form submit, we’d need to first create the incident, return the sys_id from the result, and then use that sys_id to upload the file. Our single submit would create one request, parse a response, and then use values from that response to create yet another request.

And I must say that it worked very nicely. Obviously only after I had assistance from a number of co-workers to help me debug CURLFile, CURLOPT_RETURNTRANSFER, and then figuring out that CURLFile’s somehow send their own Content_type. (Thanks James, Kevin, and Alex for the help.)

$ch = curl_init();

$realpath = drupal_realpath($file_pathinfo['dirname'] . '/' . $file_pathinfo['basename']);

$data = [
'table_name' => 'incident',
'table_sys_id' => $sys_id,
'encryption_context' => 'undefined',
'uploadFile' => new CURLFile($realpath, $file_mimetype, $file_pathinfo['basename']),
];

curl_setopt($ch, CURLOPT_URL, $request);
curl_setopt($ch, CURLOPT_HTTPHEADER, Array('Accept:application/json'));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
curl_setopt($ch, CURLOPT_USERPWD, $key);

$result = curl_exec($ch);

if (curl_errno($ch)) {
$result = curl_error($ch);
}
curl_close ($ch);

We took the majority of the examples from the ServiceNow Attachment API documentation using the multipart/form-data version.

This worked and we were able to upload files along with the incidents in our ServiceNow system.

Being on the other end of thinking that maybe the SN API was broken (it wasn’t) or that Drupal was having a problem (it wasn’t), I can definitely see how simple this process is and how easy to implement it on our sites will be.

Build Step Six: Documenting the integration
I attempted to create a decent README for this module since it’s somewhat complex with multiple layers all over the place.

# Using CU ServiceNow

## Configuring your ServiceNow Key
This module is meant to use a single service account username and password.
This must be set in the site's settings.php file under the conf variable:

`$conf['sn_key'] = 'username:password';`

Without this key, your module will not be able to make any calls to the
ServiceNow (SN) URL.

If known username and password aren't working, contact OIT for a new password.

## Configure ServiceNow environment to use.
The configuration page for cu_servicenow, go to
"admin/config/content/servicenow".

On that page there will be two options: environment and status.

There are two options for the environment:

- Sandbox
- Production

The "Sandbox" option can be used for testing. The "Production" option will send
form submissions directly to the live environment and create actual incidents.
Choose the option that best suites your needs.

## Turning on the ServiceNow integration per form
Each form must have the SN integration enabled to be used. It is disabled by
default.

To enable this option on your form:

- Go to your form's edit page (Ex. https://mywebsite.com/node/1/edit)
- Go to "Publishing options" in the bottom left corner of the page
- Below "Published" and "Sticky at top of lists", select "Enable ServiceNow
Integration"
- Save webform

Now your webform will make a call to the configured SN environment.

## Configure the form's "Assignment Map"
Our ServiceNow incident submissions require a number of fields.

- Caller (person submitting the incident)
- Assignment Group (what group the incident is coming from)
- Business Service (the nature of issue is dependent on business)
- Nature of Issue (second part of assignment piece)
- Contact Email (email address of submitter)
- Short Description (think email title of incident)
- Description (this is where the bulk of the information is kept)

The "Caller" and "Contact Email" fields are populated automatically from your
Drupal account. "Assignment Group" has been prepopulated to use "Office of
Strategic Relations" as this is coming from a Web Express site.

"Short description" and "Description" must be created as fields inside your
form.

### Setting the Business Service and Nature of Issue for the form
A list of all available options for these fields has already been loaded upon
form creation. It is located on the "form settings" tab under, "Edit webform".

Look for the field labeled, "ServiceNow Incident Assignment".

The select list options are populated directly from the SN environment for your
site.

Select the incident assignment that's correct for your form and click "save
configuration". This will fulfill both the "Business Service" and "Nature of
Issue" fields that are required.

### Creating the "Short description" and "Description" fields
This is the same as creating a field for a field with an additional select
option during the field creation process.

For example, if I create a textfield form field, in the "edit component" stage
of the field creation process--and if SN integration is enabled--I'll see a
select list with the following options under the label, "Map this field to a
ServiceNow Incident field below:"

- -No Selection- (effectively leaving it blank)
- Short Description
- Description

As you'd expect, selecting one of them will then send the values of that field
to SN as the "Short description" or "Description" field.

A complete form should have both "Short description" and
"Description" setup like this to meet the incident submission requirements.

## Testing a form submission
Once the form has been created and published, a simple test to see if the form
is sending incidents to SN correctly is to simply fill in and submit the form.

If the submission is successful, a message with the following text will be
shown on the "form submitted" page.

`Your incident number is INC0000001, please keep this number for your records.`

If there was a problem sending the incident request to SN, you will see the
following message instead:

`There was a problem sending your request to ServiceNow. Please contact the
site owner.`

### Submitting a form while SN integration is disabled
If you submit a form when either the form specific integration is disabled
(under "Edit form" in "Publishing options"), or the global SN integration is
disabled (located at "admin/config/content/servicenow"), you will see not see
either message from the previous section.

Instead, you will see a usual Drupal form submit take place. So if incidents
are not being created from form submissions, it's likely that one of the
methods of disabling the SN integration have been selected.

## Troubleshooting
Logs are kept of both successful and unsuccessful SN requests inside the
watchdog table.

For an error, look for the following:

`ServiceNow api not communicating correctly: `

It will be followed by the request information.

For a successful SN submission, look for the following:

`ServiceNow incident was successfully sent: INC0000001`

Primary location of documentation for SN API: https://docs.servicenow.com/bundle/geneva-servicenow-platform/page/integrate/inbound_rest/concept/c_RESTAPI.html

Obviously not exactly software documentation, but it will be available as help text for the module going forward. I tried to make comments throughout the code to help explain why certain choices were made.

For instance, I use a form alter hook for three different forms because when I separated them out into their own hook_form_form_id_alter’s, the functionality broke. It was less repetitious and easier to follow logically code-wise to keep them together.

Summary and what we learned
I’m really surprised that there aren’t more ServiceNow integration pieces out there. It seems like something that would be a contrib module by now. I imagine the custom nature of the ServiceNow implementation might be the cause there. But there is a Salesforce Suite for both D7 and D8.

There is a possibility that we could take out the CU-specific pieces and make this into a contrib module — I’m just not sure what the actual needs are for this in the Drupal community.

I’m pretty happy with the ultimate outcome of the integration. I think we could definitely explore more of what ServiceNow has to offer.

omorrill

Author omorrill

More posts by omorrill

Leave a Reply