Jackson Carter Goes to DevSummit!

If you missed us at the 2024 Esri Developer Summit in March, I presented on our chatbot named Jackson Carter which we built as a plugin for Esri’s ArcGIS Pro.

Before delving into the technical details, let's first review the concept of an AI chatbot. Jackson Carter is an additional feature integrated into the ArcGIS Pro interface that allows users to communicate with the software using natural language. Instead of navigating through menus or executing commands, users can converse with the chatbot, issuing commands or queries in plain language. This conversational interface streamlines workflows, reduces the learning curve for new users, and enhances accessibility for individuals with varying levels of technical proficiency. AI tools can be a great asset for allowing people to do more with technology, as long as they work the way we intend.

We designed the chatbot to allow users to build Arcade expressions using plain language and then insert the completed expressions into feature layers, allowing them to build the content of the popups knowing little to no actual Arcade.

How It works:

We used the Esri APIs to build a WPF extension for ArcGIS Pro. This WPF project contains the UI for the chat interface. It provides the prompt for the user and displays the response from the AI language model.

The user can first select the layer that they want to generate the Arcade expressions for, and then type their request in the prompt box.

The user queries are sent to a web server which authenticates users with ArcGIS Online and communicates with another server running the AI language model. We used the Azure AI platform for the language model and hosted everything in Azure for simplicity.

The code responses from the language model were in markdown, so we were able to use some packages to add syntax highlighting and formatting to make the responses easier to view in the UI.

We also added some buttons to allow the user to copy the generated Arcade, or to automatically add it to the popup for the selected layer.

We found users enjoyed using the plugin and created complex queries by making additional requests, allowing the AI model to build on the existing Arcade expressions. We also noticed that because it is a large language model, asking the same question multiple times could provide different results. It was fun to see the different Arcade expressions that could be generated with the same prompt.

Here are some examples of Jackson Carter in action:

This is a simple demo of creating a popup based on the attributes of the selected item and formatting them for easy display.

In this demo we can generate a popup that queries an additional feature layer called “Airport Influence Area” and finds the nearest items in that layer in a 5-mile radius from the location of the item selected in the “Parcels” layer.

This demo continues from the previous one and adds a link in the popup to show the Wikipedia article for the nearest airports in a 5-mile radius.

Take a peek at the code:

We have made the source code free and open source so everyone can see how to integrate AI into an existing project. Check it out on GitHub!

Thanks to the team for working on an awesome project!

Jackson Carter was built by: Ande Bell, Tim Purdum, Tim Rawson, and Christopher Moravec