Data expressions: combining and reshaping layers with Arcade FeatureSets

One Chart, Two Layers. Picture a storm night at a water utility. Main breaks come in through one feature layer, and customer service requests arrive in another one, owned by a different team. The operations director walks over and asks a simple question: which districts are hit hardest right now? You open your dashboard, and you realize neither layer can answer that on its own. One has the breaks, the other has the calls, and nobody wants to wait for a GIS analyst to merge them overnight. So how do you show both on one chart, updating live? That's what data expressions are for. In this lesson, you'll write a short Arcade script that reads both layers, combines them, and hands the dashboard a brand new table.

What You Will Learn. Here's the plan for the next few minutes. First, you'll understand what a data expression is, and what Arcade means by a FeatureSet, because everything else builds on that one idea. Second, you'll combine two separate layers, main breaks and service requests, into a single FeatureSet with a small loop. Third, you'll reshape that combined data with GroupBy, so a serial chart, the dashboard's bar or line chart, or an indicator gets exactly one row per district with the totals already calculated. Along the way, you'll see the same logic in Python with pandas, so if you already think in data frames, the Arcade version will feel familiar. You don't need to be an Arcade expert to follow along.

What a Data Expression Is. A data expression is an Arcade script you add as a data source when you configure a dashboard element. Instead of picking a layer from a web map, you choose "Arcade" and write a few lines. The script can read one or many layers, and it must return a FeatureSet. Let's slow down on that word. A FeatureSet is simply a table of features where every row has the same columns, like one tab in a spreadsheet. The columns and their types are called the schema, which is just the list of column names and what kind of value each one holds. Once your script returns a FeatureSet, charts, lists, indicators and tables treat it exactly like a normal layer.

The Arcade Toolbox. Arcade gives you a small toolbox for this, documented on the ArcGIS Developers site. FeatureSetByPortalItem opens a hosted layer using its item ID, the long code you see in the item's URL in ArcGIS Online. Filter keeps only the rows that match a condition, such as status equals open. GroupBy summarizes rows, much like a pivot table in a spreadsheet: pick a column to group on, then count or add up values within each group. OrderBy sorts the result, and Top keeps only the first few rows, which is handy for a "top five districts" list. With just these five functions, you can answer most of the questions executives ask during an incident.

Worked Example: Combine Two Layers. Here's the combining step. First, we connect to the ArcGIS Online portal and open both layers by item ID. Use your own portal address if you are on ArcGIS Enterprise, and paste your layers' item IDs in place of the two placeholders. Now look at the three arguments after the item ID. The 0 means the first layer in that item. Then we ask only for the district field, and false skips geometry. That keeps the download small. Next, we describe the output table in a dictionary: no geometry, an empty features list, and two text fields, district and source. Then two short loops walk through each layer and push one row per feature, tagging each row with where it came from. Finally, FeatureSet turns the dictionary into a real FeatureSet. That's it: one table, both layers, and a source column that tells you which team reported each incident.

Worked Example: Reshape with GroupBy. Now the combined table has one row per incident, but the director wants one row per district. That's reshaping. Replace the last line of the previous script with these lines, so the script keeps going instead of returning early. We call GroupBy on the combined FeatureSet, group on the district column, and ask for a statistic named incidents, which counts the rows in each group. A SUM statistic could also total a numeric field, but we don't need one here. Then OrderBy sorts the districts from most incidents to fewest. The chart doesn't need any grouping settings anymore, because the data arrives already summarized, ready for a bar chart or a ranked list.

The Same Idea in Pandas. If you work in Python, here's the exact same logic with pandas, run on tiny sample data. We build two small tables, one for main breaks with customers out, and one for service requests. Requests have no outage count, so we set customers out to zero to keep the columns the same, the same shared schema idea as in Arcade. We tag each with a source, stack them with concat, which plays the role of our Arcade loops, and then group by district, counting incidents and summing customers. Look at the output. North has two breaks and one call, East has one of each, and South has a single request. The Arcade script produces this same shape, just inside the dashboard and refreshed live every time the layers change.

Keep It Fast. A word on speed, because a storm dashboard might be open on forty screens at once. A data expression runs in each viewer's browser, and it runs again whenever the dashboard refreshes. So every field and every feature you download costs time. Ask only for the fields you need and pass false for geometry unless you're mapping the result. Filter as early as you can, for example keeping only open incidents from the last twenty-four hours, before looping. And prefer GroupBy and Filter over hand-written loops where possible, because on hosted layers they can be sent to the server as a query, instead of pulling every row down first. GroupBy on a combined, in-memory FeatureSet like ours runs in the browser, so filter the source layers before you combine them.

Common Mistake: A Mismatched Schema. The most common trap is a schema that doesn't match the data. If you declare a field called district but push a misspelled key such as distrct, the district column silently stays empty. If you declare an integer field and push text, or push a date as a plain string, the element may show nothing at all, or the editor reports that it's unable to execute the Arcade script. The other classic slip is returning the dictionary itself instead of wrapping it in FeatureSet. When something looks blank, test the expression in the editor, read the output table it shows, and check names and types column by column.

Recap. Let's pull it together. A data expression is an Arcade script that returns a FeatureSet, a table of rows with the same columns, and any dashboard element can use it as a data source. To combine layers, you declare a schema once, loop through each source, push rows into it, and wrap the result with FeatureSet. To reshape, GroupBy summarizes rows per group, like a pivot table, and OrderBy and Top sort and trim the answer. Keep scripts lean by asking for few fields, skipping geometry, and filtering early. With that, you can answer questions that no single layer could answer alone.

Your Exercise. Now it's your turn. Pick two layers you can access in ArcGIS Online that share a field, such as district, county or zone. If you don't have your own, sample incident or service request layers from the Living Atlas or your organization will do. Write a data expression that opens both, pushes their rows into one FeatureSet with a source column, and then groups by the shared field to count incidents. Add it to a serial chart, sorted from largest to smallest. Finally, test it: change a filter or add a record, and confirm the chart updates. In the next lesson, we'll dress this up with rich text and embedded content.